一.上传数据集
1.1Host方操作:
1.1.1进入party10000 client容器
# docker exec -it confs-10000_client_1 bash //进入容器
1.1.2修改examples/upload_host.json
cat > fateflow/examples/upload/upload_host.json <<EOF
{
"file": "examples/data/train-PartyA.csv",
"id_delimiter": ",",
"head": 1,
"partition": 10,
"namespace": "experiment",
"table_name": "train-PartyA"
}
EOF
file是数据源
1.1.3传输数据:
sudo docker cp 数据绝对路径/train-PartyA.csv 容器名字:放置位置
例如:
sudo docker cp /home/KubeFATE/data/train-PartyA.csv XXXXXXXXXXXX:/data/projects/fate/examples/data
1.1.4上传host数据
docker exec -it confs-9999_client_1 bash
1.2Guest方操作:
1.2.1进入party9999 client容器
docker exec -it confs-9999_client_1 bash
1.2.2如果打不开的话,可以试试:docker start 容器名字
cat > fateflow/examples/upload/upload_guest.json <<EOF
{
"file": "examples/data/train-PartyB.csv",
"id_delimiter": ",",
"head": 1,
"partition": 4,
"namespace": "experiment",
"table_name": "train-PartyB"
} EOF
1.2.3还是先传输数据,再上传guest数据:
flow data upload -c fateflow/examples/upload/upload_guest.json
二.进行训练:
2.1修改examples/test_hetero_lr_job_conf.json
cat > fateflow/examples/lr/test_hetero_lr_job_conf.json <<EOF
{
"dsl_version": "2",
"initiator": {
"role": "guest",
"party_id": 9999
},
"role": {
"guest": [
9999
],
"host": [
10000
],
"arbiter": [
10000
]
},
"job_parameters": {
"common": {
"task_parallelism": 2,
"computing_partitions": 8,
"task_cores": 4,
"auto_retries": 1
}
},
"component_parameters": {
"common": {
"intersection_0": {
"intersect_method": "raw",
"sync_intersect_ids": true,
"only_output_key": false
},
"hetero_lr_0": {
"penalty": "L2",
"optimizer": "rmsprop",
"alpha": 0.01,
"max_iter": 3,
"batch_size": 320,
"learning_rate": 0.15,
"init_param": {
"init_method": "random_uniform"
}
}
},
"role": {
"guest": {
"0": {
"reader_0": {
"table": {
"name": "train-PartyB",
"namespace": "experiment"
}
},
"dataio_0": {
"with_label": true,
"label_name": "y",
"label_type": "int",
"output_format": "dense"
}
}
},
"host": {
"0": {
"reader_0": {
"table": {
"name": "train-PartyA",
"namespace": "experiment"
}
},
"dataio_0": {
"with_label": false,
"output_format": "dense"
},
"evaluation_0": {
"need_run": false
}
}
}
}
}
}
EOF
2.2修改examples/test_hetero_lr_job_dsl.json
cat > fateflow/examples/lr/test_hetero_lr_job_dsl.json <<EOF
{
"components": {
"reader_0": {
"module": "Reader",
"output": {
"data": [
"table"
]
}
},
"dataio_0": {
"module": "DataIO",
"input": {
"data": {
"data": [
"reader_0.table"
]
}
},
"output": {
"data": [
"train"
],
"model": [
"dataio"
]
},
"need_deploy": true
},
"intersection_0": {
"module": "Intersection",
"input": {
"data": {
"data": [
"dataio_0.train"
]
}
},
"output": {
"data": [
"train"
]
}
},
"hetero_feature_binning_0": {
"module": "HeteroFeatureBinning",
"input": {
"data": {
"data": [
"intersection_0.train"
]
}
},
"output": {
"data": [
"train"
],
"model": [
"hetero_feature_binning"
]
}
},
"hetero_feature_selection_0": {
"module": "HeteroFeatureSelection",
"input": {
"data": {
"data": [
"hetero_feature_binning_0.train"
]
},
"isometric_model": [
"hetero_feature_binning_0.hetero_feature_binning"
]
},
"output": {
"data": [
"train"
],
"model": [
"selected"
]
}
},
"hetero_lr_0": {
"module": "HeteroLR",
"input": {
"data": {
"train_data": [
"hetero_feature_selection_0.train"
]
}
},
"output": {
"data": [
"train"
],
"model": [
"hetero_lr"
]
}
},
"evaluation_0": {
"module": "Evaluation",
"input": {
"data": {
"data": [
"hetero_lr_0.train"
]
}
},
"output": {
"data": [
"evaluate"
]
}
}
}
}
EOF
2.3提交任务
flow job submit -d fateflow/examples/lr/test_hetero_lr_job_dsl.json -c fateflow/examples/lr/test_hetero_lr_job_conf.json
output:
{
"data": {
"board_url": "http://fateboard:8080/index.html#/dashboard?job_id=202111230933232084530&role=guest&party_id=9999",
"code": 0,
"dsl_path": "/data/projects/fate/fate_flow/jobs/202111230933232084530/job_dsl.json",
"job_id": "202111230933232084530",
"logs_directory": "/data/projects/fate/fate_flow/logs/202111230933232084530",
"message": "success",
"model_info": {
"model_id": "arbiter-10000#guest-9999#host-10000#model",
"model_version": "202111230933232084530"
},
"pipeline_dsl_path": "/data/projects/fate/fate_flow/jobs/202111230933232084530/pipeline_dsl.json",
"runtime_conf_on_party_path": "/data/projects/fate/fate_flow/jobs/202111230933232084530/guest/9999/job_runtime_on_party_conf.json",
"runtime_conf_path": "/data/projects/fate/fate_flow/jobs/202111230933232084530/job_runtime_conf.json",
"train_runtime_conf_path": "/data/projects/fate/fate_flow/jobs/202111230933232084530/train_runtime_conf.json"
},
"jobId": "202111230933232084530",
"retcode": 0,
"retmsg": "success"
}
查看训练任务状态
flow task query -r guest -j 202111230933232084530 | grep -w f_status
未完待续….
版权声明:本文为qq_45348164原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。