Single-Server Multi-Device Training

This section takes the fine-tuning function of a caption task in OPT as an example. For the configuration, see MindSpore Distributed Parallel Training Example.

Modify the model startup script and replace the Python startup mode with the mxTuningKit startup mode. For details about the complete script after the modification, visit the Gitee repository.

• Original startup command.

  python -u src/scripts/train_caption.py \
          --config=config/caption/$task_config_file \
          --pretrained_model_path=$pretrained_model_path \
          --output_path=$output_path/$task_name \
          --use_parallel=True \
          --data_path=$data_path &

• Modified startup command.

  tk finetune \
         --quiet \
         --model_config_path $model_config_path \
         --boot_file_path $boot_file_path \
         --data_path $data_path \
         --output_path $output_path \
         --pretrained_model_path $pretrained_model_path &

  • boot_file_path is the absolute path of the training script.
  • data_path is the dataset absolute path.
  • model_config_path is the absolute path of the model configuration file. For details about the configuration file, see Model Configuration File.
  • output_path is the absolute output path.