EF1012 Invalid Graph Parallel Option Configuration

Symptom

Configuration of graph parallel option is incorrect.

Possible Cause

Tensor parallel size or data parallel size is incorrect.

Solution

Make sure the options in graph_parallel_option.json are correct:

1. If both tensor_parallel_option and data_parallel_option are set, the total parallel size (i.e. tensor_parallel_size * data_parallel_size) must equal the total device num in numa_config.json;

2. If only either tensor_parallel_option or data_parallel_option is set, the tensor_parallel_size or data_parallel_option must equal the total device num in numa_config,json.