Shard Method

Objective

The datasets are sharded in the same way as the benchmark model.

Principle

The data preprocessing part of the user model code may shard datasets to different nodes based on file name or number of files.

This results in large differences between the user model and benchmark model or even files sharded repeatedly to a single node, as the file read API sorts file names differently on different nodes.

Add debugging code to exclude such problems, ensuring the sharding policy consistent with that of the benchmark model.

Procedure

1. Print the lists of files input into the benchmark model and the ported model.
2. Check that the dataset files are sharded to the nodes in the same way.