Image Registration

Image Registration Parameter Search for Grayscale Reference Images

Before training the reference model, you need to register images with grayscale reference images. The software package provides a grid search script to obtain proper registration parameters. Table 1 describes the parameters.

**Table 1** Parameter description (**wafer_mindspore/reference/range_search.py**)
Parameter	Type	Value Range	Default Value	Description
--rgb_dir	String	-	./xxx/	Path that stores RGB images.
--chip_id	String	-	""	Processor ID.
--setupid	String	-	""	Processor layer.
--klarf_file	String	-	"xxx"	Name of a klarf file, which must be stored in rgb_dir.
--gray_file	String	-	"xxx"	Name of a grayscale image file, which must be stored in rgb_dir.
--save_path	String	-	""	Path for storing the search results.
--threshold	Float	[0,1]	0.5	Registration threshold.
--match_num	Integer	[1, 640]	320	Number of registered images that are randomly selected.
--range_thread_num	Integer	[1, 8]	8	Process quantity in multi-process mode, which is used to search for the registration area parameters x and y.
--img_thread_num	Integer	[1, 8]	8	Number of processes for image registration in multi-process mode.

A new_image folder must be created in rgb_dir to store the images to be registered.

klarf_file must be a TXT file, which describes information about images photographed at a machine and must comply with the following rules:

Each line corresponds to an image.
Each line contains 12 fields.
Fields are separated by a space.

For details about the fields, see Table 2. The following is a compilation example.

Create a data.txt file, which contains the following three lines of character strings:

img_1.jpg 001 M1 11224.435 12925.229 0 wafer_id1 01-08-22-13:09:22; device_id1 L21510106 model1 20
img_2.jpg 001 M1 13817.899 17822.569 0 wafer_id2 12-29-21-08:17:05; device_id1 L21510104 model2 20
img_3.jpg 001 M1 970.857 24809.092 0 wafer_id3 12-21-21-06:02:08; device_id3 L21510206 model3 20

**Table 2** Field description
Index	Type	Value Range	Description
0	String	-	Image path
1	String	-	Processor ID.
2	String	-	Processor layer.
3	Float	[0, 100000]	X coordinate of an image
4	Float	[0, 100000]	Y coordinate of an image
5	Integer	0 or 2	Image label
6	String	-	Wafer ID
7	String	-	Timestamp
8	String	-	Device ID
9	String	-	Batch number
10	String	-	Device model
11	Integer	[1, 1000]	Magnification ratio

The command for searching for image registration parameters is as follows:

python3 range_search.py --rgb_dir <rgb_graph_path> --chip_id <id> --setupid <setup> --klarf_file <klarf_file_path> --gray_file <gray_file_path> --save_path <save_path>

The following figure shows the results. With different registration parameters used, the registration threshold reaches 80%, and the match ratio of used-defined threshold is also obtained, thereby determining the optimal registration parameters.

Figure 1 Results

Registration Results for Grayscale Reference Images

After the registration parameters are determined using range_search.py, you need to save the graph of registration results to determine the registration threshold. The following table describes parameters in wafer_mindspore/reference/gray_reference_match.py that saves the registration results.

**Table 3** Parameter description
Parameter	Type	Value Range	Default Value	Description
--rgb_dir	String	-	./xxx/	Path that stores RGB images.
--chip_id	String	-	""	Processor ID.
--setupid	String	-	""	Processor layer.
--klarf_file	String	-	"xxx"	Name of a klarf file, which must be stored in rgb_dir.
--gray_file	String	-	"xxx"	Name of a grayscale image file, which must be stored in rgb_dir.
--save_path	String	-	""	Path for saving the registration results.
--threshold	Float	[0,1]	0.5	Registration threshold.
--match_num	Integer	[1,100]	50	Number of registered images that are randomly selected.

A new_image folder must be created in rgb_dir to store the images to be registered.

klarf_file must be a TXT file, which describes information about images photographed at a machine and must comply with the following rules:

Each line corresponds to an image.
Each line contains 12 fields.
Fields are separated by a space.

For details about the fields, see Table 4. The following is a compilation example.

Create a data.txt file, which contains the following three lines of character strings:

img_1.jpg 001 M1 11224.435 12925.229 0 wafer_id1 01-08-22-13:09:22; device_id1 L21510106 model1 20
img_2.jpg 001 M1 13817.899 17822.569 0 wafer_id2 12-29-21-08:17:05; device_id1 L21510104 model2 20
img_3.jpg 001 M1 970.857 24809.092 0 wafer_id3 12-21-21-06:02:08; device_id3 L21510206 model3 20

**Table 4** Field description
Index	Type	Value Range	Description
0	String	-	Image path
1	String	-	Processor ID.
2	String	-	Processor layer.
3	Float	[0, 100000]	X coordinate of an image
4	Float	[0, 100000]	Y coordinate of an image
5	Integer	0 or 2	Image label
6	String	-	Wafer ID
7	String	-	Timestamp
8	String	-	Device ID
9	String	-	Batch number
10	String	-	Device model
11	Integer	[1, 1000]	Magnification ratio

Run the following command to view the image registration results:

python3 gray_reference_match.py --rgb_dir <rgb_graph_path> --chip_id <id> --setupid <setup> --klarf_file <klarf_file_path> --gray_file <gray_file_path> --save_path <save_path>

Registration Results for RGB Reference Images

Before running generate_mindrecord.py, check the registration effect between images and the RGB reference images. The following table describes parameters in wafer_mindspore/inspection/rgb_reference_match.py that saves the registration results.

**Table 5** Parameter description
Parameter	Type	Value Range	Default Value	Description
--config	String	-	None	(Optional) Parameter configuration file path, whose permission must be set to 400.
--input_chip_id	String	-	""	Processor ID.
--input_setup_id	String	-	""	Processor layer.
--rgb_dir	String	-	rgb_ref/	Path that stores the RGB reference images.
--gray_dir	String	-	gray_ref/	Path for saving the grayscale images.
--mask_dir	String	-	mask_ref/	Path that stores mask images
--data_txt	String	-	None	Registered .txt file.
--save_path	String	-	""	Path for storing images.
--machine_id	Integer	0 or 1	1	Machine ID.
--threshold	Float	[0,1]	0.5	Registration threshold.
--match_num	Integer	[1,100]	50	Number of images for registration.

Images in rgb_dir, gray_dir, and mask_dir must be named in the specified format. The following is an example:

When input_chip_id is set to 001 and input_setup_id is set to M1, the reference image in rgb_dir must be named 001_M1_rgb_golden_die.png, the grayscale image in gray_dir must be named 001_M1_gray_golden_die.png, and the mask image must be named 001_M1_viaconductor_mask.png.

data_txt must be a TXT file, which describes information about images photographed at a machine and must comply with the following rules:

Each line corresponds to an image.
Each line contains 12 fields.
Fields are separated by a space.

For details about the fields, see Table 6. The following is a compilation example.

Create a data.txt file, which contains the following three lines of character strings:

img_1.jpg 001 M1 11224.435 12925.229 0 wafer_id1 01-08-22-13:09:22; device_id1 L21510106 model1 20
img_2.jpg 001 M1 13817.899 17822.569 0 wafer_id2 12-29-21-08:17:05; device_id1 L21510104 model2 20
img_3.jpg 001 M1 970.857 24809.092 0 wafer_id3 12-21-21-06:02:08; device_id3 L21510206 model3 20

**Table 6** Field description
Index	Type	Value Range	Description
0	String	-	Image path
1	String	-	Processor ID.
2	String	-	Processor layer.
3	Float	[0, 100000]	X coordinate of an image
4	Float	[0, 100000]	Y coordinate of an image
5	Integer	0 or 2	Image label
6	String	-	Wafer ID
7	String	-	Timestamp
8	String	-	Device ID
9	String	-	Batch number
10	String	-	Device model
11	Integer	[1, 1000]	Magnification ratio

The command for saving the registration results is as follows (rgb_reference_match.py):

python3 rgb_reference_match.py --config <config_path>

Parent topic: Quality Inspection on Semiconductor Wafer