Installing Caffe Patches

Prerequisites for Merging .proto Files

You have prepared the custom.proto file and uploaded it to any directory on the AMCT server.

The following is an example. The custom layers and parameters in the following example are for reference only. You can customize the parameters based on your need.

message LayerParameter {
  optional ReLU6Parameter relu6_param = 2060;
  optional ROIPoolingParameter roi_pooling_param = 8266711;
}

message ReLU6Parameter {
  optional float negative_slope = 1 [default = 0];
}

message ROIPoolingParameter {
  // Pad, kernel size, and stride are all given as a single value for equal
  // dimensions in height and width or as Y, X pairs.
  optional uint32 pooled_h = 1 [default = 0]; // The pooled output height
  optional uint32 pooled_w = 2 [default = 0]; // The pooled output width
  // Multiplicative spatial scale factor to translate ROI coords from their
  // input scale to the scale used when pooling
  optional float spatial_scale = 3 [default = 1];
}

The custom.proto file consists of two parts:

1. Register a custom layer using LayerParameter.

   message LayerParameter {
     # user definition fields, each field takes one line.
     optional FieldType0 field_name0 = field_num0;
     optional FieldType1 field_name1 = field_num1;
   }

   This field is used to declare the custom layer in LayerParameter. The custom layer needs to be added to LayerParameter to allow the layer in the Caffe framework to write to and read from it. There are four parts to this declaration:

   • optional: indicates that the definition is optional in LayerParameter and can only be set to optional.
   • FieldType: declares the custom type corresponding to the current field. The corresponding message definition is required.
   • field_name: indicates the ID of the current declaration, which must be unique. If a conflict occurs, you need to change the ID and use the ID to access the corresponding content.
   • field_num: indicates the number of the current declaration, which must be unique. If a conflict occurs, you need to change the number. It is recommended that the value range be less than 5000 and do not conflict with the number in the caffe.proto file provided by ATC. In the binary caffemodel, the number needs to be used to parse the corresponding field.

   The following is an example:

   message LayerParameter {
     optional ReLU6Parameter relu6_param = 2060;
     optional ROIPoolingParameter roi_pooling_param = 8266711;
   }

   

   • The sequence numbers of custom layers in custom.proto are maintained below 5000 and do not conflict with those of the built-in layers in the caffe.proto file provided by ATC.
   • The sequence numbers of the layers in amct_custom.proto start from 200000.
   • The sequence numbers of the ATC custom layers in caffe.proto are within the range [5000, 200000).
2. Define the custom layer parameters for the message layer.

   message ReLU6Parameter {
     optional float negative_slope = 1 [default = 0];
   }

   Parameters of the custom layer are defined. For details, see Google Protobuf.

   Ensure that the value of this field does not conflict with amct_custom.proto at the custom layer of AMCT. If there is a conflict, an error message will be displayed during .proto merging. You can modify the value based on the error message. If it conflicts with the built-in caffe.proto of ATC, the message definition takes precedence.

   Currently, the custom messages of AMCT include QuantParameter, DeQuantParameter, IFMRParameter, LSTMQuantParameter, SearchNParameter, RetrainDataQuantParameter, RetrainWeightQuantParameter, SingleLayerRecord, and ScaleOffsetRecord. The custom layer name must be unique.

Procedure

You can run the auto installation script install.py in caffe_patch. If successful, the patches in caffe_patch are installed automatically in the caffe-master project directory. In this process, the .proto files will be merged and the original source code and dynamic library files will be replaced with new ones. After completing the installation or manual modification, rebuild the Caffe environment following the steps below:

1. Decompress the Caffe patch package.

   Run the following command as the AMCT installation user to decompress the caffe_patch.tar.gz package:

   tar -zxvf caffe_patch.tar.gz

   Find the following extracted packages.

   caffe_patch
   ├── include                        // Used to store the header files and common functions of the custom layer.
   │   └── caffe
   │       ├── layers
   │       └── util
   ├── install.py                     // Script for merging proto files, installing the patches, and executing the source code and dynamic library files in the Caffe environment.
   ├── merge_proto                    // proto merge directory.
   │   ├── amct_custom.proto         // AMCT custom layers and updated layers of Caffe-master compared with Caffe 1.0.
   │   ├── caffe.proto               // Built-in caffe.proto in ATC.
   │   └── merge_proto.sh            // Script for merging the .proto files. This script is invoked by install.py during actual combination.
   ├── patch                          // Patch file related to the LSTM layer.
   │   ├── lstm_calibration_layer.cpp.patch  // Used to generate lstm_calibration_layer.cpp in the caffe-master/src/caffe/layers directory.
   │   ├── lstm_calibration_layer.hpp.patch  // Used to generate lstm_calibration_layer.hpp in the caffe-master/include/caffe/layers directory.
   │   ├── lstm_quant_layer.cpp.patch        // Used to generate lstm_quant_layer.cpp in the caffe-master/src/caffe/layers/ directory.
   │   └── lstm_quant_layer.hpp.patch       // Used to generate lstm_quant_layer.hpp in the caffe-master/include/caffe/layers/ directory.
   ├── quant_lib                       // Directory of the core dynamic libraries for the quantization algorithm.
   │   ├── libquant_gpu.so
   │   └── libquant.so
   └── src                              // Used to store the source code files and common functions for implementing the custom layer.
       └── caffe
           ├── layers
           └── util

2. Switch to the directory where the caffe_patch/install.py script is stored and run the following command:

   python3 install.py --caffe_dir CAFFE_DIR [--custom_proto CUSTOM_PROTO_FILE]

   The command-line options are described as follows.

   Table 1 Parameters in the quantization script

   Parameter	Description
   --h	Optional. Displays help information.
   --caffe_dir	Required. Sets the path of the Caffe source code, which can be relative or absolute.
   --custom_proto	Optional. Sets the user-defined path of the custom.proto file. The path can be a relative path or an absolute path.

   Sample command:

   python3 install.py  --caffe_dir caffe-master  --custom_proto custom.proto

   You should see information similar to the following if the execution is successful:

   1 2 3 4 5 6 7 8 9 10 11 12 13 14

   # Copy the new source code and dynamic library files to the caffe-master project directory in the Caffe environment. [INFO]Begin to copy source files, header files and quant_lib to '$HOME/AMCT/AMCT_CAFFE/caffe-master' [INFO]Finish copy source files, header files and quant_lib to '$HOME/AMCT/AMCT_CAFFE/caffe-master' # Install patches. [INFO]Begin to install patch. [INFO]Install patch 'lstm_calibration_layer.cpp.patch' successfully. [INFO]Install patch 'lstm_quant_layer.hpp.patch' successfully. [INFO]Install patch 'lstm_quant_layer.cpp.patch' successfully. [INFO]Install patch 'lstm_quant_layer.hpp.patch' successfully. [INFO]Finish install patch. # Merge the .proto files. [INFO]Merge and replace "caffe.proto" success. # Modify the Makefile. [INFO]Merge and replace "Makefile" success.

   The install.py script allows patches to be installed repeatedly.

   • If patch installation fails, restore the caffe-master/src/caffe/layers/lstm_layer.cpp and caffe-master/include/caffe/layers/lstm_layer.hpp files in the Caffe project to the native files of Caffe-master.
   • If an error message is displayed during .proto merging, fix the error by referring to How Can I Resolve a Proto Merging Error?
   • If Makefile fails to be modified, modify it as prompted. If the script is executed successfully, the Makefile will not be modified again when the script is executed again.
3. Check the Python layer configuration. If the Python layer is not configured, add the implementation of the Python layer.

   Modify the caffe-master/Makefile.config file in the Caffe project.

   Add the implementation of the Python layer.

   # Uncomment to support layers written in Python (will link against Python libs)
   WITH_PYTHON_LAYER := 1

4. (Optional) Add C++11 standard code support.

   New AMCT operators require C++11 support. As such, add the --std=c++11 build option to caffe-master/Makefile. As the file will be modified during the install.py execution in 2, check that the --std=c++11 build option is added to caffe-master/Makefile as shown in the following line in bold. Skip this step if it is added. Otherwise, manually modify the file.

   # Complete build flags.
   COMMON_FLAGS += $(foreach includedir,$(INCLUDE_DIRS),-I$(includedir)) --std=c++11
   CXXFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS)
   NVCCFLAGS += -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)

5. Return to the caffe-master directory and run the following commands to rebuild the Caffe and PyCaffe environments:

   # If the Caffe project has been built before the patch is installed, run the make clean command and then the build command after the patch is installed.
   make clean
   make all -j && make pycaffe -j

   After caffe.proto is modified, it needs to be rebuilt into caffe_pb2.py. AMCT needs to parse the Caffe model, and a custom layer may be added when the Caffe model is used. In this case, you need to modify the caffe.proto file. After the modification, you need to provide caffe_pb2.py built from the modified caffe.proto file for AMCT.

   

   If you use the protoc command to rebuild caffe.proto (for example, protoc --python_out=./caffe.proto), update the path of caffe.proto in PYTHONPATH accordingly. Replace ${path} with the actual path of caffe.proto.

   export PYTHONPATH=$PYTHONPATH:${path}