Installing Caffe Patches

Prerequisites for Merging proto Files

You have prepared the custom.proto file and uploaded it to any directory on the AMCT server.

The following is an example. The custom layers and parameters in the following example are for reference only. You can customize the parameters based on the site requirements.

message LayerParameter {
  optional ReLU6Parameter relu6_param = 2060;
  optional ROIPoolingParameter roi_pooling_param = 8266711;
}

message ReLU6Parameter {
  optional float negative_slope = 1 [default = 0];
}

message ROIPoolingParameter {
  // Pad, kernel size, and stride are all given as a single value for equal
  // dimensions in height and width or as Y, X pairs.
  optional uint32 pooled_h = 1 [default = 0]; // The pooled output height
  optional uint32 pooled_w = 2 [default = 0]; // The pooled output width
  // Multiplicative spatial scale factor to translate ROI coords from their
  // input scale to the scale used when pooling
  optional float spatial_scale = 3 [default = 1];
}

**HPM consists of the following:**

1. Register a custom layer using LayerParameter.

   message LayerParameter {
     # user definition fields, each field takes one line.
     optional FieldType0 field_name0 = field_num0;
     optional FieldType1 field_name1 = field_num1;
   }

   This field is used to declare the custom layer in LayerParameter. The custom layer needs to be added to LayerParameter to allow the layer in the Caffe framework to write to and read from it. There are four parts to this declaration:

   • optional: indicates that the definition is optional in LayerParameter and can only be set to optional.
   • FieldType: declares the user-defined type corresponding to the current field. The corresponding message definition is required.
   • field_name: ID of the current statement, which must be unique. If a conflict occurs, you need to change the ID and use the ID to access the corresponding content.
   • field_num: ID of the current declaration, which must be unique. If a conflict occurs, you need to change the ID. It is recommended that the value range be less than 5000 and do not conflict with the ID in the caffe.proto file provided by ATC. In the binary caffemodel, the ID needs to be used to parse the corresponding field.

   The following is an example:

   message LayerParameter {
     optional ReLU6Parameter relu6_param = 2060;
     optional ROIPoolingParameter roi_pooling_param = 8266711;
   }

   

   • Maintain the sequence number of a custom layer in custom.proto below 5000 and avoid conflict with those of the built-in layers in the ATC caffe.proto file.
   • The sequence numbers of the layers in amct_custom.proto start from 200000.
   • The sequence numbers of the ATC custom layers in caffe.proto are within the range [5000, 200000).
2. Define the custom layer parameters for the message layer.

   message ReLU6Parameter {
     optional float negative_slope = 1 [default = 0];
   }

   For details, see google protobuf.

   Ensure that the value of this field does not conflict with amct_custom.proto at the custom layer of AMCT. If there is a conflict, an error message will display during .proto merging. You can modify the value based on the error message. If it conflicts with the built-in caffe.proto of ATC, the message definition takes precedence.

   Currently, the custom messages include QuantParameter, DeQuantParameter, IFMRParameter, LSTMQuantParameter, SearchNParameter, RetrainDataQuantParameter, RetrainWeightQuantParameter, SingleLayerRecord and ScaleOffsetRecord. The custom layer name must be unique.

Procedure

You can run the auto installation script install.py in caffe_patch. If successful, the patches in caffe_patch are installed automatically in the caffe-master project directory. In this process, the .proto files will be merged and the original source code and dynamic library files will be replaced with new ones. After completing the installation or manual modification, recompile the Caffe environment following the steps below:

1. Decompress the Caffe patch package.

   Run the following command as the AMCT installation user to decompress the caffe_patch.tar.gz software package:

   tar -zxvf caffe_patch.tar.gz

   Find the following extracted files:

   caffe_patch
   ├—─  include //Stores the header files and common functions of the custom layer.
   │   └── caffe
   │       ├── layers
   │       └── util
   ├──  install.py //Script for merging .proto files, installing the patch, and executing the source code and dynamic library files in the Caffe environment.
   ├──  merge_proto //proto merge directory
   │   ├── amct_custom.proto         // AMCT custom layers and updated layers of Caffe-master compared with Caffe 1.0.
   Built-in caffe.proto in the │ ├── caffe.proto //ATC tool
   Script for combining │ └── merge_proto.sh //protos. This script is invoked by install.py during actual combination.
   ├──  patch //Patch file related to the LSTM layer
   │ ├── lstm_calibration_layer.cpp.patch //Used to generate lstm_calibration_layer.cpp in the caffe-master/src/caffe/layers directory.
   │ ├── lstm_calibration_layer.hpp.patch //Used to generate lstm_calibration_layer.hpp in the caffe-master/include/caffe/layers directory.
   │ ├── lstm_quant_layer.cpp.patch //Used to generate lstm_quant_layer.cpp in the caffe-master/src/caffe/layers/ directory.
   │ └── lstm_quant_layer.hpp.patch //Used to generate lstm_quant_layer.hpp in the caffe-master/include/caffe/layers/ directory.
   ├──  quant_lib //Stores the core dynamic library of the quantization algorithm.
   │   ├── libquant_gpu.so
   │   └── libquant.so
   ├──  src //Stores the source code files and public functions for implementing the custom layer.
       └── caffe
           ├── layers
           └── util

2. Switch to the directory where the caffe_patch/install.py script is stored and run the following command:

   python3 install.py --caffe_dir CAFFE_DIR [--custom_proto CUSTOM_PROTO_FILE]

   The command-line options are described as follows.

   Table 1 Parameters in the quantization script

   Option	Remarks
   --h	This function is optional. Help information.
   --caffe_dir	Required. Sets the directory of the Caffe source code, which can be relative or absolute.
   --custom_proto	This function is optional. User-defined path of the custom.proto file. The path can be a relative path or an absolute path.

   Sample command:

   python3 install.py  --caffe_dir caffe-master  --custom_proto custom.proto

   You should see information similar to the following if the execution is successful:

   1 2 3 4 5 6 7 8 9 10 11 12 13 14

   Copy the new source code and dynamic library files to the Caffe-master project directory in the Caffe environment. [INFO]Begin to copy source files, header files and quant_lib to '$HOME/AMCT/AMCT_CAFFE/caffe-master' [INFO]Finish copy source files, header files and quant_lib to '$HOME/AMCT/AMCT_CAFFE/caffe-master' Installing Patch [INFO]Begin to install patch. [INFO]Install patch 'lstm_calibration_layer.cpp.patch' successfully. [INFO]Install patch 'lstm_quant_layer.hpp.patch' successfully. [INFO]Install patch 'lstm_quant_layer.cpp.patch' successfully. [INFO]Install patch 'lstm_quant_layer.hpp.patch' successfully. [INFO]Finish install patch. # Merging proto [INFO]Merge and replace "caffe.proto" success. Modifying Makefile [INFO]Merge and replace "Makefile" success.

   The install.py script allows patches to be installed repeatedly.

   • If patch installation fails, restore the caffe-master/src/caffe/layers/lstm_layer.cpp and caffe-master/include/caffe/layers/lstm_layer.hpp files in the Caffe project to the native files of Caffe-master.
   • If an error message is displayed during .proto merging, fix the error by referring to How Can I Resolve a Proto Merging Error?.
   • If the Makefile fails to be modified, modify it as prompted. If the script is executed successfully, the Makefile will not be modified again when the script is executed again.
3. Check the Python layer configuration. If the Python layer is not configured, add the implementation of the Python layer.

   Modify the caffe-master/Makefile.config file in the Caffe project.

   Add the implementation of the Python layer.

   # Uncomment to support layers written in Python (will link against Python libs)
   WITH_PYTHON_LAYER := 1

4. (Optional) Add C++11 standard code support.

   New AMCT operators require C++11 support. As such, add the --std=c++11 compilation option to caffe-master/Makefile. As the file will be modified during the install.py execution in 2, check that the --std=c++11 compilation option is added to caffe-master/Makefile as shown in the following line in bold. Skip this step if it is added.

   # Complete build flags.
   COMMON_FLAGS += $(foreach includedir,$(INCLUDE_DIRS),-I$(includedir)) --std=c++11
   CXXFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS)
   NVCCFLAGS += -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)

5. Return to the caffe-master directory and run the following commands to recompile the Caffe and PyCaffe environments:

   # If the Caffe project has been compiled before the patch is installed, run the make clean command and then the compilation command after the patch is installed.
   make clean
   make all -j && make pycaffe -j

   As AMCT needs to parse your Caffe model, whenever the caffe.proto file is updated (by adding custom layers, for example), you must rebuild a caffe_pb2.py file for AMCT.

   

   If you use the protoc command to rebuild caffe.proto (for example, protoc --python_out=./caffe.proto), update the path of caffe.proto in PYTHONPATH accordingly. Replace ${path} with the actual path of caffe.proto.

   export PYTHONPATH=$PYTHONPATH:${path}