Adding New Layers to Caffe Model

Description¶

This chapter applies to Caffe model only. For layers not yet implemented by SigmaStar, you can use the underlying operators provided by SigmaStar to add them. The underlying operators are similar to those of the TensorFlow. For supported operators, please refer to TensorFlow-Supported Operators. The flow for conversion is as illustrated below:

CaffeConvert Tool (code located atSGS_IPU_SDK/Scripts/ConvertTool/ConvertTool.py) is developed based on the design of Xiaomi’s open source Mace framework. To add a new layer, go to Xiaomi MACE to download the source code, then configure the compilation environment following the official Mace documentation or use the dock environment provided by SigmaStar.

Taking reorg layer as an example, we will explain, in the sections that follow, how to add a new layer, principally by modifying the following files.

Modifying Caffe Proto File¶

Modifying Caffe Proto

After downloading the Mace source code, replace the caffe.proto file in the Mace engineering path mace/third_party/caffe with SGS_IPU_SDK/Scripts/CaffeConvertTool/third_party/caffe/caffe.proto. The reason behind this is that, the Caffe layers supported by SigmaStar are much greater than the number of Caffe layers supported by the native Mace, because SigmaStar has done a second development based on the Mace framework. Hence, make sure the new layer is added on the basis of SigmaStar’s Caffe proto file.

optional ReorgParameter reorg_param = 157;
}
message ReorgParameter {
optional uint32 stride = 1;
optional bool reverse = 2 [default = false];
}

Compiling Proto File

Under the Mace development environment, enter the following command:

cd pathToMace/
bazel build third_party/caffe/caffe_py

A caffe_pb2.py file will be generated in bazel-genfiles. Use the modified caffe.proto and the generated caffe_pb2.py to replace the same-name files located at SGS_IPU_SDK/Scripts/CaffeConvertTool/third_party/caffe, then modify the code in the order specified below.

Modifying caffe_converter.py¶

The caffe_converter.py file is located at SGS_IPU_SDK/Scripts/CaffeConvertTool/mace/python/tools/converter_tool/caffe_converter.py. This file will convert Caffe model into Mace model while maintaining the parameters of the layer.

Reorg':self.convert_Reorg,
... 
... 
def convert_Reorg(self, caffe_op): 
    op = self.convert_general_op(caffe_op) 
    op.type = "Reorg" 
    param = caffe_op.layer.reorg_param 
    stride_arg = op.arg.add() 
    stride_arg.name = "stride" 
    stride_arg.i = 1 #default is true 
    if param.HasField('stride'): 
        stride_arg.i = int(param.stride)

Modifying shape_inference.py¶

The shape_inference.py file is located at SGS_IPU_SDK/Scripts/CaffeConvertTool/mace/python/tools/converter_tool/shape_inference.py. This file is used to calculate the output shape of the layer.

Reorg': self.infer_shape_Reorg, 
... 
... 
def infer_shape_Reorg(self, op): 
    #only support stride is 2 
    output_shape = self._output_shape_cache[op.input[0]] 
    input_shape = list(self._output_shape_cache[op.input[0]]) 
    input_n = input_shape[0] 
    input_c = input_shape[1] 
    input_h = input_shape[2] 
    input_w = input_shape[3] 
    output_shape = [input_n,int(input_c*4),int(input_h/2),int(input_w/2)] 
    self.add_output_shape(op, [output_shape])

Modifying SGSModel_transform.py¶

As illustrated in the figure below, the reorg layer, in the order of NCHW, has done a rearrangement of the data.

Therefore, the combination of operators we finally splitted goes this like:

Now, compile the code located at SGS_IPU_SDK/Scripts/CaffeConvertTool/mace/python/tools/SGSModel_transform.py, in the order of sequence of the operator combination. This file is used to increase the split functions of the layer, so that you can use the basic operators provided by SigmaStar to implement the layer.

Reorg':self.split_Reorg,
...
...
def split_Reorg(self, op):
#only support strid is 2
[n,c,h,w] = op.output_shape[0].dims[:]
c = int(c/4)
op_name = op.name
xi = op.input[0]
...
...

By now you have completed the new layer support.

Notes and Tips on Layer Splitting¶

Data Dimension Issue¶

The Caffe network arranges data in the order of NCHW, whereas SigmaStar calculates data in the order of NHWC (similar to TensorFlow). Therefore, when converting a model into sim, the shapes and data of the four dimensions will be transposed, from NCHW to NHWC. A reference code is available from: SGS_IPU_SDK/Scripts/CaffeConvertTool/mace/python/tools/SGSModel_converter.py.

To transpose tensor data:

def _creatBuffer(self):
... 
... 
if len(ori_shape) == 4: 
#transpose data to NHWC 
    six.print_("Reshape ",tensor.name,"to NHWC") 
    data = np.array(ori_data) 
    data = data.reshape(ori_shape) 
    data = data.transpose(0,2,3,1)

To convert a shape:

def _creatTensor(self): 
... 
... 
if len(shape) == 4 and data_format == mace_pb2.DT_NCHW: 
    Transformer.transpose_shape(shape, [0, 2, 3, 1]) 
tflite.Tensor.TensorStartShapeVector(self._builder,len(shape))

Since the shape will be forcibly changed while creating the sim model, special attention must be paid to the following when doing the splitting:

Exchange sequence with regard to parameters involving axis:

For example, in the Caffe model, ‘concat layer’ needs to do concatenation against C latitude, whose axis is 1. After the network conversion, C is changed to the lowest dimension, so the axis must be modified to 3. The code to this effect is as follows. Similar cases include split, PRelu, etc.
```
def split_Concat(self, op): 
... 
... 
if len(output_shape[0].dims) == 4:#nchw -> nhwc 
    if arg[i].i == 1: 
        arg[i].i = 3 
    else if arg[i].i == 2: 
        arg[i].i = 1 
    else if arg[i].i == 3: 
        arg[i].i = 2
```
When specifying the operator output_shape, especially for 4-dimension output, watch out for the effect of order changes. For example, if you want to have the tensor output in the order of [A,B,C,D], then the value set in the code should be [A,D,B,C], because [A,D,B,C] will become [A,B,C,D] after the code conversion.
Layers under NCHW data arrangement have very strong regularity. Hence, you can start with NCHW operation against the reorg layer, for example.
To view the various properties and shapes, open the converted sim model with the Netron tool in the SDK.

Recommended Splitting Process¶

In summary, we recommend following the flow illustrated below to do the splitting.

Data Comparison and Verification¶

In the early stage, you can use the TensorFlow operator created by Python to perform verification. After the verification is done, you can modify the related codes in SGS_IPU_SDK with reference to the method described in DUMPDEBUG TOOL to dump the final result for comparison with the golden data.