Simulator
Usage¶
The Simulator tool is located at SGS_IPU_SDK/Scripts/calibrator/simulator.py. This tool performs the following functions:
- Verification of classified network dataset
- Verification of detection network dataset (COCO dataset supported)
- Single frame image prediction
- Drawing of the target detection result of a single frame image
Run the following script under SGS_IPU_SDK directory to output the Library path (can be ignored, if this has already been done):
cd ~/SGS_IPU_SDK source cfg_env.sh
Enter the working directory for this tool. Here is a usage example:
To test the ILSVRC dataset and return the model precision results:
python3 simulator.py \ -i ~/SGS_Models/resource/classify/ilsvrc2012_val_set100/ \ -m ~/SGS_Models/tensorflow/mobilenet_v2/mobilenet_v2_float.sim \ -l ~/SGS_Models/resource/classify/tensorflow_labels.txt \ -c Classification \ -t Float \ -n mobilenet_v2 \ --num_process 20
You can also achieve this by importing specified image path list file:
python3 simulator.py \ -i ~/SGS_Models/resource/classify/ilsvrc2012_val_set100/file.list \ -m ~/SGS_Models/tensorflow/mobilenet_v2/mobilenet_v2_float.sim \ -l ~/SGS_Models/resource/classify/tensorflow_labels.txt \ -c Classification \ -t Float \ -n mobilenet_v2 \ --num_process 20
To test the COCO2017 dataset and return the model detection result:
python3 simulator.py \ -i ~/SGS_Models/resource/detection/coco2017_val_set100 \ -m ~/SGS_Models/tensorflow/ssd_mobilenet_v1/ssd_mobilenet_v1_float.sim \ -l ~/SGS_Models/resource/detection/coco2017_val_set100.json \ -c Detection \ -t Float \ -n ssd_mobilenet_v1 \ --num_process 10
You can also achieve this by importing specified image path list file:
python3 simulator.py \ -i ~/SGS_Models/resource/detection/coco2017_val_set100/file.list \ -m ~/SGS_Models/tensorflow/ssd_mobilenet_v1/ssd_mobilenet_v1_float.sim \ -l ~/SGS_Models/resource/detection/coco2017_val_set100.json \ -c Detection \ -t Float \ -n ssd_mobilenet_v1 \ --num_process 10
To test a single frame image, draw the detection results on the image, and save it to the ./results/ folder:
python3 simulator.py \ -i ~/SGS_Models/resource/detection/coco2017_val_set100/000000567877.jpg \ -m ~/SGS_Models/tensorflow/ssd_mobilenet_v1/ssd_mobilenet_v1_float.sim \ -c Detection \ -t Float \ -n ssd_mobilenet_v1 \ --draw_result ./results
The following is a description of the parameters used.
Mandatory Parameter¶
-i, --image: Path to image file or image folder (specified image path list file).
-m, --model: Network model file path.
-c, --category: Category of the model, mainly including Classification, Detection, and Unknown.
Classification: The model has one output, which will output the top 5 scores from high to low based on the output score.Detection: The model has four outputs, which will be converted to the bbox position and category of the input image based on the output. Only SigmaStar post-processing operator is supported. For details, see SigmaStar Post-Processing Model. Please use Unknown for other post-processing.Unknown: The model output does not belong to the above two types. It will output all the Tensor values.
-t, --type: Data type of the model.
Float: Floating-point modelFixed: Fixed-point online modelOffline: Fixed-point offline model
Note
-
If the parameter -n/--preprocess is not used, -i/--image cannot designate a path to an image or image folder.
-
If the parameter -i/--image uses imported specified image path list file, the path will be as follows:
When the network model is single-input:
~/SGS_IPU_SDK/image_test/2007000364.jpg ~/SGS_IPU_SDK/image_test/ILSVRC2012_test_00000002.bmp ...
When the network model is multi-input:
~/SGS_IPU_SDK/image_test/2007000364.jpg,~/SGS_IPU_SDK/image_test/ILSVRC2012_test_00000002.bmp ~/SGS_IPU_SDK/image_test/2007000365.jpg,~/SGS_IPU_SDK/image_test/ILSVRC2012_test_00000003.bmp ...
Optional Parameter¶
-l, --label: Label file path of dataset / label of image text description.
-n, --preprocess: Model name, which is related to the image pre-processing method. See Image Pre-Processing Method for details. You can also specify the pre-processing file path after completing the pre-processing file configuration. Without this parameter, the image parameters will need the original data. You can use --save_input to save image data, and then configure the other original data according to their format.
Note
If the model is multi-input, the parameter -n,--preprocess will require multiple pre-processing methods, e.g. -n preprocess1.py,preprocess2.py or --preprocess preprocess1.py,preprocess2.py.
--tool: sgs_simulator file path.
--skip_garbage: Skips useless data of Fixed and Offline model output results.
--dump_rawdata: Saves the binary data of Fixed and Offline model input, with the filename being image name + .bin, in the current directory.
--num_process: The number of processes running simultaneously. (Optional parameter, if not specified, single process will be run by default.) To verify the dataset, this parameter must be set, with the number of processes > 1.
--draw_result: Draws result of the target detection network checkbox. The parameter is the save path of the results file folder (folder will be created automatically if it does not exist) and the image frame threshold value. Use a comma (,) to separate the save path and the threshold value. Input the threshold value to draw results of detection higher than the threshold value. If no threshold value is set, the tool will draw all detection results.
--continue_run: Continues processing of the remaining dataset.
--save_input: Saves the simulator pre-processed file. By default, the pre-processed file is not saved. If this parameter is enabled, the pre-processed file will be saved as is under the path where it is processed in a new folder called tmp_image under the directory ./tmp_image/imagename.
Note¶
-
The simulator tool will search for system variables to obtain the tool path required for the corresponding stage task. Therefore, the parameter -t/--tool, by default, does not have to specify the location of the relevant tool.
-
The simulator tool will create a log folder in the current directory after each run and store the results of model deduction at the output directory under the log folder.
-
When the
-i/--imageparameter is the path of a single image, the simulator will do deduction against that image only; when the -i/--image parameter is the path of an image folder, the simulator will do deduction against all the images within that folder. In the latter case, adding the parameter --num_process (parameter > 1) can enable multi-process deduction. -
When the parameter
-l/--labelis the path of the dataset label file, you should add the --num_process parameter; the dataset verification supports ImageNet (Top1, Top5) and COCO target detection (mmAP). When the parameter of -l/--label is a label text describing the image, the parameter --num_process cannot be used. -
When Unknown is selected for the parameter
-c/--category, the last dimension of the model output Tensor will be aligned upwards to 8-byte:layer46-conv Tensor: { tensor dim:4, Original shape:[1 13 13 255], Alignment shape:[1 13 13 256] The following tensor data shape is alignment shape. tensor data: ... -
For fixed-point and offline models, useful data can be obtained by using the following Python codes (assuming the variable Data is the np.array data with the dimension of [1, 13, 13, 256], the dimension of the real useful data Useful_Data will be [1, 13, 13, 255]):
Useful_Data = Data[:, :, :, :255]
calibrator_custom.simulator¶
calibrator_custom.simulator is a Python-based module for quick quantization and model conversion. With the help of the calibrator_custom.simulator, user can perform quantization and conversion against multi-input and multi-segment network in a more convenient and flexible fashion. The calibrator_custom.simulator includes three types, for parsing models in different stages.
-
calibrator_custom.float_simulatoris used for parsing floating-point models. -
calibrator_custom.fixed_simulatoris used for parsing fixed-point models. -
calibrator_custom.offline_simulatoris used for parsing offline models.
Based on the current docker environment, a Python3.7-precompiled Python module is provided. Since fixed-point models and offline models both employ 32-bit runtime library, the module should be operated under 32-bit Python3.5 environment. For this reason, the docker provided has been updated to 32-bit Python3.5 environment, so using python32 should be fine. For floating-point models to operate normally, 64-bit Python environment should be used instead.
The method of use and the API interface associated with calibrator_custom.float_simulator are as follows:
import calibrator_custom model_path = './mobilenet_v2_float.sim' model = calibrator_custom.float_simulator(model_path)
calibrator_custom.float_simulator¶
When using the calibrator_custom.float_simulator, you have to assign the float.sim model path for creating the float_simulator instance. If the assigned value is incorrect, you will not be able to create the float_simulator instance, and a ValueError message will be returned.
calibrator_custom.simulator¶
get_input_details:
This will return the network model input information (list).
Floating-point model returns the following:
input_details = model.get_input_details() print(input_details) [ { 'name': 'sub_7', 'shape': array([ 1, 513, 513, 3], dtype=int32), 'dtype': <class 'numpy.float32'>, 'index': 0 } ]
The returned list would contain the following dict information depending on the number of input models:
index: Index of the Input Tensor
name: Name of the Input Tensor
shape: Shape of the Input Tensor
dtype: Data type of the Input Tensor
Fixed-point and offline models return the following:
>>> input_details = model.get_input_details() >>> print(input_details) [ { 'index': 0, 'shape': array([ 1, 513, 513, 3]), 'dtype': <class 'numpy.uint8'>, 'name': 'sub_7' , 'input_formats': 'RGB', 'training_input_formats': 'RGB' } ]
get_output_details:
The returned list would contain the following dict information depending on the number of input models:
index: Index of the Input Tensor
name: Name of the Input Tensor
shape: Shape of the Input Tensor
dtype: Data type of the Input Tensor
input_formats: Image input format in actual operation of the network models
training_input_formats: Image input format used in network model training
get_output_detail: Returns the network model output information (list)
Floating-point model returns the following:
>>> output_details = model.get_output_details() >>> print(output_details) [ { 'name': 'MobilenetV2/Conv/Conv2D', 'shape': array([ 1, 257, 257, 30], dtype=int32), 'dtype': <class 'numpy.float32'>, 'index': 0 } ]
The returned list would contain the following dict information depending on the output model(s):
index: Index of the Output Tensor
name: Name of the Output Tensor
shape: Shape of the Output Tensor
dtype: Data type of the Output Tensor
Fixed-point and offline models return the following:
>>> output_details = model.get_output_details() >>> print(output_details) [ { 'index': 0, 'shape': array([ 1, 257, 257, 30]), 'name': 'MobilenetV2/Conv/Conv2D', 'dtype': <class 'numpy.int16'>, 'quantization': (0.00013832777040079236, 0) } ]
The returned list would contain the following dict information depending on the output model(s):
index: Index of the Output Tensor
name: Name of the Output Tensor
shape: Shape of the Output Tensor
dtype: Data type of the Output Tensor
quantization: Scale and zero_point of the Output Tensor (To get the number of floating points, multiply the model output Tensor by scale.)
set_input:
This will set the network model input data.
>>> model.set_input(0, img_data)
The 0 in the input data represents the index of the Input Tensor, which can be obtained from the return value of get_input_details(). The img_data refers to the data in numpy.ndarray format which correspond to the model’s input shape and dtype. Incorrect shape or dtype will cause set_input to return a ValueError message. If the model has multiple inputs, you can call set_input multiple times to get the input data of the Tensor corresponding to the index based on the return value of the get_input_details().
invoke:
This will invoke the model to operate once.
>>> model.invoke()
Before calling the invoke command, please use set_input to set the input data first. Calling invoke directly without calling set_input beforehand will return a ValueError message.
get_output:
This will get the network model output data.
>>> result = model.get_output(0)
The 0 in the output data represents the index of the Output Tensor, which can be obtained from the return value of get_output_details(). The output data returned is in numpy.ndarray format. If the model has multiple outputs, you can call get_output multiple times to get the output data of the Tensor corresponding to the index based on the return value of the get_output_details().
Suppose the model output shape is [1, 257, 257, 30]:
The shape of floating-point model in the print result is:
>>> print(result.shape) (1, 257, 257, 30)
The shape of fixed-point and offline models in the print result is:
>>> print(result.shape) (1, 257, 257, 32)
Because fixed-point and offline model output Tensor simulates the storage method on hardware, the last dimension of the Tensor will be aligned upwards, and so the output dimension will be the Tensor aligned dimension. The last aligned data is of no use; you can use the following method to remove it:
>>> result = result[..., :output_details[0]['shape'][-1]] >>> print(result.shape) (1, 257, 257, 30)
get_tensor_details:
This will get the tensor information (list) of the network model.
Floating-point model returns the following:
>>> tensor_details = model.get_tensor_details() >>> print(tensor_details) [ { 'name': 'MobilenetV2/Conv/Conv2D', 'shape': array([ 1, 257, 257, 30], dtype=int32), 'dtype': 'FLOAT32', 'qtype': 'INT16' }, { 'name': 'MobilenetV2/Conv/Conv2D_bias', 'shape': array( [ 2, 30], dtype=int32), 'dtype': 'FLOAT32', 'qtype': 'INT16' }, { 'name': 'MobilenetV2/Conv/weights/read', 'shape': array( [30, 3, 3, 3], dtype=int32), 'dtype': 'FLOAT32', 'qtype': 'INT8' }, { 'name': 'sub_7', 'shape': array([ 1, 513, 513, 3], dtype=int32), 'dtype': 'FLOAT32', 'qtype': 'UINT8' } ]
The returned list would contain the following dict information depending on the model Tensor:
name: Name of the Tensor
shape: Shape of the Tensor
dtype: Data type of the Tensor
qtype: Quantization type of Tensor
Fixed-point and offline models return the following, if the dequantizations in [OUTPUT_CONFIG] of input_config.ini is configured as FALSE:
>>> tensor_details = model.get_tensor_details() >>> print(tensor_details) [ { 'shape': array([ 1, 257, 257, 30]), 'quantization': [(0.00013832777040079236, 0)], 'min': [-4.230099201202393], 'max': [4.532586097717285], 'name': 'MobilenetV2/Conv/Conv2D', 'dtype': 'INT16' }, { 'shape': array([ 2, 30]), 'quantization': [], 'min': [0.0], 'max ': [1.0], 'name': 'MobilenetV2/Conv/Conv2D_bias', 'dtype':'INT16' }, { 'shape': array([30, 3, 3, 3]), 'quantization': [(0.004813921172171831, 0)], 'min': [-0.5498989820480347], 'max': [0.6113680005073547], 'name': 'MobilenetV2/Conv/weights/read', 'dtype': 'INT8' }, { 'shape': array([ 1, 513, 513, 3 ]), 'quantization': [(0.007843137718737125, 128)], 'min': [-1 .0], 'max': [1.0], 'name': 'sub_7', 'dtype': 'UINT8' } ]
The returned list would contain the following dict information depending on the number of the model Tensors:
name: Name of the Tensor
shape: Shape of teh Tensor
dtype: Data type of the Tensor
quantization: Scale and zero_point of the Tensor
min: Min. value of the Tensor
max: Max. value of the Tensor
calibrator_custom.SIM_Simulator¶
For multi-input and multi-segment network, calibrator_custom.SIM_Simulator can be utilized to realize simultaneous conversion through a simple definition.
The calibrator_custom.SIM_Simulator is a class which has already been implemented, in which only the forward parameter needs to be defined. Once the forward method is implemented, you will be able to complete the conversion successfully.
In the following example, which is based on SGS_IPU_SDK/Scripts/examples/sim_simulator.py, we will illustrate the method of use of calibrator_custom.SIM_Simulator:
import calibrator_custom class Net(calibrator_custom.SIM_Simulator): def __init__(self): super().__init__() self.model = calibrator_custom.fixed_simulator(model_path) def forward(self, x): out_details = self.model.get_output_details() self.model.set_input(0, x) self.model.invoke() result_list = [] for idx in range(len(out_details)): result = self.model.get_output(idx) # for Fixed and Offline model if result.shape[-1] != out_details[idx]['shape'][-1]: result = result[..., :out_details[idx]['shape'][-1]] if out_details[idx]['dtype'] == np.int16: scale, _ = out_details[idx]['quantization'] result = np.dot(result, scale) result_list.append(result) return result_list
In the foregoing, the forward parameter is the model input. If there are multiple inputs, the forward parameter can be increased.
To create a calibrator_custom.SIM_Simulator instance:
net = Net()
To call a calibrator_custom.SIM_Simulator instance:
result = net(img_gen, num_process=4)
To call a calibrator_custom.SIM_Simulator instance, you have to assign the numpy.ndarray format or the image generator of the input image.
When num_process > 1, img_gen must be an image generator.
Image generator (img_gen)
For the convenience of multi-input and multi-segment network model conversion, img_gen is provided to facilitate the arrangement of input image sequence. If the model has multiple inputs, the generator will follow the input sequence defined by the forward parameter and return a list with multiple numpy.ndarray output data.
calibrator_custom.utils.image_preprocess_func will use the pre-defined preprocessing method.
preprocess_func = calibrator_custom.utils.image_preprocess_func(model_name) def image_generator(folder_path, preprocess_func, norm): images = [os.path.join(folder_path, img) for img in os.listdir(folder_path)] for image in images: img = preprocess_func(image, norm) yield [img] img_gen = image_generator('./images', preprocess_func, False)