Export

In this section, we will present how to generate an N2D2 export with the Python API. Exports are standalone code which are generated by N2D2.

If you want more specific information about an export please refer to it in the export section.

With the Python API, you can only export an n2d2.cells.DeepNetCell.

Once you have trained your model, you can convert your n2d2.cells.Sequence into a n2d2.cells.DeepNetCell with the method n2d2.cells.Sequence.to_deepnet_cell().

If you have used another way to train your model such as the interoperability with Keras or PyTorch, you can retrieve the n2d2.cells.DeepNetCell with the appropriate get_deepnet_cell method.

Warning

When using interoperability, you do not associate a n2d2.provider.DataProvider to the n2d2.cells.DeepNetCell. So if you want to calibrate your network, you need ot specify a data provider otherwise N2D2 will fail to generate the export.

Listing available cells for an export

If you want to get the list available cells for an export you can use the function n2d2.export.list_exportable_cell().

n2d2.export.list_exportable_cell(export_name: str) → None

Print a list of exportable cells.

Parameters:

export_name (str) – Can be one of : C, CPP, CPP_TensorRT, CPP_STM32.

Export C

Exportable cells

Documentation

n2d2.export.export_c(deepnet_cell: DeepNetCell, provider: Provider = None, **kwargs) → None

Generate a C export of the neural network.

List of exportable cells :

Cell Name

Available

Activation

No

BatchNorm2d

Yes

Conv

Yes

Deconv

No

Dropout

No

ElemWise

Yes

Fc

Yes

Padding

No

Pool

Yes

Reshape

No

Resize

Yes

Scaling

Yes

Softmax

Yes

Transformation

No

Transpose

No
Parameters:

  • deepnet_cell – The Neural network you want to export.

  • provider – Data provider to use for calibration, default=None

  • nb_bits (int, optional) – Number of bits for the weights and signals. Must be 8, 16, 32 or 64 for integer export, or -32, -64 for floating point export, default=8

  • qat_sat (bool, optional) – Fuse a QAT trained with SAT method, default=False

  • export_no_unsigned (bool, optional) – If True, disable the use of unsigned data type in integer exports, default=False

  • calibration (int, optional) – The number of stimuli used for the calibration (0 = no calibration, -1 = use the full test dataset), default=0

  • export_no_cross_layer_equalization (bool, optional) – If True, disable the use of cross layer equalization in integer exports, default=False

  • wt_clipping_mode (str, optional) – Weights clipping mode on export, can be NONE, MSE or KL_DIVERGENCE, default=”NONE”

  • act_clipping_mode (str, optional) – activation clipping mode on export, can be NONE, MSE or KL_DIVERGENCE or Quantile, default=”MSE”

  • act_scaling_mode (str, optional) – activation scaling mode on export, can be NONE, FIXED_MULT16, SINGLE_SHIFT or DOUBLE_SHIFT, default=”SINGLE_SHIFT”

  • act_quantile_value (float, optional) – Quantile value for Quantile clipping mode, default=0.9999

  • act_rescale_per_output (bool, optional) – If True, rescale activation per output on export, default=False

  • calibration_reload (bool, optional) – If True, reload and reuse the data of a previous calibration, default=False

  • report (int, optional) – Number of steps between reportings, default=100

  • export_nb_stimuli_max (int, optional) – Maximum number of stimuli to export (0 = no dataset export, -1 = unlimited), default=-1

  • wt_round_mode (str, optional) – Weights clipping mode on export, can be NONE, RINTF, default=”NONE”

  • b_round_mode (str, optional) – Biases clipping mode on export, can be NONE, RINTF, default=”NONE”

  • c_round_mode (str, optional) – Clip clipping mode on export, can be NONE, RINTF, default=”NONE”

  • find_lr (int, optional) – Find an appropriate learning rate over a number of iterations, default=0

Example

n2d2.export.export_c(
    DEEPNET_CELL,
    nb_bits=8,
    export_nb_stimuli_max=-1,
    calibration=-1)

Frequently asked question

Scaling and ElemWise are the only layers supported in Fixed-point scaling mode

If you try to export an untrained model to C in int8, you may come across this error :

RuntimeError: Scaling and ElemWise are the only layers supported in Fixed-point scaling mode.

This is due to the weights parameter being initialized with high value. If you look at the log you may see warning lines like this :

Scaling (8.78614) > 1 for layer "model/dense/MatMul:0" is not supported with Single/Double-shift scaling. Falling back to Fixed-point scaling for this layer.

Training the model before exporting it will probably solve this issue.

Export CPP

Exportable cells

Documentation

n2d2.export.export_cpp(deepnet_cell: DeepNetCell, provider: Provider = None, optimize_buffer_memory: bool = True, **kwargs) → None

Generate a CPP export of the neural network.

List of exportable cells :

Cell Name

Available

Activation

Yes

BatchNorm2d

Yes

Conv

Yes

Deconv

Yes

Dropout

No

ElemWise

Yes

Fc

Yes

Padding

Yes

Pool

Yes

Reshape

Yes

Resize

Yes

Scaling

Yes

Softmax

Yes

Transformation

No

Transpose

Yes
Parameters:

  • deepnet_cell – The Neural network you want to export.

  • provider – Data provider to use for calibration, default=None

  • nb_bits (int, optional) – Number of bits for the weights and signals. Must be 8, 16, 32 or 64 for integer export, or -32, -64 for floating point export, default=8

  • qat_sat (bool, optional) – Fuse a QAT trained with SAT method, default=False

  • export_no_unsigned (bool, optional) – If True, disable the use of unsigned data type in integer exports, default=False

  • calibration (int, optional) – The number of stimuli used for the calibration (0 = no calibration, -1 = use the full test dataset), default=0

  • export_no_cross_layer_equalization (bool, optional) – If True, disable the use of cross layer equalization in integer exports, default=False

  • wt_clipping_mode (str, optional) – Weights clipping mode on export, can be NONE, MSE or KL_DIVERGENCE, default=”NONE”

  • act_clipping_mode (str, optional) – activation clipping mode on export, can be NONE, MSE or KL_DIVERGENCE or Quantile, default=”MSE”

  • act_scaling_mode (str, optional) – activation scaling mode on export, can be NONE, FLOAT_MULT, FIXED_MULT16, SINGLE_SHIFT or DOUBLE_SHIFT, default=”FLOAT_MULT”

  • act_quantile_value (float, optional) – Quantile value for Quantile clipping mode, default=0.9999

  • act_rescale_per_output (bool, optional) – If True, rescale activation per output on export, default=False

  • calibration_reload (bool, optional) – If True, reload and reuse the data of a previous calibration, default=False

  • report (int, optional) – Number of steps between reportings, default=100

  • export_nb_stimuli_max (int, optional) – Maximum number of stimuli to export (0 = no dataset export, -1 = unlimited), default=-1

  • wt_round_mode (str, optional) – Weights clipping mode on export, can be NONE, RINTF, default=”NONE”

  • b_round_mode (str, optional) – Biases clipping mode on export, can be NONE, RINTF, default=”NONE”

  • c_round_mode (str, optional) – Clip clipping mode on export, can be NONE, RINTF, default=”NONE”

  • find_lr (int, optional) – Find an appropriate learning rate over a number of iterations, default=0

  • optimize_buffer_memory (bool, optional) – If False deactivate memory optimization, default=True

Example

n2d2.export.export_cpp(
    DEEPNET_CELL,
    nb_bits=8,
    export_nb_stimuli_max=-1,
    calibration=-1)

Export CPP TensorRT

Exportable cells

Documentation

n2d2.export.export_tensor_rt(deepnet_cell: DeepNetCell, provider: Provider = None, **kwargs) → None

Generate a TensorRT export of the neural network.

List of exportable cells :

Cell Name

Available

Activation

Yes

BatchNorm2d

Yes

Conv

Yes

Deconv

Yes

Dropout

No

ElemWise

Yes

Fc

Yes

Padding

Yes

Pool

Yes

Reshape

Yes

Resize

Yes

Scaling

No

Softmax

Yes

Transformation

No

Transpose

Yes
Parameters:

  • deepnet_cell – The Neural network you want to export.

  • provider – Data provider to use for calibration, default=None

  • nb_bits (int, optional) – Only 32 floating point precision is available for this export. You can calibrate your network later with the export tools, default=-32

  • qat_sat (bool, optional) – Fuse a QAT trained with SAT method, default=False

  • export_no_unsigned (bool, optional) – If True, disable the use of unsigned data type in integer exports, default=False

  • calibration (int, optional) – The number of stimuli used for the calibration (0 = no calibration, -1 = use the full test dataset), default=0

  • export_no_cross_layer_equalization (bool, optional) – If True, disable the use of cross layer equalization in integer exports, default=False

  • wt_clipping_mode (str, optional) – Weights clipping mode on export, can be NONE, MSE or KL_DIVERGENCE, default=”NONE”

  • act_clipping_mode (str, optional) – activation clipping mode on export, can be NONE, MSE or KL_DIVERGENCE or Quantile, default=”MSE”

  • act_scaling_mode (str, optional) – activation scaling mode on export, can be NONE, FLOAT_MULT, FIXED_MULT16, SINGLE_SHIFT or DOUBLE_SHIFT, default=”FLOAT_MULT”

  • act_quantile_value (float, optional) – Quantile value for Quantile clipping mode, default=0.9999

  • act_rescale_per_output (bool, optional) – If True, rescale activation per output on export, default=False

  • calibration_reload (bool, optional) – If True, reload and reuse the data of a previous calibration, default=False

  • report (int, optional) – Number of steps between reportings, default=100

  • export_nb_stimuli_max (int, optional) – Maximum number of stimuli to export (0 = no dataset export, -1 = unlimited), default=-1

  • wt_round_mode (str, optional) – Weights clipping mode on export, can be NONE, RINTF, default=”NONE”

  • b_round_mode (str, optional) – Biases clipping mode on export, can be NONE, RINTF, default=”NONE”

  • c_round_mode (str, optional) – Clip clipping mode on export, can be NONE, RINTF, default=”NONE”

  • find_lr (int, optional) – Find an appropriate learning rate over a number of iterations, default=0

Example

n2d2.export.export_tensor_rt(DEEPNET_CELL)