Pruning¶

Getting Started¶

N2D2 provides a pruning module to perform pruning operations on your model in order to reduce its memory footprint. The module works like the QAT module i.e. it is possible to carry out trainings with pruned weights in order to improve the performance of the network. Only weights can be pruned so far.

Example with Python¶

class n2d2.quantizer.PruneCell(**config_parameters)¶

Finetune Pruning

N2D2()¶: Return the N2D2 object.

__init__(**config_parameters)¶

Parameters:

threshold (float, optional) – Range of Quantization, can be 1 for binary, 255 for 8-bits etc.., default=255
prune_mode (string, optional) – Type of pruning mode, can be Identity, Static or Gradual, default=``Identity``
prune_filler (string, optional) – Type of pruning filler, can be Random or IterNonStruct, default=``Random``
delta (float, optional) – For IterNonStruct filler, factor to iteractively prune the data
start (float, optional) – For Gradual mode, start value for threshold
stepsize (int, optional) – For Gradual mode, size of the steps before update the masks
gamma (float, optional) – For Gradual mode, adding threshold value when updating masks

add_biases(biases: Tensor, diff_biases: Tensor)¶

Parameters:

biases – Biases
biases – n2d2.Tensor
diff_biases – Diff Biases
diff_biases – n2d2.Tensor

add_weights(weights: Tensor, diff_weights: Tensor)¶

Parameters:

weights – Weights
weights – n2d2.Tensor
diff_weights – Diff Weights
diff_weights – n2d2.Tensor

get_current_threshold()¶: Get the current threshold of the cell.

get_parameter(key)¶

Parameters:: key (str) – Parameter name

get_pruned_masks(input_idx)¶: Access the masks of the pruner.

get_quantized_weights(input_idx)¶: Access the quantized weights of the cell the quantizer is attached to.

Example of code to use the n2d2.quantizer.PruneCell in your scripts:

for cell in model:
    ### Add Pruning ###
    if isinstance(cell, n2d2.cells.Conv) or isinstance(cell, n2d2.cells.Fc):
        cell.quantizer = n2d2.quantizer.PruneCell(prune_mode="Static", threshold=0.3, prune_filler="IterNonStruct")

Some explanations with the differents options of the n2d2.quantizer.PruneCell :

Pruning mode¶

3 modes are possible:

Identity: no pruning is applied to the cell
Static: all weights of the cell are pruned to the requested threshold at initialization
Gradual: the weights are pruned to the start threshold at initialization and at each update of the current threshold, it is increased by gamma until it reaches threshold. By default, the update is performed at the end of each epoch (possible to change it with stepsize)

Warning: if you use stepsize, please indicate the number of steps and not the number of epochs. For example, to update each two epochs, write:

n2d2.quantizer.PruneCell(prune_mode="Gradual", threshold=0.3, stepsize=2*DATASET_SIZE)

Where DATASET_SIZE is the size of the dataset you are using.

Pruning filler¶

2 fillers are available to fill the masks:

Random: The masks are filled randomly
IterNonStruct: all weights below than the delta factor are pruned. If this is not enough to reach threshold, all the weights below 2 “delta” are pruned and so on…

Important: With n2d2.quantizer.PruneCell, quant_mode and range are not used.

Example with INI file¶

The common set of parameters for any kind of Prune Quantizer.

Option [default value]	Description
`QWeight`	Quantization / Pruning method, choose `Prune` to activate the Pruning mode.
`QWeight.PruningMode` [`Identity`]	Pruning mode, can be `Identity`, `Static` or `Gradual`
`QWeight.PruningFiller` [`Random`]	Pruning filler for the weights, can be `Random`, `IterNonStruct` or `None`
`QWeight.Threshold` [`0.2`]	Weight threshold to be pruned, 0.2 means 20% for example
`QWeight.Delta` [`0.001`]	Factor for iterative pruning, use it with `IterNonStruct` pruning filler
`QWeight.StartThreshold` [`0.1`]	Starting threshold, use it with `Gradual` pruning mode
`QWeight.StepSizeThreshold` [`0`]	Step size for the threshold update, use it with `Gradual` pruning mode
`QWeight.GammaThreshold` [`0.05`]	Value to add to current threshold during its update, use it with `Gradual` pruning mode

Example of code to use the Prune Quantizer in your scripts:

[conv1]
Input=sp
Type=Conv
KernelDims=5 5
NbOutputs=6
ActivationFunction=Rectifier
WeightsFiller=HeFiller
ConfigSection=common.config
QWeight=Prune
QWeight.PruningMode=Static
QWeight.PruningFiller=IterNonStruct
QWeight.Threshold=0.3
QWeight.StartThreshold=0.1
QWeight.GammaThreshold=0.1

All explanations in relation to the parameters of Prune Quantizer are provided in the python section of this page.