Model Optimization - NNCF
===============================================================================================

.. meta::
   :description: Learn about the optimization methods offered by OpenVINO's NNCF

.. toctree::
   :maxdepth: 1
   :hidden:

   model-optimization-guide/weight-compression
   model-optimization-guide/quantizing-models-post-training
   model-optimization-guide/compressing-models-during-training


Model optimization means altering the model itself to improve its performance and reduce
its size. It is an optional step, typically used only at the development stage, so that
a pre-optimized model is used in the final AI application.

In OpenVINO, the default optimization tool is NNCF (Neural Network Compression Framework).
It is a `set of compression algorithms <https://github.com/openvinotoolkit/nncf/blob/develop/README.md>`__,
organized as a Python package, that make your models smaller and faster. Note that NNCF
is **not part of the OpenVINO package**, so it needs to be installed separately. It supports
models in **OpenVINO IR**, **PyTorch** and **ONNX** formats, offering
the following main optimizations:

.. image:: ../assets/images/WHAT_TO_USE.svg


| :doc:`Weight Compression <model-optimization-guide/weight-compression>`:
|      An easy-to-use method for Large Language Model footprint reduction and inference
       acceleration.

| :doc:`Post-training Quantization <model-optimization-guide/quantizing-models-post-training>`:
|      Designed to optimize deep learning models by applying 8-bit integer quantization. Being
       the easiest way to optimize a model it does not require its retraining or fine-tuning
       but may result in a drop in accuracy. If the accuracy-performance tradeoff is not
       acceptable, Training-time Optimization may be a better option.

| :doc:`Training-time Optimization <model-optimization-guide/compressing-models-during-training>`:
|      Involves a suite of advanced methods such as Sparsity, as well
       as Quantization-aware Training. This kind of optimization requires the use of the PyTorch framework.



Recommended workflows
##########################

* A common approach for most cases is to:

  1. Perform post-training quantization first, as it is the easiest option.
  2. If the accuracy drop is unacceptable, use quantization-aware training instead. It will give
     you the same level of performance boost, with a smaller impact on accuracy.

* **Weight compression** works with **LLMs**, **VLMs** and other Transformer-based models.



.. image:: ../assets/images/DEVELOPMENT_FLOW_V3_crunch.svg



Installation and usage
###########################

To learn about the full scope of the framework, its installation, and technical details, visit
both `the NNCF repository <https://github.com/openvinotoolkit/nncf?tab=readme-ov-file>`__ and
`NNCF API documentation <https://openvinotoolkit.github.io/nncf/autoapi/nncf/>`__.



.. tab-set::

   .. tab-item:: Installation
      :sync: install

      .. tab-set::

         .. tab-item:: PyPI
            :sync: pip

            .. code-block::

               pip install nncf

         .. tab-item:: Conda
            :sync: conda

            .. code-block::

               conda install -c conda-forge nncf

      For more installation details, see the page on
      `NNCF Installation <https://github.com/openvinotoolkit/nncf/blob/develop/docs/Installation.md>`__.

   .. tab-item:: System Requirements
      :sync: sys-req

      Full requirement listing is available in the
      `NNCF GitHub Repository <https://github.com/openvinotoolkit/nncf?tab=readme-ov-file#system-requirements>`__

      Note that to optimize a model, you will need to install this model's framework as well.
      Install NNCF in the same Python environment as the framework. For a list of recommended
      framework versions, see the
      `framework compatibility table <https://github.com/openvinotoolkit/nncf/blob/develop/docs/Installation.md#corresponding-versions>`__.


.. note::

   Once optimized, models may be executed with the typical OpenVINO inference workflow,
   no additional changes to the inference code are required.

   This is true for models optimized using NNCF, as well as those pre-optimized in their source
   frameworks, such as PyTorch, and ONNX (in Q/DQ; Quantize/DeQuantize format).
   The latter may be easily converted to the
   :doc:`OpenVINO Intermediate Representation format (IR) <../../documentation/openvino-ir-format>`
   right away.


`Hugging Face Optimum Intel <https://huggingface.co/docs/optimum/intel/optimization_ov>`__
offers OpenVINO integration with Hugging Face models and pipelines. NNCF serves as the compression
backend within the Hugging Face Optimum Intel, integrating with the widely used transformers
library to enhance model performance.


Additional Resources
#######################

* `NNCF GitHub repository <https://github.com/openvinotoolkit/nncf>`__
* `NNCF Architecture <https://github.com/openvinotoolkit/nncf/blob/develop/docs/NNCFArchitecture.md>`__
* `NNCF Tutorials <https://github.com/openvinotoolkit/nncf?tab=readme-ov-file#demos-tutorials-and-samples>`__
* `NNCF Compressed Model Zoo <https://github.com/openvinotoolkit/nncf/blob/develop/docs/ModelZoo.md>`__
* :doc:`Deployment optimization <running-inference/optimize-inference>`
