Release v2024.1.2 updates

_sources/architecture_2.0/api_ref/class_GVA_RegionOfInterest.rst

@@ -23,6 +23,7 @@ This class represents region of interest - object describing detection result (b
 		// construction
 
 		:ref:`RegionOfInterest<doxid-class_g_v_a_1_1_region_of_interest_1a68e47be95f3d19b9537fa1b6db230ab9>`(GstVideoRegionOfInterestMeta* meta);
+		:target:`RegionOfInterest<doxid-class_g_v_a_1_1_region_of_interest_1a6f186a48bab9836c7f4c287b56e8cbf1>`(GstAnalyticsODMtd meta);
 
 		// methods
 

_sources/architecture_2.0/samples_2.0.rst

@@ -47,8 +47,6 @@ Architecture 2.0 brings new samples separated into 3 categories by programming m
    * - GStreamer low-level
      - samples/gstreamer/gst_launch/instance_segmentation/
      - `instance_segmentation.sh <https://github.com/dlstreamer/dlstreamer/blob/master/samples/gstreamer/gst_launch/instance_segmentation/instance_segmentation.sh>`_
-       `roi_background_removal.sh <https://github.com/dlstreamer/dlstreamer/blob/master/samples/gstreamer/gst_launch/instance_segmentation/roi_background_removal.sh>`_
-       `classification_with_background_removal.sh <https://github.com/dlstreamer/dlstreamer/blob/master/samples/gstreamer/gst_launch/instance_segmentation/classification_with_background_removal.sh>`_
      - No
      - Yes
      - Yes

_sources/dev_guide/converting_deepstream_to_dlstreamer.rst

@@ -13,25 +13,81 @@ each step to help show the modifications being described.
 Preparing Your Model
 --------------------
 
-To use Intel® DL Streamer Pipeline Framework and OpenVINO™ Toolkit the
-model needs to be in Intermediate Representation (IR) format. To convert
-your model to this format, use the `Model Optimizer <https://docs.openvino.ai/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html>`__
-tool from OpenVINO™ Toolkit.
+To use Intel® DL Streamer Pipeline Framework and OpenVINO™ Toolkit the model needs to be in
+Intermediate Representation (IR) format. To convert your model to this format, please follow
+:doc:`model preparation <model_preparation>` steps.
+
+Configuring Model for Intel® DL Streamer
+------------------------------------------------------------
+
+NVIDIA DeepStream uses a combination of model configuration files and DeepStream element properties
+to specify interference actions as well as pre- and post-processing steps before/after running inference
+as documented here: `here <https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html>`__.
+
+Similarly, Intel® DL Streamer Pipeline Framework uses GStreamer element properties for inference
+settings and :doc:`model proc <model_proc_files>` files for pre- and post-processing steps. 
+
+The following table shows how to map commonly used NVIDIA DeepStream configuration properties
+to Intel® DL Streamer settings.
+
+.. list-table::
+   :header-rows: 1
+
+   * - NVIDIA DeepStream config file
+     - NVIDIA DeepStream element property
+     - Intel® DL Streamer model proc file
+     - Intel® DL Streamer element property
+     - Description
+   * - model-engine-file <path>
+     - model-engine-file <path>
+     - 
+     - model <path>
+     - Path to inference model network file.
+   * - labelfile-path <path>
+     - 
+     - 
+     - labels-file <path>
+     - Path to .txt file containing object classes.
+   * - network-type <0..3>
+     - 
+     - 
+     - | gvadetect for detection, instance segmentation
+       | gvaclassify for classification, semantic segmentation
+     - Type of inference operation.
+   * - batch-size <N>
+     - batch-size <N>
+     - 
+     - batch-size <N>
+     - Number of frames batched together for a single inference.
+   * - maintain-aspect-ratio
+     - 
+     - resize: aspect-ratio
+     - 
+     - Number of frames batched together for a single inference.
+   * - num-detected-classes
+     - 
+     - 
+     - 
+     - Number of classes detected by the model, inferred from label file by Intel® DL Streamer.
+   * - interval <N>
+     - interval <N>
+     - 
+     - inference-interval <N+1>
+     - Inference action executed every Nth frame, please note Intel® DL Streamer value is greater by 1.
+   * - 
+     - threshold
+     - 
+     - threshold
+     - Threshold for detection results.
 
-Pipeline Framework's inference plugins optionally can do some pre- and
-post-processing operations before/after running inference. These
-operations are specified in a model-proc file. Visit :doc:`this page <model_preparation>`
-for more information on creating a model-proc file and examples with
-various models from `Open Model Zoo <https://github.com/openvinotoolkit/open_model_zoo>`__.
 
 GStreamer Pipeline Adjustments
 ------------------------------
 
-In the following sections we will be converting the below pipeline that
-is using DeepStream elements to Pipeline Framework. It is taken from one of the
-examples
-`here <https://github.com/NVIDIA-AI-IOT/redaction_with_deepstream>`__.
-It takes an input stream from file, decodes, runs inference, overlays
+In the following sections we will be converting DeepStream pipeline to Pipeline Framework.
+The DeepStream pipeline is taken from one of the examples
+`here <https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps>`__.
+It reads a video stream from the input file, decodes it, runs inference, overlays
 the inferences on the video, re-encodes and outputs a new .mp4 file.
 
 .. code:: shell
@@ -49,12 +105,11 @@ Mux and Demux Elements
 -  Remove ``nvstreammux`` and ``nvstreamdemux`` and all their
    properties.
 
-   -  These elements are used in the case of multiple input streams to
-      connect all inputs to the same inferencing element. In DL
-      Streamer, the inferencing elements share properties and instances
-      if they share the same ``model-instance-id`` property.
-   -  In our example, we only have one source so we will skip this for
-      now. See more on how to do this with Pipeline Framework in the section
+   -  These elements combine multiple input streams into a single batched video stream (NVIDIA-specific).
+      Intel® DL Streamer takes a different approach: it employs generic GStreamer syntax to define parallel streams.
+      The cross-stream batching happens at the inferencing elements by setting the same ``model-instance-id`` property.
+   -  In this example, there is only one video stream so we can skip this for now.
+      See more on how to construct multi-stream pipelines in the following section
       :ref:`Multiple Input Streams <Multiple-Input-Streams>` below.
 
 At this stage we have removed ``nvstreammux`` and the ``queue`` that
@@ -104,29 +159,23 @@ In this example we will use gvadetect to infer on the full frame and
 output region of interests. ``batch-size`` was also added for
 consistency with what was removed above (the default value is 1 so it is
 not needed). We replaced ``config-file-path`` property with ``model``
-and ``model-proc`` properties. See the section above about “Preparing
-your model” for converting the model to IR format and creating a
-model-proc file.
-
-.. note:: 
-   The ``model-proc`` file is not always needed depending on the model's inputs and outputs.
+and ``model-proc`` properties as described in “Configuring Model for Intel® DL Streamer” above.
 
 .. code:: shell
+   :emphasize-lines: 2
 
    filesrc location=input_file.mp4 ! decodebin ! \
-   gvadetect model=./model.xml model-proc=./model_proc.json batch-size=1 ! \
+   gvadetect model=./model.xml model-proc=./model_proc.json batch-size=1 ! queue ! \
    nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! \
    nvdsosd ! queue ! \
    nvvideoconvert ! "video/x-raw, format=I420" ! videoconvert ! avenc_mpeg4 bitrate=8000000 ! qtmux ! filesink location=output_file.mp4
 
 Video Processing Elements
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
--  Replace video processing elements with VA equivalents for GPU or
-   native GStreamer elements for CPU.
+-  Replace NVIDIA-specific video processing elements with native GStreamer elements.
 
-   -  ``nvvideoconvert`` with ``vapostproc`` or ``mfxvpp`` (GPU) or
-      ``videoconvert`` (CPU).
+   -  ``nvvideoconvert`` with ``vapostproc`` (GPU) or ``videoconvert`` (CPU).
 
       -  If the ``nvvideoconvert`` is being used to convert to/from
          ``memory:NVMM`` it can just be removed.
@@ -151,9 +200,10 @@ on Intel Graphics. Also, we will use the GStreamer standard element
 the input stream as well as what is available on the system.
 
 .. code:: shell
+   :emphasize-lines: 4
 
    filesrc location=input_file.mp4 ! decodebin ! \
-   gvadetect model=./model.xml model-proc=./model_proc.json batch-size=1 ! \
+   gvadetect model=./model.xml model-proc=./model_proc.json batch-size=1 ! queue ! \
    nvdsosd ! queue ! \
    videoconvert ! avenc_mpeg4 bitrate=8000000 ! qtmux ! filesink location=output_file.mp4
 
@@ -195,9 +245,10 @@ The only metadata processing that is done in this pipeline is to overlay
 the inferences on the video for which we use ``gvawatermark``.
 
 .. code:: shell
+   :emphasize-lines: 3
 
    filesrc location=input_file.mp4 ! decodebin ! \
-   gvadetect model=./model.xml model-proc=./model_proc.json batch-size=1 ! \
+   gvadetect model=./model.xml model-proc=./model_proc.json batch-size=1 ! queue ! \
    gvawatermark ! queue ! \
    videoconvert ! avenc_mpeg4 bitrate=8000000 ! qtmux ! filesink location=output_file.mp4
 
@@ -206,21 +257,29 @@ the inferences on the video for which we use ``gvawatermark``.
 Multiple Input Streams
 ----------------------
 
-| Unlike DeepStream, where all sources need to be linked to the sink
-  pads of the ``nvstreammux`` element, Pipeline Framework shares all model and
-  Inference Engine properties between elements that have the same
-  ``model-instance-id`` property. Meaning that you do not need to mux
-  all sources together before inference and you can remove any
-  instances of ``nvstreammux`` and ``nvstreamdemux``. Below is a pseudo
-  example of a pipeline with two streams.
-| For DeepStream, the pipeline would look like this:
+| Unlike DeepStream, where all sources need to be linked to the sink pads of the ``nvstreammux`` element, 
+  Pipeline Framework uses existing GStreamer mechanisms to define multiple parallel video processing streams.
+  This approach allow to reuse native GStreamer elements within the pipeline. 
+  The input stream can share same Inference Engine if they have same ``model-instance-id`` property.
+  This allows creating inference batching across streams.
+
+| For DeepStream, the simple pipeline involving two streams would look like code snippet below. 
+  The first line defines a common inference element for two (muxed and batched) streams. 
+  The second line defines per-stream input operations prior to muxing. 
+  The third line defines per-stream output operations after de-muxing.
 
 .. code:: shell
 
-   nvstreammux ! nvinfer config-file-path=./config.txt ! nvstreamdemux filesrc ! decode ! mux.sink_0 filesrc ! decode ! mux.sink_1 demux.src_0 ! encode ! filesink demux.src_1 ! encode ! filesink
+   nvstreammux ! nvinfer batch-size=2 config-file-path=./config.txt ! nvstreamdemux \ 
+   filesrc ! decode ! mux.sink_0 filesrc ! decode ! mux.sink_1 \
+   demux.src_0 ! encode ! filesink demux.src_1 ! encode ! filesink
+
+When using Pipeline Framework, the command line defines operations for two parallel streams using native GStreamer syntax.
+By setting ``model-instance-id`` to the same value, both streams share the same instance ``gvadetect`` element.
+Hence, the shared inference parameters (model, batch size, ...) can be defined only in the first line.
 
-When using Pipeline Framework, the pipeline will look like this:
 
 .. code:: shell
 
-   filesrc ! decode ! gvadetect model=./model.xml model-proc=./model_proc.json model-instance-id=model1 ! encode ! filesink filesrc ! decode ! gvadetect model-instance-id=model1 ! encode ! filesink
+   filesrc ! decode ! gvadetect model-instance-id=model1 model=./model.xml batch-size=2 ! encode ! filesink \ 
+   filesrc ! decode ! gvadetect model-instance-id=model1 ! encode ! filesink

_sources/dev_guide/deepsort_implementation.rst

@@ -26,7 +26,7 @@ This algorithm like other tracking algorithms uses the common structure:
     od -> fe -> id
   }
 
-Various tracking algorithms implement each part differenty,
+Various tracking algorithms implement each part differently,
 but generally the pattern remains unchanged.
 
 Object detection

_sources/dev_guide/model_info_xml.rst

@@ -85,7 +85,7 @@ Intel® Deep Learning Streamer supports the following fields in the model info s
      - float
      - 255.0
      - Divide input image values by 'scale' before mapping to model input tensor
-       (typicall used when model was trained with input data normalized in <0,1> range).
+       (typically used when model was trained with input data normalized in <0,1> range).
      - range: [0.0, 1.0]
 
 Please also refer to `Deep Learning Workbench <https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Introduction.html>`__

_sources/dev_guide/model_proc_file.rst

@@ -115,7 +115,18 @@ should be described in a key-value format. Valid keys for
      - | no,
        | no-aspect-ratio,
        | aspect-ratio
-     - Resize an image.
+     - Resize an image to match input model layer dimensions.
+     - | opencv,
+       | va,
+       | va-surface-sharing
+   * - params
+     - crop
+     - | central,
+       | top_left,
+       | top_right,
+       | bottom_left,
+       | bottom_right
+     - Crop image to fit input model layer dimensions.
      - | opencv,
        | va,
        | va-surface-sharing