Main Content

MultiPixel-MultiComponent Video Streaming

This example uses:

Open Model

This example shows how to work with a multipixel-multicomponent pixel stream. Multipixel-multicomponent streaming enables real-time processing of high-resolution or high-frame-rate color video streams.

To demonstrate working with such a video stream, this example implements the well-known bloom effect image post-processing technique. The bloom effect introduces or enhances the glow of light sources in an image.

Top Level I/0

Each pixel of a high-resolution or high-frame-rate pixel stream is modeled as a NumPixel-by-NumComponent matrix. Matrix data types are supported for HDL code generation within a design, but not for the ports of the top-level subsystem. In this case, the input pixel stream is split into three 4-by-1 vectors at the input of the DUT, and then recombined at the output into a 4-by-3 matrix for the Pixels To Frame block.

Bloom Effect

The example model follows these three steps to add a bloom effect to the input image.

  1. The BrightSpotFilter subsystems find bright spots in the intensity image by checking pixel values against a threshold. These pixel values have been converted from RGB to intensity by the Color Space Converter.

  2. The 15x15 Blur subsystem spreads out the bright spots by applying a Gaussian filter.

  3. The BlendBloom subsystem adds the Gaussian-enhanced bright spots back to the original image.

Matrix Operations

Vision HDL Toolbox™ neighborhood-processing blocks can operate on vector inputs, but do not support matrix inputs. The line buffer used inside the blocks returns a NumPixels-by-KernelHeight matrix. Using multicomponent inputs would result in a NumPixels-by-KernelHeight-by-NumComponents output matrix, however, 3-D matrices are not supported for HDL code generation. To work around this the model uses For Each subsystems which support HDL code generation with scalar, vector, and matrix inputs.

The model sets the Partition Width and Partition Dimension of the BrightSpotFilter to 1. The subsystem applies a threshold in parallel to each 1x3 RGB multi-component pixel of its input multipixel-multicomponent matrix.

The 15x15 Blur subsystem has the Partition Width set to 1 and Partition Dimension set to 2. The subsystem applies a Gaussian filter in parallel to each 4x1 multipixel RGB component.

The BlendBloom subsystem has the Partition Width and Partition Dimension set to 1. The subsystem adds in parallel each 1x3 multicomponent pixel to its respective filtered 1x3 multicomponent pixel.

Simulation Results

Simulating the model displays these input and output images. The bloom effect makes the lighted areas of the scene look brighter and shows a halo effect.

Implementation Results

This table shows the synthesis results of HDL code generated from the DUT subsystem and synthesized for an AMD™ Zynq™ ZC706 board. Because none of the resources exceed 25% of their respective category, the design has a relatively small footprint.

T =

  4x2 table

    Resource     Usage
    _________    _____

    DSP48        84   
    Flip Flop    61739
    LUT          36966
    BRAM         132

See Also

|

Related Topics