Nvidia DeepStream-6.0 : Jetson Setup

Nvidia DeepStream

What is Video Analytics?

Video analytics is a process wherein a real-time video feed is captured, processed, analyzed, and visualized. In a nutshell, the task is to convert the raw data into intelligent data and generate alerts and analytics.

Flow:

· Capture the data (camera, sensors, storage)

· Decode the feed

· Pre-processing the data

· Create inference using machine learning and deep learning algorithms.

· Encode the feed

· Make decisions (analytics, alerts and responses)

Where is the problem???

The task of analyzing videos and generating meaningful insights is simple when you have, say one or two video sources, but it becomes a very tedious task in terms of scaling when you have over 9 to 12 sources running in parallel on a single machine. As the number of sources increases and the AI models becomes more and more complex the whole task of video analytics becomes resource hungry and difficult to manage.

In a general pipeline, most of the tasks run on CPU and very few on GPUs, in such a case if a single block in the pipeline creates a delay the whole process is delayed and affects the overall performance of an application.

So, what to do???????

Solution: DeepStream

DeepStream SDK by NVIDIA is a toolkit that provides you with an end-to-end solution from capturing the data to deploying the service/application on the cloud. It’s a low-code platform that provides you with the necessary tools and plugins for building AI-powered applications. DeepStream comes with accelerated plugins that ensure all the heavy operations and computations take place on dedicated accelerators such as GPUs, DLAs (Deep Learning Accelerator) and VIC. With GPU-optimized plugins, every segment of the pipeline is able to leverage the power of GPU and increase the efficiency (except capturing as it generally happens over the network in most cases).

DeepStream supports development in both C/C++ and python. It accepts all sorts of feed be it usb/csi camera, video file, RTSP, and uses optimized AI models to generate insights.

How to get STARTED

1. Install the necessary packages

sudo apt install \
libssl1.0.0 \
libgstreamer1.0-0 \
gstreamer1.0-tools \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
libgstrtspserver-1.0-0 \
libjansson4=2.11-1

2. Clone librdkafka from Github

git clone https://github.com/edenhill/librdkafka.git

3. Configure and build Kafka

cd librdkafka
git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a
./configure
make
sudo make install
cd ..

4. Copy the generated libraries to deepstream directory

sudo mkdir -p /opt/nvidia/deepstream/deepstream-6.0/lib
sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-6.0/lib
cd ..

5. Install DeepStream SDK

Download the DeepStream tar package from here.

Download deepstream_sdk_v6.0.0_jetson.tbz2 on your Jetson Module.

Following command will extract and install DeepStream SDK on your Jetson Module:

sudo tar -xvf deepstream_sdk_v6.0.0_jetson.tbz2 -C /
cd /opt/nvidia/deepstream/deepstream-6.0
sudo ./install.sh
sudo ldconfig

Almost there …..

After DeepStream SDK is installed run the following to boost the clocks:

sudo nvpmodel -m 0
sudo jetson_clocks

Installing python binding

Through python bindings, you can develop AI applications in DeepStream using python as the primary language. DeepStream ships with several reference applications that you can use to develop your custom application.

7. Clone the python apps

git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps

8. Copy these files to DeepStream directory

sudo cp -r deepstream_python_apps /opt/nvidia/deepstream/apps

Now you are ready to ROCK…..

To understand each python application in detail visit the following link:

Python Sample Apps and Bindings Source Details — DeepStream 6.0 Release documentation

Few examples for reference

GitHub — HitPant/nv_deepstream_apps