Changing the Engine and Accelerator

An Engine and Accelerator must be specified for the core CV services. The engine is the software backend running the model, and the accelerator is the target hardware the engine is running on. Specify the engine and accelerator in the load() function. If only the engine is provided, the accelerator listed as default for that engine will be used. You can refer to the model catalog and the supported devices page to see if your model or device is supported by an engine and accelerator.

OpenCV’s DNN Engine

OpenCV’s DNN engine (DNN) will run on all supported devices and supports most models, so it’s a great starting point. The default accelerator is GPU, which attempts to run the model on the GPU and falls back to the CPU if not. If desired, you can manually select the CPU, which runs the model on the CPU.

Set the engine parameter of your core CV service as follows to use OpenCV’s DNN engine:

cv_service.load(engine=edgeiq.Engine.DNN)

or:

cv_service.load(
    engine=edgeiq.Engine.DNN, accelerator=edgeiq.Accelerator.CPU)

OpenCV’s CUDA Engine

OpenCV’s CUDA engine (DNN_CUDA) will perform inferencing on a CUDA-supported GPU. The default accelerator is NVIDIA, and for some models an additional performance boost comes with using the NVIDIA_FP16 accelerator.

Set the engine parameter of your core CV service as follows to use OpenCV’s DNN engine:

cv_service.load(engine=edgeiq.Engine.DNN_CUDA)

NVIDIA’s TensorRT Engine

NVIDIA’s TensorRT engine (TENSOR_RT) is a high-performance model optimizer and inference engine for CUDA-supported GPU’s.

Set the engine parameter of your core CV service as follows to use OpenCV’s DNN engine:

cv_service.load(engine=edgeiq.Engine.TENSOR_RT)

OpenCV’s OpenVINO Engine for NCS2 and Myriad Processors

OpenCV’s OpenVINO engine (DNN_OPENVINO) enables inferencing on the Intel Neural Compute Stick 2 as well as other Myriad-based devices. It supports models built with the OpenVINO model optimizer as well as some other frameworks.

Set the engine parameter of your core CV service as follows to use OpenCV’s DNN engine:

cv_service.load(engine=edgeiq.Engine.DNN_OPENVINO)