edgeIQ API

bounding_box module

class BoundingBox(start_x, start_y, end_x, end_y)

A rectangular bounding box.

Checking if two BoundingBox instances are equal checks the equivalence of their coordinates.

Multiplying a BoundingBox by a scalar scales the bounding box and position. This can be helpful when scaling an image up or down.

Parameters
  • start_x (integer) – The starting x value.

  • start_y (integer) – The starting y value.

  • end_x (integer) – The ending x value.

  • end_y (integer) – The ending y value.

property start_x

The starting x value.

Type

integer

property start_y

The starting y value.

Type

integer

property end_x

The ending x value.

Type

integer

property end_y

The ending y value.

Type

integer

property width

The width of the bounding box.

Type

integer

property height

The height of the bounding box.

Type

integer

property area

The area of the bounding box.

Type

integer

property center

The center of the bounding box.

Type

float

compute_distance(other_box)

Compute the distance between the center of this and another bounding box.

Parameters

other_box (BoundingBox) – The second bounding box for the distance computation.

Returns

float – The Euclidean distance between bounding box centers.

get_intersection(other_box)

Get the intersection of this box and other_box represented as another bounding box.

Parameters

other_box (BoundingBox) – The overlapping box.

Returns

BoundingBox – The bounding box representing the intersection of this box and other_box.

compute_overlap(other_box)

Compute the fraction of this box which is overlapped by other_box.

Parameters

other_box (BoundingBox) – The overlapping box.

Returns

float – A value [0.0, 1.0] indicating the fraction of this box that is overlapped by other_box.

camera_blockage module

class SimpleCameraBlockageResults(blocked_status, duration, std_dev, blocked_ratio)

The results of detect_simple_camera_blockage().

property blocked_status

True if the camera is blocked, False otherwise.

Type

boolean

property duration

The duration of the detection in seconds.

Type

float

property std_dev

The standard deviation for each pixel of the image.

Type

numpy array of floats the same dimension of the input image

property blocked_ratio

The ratio of blocked portion to overall image size.

Type

list of floats if detect_quadrants=True; else float

detect_simple_camera_blockage(image, blocked_threshold=0.9, std_dev_threshold=2.0, detect_quadrants=True)

Detect a camera blockage using a simple standard deviation-based algorithm.

A blockage is detected when the standard deviation falls below a specified level for a portion of the image or quadrant. It is great for detecting simple cases where the blockage has low standard deviation, for example a solid color object. It fails when the blockage also has a high standard deviation, such as a colorful notebook.

Parameters
  • image (numpy array of image in BGR format) – The image to analyze.

  • blocked_threshold (float in range [0, 1]) – The portion of the image that must be under the std_dev_threshold to determine that the camera is blocked. For example, a value of 0.9 means that the camera is considered blocked when 90% of its pixels fall below the std_dev_threshold.

  • std_dev_threshold (float) – The level of the standard deviation to consider for the blockage threshold. For example, a value of 2.0 means that all pixels under a standard deviation of 2.0 are considered blocked.

  • detect_quadrants (boolean) – When True, the blocked_threshold will be applied to each quadrant of an image. If any quadrant is blocked the camera is considered blocked. This is useful to detect when a portion of the field of view is blocked. When False, the blocked_threshold will be applied to the entire image.

Returns

SimpleCameraBlockageResults – The results of the blockage detection.

constants module

class Engine

Bases: enum.Enum

Software inference engines supported by edgeIQ

DNN = 'DNN'

OpenCV’s DNN backend.

Supports:

DNN_OPENVINO = 'DNN_OPENVINO'

OpenCV’s OpenVINO Inference Engine backend.

Supports:

DNN_CUDA = 'DNN_CUDA'

OpenCV’s CUDA Inference Engine backend.

Supports:

class Accelerator

Bases: enum.Enum

HW accelerators supported by edgeIQ

DEFAULT = 'DEFAULT'

Selects the default accelerator for the given Engine

CPU = 'CPU'

Run the Engine on the CPU

GPU = 'GPU'

Run the Engine on the GPU

MYRIAD = 'MYRIAD'

Run the Engine on a MYRIAD accelerator (NCS1, NCS2)

NVIDIA = 'NVIDIA'

Run the Engine on a NVIDIA gpu accelerator

edge_tools module

These are a set of utility services that can be used for things like capturing segmented video clips, FPS speeds, interfacing with your webcam, saving event-based triggered videos, and image manipulations.

class WebcamVideoStream(cam=0)

Capture video frames from a webcam or CSI camera attached to your device.

WebcamVideoStream can be instantiated as a context manager:

with edgeiq.WebcamVideoStream() as video_stream:
    ...

To use WebcamVideoStream directly, use the start() and stop() functions:

video_stream = edgeiq.WebcamVideoStream().start()
...
video_stream.stop()

Typical usage:

with edgeiq.WebcamVideoStream() as video_stream:
    while True:
        frame = video_stream.read()
Parameters

cam (integer) – The integer identifier of the camera.

read()

Return the most recent frame from the webcam.

Returns

numpy array – The frame that was read from the camera

start()

Start reading frames from the video stream.

Returns

self

stop()

Stop and clean up the webcam connection.

class FrameRotation

Amount of rotation applied to each frame in degrees

ROTATE_NONE = 0
ROTATE_90 = 90
ROTATE_180 = 180
class GStreamerVideoStream(cam=0, rotation=<FrameRotation.ROTATE_NONE: 0>)

Capture video frames using the gstreamer plugin.

This can be useful for capturing videos from CSI cameras when V4L is not supported.

GStreamerVideoStream can be instantiated as a context manager:

with edgeiq.GStreamerVideoStream() as video_stream:
    ...

To use GStreamerVideoStream directly, use the start() and stop() functions:

video_stream = edgeiq.GStreamerVideoStream().start()
...
video_stream.stop()

Typical usage:

with edgeiq.GStreamerVideoStream() as video_stream:
    while True:
        frame = video_stream.read()
Parameters
  • cam (integer) – The integer identifier of the camera.

  • rotation (FrameRotation) – The rotation applied to each frame

read()

Return the most recent frame from the webcam.

Returns

numpy array – The frame that was read from the camera

start()

Start reading frames from the video stream.

Returns

self

stop()

Stop and clean up the webcam connection.

class IPVideoStream(url)

Capture video frames from an IP stream.

IPVideoStream can be instantiated as a context manager:

with edgeiq.IPVideoStream('tcp://0.0.0.0:3333') as video_stream:
    ...

To use IPVideoStream directly, use the start() and stop() functions:

video_stream = edgeiq.IPVideoStream('tcp://0.0.0.0:3333').start()
...
video_stream.stop()

Typical usage:

with edgeiq.IPVideoStream('tcp://0.0.0.0:3333') as video_stream:
    while True:
        frame = video_stream.read()
Parameters

url (string) – The URL of the IP stream

read()

Return the most recent frame from the webcam.

Returns

numpy array – The frame that was read from the camera

start()

Start reading frames from the video stream.

Returns

self

stop()

Stop and clean up the webcam connection.

class FPS

Monitor the frames per second (FPS) of video streams for performance tracking.

Typical usage:

fps = edgeiq.FPS().start()
while True:
    <main processing loop>
    fps.update()

# Get the elapsed time and FPS
fps.stop()
print("Elapsed seconds: {}".format(fps.get_elapsed_seconds()))
print("FPS: {}".format(fps.compute_fps()))

compute_fps() may also be called in the main processing loop to compute an instantaneous estimate of the FPS.

start()

Start tracking FPS.

stop()

Stop tracking FPS.

update()

Increment the total number of frames examined during the start and end intervals.

Raises

RuntimeError

get_elapsed_seconds()

Return the total number of seconds between the start and end intervals.

Returns

float – The elapsed time in seconds between start and end, or since start if stop() has not been called.

compute_fps()

Compute the (approximate) frames per second.

Returns

float – the approximate frames per second.

class FileVideoStream(path, queue_size=128, play_realtime=False)

Stream a video file for analysis.

FileVideoStream can be instantiated as a context manager:

with edgeiq.FileVideoStream('/path/to/video.mp4') as video_stream:
    ...

To use FileVideoStream directly, use the start() and stop() functions:

video_stream = edgeiq.FileVideoStream('/path/to/video.mp4').start()
...
video_stream.stop()

Typical usage:

with edgeiq.FileVideoStream('/path/to/video.mp4') as video_stream:
    while video_stream.more():
        frame = video_stream.read()
Parameters
  • path (string) – The path of the video file location.

  • queue_size (integer) – The size of the buffer queue.

  • play_realtime (boolean) – If ‘True’ play video in simulated realtime, where frames are dropped as they become outdated.

start()

Start a thread to read frames from the file video stream.

Returns

self

read()

Read the next frame in the queue.

Returns

numpy array – The next frame

more()

Check if there are more frames in the queue.

Returns

True if there are frames in the queue.

stop()

Stop the FileVideoStream object.

class VideoWriter(output_path='output.avi', fps=30, color=True, codec='MJPG')

Save a video clip to a file.

VideoWriter can be instantiated as a context manager:

with edgeiq.VideoWriter('/path/to/output/video.avi') as video_writer:
    ...

To use VideoWriter directly, make sure to call close() when usage is complete:

video_writer = edgeiq.VideoWriter('/path/to/output/video.avi')
...
video_writer.close()

Typical usage:

with edgeiq.VideoWriter('/path/to/output/video.avi') as video_writer:
    while True:
        <read and process frame>
        video_writer.write_frame(frame)
Parameters
  • output_path (string) – The path to save the video file at.

  • fps (integer) – The number of frames per second to save.

  • color (boolean) – If True, save the video in color.

  • codec (string) – The video codec to use for saving the video clip.

write_frame(frame)

Save the frame to the video clip.

Parameters

frame (numpy array of frame) – The video frame to save.

close()

Finish creating the video clip.

class EventVideoWriterState

States of the EventVideoWriter

IDLE = 'Idle'
RECORDING = 'Recording'
POST_ROLL = 'Post-roll'
class EventVideoWriter(pre_roll=150, post_roll=150, fps=30, color=True, codec='MJPG')

Save a segment of a video stream based upon an event trigger.

EventVideoWriter can be instantiated as a context manager:

with edgeiq.EventVideoWriter() as video_writer:
    ...

To use EventVideoWriter directly, make sure to call close() when usage is complete:

video_writer = edgeiq.EventVideoWriter()
...
video_writer.close()

Typical usage:

def video_writer_callback(event_name):
    print('Event {} has completed'.format(event_name))


with edgeiq.EventVideoWriter() as video_writer:
    while True:
        <read and process frame>
        ...
        video_writer.update(frame)

        if event_started:
            video_writer.start_event(
                '/path/to/output/video.avi', video_writer_callback, 'Event 1')

        if event_ended:
            video_writer.finish_event()
Parameters
  • pre_roll (integer) – The number of frames to capture preceding the event trigger.

  • post_roll (integer) – The number of frames to capture after the event completes.

  • fps (integer) – The number of frames per second to save.

  • color (boolean) – If True, save the video with color.

  • codec (string) – The video codec to use for saving the video clip.

property state

The current recording state.

Type

EventVideoWriterState

update(frame)

Provide a frame to be written to the video file if recording.

Parameters

frame (numpy array) – The frame to save.

start_event(output_path, callback_function=None, callback_args=())

Start a video clip recording.

Parameters
  • output_path (string) – The path to save the video.

  • callback_function (function) – A function to be called upon completing this video write. When set to None no callback is called. Actions in callback_function must be thread-safe.

  • callback_args (tuple) – Arguments to be passed to the callback function. Arguments must be thread-safe.

finish_event()

Finish a recording for an event. Note that recording continues for post_roll frames.

close()

Close the EventVideoWriter.

If a video recording is ongoing, it will be stopped and the recording will be saved.

translate(image, x, y)

Translate an image.

Parameters
  • image (numpy array of image) – The image to manipulate.

  • x (integer) – Translate image on X axis by this amount.

  • y (integer) – Translate image on Y axis by this amount.

Returns

numpy array – The translated image.

rotate(image, angle, center=None, scale=1.0)

Rotate an image by specified angle.

Parameters
  • image (numpy array of image) – The image to manipulate.

  • angle (integer) – The angle to rotate the image by (degrees).

  • center (tuple of two integers) – The center coordinates of the rotation.

  • scale (integer) – The scale factor.

Returns

numpy array – The rotated image.

resize(image, width=None, height=None, keep_scale=True, inter=3)

Resize an image to specified height and width.

When both a width and height are given and keep_scale is True, these are treated as the maximum width and height.

Parameters
  • image (numpy array of image) – The image to manipulate.

  • height (integer) – The new height of image.

  • width (integer) – The new width of image.

  • keep_scale (boolean) – Maintain the original scale of the image.

  • inter (integer) – The interpolation method (One of OpenCV InterpolationFlags).

Returns

numpy array – The resized image.

list_images(base_path, contains=None)

List all images in specified path.

Finds images with the following extensions:

  • .jpg

  • .jpeg

  • .png

  • .bmp

  • .tif

  • .tiff

Parameters
  • base_path (string) – The base path of folder where images are located.

  • contains (string) – Select only filenames that contain this string.

Returns

list of strings – The valid image file paths.

list_files(base_path, valid_exts=('.jpg', '.jpeg', '.png', '.bmp', '.tif', '.tiff'), contains=None)

List all files in specified path.

Parameters
  • base_path (string) – The base path of folder where files are located.

  • valid_exts (list of strings) – The list of valid extentions to filter for.

  • contains (string) – Select only filenames that contain this string.

Returns

list of strings – The valid file paths.

preprocess_image(input_image, dim)

Preprocess an image for analysis. Preprocessing performs resizing, adjusting the mean, scaling, and setting the type to be a 32 bit float.

Parameters
  • input_image (numpy array of image) – The image to process.

  • dim (list of two integers) – The dimensions of the output image.

Returns

numpy array – The preprocessed image.

color_converter(image, flag='RGB')

Convert the color of an image for analysis.

Parameters
  • image (numpy array of image) – The image to process.

  • flag (string) – The output image format: “RGB” or “BGR”.

Returns

numpy array – The converted image.

markup_image(image, predictions, show_labels=True, show_confidences=True, colors=None, line_thickness=2, font_size=0.5, font_thickness=2)

Draw boxes, labels, and confidences on the specified image.

Parameters
  • image (numpy array of image in BGR format) – The image to draw on.

  • predictions (list of ObjectDetectionPrediction) – The list of prediction results.

  • show_labels (boolean) – Indicates whether to show the label of the prediction.

  • show_confidences (boolean) – Indicates whether to show the confidence of the prediction.

  • colors (list of tuples in the format (B, G, R)) – A custom color list to use for the bounding boxes. The index of the color will be matched with a label index.

  • line_thickness (float) – The thickness of the lines that make up the bounding box.

  • font_size (float) – The scale factor for the text.

  • font_thickness (float) – The thickness of the lines used to draw the text.

Returns

numpy array – The marked-up image.

transparent_overlay_boxes(image, predictions, alpha=0.5, colors=None, show_labels=False, show_confidences=False)

Overlay an image with a full mask or just area(s) of interest. This utility is designed to work with either object detection results to display overlayed bounding boxes or display semantic segmentation results mask overlayed on the original image.

Parameters
  • image (numpy array of image in BGR format) – The image to manipulate.

  • predictions (list of ObjectDetectionPrediction) – The list of prediction results.

  • alpha (integer in range [0.0, 1.0]) – Transparency of the overlay. The closer alpha is to 1.0, the more opaque the overlay will be. Similarly, the closer alpha is to 0.0, the more transparent the overlay will appear.

  • colors (list of tuples in the format (B, G, R)) – A custom color list to use for the bounding boxes or object classes pixel map

  • show_labels (boolean) – Indicates whether to show the label of the prediction.

  • show_confidences (boolean) – Indicates whether to show the confidence of the prediction.

Returns

numpy array – The overlayed image.

filter_predictions_by_label(predictions, label_list)

Filter a prediction list by label.

Typical usage:

people_and_apples = edgeiq.filter_predictions_by_label(predictions, ['person', 'apple'])
Parameters
  • predictions (list of ObjectDetectionPrediction) – A list of predictions to filter.

  • label_list (list of strings) – The list of labels to keep in the filtered output.

Returns

list of ObjectDetectionPrediction – The filtered predictions.

filter_predictions_by_area(predictions, min_area_thresh)

Filter a prediction list by bounding box area.

Typical usage:

larger_boxes = edgeiq.filter_predictions_by_area(predictions, 450)
Parameters
  • predictions (list of ObjectDetectionPrediction) – A list of predictions to filter.

  • min_area_thresh (integer) – The minimum bounding box area to keep in the filtered output.

Returns

list of ObjectDetectionPrediction – The filtered predictions.

find_usb_device(id_vendor, id_product)

Check if a USB device is connected.

Parameters
  • id_vendor (integer) – The vendor ID.

  • id_product (integer) – The product ID.

Returns

True if device found, otherwise False

find_ncs1()

Check if an NCS1 is connected.

Note that once a connection to the NCS1 is opened, the device’s VID and PID will change and this function will no longer find it.

Returns

True if NCS1 found, otherwise False

find_ncs2()

Check if an NCS2 is connected.

Note that once a connection to the NCS2 is opened, the device’s VID and PID will change and this function will no longer find it.

Returns

True if NCS2 found, otherwise False

cutout_image(image, box)

Cut out the portion of an image outlined by a bounding box.

Parameters
  • image (numpy array of image) – The image to cut out from.

  • box (BoundingBox) – The bounding box outlining the section of the image to cut out.

Returns

numpy array – The segment of the image outlined by the bounding box. Will be independent from the original image.

blur_objects(image, predictions)

Blur objects detected in an image.

Parameters
  • image (numpy array of image) – The image to draw on.

  • predictions (list of ObjectDetectionPrediction) – A list of predictions objects to blur.

Returns

numpy array – The image with objects blurred.

blend_images(background_image, foreground_image, alpha)

Blend a background image with a foreground image, background image and foreground image must have the same dimensions and same color format (RGB/BGR).

Parameters
  • background_image (numpy array of image) – The background image which gets overlayed

  • foreground_image (numpy array of image) – The background image which gets overlayed

  • alpha (float in range [0.0, 1.0]) – transparency of the overlay. The closer alpha is to 1.0, the more opaque the overlay will be. Similarly, the closer alpha is to 0.0, the more transparent the overlay will appear

Returns

numpy array – The blended image.

human_pose.human_pose module

class Pose(key_points, score)
property key_points

Key Points corresponding to body parts.

Nose

Neck

Right Shoulder

Right Elbow

Right Wrist

Left Shoulder

Left Elbow

Left Wrist

Right Hip

Right Knee

Right Ankle

Left Hip

Left Knee

Left Ankle

Right Eye

Left Eye

Right Ear

Left Ear

Returns

dict of coordinate tuples mapped to their body part names

property score

Confidence level associated with pose.

Type

float in range [0.0, 1.0]

class HumanPoseResult(net, image, pad)

Find poses within an image.

property image

The image results where processed on.

Type

numpy array – The image in BGR format

property poses

Poses found in image.

Type

list of Pose

property duration

The duration of the inference in seconds.

Type

float

draw_poses_background(color)

Draw poses found on image on a background color.

Parameters

color (tuple which contains values for B, G, R color channels.) – The color of the background in which the poses will be drawn on.

Returns

image: numpy array of image in BGR format

draw_poses(image=None)

Draws poses found on image.

Parameters

image (numpy.array) – An image to draw the poses found

Returns

image: numpy array of image in BGR format

property raw_results

The raw results returned from model.

Type

list of 2 numpy arrays shaped (1, 38, 32, 57) (1, 19, 32, 57) respectively, data type float32

property feature_maps

The feature map returned by the model.

Type

numpy arrays shape (18, 32, 57), data type float

property partial_affinity_fields

The partial affinity fields returned by the model.

Type

numpy arrays shape (38, 32, 57), data type float

draw_aliens()

image_classification module

class ClassificationPrediction(confidence, label)

A single prediction from Classification.

property label

The label describing this prediction result.

Type

string

property confidence

The confidence of this prediction.

Type

float

class ClassificationResults(confidences, labels, start, end)

All the results of classification from Classification.

Predictions are stored in sorted order, with descending order of confidence.

property duration

The duration of the inference in seconds.

Type

float

property predictions

The list of predictions.

Type

list of ClassificationPrediction

class Classification(model_id)

Identify the most prominent object in an image.

Typical usage:

classifier = edgeiq.Classification('alwaysai/googlenet')
classifier.load(engine=edgeiq.Engine.DNN)

<get image>
results = classifier.classify_image(image)
for prediction in results.predictions:
    print('Label: {}, confidence: {}'.format(
        prediction.label, prediction.confidence))
Parameters

model_id (string) – The ID of the model you want to use for image classification.

property accelerator

The accelerator being used.

Type

string

classify_image(image, confidence_level=0.3)

Identify the most prominent object in the specified image.

Parameters
  • image (numpy array of image) – The image to analyze.

  • confidence_level (float in range [0.0, 1.0]) – The minimum confidence level required to successfully accept a classification.

Returns

ClassificationResults

property colors

The auto-generated colors for the loaded model.

Note: Initialized to None when the model doesn’t have any labels. Note: To update, the new colors list must be same length as the label list.

Type

list of (B, G, R) tuples.

property engine

The engine being used.

Type

string

property labels

The labels for the loaded model.

Note: Initialized to None when the model doesn’t have any labels.

Type

list of strings.

load(engine=<Engine.DNN: 'DNN'>, accelerator=<Accelerator.DEFAULT: 'DEFAULT'>)

Initialize the inference engine and accelerator.

Parameters
  • engine (Engine) – The inference engine to use.

  • accelerator (Accelerator) – The hardware accelerator on which to run the inference engine.

property model_id

The ID of the loaded model.

Type

string

object_detection module

class ObjectDetectionPrediction(box, confidence, label, index)

A single prediction from ObjectDetection.

property label

The label describing this prediction result.

Type

string

property index

The index of this result in the master label list.

Type

integer

property box

The bounding box around the object.

Type

BoundingBox

property confidence

The confidence of this prediction.

Type

float

class ObjectDetectionResults(boxes, confidences, labels, indexes, start, end, image)

All the results of object detection from ObjectDetection.

Predictions are stored in sorted order, with descending order of confidence.

property duration

The duration of the inference in seconds.

Type

float

property predictions

The list of predictions.

Type

list of ObjectDetectionPrediction

class ObjectDetection(model_id)

Analyze and discover objects within an image.

Typical usage:

obj_detect = edgeiq.ObjectDetection(
        'alwaysai/ssd_mobilenet_v1_coco_2018_01_28')
obj_detect.load(engine=edgeiq.Engine.DNN)

<get image>
results = obj_detect.detect_objects(image, confidence_level=.5)
image = edgeiq.markup_image(
        image, results.predictions, colors=obj_detect.colors)

for prediction in results.predictions:
        text.append("{}: {:2.2f}%".format(
            prediction.label, prediction.confidence * 100))
Parameters

model_id (string) – The ID of the model you want to use for object detection.

property accelerator

The accelerator being used.

Type

string

property colors

The auto-generated colors for the loaded model.

Note: Initialized to None when the model doesn’t have any labels. Note: To update, the new colors list must be same length as the label list.

Type

list of (B, G, R) tuples.

property engine

The engine being used.

Type

string

property labels

The labels for the loaded model.

Note: Initialized to None when the model doesn’t have any labels.

Type

list of strings.

load(engine=<Engine.DNN: 'DNN'>, accelerator=<Accelerator.DEFAULT: 'DEFAULT'>)

Initialize the inference engine and accelerator.

Parameters
  • engine (Engine) – The inference engine to use.

  • accelerator (Accelerator) – The hardware accelerator on which to run the inference engine.

property model_id

The ID of the loaded model.

Type

string

detect_objects(image, confidence_level=0.3, overlap_threshold=0.3)

Detect objects within the specified image.

Parameters
  • image (numpy array of image) – The image to analyze.

  • confidence_level (float) – The minimum confidence level required to successfully accept an object identification.

  • overlap_threshold (float) – The maximum overlap for accepting a bounding box. This threshold is only applied for Darknet models.

Returns

ObjectDetectionResults

object_tracking module

class CorrelationTracker(max_objects=None)

Track objects based on a correlation tracking algorithm.

Typical usage:

obj_detect = edgeiq.ObjectDetection(
        'alwaysai/ssd_mobilenet_v1_coco_2018_01_28')
obj_detect.load(engine=edgeiq.Engine.DNN)

tracker = edgeiq.CorrelationTracker(max_objects=5)

while True:
    <get video frame>

    # Perform detection once every detect_period
    if frame_idx % detect_period:
        results = obj_detect.detect_objects(frame, confidence_level=.5)

        # Stop tracking old objects
        if tracker.count:
            tracker.stop_all()

        for prediction in results.prediction:
            tracker.start(frame, prediction)

    else:
        if tracker.count:
            predictions = tracker.update(frame)

    frame = edgeiq.markup_image(
                frame, predictions, colors=obj_detect.colors)
Parameters

max_objects (integer) – The maximum number of objects to track.

start(image, prediction)

Start tracking an object.

Parameters
  • image (numpy array of an image or video frame) – The image or frame to provide objects to be tracked.

  • prediction (ObjectDetectionPrediction) – The object to track.

property count

Get the number of objects being tracked.

Type

integer

update(image)

Update the tracker with a new image.

Parameters

image (numpy array of an image or video frame) – The image or frame to provide objects to be tracked

Returns

list of ObjectDetectionPrediction – The bounding boxes updated with new location and tracker confidence.

stop_all()

Stop tracking all objects currently being tracked.

class CentroidTracker(deregister_frames=50, max_distance=50)

Associate a bounding box with an object ID based on distances from previous detections.

Typical usage:

obj_detect = edgeiq.ObjectDetection(
        'alwaysai/res10_300x300_ssd_iter_140000')
obj_detect.load(engine=edgeiq.Engine.DNN)

centroid_tracker = edgeiq.CentroidTracker(
        deregister_frames=20, max_distance=50)

while True:
    <get video frame>
    results = obj_detect.detect_objects(frame, confidence_level=.5)

    objects = centroid_tracker.update(results.predictions)

    # Use the object dictionary to create a new prediction list
    for (object_id, prediction) in objects.items():
        new_label = 'face {}'.format(object_id)
        prediction.label = new_label
        text.append(new_label)
        predictions.append(prediction)

    frame = edgeiq.markup_image(frame, predictions)
Parameters
  • deregister_frames (integer) – The number of frames before deregistering an object that can no longer be found.

  • max_distance (integer) – The maximum distance between two centroids to associate an object.

update(predictions)

Update object centroids based on a new set of bounding boxes.

Parameters

predictions (list of ObjectDetectionPrediction) – The list of bounding boxes to track.

Returns

A dictionary that utilizes the object ID as the key and the ObjectDetectionPrediction as the value.

pose_estimation module

class PoseEstimation(model_id)

Find poses within an image.

Typical usage:

pose_estimator = edgeiq.PoseEstimation("alwaysai/human-pose")
pose_estimator.load(
        engine=edgeiq.Engine.DNN_OPENVINO,
        accelerator=edgeiq.Accelerator.MYRIAD)

<get image>
results = pose_estimator.estimate(image)

for ind, pose in enumerate(results.poses):
            print('Person {}'.format(ind))
            print('-'*10)
            print('Key Points:')
            for key_point in pose.key_points:
                print(str(key_point))
image = results.draw_poses(image)
Parameters

model_id (string) – The ID of the model you want to use for pose estimation.

estimate(image)

Estimate poses within the specified image.

Parameters

image (numpy array of image) – The image to analyze.

Returns

HumanPoseResult

property accelerator

The accelerator being used.

Type

string

property colors

The auto-generated colors for the loaded model.

Note: Initialized to None when the model doesn’t have any labels. Note: To update, the new colors list must be same length as the label list.

Type

list of (B, G, R) tuples.

property engine

The engine being used.

Type

string

property labels

The labels for the loaded model.

Note: Initialized to None when the model doesn’t have any labels.

Type

list of strings.

load(engine=<Engine.DNN: 'DNN'>, accelerator=<Accelerator.DEFAULT: 'DEFAULT'>)

Initialize the inference engine and accelerator.

Parameters
  • engine (Engine) – The inference engine to use.

  • accelerator (Accelerator) – The hardware accelerator on which to run the inference engine.

property model_id

The ID of the loaded model.

Type

string

semantic_segmentation module

class SemanticSegmentationResults(class_map, start, end)

The results of semantic segmentation from SemanticSegmentation.

property duration

The duration of the inference in seconds.

Type

float

property class_map

The class label with the highest probability for each and every (x, y)-coordinate in the image.

Type

numpy array

class SemanticSegmentation(model_id)

Classify every pixel in an image.

The build_legend() is useful when used with the Streamer.

Typical usage:

semantic_segmentation = edgeiq.SemanticSegmentation(
        'alwaysai/enet')
semantic_segmentation.load(engine=edgeiq.Engine.DNN)

with edgeiq.Streamer() as streamer:
    <get image>
    results = semantic_segmentation.segment_image(image)

    text = 'Inference time: {:1.3f} s'.format(results.duration)
    text.append('Legend:')
    text.append(semantic_segmentation.build_legend())

    mask = semantic_segmentation.build_image_mask(results.class_map)
    blended = edgeiq.blend_images(image, mask, alpha=0.5)

    streamer.send_data(blended, text)
Parameters

model_id (string) – The ID of the model you want to use for semantic segmentation.

segment_image(image)

Classify every pixel within the specified image.

Parameters

image (numpy array of image) – The image to analyze.

Returns

SemanticSegmentationResults

build_image_mask(class_map)

Create an image mask by mapping colors to the class map. Colors can be set by the colors attribute.

Parameters

class_map (numpy array) – The class label with the highest probability for each and every (x, y)-coordinate in the image

Returns

numpy array – Class color visualization for each pixel in the original image.

build_legend()

Create a class legend that associates color with a class object

Returns

string – An HTML table with class labels and colors that can be used with the streamer.

property accelerator

The accelerator being used.

Type

string

property colors

The auto-generated colors for the loaded model.

Note: Initialized to None when the model doesn’t have any labels. Note: To update, the new colors list must be same length as the label list.

Type

list of (B, G, R) tuples.

property engine

The engine being used.

Type

string

property labels

The labels for the loaded model.

Note: Initialized to None when the model doesn’t have any labels.

Type

list of strings.

load(engine=<Engine.DNN: 'DNN'>, accelerator=<Accelerator.DEFAULT: 'DEFAULT'>)

Initialize the inference engine and accelerator.

Parameters
  • engine (Engine) – The inference engine to use.

  • accelerator (Accelerator) – The hardware accelerator on which to run the inference engine.

property model_id

The ID of the loaded model.

Type

string

streamer module

class Streamer(queue_depth=2, inter_msg_time=0, drop_frames=True)

Host a video and data streaming server on the device.

Streamer can be instantiated as a context manager:

with edgeiq.Streamer() as streamer:
    ...

To use Streamer directly, use the setup() and close() functions:

streamer = edgeiq.Streamer().setup()
...
streamer.close()

For a video stream, the default parameters are all you need. The Streamer web page has a “stop” button which you can check with the check_exit() function:

with edgeiq.Streamer() as streamer:
    while True:
        <get video frame>
        streamer.send_data(frame, 'This will be printed on the Streamer')

        if streamer.check_exit():
            break

For a series of images (slideshow mode), set the queue depth to the total number of images to be displayed, and set the inter-message time to the time in seconds you’d like each image to be displayed. Note that the inter-message time doesn’t take into account any delay in sending the message to the Streamer. You’ll also need to call the wait() function to make sure the app keeps running until all images have been sent to the web page. The “stop” button also works in this mode, but doesn’t need an explicit check since it’s checked in the wait() function.

with edgeiq.Streamer(
        queue_depth=len(image_list), inter_msg_time=4) as streamer:
    for image in image_list:
        streamer.send_data(image, 'This image will be displayed for 4 seconds')

    streamer.wait()
Parameters
  • queue_depth (integer) – The depth of the data queue used for communication between the application process and server process.

  • inter_msg_time (float) – The time in seconds between message sends to attached clients.

  • drop_frames (boolean) – Indicates whether send_data() should block when queue is full or drop frames.

setup()

Setup and start the streamer.

Returns

self

send_data(image=None, text=None)

Send image and text data to be displayed by the streamer.

Raises any exception that occurs in the Streamer process.

Parameters
  • image (numpy array) – The image to be displayed by the streamer.

  • text (string or list of strings) – The text to be displayed by the streamer. Each list element is shown on its own line on the streamer.

wait()

Wait for all packets in the queue to be sent.

Will exit if the “stop” button has been pressed.

check_exit()

Check if the “stop” button has been pressed.

close()

Stop the Streamer server.