Understanding the Results Objects

The edgeIQ results objects make it easy to work with the inference outputs, and to use multiple methods (classification, detection, tracking, etc.) together.

Classification Results

The Classification class provides the inference time and a list of predictions in a ClassificationResults object. Each prediction is of type ClassificationPrediction and contains the label given by the model, and the confidence the model has in that label. The confidence_level input to classify_image() will filter the results by confidence, and setting it to 0 will return a confidence for each label. Since the confidence values are determined by the model, it can be helpful to first return the full list before choosing a confidence level threshold.

Object Detection Results

The ObjectDetection class provides the inference time and a list of predictions in an ObjectDetectionResults object. Each prediction is of type ObjectDetectionPrediction and contains the label given by the model, the confidence the model has in that label, a BoundingBox around the detected object, and the index of the label in the model’s master list. The confidence_level input to detect_objects() will filter the results by confidence, and setting it to 0 will return all detections for the image.

Once you have a list of ObjectDetectionPrediction objects, you may want to perform processing for only a few specific objects. For example, you may want to perform processing on all potted plants and chairs in the image. To do that, call filter_predictions_by_label:

filtered_predictions = edgeiq.filter_predictions_by_label(predictions, ['pottedplant', 'chair'])

Another tool enables filtering predictions by bounding box area. This could be used, for example, to filter out objects that are far in the distance:

filtered_predictions = edgeiq.filter_predictions_by_area(predictions, 1000)

The ObjectDetectionPrediction objects can be used to perform more specific classification on detected objects. For example, you may want to detect all birds flying by a window, then use a classifier to determine the type of each bird. Start by filtering out all detected birds:

filtered_predictions = edgeiq.filter_predictions_by_label(predictions, ['bird'])

Then, for each detected bird slice out that portion of the image and pass it to a classifier:

for prediction in filtered_predictions:
  bird_image = edgeiq.cutout_image(image, prediction.box)
  bird_type = classification.classify_image(bird_image)

ObjectDetectionPrediction objects, or really any predictions with a box (of type BoundingBox) and confidence, can be used in conjunction with object tracking. The CorrelationTracker class takes in a prediction with a bounding box and updates the location of the bounding box as new frames are provided. For each frame, the tracker returns all the predictions with updated bounding box location and confidence. The CentroidTracker class takes in a list of predictions and associates each with an object ID. For each frame, the tracker will provide a dictionary mapping the object ID to each provided prediction.

Bounding Boxes

The BoundingBox class represents a box that bounds a region of interest in an image. It is defined by two points, (start_x, start_y) and (end_x, end_y).

box = edgeiq.BoundingBox(start_x=100, start_y=100, end_x=200, end_y=200)

Two bounding boxes can be compared to see if they represent the same box, and a bounding box can be multiplied by a scalar to perform scaling along with scaling an image. Attributes include width, height, area, and center. center returns a tuple representing the center point.

An important aspect of bounding boxes is their relationship with other boxes or regions of interest in the image. The compute_distance() function can be used to compute the distance between the centers of two bounding boxes. The get_intersection() function returns a new BoundingBox object representing the intersection between the current bounding box and another, and the compute_overlap() function returns the fraction of the current bounding box that is overlapped by another.

The compute_distance() and compute_overlap() functions could be used together, for example, to determine if a person is riding a bicycle or walking. If a person is riding a bicycle, the distance between the person and bicycle should remain fairly consistent over multiple frames, and the person’s bounding box should be significantly overlapped by the bicycle’s bounding box.