Semantic Segmentation

Semantic segmentation is essentially image classification at a pixel level, as it takes an image and assigns a class to each pixel in the image. This can be helpful whenever you need more precision than is provided by object detection, since semantic segmentation gives you a more complete picture of the exact boundaries of the different elements in the image. Semantic segmentation can be used for tasks such as medical imaging analysis, surface area calculations, precision agriculture, and detection of surface defects during manufacturing.


Semantic segmentation can be performed on an image using the SemanticSegmentation class. The first step is to instantiate a SemanticSegmentation object with the ID of the model to use. For example:

semantic_segmentation = edgeiq.SemanticSegmentation("alwaysai/enet")

(If the model is not a semantic segmentation model, the instantiation will fail with an error message indicating that the model can’t be used.)

Next, call the object’s load() function to initialize the inference engine and accelerator:


Unless directly specified, the accelerator chosen will be the default for the provided Engine. Now the semantic segmentation object is ready. Use the segment_image() function to detect objects in an image:

results = semantic_segmentation.segment_image(image)

The results object is of type SemanticSegmentationResults and contains the duration of the inference (in seconds), and a class map, which maps a class to each pixel.

Since it can be useful to visualize the segmentation, you can create a color mask by using build_image_mask() and then apply the mask to the image using blend_images():

color_mask = semantic_segmentation.build_image_mask(results.class_map)
blended_image = edgeiq.blend_images(image, color_mask, alpha=0.5)

The blend_images() function takes as its input an image (the same image that you used in the segment_image() function), the color map generated by build_image_mask(), and the alpha factor, which determines the opacity of the color mask and image. By setting alpha to 0.5, both the mask and the image will have equal opacity.