Back to Results...

This research is performed within the scope of the project "Real Time Customer Action and Product Recognition" funded by AlfaZet Systems, Heusden-Zolder, Belgium.

The primary objective of the project is to design an automated system for product recognition, which would enable a completely automatic checkout process in closed-type restaurants. The system uses machine learning-based techniques to correctly identify multiple types of dishes and bottled beverages using only color images captured by strategically placed cameras overlooking the product tray. The tray containing the products is placed inside a setup with three cameras with different views: one top-view camera and two side-view cameras.

To enable automated checkout, the project tackles two main challenges:
1. Dish recognition
Designing an algorithm to detect and recognize 6 different types of white, circular dishes: 5 plates and 2 bowls, regardless of the food placed in it.
2. Bottled beverage recognition
Designing an algorithm to detect and recognize 50 different types of drinks packed in small bottles, regardless of the bottle position (horizontal or vertical).

The product recognition algorithm uses neural networks and digital signal processing techniques to detect, classify, and localize the different types of products. Two networks are trained for dealing with the different types of bottle placement on the tray: one for bottles positioned upright, and another for bottles that are lying down (positioned horizontally). By combining the outputs from both models, we get the final decisions for all detected bottles on the tray.
The algorithm achieves high detection and classification accuracy regardless of the type and intensity of the ambient lighting in the scene.
Below, we showcase sample results detected dishes and beverages.