The demand for convenient retail is the main force shaping the industry today and both online shops and brick-and-mortar stores strive to offer the best experience to their customers. And while there are many strategies devised to increase customer satisfaction in digital retail, physical stores still struggle with meeting shoppers’ expectations. At Abto Software, we have already developed an easily deployable Cashierless Checkout for retail outlets that eliminates queues and ensures the growth of the store attendance. Today we offer yet another retail automation technique: camera-based product recognition technology for smart fridges and intelligent vending machines.

Recognition of Beverages for Vending Machines and Fridges

Abto Computer Vision engineers have developed an AI-driven grocery product recognition model for vending machines and smart fridges. By automatically detecting what products the customer picks from the shelf it provides a frictionless shopping experience and simplifies the purchasing process altogether.

Only one camera is needed to automate vending machines and fridges: placed in the outer door, facing inwards it monitors the shelves and detects which products the customer takes out. Such an approach allows both retrofitting existing vending machines and developing new self-serve shopping solutions. It can also be used to upgrade micro markets to fully automated convenience stores.

The next videos demonstrate how our grocery product recognition model works for two scenarios:

  1. two customers take out and put back different groceries from the fridge;
  2. a customer places different products on the table then takes them off.

These demos show how our grocery product recognition model works specifically for beverages. In order to test the robustness of the AI model we have selected a wide variety of drinks: baby food, yogurts, soft drinks, mineral water, beer, energy drinks, iced tea, etc.

Real-time Beverage Recognition in Fridge

Real-time Beverage Recognition on Table

The Accuracy of Grocery Product Recognition Prototype

The following table lists the accuracy of product recognition for 11 beverages selected for the demo. GPU-based implementation of the prototype provides real-time processing at 13 FPS with an average accuracy of 87%.

Product TypeProduct ClassProduct Recognition Accuracy
Plastic bottlesActivia yogurt100%
Coca-Cola bottle98%
Fanta Orange bottle100%
Lipton Ice Tea96%
Glass bottlesBorjomi mineral water81%
Gerber juice96%
Corona Extra Lager Beer bottle57%
CansCoca-Cola can100%
Pepsi can88%
Bud Light Beer can78%
Red Bull Energy Drink66%
Total Average Accuracy of Product Recognition Prototype – 87%

Tools and Technologies

Computer Vision & AI tools: Single Shot MultiBox Detector (SSD), fastai, ResNet, advanced video and image processing, image stitching.

Product recognition model specifications: the highest accuracy is achieved for grocery products distanced 0.5 – 1 m from the camera that should be positioned at approximately 45-degree angle to the products.

Data required for product recognition model training: one of the benefits of the proposed grocery product recognition model is the reduced amount of data required for its training. The average recognition accuracy of 87% has been achieved for a scarce training dataset consisting of two 30-second videos per class.

How Product Recognition Boosts Vending Machine Business

Abto product recognition approach for smart fridges and intelligent vending machines ensures your customers have the most positive buying experience. What can be more convenient than just grabbing a snack and continuing on your way without concerning yourself with stuck items, incomprehensible instructions on the screen, or having enough change? And on top of that, most people want to check the nutrition facts panel on the product label before buying it: according to the January 2020 NPD Group press release, nearly 90% of consumers actually read food labels before making a purchase. Our approach to intelligent vending allows your customers to do just that! They can pick and examine the products before making their mind up as the purchase is finalized only after the door is closed.

Such a strong focus on increased customer satisfaction contributes directly to the bottom line and allows you to expand the vending machine business quickly and with little overhead.

Apart from smoothing the shopping experience, the benefits of incorporating camera-based product recognition into vending machines and refrigerators include:

  • effortless modernization of the existing vending machines, refrigerators, freezers, and shelving fixtures;
  • low hardware cost of the upgrade as our approach DOES NOT require RFID chips, weight sensing plates, or smart tags;
  • simple extension of the list of products recognized by the model with your own-brand products;
  • data-driven insights on consumer habits and preferences gained through gathering real-time purchase data;
  • timely restocking due to 24/7 inventory management and efficient stock control;
  • using in-depth sales analytics for implementing and assessing targeted marketing campaigns.

Upgraded smart vending machines can be placed in offices, employee breakrooms, hotel lobbies, hospitals, universities, campuses, sports facilities, residential complexes, co-working spaces, malls, metro stations, and about any other public venue.

AI-driven Product Recognition for Variety of Applications

Retail automation has long ceased to be an option – it’s an urgent necessity of the 21st century. Even though we developed the described camera-based grocery product recognition to reinforce the services of traditional vending machine business, it has a variety of other applications.

  • Micro markets & unmanned stores
    Abto food recognition technology is basically a cashierless checkout made adaptable for different implementations. It can transform micro markets, that is small self-serving unmanned shops, into fully automated Amazon Go-like cashierless stores.
  • Food delivery services & ghost kitchens
    Ghost kitchens, also known as virtual kitchens, are delivery-centric restaurants without an actual dining area that prepare food for delivery only. Abto food recognition technology can be used to track dispatched orders in such facilities and help in speeding up the overall food delivery time.
  • Restaurants & intelligent catering solutions
    One of the ways to change the typical restaurant experience is to introduce an intelligent catering system. Coupled with accurate product recognition it allows to fully automate food delivery within the dining area.
  • Brick-and-mortar stores
    Abto grocery product recognition technology can be incorporated into smart shopping carts, self-checkout kiosks, and scanning tunnels to streamline the purchasing process, reduce shoplifting, and cut costs on RFID tags that are usually used in self-checkout solutions. It can also be applied to the automation of the planogram compliance checks.

Fill out the form below to learn how implementing Abto food recognition technology can accelerate your business growth.


Contact Us

To find out more about Abto Software expertise, request a quote or get a demo of your custom solution.

  • Clicking this button, I agree to the processing of my personal data.

Contact us

Tell your idea, request a quote or ask us a question