Product recognition for smart fridges and intelligent vending machines

The demand for convenient retail is the main force shaping the industry today and both online shops and brick-and-mortar stores strive to offer the best experience to their customers. And while there are many strategies devised to increase customer satisfaction in digital retail, physical stores still struggle with meeting shoppers’ expectations. At Abto Software, we have already developed an easily deployable Cashierless Checkout for retail outlets that eliminates queues and ensures the growth of the store attendance. Today we offer yet another retail automation technique: camera-based AI product recognition technology for smart fridges and intelligent vending machines.

Abto Computer Vision engineers have developed an AI-driven grocery product recognition model for vending machines and smart fridges. By automatically detecting what products the customer picks from the shelf it provides a frictionless shopping experience and simplifies the purchasing process altogether.

Only one camera is needed to automate vending machines and fridges: placed in the outer door, facing inwards it monitors the shelves and detects which products the customer takes out. Such an approach allows both retrofitting existing vending machines and developing new self-serve shopping solutions. It can also be used to upgrade micro markets to fully automated convenience stores.

The next videos demonstrate how our grocery product recognition model works for two scenarios:

• two customers take out and put back different groceries from the fridge;
• a customer places different products on the table then takes them off.

These demos show how our grocery product recognition model works specifically for beverages. In order to test the robustness of the AI model we have selected a wide variety of drinks: baby food, yogurts, soft drinks, mineral water, beer, energy drinks, iced tea, etc.

The following table lists the accuracy of product recognition for 11 beverages selected for the demo. GPU-based implementation of the prototype provides real-time processing at 13 FPS with an average accuracy of 87%.

Abto AI product recognition approach for smart fridges and intelligent vending machines ensures your customers have the most positive buying experience. What can be more convenient than just grabbing a snack and continuing on your way without concerning yourself with stuck items, incomprehensible instructions on the screen, or having enough change? And on top of that, most people want to check the nutrition facts panel on the product label before buying it: according to the January 2020 NPD Group press release, nearly 90% of consumers actually read food labels before making a purchase. Our approach to intelligent vending allows your customers to do just that! They can pick and examine the products before making their mind up as the purchase is finalized only after the door is closed.

Such a strong focus on increased customer satisfaction contributes directly to the bottom line and allows you to expand the vending machine business quickly and with little overhead.

Apart from smoothing the shopping experience, the benefits of incorporating camera-based product recognition into vending machines and refrigerators include:

• effortless modernization of the existing vending machines, refrigerators, freezers, and shelving fixtures;
• low hardware cost of the upgrade as our approach DOES NOT require RFID chips, weight sensing plates, or smart tags;
• simple extension of the list of products recognized by the model with your own-brand products;
• data-driven insights on consumer habits and preferences gained through gathering real-time purchase data;
• timely restocking due to 24/7 inventory management and efficient stock control;
• using in-depth sales analytics for implementing and assessing targeted marketing campaigns.

Upgraded smart vending machines can be placed in offices, employee breakrooms, hotel lobbies, hospitals, universities, campuses, sports facilities, residential complexes, co-working spaces, malls, metro stations, and about any other public venue.

Retail automation has long ceased to be an option – it’s an urgent necessity of the 21st century. Even though we developed the described camera-based grocery product recognition to reinforce the services of traditional vending machine business, it has a variety of other applications.

• Micro markets & unmanned stores
  Abto food recognition technology is basically a cashierless checkout made adaptable for different implementations. It can transform micro markets, that is small self-serving unmanned shops, into fully automated Amazon Go-like cashierless stores.
• Food delivery services & ghost kitchens
  Ghost kitchens, also known as virtual kitchens, are delivery-centric restaurants without an actual dining area that prepare food for delivery only. Abto food recognition technology can be used to track dispatched orders in such facilities and help in speeding up the overall food delivery time.
• Restaurants & intelligent catering solutions
  One of the ways to change the typical restaurant experience is to introduce an intelligent catering system. Coupled with accurate product recognition it allows to fully automate food delivery within the dining area.
• Brick-and-mortar stores
  Abto grocery product recognition technology can be incorporated into smart shopping carts, self-checkout kiosks, and scanning tunnels to streamline the purchasing process, reduce shoplifting, and cut costs on RFID tags that are usually used in self-checkout solutions. It can also be applied to the automation of the planogram compliance checks.

Fill out the form below to learn how implementing Abto food recognition technology can accelerate your business growth.