On My Plate – keep an eye on your calories

Mobile applications recording everything from steps walked to heart rate have become popular on an incomparable scale. The essence of them is to wake user healthy habits and enhance the quality of life.

Abto Software developers built a mobile application prototype named OnMyPlate. They employed computer vision technology for food recognition and analysis that uses images to count calories and consequently provides nutritional information. Our engineers came up with an incisive way to apply image and volume recognition methodology into mainstream consumer life. The real trailblazer of Abto’s prototype is self-consistent algorithm for meal recognition with the highest level of simplicity while providing the right functionality.

Team and Technologies

The team of 7 software engineers has been working on the research and implementation of the project for 6 months.

Technologies applied:

MATLAB,C++, Python, OpenCV, Qt,Android, image processing, machine learning, feed-forward artificial neural networks (multi-layer perceptrons), image segmentation (k-means)

Main features of the proposed method include:

• Identification of food items from camera-enabled deviceimages
• Ability to quickly search and log food items.
• Interactivity with no communication delays
• Real-time calorie count
• Simple, lean and intuitive interface
• Advanced search engine
• Extensive food database
• Possibility to add and edit food composition
• Automatic updates available

Calorie Tracking Application  and key benefits

The purpose of a mobile food recognition system is estimating calorie intake and recording eating habits of a user. All processes are performed on a smartphone in a real-time mode. The user only has to download an app,  take a photo of the plate and the rest is processed by the program.

– Simplified tracking calories method

– Proved feasibility of the method with 75% precision rate

– Improved estimation by utilizing contextual clues

– Real-time performance

– Available for Android and iPhone devices

Value delivered

Established on extensive technical investigation, Abto Software food recognition method combines two parts: qualitative analysis (food recognition, based on neural network model) and quantitative analysis (calories estimation). Image classifiers are trained to identify and categorize individual food items on a plate from a single image.

Among other challenges, a profound food database formation is one of the milestones of the project. Second, a category of food usually contains multi-view variations due to how it’s prepared and served. Also, arbitrary lighting conditions, volume of food may lead to varying visual appearance of meals. Besides, the quality of images taken with mobile phones makes the task even more complex.

For the design and evaluation of the prototype system, a visual dataset with nearly 10000 food images is being created and organized into 100 classes.

By creation of this app  we demonstrate the potential of computer vision, clustering and machine learning.  We aim for implementing a mobile recognition system which can run in the real-time way, suitable for daily use with image features to be extracted.