Shazam for food! HealthifyMe’s Snap AI tool makes calorie tracking fun and easy

Remember SeeFood? The fictional “Shazam for food” app featured in the HBO sitcom Silicon Valley. Well, it appears to be a reality now with HealthifyMe’s new AI feature – Snap!

If you’re tired of the arduous process of manually logging your meals and tracking your calorie consumption through various health apps, you might find HealthifyMe’s newest innovation, Snap, quite appealing. Snap is a food recognition system that uses artificial intelligence to recognize and monitor your food intake through photographs. You don’t even need to open the app for it to work. Just capture a screenshot of your meal on your phone’s lock screen and let Snap take care of the rest later.

Abhijit Khasnis, Vice President of Engineering, HealthifyMe spoke to Business Today about how Snap makes tracking nutrition information a breeze and simple.

How does Snap leverage AI for food recognition? What makes it significant from a technical point of view?

Snap, the world’s first ‘Fire & Forget’ food recognition system, automates the nutrition tracking process, using innovative computer vision AI to identify and track foods directly from the photo gallery. It combines lightweight, efficient AI models on the user’s phone with larger, more complex foundational AI models on HealthifyMe’s servers that have been fine-tuned using thousands of food photos. Users just need to take a photo of their plate of food, which they can do from their phone’s lock screen without even launching the HealthifyMe app. HealthifyMe’s advanced AI features take over, automatically detecting foods to record meal facts. It overlays heuristic data to personalize results. Once meals are recorded, our AI systems also provide feedback on the food consumed. This includes insights into macros and micros, along with user-specific nutritional recommendations and information.

What led you to launch Snap?

Nutritional tracking is an important tool for achieving your fitness goals. However, unlike steps or heart rate, it requires significant discipline and involvement from the user. With Snap, we make meal tracking as easy as clicking a picture. This ease of tracking food is a game changer for anyone looking to elevate their fitness level. As a result, clients are more aware of their dietary intake patterns, allowing them to make healthier choices on their own or with the help of their HealthifyMe coaches.

How do you ensure accurate calories and macros, especially when it comes to health-related matters?

The Snap AI model has access to heuristic analysis to personalize Snap’s recommendations using billions of previously recorded foods, as well as informed people whenever necessary. The hybrid technique used produces a remarkable 75% accuracy rate, which is expected to increase as HealthifyMe Snap processes more and more images. Currently, Snap is tailored to Indian cuisine, which is typically a cuisine that is difficult to recognize due to the lack of images/reference data and the diversity of foods in India. With HealthifyMe’s extensive knowledge and model training, Snap is able to recognize a variety of popular Indian dishes from across the country. It can also recognize packaged foods and drinks typically found on store shelves and is well trained to detect international foods popular in India, such as pasta, burgers and salads. Additionally, each food image is shown to the user when they open the app for review to ensure accuracy. This ensures that the power is in the user’s hands, with Snap simply doing the first round of detection to save the user time and effort.

What has Snap’s initial response/traction been like?

The consistency of customer meal tracking has already increased by more than 80%. We’re also piloting these features on WhatsApp to allow more customers to try SNAP.

See how it works:
You click a food photo or upload it to your HealthifySnap number.
Snap will send you calories, PFCF, and information about your nutrition.
If you have registered a Healthifyme account with the same WhatsApp number, it automatically tracks food for you.

How is user privacy maintained with Snap?

HealthifyMe attaches utmost importance to protecting user privacy. Snap is designed to ensure that only photos recognized as food are uploaded to the app, thanks to an “on-device” model, after explicit permissions from the user. No non-food images are uploaded to HealthifyMe servers. Additionally, the ability to automatically analyze photos is only accessible to those who have given Snap access to their camera roll.

What data did you train your AI model on?

Over the years, HealthifyMe has tried to bring image-based food recognition; however, the development of more advanced core AI models enabled the creation of Snap. We trained Snap on thousands of Indian foods and hundreds of photos of those foods. Currently, 150,000 Indian cuisine items can be recognized by the resource thanks to training. Human reviewers examine and rectify erroneous recognitions. To improve the model, users can also manually identify these misidentified images.

What innovations does HealthifyMe foresee from what it learned from Snap?

We are monitoring the feedback we receive on its use to increase the number of foods detected in global cuisines, which can help us with our upcoming international launches and improve food quantity recognition. Furthermore, Snap is a step towards automatically tracking important data. We’ve already done this with SmartScale. We will continue to add more health attributes to this “passive tracking” system over time.

Leave a Comment