Shopping is a daily necessity, but for many, especially the visually impaired, it can be a frustrating and overwhelming experience when trying to locate items. We wanted to create a solution that not only makes shopping easier but also empowers individuals to do so independently.
Scout is a portable hardware solution that can be easily attached to trolleys. Once in place, it scans nearby items and provides audio cues when an item from the user's grocery list appears. It allows the user to add and remove items with simply their voice.
We built Scout using a Raspberry Pi to analyze items, process voice commands, and generate audio responses. A webcam scans nearby items, while a built-in microphone enables users to add, remove, or clear items from their shopping list through voice interaction.
We faced several challenges, including connecting peripherals to the Raspberry Pi, fine-tuning the YOLO object recognition model, and dealing with speech recognition issues, particularly the failure to recognize key commands.
We're proud to have created a portable solution that provides tremendous support to visually impaired users to shop independently. Despite the technical challenges, we successfully built a working prototype that integrates object recognition and voice interaction in a compact, user-friendly device.
We learnt a lot about hardware integration, particularly with the Raspberry Pi, since none of the team members have worked with it before. Additionally, we gained a deeper understanding of the challenges faced by visually impaired individuals in everyday tasks that some may deem as mundane, like grocery shopping
We aim to make the device even smaller and portable by using more built-in components to save space. We will also work to print a 3d stand for the camera, allowing it to move on its own without the trolley being pushed when it comes to a halt.