A Personal Handheld Multi-Modal Shopping Assistant

We describe a personal handheld multi-modal shopping assistant that we have developed to aid the everyday consumer in one of the most routine tasks, grocery shopping. The shopping assistant provides the user with the ability to create a shopping list with up-to-date prices and item images, search for items in a store database, suggest recipes based on items in the shopping list, and display a store map with the items in the shopping list highlighted to make them easier to find. The user can interact with the personal handheld shopping assistant using speech, graphics, and text. We not only describe the shopping assistant but also demonstrate its many useful features