Project Description


Speech-Activated Residential Assistant (SARA), is a computer based assistant that can help with scheduling, play music, tell the current and forecasted weather, and even wake the user up in the morning. SARA is implemented using the Microsoft Speech API (SAPI), which provides both voice recognition and speech synthesis. SARA uses the Windows Presentation Foundation (WPF) for its GUI, to provide a clean, consistent interface.

Design Philosophy

The design philosophy of the voice recognition part of SARA is that a user should be able to issue commands in the most natural way possible. I will use setting the alarm as an example to show the changes this design philosophy brought about. To set the alarm, the user can say "Set alarm for seven-o'-clock AM", but this feels very stilted and unnatural. The first way this command was improved was to simply add an alternate command for setting the alarm, so the user can say "Wake me up at seven-o'-clock AM". This is more conversational, and therefore more, natural, but it can still be better. The next improvement made was to rewrite the time recognition module to allow the user to omit "o'-clock", which should be obvious from the context, and also to omit "AM", which, while less obvious, it is pretty safe to assume that if the user says to wake them up at seven, they mean 7:00 AM. Of course, the user can still explicitly tell SARA to set the alarm for "seven-o'-clock PM" or "seven PM" if this is what he actually wants. As a result of these changes, the user can now say "Wake me up at seven", which is much more natural and conversational. This design philosophy is used throughout SARA, and, as a result, there can be several commands for every action.

Primary Features

Sleep State

When SARA starts up, the system is in sleep mode. In sleep, mode the system will not respond to any of its normal commands. The only command the system will recognize in this state is its own name. Saying "Sara" will cause the system to enter its normal listening state and respond to commands normally. When a command such as "goodnight" or "go to sleep" is issued, the system will transition back into its sleep state. It will also transition to sleep state after 60 seconds of not recognizing any commands. The sleep state was included in SARA so that the system will not interpret normal conversation as commands. If the system is asleep, saying "Sara, <command>" will wake SARA up and cause it to process the command issued. This is a very natural way of giving a command to SARA, and requiring a direct address when SARA is asleep is a good way of ensuring that it does not misinterpret normal conversation as a command.

Alarm System

The only set of normal commands that is currently fully-functional is the alarm system. If you ask SARA to wake you up at a certain time, or to set the alarm for a certain time, she will set an alarm for that time. At the time of the alarm, she will repeatedly tell you that it is time to get up, until you turn the alarm off or snooze it (5-minute intervals). See the documentation (forthcoming) or the code for the exact commands used for these and other features.

Calendar and Scheduling System

The Calendar and Scheduling System of SARA is currently unimplemented. There is a command that the user can use to ask SARA about his schedule, but SARA will respond with a hard-coded message stating that the feature has not yet been implemented. When this feature is implemented, SARA will use Outlook or a similar API to look at the user's calendar and both determine what the his schedule is and be able to add new appointments or reschedule existing ones.

Future Developments

Long-term improvements include voice and/or facial recognition to identify the user and guests.

Last edited Jan 5, 2010 at 8:12 PM by Arathald, version 8