Architecture
Desktop Assistant
For Desktop Assistant part, on the top there is a tools bar which has functions like document, settings, set up alarms, search, see local news based on persona, bus lines, map, enter AR mode, play music, turn on/off other functions. There is also search bar, music player, message notifier – which gives notification messages like alarm, interesting news in the form of a speech bubble. And, of course, the pet – there is a pet house, when we click on the house, the pet walks out with some animations (Figure 1). All bars can be hidden, except for the pet.
AR Mode
When we enter AR mode we can feed and play with the pet. However, the other functions (search, alarm, local news, music player, etc) are not available. Functions like TTS, ASR and music player are still available under AR mode (Figure 2). The server synchronizes the position of the pet, and under all modes, the server stores user profile and pet info.
Structural Diagram
Design Patterns
Singleton
Observer
AI Design
Pet AI
We chose the State Machine AI to develop the pet AI. Why? This project is more of an AI assistant in the form of a pet, rather than a pet as complex as learning the user’s previous pet behavior. Also, we don’t want a pet that is 100% real, such that for example if you forget to feed them in a long time it will die. Hence, using a State Machine AI will be sufficient for this project's purposes.
Typical states include: Idle --- licking, sleeping, stretching; Moving --- walking, running, jumping around; Reacting to human gesture --- sit, stand, rolling, enjoying petting; Playing; Eating (Figure 3). The pet will automatically turn to the user and chase if distance is larger than 1 meter. It might also want the user to notice if it is hungry.
Others
For other AI like speech recognition, we can say “Set an alarm clock at seven to remind me to get up”, so the new alarm pops up on top right. For text-to-speech, we can have it say “Jobs done”, or when we ask it to show bus timetable, we can have it respond with “The next bus No. 82 will arrive in 10 minutes”.
APIs
WAPET.login(password, account)
Sends the login account and its corresponding password to server, verifies it and if matches, the user can login into their account.
WAPET.GetStatus()
Get the status of the pet from server every certain period, including its name, age, and other info.
WAPET.GetStatus() (Overwrites)
Get the status of the pet from server every certain period, including its name, age, and other info, and its current state (under AR mode), position relative to the detected plane. Client on different platform can operate different actions once receive status, e.g. synchronizing actions with entity Interpolation.
WAPET.ChangeState(stateName)
Send the current state message to server once one device enters AR mode, so that server can send status message to each client. Typical states include Idle, Reacting to Human, Walking around, Eating, Sleeping. Check Pet AI part for more.
WAPET.TextToSpeech(text)
Send a text message to server and get an audio clip back.
WAPET.SpeechToText(audio)
Send an audio to server and get text back.
WAPET.GetBusTimeable()
Get local bus timetable, play the audio.
WAPET.GetLocalNews()
Get Recommended local news, personal interests are stored in database.