This is the first iteration of LeChefu.

Prototype

For prototyping, we have created storyboards to show how LeChefu would interact with a user through carrying out key tasks and an architecture diagram that shows what we intend to link all components to implement the key features. Since we do not create a website or application, we do not have wireframes for the user interface.

Storyboards

We have created several storyboards for our chosen use case, the kitchen assistant use case. These storyboards highlights our program’s key features by illustrating several user stories and how Sota would interact with the user.

1. Updating the user’s groceries list

1. Identifying ingredients and accessing recipe information

1. Simple, straightforward interaction and setting a timer

Architecture Diagram

Infrastructure

To implement our Cooking Assistant idea, we require three different types of APIs for the following functions:

• Speech recognition
• Image recognition
• Recipes

In this section, we look at the research we conducted into several APIs for each area and the justification of why we chose to use these APis.

Speech API

Google Cloud Speech API [source]

• convert audio to text using neural network models
• easy to use
• recognizes over 80 languages and variants
• enable command-and-control through voice
• accurate in noisy environments

Microsoft Cognitive Services Bing Speech API [source]

• Various add ons like voice authentication.
• Can be directed to turn on and recognize audio coming from the microphone in real-time

API.AI [source]

API.AI provides Natural Language Understanding Tools and allows integration with external apps and integrations with Alexa, Cortana and messaging platforms.

• Useful to build command applications.
• Supports 14 languages

Conclusion - For the Speech API we decided to use Google Cloud Speech API. Although Microsoft offers quite a few addons such as voice authentication, we feel that is not a core or necessary requirement for our project. Google Cloud Speech API offers the widest variety of languages, which would be beneficial for our client in the future, if they decide to expand the market for this product. It is described as being “accurate in noisy environments”, which is important in a kitchen environment. Furthermore, reports suggest that Google Cloud Speech API is the best speech API in the market.

Image Recognition API

Google Cloud Vision API

This API includes “image labeling, face and landmark detection, optical character recognition (OCR), and tagging of explicit content.” [source]

Clarifai [source]

• Has a dedicated “Food Recognition Model”
• “Create a custom model that fits perfectly with your unique use case.”
• Automatically tags images and videos to allow quick search through content

Conclusion - For the image recognition API we chose Clarifai. We chose Clarifai over the Google Vision API since Clarifai has a dedicated “Food recognition Model”, which is the most important image recognition feature that we need. Moreover, Clarifai helps to determine how healthy your food is based on the photo, something that would make Sota even more helpful to users. Additionally, although Google Vision API offers many different image recognition abilities, most are not needed for our program. Clarifai’s ability to automatically tag images makes scanning its environment quicker and more accurate.

Recipe API

Spoonacular API

• Searching for Recipes
  • find recipes to use ingredients you already have (“what’s in your fridge” search) ✓
  • find recipes based on nutritional requirements ✓
  • semantically search recipes ✓
  • find similar recipes ✓
• Extract/Compute
  • extract recipes from any website ✓
  • classify a recipe cuisine ✓
  • convert ingredient amounts ✓
  • compute an entire meal plan ✓
  • map ingredients to products ✓
• Get Information
  • get the price of a recipe ✓
  • get nutritional information for ingredients ✓
  • get in-depth recipe and product information ✓

Yummly API

Yummly API uses the world’s largest and most powerful recipe search site. It includes:

• Weight Loss/Tracking
• Allergy
• Shopping List
• Cooking and meal planning
• Food Brands
• Social Apps

Conclusion - For the Recipe API we decided to use Spoonacular API. This API offers numerous features, as described before, and is considered one of the best APIs regarding recipes. Yummly API uses a single site to gather the data while Spoonacular uses multiple sites, which enables a more extensive search and returns a more diverse collection of recipes.

Natural Language Processing (NLP)

Problem: In order to parse strings accurately and allow more natural conversations between Sota and the user, we needed to analyse the sentence to enable the program to understand the request.

For example, when parsing for ingredients to use in querying the API, some ingredients can be made of two words.

“Find a recipe using sweet corn and flour” -> “corn” and “flour”

This is not ideal, as adjectives can be significant to the description of an ingredient.

Solution: We used an NLP API to recognise the parts of speech that each word belongs to and use these tags in the logic of our functions.

For example, in the previous case case, we used NLP to recognise parts of speech and avoid dropping adjectives that come right before nouns in the output.

“Find a recipe using sweet corn and flour” -> “sweet corn” and “flour”

While exploring the idea of using NLP, we looked at several NLP APIs.

Apache Open NLP [source]

Open NLP is an Apache-licensed machine learning based toolkit for “the processing of natural language text”. It supports part-of-speech tagging, which is the NLP feature that we require. As it is open source, we are able to use it with no additional costs to the project.

User Interface

Most of our user interface is done through speech, since the main role of LeChefu is to be a hands-free cooking assistant. We have identified several activation words, for each of the main features, which we have included in the user manual.

Human Computer Interaction

As our client has highlighted that, although Sota was not built to hold natural conversations with users, we should make user interaction as natural as possible.

Throughout development, we will be using tools like a Natural Language Processing API to give our software a better idea of sentence structure and using our User Acceptance Tests to identify factors like most commonly used phrases.

One potential problem with relying on Speech UI that we have found is the action of signing into Google Drive. Users will most probably prefer not to say their email address and password aloud.

User Acceptance Testing

We will employ User Acceptance Testing to test the product on our end users. User Acceptance Testing is a method of checking that previously defined requirements are still on track. They portray user requirements in a directly verifiable way and provide an overview of how complete our system is. Carrrying out User Acceptance Testing in addition to unit and integration testing will ensure that we identify problems that those have missed, since it defines behavior instead of functionality. We would like to test LeChefu’s features in its environment and judge how easy it is to use the SOTA for our use case.

The specific approach that we are taking for this is Alpha & Beta testing. This involves two User Acceptance tests, once during the development of the system and another at the end of development. The initial Alpha test enables us to catch and fix problems before testing our final product. The Beta test allows us to test our final product with our client and potential users.

To create effective User Acceptance Tests, our tests will be based on our User Stories and we will define the acceptance criteria according to the system’s requirements.

Testing Strategies

During and after the development of PoC finishes, we will conduct testing to ensure that the product works as intended.After researching on various approaches to and tools for testing, we decided to use the following testing methods:

Unit Testing

We will use Unit Testing to test the functionality of the separate components, including units conversion, grocery list and ingredient recognition. We have compared multiple testing frameworks, such as Arquillian (highly innovative and extensible testing platform for JVM allowing developers to create automated integration, functional and acceptance tests), JTest (automated software testing and static analysis toolbox) and JUnit (unit-testing framework).

In the end we decided to use JUnit; writing JUnit tests is simple and elegant, JUnit tests can be bundled into multiple test-suites testing different parts of the system and they are written itself in Java, bringing the learning curve of a new framework to minimum.

Unit Testing has several main advantages that relate to our system¹

• They increase confidence in changing and maintaining code. This is especially useful when developing code concurrently and in iterations, where we have to go back to previously written code to improve it. After every change, we are able to quickly verify that the units still work as intended by running the tests. Furthermore, unit tests can be useful for future developers of the project when they are maintaining the code.
• Making unit tests encourages us to develop more reusable code because the functions used in unit tests should be modular to enable its unit test to be written.
• Costs of fixing problems found at the unit testing level is less compared to if they were discovered at higher levels, since it usually take a shorter time and less effort.
• Once unit tests are written, they can be run after every small change, meaning that developers only have to debug the latest changes. Narrowing the scope in which developers have to look saves a considerable amount of time.
• Unit testing enables us to write more reliable code, as the code works according to our end goal for the program.

We plan to write test suites for the main algorithm, including, but not limited to, these features:

• The conversion function for all measurement units
• Updating to and reading from grocery lists
• Recipe finding functions

Integration Testing

The phase in software testing in which individual software modules are combined and tested as a group. It occurs after unit testing and before validation testing. It takes, as its input, modules that have been unit tested, groups them in larger aggregates, applies tests defined in an integration test plan to those aggregates, and delivers as its output the integrated system ready for system testing.²

Integration tests are used to show that the major parts of a system work well together.²

There are two methods of Integration Testing:

Bottom-up

• requires the lowest-level units be tested and integrated first (referred to as utility modules)
• utility modules are tested early in the development process
• need for stubs is minimized
• the need for drivers complicates test management and high-level logic and data flow are tested late. Like the top-down approach, the bottom-up approach also provides poor support for early release of limited functionality.

Top-down⁴

• requires the highest-level modules be test and integrated first
• allows high-level logic and data flow to be tested early
• minimizes the need for drivers
• the need for stubs complicates test management and low-level utilities are tested relatively late in the development cycle.
• poor support for early release of limited functionality.

Functional Testing

What are functional tests? Functional testing involves testing the system against the preset functional requirements and specifications. In functional testing, the structure of the program is not considered. “Functional testing is normally called Black Box testing. The test cases for the entire system are designed from the requirement specification document for the system.”⁵

Typically functional testing involves the following steps:

• Identify functions that the software is expected to perform.
• Create input data based on the function’s specifications.
• Determine output based on function’s specifications.
• Execute the test case.
• Compare actual and expected outputs.

These are some of the techniques for Black Box testing:

• Equivalence class partitioning: This method involves dividing the domain of all inputs into a set of equivalence classes, so that if any test in an equivalent class succeeds then every test in that class succeeds. This technique identifies classes of test cases such that if one test case passes then so do the rest. Without looking at the internal representation of the program it is difficult to pick out these equivalent classes. This method tries to approximate this ideal. This approximation is an assumption that if the specifications require exactly the same behavior for each element in a class of values, then the program is likely to be constructed so that it either passes or fails for each of the values in the class.⁵

• Boundary Value Analysis: Many systems have the tendency to fail on boundary values, hence making it important to test these boundary values.This testing technique is where the extreme boundary values are chosen Boundary values include maximum, minimum, typical values, and error values. This testing technique forces attention to exception handling.⁶

ADVANTAGES OF BLACK BOX TESTING:

• Tester is free from any pressure of knowledge of specific programming languages to test the reliability and functionality of an application / software
• Facilitates identification of contradictions and vagueness in functional specifications
• Test is performed from a user’s point-of-view and not of the designer’s. It simulates actual system usage.
• Test cases can be designed immediately after the completion of specifications. ⁷

DISADVANTAGES OF BLACK BOX TESTING:

• Some tests can be redundant if they have already been executed by the designer.
• Test cases can be very difficult to design if the specification is unclear.
• This method can not be used to test complex code segments. [3]

Black Box testing does not test a system thoroughly but it is reliable when it comes to functional verification and ensuring a bug free system.

How are we planning to carry out our functional tests?

We are planning to use the Boundary Value Analysis technique to test extreme boundary values in the system. We plan on using this method to ensure the system does not break when the user enters too many ingredients in their grocery list or when searching for a recipe. We will also carry out some exploratory testing to ensure that the system fully conforms to the stated functional requirements. This will involve designing use cases from the functional requirements specifications.

FUNCTIONAL REQUIREMENTS:

• The robot should incorporate voice UI by allowing the user to create and update a grocery list by speaking to it: This feature will be tested by creating various grocery lists of different sizes. We will also try recording in noisy environments to check how the system performs under such conditions.

• The robot should verbally respond to user input: This can be tested by asking SOTA simple conversion questions such as “Convert 5 cups to ml”.

• SOTA should also incorporate Image recognition by being able to identify ingredients that the user might not know: We plan to test this feature by showing SOTA a variety of ingredients. We will display these ingredients to SOTA in various light intensities to check how it responds under such conditions.

• SOTA should be intelligent enough to recommend a recipe to the user depending on the ingredients the user has suggested to SOTA: This can be simply tested by suggesting certain groups of ingredients and ensuring the response from SOTA is actually correct.

• SOTA should also be able to walk the user through recipes using voice UI.
• SOTA should be able to carry out basic conversions e.g ounces to grams.
• SOTA should be able to set alarms and timers.

References

[1] Theme and Powered, C. (2016) Unit testing. Available at: http://softwaretestingfundamentals.com/unit-testing/ (Accessed: 26 February 2017).

[2] Martyn A Ould & Charles Unwin (ed), Testing in Software Development, BCS (1986), p71. Accessed 31 Oct 2014

[3] Hunt, A. and Thomas, D. (1999) The pragmatic programmer. Available at: https://www.amazon.co.uk/Pragmatic-Programmer-Andrew-Hunt/dp/020161622X (Accessed: 28 February 2017).

[4] Microsoft (2017) Integration testing. Available at: https://msdn.microsoft.com/en-us/library/aa292128(v=vs.71).aspx (Accessed: 28 February 2017).

[5] Tutorials Point (ed), Functional Testing. Available at: https://www.tutorialspoint.com/software_testing_dictionary/functional_testing.htm (Accessed: 9 March 2017)

[6] TechTarget (2017), Six functional tests to ensure software quality. Available at: http://searchsoftwarequality.techtarget.com/report/Six-functional-tests-to-ensure-software-quality

[7]