Evaluation
Summary
Achievements
| Requirement | Priority | State |
|---|---|---|
| Block-based interface | Must have | Done |
| 3D scene view that updates instantly | Must have | Done |
| Intuitive flying camera | Must have | Done |
| Material parameter blocks | Must have | Done |
| Texture imports applied on materials | Must have | Done |
| Multiple objects with position and rotation | Must have | Done |
| Object animation | Must have | Done |
| Generated JS code view pane | Must have | Done |
| AI chat assistant | Should have | Done |
| Point and directional light blocks | Should have | Done |
| Object hierarchies | Should have | Done |
| Sky parameter blocks | Could have | Done |
| Day-night cycle animation | Could have | Done |
| Color name AI model | Could have | Done |
| Export to Babylon.js Playground | Could have | Not done |
| Shared library of users' creations | Could have | Not done |
| glTF and other 3D model format import | Could have | Not done |
| Web-only app | Must have | Done |
| UI is smooth and updates instanly | Must have | Done |
| Intuitive and simple UI | Must have | Done |
| Runs on a wide range of hardware | Should have | Done |
| Easily extensible code structure | Could have | Done |
Thus, we have completed 100% of our key requirements and 57% of our optional requirements.
Known bugs
| Description | Priority |
|---|---|
| As discussed in System Design, Babylon.js is not optimized for low-latency scene editing, so in certain cases the scene can flicker briefly on edit. | Low |
| Upon Initialisation, if a network disconnection causes the code evaluation of babylon to not trigger, the JS view pane with be entirely blank until blocky is re-initialised by switching back to its tab. | Low |
Individual contribution breakdown
System
| Item | Aditya | Adwit | Sara | Shaye |
|---|---|---|---|---|
| Website setup | 10% | 70% | 10% | 10% |
| Babylon.JS integration | 10% | 60% | 10% | 20% |
| Blockly integration | 10% | 10% | 60% | 20% |
| Resizable panels | 40% | 40% | 10% | 10% |
| Object creation | 10% | 30% | 50% | 10% |
| Lights | 10% | 10% | 40% | 40% |
| Material parameters | 10% | 10% | 10% | 70% |
| Sky parameters | 10% | 10% | 10% | 70% |
| Object positioning | 70% | 10% | 10% | 10% |
| Object rotation | 40% | 10% | 10% | 40% |
| Hierarchies | 10% | 10% | 70% | 10% |
| Animation | 10% | 40% | 10% | 40% |
| Textures | 10% | 10% | 10% | 70% |
| Color name AI | 35% | 35% | 20% | 10% |
| JavaScript code view | 10% | 10% | 70% | 10% |
| AI help chatbot | 10% | 70% | 10% | 10% |
| UI layout | 70% | 10% | 10% | 10% |
| UI styling | 70% | 10% | 10% | 10% |
| Optimization | 60% | 10% | 10% | 20% |
| Unit testing | 10% | 10% | 10% | 70% |
| End-to-end testing | 25% | 25% | 25% | 25% |
| Overall contribution | 25.7% | 23.8% | 22.7% | 27.8% |
Note that the 10% contribution is due to everyone reviewing the code and providing useful insights and suggestions.
Report
| Item | Aditya | Adwit | Sara | Shaye |
|---|---|---|---|---|
| Template and Setup | 0% | 0% | 100% | 0% |
| Home | 0% | 0% | 100% | 0% |
| Video | 0% | 100% | 0% | 0% |
| Requirements | 0% | 0% | 50% | 50% |
| HCI | 0% | 50% | 50% | 0% |
| Research | 0% | 50% | 0% | 50% |
| UI Design | 100% | 0% | 0% | 0% |
| System Design | 0% | 0% | 20% | 80% |
| Implementation | 0% | 100% | 0% | 0% |
| Testing | 0% | 0% | 50% | 50% |
| Evaluation | 20% | 0% | 0% | 80% |
| Deployment Manual | 90% | 10% | 0% | 0% |
| Legal issues and Processes | 100% | 0% | 0% | 0% |
| Overall contribution | 23.8% | 23.9% | 28.5% | 23.8% |
Section 1 - User Interface
Our User-Interface/User-Experience, having undergone months of iterations and re-evaluations, seems well suited when considering our target audience and their possible expectations from a learning platform. Concious efforts were made to maintain clean and descriptive UI elements that can be easily understood. We also tried to restrict certain actions such as minimizing a pane to 0 width to prevent anomalous bugs on different web platforms. We are satisfied with the current interation of our UI, although further improvements could be made such as overlaying help markers based on the AI assistants response to a query.
Section 2 - Functionality
In retrospective, our feature set feels bare-bones, with not enough blocks for advanced users to build impressive scenes and projects. However, since our target audience is children aged 8 - 14, we need to strike the balance between an extensive feature set and the intuitiveness of the application. Since our original problem statement is that 3D development is very overwhelming for someone starting out, we need to ensure that we do not become overwhelming ourselves. Keeping this in mind, we feel that we have an optimal feature set for our age range, with enough complexity to keep older children engaged and creating, while being simple enough for a 10 year old to understand.
Section 3 - Stability
During our development, unit testing, and end-to-end testing phases, we encountered very few crashes or other major bugs, so we feel that the system as a whole is quite stable.
Section 4 - Efficiency
Due to the requirement for changes in the block workspace to instantly reflect in the Babylon.js scene, the entire system was designed with performance from the get-go. We employ caching wherever possible, especially since Babylon.js isn't optimally designed for our usecase - it is designed for scenes with mostly static objects, and only a few dynamic ones - while we need all objects to be dynamic and re-creatable, since the user can edit them at any time. As mentioned in the known bugs section, this isn't perfect, but we feel that the bug is barely noticeable to most people, and it does not become worse even on low-end hardware.
Section 5 - Compatibility
Being a web-based application for education, compatibility is of utmost importance. However, there are still some constraints:
- Being an app with multiple panes, both of which require a decent space, we only support laptops and desktops, and some tablets. We also require a mouse and keyboard, since navigation in a 3D space is difficult with a touchscreen
- Due to the 3D scene viewport rendering, we have higher performance requirements than most websites, and so, require a relatively modern system to run.
However, our system is able to run on a 12 year old laptop with only an integrated GPU with a usable framerate. We weren't able to conduct extensive testing on computers older and slower than that, but we believe that we have wide enough compatibility that Babylon Studio should be able to run in the computer labs of most schools.
Section 6 - Maintainability
From the outset, our project was designed to be as extensible and maintainable as possible. Further extensions are possible in many ways:
- Adding an entirely new feature with a UI can be done by just adding a new pane to the right side of the UI
- Adding more blocks can be done by following a simple JSON schema with a code generation function.
If the project scope grows far beyond the current scope, some refactoring will obviously be required, but it should be limited to splitting some files that might be getting too large.
Section 7 - Project Management
The project was managed by splitting our MoSCoW requirements into granular tasks, ordered by priority. Each team member was then assigned tasks, ensuring that the assignments were as equal as possible to ensure even contribution. Finally, when features or tasks were finished, they were reviewed by each other team member, and suggestions, feedback, and changes were taken into consideration. We believe that this was an effective method of project management, as it led to nearly equivalent contribution from all team members, as shown above. All key requirements were also completed on time.
Section 8 - Future Work
As mentioned in Section 6, due to our maintainable, future extensions to the system should be fairly easy, even for a new team. Some examples of features that could be added include:
- Importing 3D model files as objects
- Adding a spotlight type
- Adding an easy way to export the code to the Babylon.js Playground for further experimentation
- Adding more animations, as well as more loop types for custom animations
- Adding more material types, as well as more material parameters
- Add support for custom skyboxes, instead of just the atmosphere model