Bi-Weekly Report 6

January 29, 2016

Summary

During the last two weeks, we consolidated the development efforts each member made over the break and resolved issues with the architecture and deployment strategy before starting with the implementation of features. We made ample progress with development, having finished implementing almost every “Must Have” requirement.

Meeting 16

Date 19/01/2016 Time 14-16

During our first meeting as a team after break, we caught up on the progress in technical research and development that each of us made. We realized that due to issues with communication as well as the deployment of skeleton architecture, none of us were able to make as much progress as we had hoped. Therefore, we spent the lab session resolving issues with the deploying the architecture.

Meeting 17

Date 20/01/2016 Time 14-15

We discussed our progress with the client detailing our achievements and setbacks before presenting a demo of our work so far. They asked us if we were on track with the project, and we reassured to them that features would now be developed very rapidly as we now have a functional architecture and deployment strategy. At the end of the meeting, our clients explained that their business is undergoing changes and thus they will not be able to continue meeting once a week in the weeks to come. After the meeting, we referred to our requirements list to delegate tasks and decide which features would be developed by whom before the next meeting.

Meeting 18

Date 21/01/2016 Time 14-16

After a lecture for the module, we met up briefly to agree on a timeline with five milestones/internal deadlines which we can refer to in the future to gauge our progress.

Meeting 19

Date 26/01/2016 Time 14-16

We started off the meeting by finalizing the two minute elevator pitch and then reviewed the current state of development, testing features and trying to find bugs. During the remaining time, we developed the “find rows with missing values” feature as a team and decided on what each of us would do before the next lab session.

Meeting 20

Date 28/01/2016 Time 14-16

After giving the two minutes elevator pitch, we had a brief meeting to catch up on development progress and test the new features that were implemented after the last meeting. We also evaluated the current user interface of our application and agreed that the sidebar needs to be redesigned with a less cluttered layout. It was decided that the sidebar will be restructured with tabs.

Progress

Resolved issues with development architecture and deployment strategy
Integrated Angular Material front end into skeleton architecture
Implemented features: upload screen, change column name and type, remove selected rows, view rows with missing values, clean missing values
Began unit testing on backend
Decided on roadmap/timeline of project with five milestones

Outlook

As we become more familiar with the architecture including the frameworks and libraries, our development pace has become quicker and more consistent. We are on track if not ahead of schedule to complete our first milestone which is to develop all the “Must Have” functional requirements by February 3rd.

Plan

We aim to maintain the momentum we have gathered with development and continue to implement features as per our functional requirements. As our codebase grows, we need to begin unit testing on the front-end and start integration testing as soon as our system is ready for an end to end data cleaning use case.

Bandi Enkh-Amgalan

My teammates revealed that they were unable to conduct development because the skeleton architecture could not be deployed onto their computers, so my first task during the last two weeks was to fix this major issue. I overhauled the deployment strategy by switching from using Vagrant provisioning scripts to packaging a preconfigured virtual machine image. This resolved the issues my teammates were experiencing and allowed them to begin developing features on top of the skeleton architecture. Besides fixing issues with deploying the development environment, I implemented the rename column, change data type and visualize (line chart, bar chart and histogram) features and helped with the “show rows with missing values” feature. I also performed code refactoring on the front-end using directives in order to follow AngularJS best practices and improve code maintainability.

Shivam Dhall

Over the course of the previous two weeks, I have been working on familiarising myself with our front-end framework which is AngularJS, this included learning how to use features such as Two-way Data Binding and Routing using the ngRoute module. I also acquainted myself with Angular Material which we make use of within our interface to implement specific UI components. Using this I was able to assist my teammates in the development of some aspects of the user interface. I have also been working on the testing of our application, so far I have begun implementing unit-tests on some of our python modules which are used to carry out data cleaning operations – I implemented these tests using a testing tool called Pytest. Finally, I also spent some time preparing and practising for the elevator pitch on Thursday as I was presenting.

Gordon Cheng

Over the course of the last two weeks, our team started development phase of the project and I focused on building the frontend of the web-app and implementing some of the backend features. I was tasked to implement most of the features related to cleaning missing values in the dataset. This included the function to fill missing values using the average value, the nearest valid value, a custom value, and by interpolation. After completing the missing values features, I moved on to implementing more cleaning features such as feature scaling. I also worked on refining the user interface of the cleaning tab, this included managing the separation between the elements on screen to create a more cohesive layout; and refining the scrolling behaviour of both the toolbar and the spreadsheet to make operations easier. Other than the development of our PoC, I also worked with my teammates to prepare slides and scripts for the elevator pitch presentation.