Bi-Weekly Report 9
March 11, 2016
Summary
During the last two weeks, we followed through with our plan of completing all of the Should Have functional requirements after our stagnated progress during the Reading week and Scenario week. We made major headway with development and are now in a position to begin testing and potentially making further refinements to the feature set. We also met up with the client after not being able to meet with them for several weeks and we received very positive feedback on the current state of our application.
Meetings
Tuesday, March 1st
During this two-hour lab session, we reviewed the current status of our project and merged minor feature branches that each of us had worked on after Scenario Week. We conducted some manual regression testing as we had implemented some new features and uncovered some bugs. These bugs were assigned to members to be fixed by next week. Finally, we agreed on a list of Should Have requirements that each of us would implement by the following week.
Thursday, March 3rd
We had a brief meeting to test our application on Windows and discovered that the deployment did not work at all due to an issue with Celery workers getting killed by the guest OS. We attempted to diagnose the issue but could not come up with a solution in time, so the task of fixing Windows deployment was assigned to a team member.
Tuesday, March 8th
In this lab session, we merged the branches containing the features that each of us were assigned in the previous week, including matplotlib visualization, file exporting and data filters. We also took time to update our official requirements list on our project website with the revised set of requirements that we had been following internally. Finally, we made sure to perform some last minute testing on the application in order to prepare for the demo we would be giving for our client later that day.
Tuesday, March 8th
After not being able to get through to our clients for quite some time, we finally had a meeting to catch up and demo all of the new features we had implemented in the last couple of weeks. There was an extensive list of changes and new features to demo to the clients, but at the end of the meeting, they confirmed that they were happy with our work. In fact, they believed that the feature set was complete enough for us to begin distributing a beta release of our application to their developer community in order to get real feedback from users. We agreed to provide them with one as soon as we make refinements to some features and conduct alpha testing internally.
Progress
- Resolve issues with development on Windows
- Reimplement all visualization features with matplotlib
- Implement basic undo function
- Implement export to CSV and JSON feature
- Implement filter by duplicates and outliers feature
- Redesign UI for “Visualize” tab
- Redesign UI for “Analyze” tab
- Updated official requirements list
Plan
We aim to perform manual end to end testing on our application next week to uncover any bugs and oversights with features, in order to provide our client with a beta release by the end of the week. We also want to begin working on our internal deliverables for the module including the project video and documentation and updating the website.
Personal Paragraphs
Bandi Enkh-Amgalan
I finished reimplementing the visualization features with matplotlib during the first week, also adding scatter plots with regression metrics as a new feature. I supported Gordon in developing the filter by duplicates and outliers feature by creating the front-end and integrating it with the backend functions that he wrote. I was also the one responsible for fixing the deployment issues that were occurring on Windows machines which I was able to deliver by the end of the second week. Finally, I implemented a basic undo function on the front-end and I am currently working work on the sort by column feature.
Gordon Cheng
During the last two weeks, I continued to develop the system with the rest of the team. I primarily focused on the user interface of the system, refining it to make it look better and easier to use. I overhauled the layout of the analyze and visualize tabs. For the analyze tab, I made it so that the user can see up to 4 column analyses at once, as opposed to maximum of 2 previously. I believe this makes it easier for the user to compare columns. For the visualize tab, I put all the controls into a hideable sidebar and made the graph generated to fill most of the screen. I also added in a zoom functionality so the user has greater flexibility when it comes to viewing the graph. Other than the UI, I also worked on some data cleaning features, namely split and combine columns, and edit individual cell values. The prior allows the user to create new columns from existing columns which can be useful for dealing with data such as dates.
Shivam Dhall
Over the course of the last two weeks, I spent most of my time working on the file import and export feature of our application. One of the features I implemented was to allow the upload of JSON files to the application, this feature was particularly important considering a lot of the data received by our client is in JSON format. In order to deal with large files, I also added a feature which would allow the user to upload a random sample of the file using a specified percentage and an optional seed value. Another feature I am currently working on is to allow the user to export as well as import data sets in both SQL as well as XLSS (Excel file) formats as they are both widely used. Finally, I have also updated and added a number of unit tests to our existing suite in response to the addition of more functions to our backend python modules.