Team 3

Summary of Achievements

Critical Evaluation

User Interface

The user interface was designed with simplicity and usability in mind. We initially proposed a design which displayed related metrics of the input paper along with the predicted citation count on the analysis page. In the final design we decided against this because of the difficulties obtaining these metrics, most papers were locked behind a pay wall and some metrics were simply not available.

Instead, the analysis page now shows the title and abstract of the input paper on the left, for reference, and shows the predicted citation count on the right. Our upload page, which is also the main page, is similarly intuitive with labelled input boxes for both the title and abstract. The title is input into a text box, while the abstract is input as a file that is uploaded to the page. The user has the option to drag and drop or browse the pc and select the file, depending on their preference. Users can also easily change between models with the box at the bottom.

Both pages are simplistic in design and easy to navigate. They meet all the users’ needs and give the users alternative ways to interact with the interface.

Functionality

The program functions without problems and serves its purpose well. The upload page takes the title and abstract of the paper as input and the analysis page displays the citation count prediction for the user. The models that we used all meet an acceptable value of R^2 which is the metric we are using to measure the model’s performance.

Stability

We made sure to write unit tests for all the key functions in our web application to ensure the program is stable and any bugs can be found and fixed. The final product contains very few known bugs which are all very low priority.

In terms of the models themselves, we have documented our experiments testing the effects of tuning different hyperparameters and the effect it has on the performance of the models. These tests were carried out to find the optimal values of hyperparameters to increase the performance of our models. However, the experiments we have carried out are difficult to reproduce. This is mainly because it would take too long to run all the tests from the beginning. The pre-processing steps took especially long since we were dealing with a large data set of over 600,000 samples. Training the models can take very long as well, because we used cross-validation to get a more representative score when testing the models. Also, running the tests again would yield slightly different results because of the random nature of the testing and training split used to evaluate the models. We decided to do it this way to make sure the scores we got represent the model performance well, since certain splits will yield better scores than other splits.

Efficiency

Efficiency in our code was not a problem since the application code is very simple and only requires the use of the model to display the predicted citation count. Because of this, the code runs efficiently since the models are pre-trained and are used by the application for prediction. Training a new model every time a prediction is made would take very long, especially for the more complex models, so it was very necessary that we trained the models before hand and loaded them in our application when they are used for the prediction.

Compatibility

Our application is not platform dependent and will for both Windows and Linux machines. The application does require python to be installed along with some dependencies that can be found in the requirements.txt file.

Maintainability

Our source code is well documented, and we have used meaningful labels for naming variables and functions so that anyone reading it can easily understand and change anything. Adding models is also very simple, they can just be imported for use in the predict method.

Project Management

Our project was very well managed throughout all stages. We made sure to divide the work fairly between team members and we played to each other’s strengths when assigning tasks to make the development process as smooth as possible. We also met very frequently, usually once or twice every week, to share any progress or issues and to plan our next steps. This helped us meet all our deadlines without much trouble.

Future Work