Implementation Details

1. Neurosynth Dataset generation.

INcDb requires a Neurosynth dataset that it uses to generate Term Analyses. The generation of Neurosynth Dataset is a memory-intensive operation which is not suitable to be done on a remote server, which typically has less RAM.

We suggest the Dataset to be generated on a local machine. Follow the instructions on https://github.com/neurosynth/neurosynth to build your own Dataset (neurosynth_dataset.pkl) and copy the Dataset to incdb-poc/app/data/assets/.

With the Dataset in place, we can initialise the database and add the Analyses to the database. These analyses will then be used to compute the correlation of the Components with the Terms.

2. Flask Application

The main function of the INcDb is the repository browsing. This is implemented using database query and Flask routing mechanism. Flask routing function help to pass variable name through the URL and this variable is used to select the necessary data from the database to be displayed.

The user management was implemented using the Flask extension - Flask-User. The user management allow signed in users to create Collection and upload Raw Dataset.

Flask-WTF, a Flask extension, was used to simplify the creation of the complicated Collection’s form.

3. Computation of correlation between Component and Terms

Another key function performed by the INcDb is to compute the correlations of the Components uploaded by our client with the Terms Analyses generated by the Neurosynth Dataset generation. As requested by our client, we referenced the computation done by the open-sourced Neurosynth-Web, https://github.com/neurosynth/neurosynth-web/blob/master/nsweb/tasks/__init__.py, and implemented a similar decoding function for INcDb.

The decoding is done when new Processed Dataset is uploaded to the server by our client and the top 10 Terms (with the highest correlations) for each Component are stored in a text files on the remote server. These text files are retrieved to display the correlation values when user view the Component.

For each Component of the Processed Dataset, a row in the Decodings table (in the database) is also created and tagged with the Term that it has the higest correlation with.

4. Neurosynth Viewer

The main aspect of the web application is to provide a way to view different brain images. This was done using the Neurosynth Viewer framework to view and interact with brain imaging data on a web page, as well as Nipype + AFNI to prepare the imaging data for Terms for a specific Movie.

The Neurosynth Viewer is implemented within the HTML code as part of a Javascript block. Since this is a Python Flask application, macro fields are added as a placeholder for the location of the brain imaging data to help load the relevant images for dynamically generated web pages. These macro fields run Python functions written earlier that, when given the name of a Component or Movie, it will search within the directories in the server and send the files as an attachment to the Viewer.

Part of the functionality is assisted by the AFNI toolset, which makes it possible to merge multiple brain images into one image. This is useful for deriving a brain image for an individual Term for a specific Movie. The selection of these brain images is done by querying through the list if Components, and identifying the Components for the specific Movie and Term. This is saved in an array and fed to the AFNI toolset to merge.

This merging process is done after the decoding process upon upload by the admin. The Nipype Python package serves as an interface to provide Python functions to easily interact with the AFNI toolset.

5. Background task using Celery and Redis

To improve the user experience and efficiency, we have decided to make the decoding funciton a background task. By running the decoding asynchronously, the user (Dr. Skipper) will be able to turn to other pages of INcDb and continue with other tasks, or assign more decoding tasks for INcDb to work on, instead of having to wait on the page while the long-running task is being completed.

For implementing background tasks, the Celery task queue and Redis message broker are used. First of all, a Celery Worker as well as the Redis Server must be running (which we have set to start automatically with the server). A task is set to be run asynchronously by setting the method as a Celery Task in Flask, and calling it with a delay() decoration, which then a message is sent to the Redis server, translated and forwarded to the Celery Worker, and pushing that task onto the task queue.

Then the tasks in the task queue are completed in order by the Celery Worker.

Dependencies

The application is developed with a list of dependecies. The complete list of Python packages dependencies can be found at https://github.com/UCL-CS35/incdb-poc/blob/master/requirements.txt.

We will like to highlight a few key dependencies that were critical for the development of INcDb.

Name Role
Flask The micro-framework that INcDb runs on.
Flask-User Flask extension used to implement User Management feature.
Flask-WTF Flask extension used to simply form creation.
Neurosynth Neurosynth is used to generate analyses that is used for the decoding of the Processed Dataset uploaded by our client.
gunicorn A Python WSGI HTTP Server to serve the Flask application.
celery A task processing system used to manage background tasks like decoding.
Redis Redis is used as message broker to pass messages between Flask application and Celery workers.

Code Structure

The application first-level code structure is presented below and the role of each file or folder is explained.

  • incdb-poc/
    • app/
      • Contains Models, Views (templates), Controllers, Static files (CSS, JS, Images, Fonts), Setup scripts
    • data/
      • Contains Neurosynth Dataset, Decoding results, Admin uploaded Processed Dataset, Generated memmaps, FAQ data
    • default/
    • docs/
      • Documentations generated by Sphinx
    • migrations/ * Migrations genereted (Currently empty)
    • node_modules/
      • Node modules required to run Neurosynth Viewer
    • src/
      • Coffeescript requred to run Neurosnyth Viewer
    • tests/
      • pytests written to test User Managemnent, Search, Creation of Collection
    • uploads/
      • Contains User uploaded Raw Dataset
    • .gitignore
      • Ignore Database and Data folders
    • Aptfile
    • Cakefile
    • Guardfile
    • LICENSE.txt
    • manage.py
      • Use to start the Flask application
    • README.md
    • requirements.txt
      • List of Python packages required
    • runserver.sh
      • Bash script to run the the Flask application in development mode
    • runtests.sh
      • Bash script to run the Test Cases in /tests folder
    • supervsiord.conf
      • Supervisor configuration file used for Deployment

For the complete code, you can view them on GitHub.