Skip to content

For Developers

Repository Organization

  • Library and Application code in the /neuroport_dbs folder
  • Unit tests in the /tests folder. Note this uses the pytest framework and conventions.
  • Documentation in the /docs folder

Maintaining the Documentation

You will need to install several Python packages to maintain the documentation.

  • pip install mkdocs mkdocstrings mknotebooks mkdocs-material Pygments

The /docs/{top-level-section} folders contain a mix of .md and .ipynb documentation. The latter are converted to .md by the mknotebooks plugin during building.

Run mkdocs gh-deploy to build the documentation, commit to the gh-deploy branch, and push to GitHub. This will make the documentation available at https://SachsLab.github.io/NeuroportDBS/

Here is a great cheat sheet for mkdocs and mkdocstrings syntax.

Autogenerated Documentation

The /docs/neuroport_dbs folder can hold stubs to tell the mkdocstrings plugin to build the API documentation from the docstrings in the library code itself. Currently this is empty. If any stubs are added then it's necessary to build the documentation from a Python environment that has the package installed. A stub takes the form

# Title
::: neuroport_dbs.module.name

Testing the documentation locally

  • mkdocs serve

If you build the docs locally then you'll also get the /site directory, but this should be git ignored.

Running the unit tests

TODO

Maintaining the Zip Distribution

  • Download the latest WinPython release.
    • These instructions were tested with Winpython64-3.8.5.0
  • Run the WinPython self-extracting executable. This will create a folder containing a full Python distribution with many useful packages installed (see full list here).
  • Edit the WPy64-3850\settings\winpython.ini file and add the following line: PATH = %WINPYDIR%\Lib\site-packages\PyQt5\Qt\bin;%PATH%
  • Download MySQL Windows ZIP Archive
    • Tested with mysql-8.0.2.1-win64.zip
  • Next to the WinPython folder, extract the mysql zip and rename the extracted folder to mysql
  • In the WinPython folder, run "WinPython Command Prompt". This will open a Command Prompt with all the paths configured to use this new Python distribution.
  • Install all of the Python packages listed in the table below.
    • Version numbers may not be important. Please try the latest version and report to us if it does not work.
    • The method to install the packages isn't important. If you're on an internet-connected computer then you can use the pip commands. Otherwise you can first download the wheels then bring them to the development computer to pip install the wheels.
    • If you wish to be able to modify any of the SachsLab packages that are pure python (mspacman, cerebuswrapper, serf, neuroport_dbs) then you may do so by first cloning the repository to get the source and installing the package in-place: Using the WinPython command prompt, run pip install -e . from within the cloned directory.
    • The cerebus package may complain "DLL load failed". This happens when cerebus.cbpy can't find Qt5 or it finds the wrong version. This SHOULD be fixed by editing the PATH in the 3rd step above, but I also found it necessary to copy Qt5Core.dll and Qt5Xml.dll from the above path directly into the site-packages\cerebus folder. We hope to remove the qt dependency from cerebus to avoid this in the future.
  • In the command prompt, cd into the bin subfolder of the unzipped mysql folder.
  • Create a mysql\data folder along with the base databases: mysqld --initialize-insecure --console
    • You can change the default data directory, username, and password. See the section below "Configuring MySQL Database Server"
  • Double-click on the mysqld executable in the bin folder.
  • Back in the command prompt, run mysqladmin --user=root create serf
  • Install the serf databases with the following commands: 
    serf-makemigrations
serf-migrate
  • Make a batch file WPy64-3850\scripts\NeuroportDBS.bat with the following contents: shell script @echo off call "%~dp0env_for_icons.bat" start "" "%WINPYDIR%\Scripts\dbs-sweep.exe" /command:%1 /B start "" "%WINPYDIR%\Scripts\dbs-raster.exe" /command:%1 /B start "" "%WINPYDIR%\Scripts\dbs-waveform.exe" /command:%1 /B start "" "%WINPYDIR%\Scripts\dbs-ddu.exe" /command:%1 /B start "" "%WINPYDIR%\Scripts\dbs-features.exe" /command:%1 /B
  • Jump ahead to Usage Instructions below.

Required Python Packages

Package Version Wheel pip command
pyFFTW 0.12.0 Link
mysqlclient 2.0.1 Link
Django 3.1 Link
quantities 0.12.4
python-neo 0.9.0 pip install git+https://github.com/NeuralEnsemble/python-neo.git
pylsl 1.13.6 Link
pytf 0.1 Link pip install git+https://github.com/SachsLab/pytf.git
mspacman 0.1 Link pip install git+https://github.com/SachsLab/mspacman.git
cerebus 0.0.4 Link N/A - must use wheel
cerebuswrapper 0.1 Link pip install git+https://github.com/SachsLab/cerebuswrapper.git
serf 1.1 Link pip install git+https://github.com/cboulay/SERF.git#subdirectory=python
neurport_dbs 1.0 Link pip install git+https://github.com/SachsLab/NeuroportDBS.git

Configuring MySQL Database Server

  • If you wish to use a different datadir then you must first create a my.cnf file in the root mysql folder with the following contents (commented out lines aren't necessary, just keeping them here for reference): 
    [mysqld]
datadir=path/to/data
#port = 3306
#socket = /tmp/mysql.sock
#pid-file = /Volumes/STORE/eerfdata/Chadwicks-MacBook-Pro.local.pid
#default-storage-engine = MyISAM
#default_tmp_storage_engine = MyISAM
#query_cache_type = 1
#key_buffer_size = 2G
#query_cache_limit = 400M
  • If you wish to secure the database then you'll need to give the root account a password. Do so with mysql_secure_installation.
  • If you change from the default username (root) and password (none) then you will have to tell serf what the username and password are. Create a file named my_serf.cnf and put it in the path identified by the following command: python -c "import os; print(os.path.expanduser('~'))" The file contents should be 
    [client]
user = root
password = {password}
#port = 3306
#socket = /tmp/mysql.sock

For experts who want to use their existing environment

We assume you know how to work with conda environments and that you have a MySQL database server running and configured to your liking.

  • Install the Python packages from the table above.
  • Adapt the instructions at Segmented Electrophys Recordings and Features Database (SERF) to prepare the database server for these tools.
  • If you have a hybrid distribution/system-MySQL environment (i.e., your name is Guillaume) then you may also wish to use some of the MySQL DB config settings from above.