This fork of tryolabs/luminoth accompanies the following paper/preprint: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1009257

---

[

Luminoth is an open source toolkit for computer vision. Currently, we support object detection, but we are aiming for much more. It is built in Python, using TensorFlow and Sonnet.

Read the full documentation here.

DISCLAIMER: Luminoth is still alpha-quality release, which means the internal and external interfaces (such as command line) are very likely to change as the codebase matures.

Installation

Luminoth currently supports Python 2.7 and 3.4–3.6.

Pre-requisites

To use Luminoth, TensorFlow must be installed beforehand. If you want GPU support, you should install the GPU version of TensorFlow with pip install tensorflow-gpu, or else you can use the CPU version using pip install tensorflow.

Installing from source

First, clone the repo on your machine and then install with pip:

bash git clone https://github.com/czbiohub/luminoth-uv-imaging.git cd luminoth python setup.py install

Check that the installation worked

Simply run lumi --help.

Supported models

Currently, we support the following models:

• Object Detection
  • Faster R-CNN
  • SSD

We also provide pre-trained checkpoints for the above models trained on popular datasets such as COCO and Pascal.

We also provide pre-trained checkpoints for the Faster R-CNN trained on RBC cell detection/parasite stage classification are available in release https://github.com/czbiohub/luminoth/releases/tag/v0.4.0 1a0f3002f674 - Faster R-CNN with customized UV microscope dataset to detect healthy/ring/schizont/normal RBC 1fbb5e928fd5 - Faster R-CNN with Leica commercial microscope dataset to detect healthy/ring/schizont/normal RBC

Use lumi predict and the checkpoints above to test on your own dataset with similar 4 or 2 classes as mentioned above

Usage

There is one main command line interface which you can use with the lumi command. Whenever you are confused on how you are supposed to do something just type:

lumi --help or lumi <subcommand> --help

and a list of available options with descriptions will show up.

Working with datasets

See Adapting a dataset.

Training

See Training your own model to learn how to train locally or in Google Cloud.

Visualizing results

We strive to get useful and understandable summary and graph visualizations. We consider them to be essential not only for monitoring (duh!), but for getting a broader understanding of what's going under the hood. The same way it is important for code to be understandable and easy to follow, the computation graph should be as well.

By default summary and graph logs are saved to jobs/ under the current directory. You can use TensorBoard by running:

bash tensorboard --logdir path/to/jobs

Docker

It is recommended that you run luminoth inside a Docker container, especially if you're using shared resources like a GPU server. you can do so: buildoutcfg docker build -t imaging_docker:gpu_py36_cu90 -f Dockerfile . Now you want to start a Docker container from your image, which is the virtual environment you will run your code in. buildoutcfg nvidia-docker run -it -p <your port>:<exposed port> -v <your dir>:/<dirname inside docker> imaging_docker:gpu_py36_cu90 bash Once docker is run from cloned luminoth folder run the below command, make sure you have tensorflow-gpu and not tensorflow python setup.py install This will install luminoth, test it by running lumi --help If you encounter any error with export LANG errors, run them as the error suggests and it should work

If you look in the Dockerfile, you can see that there are two ports exposed, one is typically used for Jupyter (8888) and one for Tensorboard (6006). To be able to view these in your browser, you need map the port with the -p argument. The -v arguments similarly maps directories. You can use multiple -p and -v arguments if you want to map multiple things. The final 'bash' is to signify that you want to run bash (your usual Unix shell).

If you want to launch a Jupyter notebook inside your container, you can do so with the following command: buildoutcfg jupyter notebook --ip=0.0.0.0 --port=8888 --allow-root --no-browser Then you can access your notebooks in your browser at: buildoutcfg http://<your server name (e.g. fry)>:<whatever port you mapped to when starting up docker> You will need to copy/paste the token generated in your Docker container.

Why the name?

The Dark Visor is a Visor upgrade in Metroid Prime 2: Echoes. Designed by the Luminoth during the war, it was used by the Champion of Aether, A-Kul, to penetrate Dark Aether's haze in battle against the Ing.

-- Dark Visor - Wikitroid

Dataset to model to prediction in a few steps

Also if you have image formats that are not compatible with lumi, use imagemagick to convert image from one format to another brew install imagemagick and it installs mogrify in addition to a command called convert mogrify -format png *.tif or for videos use ffmpeg
brew install ffmpeg ffmpeg -pattern_type glob -i '*.jpg' -vf "setpts=5*PTS" test_r5.mp4 ffmpeg -i in.MOV -pix_fmt rgb24 output_tflite.gif ffmpeg -i in.m4v out.mp4 ffmpeg -i out.mp4 -vf format=gray gray.mp4 If you have any trouble please print lumi command --help example lumi predict --help or refer to documentation files in ./docs/tutorial/ 1. Use the first command below, To test if your bounding boxes and objects in csv file and images look right 2. Split data given a csv file containing path to image and bounding boxes with extreme corners of rectangles of objects with labels. The csv file contains xmin, xmax, ymin, ymax, label, imageid. Given that csv file creates 2 folders with train, val and 2 csv files with train.csv and val.csv 3. Third command below converts different images and csv files to tensorflow record files, and depending on percentage paramter creates - train.tfrecords containing that fraction of data 1-fraction of data into val.tfrecords 4. Fourth command, Train using the config, example config is examples folder, change and copy the dictionary from baseconfig.yml if any other parameters should be changed for your training, dataset and it your config.yml 5. Fifth command below, Evaluate training data - prints precision, losses for each class on training data 6. Sixth command below, Evaluate validation - data prints precision, losses for each class on validation data 7. Checkpoint creation, Create checkpoint to save the model so far created 8. Export the checkpoint to a .tar folder of same name as checkpoint so if you lose it or the docker exits you can import it using lumi checkpoint import cb0e5d92a854.tar 9. Predict on a dataset given a checkpoint creates a predval.csv file and boundary boxes overlaid images in a folder as well 10. Print confusion matrix and confusion matrix png and other metrics comparing ground truth csv file and prediction csv file ``` bash lumi overlaybbs --imdir "images" --csvpath "bblabels.csv" --outputdir overlaidmosaiccells/ --inputimageformat tif lumi splittrainval annotatedboundingboxes.csv annotatedboundingboxes1.csv annotatedboundingboxes2.csv --outputdir lumicsv --percentage 0.9 --randomseed 42 --inputimageformat .tif lumi dataset transform --type csv --data-dir /lumicsv/ --output-dir /tfdata/ --split train --split val --only-classes=table lumi train -c config.yml lumi eval --split train -c config.yml --no-watch lumi eval --split val -c config.yml --no-watch lumi checkpoint create config.yml -e name='Faster RCNN' -e alias=cnntrial lumi checkpoint export cb0e5d92a854 lumi predict val/ -d predsval/ --checkpoint trial -f predval.csv lumi confusionmatrix --groundtruthcsv val.csv --predictedcsv predval.csv --outputtxt output.txt --classes_json classes.json ```

License

Copyright © 2018, Tryolabs. Released under the BSD 3-Clause.