Science Score: 26.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (14.3%) to scientific vocabulary
Repository
Aim-spaCy integration
Basic Info
Statistics
- Stars: 34
- Watchers: 5
- Forks: 9
- Open Issues: 2
- Releases: 1
Metadata Files
README.md
Aim-spaCy
Aim-spaCy is an Aim-based spaCy experiment tracker. Its mission is to help AI researchers compare their spaCy metadata dramatically faster at scale.
About
When running spaCy for training and inference a variety of metadata gets generated (hyperparams, metrics, visualizations etc). The runs/inferences need to be compared to do the next iterations quickly. Aim is the most advanced open-source self-hosted AI experiment / metadata comparison tool.
Aim-spaCy helps to easily collect, store and explore training logs for spaCy, including: - metrics - hyper-parameters - displaCy visualizations
More about Aim: https://github.com/aimhubio/aim
More about spaCy: https://github.com/explosion/spaCy
Examples
- A quick example on how to track metrics, hparams and displaCy outputs with AimLogger
- https://github.com/aimhubio/aim-spacy/tree/master/examples/training_demo
- Example of tracking displaCy visualizations with AimDisplaCy
- https://github.com/aimhubio/aim-spacy/tree/master/examples/imagetrackingdemo
Getting Started
1. Installation
In order to use aim for experimentation tracking, you must install it onto your system.
pip install aim-spacy
2. Tracking metrics, hparams and displacy visualizations
1. Tracking with AimLogger
`spacy.AimLogger.v1` allows the user to leverage [`Aim`](https://aimstack.io/) as the experiment tracker throughout the model development cycles. All of the training metrics will be tracked by the `Aim` through the training steps and can be easily accessed through Aim UI. It is important to note that you can observe the changes in the metrics live throughout the training. Aim also supports tracking multiple experiments simultaneously. Aim will store all of the [training config](https://spacy.io/usage/training#config) hyperparameters that are used during experimentation along with [system-related information](https://aimstack.readthedocs.io/en/latest/ui/pages/run_management.html#id7) ranging from GPU/CPU usability to Disk IO. This comes in with an added benefit that you can search/filter across the experimentation rather granularly, using our pythonic search [AimQL](https://aimstack.readthedocs.io/en/latest/using/search.html?highlight=AimQL#searching), live from the UI. To access the Aim Logger one can simply add a config akin to this Example config: ```ini [training.logger] @loggers = "spacy.AimLogger.v1" repo = "path/to/save/logs" experiment_name = "your_experiment_name" ``` The complete overview of Aim Logger inputs looks like this. | Name | Type | Description| | ---------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `repo` | `str` | The path for saving the logs | | `experiment_name` | `str` | The name of the experiment to track(default: `None`, the experiment will be determined by hash). | | `viz_path` | `str` | The path of the dataset (in a .spacy format) that will be used for saving the visualisations during training. Usually, this is taken as the dev set (dev.spacy) For a standalone run, this option is not needed (default: `None`). | | `model_log_interval` | `int` |How often should the model log the experimentation results | | `image_size` | `str` | A string with a ',' separation designating the height and width of the image that is going to be saved during the training | | `experiment_type` | `str` | This flag designates what kind of an experiment is currently being tracked. If 'ner' or 'dep' is specified then the tracker will also use the `viz_path` and `image_size` for tracking displacy outputs during training. For a standalone run, this option is not needed (default: `None`). ||2. Tracking in any python script
Track metrics and hparams ```py from aim import Run run = Run() # Save inputs, hparams or any other `key: value` pairs run['hparams'] = { 'learning_rate': 0.001, 'batch_size': 32, } # ... for step in range(10): # Log metrics to visualize performance run.track(step, name='metric_name') # ... ``` Track displacy visulizations ```py from aim_displacy import AimDisplaCy from aim import Run aim_run = Run() aim_displacy = AimDisplaCy(image_size=(600, 200)) for index, text in tqdm(enumerate(data), total=len(data)): doc = nlp(text) aim_run.track( aim_displacy(doc, style='dep', caption=f'Dependecy for data: {index}'), name='Parsing', context={'type': 'dependency'} ) aim_run.track( aim_displacy(doc, style='ent', caption=f'Entity for data: {index}'), name='Parsing', context={'type': 'entity'} ) ```3. Browsing results with Aim UI
Execute the below command to run Aim UI:
shell
aim up
You should see the following output meaning Aim UI is up and running:
Running Aim UI on repo `<Repo#-5930451821203570655 path=/.aim read_only=None>`
Open http://127.0.0.1:43800
Press Ctrl+C to exit
Open your browser and navigate to http://127.0.0.1:43800
You should be able to see the home page of Aim UI!
Using the full scale of aim-spacy integration it is possible to aggregate/group and filter your experiments with various levels of granularity
You can also filter using all the parametrs with Aims pythonic search engine AimQL
Tracking NER's and dependecies during training is also integrated within the pipeline
Grouping images along various axis is also entirely possible
Owner
- Name: Aim
- Login: aimhubio
- Kind: organization
- Location: Berkeley, California
- Website: https://aimstack.io
- Twitter: aimstackio
- Repositories: 10
- Profile: https://github.com/aimhubio
AI Development Environment
GitHub Events
Total
- Watch event: 2
Last Year
- Watch event: 2
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 1
- Total pull requests: 5
- Average time to close issues: N/A
- Average time to close pull requests: 24 days
- Total issue authors: 1
- Total pull request authors: 4
- Average comments per issue: 1.0
- Average comments per pull request: 1.0
- Merged pull requests: 3
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- ceesroele (1)
Pull Request Authors
- osoblanco (2)
- alberttorosyan (1)
- cceyda (1)
- gorarakelyan (1)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- aim *
- html2image *
- lxml *
- pillow *
- reportlab *
- spacy *
- svglib *
- tqdm *