baal - Baal 2.0: Experiment API

Quite a large update with a renewed experience for researchers: Experiment API.

Nonetheless, production users will benefit from EPIG, the new SotA in Active Learning!

Experiment API The new experiment object will simplify your experiments by reducing it to a 1-liner.

Experiment API

EPIG Contribution from @reeshipaul @Dref360 and @fbickfordsmith, EPIG is the new state-of-the-art in Bayesian Active Learning. It better estimates the predictive uncertainty than PowerBALD and considers the labeled training set.

EPIG

Stopping Criterion A new object that helps you stop an experiment when reaching your labeling budget or a plateau in uncertainty/performance.

Stopping criterion

Breaking Changes

Arguments in ModelWrapper.*_on_dataset have been for the most part moved to TrainingArguments and are included in ModelWrapper constructor. Visit the documentation for more details.

What's Changed

Add Stopping Criteria for loop by @Dref360 in https://github.com/baal-org/baal/pull/286
Add EPIG by @reeshipaul @Dref360 @fbickfordsmith in https://github.com/baal-org/baal/pull/293
Add Args to ModelWrapper to simplify common API by @Dref360 in https://github.com/baal-org/baal/pull/294
Add documentation for criterion by @Dref360 in https://github.com/baal-org/baal/pull/297
Experiment API v2 by @Dref360 in https://github.com/baal-org/baal/pull/296

Full Changelog: https://github.com/baal-org/baal/compare/v1.9.2...v2.0.0

- Python
Published by Dref360 over 1 year ago

baal - v1.9.2

What's Changed

Allow HF to not replicate in memory by @Dref360 in https://github.com/baal-org/baal/pull/281
Run poetry.lock by @Dref360 in https://github.com/baal-org/baal/pull/282
Fix typo in logging and bump structlog by @Dref360 in https://github.com/baal-org/baal/pull/285

Full Changelog: https://github.com/baal-org/baal/compare/v1.9.1...v1.9.2

- Python
Published by Dref360 almost 2 years ago

baal - v1.9.1 Heuristic Aliases!

Full Changelog: https://github.com/baal-org/baal/compare/v1.9.0...v1.9.1

- Python
Published by Dref360 over 2 years ago

baal - v1.9.0

What's Changed

Handle NER usecase by @Dref360 in https://github.com/baal-org/baal/pull/263
Add warnings when using replicateinmemory=True and MCCachingModule by @Dref360 in https://github.com/baal-org/baal/pull/273
weights=None in vgg models by @arthur-thuy in https://github.com/baal-org/baal/pull/276
fix: no pool preds for random heuristic by @arthur-thuy in https://github.com/baal-org/baal/pull/277

Full Changelog: https://github.com/baal-org/baal/compare/v1.8.0...v1.9.0

- Python
Published by Dref360 over 2 years ago

baal - V1.8.0

New features

MCCachingModule: Cache computation between Monte Carlo iteration link. Special thanks to @pieterblok for introducing this capability!
We can now evaluation every x epochs in ModelWrapper.train_and_test_on_datasets.

What's Changed

Fix doc for NLP example by @Dref360 in https://github.com/baal-org/baal/pull/241
#246 Fix issue where training was not kept consistent by @Dref360 in https://github.com/baal-org/baal/pull/249
Baal in Production Notebook | Classification | NLP | Hugging Face by @nitish1295 in https://github.com/baal-org/baal/pull/245
Add code walkthrough for CIFAR10 by @Dref360 in https://github.com/baal-org/baal/pull/253
Deprecate shuffle prop in favor of PowerSampling by @Dref360 in https://github.com/baal-org/baal/pull/260
docs: use correct link to experiment script by @arthur-thuy in https://github.com/baal-org/baal/pull/267
added setting to skip x epochs when using trainandtestondatasets by @studentWUR in https://github.com/baal-org/baal/pull/258
Add caching mechanism for MCDropout by @Dref360 in https://github.com/baal-org/baal/pull/268
Improve getindicesforactivestep by caching the result according t… by @Dref360 in https://github.com/baal-org/baal/pull/269

New Contributors

@nitish1295 made their first contribution in https://github.com/baal-org/baal/pull/245
@arthur-thuy made their first contribution in https://github.com/baal-org/baal/pull/267
@studentWUR made their first contribution in https://github.com/baal-org/baal/pull/258

Deprecation

We will deprecate shuffle_prop in favor of PowerSampling. Look here to know more about PowerBALD!

Full Changelog: https://github.com/baal-org/baal/compare/v1.7.0...v1.8.0

- Python
Published by Dref360 over 2 years ago

baal - v1.7.0

Large release where we added support for torchmetrics, moved to mkdocs and made the app more robust!

We also updated our visual identity, huge thanks to @Donasolo for the new logos!

What's Changed

Move to Github Actions by @Dref360 in https://github.com/baal-org/baal/pull/212
Added None to act as max for np.clip by @GeorgePearse in https://github.com/baal-org/baal/pull/213
Assign version to pytorch-lightning by @Dref360 in https://github.com/baal-org/baal/pull/226
Update README and CONTRIBUTING by @Dref360 in https://github.com/baal-org/baal/pull/222
Add metric logging by @Dref360 in https://github.com/baal-org/baal/pull/223
Throw away value for custom dataloaders by @bresilla in https://github.com/baal-org/baal/pull/228
#232 Allow for Python <4 by @Dref360 in https://github.com/baal-org/baal/pull/233
#156 Support torchmetrics by @Dref360 in https://github.com/baal-org/baal/pull/230
Add warning when data augmentation is applied on the pool by @Dref360 in https://github.com/baal-org/baal/pull/229
Add structure for mkdocs instead of Sphinx by @Dref360 in https://github.com/baal-org/baal/pull/225

New Contributors

@GeorgePearse made their first contribution in https://github.com/baal-org/baal/pull/213
@bresilla made their first contribution in https://github.com/baal-org/baal/pull/228

Full Changelog: https://github.com/baal-org/baal/compare/v1.6.0...v1.7.0

- Python
Published by Dref360 over 3 years ago

baal - v1.6.0

What's Change

Update faq.md by @Dref360 in https://github.com/ElementAI/baal/pull/203
#205 Add stochastic heuristics from Kirsch et al. by @Dref360 in https://github.com/ElementAI/baal/pull/206

Full Changelog: https://github.com/ElementAI/baal/compare/v1.5.2...v1.6.0

- Python
Published by Dref360 almost 4 years ago

baal - V1.5.2

What's Changed

Better indexing support arrow dataset by @parmidaatg in https://github.com/ElementAI/baal/pull/183
Raises an error instead of a warning when label has no label by @Dref360 in https://github.com/ElementAI/baal/pull/187
#192 Use configure instead of configure_once to remove warnings by @Dref360 in https://github.com/ElementAI/baal/pull/193
#190 Fix MRO for Lightning examples and deprecate said example by @Dref360 in https://github.com/ElementAI/baal/pull/191
Can easily unpatch modules and use as a context manager #198 #194 :

```python mcdropoutmodel = MCDropoutModule(your_model)

this is stochastic

predictions = [mcdropoutmodel(input) for _ in range(ITERATIONS)]

model = mcdropoutmodel.unpatch()

this is deterministic

output = model(input)

Context manager

with MCDropoutModule(your_model) as model: # this is stochastic predictions = [model(input) for _ in range(ITERATIONS)]

this is deterministic

output = model(input) ```

Full Changelog: https://github.com/ElementAI/baal/compare/1.5.1...v1.5.2

- Python
Published by Dref360 almost 4 years ago

baal - v1.5.1

What's Changed

Solve bug where a dataset wouldnt be able to label by @Dref360 in https://github.com/ElementAI/baal/pull/178

Full Changelog: https://github.com/ElementAI/baal/compare/1.5.0...1.5.1

- Python
Published by Dref360 about 4 years ago

baal - v1.5.0

What's Changed

Split API documentation in multiple files by @Dref360 in https://github.com/ElementAI/baal/pull/158
Lightning Flash example by @parmidaatg in https://github.com/ElementAI/baal/pull/154
Add bandit to CircleCI by @Dref360 in https://github.com/ElementAI/baal/pull/164
Add Label function to HFDataset @Dref360 in https://github.com/ElementAI/baal/pull/165
#161 update query size by @Dref360 in https://github.com/ElementAI/baal/pull/166
Same fix in dropconnect and consistent dropout as in dropout by @Dref360 in https://github.com/ElementAI/baal/pull/172
Add lastactivestep iteration to iterate over the last N active steps. by @Dref360 in https://github.com/ElementAI/baal/pull/174

Deprecated

We now deprecate our PL integration in favor of Lightning Flash. More information to come.
We renamed n_data_to_label to query_size to match academic papers notation.

Full Changelog: https://github.com/ElementAI/baal/compare/1.4.0...1.5.0

- Python
Published by Dref360 about 4 years ago

baal - v1.4.0

What's Changed

Support arrowdataset by @parmidaatg in https://github.com/ElementAI/baal/pull/142
Give ability for users to get uncertainty values. by @Dref360 in https://github.com/ElementAI/baal/pull/144
#146 Fix issue where at most a single submodule was affected by Dropout by @Dref360 in https://github.com/ElementAI/baal/pull/147
#131 Use poetry instead of setup.py by @Dref360 in https://github.com/ElementAI/baal/pull/148
#145 Example using MLP on MNIST by @Dref360 in https://github.com/ElementAI/baal/pull/150
mlp regression experiment by @parmidaatg in https://github.com/ElementAI/baal/pull/152
#130 Add mypy and step to test imports by @Dref360 in https://github.com/ElementAI/baal/pull/155

Full Changelog: https://github.com/ElementAI/baal/compare/v1.3.1...1.4.0

- Python
Published by Dref360 over 4 years ago

baal - v1.3.1

Changelog: * Update pytorch-lightning to > 1.3.0 API * Make torchvision and huggingface optional dependencies. * New tutorial on Fairness and how to use Label Studio.

- Python
Published by Dref360 over 4 years ago

baal - v1.3.0

BaaL 1.3.0 is a release focused on UX.

Features

Initial support for HF Trainer along with tutorials to use HuggingFace.
Initial support for Semi-supervised learning, we are eager to see what the community will do with such a powerful tool!
Fixes in ECE computation

Documentation

The biggest change in this release is the new website along with tons of content. 1. Tutorial on Deep Ensembles #94 2. Tutorial on NLP Classification #87 3. Tutorial on visualisation. 4. Added a BaaL cheatsheet to translate equations to code easily. 5. Added a list of "Core papers" to get new users started in Bayesian deep learning.

- Python
Published by Dref360 almost 5 years ago

baal - BaaL v1.2.1

Changelogs

Features * Initial support for ensembles. Example to come. * Initial support for Pytorch Lightning. Example here.

Bugfixes * Fix BALD for binary classification * Fix Random heuristic for generators * Fix to_cuda for strings. * Fix a bug where MCDropconnect would not work with DataParallel

Misc * Warning when no layers are affected by patch_layers in MCDropout, MCDropconnect.

- Python
Published by Dref360 over 5 years ago

baal - v1.2.0

Changelist for v1.2.0

Add DirichletCalibration (Kull et al. 2019), see our blog post.
Add ECE Metrics for computing model's calibration.
Add support for Multi-Input/Output for ModelWrapper
Fix BatchBALD to be consistent with the official implementation
Add ConsistentDropout, where the masks used in MC-Dropout are the same for each input.

Important notes

BaaL is now part of Pytorch Ecosystem!

- Python
Published by Dref360 almost 6 years ago

baal - V1.1.0

BaaL v1.1 release notes

Changelog

Support for MC-Dropconnect (Mobiny, 2019)
ActiveLearningDataset now has better support for attributes specifics to the pool (see below).
More flexible support multi-inputs/outputs in ModelWrapper.
- Can support list of inputs or outputs.
QoL features on ActiveLearningDataset
- Can use a RandomState and add load_state_dict.
Add replicate_in_memory flag to ModelWrapper.
- If False, the MC iterations are done in a for-loop instead of making a batch in memory.
- (This means predict_on_batch would not take up more memory than e.g. test_on_batch)
Add patience and min_epoch_for_es to ModelWrapper.train_and_test_on_datasets.
- Allows early stopping.
New tutorial on how to use BaaL with scikit-learn.
Can now combine heuristics for multi-outputs models (see baal.active.heuristics.Combine).
Fix documentation

New ActiveLearningDataset

To better support new tasks, ActiveLearningDataset can now support any attributes to be overrided when the pool is created.

Example: ```python from PIL import Image from torch.utils.data import Dataset from torchvision.transforms import Compose, ToTensor, RandomHorizontalFlip from baal.active.dataset import ActiveLearningDataset

class MyDataset(Dataset): def init(self): self.my_tansforms = Compose([RandomHorizontalFlip(), ToTensor()])

def __len__(self):
    return 10

def __getitem__(self, idx):
    x = Image.open('an_image.png')
    return self.my_tansforms(x)

aldataset = ActiveLearningDataset(MyDataset(), poolspecifics={ 'my_tansforms': ToTensor() })

Now `pool.my_tansforms = ToTensor()`

pool = al_dataset.pool ```

- Python
Published by Dref360 over 6 years ago

Recent Releases of baal

baal - Baal 2.0: Experiment API

Breaking Changes

What's Changed

baal - v1.9.2

What's Changed

baal - v1.9.1 Heuristic Aliases!

baal - v1.9.0

What's Changed

baal - V1.8.0

New features

What's Changed

New Contributors

Deprecation

baal - v1.7.0

What's Changed

New Contributors

baal - v1.6.0

What's Change

baal - V1.5.2

What's Changed

this is stochastic

this is deterministic

Context manager

this is deterministic

baal - v1.5.1

What's Changed

baal - v1.5.0

What's Changed

Deprecated

baal - v1.4.0

What's Changed

baal - v1.3.1

baal - v1.3.0

Features

Documentation

baal - BaaL v1.2.1

Changelogs

baal - v1.2.0

Changelist for v1.2.0

Important notes

baal - V1.1.0

BaaL v1.1 release notes

Changelog

New ActiveLearningDataset

Now pool.my_tansforms = ToTensor()

Now `pool.my_tansforms = ToTensor()`