Recent Releases of nederlab-pipeline
nederlab-pipeline - v0.9.11
Bugfix release and added extra check to guard against a certain corruption in the metadata database
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.9
Further fixes in DBNL fix pipeline
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.8
Incorporated extra validation step in fix pipeline
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.7
Minor update for dbnl fix pipeline: Reverted unnecessary patch from previous release and ensure output files can't overwrite input
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.6
Minor update for dbnl fix pipeline: added sync in an attempt to tackle an elusive issue
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.5
Bugfix release in fix pipeline: Unassign unused metadata
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.4
Bugfix release for fix pipeline: Acts can also be independent chapters
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.3
Minor release update; better gz compression in fix pipeline
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.2
Further bugfix release for dbnl fix pipeline
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.1
bugfix release for dbnl fix pipeline
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.9.0
Recent changes:
- Implemented a script that fixes the DBNL FoLiA v2 documents as delivered in 2019. This scripts fixes the IDs and adds the necessary (sub)metadata. Discussed in Jira ticket: https://jira.socialhistoryservices.org/browse/TT-709
Older changes (2019):
- enable ignore option for wikiente
- implement support for language constrain in modernisation
- added resources (migrated from inl/nederlab-linguistic-enrichment)
- only do language identification on sentences!
- simplifying the pipeline, do not run frog in batches anymore but one frog per file (at cost of init time and extra memory, but easier to handle potential errors)
- replacing folialangid with colibri-lang, use --subcodes for colibri-lang
- do language detection before tokenization
- Groovy
Published by proycon over 5 years ago
nederlab-pipeline - v0.8.0
This is a first release after the split from PICCL. Introduces a renewed Nederlab pipeline consisting of the following components (most can be enabled/disabled when needed):
* Conversion from TEI to FoLiA through tei2folia (part of FoliA tools, this replaces the old scripts by Erik Tjong Kim Sang.
* FoLiA v2 compatible, stricter and explicit validation and explicit provenance information.
* Tokenisation through ucto
* Language detection using folialangid (part of FoliA tools)
* Linguistic enrichment through Frog (part-of-speech tagging, lemmatisation, named entity recognition)
* Named Entity Linking (after another recognition stage) is handled by wikiente, which communicates with the DBPedia Spotlight service, and replaces the older foliaentity tool by Erwin Komen.
- Groovy
Published by proycon about 7 years ago