Recent Releases of https://github.com/clariah/wp6-missieven

https://github.com/clariah/wp6-missieven - With entity and ent nodes

Now the entity occurrences are represented as ent nodes and these nodes have the features eid and kind for entity ID and entity kind. There are also entity nodes that collect entity occurrences with the same eid and kind. The edge feature eoccs links entity nodes to their occurrences, the ent nodes.

So, multiword entity occurrences now corresponds to a single ent node, linked to the words the entity occupies.

The ent and entity nodes are added to the original dataset. The version of the dataset is still 1.0e.

Note that most tutorials work with version 1.0, but not version 1.0e.

If you need to work with earlier versions of the missieven, specify the version in the use command, like so:

A = use("CLARIAH/wp6-missieven", version="1.0")

This works best if you have installed Text-Fabric as

pip install --upgrade 'text-fabric[all]'

because then TF can use the GitHub API to fetch the data.

If you only work with the latest version (1.0e) this is not needed.

- Jupyter Notebook
Published by dirkroorda over 2 years ago

https://github.com/clariah/wp6-missieven - With entities as nodes

Now the entities are represented as nodes and these nodes have the features eid and kind for entity ID and entity kind. So, a multiword entity occurrences now corresponds to a single entity node, linked to the words the entity occupies.

The entity nodes are added to the original dataset. The new version of the dataset is 1.0e.

Not that all tutorials work with version 1.0, but not version 1.0e.

If you need to work with earlier versions of the missieven, specify the version in the use command, like so:

A = use("CLARIAH/wp6-missieven", version="1.0")

This works best if you have installed Text-Fabric as

pip install --upgrade 'text-fabric[all]'

because then TF can use the GitHub API to fetch the data.

If you only work with the latest version (1.0e) this is not needed.

- Jupyter Notebook
Published by dirkroorda over 2 years ago

https://github.com/clariah/wp6-missieven - With a new entities export

Metadata added for harvesting by CLARIAH.

New version of entity annotations by Sophie Arnoult.

Note on the attachments:

No need to download them manually. They will be fetched by Text-Fabric when needed.

  • tf: the main corpus
  • voc-missives-export entity annotations as produced in cltl/voc-missives
  • voc-missives-migrated entity annotations as migrated from an earlier version in cltl/voc-missives
  • exercises-entities: results of toy example of creating entity annotations
  • exercises-numerics: results of toy example of creating other annotations

- Jupyter Notebook
Published by dirkroorda almost 4 years ago

https://github.com/clariah/wp6-missieven - Includes volume 14

Volume 14 was not included so far. It has two bands. We converted this material from a textual pdf, produced the same kind of xml as for volumes 1-13, and generated TF from the result.

- Jupyter Notebook
Published by dirkroorda about 4 years ago

https://github.com/clariah/wp6-missieven - All letters in a page, space corrections

All letters now are part of a page, also the letters that do not have a <pb> element in their text. The space corrections by Sophie have been applied.

- Jupyter Notebook
Published by dirkroorda almost 5 years ago

https://github.com/clariah/wp6-missieven - Spaces

Added spaces to feature punc and friends on the basis of a correction set by Sophie Arnoult.

- Jupyter Notebook
Published by dirkroorda about 5 years ago

https://github.com/clariah/wp6-missieven - Fixed words outside lines

When multiple letters occur on a single page, the non-first letters on such pages end up with the words on the first line not wrapped in a line node. This hinders a space-optimization in the layered search app. Corrected.

- Jupyter Notebook
Published by dirkroorda about 5 years ago

https://github.com/clariah/wp6-missieven - New data version 0.7

Data version 0.7 has a different treatment of footnotes. Before, the footnote bodies were mere feature values. Now they occupy slots and lines themselves.

- Jupyter Notebook
Published by dirkroorda over 5 years ago

https://github.com/clariah/wp6-missieven -

Dataversion 0.6.

Small fixes in folio references.

There is also a simple data export of all words plus basis information. You can use this as input for natural language tools and named entity recognition. The data can be used to run this tools on orginal words and editorial words separately.

See export notebook for a detailed description and the way it is generated.

- Jupyter Notebook
Published by dirkroorda over 5 years ago

https://github.com/clariah/wp6-missieven - Data version 0.5

Fixed the generation of spurious newlines in footnote bodies

- Jupyter Notebook
Published by dirkroorda over 5 years ago

https://github.com/clariah/wp6-missieven - Data version 0.4

All footnotes resolve. Rather good dataset, still full of OCR mistakes, but structurally clean.

- Jupyter Notebook
Published by dirkroorda over 5 years ago

https://github.com/clariah/wp6-missieven - New data release

Footnote bodies are almost all checked and corrected (12247 in total), footnote marks have been checked en corrected for volumes 1-4, there remain at least (300) pages with unlinked footnotes out of the 5270 pages that have footnotes. Editorial text is now in the main text, on equal footing with the original letter content, but separable from it in a number of ways.

- Jupyter Notebook
Published by dirkroorda over 5 years ago

https://github.com/clariah/wp6-missieven - Decent conversion to tf

The source is messy. Lots of things have been detected and corrected, but by no means all issues got treated.

Still outstanding:

  • correcting footnote marks so that the proper footnote bodies can be linked to the proper marks
  • tables have bad TEI encodings, especially tables in landscape mode. Better OCR is available and can be manually encoded
  • words and punctuation must be separated
  • etc.

- Jupyter Notebook
Published by dirkroorda over 5 years ago

https://github.com/clariah/wp6-missieven - Initial commit

Everything set up, preliminary docs, provenance, acknowledgements, but not yet code.

- Jupyter Notebook
Published by dirkroorda almost 6 years ago