https://github.com/clariah/wp6-daghregisters
Daghregister in Text-Fabric
Science Score: 13.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.0%) to scientific vocabulary
Repository
Daghregister in Text-Fabric
Basic Info
- Host: GitHub
- Owner: CLARIAH
- License: mit
- Language: Jupyter Notebook
- Default Branch: master
- Size: 11.9 MB
Statistics
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
- Releases: 2
Metadata Files
README.md
Daghregister in Text-Fabric
Source
Lodewijk Petram referred me to the Daghregister, and gave me a link to archive.org from where I have downloaded a volume, apparently as digitized by Google.
Conversion
HOCR is simple HTML in which the results of an OCR process are stored: text, bounding boxes, confidences. All organized by page, area, paragraph, line, word and character.
This is a big file (180 MB) for a source that has 500 pages.
Step 1: reduce
We reduced and simplified the HOCR to a TSV file (72 MB) with identical essential information. The result is much easier viewable in the editor, mainly because the syntax highlighting is not triggered, which helps performance greatly.
Step 2: wordify
We converted the character-based lines of information to word-based lines. Some words that were split by a small space, have been joined again. We averaged the character confidences to word confidences.
Step 3: clean
We removed all front matter pages, plus all words "Digitized by Google". Funnily enough, these strings were OCRed too, and sometimes not entirely correctly. We still found them by using the Levenshtein edit distance.
Also the header lines were removed.
Step 4: make text-fabric
We generated straightforward text-fabric out of it, and decided to loose some of the information. We do not use the area containers. We do not retain the bounding box information.
Step 5: use text-fabric
If you have installed text-fabric (pip install text-fabric),
and if you have cloned this repository to ~/github/CLARIAH/wp6-daghregisters,
then you can give this command on the prompt:
sh
text-fabric CLARIAH/wp6-daghregisters/tf/daghregister/001/0.1:clone --checkout=clone
after which your browser opens with an interface on this volume. You can browse pages and execute queries.
For example, enter this in the search box and click the search icon:
word letters=Oppercoopman
That has 8 results, and you can see them in context. Here is the one on page 25 line 7:
You can also use text-fabric in a jupyter notebook.
A tutorial will follow, but it is not unlike this one for the General Missives.
Author
Owner
- Name: CLARIAH
- Login: CLARIAH
- Kind: organization
- Website: http://www.clariah.nl
- Repositories: 65
- Profile: https://github.com/CLARIAH
CLARIAH offers humanities scholars a Common Lab providing access to large collections of digital resources and innovative tools for research
GitHub Events
Total
Last Year
Committers
Last synced: over 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| Dirk Roorda | d****a@d****l | 22 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: over 1 year ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0