EtymDB 2.1

EtymDB 2.1 : An etymological database extracted from the Wiktionary (described in Methodological Aspects of Developing and Managing an Etymological Lexical Resource: Introducing EtymDB-2.0).

Previous versions available here. Logo upgraded by Alix Chagu.

Organisation of the repo (and the base)

• data

  • etymdb.csv is the raw extracted DB csv file
    • Extracted from wiktionary.xml, itself extracted from enwiktionary-latest-pages-articles.xml - neither have been added to the repo because of their size, if you need them, please contact the repo owner
  • split_etymdb contains the extracted database, separated in several files for easier data analysis
    • etymdb_values: Word ix, Lang identifier (in wiki code), Lexeme, Gloss (English translation)
      
    • etymdb_links_info: Direct relation type, child word ix, parent word ix
      • If the parent index is negative (usually for derivation or compounding relations), it means that several parents are implied: the negative index will be found in etymdb_links_index, in association with the several parents indices
    • etymdb_links_index: Multiple parents relation ix, parent 1 ix, parent 2 ix, ... parent n ix

• extraction_scripts contains all the scripts used for data extraction, included for reproducibility

• analysis_notebooks contains 2 Jupyter notebooks to help you get a quick start with the database. One is the reproduction of part 7 of the paper

• static contains the logos

Data extraction

You can reproduce all steps of data extraction by using the following commands on your data dump of interest.

Extract your data dump

Download and extract the xml data dump that you want to use, and put it in data/.

tar -xvjf enwiktionary-date-pages-articles.xml.bz2 mv enwiktionary-date-pages-articles.xml data/

From xml to csv

Then, from the script folder.

cat ../data/enwiktionary-date-pages-articles.xml | perl enwiktionary2xml.pl > ../data/enwiktionary.xml cat ../data/enwiktionary.xml | perl etymology_analyser.pl > ../data/enwiktionary.csv

From csv to split csv

From the data folder.

```

Get only links_info

awk '$1 ~ /^-/'etymdb.csv > splitetymdb/etymdblinks_index.csv

Get no links info

awk '$1 !~ /^-/'etymdb.csv > splitetymdb/etymdbnotlinksindex.csv

Get only lexeme info

awk 'NF > 3 { print $0 }' splitetymdb/etymdbnotlinksindex.csv > splitetymdb/etymdbvalues.csv

Get values info

awk 'NF == 3 { print $0 }' splitetymdb/etymdbnotlinksindex.csv > splitetymdb/etymdblinks_info.csv ```

Citation

@inproceedings{fourrier-sagot-2020-methodological, title = "Methodological Aspects of Developing and Managing an Etymological Lexical Resource: Introducing {E}tym{DB}-2.0", author = "Fourrier, Cl{\'e}mentine and Sagot, Beno{\^\i}t", booktitle = "Proceedings of the 12th Language Resources and Evaluation Conference", month = may, year = "2020", address = "Marseille, France", publisher = "European Language Resources Association", url = "https://aclanthology.org/2020.lrec-1.392", pages = "3207--3216", ISBN = "979-10-95546-34-4", }