noisy-sentences-dataset
550K sentences in 5 European languages augmented with noise for training and evaluating spell correction tools or machine learning models.
Science Score: 67.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 1 DOI reference(s) in README -
✓Academic publication links
Links to: zenodo.org -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.2%) to scientific vocabulary
Keywords
Repository
550K sentences in 5 European languages augmented with noise for training and evaluating spell correction tools or machine learning models.
Statistics
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
- Releases: 1
Topics
Metadata Files
README.md
Noisy Sentences Dataset
550K sentences in 5 European languages augmented with noise for training and evaluating spell correction tools or machine learning models. We have constructed our dataset to cover representatives from the language families used across Europe. - Germanic - English, German; - Romance - French; - Slavic - Bulgarian; - Turkic - Turkish;
Use case example: Apply language models or other techniques to compare the sentence pairs and reconstruct the original sentences from the augmented ones. You can use a single multilingual solution to solve the challenge or employ multiple models/techniques for the separate languages. Per-word dictionary lookup is also an option.
Files
- train.csv - the training set
- test.csv - the test set
Because of an ongoing Kaggle competition, the testing set labels will be released in March 2023.
Columns
-
Id- unique id for each sentence pair -
Language- one ofen(English),bg(Bulgarian),tr(Turkish),fr(French),de(German) -
Text- noisy text -
Expected- original sentence before augmentation
Cite
bibtex
@software{Cholakov_Noisy_Sentences_Dataset_2023,
author = {Cholakov, Radostin},
doi = {10.5281/zenodo.7602333},
month = {2},
title = {{Noisy Sentences Dataset}},
url = {https://github.com/radi-cho/noisy-sentences-dataset},
version = {0.0.1},
year = {2023}
}
Credits
The sentences in our dataset are sourced from Wikipedia. The contents of Wikipedia are licensed under the Creative Commons 3.0 License.
Owner
- Name: Radi Cho
- Login: radi-cho
- Kind: user
- Location: Plovdiv, Bulgaria
- Company: Stanford University
- Website: https://www.linkedin.com/in/radostin-cholakov-bb4422146/
- Twitter: radi_cho
- Repositories: 62
- Profile: https://github.com/radi-cho
Student, 19. Machine Learning Research.
Citation (CITATION.cff)
cff-version: 1.2.0 message: "If you use this dataset, please cite it as below." authors: - family-names: "Cholakov" given-names: "Radostin" orcid: "https://orcid.org/0000-0002-5460-8252" title: "Noisy Sentences Dataset" version: 0.0.1 doi: 10.5281/zenodo.7602333 date-released: 2023-02-03 url: "https://github.com/radi-cho/noisy-sentences-dataset"
GitHub Events
Total
Last Year
Issues and Pull Requests
Last synced: 11 months ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0