Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of corelli

corelli - Op02n06a

first movement ready for review


Published by github-actions[bot] 10 months ago

corelli - Updated paradata


Published by github-actions[bot] 10 months ago

corelli - Addresses phrase and pedal incongruency warnings

This is a README file for a data repository originating from the DCML corpus initiative and serves as welcome page for both

For information on how to obtain and use the dataset, please refer to this documentation page.

Arcangelo Corelli – Trio Sonatas (A corpus of annotated scores)

This corpus of annotated MuseScore files has been created within the DCML corpus initiative and employs the DCML harmony annotation standard. It was relased together with and as part of the "workflow paper" (for the reference see below or the file CITATION.cff).

The corpus comprises 36 Sonate a tre, divided into 149 separate movements. Together they make up for three of the four famous cycles of 12 trio sonatas each:

| Opus | Cycle | Publication | Included | |------|---------------------|-------------|----------| | 1 | 12 sonate da chiesa | Rome 1681 | Yes | | 2 | 12 sonate da camera | Rome 1685 | No | | 3 | 12 sonate da chiesa | Rome 1689 | Yes | | 4 | 12 sonate da camera | Rome 1694 | Yes |

Getting the data

Data Formats

Each piece in this corpus is represented by five files with identical name prefixes, each in its own folder. For example, the first movement of the first trio sonata has the following files:

  • MS3/op01n01a.mscx: Uncompressed MuseScore 3.6.2 file including the music and annotation labels.
  • notes/op01n01a.notes.tsv: A table of all note heads contained in the score and their relevant features (not each of them represents an onset, some are tied together)
  • measures/op01n01a.measures.tsv: A table with relevant information about the measures in the score.
  • chords/op01n01a.chords.tsv: A table containing layer-wise unique onset positions with the musical markup (such as dynamics, articulation, lyrics, figured bass, etc.).
  • harmonies/op01n01a.harmonies.tsv: A table of the included harmony labels (including cadences and phrases) with their positions in the score.

Each TSV file comes with its own JSON descriptor that describes the meanings and datatypes of the columns ("fields") it contains, follows the Frictionless specification, and can be used to validate and correctly load the described file.

Opening Scores

After navigating to your local copy, you can open the scores in the folder MS3 with the free and open source score editor MuseScore. Please note that the scores have been edited, annotated and tested with MuseScore 3.6.2. MuseScore 4 has since been released which renders them correctly but cannot store them back in the same format.

Opening TSV files in a spreadsheet

Tab-separated value (TSV) files are like Comma-separated value (CSV) files and can be opened with most modern text editors. However, for correctly displaying the columns, you might want to use a spreadsheet or an addon for your favourite text editor. When you use a spreadsheet such as Excel, it might annoy you by interpreting fractions as dates. This can be circumvented by using Data --> From Text/CSV or the free alternative LibreOffice Calc. Other than that, TSV data can be loaded with every modern programming language.

Loading TSV files in Python

Since the TSV files contain null values, lists, fractions, and numbers that are to be treated as strings, you may want to use this code to load any TSV files related to this repository (provided you're doing it in Python). After a quick pip install -U ms3 (requires Python 3.10 or later) you'll be able to load any TSV like this:

```python import ms3

labels = ms3.loadtsv("harmonies/op01n01a.harmonies.tsv") notes = ms3.loadtsv("notes/op01n01a.notes.tsv") ```

Version history

See the GitHub releases.

Questions, Suggestions, Corrections, Bug Reports

Please create an issue and/or feel free to fork and submit pull requests.

Cite as

Hentschel, J., Moss, F. C., Neuwirth, M., & Rohrmeier, M. A. (2021). A semi-automated workflow paradigm for the distributed creation and curation of expert annotations. Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR, 262–269. https://doi.org/10.5281/ZENODO.5624417

License

Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0).


Published by github-actions[bot] 12 months ago

corelli - Updating DCML annotation workflow (v2.4)

What's Changed

  • Updating DCML annotation workflow by @johentsch in https://github.com/DCMLab/corelli/pull/259

Full Changelog: https://github.com/DCMLab/corelli/compare/v2.3...v2.4


Published by johentsch over 1 year ago

corelli - Full extraction with ms3 v2.4.1; updated workflow to v4.3


Published by github-actions[bot] about 2 years ago

corelli - Updated extracted data with ms3 v2.4.0

Apart from updating the metadata, this update also adds

  • the chords facet that includes control events (such as dynamics) and thorough bass figures
  • the annotation workflow v4.1


Published by github-actions[bot] over 2 years ago

corelli - v2.1: Re-extracts all files using ms3 v2.2.1

Changes to the TSV facets

Updated filenames

The TSV files have been renamed and now come with a suffix specifying the data facet that they represent. The three facets are

  • measures
  • notes
  • harmonies

so, for example, the file measures/op01n01a.tsv has been renamed to measures/op01n01a.measures.tsv.

Added resource descriptors

Each TSV file is now accompanied with a JSON resource descriptor following the frictionless specification for Tabular Data Resources. They can and have be used to validate the tabular data.

These files have the same names as the TSV files they describe but replacing the .tsv extension with .resource.json. For example, the file measures/op01n01a.tsv has the corresponding descriptor measures/op01n01a.resource.json.

These metadata files replace the previous csv-metadata.json files (which followed the CSVW standard) that had described the data inadequately.

Added column quarterbeats_all_endings

The quarterbeats column (also known as qstamp) that had already been available does not provide values for first or third endings, expressing the dimensions of a singular playthrough without taking into account any repeats. The newly added column quarterbeats_all_endings does not have any empty values, providing continuous positions as if each measure followed the previous one.

Renamed index column

The first column of metadata.tsv was renamed from fname to piece.

Changes to the reviewed folder

The _reviewed.mscx files compare the current set of annotation labels against those from the previous version (v2.1). They are identical, so no differences are displayed.

Wherever the MuseScore parser throws one or several warnings (e.g. because an annotation label does not match the score very well), these warnings appear in a .warnings file. For example, the warnings for op01n01a can be seen in reviewed/op01n01a.warnings. One goal for future versions of this dataset would be to reduce the number of warnings to the point of addressing all of them.

Frictionless datapackage

This release is the first one that has a frictionless datapackage attached (below). It consists of

  • corelli.zip, a ZIP file containing one TSV file per facet, that corresponds to a concatenation of the TSV files in the respective folder, that is
    • corelli.expanded.tsv
    • corelli.measures.tsv
    • corelli.metadata.tsv (concatenation of a single file)
    • corelli.notes.tsv
  • corelli.datapackage.json, the package descriptor.

If one has the frictionless framework installed, one can use the descriptor to validate the package using the command

bash frictionless validate corelli.datapackage.json


Published by johentsch over 2 years ago

corelli - Arcangelo Corelli - Trio Sonatas (A corpus of annotated scores) (v2.0)

This corpus of annotated MuseScore files has been created within the DCML corpus initiative and employs the DCML harmony annotation standard. It was relased together with and as part of the "workflow paper"

Hentschel, J., Moss, F. C., Neuwirth, M., & Rohrmeier, M. A. (2021). A semi-automated workflow paradigm for the distributed creation and curation of expert annotations. Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR, 262–269. https://doi.org/10.5281/ZENODO.5624417

The corpus comprises 36 Sonate a tre, divided into 149 separate movements. Together they make up for three of the four famous cycles of 12 trio sonatas each:

| Opus | Cycle | Publication | Included | |------|---------------------|-------------|----------| | 1 | 12 sonate da chiesa | Rome 1681 | Yes | | 2 | 12 sonate da camera | Rome 1685 | No | | 3 | 12 sonate da chiesa | Rome 1689 | Yes | | 4 | 12 sonate da camera | Rome 1694 | Yes |

Versions

Version 2.0

  • TSV files now come with the column quarterbeats, which measures in quarter notes each event's position as its distance from the beginning
  • Extracted notes now come with the columns name and octave.
  • Column volta (containing first and second endings) removed from pieces that don't have any.
  • metadata.tsv has been enriched with further columns, in particular information about each movement's dimensions, including dimensions upon unfolding repeats (for instance, last_mn has the number of measures, last_mn_unfolded the number of measures when playing all repeats)
  • The folder reviewed contains two files per movement:
    • A copy of the score where all out-of-label notes have been colored in red; are shown in these files in a diff-like manner (removed in red, added in green).
    • A copy of the harmonies TSV with six added columns that reflect the coloring of out-of-label notes ("coloring reports")
  • As long as the ms3 review has any complaints, it stores them in the file warnings.log. Currently, it is showing those labels where over 60% of the notes in the segment have been colored in red and probably need revisiting ( Pull Requests welcome)
  • TSV files are automatically kept up to date using the new GitHub action dcmlcorpusworkflow which is the successor of the implementation used in the creation of this dataset.

Version 1.1

This release marks the moment where all 149 movements include a reviewed set of annotations that adhere to version 2.3.0 of the DCML harmony annotation standard. The metadata have not been completed yet and the data were extracted one last time with the now deprecated version 0.4.11 of the MuseScore parser ms3 for matters of completeness and homogeneity. The purpose is mainly to substantiate the claim that the "semi-annotated workflow paradigm", as it had been implemented at publication time (see the ISMIR paper cited above), can indeed be put to effective use in the creation of a large dataset. This version is, however, to be followed by a version with upgraded tabular data based on the more mature ms3 > 1.0.0.

Version 1.0

The first release reflects the state of the dataset when finalizing chapter 4 of the workflow paper cited above.


Published by johentsch about 3 years ago

corelli - Arcangelo Corelli - Trio Sonatas (A corpus of annotated scores) (v1.1)

This corpus of annotated MuseScore files has been created within the DCML corpus initiative and employs the DCML harmony annotation standard. It was relased together with and as part of the "workflow paper"

Hentschel, J., Moss, F. C., Neuwirth, M., & Rohrmeier, M. A. (2021). A semi-automated workflow paradigm for the distributed creation and curation of expert annotations. Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR, 262–269. https://doi.org/10.5281/ZENODO.5624417

The corpus comprises 36 Sonate a tre, divided into 149 separate movements. Together they make up for three of the four famous cycles of 12 trio sonatas each:

Opus | Cycle | Publication | Included -- | -- | -- | -- 1 | 12 sonate da chiesa | Rome 1681 | Yes 2 | 12 sonate da camera | Rome 1685 | No 3 | 12 sonate da chiesa | Rome 1689 | Yes 4 | 12 sonate da camera | Rome 1694 | Yes

Version 1.0 reflects the state of the dataset when finalizing chapter 4 of the workflow paper cited above.

Version 1.1 marks the moment where all 149 movements include a reviewed set of annotations that adhere to version 2.3.0 of the DCML harmony annotation standard. The metadata have not been completed yet and the data were extracted one last time with the now deprecated version 0.4.11 of the MuseScore parser ms3 for matters of completeness and homogeneity. The purpose is mainly to substantiate the claim that the "semi-annotated workflow paradigm", as it had been implemented at publication time (see the ISMIR paper cited above), can indeed be put to effective use in the creation of a large dataset. This version is, however, to be followed by a version with upgraded tabular data based on the more mature ms3 > 1.0.0.

This corpus of annotated MuseScore files has been created within the DCML corpus initiative and employs the DCML harmony annotation standard. It was relased together with and as part of the "workflow paper"

Hentschel, J., Moss, F. C., Neuwirth, M., & Rohrmeier, M. A. (2021). A semi-automated workflow paradigm for the distributed creation and curation of expert annotations. Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR, 262–269. https://doi.org/10.5281/ZENODO.5624417

The corpus comprises 36 Sonate a tre, divided into 149 separate movements. Together they make up for three of the four famous cycles of 12 trio sonatas each: Opus Cycle Publication Included 1 12 sonate da chiesa Rome 1681 Yes 2 12 sonate da camera Rome 1685 No 3 12 sonate da chiesa Rome 1689 Yes 4 12 sonate da camera Rome 1694 Yes

Version 1.0 reflects the state of the dataset when finalizing chapter 4 of the workflow paper cited above.

Version 1.1 marks the moment where all 149 movements include a reviewed set of annotations that adhere to version 2.3.0 of the DCML harmony annotation standard. The metadata have not been completed yet and the data were extracted one last time with the now deprecated version 0.4.11 of the MuseScore parser ms3 for matters of completeness and homogeneity. The purpose is mainly to substantiate the claim that the "semi-annotated workflow paradigm", as it had been implemented at publication time (see the ISMIR paper cited above), can indeed be put to effective use in the creation of a large dataset. This version is, however, to be followed by a version with upgraded tabular data based on the more mature ms3 > 1.0.0.


Published by johentsch about 3 years ago

corelli - Arcangelo Corelli - Sonate a tre

This version represents the state of the dataset when finalizing chapter 4 of

Hentschel, J., Moss, F. C., Neuwirth, M., & Rohrmeier, M. A. (2021). A semi-automated workflow paradigm for the distributed creation and curation of expert annotations. Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR, 262–269. https://doi.org/10.5281/ZENODO.5624417


Published by johentsch about 3 years ago