Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of oaebu-workflows

oaebu-workflows - v0.6.2

What's Changed

  • Work with Row objects, not strings by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/254

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.6.1...v0.6.2

- Python
Published by keegansmith21 about 1 year ago

oaebu-workflows - v0.6.1

What's Changed

  • Remove partition check by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/253

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.6.0...v0.6.1

- Python
Published by keegansmith21 about 1 year ago

oaebu-workflows - v0.6.0

What's Changed

  • Error on late GB uploads by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/252
  • crossref events removal by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/249

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.5.3...v0.6.0

- Python
Published by keegansmith21 about 1 year ago

oaebu-workflows - v0.5.3

What's Changed

  • Set trigger rule to none failed by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/250
  • Add filter on SubjectCode to avoid nulls in array by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/251

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.5.2...v0.5.3

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.5.2

What's Changed

  • Updated schedule for sftp telescopes to everyday by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/248

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.5.1...v0.5.2

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.5.1

What's Changed

  • Add subtitle_title to country exoprt SELECT by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/247

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.5.0...v0.5.1

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.5.0

What's Changed

  • Onix wf custom schedule by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/244
  • Execution date -> Logical date by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/245
  • Add subtitle to filter by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/246

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.4.3...v0.5.0

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.4.3

What's Changed

  • Fix/metadata filename regex by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/243

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.4.2...v0.4.3

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.4.2

What's Changed

  • Find format from filename, not filepath by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/242

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.4.1...v0.4.2

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.4.1

What's Changed

  • Subtitle and subjects update by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/241

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.4.0...v0.4.1

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.4.0

What's Changed

  • Elevate relation code 13 by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/240
  • UCL Sales Telescope by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/239

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.3.2...v0.4.0

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.3.2

What's Changed

  • SHMP fixes by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/237

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.3.1...v0.3.2

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.3.1

What's Changed

  • Added codecov secret token to actions by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/234
  • Added product elevation to onix telescope by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/235
  • Dataset API and Release updates by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/236

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.3.0...v0.3.1

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.3.0

What's Changed

  • Fixed publisher name in aggregated queries by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/230
  • Added clone protection by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/232
  • Bad 691 include additional metadata in exports by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/233
  • Feature/bespoke data partners by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/231

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.2.4...v0.3.0

- Python
Published by keegansmith21 over 1 year ago

oaebu-workflows - v0.2.4

What's Changed

  • Ignore additions to ONIX schema by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/228
  • Run oapen_metadata transform task on different queue by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/229

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.2.3...v0.2.4

- Python
Published by jdddog almost 2 years ago

oaebu-workflows - v0.2.3

What's Changed

  • Add a toggle to turn Crossref Events off by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/227

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.2.2...v0.2.3

- Python
Published by jdddog almost 2 years ago

oaebu-workflows - v0.2.2

What's Changed

  • Feature/crossref events doi prefix query by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/226

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.2.1...v0.2.2

- Python
Published by jdddog almost 2 years ago

oaebu-workflows - v0.2.1

What's Changed

  • Added exception for different entity format by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/225

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.2.0...v0.2.1

- Python
Published by jdddog almost 2 years ago

oaebu-workflows - v0.2.0

What's Changed

  • Updated JSTOR collections regex by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/224
  • Alternative ONIX snapshot date source by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/223

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.1.4...v0.2.0

- Python
Published by keegansmith21 almost 2 years ago

oaebu-workflows - v0.1.4

What's Changed

  • Fix/irus sa by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/220
  • Renamed pool to comply with standard by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/222

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.1.3...v0.1.4

- Python
Published by keegansmith21 almost 2 years ago

oaebu-workflows - v0.1.3

What's Changed

  • Publisher field by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/217
  • Fix/onix xform file save by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/219
  • Fix/sensor defaulting by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/221

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.1.2...v0.1.3

- Python
Published by keegansmith21 almost 2 years ago

oaebu-workflows - v0.1.2

What's Changed

  • Fix/post astro hotfixes by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/218

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.1.1...v0.1.2

- Python
Published by keegansmith21 almost 2 years ago

oaebu-workflows - v0.1.1

What's Changed

  • Add astronomer-starship Airflow plugin by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/216

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/v0.1.0...v0.1.1

- Python
Published by jdddog almost 2 years ago

oaebu-workflows - v0.1.0

First release for astro workflows

What's Changed

  • Bad 685 thoth rp elevation by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/206
  • BAD-687 by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/208
  • fix to gitbook link by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/209
  • updated author list by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/210
  • Updated collections regex by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/211
  • Added entity_name exception catch by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/213
  • ONIX WF Refactor - Modular Product Tables by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/207
  • astro migration by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/212
  • Remove github actions publish pypi by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/215

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/2023.12.0...v0.1.0

- Python
Published by keegansmith21 almost 2 years ago

oaebu-workflows - 2023.12.0

What's Changed

  • Fix oapen metadata 'invalid date' issue by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/66
  • MEL-399 created new hierarchy by @niamhq in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/65
  • MEL-399 Added Fulcrum content by @niamhq in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/67
  • MEL-399 Added BISAC content by @niamhq in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/69
  • Add DOI badge by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/71
  • Feature/onix schema descriptions by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/72
  • MEL-399 Added overview content by @niamhq in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/73
  • MEL-399 Added Book ISBNs content by @niamhq in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/74
  • Use country.googleanalyticsname for matching with Google Analytics by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/68
  • Mel 773/update oaebu docs by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/70
  • Add fields for google analytics by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/61
  • Mel 399 remaining dashboard content by @niamhq in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/78
  • Mel 399 dashboard content by @niamhq in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/75
  • MEL-399 New FAQ, Known issues and Contact content by @niamhq in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/79
  • Mel 778/update overview and new schema by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/80
  • Update CITATION.cff by @bechandcock in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/77
  • Create .zenodo.json by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/76
  • Author citation missing initial by @bechandcock in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/81
  • Update telescope summaries for readthedocs by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/64
  • Update onix_1900-01-01.json by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/82
  • Update CITATION.cff by @bechandcock in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/84
  • Mel 789/final documentation updates by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/85
  • Update .zenodo.json by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/83
  • MEL-797 included data trust community and update direct link to embed… by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/88
  • Update Google Books schedule interval by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/86
  • INF-384: Use 2nd gen cloud function by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/89
  • Mel 670/dataset release creation by @tuanchien in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/57
  • Fix/api update by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/90
  • Fix oapen metadata dc.date.issued field by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/91
  • Use DatasetTypeId class by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/93
  • Update Oapen IRUS UK by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/92
  • Inf 406/add scripts to seed api db by @tuanchien in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/96
  • Use findfreeport in tests by @tuanchien in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/97
  • Parallelise some onix workflow tasks by @tuanchien in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/99
  • Add dag tags by @tuanchien in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/98
  • BAD-142: ANU google analytics update by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/95
  • Fixed the google analytics schema by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/100
  • Increased lookback time to October 2016 by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/101
  • Bad 214 oapen onix unittest table updates by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/102
  • Change start date of oapenirusuk_telescope by @alexmassen-hane in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/104
  • Feature/kibana spaces by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/106
  • Bad 241 - UCL Discovery data collection - Added ISBN10s by @alexmassen-hane in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/109
  • INF-532: Pre-commit update by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/110
  • INF-540: Updated JSTOR telescope to ignore new fields in the schema by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/111
  • Bad 210 wp2 standardised data ingest pipelines workflows by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/105
  • INF-550: Updated JSTOR transform for new report format by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/114
  • INF-562: JSTOR schedule update by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/115
  • Removed the loading of the OAEBU service account credentials by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/117
  • BAD-308: schema and BQ description updates by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/116
  • Update github unit tests in github workflow/actions by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/120
  • Inf 574 onix schema fix by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/121
  • Bad 257 OAPEN ONIX Integration by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/118
  • BAD-412: Added view creation to onix workflow by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/122
  • OAPEN API seed fix by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/125
  • BAD-372: Thoth Telescope by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/119
  • OAPEN metadata telescope type update by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/126
  • Onix keyword null ammendment by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/127
  • Oapen docs by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/123
  • BAD-314: CEU Press onboarding by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/124
  • INF-591: Updated HTTP requests to use new retry function by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/129
  • Onix workflow's integration with onix telescope alternatives by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/128
  • BAD-490: Removed field selection from crossref metadata uri by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/133
  • Remove helper functions from Thoth telescope by @alexmassen-hane in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/134
  • Added OBP ONIX workflow dataset to seed by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/130
  • Updated filenames in dag to use telescope default by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/131
  • BAD-489: Retry loop for crossref metadata by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/132
  • Fixes incorrect parameter in selecttableshard_dates function by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/138
  • INF-604: Crossref pools for the ONIX workflow by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/137
  • INF-598: ONIX telescope keyword parsing by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/136
  • Oapen metadata structure change and personname field by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/135
  • Handle case: OAPEN Metadata returns only the XML header by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/140
  • Updated telescope documentation index by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/139
  • BAD-276: Fulcrum Telescope Development by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/113
  • Oapen metadata transform bucket consistency by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/141
  • Feature/crossref academic observatory by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/144
  • Changed crossref metadata table load by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/145
  • Feature/workflow config file by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/142
  • BAD-507: Add personname field to Thoth feed by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/143
  • Remove ISBN filter on OA books by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/146
  • Changed partition write disposition to append by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/149
  • Update ONIX parsing and function consolidation by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/147
  • Google books sftp update by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/148
  • Add dags folder by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/151
  • Fixed file saving for empty contents by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/152
  • Fix GA schema find by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/153
  • Fixed the file matching regex by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/154
  • Added the LCCN column to the ONIX schema by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/155
  • fixed jstor write disposition by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/157
  • Fixed the name used for crossref metadata dataset by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/156
  • Fixed GA partition date and added tests by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/159
  • Added fulcrum to data export SQL by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/158
  • Removed additional table load in test by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/161
  • IRUS schedule change by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/162
  • oapen irus uk rename to irus oapen by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/160
  • COK-531/update bad readthedocs by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/163
  • Update of UCL discovery telescope by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/164
  • Rename empty IRUS tables by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/165
  • Fixed irus names by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/168
  • Update for Airflow 2.6.3 by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/167
  • Fix/jstor heading update by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/169
  • Updated oapen metadata to work with additional OAPEN data sources by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/166
  • Remove Elastic mappings by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/171
  • Fixed empty table creation in SQL by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/170
  • Add Github Actions merge_group trigger by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/182
  • Update cloud function version by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/172
  • Moved crossref dataset creation to metadata table task by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/173
  • Added worldreader and internet archive by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/183
  • Updated parser fields for tests by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/184
  • Include published date field by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/185
  • Added InvertedPersonName field by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/186
  • templated ga3 views type by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/188
  • Fix published_date for YYYY entries by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/189
  • Fix issue with OAPEN metadata filter by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/190
  • Bad 638 update readthedocs by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/191
  • Added readthedocs config file by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/193
  • OaebuPartner Class Expansion by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/187
  • Bad 645 rtd amendments by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/195
  • python 3.10 update by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/192
  • BAD-646 updates to formatting after review from Laura. Changes made t… by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/196
  • added CODEOFCONDUCT.md and CONTRIBUTING.md by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/198
  • Related Products Elevation & Normalisation by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/194
  • BAD-657 inserted new text into FAQ as requested by Laura and amended … by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/200
  • Add collections to JSTOR Telescope by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/197
  • NOT NULL statements for crossref and GA by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/202
  • include jstor country_name join for collections by @kathrynnapier in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/204
  • Update irus oapen cloud function version by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/201
  • fix elevation to be optional by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/205
  • File structure change by @keegansmith21 in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/203

New Contributors

  • @niamhq made their first contribution in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/65
  • @kathrynnapier made their first contribution in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/70
  • @bechandcock made their first contribution in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/77
  • @alexmassen-hane made their first contribution in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/104

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/2022.03.0...2023.12.0

- Python
Published by jdddog about 2 years ago

oaebu-workflows - 2022.03.0

What's Changed

  • Update StreamTelescope telescopes by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/25
  • Update Google Books to handle multiple accounts for 1 publisher by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/22
  • Remove 'expand=metadata' param to fix hanging request. by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/27
  • Add data partners and onix telescope fix by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/26
  • INF-166/airflow-2.2 by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/28
  • Fix JSTOR missing reports by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/29
  • Inf 166/add operator by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/30
  • Fix JSTOR failing DAG due to updated report header by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/31
  • changed month metrics filter by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/32
  • MEL-534 updating mapping to cover missing UCL Discovery field by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/33
  • MEL-535 Ensures unmatched isbns in ucl_discovery are exported by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/35
  • MEL-556 added usage flag to book product list by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/34
  • onix aggregation: filter out duplicate isbns by @tuanchien in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/41
  • Fix DOAB telescope by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/39
  • Fix OAPEN Metadata telescope by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/40
  • MEL-569 Changed mapping naming of google books by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/38
  • Mel 557 bisac and thema codes by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/36
  • Mel 553 autocomplete lists by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/42
  • Mel 559 individual author book metrics by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/43
  • changed jinja variable name by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/44
  • Add institution_list mapping by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/45
  • MEL quickfix add subject code by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/46
  • Updated country metrics export by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/47
  • Update OAPEN IRUS UK telescope by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/48
  • Fix JSTOR email filtering by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/49
  • Mel 581/onix workflow use api by @tuanchien in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/51
  • Mel-615 - ONIX workflow readthedocs by @rhosking in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/50
  • Ignore unknown fields by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/52
  • Test fixes after bq quota check feature by @tuanchien in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/53
  • Change OaebuPartner(name=telescope.organisation.name) to to OaebuPartner(name=dataset.name) by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/54
  • Update config instances for elastic import workflow by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/55
  • Change DAG id for public umich export by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/56
  • Fix error in createoaebubookproducttable task. by @aroelo in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/58
  • Inf 339/read the docs windows by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/59
  • Add ONIX telescope documentation by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/62
  • Use new countries table, rather than the two previous tables. by @jdddog in https://github.com/The-Academic-Observatory/oaebu-workflows/pull/63

Full Changelog: https://github.com/The-Academic-Observatory/oaebu-workflows/compare/2021.09.1...2022.03.0

- Python
Published by jdddog almost 4 years ago

oaebu-workflows - 2021.09.1

This release adds the following changes / features: * Add black to precommit config. * Refactor relevant workflows to use OrganisationTelescope. * Use BITSTREAMISBN for Oapen Workflow. * Refactor workflows to use XComless StreamTelescope. * Removed getfile hashfile from workflows.

This release adds the following bug fixes: * Oapen Workflow: fix pendulum type hints, use correct Google Cloud project id, use new functions from observatory platform. * Onix Workflow: fix pendulum type hints, use new functions from observatory platform. * elasticimportworkflow.py: fix configuration.

- Python
Published by jdddog over 4 years ago

oaebu-workflows - 2021.09.0

This release adds the following telescope workflows: * DOAB * Google Analytics * Google Books * JSTOR * OAPEN IRUS UK * OAPEN Metadata * ONIX * UCL Discover

This release adds the following analytic workflows: * OAPEN Workflow * ONIX Workflow

- Python
Published by jdddog over 4 years ago