Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of academic-observatory-workflows

academic-observatory-workflows - v0.1.7

What's Changed

  • Updates to Unpaywall docstrings by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/258
  • Docs update by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/259
  • Use ROR API v2 by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/260

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/v0.1.6...v0.1.7

- Python
Published by keegansmith21 6 months ago

academic-observatory-workflows - v0.1.6

What's Changed

  • Retry and use user agent for wikipedia by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/257

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/v0.1.5...v0.1.6

- Python
Published by keegansmith21 7 months ago

academic-observatory-workflows - v0.1.5

What's Changed

  • Fix/unpaywall schema authors by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/256

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/v0.1.4...v0.1.5

- Python
Published by keegansmith21 7 months ago

academic-observatory-workflows - v0.1.4

What's Changed

  • pubmed branching logic fix by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/254
  • Unpaywall schema update by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/255

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/v0.1.3...v0.1.4

- Python
Published by keegansmith21 8 months ago

academic-observatory-workflows - v0.1.3

What's Changed

  • Unpaywall file download check by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/253

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/v0.1.2...v0.1.3

- Python
Published by keegansmith21 8 months ago

academic-observatory-workflows - v0.1.2

What's Changed

  • Automated build and push image by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/250
  • secret name change by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/251
  • remove main branch push by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/252

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/v0.1.1...v0.1.2

- Python
Published by keegansmith21 8 months ago

academic-observatory-workflows - v0.1.1

What's Changed

  • readthedocs build update by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/247
  • Increase orcid threads in test by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/246
  • unpaywall api updates by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/248
  • Pubmed unstring nulls by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/249

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/v0.1.0...v0.1.1

- Python
Published by keegansmith21 8 months ago

academic-observatory-workflows - v0.1.0

What's Changed

  • Zenodo API changes by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/195
  • Optimise OpenAlex table loading by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/205
  • Add more countries and institutions to OA Dashboard by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/206
  • Fix/PubMed: No updatefiles found in release period by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/207
  • Inf 694 contributions update by @kathrynnapier in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/210
  • Fix/OpenAlex: Account for new top level field when merging generated schemas by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/209
  • Enable files larger than 100 Mb to be uploaded to Zenodo by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/208
  • Add upper trim for unpaywall join by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/212
  • update CIC to CIDS by @kathrynnapier in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/214
  • Increase OpenAlex upsert byte limit by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/213
  • Restructure workflow layout by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/215
  • Fix fetch wikipedia descriptions by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/217
  • Refactor Airflow DAGs to use Airflow Taskflow API by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/216
  • More specific ROR data file regex by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/219
  • OpenAlex schema for 2024-04-25 snapshot by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/220
  • Fix/workflow bugs by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/223
  • Skip cases where a relationship org is a both parent and a child by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/225
  • Fix OA Dashboard divide by zero null error by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/226
  • Remove execution_delta from PreviousDagRunSensor by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/227
  • Update nltk and use punkt_tab by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/230
  • Add field 'affiliations' to OpenAlex ingest by @bmkramer in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/231
  • Re-order new affiliations field in works table schema by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/232
  • Add Make Data Count Data Citation Corpus schema and instructions by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/229
  • Add DataCite schema and transformation script by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/228
  • DataCite Public Data File 2024 by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/235
  • Feature/astro2 by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/224
  • Astro Deployment by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/238
  • Upload task merge by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/239
  • Remaining Airflow Astro workflow fixes by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/241
  • Deploy to astro by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/244
  • removed fields as per unpaywall docs by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/243

New Contributors

  • @bmkramer made their first contribution in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/231

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/2023.12.0...v0.1.0

- Python
Published by keegansmith21 8 months ago

academic-observatory-workflows - 2023.12.0

What's Changed

  • Update file regex by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/94
  • Save JSON index for the filtering API and COKI Open Access Dataset release by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/89
  • Add DOI badge by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/96
  • Fix/country table schema by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/95
  • Create CITATION.cff by @bechandcock in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/98
  • Create .zenodo.json by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/97
  • Added Kathryn's middle initial by @bechandcock in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/99
  • Update telescope summaries for readthedocs by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/93
  • Fix error for Orcid telescope by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/100
  • Updated crossref metadata schema by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/102
  • Fix gsutil bug for OpenAlex by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/103
  • Fix empty line in OpenAlex file by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/105
  • Fix wiki descriptions by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/106
  • Update createbigquerytablefromquery call by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/109
  • MEL-800 added data trust zenodo community by @kathrynnapier in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/108
  • Remove largest remainder rounding by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/107
  • Rename 'metricsgreenonly' to 'metricspercentgreen_only' by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/111
  • Publish new version of dataset to Zenodo automatically by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/110
  • Feature/oa web acronyms by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/104
  • Fix/zenodo date version by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/112
  • Inf 326/dataset release creation by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/84
  • Fix/api update by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/113
  • Download pre-generated twitter cards by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/114
  • Unpaywall: use merged daily feeds to update by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/115
  • Update fieldname "metricscitationspercitedoutput" by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/117
  • Update OpenAlex by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/116
  • Inf 406/seed ao workflows db by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/120
  • Update OpenAlex to use transfer manifest file by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/121
  • Use findfreeport in tests by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/122
  • Add dag tag by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/123
  • Update elasticimportworkflow.py by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/125
  • Compute histograms and change country and institution inclusion thresholds by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/126
  • Closes COK-322 Added All Access Type Counts to Generic and Discipline Exports by @cameronneylon in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/128
  • Add COKI open access outputs and repositories to the DOI and aggregate tables by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/129
  • OpenAlex: set startdate to enddate of recent release by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/134
  • INF-521 DOI Workflow: Remove DOI table being output to coki-dashboards by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/132
  • INF-525: Removing MAG and GRID telescopes, workflows and unit tests by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/137
  • OpenAlex: add extra release date management tests by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/136
  • OA web workflow other platform open updates by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/133
  • Pre-commit update by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/138
  • Replace MAG with OpenAlex by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/87
  • Migrate aggregate/export doi metrics to openalex by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/140
  • INF-557: Removed authors from the list of aggregations by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/141
  • Check for invalid lat lng values by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/143
  • DOI Workflow: add genealogical institutions by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/144
  • OA Web Open Alex by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/142
  • Bad 308 schema updates by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/145
  • Additional kibana spaces by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/148
  • Fix space name by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/151
  • Update github unit tests in github workflow/actions by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/149
  • Updated regex pattern match by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/153
  • Remove ORCID sensor by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/154
  • INF-588: HTTP request updates by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/155
  • Feature/oa web d1 database by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/152
  • Added crossref events download throttling by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/157
  • Add descriptions to DOI and aggregate tables by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/159
  • Fix/doi ror affiliation matcher errors by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/161
  • Revert changes for re-raising non 500 errors by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/163
  • Fix Crossref Metadata release date by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/162
  • Feature/workflow config file by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/158
  • Added .env to gitignore by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/165
  • Fix/deploy may 2023 by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/166
  • Filter Crossref type and fix release_date naming in oa web workflow by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/167
  • Fix/oa web repo dispatch by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/168
  • Fix write disposition by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/170
  • Fix/Add s3:// prefix to OpenAlex AWS blob URL if not present by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/172
  • Fix/OpenAlex "abstractinvertedindex" field is sometimes a string, not a dictionary by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/173
  • Refactor DOI table by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/171
  • Feature/airflow2.6.3 by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/175
  • Remove Elastic mappings by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/176
  • Convert updated_date into timestamp when it is supplied as a date by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/177
  • Pubmed Telescope by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/160
  • Ignore ROR relationships where the ROR IDs don't exist by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/179
  • Download and read data files with a glob pattern by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/180
  • Updates for Cloudflare D1 branch by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/178
  • Fix/Pubmed Telescope: Merge upserts and deletes by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/181
  • Updated parser fields for tests by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/184
  • INF-648: Add Pubmed to DOI table by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/182
  • Fix/OpenAlex: abstractinvertedindex field now a JSON structure by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/183
  • INF-652:Add new fields to OpenAlex by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/185
  • Remove duplicate DOIs from Crossref Metadata by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/186
  • Orcid Telescope v2 by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/169
  • DOI Workflow: Comment out PubMed by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/188
  • OpenAlex: use taskgroups and process each table as separate tasks by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/187
  • Fix ORCID telescope queue name by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/189
  • Remove any PubMed record that has duplicate DOIs by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/191
  • Added readtehdocs config file by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/192
  • python 3.10 update by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/190
  • Fixed missing _latest.csv schemas by @keegansmith21 in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/193
  • added code of conduct file, and contributing guide that links to the … by @kathrynnapier in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/194
  • Use more SQL for OA Dashboard Workflow by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/196
  • Fix/oa dashboard by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/200
  • Feature/Update OpenAlex schema - November 2023 snapshot by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/201
  • INF-665: OpenAlex Telescope - Schema detect feature by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/198
  • INF-638: Data Quality Workflow by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/202
  • Fix: OpenAlex Schema Generator by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/203
  • Update: Fix OpenAlex schema and add missing fields by @alexmassen-hane in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/204

New Contributors

  • @bechandcock made their first contribution in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/98
  • @kathrynnapier made their first contribution in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/108
  • @cameronneylon made their first contribution in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/128
  • @alexmassen-hane made their first contribution in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/132
  • @keegansmith21 made their first contribution in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/138

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/2022.03.0...2023.12.0

- Python
Published by jdddog about 2 years ago

academic-observatory-workflows - 2022.03.0

What's Changed

  • Inf 40/create unpaywall data feed telescope by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/39
  • Rename snapshot to unpaywall_snapshot, datafeed to unpaywall by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/44
  • Update StreamTelescope telescopes by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/45
  • INF-166/airflow-2.2 by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/49
  • Inf 170/unpaywall download changes by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/51
  • ROR Telescope by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/50
  • Inf 60/add operator by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/53
  • Port open citations to telescope template. by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/48
  • Inf 164/port wos telescope template by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/55
  • port scopus telescope by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/56
  • INF-181 ORCID records not being correctly matched by @rhosking in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/58
  • Added new oa fields and repo names by @rhosking in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/57
  • Remove Unpaywall history by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/54
  • Update ror_telescope.py by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/63
  • INF-65: MAG updates by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/60
  • INF-191 Add discipline to affiliation list by @rhosking in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/61
  • INF-182 use issn-l for journal ids by @rhosking in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/59
  • INF-60: Open Access Website Workflow by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/52
  • Add xcom cleanup to mag telescope by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/65
  • Inf 183 - Move to ROR by @rhosking in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/64
  • Inf 304/scopus deployment fix extra documentation by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/70
  • Feature/crossref metadata schema update by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/71
  • Ignore unknown fields by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/69
  • INF-193 Commenting the SQL scripts by @rhosking in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/62
  • Test fixes after bq quota checks feature by @tuanchien in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/72
  • INF-208 Updated DOI workflow page in readthedocs by @rhosking in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/68
  • Remove MAG sensor by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/73
  • INF-320: Fix duplicate DOIs by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/76
  • Update config instance for elastic import workflow by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/75
  • Fix crossref metadata invalid timestamp by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/77
  • INF-328: Ignore unknown values on BigQuery load by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/79
  • INF-287: Update generated data to work with Next.js web application by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/66
  • Fix ORCID telescope gcloud error by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/85
  • OpenAlex telescope by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/80
  • Add wikipedia descriptions during oa web workflow by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/82
  • Inf 333/publish oa web data by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/83
  • OpenAlex fixes by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/88
  • Small fixes OA web workflow wiki descriptions by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/86
  • Fix error getting wiki description oawebworkflow by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/90
  • Fix transform for cases where the transformed field does not exist. by @aroelo in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/92
  • Refactor to use new country and region table by @jdddog in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/91

New Contributors

  • @rhosking made their first contribution in https://github.com/The-Academic-Observatory/academic-observatory-workflows/pull/58

Full Changelog: https://github.com/The-Academic-Observatory/academic-observatory-workflows/compare/2021.09.1...2022.03.0

- Python
Published by jdddog almost 4 years ago

academic-observatory-workflows - 2021.09.1

This release includes the following changes / new features: * Add black to pre-commit config. * Added a standardised makereleasedate function to observatory-platform, which is now used in Onix Workflow. It returns a pendulum.DateTime rather than a pendulum.Date. * Remove uploaddownloaded from SnapshotTelescope subclasses. * Refactor relevant workflows to use XComless StreamTelescope. * Ported Unpaywall workflow to use the SnapshotTelescope. * Removed getfile and hashfile. * Crossref Metdata: * Can transform JSON input files whether pretty printed or not. * Refactored transform functions to use Python rather than mawk as it is easier to understand what the transformation does and it is faster. * Added explicit tests for transformfile and transformitem.

This release includes the following bugfixes: * Remove RUN statement from requirements shell script. * Crossref Metadata now needs the API key to make a head request, which is now added to the head request. * Fix bug in Fundref getreleaseinfo, which was passing a string to bigqueryshardedtable_id rather than a DateTime.

- Python
Published by jdddog over 4 years ago

academic-observatory-workflows - 2021.09.0

This release includes telescope workflows for: * Crossref Events * Crossref Fundref * Crossref Metadata * Geonames * GRID * Microsoft Academic Graph * Open Citations * ORCID * Scopus * Unpaywall * Web of Science

And the following analytical workflows: * DOI Workflow * Elastic Import Workflow setup to export output from the DOI Workflow into Elasticsearch and Kibana.

- Python
Published by jdddog over 4 years ago