https://github.com/caltechlibrary/library-shell-curl-and-api
An intermediate exploration of Bash, curl and working with content from web API
https://github.com/caltechlibrary/library-shell-curl-and-api
Science Score: 26.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (9.6%) to scientific vocabulary
Repository
An intermediate exploration of Bash, curl and working with content from web API
Basic Info
- Host: GitHub
- Owner: caltechlibrary
- Default Branch: master
- Size: 31.3 KB
Statistics
- Stars: 1
- Watchers: 8
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
Bash, curl and working with web API
Overview
One of the benefits of the Bash shell (and other Unix shells) is it provides a great deal of capability right out of the box. When you combine that with other Unix tools you have a powerful environment for text processing and manipulation. The web provides a massive ammount on content. In this lesson will be combining the two.
What is covered
In this tutorial we assume you have some knowledge of Unix/Bash basics
- How to start the Bash shell (e.g. 'Terminal' on Mac OS X)
- How to list, view, copy, move and delete files and directories
We'll be expanding on that basic knowledge with a discussion of
- Commannd history and command line editing
- Working environment variable
- remembering things for later
- simple text manipulation
- Building complex pipe lines
- Making decisions
- Repeating things
- Simple templated text output with Bash (advanced uses of redirection and cat)
Additionally we will be using two new programs you may not be familiar with
- sed - a automated editor (stream editor) for manipulating text
- curl - for retrieving and interacting with content from the web
- find - walking the content you've fetched from the web
- cut - a tool for working with column oriented data
- paste - a tool for putting text side by side
- join - a tool for combining things
- jq - a tool for working with JSON data
Some data sources for the examples
References
- James Baker, "Preserving Your Research Data", (April 2014), http://programminghistorian.org/lessons/preserving-your-research-data
- Ian Milligan and James Baker, "Introduction to the Bash Command Line", (September 2014) http://programminghistorian.org/lessons/intro-to-bash
- Billy Wideling alien@koping.net, "Alien's Bash Tutorial", (in circulation before 2004, this version apprears to be from circa 2006) http://www.subsignal.org/doc/AliensBashTutorial.html
- William J Turkel, "Basic Text Analysis with Command Line Tools in Linux", (June 2013), https://williamjturkel.net/2013/06/15/basic-text-analysis-with-command-line-tools-in-linux/
- William J Turkel, "Pattern Matching and Permuted Term Indexing with Command Line Tools in Linux", (June 2013) https://williamjturkel.net/2013/06/20/pattern-matching-and-permuted-term-indexing-with-command-line-tools-in-linux/
- Brad Yoes, "Introduction to text manipulation on Unix-based systems" (March 2012) https://www.ibm.com/developerworks/aix/library/au-unixtext/
- Geert Jansen, "Thoughts on RESTful API Design" (November 2012) https://restful-api-design.readthedocs.io/en/latest/
Owner
- Name: Caltech Library
- Login: caltechlibrary
- Kind: organization
- Email: helpdesk@library.caltech.edu
- Location: Pasadena, CA 91125
- Website: https://www.library.caltech.edu/
- Repositories: 84
- Profile: https://github.com/caltechlibrary
We manage the physical and digital holdings of the California Institute of Technology, provide services and training, and develop open-source software.
GitHub Events
Total
Last Year
Committers
Last synced: about 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| R. S. Doiel | r****l@g****m | 27 |
| Juliane Schneider | j****r@g****m | 2 |
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0