springytools

A set of Go based command line tools for working with LibGuides export XML

https://github.com/caltechlibrary/springytools

Last synced: 6 months ago · JSON representation ·

Repository

A set of Go based command line tools for working with LibGuides export XML

Basic Info

Host: GitHub
Owner: caltechlibrary
License: other
Language: Go
Default Branch: main
Size: 103 KB

Statistics

Stars: 0
Watchers: 5
Forks: 0
Open Issues: 0
Releases: 3

Created over 4 years ago · Last pushed over 4 years ago

Metadata Files

Readme Changelog Contributing License Code of conduct Citation Support Codemeta

LibGuides Tools

A Golang package for working with LibGuides exported XML.

Introduction

There is a periodic need to work with exported LibGuides XML in Caltech Library. This is a Golang package for working with the exported data. Go provides a robust may of mapping simple data structures to and from XML (or JSON). This makes working with XML very easy in a consistent fashion. It seem time to move beyond my usual Bash/sed/python scripts.

One program is currently provided with springytools, lgxml2sjon which converts a LibGuides XML export file into JSON.

Installation

This is a Golang package providing two commands for working with LibGuides' exported XML. To compile you will need Go 1.16 or better, GNU Make and Stephen Dolan's jq for browser JSON output.

Steps to compile from source

clone the repository
change into the clone directory
test
build the command line tool lgxml2json
use lgxml2json and test output with jq
- Replace "LibGuidesexportXXXXX.xml" with the file path to your exported LibGuides XML file
install lgxml2json

Example commands to execute in the shell (e.g. Terminal on macOS, xterm on Linux)

~~~ git clone git@github.com:caltechlibrary/springytools cd springytools make make test make install ~~~

By default installation is to your $HOME/bin directory. This directory should be in your shell's "PATH".

You can get a brief description of the commands using the -h option with the command.

~~~ lgxml2json -h lglinkreport -h ~~~

Known issues and limitations

This library is currently written to perform the LibGuides link analysis. It only provides the commands I needed to do the data analysis. It will grow as needed.

The exported XML output from the LibGuides may not be valid UTF-8. UTF-8 encoding is required to successfully parse the export file. Looking at the raw XML markup in vim I noticed a number of control code sequences. This corresponded to the errors on parsing the unsanitized XML file. The problem characters appear as ^A, ^K, ^L, ^S, ^C, ^R. These maybe non-UTF-8 characters embedded as UTF-8 when the rich text documents were pasted in via the LibGuides edit UI. My hunch is these were pasted in/imported from Word documents. Remove the offending characters allowed the export to parse successfully. These edits are destructive as some of the codes probably represent UTF-8 characters used in non-English European names or terminology.

Getting help

File an issue on GitHub.

License

Software produced by the Caltech Library is Copyright © 2021 California Institute of Technology. This software is freely distributed under a BSD/MIT type license. Please see the LICENSE file for more information.

Authors and history

R. S. Doiel, Software Developer, Digital Library Development, Caltech Library

Acknowledgments

This work was funded by the California Institute of Technology Library.

(If this work was also supported by other organizations, acknowledge them here. In addition, if your work relies on software libraries, or was inspired by looking at other work, it is appropriate to acknowledge this intellectual debt too.)

Owner

Name: Caltech Library
Login: caltechlibrary
Kind: organization
Email: helpdesk@library.caltech.edu
Location: Pasadena, CA 91125

Website: https://www.library.caltech.edu/
Repositories: 84
Profile: https://github.com/caltechlibrary

We manage the physical and digital holdings of the California Institute of Technology, provide services and training, and develop open-source software.

Citation (CITATION.cff)

# YAML 1.2
---
authors: 
  -
    affiliation: "Caltech Library"
    family-names: Doiel
    given-names: Robert
    orcid: "https://orcid.org/0000-0003-0900-6903"
cff-version: "0.0.0"
date-released: 2021-07-13
doi:
keywords: 
  - libguides
  - xml
  - json
message: "If you use this software, please cite it using these metadata."
repository-code: "https://github.com/caltechlibrary/springytools"
title: "springytools"
version: "0.0.3"
...

CodeMeta (codemeta.json)

{
  "@context": "https://doi.org/10.5063/schema/codemeta-2.0",
  "@type": "SoftwareSourceCode",
  "description": "A set of command line tools for working LibGuides export XML.",
  "name": "springytools",
  "codeRepository": "https://github.com/caltechlibrary/springytools",
  "issueTracker": "https://github.com/caltechlibrary/springytools/issues",
  "license": "https://github.com/caltechlibrary/springytools/blob/main/LICENSE",
  "version": "0.0.3",
  "author": [
    {
      "@type": "Person",
      "givenName": "Robert",
      "familyName": "Doiel",
      "affiliation": "Caltech Library",
      "email": "rsdoiel@caltech.edu",
      "@id": "https://orcid.org/0000-0003-0900-6903"
    }
  ],
  "developmentStatus": "concept",
  "downloadUrl": "https://github.com/caltechlibrary/springytools/archive/main.zip",
  "keywords": [
    "software",
    "LibGuides",
    "xml",
    "json",
    "csv"
  ],
  "maintainer": "https://orcid.org/0000-0003-0900-6903",
  "funder": {
    "@id": "https://doi.org/10.13039/100006961",
    "@type": "Organization",
    "name": "Caltech Library"
  },
  "programmingLanguage": [
    "Go"
  ]
}

GitHub Events

Total

Last Year

Issues and Pull Requests

Last synced: 8 months ago

All Time

Total issues: 0
Total pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Total issue authors: 0
Total pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 0
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 0
Pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

Total packages: 1
Total downloads: unknown

Total dependent packages: 0
Total dependent repositories: 0
Total versions: 3

proxy.golang.org: github.com/caltechlibrary/springytools

expected.go is a set of testing functions Author: R. S. Doiel <rsdoiel@caltech.edu> Copyright (c) 2021, Caltech All rights not granted herein are expressly reserved by Caltech. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. extractors.go provides funcs for processing text and pulling out elements like URL links. Author: R. S. Doiel <rsdoiel@caltech.edu> Copyright (c) 2021, Caltech All rights not granted herein are expressly reserved by Caltech. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. libguides.go implements the data structures for working with with LibGuides exported XML. Author: R. S. Doiel <rsdoiel@caltech.edu> Copyright (c) 2021, Caltech All rights not granted herein are expressly reserved by Caltech. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. reports.go provides the functions that work in an input filename and output filename generating reports or data conversions. Author: R. S. Doiel <rsdoiel@caltech.edu> Copyright (c) 2021, Caltech All rights not granted herein are expressly reserved by Caltech. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. tables.go provides a XML, JSON and CSV rendering of the Table datastructure. Author: R. S. Doiel <rsdoiel@caltech.edu> Copyright (c) 2021, Caltech All rights not granted herein are expressly reserved by Caltech. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Homepage: https://github.com/caltechlibrary/springytools
Documentation: https://pkg.go.dev/github.com/caltechlibrary/springytools#section-documentation
License: BSD-3-Clause
Latest release: v0.0.3
published over 4 years ago

Versions: 3
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 8.7%

Average: 9.3%

Dependent repos count: 9.8%

Last synced: 7 months ago