Data

Tools

Indexes

Applications

Experiments

Open Source Science

RSocrata

Provides easier interaction with Socrata open data portals http://dev.socrata.com. Users can provide a 'Socrata' data set resource URL, or a 'Socrata' Open Data API (SoDA) web query, or a 'Socrata' "human-friendly" URL, returns an R data frame. Converts dates to 'POSIX' format. Manages throttling by 'Socrata'.

https://github.com/chicago/rsocrata

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.6%) to scientific vocabulary

Keywords

chicago government open-data r socrata soda
Last synced: 6 months ago · JSON representation

Repository

Provides easier interaction with Socrata open data portals http://dev.socrata.com. Users can provide a 'Socrata' data set resource URL, or a 'Socrata' Open Data API (SoDA) web query, or a 'Socrata' "human-friendly" URL, returns an R data frame. Converts dates to 'POSIX' format. Manages throttling by 'Socrata'.

Basic Info
Statistics
  • Stars: 237
  • Watchers: 19
  • Forks: 83
  • Open Issues: 43
  • Releases: 18
Topics
chicago government open-data r socrata soda
Created over 12 years ago · Last pushed 10 months ago
Metadata Files
Readme Changelog Contributing License

README.md

RSocrata

Gitter downloads cran version

Master

Stable beta branch. Test about-to-be-released features in a stable pre-release build before it is submitted to CRAN.

Linux build - MasterWindows build - MasterCoverage - Master

Dev

"Nightly" alpha branch. Test the latest features and bug fixes -- enjoy at your own risk.

Linux build - DevWindows build - DevCoverage - Dev

A tool for downloading and uploading Socrata datasets

Provided with a URL to a dataset resource published on a Socrata webserver, or a Socrata SoDA (Socrata Open Data Application Program Interface) web API query, or a Socrata "human-friendly" URL, read.socrata() returns an R data frame. Converts dates to POSIX format. Supports CSV and JSON download file formats from Socrata. Manages the throttling of data returned from Socrata and allows users to provide an application token. Supports SoDA query parameters in the URL string for further filtering, sorting, and queries. Upload data to Socrata data portals using "upsert" and "replace" methods.

Use ls.socrata() to list all datasets available on a Socrata webserver.

testthat test coverage.

Installation

To get the current released version from CRAN:

R install.packages("RSocrata")

The most recent beta with soon-to-be-released changes can be installed from GitHub:

```R

install.packages("devtools")

devtools::install_github("Chicago/RSocrata") ```

The "nightly" version with the most recent bug fixes and features is also available. This version is always an alpha and may contain significant bugs. You can install it from the dev branch from GitHub:

```R

install.packages("devtools")

devtools::install_github("Chicago/RSocrata", ref="dev") ```

Examples

Reading SoDA valid URLs

r earthquakesDataFrame <- read.socrata("https://soda.demo.socrata.com/resource/4334-bgaj.csv") nrow(earthquakesDataFrame) # 1007 (two "pages") class(earthquakesDataFrame$Datetime[1]) # POSIXlt

Reading "human-readable" URLs

r earthquakesDataFrame <- read.socrata("https://soda.demo.socrata.com/dataset/USGS-Earthquakes-for-2012-11-01-API-School-Demo/4334-bgaj") nrow(earthquakesDataFrame) # 1007 (two "pages") class(earthquakesDataFrame$Datetime[1]) # POSIXlt

Using API key to read datasets

r token <- "ew2rEMuESuzWPqMkyPfOSGJgE" earthquakesDataFrame <- read.socrata("https://soda.demo.socrata.com/resource/4334-bgaj.csv", app_token = token) nrow(earthquakesDataFrame)

Download private datasets from portal

```r

Store user email and password

socrataEmail <- Sys.getenv("SOCRATAEMAIL", "mark.silverberg+soda.demo@socrata.com") socrataPassword <- Sys.getenv("SOCRATAPASSWORD", "7vFDsGFDUG")

privateResourceToReadCsvUrl <- "https://soda.demo.socrata.com/resource/a9g2-feh2.csv" # dataset

read.socrata(url = privateResourceToReadCsvUrl, email = socrataEmail, password = socrataPassword) ```

List all datasets on portal

r allSitesDataFrame <- ls.socrata("https://soda.demo.socrata.com") nrow(allSitesDataFrame) # Number of datasets allSitesDataFrame$title # Names of each dataset

Upload data to portal

```r

Store user email and password

socrataEmail <- Sys.getenv("SOCRATAEMAIL", "mark.silverberg+soda.demo@socrata.com") socrataPassword <- Sys.getenv("SOCRATAPASSWORD", "7vFDsGFDUG")

datasetToAddToUrl <- "https://soda.demo.socrata.com/resource/xh6g-yugi.json" # dataset

Generate some data

x <- sample(-1000:1000, 1) y <- sample(-1000:1000, 1) df_in <- data.frame(x,y)

Upload to Socrata

write.socrata(df_in,datasetToAddToUrl,"UPSERT",socrataEmail,socrataPassword) ```

Issues

Please report issues, request enhancements or fork us at the City of Chicago github.

Contributing

If you would like to contribute to this project, please see the contributing documentation and the product roadmap.

Owner

  • Name: City of Chicago
  • Login: Chicago
  • Kind: organization
  • Email: developers@cityofchicago.org
  • Location: Chicago, IL

Open source projects created by the City of Chicago. City staff can request access through developers@cityofchicago.org from their work email.

GitHub Events

Total
  • Issues event: 3
  • Watch event: 6
  • Member event: 1
  • Issue comment event: 11
  • Gollum event: 1
  • Pull request event: 2
  • Pull request review event: 1
  • Fork event: 2
  • Create event: 1
Last Year
  • Issues event: 3
  • Watch event: 6
  • Member event: 1
  • Issue comment event: 11
  • Gollum event: 1
  • Pull request event: 2
  • Pull request review event: 1
  • Fork event: 2
  • Create event: 1

Committers

Last synced: over 2 years ago

All Time
  • Total Commits: 297
  • Total Committers: 16
  • Avg Commits per committer: 18.563
  • Development Distribution Score (DDS): 0.65
Past Year
  • Commits: 11
  • Committers: 3
  • Avg Commits per committer: 3.667
  • Development Distribution Score (DDS): 0.364
Top Committers
Name Email Commits
Tom Schenk Jr t****r@g****m 104
Lucius n****2@c****g 36
Hugh H****n@c****l 32
Tom Schenk Jr t****k@c****g 32
dmpe c****o@s****z 22
Gene Leynes g****s@g****m 20
Gene Leynes g****s@c****g 13
unknown 3****2@M****l 12
Tom Schenk Jr t****r 10
Peter Schmiedeskamp p****r@t****t 8
Mark Silverberg m****g@s****m 3
Kent Johnson k****7@g****m 1
jjchern j****n@g****m 1
Gene Leynes G****s@c****g 1
Nick Lucius l****s@N****l 1
Nick Lucius n****s@g****m 1
Committer Domains (Top 20 + Academic)
cityofchicago.org: 4 socrata.com: 1 thoughtspot.net: 1 seznam.cz: 1

Issues and Pull Requests

Last synced: 9 months ago

All Time
  • Total issues: 83
  • Total pull requests: 42
  • Average time to close issues: 4 months
  • Average time to close pull requests: 3 days
  • Total issue authors: 30
  • Total pull request authors: 10
  • Average comments per issue: 3.89
  • Average comments per pull request: 1.76
  • Merged pull requests: 30
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 3
  • Pull requests: 2
  • Average time to close issues: 5 months
  • Average time to close pull requests: 19 minutes
  • Issue authors: 1
  • Pull request authors: 2
  • Average comments per issue: 2.33
  • Average comments per pull request: 1.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • geneorama (25)
  • tomschenkjr (18)
  • nicklucius (9)
  • chursaner (2)
  • finestjava (2)
  • LoveYukee (2)
  • Lulliter (1)
  • jbrnbrg (1)
  • heathyates (1)
  • ecorreig (1)
  • daniel-kramer (1)
  • marks (1)
  • bartling (1)
  • delsea (1)
  • jospueyo (1)
Pull Request Authors
  • geneorama (16)
  • nicklucius (14)
  • tomschenkjr (5)
  • JordanMPDS (2)
  • ryan-hall (1)
  • danielOKeefe (1)
  • Elsiemtz (1)
  • userfog (1)
  • shorty0789 (1)
  • flynn-d (1)
Top Labels
Issue Labels
bug (14) enhancement (10) unit testing (9) question (4) documentation (4) wontfix (3) in progress (2) duplicate (1) invalid (1)
Pull Request Labels
bug (2) enhancement (1) documentation (1) wontfix (1)

Packages

  • Total packages: 3
  • Total downloads:
    • cran 1,040 last-month
  • Total docker downloads: 21,255
  • Total dependent packages: 2
    (may contain duplicates)
  • Total dependent repositories: 13
    (may contain duplicates)
  • Total versions: 53
  • Total maintainers: 1
proxy.golang.org: github.com/Chicago/RSocrata
  • Versions: 16
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.5%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 6 months ago
proxy.golang.org: github.com/chicago/rsocrata
  • Versions: 16
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.5%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 6 months ago
cran.r-project.org: RSocrata

Download or Upload 'Socrata' Data Sets

  • Versions: 21
  • Dependent Packages: 2
  • Dependent Repositories: 13
  • Downloads: 1,040 Last month
  • Docker Downloads: 21,255
Rankings
Forks count: 0.7%
Stargazers count: 1.9%
Dependent repos count: 8.0%
Average: 10.1%
Downloads: 12.0%
Dependent packages count: 13.8%
Docker downloads count: 23.9%
Maintainers (1)
developers@cityofchicago.org
Last synced: 9 months ago

Dependencies

DESCRIPTION cran
  • R >= 3.3.0 depends
  • httr >= 1.0.0 imports
  • jsonlite >= 0.9.16 imports
  • mime >= 0.3 imports
  • plyr >= 1.8.4 imports
  • readr >= 0.2.2 suggests
  • roxygen2 >= 4.1.0 suggests
  • testthat >= 0.10.0 suggests