Data

Tools

Indexes

Applications

Experiments

Open Source Science

gsky

Distributed Scalable Geospatial Data Server

https://github.com/nci/gsky

Science Score: 28.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    5 of 12 committers (41.7%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.1%) to scientific vocabulary

Keywords

geospatial golang grpc postgis remote-sensing wcs wms wps
Last synced: 6 months ago · JSON representation ·

Repository

Distributed Scalable Geospatial Data Server

Basic Info
  • Host: GitHub
  • Owner: nci
  • Language: Go
  • Default Branch: master
  • Size: 4.58 MB
Statistics
  • Stars: 32
  • Watchers: 4
  • Forks: 16
  • Open Issues: 2
  • Releases: 37
Topics
geospatial golang grpc postgis remote-sensing wcs wms wps
Created almost 8 years ago · Last pushed almost 3 years ago
Metadata Files
Readme License Citation

README.md

GSKY: Distributed Scalable Geospatial Data Server

What Is This?

GSKY was developed at NCI and is a scalable, distributed server which presents a new approach for geospatial data discovery and delivery using OGC standards. The most recent release is here.

Build Status

GSKY Docker Image

The quickest way to try out GSKY is via GSKY docker image.

GSKY's docker image ships with a fully functional environment of GSKY cluster, as well as sample data files and configuration files for testing.

docker pull gjmouse/gsky:latest

For more details of GSKY docker image, please refer here.

Overview of Servers Components

GSKY mainly consists of three servers working together to deliver services. The main server (ows.go) is the front-end server that takes WMS/WCS/WPS HTTP requests as inputs. The main server talks to the MAS Restful API server (mas/api/api.go) for the data files that intersect with the polygon bounding box in the WMS/WCS/WPS requests. With those data files, the main server talks to the RPC worker nodes (grpc-server/main.go) for compute and IO intensive tasks and then sends the results back to the client side.

How To Start the Servers

  • Start the MAS Restful API server: /opt/gsky/sbin/masapi -port 8888

    The -port option sets the API server listening port. The default is port 8080.

  • Start all the RPC worker nodes: /opt/gsky/sbin/gsky-rpc -p 6000

    The -p option sets the gRPC listening port. The default is port 6000.

  • Start the main server: /opt/gsky/sbin/gsky-ows -p 8080

    The -p option sets the main server listening port. The default is port 8080.

Configuration Files

  1. config.json: Contains the list of WMS, WCS and WPS layers exposed by the server. It also contains the IP address of the index API used in the workflow. In addtion, it contains the list of worker nodes specifying the IP address and list of ports per worker. Several workers can be specified on a single machine by adding several entries using the same IP address and different ports. These services have to be locally started at the specified machines.

  2. Serveral config.json files can be organized into directories to form namespaces to group logical collection of datasets together. For example, the server serves two science projects with the following URLs:

1) http://<server address>/ows/project1 2) http://<server address>/ows/project2

The directory structure of the config files will be as follows:

``` 

   project1
      config.json

   project2
      config.json

```

How To Compile the Source

Dependencies:

  • Go 1.16.x+
  • C++ 14+
  • GDAL 3.x+
  • PostgreSQL 11+
  • PostGIS 3.x+
  • Various Go packages (handled by go modules)

console $ git clone https://github.com/nci/gsky $ cd gsky $ ./configure $ make all install

The configure script takes all of the standard GNU configure flags such as --prefix (to specify where to install GSKY).

Contributions

Suggestions, enhancement requests, bug reports and patches to GSKY are welcome via this GitHub page. Please submit patches as a GitHub pull request. Authors retain copyright over their contributions.

Citing GSKY in publications

When referring to GSKY in publications please use the citation in CITATION.md. A ready-to-use BibTeX entry for LaTeX users can also be found in this file.

License

Copyright 2016 Australian National University

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this package except in compliance with the License. A copy of the License may be found in this source distribution in LICENSE-2.0.txt.

Owner

  • Name: NCI
  • Login: nci
  • Kind: organization
  • Location: Australian National University, Canberra, Australia

Citation (CITATION.md) 

To cite GSKY in publications please use:

  National Computational Infrastructure and contributors (2018). The
  GSKY Distributed Scalable Geospatial Data Server. National
  Computational Infrastructure. URL
  [http://nci.org.au](http://nci.org.au)

A BibTeX entry for LaTeX users is

```
  @Manual{,
    title = {GSKY: A scalable, distributed geospatial data-server},
    author = {Pablo R. Larraondo, Sean Pringle, Joseph Antony, Ben Evans},
    journal={Proceedings of the Academic Research Stream at the Annual Conference Locate, Research@Locate 2017. Sydney, Australia, April 3-6, 2017.},
    year = {2017},
    url = {http://ceur-ws.org/Vol-1913/RL17_paper_14.pdf},
  }
```

GitHub Events

Total
Last Year

Committers

Last synced: about 2 years ago

All Time
  • Total Commits: 571
  • Total Committers: 12
  • Avg Commits per committer: 47.583
  • Development Distribution Score (DDS): 0.459
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
edisonguo e****o@a****u 309
Ben Elliston b****n@a****u 119
edisonguo g****e@g****m 103
Ben Elliston b****e@a****u 26
Sean Pringle s****e@g****m 6
root r****t@g****l 2
root r****t@g****l 1
Joseph Antony j****y@a****u 1
Pablo Rozas Larraondo p****o@a****u 1
Matthew Sanderson m****n@a****u 1
Sean Pringle 3****0 1
Sean Pringle s****e@n****u 1
Committer Domains (Top 20 + Academic)
anu.edu.au: 5 nci.org.au: 1 gsky-datamgt.novalocal: 1 gsky-edison-bug-fix.novalocal: 1 air.net.au: 1

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 9
  • Total pull requests: 91
  • Average time to close issues: 9 months
  • Average time to close pull requests: less than a minute
  • Total issue authors: 7
  • Total pull request authors: 2
  • Average comments per issue: 2.33
  • Average comments per pull request: 0.0
  • Merged pull requests: 89
  • Bot issues: 0
  • Bot pull requests: 2
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • erick-otenyo (3)
  • sempervictus (1)
  • whatnick (1)
  • pbranson (1)
  • gaellafond (1)
  • bje- (1)
  • dtpc (1)
Pull Request Authors
  • edisonguo (89)
  • dependabot[bot] (2)
Top Labels
Issue Labels
Pull Request Labels
dependencies (2)

Packages

  • Total packages: 1
  • Total downloads: unknown
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 37
proxy.golang.org: github.com/nci/gsky
  • Versions: 37
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Forks count: 3.7%
Stargazers count: 4.3%
Average: 6.1%
Dependent packages count: 7.0%
Dependent repos count: 9.3%
Last synced: 6 months ago

Dependencies

go.mod go
  • github.com/CloudyKit/fastprinter v0.0.0-20200109182630-33d98a066a53
  • github.com/edisonguo/govaluate v3.0.1-0.20200610034316-8fb8220f46f3+incompatible
  • github.com/edisonguo/jet v2.1.2+incompatible
  • github.com/golang/protobuf v1.5.2
  • github.com/kavu/go_reuseport v1.5.0
  • github.com/lib/pq v1.10.1
  • github.com/nci/geometry v0.0.0-20170727004624-e73695b914d9
  • github.com/nci/gomemcache v0.0.0-20170208213004-1952afaa557d
  • golang.org/x/crypto v0.0.0-20210505212654-3497b51f5e64
  • golang.org/x/net v0.0.0-20210505214959-0714010a04ed
  • google.golang.org/grpc v1.37.0
  • gopkg.in/yaml.v2 v2.4.0
go.sum go
  • cloud.google.com/go v0.26.0
  • github.com/BurntSushi/toml v0.3.1
  • github.com/CloudyKit/fastprinter v0.0.0-20200109182630-33d98a066a53
  • github.com/census-instrumentation/opencensus-proto v0.2.1
  • github.com/client9/misspell v0.3.4
  • github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403
  • github.com/davecgh/go-spew v1.1.0
  • github.com/edisonguo/govaluate v3.0.1-0.20200610034316-8fb8220f46f3+incompatible
  • github.com/edisonguo/jet v2.1.2+incompatible
  • github.com/envoyproxy/go-control-plane v0.9.0
  • github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473
  • github.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d
  • github.com/envoyproxy/protoc-gen-validate v0.1.0
  • github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
  • github.com/golang/mock v1.1.1
  • github.com/golang/protobuf v1.2.0
  • github.com/golang/protobuf v1.3.2
  • github.com/golang/protobuf v1.4.0-rc.1
  • github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208
  • github.com/golang/protobuf v1.4.0-rc.2
  • github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0
  • github.com/golang/protobuf v1.4.0
  • github.com/golang/protobuf v1.4.1
  • github.com/golang/protobuf v1.4.2
  • github.com/golang/protobuf v1.5.0
  • github.com/golang/protobuf v1.5.2
  • github.com/google/go-cmp v0.2.0
  • github.com/google/go-cmp v0.3.0
  • github.com/google/go-cmp v0.3.1
  • github.com/google/go-cmp v0.4.0
  • github.com/google/go-cmp v0.5.0
  • github.com/google/go-cmp v0.5.5
  • github.com/google/uuid v1.1.2
  • github.com/kavu/go_reuseport v1.5.0
  • github.com/lib/pq v1.10.1
  • github.com/nci/geometry v0.0.0-20170727004624-e73695b914d9
  • github.com/nci/gomemcache v0.0.0-20170208213004-1952afaa557d
  • github.com/pmezard/go-difflib v1.0.0
  • github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
  • github.com/stretchr/objx v0.1.0
  • github.com/stretchr/testify v1.5.1
  • golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
  • golang.org/x/crypto v0.0.0-20210505212654-3497b51f5e64
  • golang.org/x/exp v0.0.0-20190121172915-509febef88a4
  • golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
  • golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961
  • golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3
  • golang.org/x/net v0.0.0-20180724234803-3673e40ba225
  • golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
  • golang.org/x/net v0.0.0-20190213061140-3a22650c66bd
  • golang.org/x/net v0.0.0-20190311183353-d8887717615a
  • golang.org/x/net v0.0.0-20210226172049-e18ecbb05110
  • golang.org/x/net v0.0.0-20210505214959-0714010a04ed
  • golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
  • golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
  • golang.org/x/sync v0.0.0-20181108010431-42b317875d0f
  • golang.org/x/sync v0.0.0-20190423024810-112230192c58
  • golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
  • golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
  • golang.org/x/sys v0.0.0-20201119102817-f84b799fce68
  • golang.org/x/sys v0.0.0-20210423082822-04245dca01da
  • golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1
  • golang.org/x/text v0.3.0
  • golang.org/x/text v0.3.3
  • golang.org/x/text v0.3.6
  • golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
  • golang.org/x/tools v0.0.0-20190114222345-bf090417da8b
  • golang.org/x/tools v0.0.0-20190226205152-f727befe758c
  • golang.org/x/tools v0.0.0-20190311212946-11955173bddd
  • golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135
  • golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
  • google.golang.org/appengine v1.1.0
  • google.golang.org/appengine v1.4.0
  • google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
  • google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
  • google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013
  • google.golang.org/grpc v1.19.0
  • google.golang.org/grpc v1.23.0
  • google.golang.org/grpc v1.25.1
  • google.golang.org/grpc v1.27.0
  • google.golang.org/grpc v1.37.0
  • google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd
  • google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64
  • google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60
  • google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967
  • google.golang.org/protobuf v1.21.0
  • google.golang.org/protobuf v1.22.0
  • google.golang.org/protobuf v1.23.0
  • google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc
  • google.golang.org/protobuf v1.25.0
  • google.golang.org/protobuf v1.26.0-rc.1
  • google.golang.org/protobuf v1.26.0
  • gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
  • gopkg.in/yaml.v2 v2.2.2
  • gopkg.in/yaml.v2 v2.4.0
  • honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099
  • honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc