https://github.com/alan-turing-institute/science-gateway-middleware

Middleware endpoints for the Science Gateway project

https://github.com/alan-turing-institute/science-gateway-middleware

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
  • DOI references
  • Academic publication links
  • Academic email domains
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (14.3%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Middleware endpoints for the Science Gateway project

Basic Info
Statistics
  • Stars: 3
  • Watchers: 22
  • Forks: 0
  • Open Issues: 14
  • Releases: 0
Created about 9 years ago · Last pushed over 8 years ago
Metadata Files
Readme License

README.md

science-gateway-middleware

Build Status Build Status

Overview

Currently this is a skeleton app that is designed to take values via HTTP POST, build a bash command to multiply them together and run that command via ssh on a remote server.

Local deployment

This code is tested on python 3.4 and all dependencies can be installed via pip:

pip install -r requirements.txt

At present this app uses a config.py file to store login details. For obvious reasons this is not committed with the rest of the code, but can be created locally. The code expects a config.py file placed in a root-level instance of the directory with the following variables:

``` USERNAME = "" PASSWORD = "" SSHUSR = "" SSHHOSTNAME = "" SSH_PORT =

SSHPRIVATEKEY_PATH = None

SSHPRIVATEKEY_PATH = "keys/development" # where development is a private RSA key

SQLALCHEMYDATABASEURI = 'sqlite://'

SQLALCHEMYDATABASEURI = 'sqlite:////tmp/test.db' SQLALCHEMYTRACKMODIFICATIONS = False ```

The SHH_* variables will need to point to science-gateway-cluster but at present I have been testing this via one of UCL's servers. The ssh code assumes that you already have ssh keys configured on the machine where the app is running, and has limited error handling if this is not the case.

The middleware can be hosted locally on localhost for testing via the middleware/app.py entry point. To achieve this, run the following command:

shell ./run_test.sh

Similarly, the production config file checked into source control can be tested using ./run_production.sh. However, note that the same instance/config.py will be used in all cases, so it will nbot be a true test of production

Azure deployment

To create an MS Azure Web App Service run the following sequence of commands. First, choose a name for the app (note, the resource group, app service plan and web app will all share this name, but unique names could be set for each if needed).

shell APP_NAME=Science-Gateway-Middleware

Set a username and password for the Web App Service:

az login az webapp deployment user set --user-name <username> --password <password>

Create the Azure web app:

shell az group create --name $APP_NAME --location westeurope az appservice plan create --name $APP_NAME --resource-group $APP_NAME --sku S1 # use S1 or higher for access to development slots az webapp create --name $APP_NAME --resource-group $APP_NAME --plan $APP_NAME az webapp config set --python-version 3.4 --name $APP_NAME --resource-group $APP_NAME # set python version az webapp config appsettings set --name $APP_NAME --resource-group $APP_NAME --settings APP_CONFIG_NAME=production

Configure web app environment variables.

```shell APPNAME=Science-Gateway-Middleware APPSLOT=dev

az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SSHUSR=

az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SSHHOSTNAME=

az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SSHPORT=22

az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SIMROOT=

az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SSHPRIVATEKEYSTRING= ```

The develop branch will deploy to a development "slot" on Azure via the following branch-specific .travis.ymlsettings:

yaml deploy: - provider: azure_web_apps verbose: true skip_cleanup: false on: develop slot: science-gateway-middleware-dev

The staging branch will deploy to a staging "slot" on Azure via the following branch-specific .travis.ymlsettings:

yaml deploy: - provider: azure_web_apps verbose: true skip_cleanup: false on: staging slot: science-gateway-middleware-staging

Manual deployment to Azure is possible via dpl:

sudo gem install dpl

For example, run the following to deploy to the Azure dev slot:

export AZURE_WA_PASSWORD=<password> export AZURE_WA_USERNAME=<user-name> export AZURE_WA_SITE=science-gateway-middleware export AZURE_WA_SLOT=science-gateway-middleware-dev dpl --provider=AzureWebApps --verbose

Any un-committed files can be added using using ftp. The ftp address is available via App Service > Properties in Azure portal. An example ftp credentials are:

server address: ftps://<address-details>.ftp.azurewebsites.windows.net user: Science-Gateway-Middleware\<username>

Testing

Tests can be run via python -m pytest.

GitHub Events

Total
  • Issues event: 2
Last Year
  • Issues event: 2

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 57
  • Total pull requests: 57
  • Average time to close issues: 29 days
  • Average time to close pull requests: about 1 month
  • Total issue authors: 4
  • Total pull request authors: 5
  • Average comments per issue: 4.26
  • Average comments per pull request: 3.25
  • Merged pull requests: 47
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • masonlr (12)
  • martintoreilly (11)
  • sgrieve (6)
  • rwood-97 (1)
  • myyong (1)
Pull Request Authors
  • masonlr (11)
  • sgrieve (8)
  • martintoreilly (8)
  • dpshelio (1)
  • myyong (1)
Top Labels
Issue Labels
in progress (3) enhancement (3) bug (3) help wanted (2) Testing (1) Documentation (1)
Pull Request Labels
in progress (3) bug (1)

Dependencies

requirements.txt pypi
  • Flask ==0.12.2
  • Flask-Cors ==3.0.3
  • Flask-RESTful ==0.3.6
  • Flask-SQLAlchemy ==2.2
  • Jinja2 ==2.9.6
  • Mako ==1.0.7
  • MarkupSafe ==1.0
  • PyNaCl ==1.1.2
  • SQLAlchemy ==1.1.12
  • SQLAlchemy-Utils ==0.32.14
  • Werkzeug ==0.12.2
  • aniso8601 ==1.2.1
  • arrow ==0.10.0
  • asn1crypto ==0.22.0
  • bcrypt ==3.1.3
  • cffi ==1.10.0
  • click ==6.7
  • cryptography ==2.0.2
  • flake8 ==3.3.0
  • flask-marshmallow ==0.8.0
  • idna ==2.5
  • itsdangerous ==0.24
  • json-merge-patch ==0.1
  • marshmallow-sqlalchemy ==0.13.1
  • paramiko ==2.2.1
  • py ==1.4.34
  • pyasn1 ==0.3.1
  • pycparser ==2.18
  • pytest ==3.1.3
  • python-dateutil ==2.6.1
  • pytz ==2017.2
  • scp ==0.10.2
  • six ==1.10.0