https://github.com/alan-turing-institute/science-gateway-middleware
Middleware endpoints for the Science Gateway project
https://github.com/alan-turing-institute/science-gateway-middleware
Science Score: 13.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (14.3%) to scientific vocabulary
Repository
Middleware endpoints for the Science Gateway project
Basic Info
- Host: GitHub
- Owner: alan-turing-institute
- License: mit
- Language: Python
- Default Branch: master
- Homepage: https://github.com/alan-turing-institute/science-gateway
- Size: 18.8 MB
Statistics
- Stars: 3
- Watchers: 22
- Forks: 0
- Open Issues: 14
- Releases: 0
Metadata Files
README.md
science-gateway-middleware
Overview
Currently this is a skeleton app that is designed to take values via HTTP POST, build a bash command to multiply them together and run that command via ssh on a remote server.
Local deployment
This code is tested on python 3.4 and all dependencies can be installed via pip:
pip install -r requirements.txt
At present this app uses a config.py file to store login details. For obvious reasons this is not committed with the rest of the code, but can be created locally. The code expects a config.py file placed in a root-level instance of the directory with the following variables:
```
USERNAME = ""
PASSWORD = ""
SSHUSR = "
SSHPRIVATEKEY_PATH = None
SSHPRIVATEKEY_PATH = "keys/development" # where development is a private RSA key
SQLALCHEMYDATABASEURI = 'sqlite://'
SQLALCHEMYDATABASEURI = 'sqlite:////tmp/test.db' SQLALCHEMYTRACKMODIFICATIONS = False ```
The SHH_* variables will need to point to science-gateway-cluster but at present I have been testing this via one of UCL's servers. The ssh code assumes that you already have ssh keys configured on the machine where the app is running, and has limited error handling if this is not the case.
The middleware can be hosted locally on localhost for testing via the middleware/app.py entry point. To achieve this, run the following command:
shell
./run_test.sh
Similarly, the production config file checked into source control can be tested using ./run_production.sh. However, note that the same instance/config.py will be used in all cases, so it will nbot be a true test of production
Azure deployment
To create an MS Azure Web App Service run the following sequence of commands. First, choose a name for the app (note, the resource group, app service plan and web app will all share this name, but unique names could be set for each if needed).
shell
APP_NAME=Science-Gateway-Middleware
Set a username and password for the Web App Service:
az login
az webapp deployment user set --user-name <username> --password <password>
Create the Azure web app:
shell
az group create --name $APP_NAME --location westeurope
az appservice plan create --name $APP_NAME --resource-group $APP_NAME --sku S1 # use S1 or higher for access to development slots
az webapp create --name $APP_NAME --resource-group $APP_NAME --plan $APP_NAME
az webapp config set --python-version 3.4 --name $APP_NAME --resource-group $APP_NAME # set python version
az webapp config appsettings set --name $APP_NAME --resource-group $APP_NAME --settings APP_CONFIG_NAME=production
Configure web app environment variables.
```shell APPNAME=Science-Gateway-Middleware APPSLOT=dev
az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SSHUSR=
az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SSHHOSTNAME=
az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SSHPORT=22
az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SIMROOT=
az webapp config appsettings set --name $APPNAME --resource-group $APPNAME --slot $APPSLOT --settings SSHPRIVATEKEYSTRING=
The develop branch will deploy to a development "slot" on Azure via the following branch-specific .travis.ymlsettings:
yaml
deploy:
- provider: azure_web_apps
verbose: true
skip_cleanup: false
on: develop
slot: science-gateway-middleware-dev
The staging branch will deploy to a staging "slot" on Azure via the following branch-specific .travis.ymlsettings:
yaml
deploy:
- provider: azure_web_apps
verbose: true
skip_cleanup: false
on: staging
slot: science-gateway-middleware-staging
Manual deployment to Azure is possible via dpl:
sudo gem install dpl
For example, run the following to deploy to the Azure dev slot:
export AZURE_WA_PASSWORD=<password>
export AZURE_WA_USERNAME=<user-name>
export AZURE_WA_SITE=science-gateway-middleware
export AZURE_WA_SLOT=science-gateway-middleware-dev
dpl --provider=AzureWebApps --verbose
Any un-committed files can be added using using ftp. The ftp address is available via App Service > Properties in Azure portal. An example ftp credentials are:
server address: ftps://<address-details>.ftp.azurewebsites.windows.net
user: Science-Gateway-Middleware\<username>
Testing
Tests can be run via python -m pytest.
Owner
- Name: The Alan Turing Institute
- Login: alan-turing-institute
- Kind: organization
- Email: info@turing.ac.uk
- Website: https://turing.ac.uk
- Repositories: 477
- Profile: https://github.com/alan-turing-institute
The UK's national institute for data science and artificial intelligence.
GitHub Events
Total
- Issues event: 2
Last Year
- Issues event: 2
Issues and Pull Requests
Last synced: over 1 year ago
All Time
- Total issues: 57
- Total pull requests: 57
- Average time to close issues: 29 days
- Average time to close pull requests: about 1 month
- Total issue authors: 4
- Total pull request authors: 5
- Average comments per issue: 4.26
- Average comments per pull request: 3.25
- Merged pull requests: 47
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- masonlr (12)
- martintoreilly (11)
- sgrieve (6)
- rwood-97 (1)
- myyong (1)
Pull Request Authors
- masonlr (11)
- sgrieve (8)
- martintoreilly (8)
- dpshelio (1)
- myyong (1)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- Flask ==0.12.2
- Flask-Cors ==3.0.3
- Flask-RESTful ==0.3.6
- Flask-SQLAlchemy ==2.2
- Jinja2 ==2.9.6
- Mako ==1.0.7
- MarkupSafe ==1.0
- PyNaCl ==1.1.2
- SQLAlchemy ==1.1.12
- SQLAlchemy-Utils ==0.32.14
- Werkzeug ==0.12.2
- aniso8601 ==1.2.1
- arrow ==0.10.0
- asn1crypto ==0.22.0
- bcrypt ==3.1.3
- cffi ==1.10.0
- click ==6.7
- cryptography ==2.0.2
- flake8 ==3.3.0
- flask-marshmallow ==0.8.0
- idna ==2.5
- itsdangerous ==0.24
- json-merge-patch ==0.1
- marshmallow-sqlalchemy ==0.13.1
- paramiko ==2.2.1
- py ==1.4.34
- pyasn1 ==0.3.1
- pycparser ==2.18
- pytest ==3.1.3
- python-dateutil ==2.6.1
- pytz ==2017.2
- scp ==0.10.2
- six ==1.10.0