Gumbo Python Client

A python package for reading/writing to the gumbo database

Installations

Note: the client has been switched to the v2 of the client which works very differently. The installation instructions below have changed

If using poetry, run:

poetry source add --priority=supplemental public-python https://us-central1-python.pkg.dev/cds-artifacts/public-python/simple/ poetry add --source public-python gumbo-rest-client

Otherwise, if you don't use poetry, install via pip

pip install --extra-index-url=https://us-central1-python.pkg.dev/cds-artifacts/public-python/simple/ gumbo-rest-client

This repo now only contains the new version of the gumbo client which no longer makes direct connections to the database but instead uses a hosted service. From the users perspective this just means:

1. Simpler setup (no need to download the proxy, and debug differing versions)
2. No need for the client to launch a separate task in the background
3. Fewer connection errors

To use the new client, first run gcloud auth application-default login to set up your google credentials which will be used as the default app credientials.

Afterwards, you should be able to import Client from gumbo_rest_client and then use it as you normally would to read tables such as in the below:

``` from gumborestclient import Client

client = Client() df = client.get("depmapmodeltype") ```

If you're writing a script which should use a service account instead of your user credentials, make sure the service account is set up to be the default app credentials (setting GOOGLE_APPLICATION_CREDENTIALS if necessary) and then use the following code to create the client:

``` from gumborestclient import Client, createauthorizedsession

client = Client(username="myscriptname", authedsession=createauthorizedsession(usedefaultserviceaccount=True)) df = client.get("depmapmodeltype") ```

If you want to test against the staging version, provide a different base_url ``` from gumborestclient import Client, staging_url

client = Client(baseurl=stagingurl) df = client.get("depmapmodeltype") ```

Usage

Read or write from the following tables: - model - modelcondition - omicsprofile - omicssequencing - screen - screensequence

The client will autocommit changes after insertions or updates.

``` from gumborestclient import Client

client = Client()

to read

df = client.get("table_name")

to only update existing rows:

client.updateonly("tablename", df) # throws an exception if a given row doesn't already exist

to only insert new rows:

client.insertonly("tablename", newrowsdf) # throws an exception if a given row already exists

finally, close the database connection

client.close() ```

Debugging Connection Issues

If you get a "Bad Request" error while trying to create the client (client = Client()): 1. Make sure you are logged in with Broad Google account, not a service account. You can check by running gcloud auth list 2. It's possible you have previously set application-default credentials set which are overriding your main user credentials. You can try resetting the application default credentials by running cloud auth application-default login and logging in with your Broad Google account.

If you get an error about "Unable to acquire impersonated credentials ... PERMISSION_DENIED ... iam.serviceAccounts.getAccessToken" you are probably missing a required permission. Make sure your account has been granted "Service Account Token Creator" access on the service account gumbo-client-iap-auth@depmap-gumbo.iam.gserviceaccount.com. An admin can make this change in the Google Console's Service Account page (under the depmap-gumbo project).

Using the client with a service account

If you are writing your script to run from a non-interactive process, you will need a service account for it to run under and initialize the client. This service account will need to be granted access to the API via IAP permissions. To grant, go to https://console.cloud.google.com/security/iap?referrer=search&project=depmap-gumbo and grant the "IAP Secured Web App User" role to the service account on the resource "rest-api-v2" (for the production instance) and "rest-api-v2-staging" (for the staging instance).

Also, you should pass in a useful label for username so we know the source of the updates when they are recorded to the audit log.

Running tests

The codebase is organized into a few different python modules, some of which have dependencies on one another.

After checking out repo, make sure to run ./all_install.sh to create a poetry environment for each one.

In the parent run ./all_tests.sh to run all tests. Alternatively you can cd to a particular module and run the tests for that module via poetry run pytest.

Publishing new versions of the gumbo client python package

There's nothing in this repo that is private or secret, however, this is an internal tool, so it doesn't seem like it should be published on Pypi. Instead we can publish to an internal package index.

To setup for publishing (Based on https://medium.com/google-cloud/python-packages-via-gcps-artifact-registry-ce1714f8e7c1 )

``` poetry self add keyrings.google-artifactregistry-auth poetry config repositories.public-python https://us-central1-python.pkg.dev/cds-artifacts/public-python/

also make sure you've authentication via "gcloud auth login" if you haven't already

```

Then you can navigate into the package directory and bump the version: cd gumbo-rest-client/ poetry version patch

And then publish via: poetry publish --build --repository public-python