penquins: a python client for Kowalski

penquins is a python client for Kowalski, a multi-survey data archive and alert broker for time-domain astronomy.

Quickstart

Install penquins from PyPI:

bash pip install penquins --upgrade

Connect to a Kowalski instance:

```python from penquins import Kowalski

username = "" password = ""

protocol, host, port = "https", "", 443

kowalski = Kowalski( username=username, password=password, protocol=protocol, host=host, port=port ) ``` When connecting to only one instance, it will be labeled as "default". Keep this in mind when retrieving the results of your queries.

Connect to multiple Kowalski instances:

```python from penquins import Kowalski

instances = { "kowalski": { "name": "kowalski", "host": "", "protocol": "https" "port": 443, "token": "" # or username and password }, ... }

kowalski = Kowalski(instances=instances) ```

When using multiple instances at once, you can specify a single instance to query using its name when calling query(name=...), or no name at all. If no name is provided and the catalog(s) being queried is/are available on multiple instances, penquins will divide the load between instances automagically.

When retrieving the results, you'll have to use the instance(s) name instead of "default", or simply iterate over the results by instance and merge the results.

It is recommended to authenticate once and then just reuse the generated token:

```python token = kowalski.token print(token)

kowalski = Kowalski( token=token, protocol=protocol, host=host, port=port ) ```

Check connection:

python kowalski.ping()

Querying a Kowalski instance

Most users will be interacting with Kowalski using the Kowalski.query method.

Retrieve available catalog names:

```python query = { "querytype": "info", "query": { "command": "catalognames", } }

response = kowalski.query(query=query) data = response.get("default").get("data") ```

Query for 7 nearest sources to a sky position, sorted by the spheric distance, with a near query:

```python query = { "querytype": "near", "query": { "maxdistance": 2, "distanceunits": "arcsec", "radec": {"querycoords": [281.15902595, -4.4160933]}, "catalogs": { "ZTFsources20210401": { "filter": {}, "projection": {"id": 1}, } }, }, "kwargs": { "maxtime_ms": 10000, "limit": 7, }, }

response = kowalski.query(query=query) data = response.get("default").get("data") ```

Retrieve available catalog names:

```python query = { "querytype": "info", "query": { "command": "catalognames", } }

response = k.query(query=query) data = response.get("default").get("data") ```

Query for 7 nearest sources to a sky position, sorted by the spheric distance, with a near query:

```python query = { "querytype": "near", "query": { "maxdistance": 2, "distanceunits": "arcsec", "radec": {"querycoords": [281.15902595, -4.4160933]}, "catalogs": { "ZTFsources20210401": { "filter": {}, "projection": {"id": 1}, } }, }, "kwargs": { "maxtime_ms": 10000, "limit": 7, }, }

response = k.query(query=query) data = response.get("default").get("data") ```

Run a cone_search query:

```python query = { "querytype": "conesearch", "query": { "objectcoordinates": { "conesearchradius": 2, "conesearchunit": "arcsec", "radec": { "ZTF20acfkzcg": [ 115.7697847, 50.2887778 ] } }, "catalogs": { "ZTFalerts": { "filter": {}, "projection": { "id": 0, "candid": 1, "objectId": 1 } } } }, "kwargs": { "filterfirst": False } }

response = kowalski.query(query=query) data = response.get("default").get("data") ```

Run a find query:

```python q = { "querytype": "find", "query": { "catalog": "ZTFalerts", "filter": { "objectId": "ZTF20acfkzcg" }, "projection": { "_id": 0, "candid": 1 } } }

response = kowalski.query(query=q) data = response.get("default").get("data") ```

Run a batch of queries in parallel:

```python queries = [ { "querytype": "find", "query": { "catalog": "ZTFalerts", "filter": { "candid": alert["candid"] }, "projection": { "_id": 0, "candid": 1 } } } for alert in data ]

responses = k.query(queries=queries, usebatchquery=True, maxnthreads=4) ```

Querying multiple instances at once

When using multiple instances at once, you can specify a single instance to query using its name when calling query(name=...), or no name at all. If no name is provided, and the catalog(s) being queried is/are available on multiple instances, penquins will divide the load between instances automagically.

When retrieving the results, you'll have to use the instance(s) name instead of "default", or simply iterate over the results by instance and merge the results.

Any of the queries mentioned for single instance querying also work here.

Examples

No instance name specified:

```python q = { "querytype": "find", "query": { "catalog": "ZTFalerts", "filter": { "objectId": "ZTF20acfkzcg" }, "projection": { "id": 0, "candid": 1 } } } response = kowalski.query(query=q) data = response.get(<instancename).get("data") # retrieving data from one instance

OR

data = [] # or {} depending on the query's expected result, differs by query type for instance, instanceresults in response.items(): for result in instanceresults: data.append(result.get('data')) ```

Instance name specified: python q = { "query_type": "find", "query": { "catalog": "ZTF_alerts", "filter": { "objectId": "ZTF20acfkzcg" }, "projection": { "_id": 0, "candid": 1 } } } response = kowalski.query(query=q, name=<instance_name>) data = response.get(<instance_name).get("data") # retrieving data from one instance

Interacting with the API

Users can interact with Kowalski's API in a more direct way using the Kowalski.api method.

Users with admin privileges can add/remove users to/from the system:

```python username = "noone" password = "nopas!" email = "user@caltech.edu"

request = { "username": username, "password": password, "email": email }

response = kowalski.api(method="post", endpoint="/api/users", data=request)

response = kowalski.api(method="delete", endpoint=f"/api/users/{username}") ```

Publish new version

Please refer to https://realpython.com/pypi-publish-python-package/ for a detailed guide.

```shell script pip install bumpversion export PENQUINS_VERSION=2.4.2

bumpversion --current-version $PENQUINSVERSION minor setup.py penquins/penquins.py python setup.py sdist bdistwheel

twine check dist/$PENQUINS_VERSION twine upload dist/$PENQUINS_VERSION

username: token token: ```