Data

Tools

Indexes

Applications

Experiments

Open Source Science

pifpaf

Python fixtures and daemon managing tools for functional testing

https://github.com/jd/pifpaf

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (8.5%) to scientific vocabulary

Keywords

aodh ceph consule couchdb elasticsearch etcd fakes3 fixtures gnocchi influxdb keystone mongodb mysql openstack postgresql python redis s3rver swift zookeeper

Keywords from Contributors

asyncio graph-theory json-serialization
Last synced: 6 months ago · JSON representation

Repository

Python fixtures and daemon managing tools for functional testing

Basic Info
  • Host: GitHub
  • Owner: jd
  • License: apache-2.0
  • Language: Python
  • Default Branch: master
  • Size: 801 KB
Statistics
  • Stars: 194
  • Watchers: 9
  • Forks: 41
  • Open Issues: 19
  • Releases: 5
Topics
aodh ceph consule couchdb elasticsearch etcd fakes3 fixtures gnocchi influxdb keystone mongodb mysql openstack postgresql python redis s3rver swift zookeeper
Created almost 10 years ago · Last pushed about 1 year ago
Metadata Files
Readme License

README.rst 

==========
 Pifpaf
==========

.. image:: https://badge.fury.io/py/pifpaf.svg
    :target: https://badge.fury.io/py/pifpaf

Pifpaf is a suite of `fixtures`_ and a command-line tool that allows to start
and stop daemons for a quick throw-away usage. This is typically useful when
needing these daemons to run `integration testing`_. It originaly evolved from
its precussor `overtest`_.

.. _fixtures: https://pypi.python.org/pypi/fixtures
.. _overtest: https://github.com/jd/overtest

Supported daemons
=================

Pifpaf currently supports:

* `PostgreSQL`_
* `MySQL`_
* `Memcached`_
* `InfluxDB`_
* `Etcd`_ (with clustering)
* `Redis`_ (with sentinel mode)
* `Valkey`_
* `Elasticsearch`_
* `ZooKeeper`_
* `Gnocchi`_
* `Aodh`_
* `Ceph`_
* `RabbitMQ`_ (with clustering)
* `FakeS3`_
* `Consul`_
* `Keystone`_
* `CouchDB`_
* `S3rver`_
* `MongoDB`_
* `OpenStack Swift`_
* `Vault`_
* `HttpBin`_

.. _Consul: https://www.consul.io/
.. _PostgreSQL: http://postgresql.org
.. _MySQL: http://mysql.org
.. _Memcached: http://memcached.org
.. _InfluxDB: http://influxdb.org
.. _Etcd: https://coreos.com/etcd/
.. _Redis: http://redis.io/
.. _Valkey: https://valkey.io/
.. _Elasticsearch: https://www.elastic.co/
.. _ZooKeeper: https://zookeeper.apache.org/
.. _Gnocchi: http://gnocchi.xyz
.. _Aodh: http://launchpad.net/aodh
.. _Ceph: http://ceph.com
.. _RabbitMQ: https://www.rabbitmq.com/
.. _FakeS3: https://github.com/jubos/fake-s3
.. _Keystone: https://launchpad.net/keystone
.. _CouchDB: http://couchdb.apache.org/
.. _S3rver: https://www.npmjs.com/package/s3rver
.. _MongoDB: https://www.mongodb.com
.. _OpenStack Swift: https://docs.openstack.org/developer/swift/
.. _Vault: https://www.vaultproject.io/
.. _HttpBin: https://httpbin.org/

Usage
=====
To use Pifpaf, simply call the `pifpaf run $daemon ` program that you
need. It will setup the temporary environment and export a few environment
variable for you to access it::

  $ pifpaf run postgresql psql template1
  Expanded display is used automatically.
  Line style is unicode.
  SET
  psql (9.4.5)
  Type "help" for help.

  template1=# \l
                                List of databases
     Name    │ Owner │ Encoding │   Collate   │    Ctype    │ Access privileges
  ───────────┼───────┼──────────┼─────────────┼─────────────┼───────────────────
   postgres  │ jd    │ UTF8     │ en_US.UTF-8 │ en_US.UTF-8 │
   template0 │ jd    │ UTF8     │ en_US.UTF-8 │ en_US.UTF-8 │ =c/jd            ↵
             │       │          │             │             │ jd=CTc/jd
   template1 │ jd    │ UTF8     │ en_US.UTF-8 │ en_US.UTF-8 │ =c/jd            ↵
             │       │          │             │             │ jd=CTc/jd
  (3 rows)

  template1=# \q
  $

You can also run it with no command line provided::

  $ eval `pifpaf run memcached`
  $ env | grep PIFPAF
  PIFPAF_PID=13387
  PIFPAF_DAEMON=memcached
  PIFPAF_URL=memcached://localhost:11212
  PIFPAF_MEMCACHED_URL=memcached://localhost:11212
  $ pifpaf_stop

Killing the daemon whose PID is contained in `$PIFPAF_PID` will stop the
launched daemon and clean the test environment. You can kill it yourself or use
the defined function `pifpaf_stop`.

Environment variables
=====================
Pifpaf exports a few environment variable:

* `PIFPAF_DAEMON` which contains the name of the daemon launched
* `PIFPAF_URL` which contains the URL to the daemon
* `PIFPAF_PID` the PID of the pifpaf daemon
* `PIFPAF_$daemon_*` variables, which contains daemon specific variables,
  such as port, database name, URL, etc.

.. _integration testing: https://en.wikipedia.org/wiki/Integration_testing


Running several programs at once
================================
Pifpaf provides the ability to change the prefix of its environment variable,
allowing you to nest several Pifpaf instances and therefore running several
daemons at once::

  $ pifpaf --env-prefix STORAGE run memcached -- pifpaf --env-prefix INDEX run postgresql $SHELL
  $ env | grep STORAGE
  STORAGE_DATA=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmpVreJ0J
  STORAGE_MEMCACHED_PORT=11212
  STORAGE_URL=memcached://localhost:11212
  STORAGE_PID=71019
  STORAGE_DAEMON=memcached
  STORAGE_MEMCACHED_URL=memcached://localhost:11212
  $ env | grep INDEX
  INDEX_DATA=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmphAG7tf
  INDEX_URL=postgresql://localhost/postgres?host=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmphAG7tf&port=9824
  INDEX_PID=71021
  INDEX_DAEMON=postgresql
  INDEX_POSTGRESQL_URL=postgresql://localhost/postgres?host=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmphAG7tf&port=9824
  $ echo $PIFPAF_URLS
  memcached://localhost:11212;postgresql://localhost/postgres?host=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmpQ2BWFH&port=9824

The `PIFPAF_URLS` environment variable will contain the list of all URLs
detected and set-up by Pifpaf. You can override this variable name with the
`--global-urls-variable` option.

How it works under the hood
===========================

Pifpaf will start the asked daemon using the current Posix user. The data file
of the daemon will be placed in a temporary directory. The system-wide
configured daemon that might exists is not touched at all.

Pifpaf expected to find daemon binaries on your system (like `mysql`, `mysqld`,
`pg_config`, `pg_ctl`, `rabbitmq-server`, etc).

When the Python fixture is cleaned or when Pifpaf is terminated, the daemon is
stopped and the temporary directory removed.

.. image:: pifpaf.jpg

Owner

  • Name: Julien Danjou
  • Login: jd
  • Kind: user
  • Location: Toulouse, France
  • Company: @Mergifyio

Open Source Software Engineer 💻, Runner 🏃🏻‍♂️ Foodie 🍲 FPS player 🎯

GitHub Events

Total
  • Release event: 1
  • Watch event: 3
  • Issue comment event: 2
  • Push event: 6
  • Pull request event: 12
  • Pull request review event: 3
  • Fork event: 2
  • Create event: 1
Last Year
  • Release event: 1
  • Watch event: 3
  • Issue comment event: 2
  • Push event: 6
  • Pull request event: 12
  • Pull request review event: 3
  • Fork event: 2
  • Create event: 1

Committers

Last synced: 9 months ago

All Time
  • Total Commits: 395
  • Total Committers: 26
  • Avg Commits per committer: 15.192
  • Development Distribution Score (DDS): 0.461
Past Year
  • Commits: 17
  • Committers: 7
  • Avg Commits per committer: 2.429
  • Development Distribution Score (DDS): 0.588
Top Committers
Name Email Commits
Julien Danjou j****n@d****o 213
Mehdi Abaakouk s****t@s****t 85
Tobias Urdin t****n@b****e 25
Felix Yan f****s@a****g 16
Junpei Kawamoto k****i@g****m 10
Kainami Takashi k****t 8
ajssmith a****h@r****m 7
gord chung g****d@l****a 5
Eli Schwartz e****3@g****m 4
Moisés Guimarães de Medeiros g****s@p****e 4
Rafael Weingärtner r****l@a****g 2
David Douard d****d@s****g 2
Daniel Zagaynov k****y@a****g 1
Rémy HUBSCHER r****r@m****m 1
Anna c****v@s****n 1
Carl Smedstad c****d@p****m 1
Davanum Srinivas d****m@g****m 1
Elancheran Subramanian e****n@g****m 1
Joshua Harlow h****a@g****m 1
Kenneth Giusti k****i@r****m 1
Matt Wheeler m@f****g 1
Michael Bayer m****p@z****m 1
Omar Trigui o****n@g****m 1
Sam Morrison s****n@g****m 1
mergify[bot] 3****] 1
ruy r****y@a****t 1
Committer Domains (Top 20 + Academic)
redhat.com: 2 ainoniwa.net: 1 zzzcomputing.com: 1 funkyhat.org: 1 godaddy.com: 1 sysrq.in: 1 mozilla.com: 1 altlinux.org: 1 sdfa3.org: 1 apache.org: 1 pm.me: 1 live.ca: 1 archlinux.org: 1 binero.se: 1 sileht.net: 1 danjou.info: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 28
  • Total pull requests: 111
  • Average time to close issues: about 1 year
  • Average time to close pull requests: about 1 month
  • Total issue authors: 21
  • Total pull request authors: 25
  • Average comments per issue: 2.89
  • Average comments per pull request: 1.65
  • Merged pull requests: 87
  • Bot issues: 0
  • Bot pull requests: 1
Past Year
  • Issues: 0
  • Pull requests: 19
  • Average time to close issues: N/A
  • Average time to close pull requests: 1 day
  • Issue authors: 0
  • Pull request authors: 5
  • Average comments per issue: 0
  • Average comments per pull request: 0.16
  • Merged pull requests: 9
  • Bot issues: 0
  • Bot pull requests: 1
View more stats
Top Authors
Issue Authors
  • felixonmars (5)
  • Alessandro-Barbieri (2)
  • zzzeek (2)
  • ajssmith (2)
  • douardda (2)
  • cristicalin (1)
  • CyberTailor (1)
  • jd (1)
  • vytas7 (1)
  • jdandrea (1)
  • sinemetu1 (1)
  • sfermigier (1)
  • arthurzenika (1)
  • maximumG (1)
  • chungg (1)
Pull Request Authors
  • tobias-urdin (22)
  • sileht (22)
  • kajinamit (16)
  • CyberTailor (12)
  • jd (12)
  • ajssmith (7)
  • eli-schwartz (5)
  • moisesguimaraes (4)
  • felixonmars (4)
  • chungg (4)
  • jkawamoto (3)
  • maeln (2)
  • zzzeek (2)
  • mergify[bot] (2)
  • douardda (2)
Top Labels
Issue Labels
enhancement (2) help wanted (2) invalid (1) bug (1)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 5,669 last-month
  • Total docker downloads: 24
  • Total dependent packages: 10
  • Total dependent repositories: 41
  • Total versions: 127
  • Total maintainers: 3
pypi.org: pifpaf

Suite of tools and fixtures to manage daemons for testing

  • Versions: 127
  • Dependent Packages: 10
  • Dependent Repositories: 41
  • Downloads: 5,669 Last month
  • Docker Downloads: 24
Rankings
Dependent packages count: 1.0%
Dependent repos count: 2.3%
Downloads: 3.0%
Docker downloads count: 3.6%
Average: 3.7%
Stargazers count: 5.1%
Forks count: 7.0%
Maintainers (3)
jd sileht tobias-urdin
Last synced: 6 months ago

Dependencies

requirements.txt pypi
  • click *
  • daiquiri *
  • fixtures *
  • jinja2 *
  • pbr *
  • psutil *
  • xattr *
Dockerfile docker
  • ubuntu focal build