Science Score: 13.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (9.3%) to scientific vocabulary
Last synced: 11 months ago
·
JSON representation
Repository
Python wrapper for libaio
Basic Info
- Host: GitHub
- Owner: vpelletier
- License: lgpl-3.0
- Language: Python
- Default Branch: master
- Size: 128 KB
Statistics
- Stars: 19
- Watchers: 2
- Forks: 6
- Open Issues: 0
- Releases: 0
Created over 8 years ago
· Last pushed over 2 years ago
Metadata Files
Readme
License
README.rst
Linux AIO API wrapper
This is about in-kernel, file-descriptor-based asynchronous I/O.
It has nothing to do with the ``asyncio`` standard module.
Linux AIO primer
----------------
When sending or expecting data, the typical issue a developer faces is knowing
when the operation will complete, so the program can carry on.
- read/write/recv/send: blocks until stuff happened
- same, on a non-blocking file descriptor: errors out instead of blocking,
developper has to implement retry somehow, and may end up wasting CPU time
just resubmitting the same operation over and over.
- select/poll/epoll: kernel tells the program when (re)submitting an operation
should not block (if developer is careful to not have competing IO sources)
AIO is the next level: the application expresses the intention that some IO
operation happens when the file descriptor accepts it *and* provides
corresponding buffer to the kernel.
Compared to select/poll/epoll, this avoids one round-trip to userland when the
operation becomes possible:
- kernel sends notification (ex: fd is readable)
- program initiates actual IO (ex: read from fd)
Instead, kernel only has to notify userland the operation is already completed,
and application may either process received data, or submit more data to send.
Edge cases
----------
Because of this high level of integration, low-level implementation
constraints which are abstracted by higher-overhead APIs may become apparent.
For example, when submitting AIO blocks to an USB gadget endpoint file, the
block should be aligned to page boundaries because some USB Device Controllers
do not have the ability to read/write partial pages.
In python, this means ``mmap`` should be used to allocate such buffer instead
of just any ``bytearray``.
Another place where implementation details appear is completion statuses,
``res`` and ``res2``. Their meaning depends on the module handling operations
on used file descriptor, so python-libaio transmits these values without
assuming their meaning (rather than, say, raise on negative values).
Yet another place is application-initiated closures: there is a fundamental
race-condition when cancelling an AIO block (maybe hardware-triggered
completion will happen first, or maybe software-initiated cancellation will).
In any case, a completion event will be produced and application may check
which origin won. A consequence of this is that AIO context closure may take
time: while requesting cancellation does not block, software should wait for
hardware to hand the buffers back.
python 2 Notes
--------------
In python 2.7, a memoryview of a bytearray, despite being writable, is rejected
by ctypes:
.. code:: python
>>> from ctypes import c_char
>>> a = bytearray(b'foo')
>>> c_char.from_buffer(a)
c_char('f')
>>> b = memoryview(a)
>>> b.readonly
False
>>> c_char.from_buffer(b)
Traceback (most recent call last):
File "", line 1, in
TypeError: expected a writeable buffer object
This means that it is not possible to only read or write a few bytes at the
beginning of a large buffer without having to copy memory.
The same code works fine with python 3.x .
This is considered a python 2.7 ctypes or memoryview bug, and not a python-libaio bug.
Also, memoryview refuses to use an mmap object:
.. code:: python
>>> import mmap
>>> a = mmap.mmap(-1, 16*1024)
>>> b = memoryview(a)
Traceback (most recent call last):
File "", line 1, in
TypeError: cannot make memory view because object does not have the buffer interface
>>>
...but ctypes is happy with it:
.. code:: python
>>> import ctypes
>>> c = (ctypes.c_char * len(a)).from_buffer(a)
>>>
...and memoryview accepts being constructed over ctype objects:
.. code:: python
>>> d = memoryview(c)
>>>
...and it really works !
.. code:: python
>>> a[0]
'\x00'
>>> c[0]
'\x00'
>>> d[0]
'\x00'
>>> d[0] = '\x01'
>>> c[0]
'\x01'
>>> a[0]
'\x01'
>>> a[0] = '\x02'
>>> c[0]
'\x02'
>>> d[0]
'\x02'
This is considered a python 2.7 memoryview or mmap bug.
Owner
- Name: Vincent Pelletier
- Login: vpelletier
- Kind: user
- Location: Japan
- Repositories: 31
- Profile: https://github.com/vpelletier
GitHub Events
Total
- Watch event: 2
Last Year
- Watch event: 2
Committers
Last synced: almost 3 years ago
Top Committers
| Name | Commits | |
|---|---|---|
| Vincent Pelletier | p****t@g****m | 80 |
| Shangyan Zhou | s****u@h****m | 1 |
| Mark Harfouche | m****e@g****m | 1 |
Issues and Pull Requests
Last synced: 11 months ago
All Time
- Total issues: 11
- Total pull requests: 4
- Average time to close issues: 20 days
- Average time to close pull requests: 1 day
- Total issue authors: 4
- Total pull request authors: 2
- Average comments per issue: 4.36
- Average comments per pull request: 3.5
- Merged pull requests: 1
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- hmaarrfk (5)
- pahome (4)
- ali1234 (1)
- anonhostpi (1)
Pull Request Authors
- hmaarrfk (3)
- sphish (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 2
-
Total downloads:
- pypi 9,055 last-month
- Total docker downloads: 34
-
Total dependent packages: 0
(may contain duplicates) -
Total dependent repositories: 5
(may contain duplicates) - Total versions: 22
- Total maintainers: 1
pypi.org: libaio
Linux AIO API wrapper
- Homepage: http://github.com/vpelletier/python-libaio
- Documentation: https://libaio.readthedocs.io/
- License: LGPLv3+
-
Latest release: 0.9.3
published over 2 years ago
Rankings
Downloads: 4.8%
Dependent repos count: 6.7%
Dependent packages count: 10.0%
Average: 10.2%
Forks count: 14.2%
Stargazers count: 15.2%
Maintainers (1)
Last synced:
11 months ago
conda-forge.org: python-libaio
Linux AIO API wrapper This is about in-kernel, file-descriptor-based asynchronous I/O. It has nothing to do with the asyncio standard module.
- Homepage: https://github.com/vpelletier/python-libaio
- License: LGPL-3.0-or-later
-
Latest release: 0.9.1
published over 4 years ago
Rankings
Dependent repos count: 34.0%
Average: 45.6%
Forks count: 47.7%
Stargazers count: 49.6%
Dependent packages count: 51.2%
Last synced:
11 months ago
Dependencies
setup.py
pypi