hpi - v0.6.20250816

What's Changed

core: change get_files handling for sort argument + clean up tests by @karlicoss in https://github.com/karlicoss/HPI/pull/452
general: use full dependency specifiers (i.e. "package-name @ git+htps://..." in REQUIRES) by @karlicoss in https://github.com/karlicoss/HPI/pull/448
core: deprecate my.core.assert_never and my.core.LazyLogger by @karlicoss in https://github.com/karlicoss/HPI/pull/441
core: autodetect uv in hpi module install by @karlicoss in https://github.com/karlicoss/HPI/pull/445
core: improve uv detection in hpi module install by @karlicoss in https://github.com/karlicoss/HPI/pull/447
core/error: fix types in extracterrordatetime, allow returning date by @karlicoss in https://github.com/karlicoss/HPI/pull/450

Modules:

my.twitter.android: fix own user id guessing by @karlicoss in https://github.com/karlicoss/HPI/pull/442
my.bluemaestro: big speedups for data processing, switch to ZoneInfo by @karlicoss in https://github.com/karlicoss/HPI/pull/454

Chores/CI improvements

ty: type check core with ty by @karlicoss in https://github.com/karlicoss/HPI/pull/444
minor cleanup for tox.init and pytest.ini by @karlicoss in https://github.com/karlicoss/HPI/pull/440
chore: bring CI configs up to date, set up coverage by @karlicoss in https://github.com/karlicoss/HPI/pull/443
chore: update ty version/suppress errors and remove useless suppressions by @karlicoss in https://github.com/karlicoss/HPI/pull/446
chore: improve type annotations for better ty support by @karlicoss in https://github.com/karlicoss/HPI/pull/449
ty: add tox section for ty-all and cleanup type checking sections by @karlicoss in https://github.com/karlicoss/HPI/pull/451
various modules: minor cleanup/formatting by @karlicoss in https://github.com/karlicoss/HPI/pull/453

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.6.20250623...v0.6.20250816

- Python
Published by karlicoss 6 months ago

hpi - v0.6.20250623: hotfix for broken pypi wheel

What's Changed

fix hatch build by explicitly specifying directories to include and excluding my/ symlink by @karlicoss in https://github.com/karlicoss/HPI/pull/439
general: sort out F401 (unused variable) warnings in ruff everywhere by @karlicoss in https://github.com/karlicoss/HPI/pull/438

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.6.20250622...v0.6.20250623

- Python
Published by karlicoss 8 months ago

hpi - v0.6.20250622: rolling release

What's Changed

General

Some notable upgrades -- shouldn't break anything for existing users, but worth highlighting: * general: switch from flat layout to src layout by @karlicoss in https://github.com/karlicoss/HPI/pull/414 If something isn't working (even though I made sure everything is compatible), best to reinstall your HPI * core.common: use kompress directly instead of attempting to import previously vendorised version by @karlicoss in https://github.com/karlicoss/HPI/pull/422 * kompress: get rid of vendorised version by @karlicoss in https://github.com/karlicoss/HPI/pull/417 If something isn't working, make sure to run pip install kompress * core: migrate from deprecated appdirs to platformdirs by @karlicoss in https://github.com/karlicoss/HPI/pull/418 If something isn't working, run pip install platformdirs

Other minor changes: * ci: bring pytest config/pyproject/tox up to date/switch to hatch for building package by @karlicoss in https://github.com/karlicoss/HPI/pull/437 * ci: switch to pyproject.toml and update other CI files by @karlicoss in https://github.com/karlicoss/HPI/pull/416 * general: use Exception.addnote instead of hacky my.core.error.echain by @karlicoss in https://github.com/karlicoss/HPI/pull/421 * my.core.experimental: some fixes from importoriginalmodule by @karlicoss in https://github.com/karlicoss/HPI/pull/433 * general: remove unnecessary _init__.py in tests/ subpackages by @karlicoss in https://github.com/karlicoss/HPI/pull/434

Core

core: add my.core.json.json_loads, use orjson if available, otherwise fall back to stdlib json by @karlicoss in https://github.com/karlicoss/HPI/pull/425
core: migrate code to benefit from 3.9 stuff by @karlicoss in https://github.com/karlicoss/HPI/pull/401
general: migrate modules to use 3.9 features by @karlicoss in https://github.com/karlicoss/HPI/pull/402
minor updates: remove assert_subpackage and clean up some ruff by @karlicoss in https://github.com/karlicoss/HPI/pull/420

Modules

my.whatsapp.android: adapt to new db format by @karlicoss in https://github.com/karlicoss/HPI/pull/403
fix typo in rexport DAL by @srajangarg in https://github.com/karlicoss/HPI/pull/405
smscalls: add phone number to model by @purarue in https://github.com/karlicoss/HPI/pull/410
smscalls: allow large XML files as input by @purarue in https://github.com/karlicoss/HPI/pull/411
core.sqlite: add helper SqliteTool to get table schemas by @karlicoss in https://github.com/karlicoss/HPI/pull/412
twitter.android: fix getownuser_id for latest exports by @karlicoss in https://github.com/karlicoss/HPI/pull/413
instagram.gdpr: some cleanup and properly extract threadv2id by @karlicoss in https://github.com/karlicoss/HPI/pull/423
instagram.android: improve thread id extraction and add more comments by @karlicoss in https://github.com/karlicoss/HPI/pull/424
my.lastfm: some cleanup by @karlicoss in https://github.com/karlicoss/HPI/pull/426
new module: podcastaddict android app by @karlicoss in https://github.com/karlicoss/HPI/pull/429
fbmessenger.android: fix missing senders, fallback on empty name by @karlicoss in https://github.com/karlicoss/HPI/pull/430
my.bluemaestro: support new bmLogger app by @karlicoss in https://github.com/karlicoss/HPI/pull/431
my.bumble.android: some cleanup + switch to Protocol-based config by @karlicoss in https://github.com/karlicoss/HPI/pull/432
twitter.android: handle empty databases by @karlicoss in https://github.com/karlicoss/HPI/pull/435

Misc

chore: update urls by @purarue in https://github.com/karlicoss/HPI/pull/404
doc: spelling fixes by @purarue in https://github.com/karlicoss/HPI/pull/408
doc: some performance notes for query_range by @purarue in https://github.com/karlicoss/HPI/pull/409
src/ doc links by @PatMyron in https://github.com/karlicoss/HPI/pull/415
general: update mypy rules and fix by @karlicoss in https://github.com/karlicoss/HPI/pull/419
fix ruff error by @karlicoss in https://github.com/karlicoss/HPI/pull/428
ci: sync ruff rules with base config and fix warnings by @karlicoss in https://github.com/karlicoss/HPI/pull/436
workaround to make pip install hpi[testing] work while dependency group support is missing in pip by @karlicoss in https://github.com/karlicoss/HPI/pull/427

New Contributors

@srajangarg made their first contribution in https://github.com/karlicoss/HPI/pull/405
@PatMyron made their first contribution in https://github.com/karlicoss/HPI/pull/415

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.5.20241019...v0.6.20250622

- Python
Published by karlicoss 8 months ago

hpi - v0.5.20241019: switch min version to 3.9

What's Changed

general: python3.9 reached EOL, switch min version by @karlicoss in https://github.com/karlicoss/HPI/pull/400

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.5.20240924...v0.5.20241019

- Python
Published by karlicoss over 1 year ago

hpi - hotfix for my.core.time

What's Changed

core.time: hotfix for default force_abbreviations attribute by @karlicoss in https://github.com/karlicoss/HPI/pull/399

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.5.20240923...v0.5.20240924

- Python
Published by karlicoss over 1 year ago

hpi - rolling release

What's Changed

core

my.core: fix list constructor in alwayssupportsequence and add some tests by @karlicoss in https://github.com/karlicoss/HPI/pull/395
my.core.structure: add support for .tar.gz archives by @karlicoss in https://github.com/karlicoss/HPI/pull/390

modules

my.google.takeout.parser: speedup event merging on newer googletakeoutparser versions by @karlicoss in https://github.com/karlicoss/HPI/pull/389
my.github.gdpr/my.zulip.organization: use kompress support for tar.gz if it's available by @karlicoss in https://github.com/karlicoss/HPI/pull/392
my.fbmessenger.android: exclude unsent messages to avoid duplication by @karlicoss in https://github.com/karlicoss/HPI/pull/393
migrate bluemaestro and pdfs configs to use Protocol and @property by @karlicoss in https://github.com/karlicoss/HPI/pull/384
migrate some modules to 'lazy config' pattern and clean up tests by @karlicoss in https://github.com/karlicoss/HPI/pull/386
small fixes for my.twitter.android, dedupicate tweets in my.twitter.archive by @karlicoss in https://github.com/karlicoss/HPI/pull/394
my.youtube.takeout: deduplicate watched videos and sort out a few minor errors by @karlicoss in https://github.com/karlicoss/HPI/pull/397

misc

update ruff config, enable/suppress some checks by @karlicoss in https://github.com/karlicoss/HPI/pull/387
ci: update mypy config and make ruff config more consistent with other projects by @karlicoss in https://github.com/karlicoss/HPI/pull/388
tox: some prep for potentially using uv on CI instead of pip by @karlicoss in https://github.com/karlicoss/HPI/pull/396
tox: try using uv for CI, should result in speedup by @karlicoss in https://github.com/karlicoss/HPI/pull/398

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.5.20240824...v0.5.20240923

- Python
Published by karlicoss over 1 year ago

hpi - rolling release

What's Changed

Mostly big cleanup & deprecations for core.common module https://github.com/karlicoss/HPI/pull/380 https://github.com/karlicoss/HPI/pull/382

New core dependency: typing-extensions

Lots of cleanup/deprecations in core. This mostly shouldn't break any runtime code, but mypy might complain -- so best to migrate the imports appropriately

core.common: deprecate outdated LazyLogger alias
core.common: move Json, datetimeaware, datetimenaive, is_namedtuple, asdict to my.core.types
my.core: deprecate Path/dataclass imports from my.core during type checking

runtime still works for backwards compatibility

core.common: move assert_subpackage to my.core.internal
core.common: move mime-related stuff to my.core.mime

no backward compat, unlikely it was used by anyone else

core.common: move stats-related stuff to my.core.stats and add more thorough tests/docs

deprecate core.common.stat and core.common.Stats with backwards compatibility

core: cleanup my.core.common.unique_everseen
- move to my.core.utils.itertools
- more robust check for hashable types -- now checks in runtime (since the one based on types purely isn't necessarily sound)
- add more testing
my.core.common: move warnifempty to my.core.utils.itertools, cleanup and add more tests
core.common: move away import related stuff to my.core.utils.imports

moving without backward compatibility, since it's extremely unlikely they are used for any external modules

in fact, unclear if these methods still have much value at all, but keeping for now just in case

core.common: move listify to core.utils.itertools, use better typing annotations for it

also some minor refactoring of my.rss

core: cleanup itertool style helpers
- deprecate groupbykey, should use itertool.bucket instead
- move makedict and ensureunique to my.core.utils.itertools
core: cleanup deprecations, exclude from type checking and show runtime warnings

among affected things:

core.common.assert_never
core.common.cproperty
core.common.isoparse
core.common.mcachew
core.common.the
core.common.tzdatetime
core.compat.sqlite_backup
- core: cleanup deprecations, exclude from type checking and show runtime warnings
among affected things: - core.common.assertnever - core.common.cproperty - core.common.isoparse - core.common.mcachew - core.common.the - core.common.tzdatetime - core.compat.sqlitebackup

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.4.20240810...v0.5.20240824

- Python
Published by karlicoss over 1 year ago

hpi - rolling release

What's Changed

fix for new instagram gdpr format by @karlicoss in https://github.com/karlicoss/HPI/pull/371
fix legacy google takeout by @karlicoss in https://github.com/karlicoss/HPI/pull/372
fbmessenger.android: fix minor issue with processing thread participants by @karlicoss in https://github.com/karlicoss/HPI/pull/373
fix my.twitter.android by @karlicoss in https://github.com/karlicoss/HPI/pull/374
tests: use updated conftest from pymplate, this allows to run individual test modules properly by @karlicoss in https://github.com/karlicoss/HPI/pull/375
improve testing consistency, simplify tox by @karlicoss in https://github.com/karlicoss/HPI/pull/376
replace deprecated utcnow and utcfromtimestamp by @karlicoss in https://github.com/karlicoss/HPI/pull/378

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.4.20240506...v0.4.20240810

- Python
Published by karlicoss over 1 year ago

hpi - rolling release

What's Changed

monzo.monzoexport: initial module by @karlicoss in https://github.com/karlicoss/HPI/pull/346
time.tz.via_location: more consistent behaviour wrt caching by @karlicoss in https://github.com/karlicoss/HPI/pull/351
my.codeforces/my.topcoder: move to the top level & cleanup by @karlicoss in https://github.com/karlicoss/HPI/pull/352
my.body.sleep: massive speedup for average temperature calculation by @karlicoss in https://github.com/karlicoss/HPI/pull/353
gpslogger: catch broken xml file error by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/356
initial module for official twitter app for android by @karlicoss in https://github.com/karlicoss/HPI/pull/359
my.youtube.takeout: fix exception handling by @karlicoss in https://github.com/karlicoss/HPI/pull/361
fbmessenger.android: expose contacts by @karlicoss in https://github.com/karlicoss/HPI/pull/362
whatsapp.android: expose all entities extracted from the db by @karlicoss in https://github.com/karlicoss/HPI/pull/363
google takeout: support multiple locales, resolves #330 by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/364
my.google.maps: intitial module for extracting saved places data from Android app by @karlicoss in https://github.com/karlicoss/HPI/pull/365
my.twitter.android: better detection of own user id by @karlicoss in https://github.com/karlicoss/HPI/pull/368
my.tinder.android: better error handing and fix case with empty db by @karlicoss in https://github.com/karlicoss/HPI/pull/369
smscalls: parse mms from smscalls export by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/370

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.4.20231102...v0.4.20240506

- Python
Published by karlicoss over 1 year ago

hpi - v0.4.20231102, fix release script

not ideal that the date in the version is the future date.. but oh well

What's Changed

ci: fix wrong release command by @karlicoss in https://github.com/karlicoss/HPI/pull/344

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.4.20231101...v0.4.20231102

- Python
Published by karlicoss over 2 years ago

hpi - v0.4.20231101

Big update, even though things should be backwards compatible, it's recommended to install the package again if you use editable git checkout (pip3 install --user -e /path/to/hpi_repo)

Also worth installing optional dependencies (pip3 install --user -e /path/to/hpi_repo[optional])

What's Changed

Core: * core: implement more methods for ZipPath and better support for getfiles by @karlicoss in https://github.com/karlicoss/HPI/pull/285 * query: add --warn-exceptions, dateparser, docs by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/290 * my.core.pandas: rely on typing annotations from types-pandas by @karlicoss in https://github.com/karlicoss/HPI/pull/292 * core: experimental import of my.inithook to configure logging/warnings/env variables by @karlicoss in https://github.com/karlicoss/HPI/pull/298 * core/logging: overhaul and many improvements -- mainly to deprecate abandoned logzero by @karlicoss in https://github.com/karlicoss/HPI/pull/299 * minor module updates + logging/warnings improvements by @karlicoss in https://github.com/karlicoss/HPI/pull/300 * general: move reddit tests into my/tests + tweak my.core.cfg to be more reliable by @karlicoss in https://github.com/karlicoss/HPI/pull/295 * core/cli: allow user to bypass PEP 668 by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/302 * core/cli: update vendored completion files by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/304 * core/logging: fix issue with logger setup called multiple times when called with different levels by @karlicoss in https://github.com/karlicoss/HPI/pull/310 * my.core.logging: compatibility with HPILOGS by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/307 * core/stats: exclude contextmanagers from guessstats by @karlicoss in https://github.com/karlicoss/HPI/pull/334 * move hpi specific compat stuff to core.hpicompate + minor updates for hypothesis and kobo by @karlicoss in https://github.com/karlicoss/HPI/pull/335 * core/config: implement a warning if config is imported from the dir other than MYCONFIG by @karlicoss in https://github.com/karlicoss/HPI/pull/336 * general: improve logging during file processing in various modules by @karlicoss in https://github.com/karlicoss/HPI/pull/339 * general: enhancle logging for various modules by @karlicoss in https://github.com/karlicoss/HPI/pull/340 * general: deprecate some old methods by hiding behind TYPECHECKING by @karlicoss in https://github.com/karlicoss/HPI/pull/342 * core: add helper for moreiterable to check that all types involved are hashable by @karlicoss in https://github.com/karlicoss/HPI/pull/343 * core/kompress: move vendorized kompress to _deprecated, use kompress library directly by @karlicoss in https://github.com/karlicoss/HPI/pull/319 * more consistent handling of zip archives in getfiles, less kompress boilerplate in modules by @karlicoss in https://github.com/karlicoss/HPI/pull/321 * experimental: add a hacky helper to import "original/shadowed" modules from within overlays by @karlicoss in https://github.com/karlicoss/HPI/pull/322 * core/stats: less duplication in output, report stats of first item, report input stats by @karlicoss in https://github.com/karlicoss/HPI/pull/324 * core.logging: ignore CollapseLogsHandler if we're not attached to a terminal by @karlicoss in https://github.com/karlicoss/HPI/pull/329 * dcotor: make compileall check a bit more defensive by @karlicoss in https://github.com/karlicoss/HPI/pull/332

modules: * my.whatsapp.android: initial module by @karlicoss in https://github.com/karlicoss/HPI/pull/284 * smscalls: make some fields optional by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/287 * cli: add option to output locations to gpx files by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/286 * smscalls: make checking for keys stricter by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/289 * browser: fix duplicate logs when fetching loglevel by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/309 * new module: Harmonic app for Hackernews by @karlicoss in https://github.com/karlicoss/HPI/pull/313 * location: make accuracy default config floats by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/314 * reddit.rexport: big cleanup + use new types from rexport.dal by @karlicoss in https://github.com/karlicoss/HPI/pull/320 * instagram: better normalising/error handling by @karlicoss in https://github.com/karlicoss/HPI/pull/325 * updates for whatsapp.android and tinder.android by @karlicoss in https://github.com/karlicoss/HPI/pull/326 * twitter.archive: deduplicate results via json.dumps by @karlicoss in https://github.com/karlicoss/HPI/pull/327 * reddit.rexport: experiment with using optional cpu pool (can potentially be used by other HPI modules) by @karlicoss in https://github.com/karlicoss/HPI/pull/328 * small zulip and dogsheep updates by @karlicoss in https://github.com/karlicoss/HPI/pull/331 * my.emfit: cleanup and pass cpu pool by @karlicoss in https://github.com/karlicoss/HPI/pull/333 * my.stackexchange.stexport: small cleanup & stat improvements by @karlicoss in https://github.com/karlicoss/HPI/pull/337 * fbmessenger.android: support processing msys database by @karlicoss in https://github.com/karlicoss/HPI/pull/341

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.3.20230327...v0.4.20231101

- Python
Published by karlicoss over 2 years ago

hpi - release

What's Changed

vkmessagesbackup updates by @karlicoss in https://github.com/karlicoss/HPI/pull/274
location fallback by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/263
add example for denylist, update ci by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/275
core.logging/core.kompress updates by @karlicoss in https://github.com/karlicoss/HPI/pull/277
google_takeout: add semantic location history by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/278
cli completion doc updates, hide legacy import warning by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/279
some updates for bumble/fbmessenger/twint by @karlicoss in https://github.com/karlicoss/HPI/pull/280
better unification of my.instagram and various enhancements by @karlicoss in https://github.com/karlicoss/HPI/pull/282
docs: fix typos by @kianmeng in https://github.com/karlicoss/HPI/pull/281
telgram_backup updates by @karlicoss in https://github.com/karlicoss/HPI/pull/283

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.3.20230221...v0.3.20230327

- Python
Published by karlicoss almost 3 years ago

hpi - release

What's Changed

my.telegram: initial module by @karlicoss in https://github.com/karlicoss/HPI/pull/270
my.fbmessenger.android updates by @karlicoss in https://github.com/karlicoss/HPI/pull/271
my.telegram.telegram_backup enhancements by @karlicoss in https://github.com/karlicoss/HPI/pull/273

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.3.20230209...v0.3.20230221

- Python
Published by karlicoss about 3 years ago

hpi - new release

What's Changed

my.fbmessenger.export: use context manager to properly close sqlite connection by @karlicoss in https://github.com/karlicoss/HPI/pull/268
core: better support for config overrides (tmp_config) by @karlicoss in https://github.com/karlicoss/HPI/pull/269

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.3.20230207...v0.3.20230209

- Python
Published by karlicoss about 3 years ago

hpi - v0.3.20230207

What's Changed

core/serialize: serialize stdlib Decimal class by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/259
google.takeout.parser: recreate cache on upgrade by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/261
switch modules from dataset to use raw sqlite3 by @karlicoss in https://github.com/karlicoss/HPI/pull/267
twitter archive + instapaper fixes by @karlicoss in https://github.com/karlicoss/HPI/pull/260
* docs: browser - fix examples for config by @seanbreckenridge in https://github.com/karlicoss/HPI/pull/258

Full Changelog: https://github.com/karlicoss/HPI/compare/v0.3.20220607...v0.3.20230207

- Python
Published by karlicoss about 3 years ago

hpi - hotfix so 'hpi module install' doesn't break for legacy modules

- Python
Published by karlicoss over 3 years ago

hpi - rolling release

mainly so hpi install --parallel is available

- Python
Published by karlicoss over 3 years ago

hpi - rolling release

mostly fixes for Windows and Python 3.10

- Python
Published by karlicoss almost 4 years ago

hpi - rolling release

new google takeout module!

- Python
Published by karlicoss almost 4 years ago

hpi - rolling release

- Python
Published by karlicoss almost 4 years ago

hpi - rolling release

- Python
Published by karlicoss about 4 years ago

hpi - rolling release

- Python
Published by karlicoss over 4 years ago

hpi - new release (see changelog for full changes)

`v0.3.20210220`

General/my.core changes:

a3305677b24694391a247fc4cb6cc1237e57f840 deprecate my.cfg, instead my.config can (and should be) used directly
0534c5c57dc420f9a01387b58a7098823e54277e new cli feature: module management

cli: add hpi module install and hpi module requires

relevant: https://github.com/karlicoss/HPI/issues/12, https://github.com/karlicoss/HPI/issues/79
97650adf3b48c653651b31c78cefe24ecae5ed4f add discovery_pure module to get modules and their dependencies via ast module
f90599d7e4463e936c8d95196ff767c730207202 make module discovery rely on =ast= module Hopefully it will make it more robust & much faster.
07f901e1e5fb2bd3009561c84cc4efd311c94733 helpers for automatic dataframes from sequences of NamedTuple/dataclass
4012f9b7c2a429170df8600591ec8d1e1407b162 more generic functions to jsonify data
746c3da0cadcba3b179688783186d8a0bd0999c5 core.pandas: allow specifying schema; add tests
5313984d8fea2b6eef6726b7b346c1f4316acd01 add tmp_config context manager for test & adhoc patching
df9a7f7390aee6c69f1abf1c8d1fc7659ebb957c core.pandas: add check for 'error' column + add empty one by default
e81dddddf083ffd81aa7e2b715bd34f59949479c proprely resolve class properties in make_config + add test

Modules: - some innitial work on filling InfluxDB with HPI data

pinboard
- 42399f6250d9901d93dcedcfe05f7857babcf834: breaking backwards compability, use pinbexport module directy
Use 'hpi module install my.pinboard' to install it

relevant: https://github.com/karlicoss/HPI/issues/79
stackexchange
- 63c825ab81bb561e912655e423c6b332fb6fd1b4 use GDPR data for votes
- ddea816a49f5da79fd6332e7f6b879b1955838af use proper pip package, add stat
bluemaestro
- 6d9bc2964b24cfe6187945f4634940673dfe9c27 populate grafana
- 1899b006de349140303110ca98a21d918d9eb049 investigation of data quality + more sanity checks
- d77ab92d8634d0863d2b966cb448bbfcc8a8d565 get rid of unnecessary file, move to top level
runnerup
- 6b451336ed5df2b893c9e6387175edba50b0719b Initial parser for RunnerUp data which I'm now using instead of Endomondo

Misc: - f102101b3917e8a38511faa5e4fd9dd33d284d7e core/windows: fix get_files and its tests - 56d5587c209dcbd27c7802d60c0bc8e8e2391672 CI: clean up tox config a bit, get rid of custom lint script - d562f00dca720fd4f6736377a41168e9a796c122

tests: run all tests, but exclude tests specific to my computer from CI
controllable via `HPI_TESTS_KARLICOSS=true`

improved mypy coverage

- Python
Published by karlicoss about 5 years ago

hpi - rolling release

General: - documentation improvements, my.config in the package now serves both as documentation & tests helper - my.kython: deprecated, moved to my.core - my.core.error: error handling enhancements - stat now supports pandas.DataFrame - minor enhancements to hpi doctor

Module changes: - new: my.time.tz: provides localize function to attach a timezone to an arbitrary date https://github.com/karlicoss/HPI/pull/96 - my.location: exposes a combined log of user's location, with fallback to 'default' home locations https://github.com/karlicoss/HPI/pull/95 https://github.com/karlicoss/HPI/pull/99 - my.fbmessenger: switch to use a proper PIP package - new: my.taplog data provider

Misc: more tests & mypy running on CI

See commit changelog for more details.

Special thanks to @seanbreckenridge for discussions!

- Python
Published by karlicoss over 5 years ago

hpi - minor release

core: - core config section, allow enabling/disabling modules - better module detection - experimental helpers to supply errors with datetime information - experimental helpers to make it easier to override config, e.g. see https://github.com/karlicoss/dashboard/blob/ff15295d8118494f986ed13757e022ae2aa18000/src/dashboard/data.py#L99-L117 for example usage - nicer warnings - initial core.pandas helpers

cli: hpi doctor enhancements

modules: - big update for instapaper, reddit, ghexport, hypothesis, rescuetime, pocket. DEPRECATED: the old way of specifying dependecies either as - config parameters (e.g. hypexport = "/path/to/hypexport_on_disk") - symlinks in the config (e.g. my/config/repos/hypexport)

These dependencies are proper PIP packages now, see https://github.com/karlicoss/HPI/issues/79 for the discussion about the change This is backwards compatible, but please reinstall them properly as soon as you can (see the updated instructions in the corresponding repos, e.g. https://github.com/karlicoss/instapexport#setting-up)

provide fake data providers for some modules
better mypy coverage for some modules
for other minor updates, see the commit messages

- Python
Published by karlicoss over 5 years ago

hpi - minor release

core helper to open databases as read-only/immutable (useful for Docker mounts)
some cachew experiments, including cachew base directory (still work in progress)
more stat functions and improvements for stats
initial work on using proper virtualenv for DAL helpers (using emfit to start with)

modules affected: - reddit - pdfs - bluemaestro - emfit - twint - rescuetime

- Python
Published by karlicoss over 5 years ago

hpi - core, github, kobo updates

core: - support '' for "empty paths" - helper for computing stats, add to more modules

github: - deprecated my.coding.github, see https://github.com/karlicoss/HPI/commit/ca39187c6347c645ba37d92246235b97e81eed84 - support data arbitration from GDPR and ghexport, see https://github.com/karlicoss/HPI/commit/d7aff1be3ff5cd4a51574bf97ecdb1395e73a91f

kobo: see https://github.com/karlicoss/HPI/pull/62 - breaking: kobuddy should be properly installed (instead of old my.config.repos.kobuddy) - breaking: export_dir renamed to export_path for consistency,

- Python
Published by karlicoss over 5 years ago

hpi - core: CLI improvements; pocket: BREAKING changes

pocket:

now uses pockexport for the data, see https://github.com/karlicoss/HPI/commit/f175acc848450265e6e7a522ebff53f381f5dbd4

- Python
Published by karlicoss over 5 years ago