https://github.com/andrewrgarcia/bcrpy - 🚀 bcrpy v3.1 - Enhanced SQL Caching & Storage Support

New in v3.1 brings streamlined and powerful SQL caching and storage options, enabling efficient data handling and retrieval in SQLite format alongside existing DataFrame support. This release optimizes workflows for handling large datasets by storing them in a structured SQL database, reducing memory usage and improving access times.

Highlights

• SQL Save and Load Support:
  Added save_df_as_sql and load_from_sqlite utility functions to easily save and retrieve data from SQLite databases. Now, users can seamlessly choose between DataFrame or SQL-based caching.

• Enhanced Fetcher Class:
  Updated the Fetcher class to support SQL caching, allowing automatic storage and retrieval of data from SQL databases. This enhances flexibility, especially for larger datasets.

• Generalized Cache Handling:
  Streamlined cache handling logic for both DataFrame and SQL formats, removing redundant methods and using a unified approach for efficient data management.

• Dependency Updates:
  Updated poetry.lock to include packaging version 24.2 for compatibility and performance improvements.

With these upgrades, bcrpy v3.1 is now more efficient for large-scale data projects, providing robust caching and storage options to meet diverse needs.

Upgrade Notes:
Existing projects using previous versions should update any custom caching workflows to leverage the new SQL functions and streamlined cache handling in Fetcher.

- Python
Published by andrewrgarcia over 1 year ago

https://github.com/andrewrgarcia/bcrpy - Flexible Data Storage and Subtle Performance Optimizations

• Enhanced with optional storage parameter to select between DataFrame and SQLite formats for data extraction, improving flexibility.
• Renamed internal attributes and methods for consistency:
  • codigos to codes in GET and largeGET methods.
  • Hacha class renamed to Axe, with methods updated to slice and forge.
• Refined caching logic:
  • Added SQLite-based caching for optimized performance in large data requests.
  • Implemented save_to_sqlite, load_from_sqlite, and save_chunk_to_sqlite methods for efficient chunk storage.
• Updated MetadataHandler with robust fallback loading for metadata and improved error handling.
• Bumped documentation and version numbers to align with 3.0 release.

- Python
Published by andrewrgarcia over 1 year ago

https://github.com/andrewrgarcia/bcrpy - A package without matplotlib dependencies

A lighter, more straightforward version

- Python
Published by andrewrgarcia about 2 years ago