nodbi - nodbi 0.13.1

• Improve mangling DuckDB version number
• Correct listfields = TRUE for src_duckdb

- R
Published by rfhb 12 months ago

nodbi - nodbi 0.13.0

• Prepare docdb_query() for duckdb 1.3.0 (e.g., use new json_tree function)
• Move warning for non-persistent connections to srcsqlite, srcduckdb calls
• Harmomise printing connections

- R
Published by rfhb about 1 year ago

nodbi - nodbi 0.12.0

Improvements

• Use duckdb internal function for writing NDJSON to file
• Removing message about RSQLite handling NDJSON file name as value

- R
Published by rfhb about 1 year ago

nodbi - nodbi 0.11.0

Potentially breaking changes

• docdb_query() modified so that it returns a data frame, in which each column has just one type (atomic or list) across all the rows of the respective column (previously, e.g. a mix of single-item lists simplified to atomic values and of multi-item lists were returned)

Changes

• docdb_create() and docdb_update() for SQLite and PostgreSQL (only if on localhost) now import directly and fast from ndjson files, in analogy to DuckDB (needs RSQLite >= 2.3.7.9014)
• Refactored docdb_update() for src_couchdb()
• Add message from docdb_create() if a data frame has column names with a dot(s) since dots in nodbi are used for JSON dot paths
• Add code to check database backend version requirements
• Adding info if PostgreSQL database is not yet created
• Factored out further code

- R
Published by rfhb over 1 year ago

nodbi - nodbi 0.10.7

• uses new features of duckdb 1.11.0 for refactoring of docdb_query(), accelerating queries
• accelerated creating and updating from file

- R
Published by rfhb over 1 year ago

nodbi - nodbi 0.10.6

Changes

• partial refactoring of docdb_query(), accelerating queries up to 20-fold for SQLite, DuckDB, and accelerating listfields = TRUE several times for DuckDB

Bug fix

• address docdb_query() not working for cases when dot paths had no counts between fields
• address wrong database size printing

- R
Published by rfhb almost 2 years ago

nodbi - nodbi 0.10.5

Bug fix

• stop if query is invalid even though JSON is valid
• print information also for MongoDB connection object

Changes

• code cleaning, parameters checking
• document that $regex in docdb_query() is case-sensitive

- R
Published by rfhb almost 2 years ago

nodbi - nodbi v0.10.4

Bug fix

• re-adding field formatting for docdb_query(src, key, query, listfields = TRUE, limit = <integer>)

- R
Published by rfhb over 2 years ago

nodbi - nodbi v0.10.3

Bug fix

• minor fixes to limit in docdb_query(src, key, query, listfields = TRUE, limit = <integer>) and speed up

- R
Published by rfhb over 2 years ago

nodbi - nodbi v0.10.2

Changes

• added vignette
• added tests internal functions, verbose option
• added caching to GitHub action workflow
• added missing fields validity check for duckdb
• more robust parameter checks in docdb_query and docdb_update
• ensure NULL also for all MongoDB returns

Bug fixes

• docTyp'ed src.R
• minify JSON with Elasticsearch in docdb_update
• moved local variable out of UseMethod in docdb_query

- R
Published by rfhb over 2 years ago

nodbi - nodbi v0.10.1

Bug fix

• make docdb_get() work again for src_sqlite() by casting JSONB back to JSON

- R
Published by rfhb over 2 years ago

nodbi - nodbi v0.10.0

Deprecated

• empty parameter query now triggers a warning as it should be a valid JSON string; change query = "" into query = "{}"

Changes

• adapted to use new, faster JSONB functions in SQLite 3.45.0 (RSQLite >= 2.3.4.9005)
• refactored parts of docdb_create() to speed up handling large data frames and lists
• made Elasticsearch to immediately refresh index after docdb_create() and other functions
• docdb_update() now reports which records failed to update and then continues
• docdb_delete() now returns harmonised success logical value across backends

Potentially breaking change

docdb_query() reimplementation to have the same functionality across all databases (DuckDB, SQLite, PostgreSQL, MongoDB, Elasticsearch, CouchDB); even though the API and unit tests remained, user provisions may break e.g. to handle return values of databases that previously were incompletely implemented (in particular Elasticsearch and CouchDB). Details:

• query can now be complex (nested, various operators) and is digested with a Javascript helper
• fields can now be nested fields (e.g., friends.name) to directly return values lifted from the nested field
• listfields parameter newly implemented to return dot paths for all fields of all or selected documents in collection
• expanded use of jq via jqr for mangling parameters, selecting documents, filtering fields and lifting nested field values
• if no data are found, returns NULL (previously some backends returned an empty data frame)
• docdb_query(src, key, query = "{}", fields = "{}") now delegates to docdb_get(src, key)
• _id is always returned, unless specified with "_id": 0 in parameter fields
• for scr_postgres, only fewer than 50 fields if any can be specified in fields
• for src_sqlite, minimise the use of the time-costly json_tree
• workaround for path collisions of MongoDB
• some acceleration of docdb_query()
• factored out common code
• expanded testing
• updated docs

- R
Published by rfhb over 2 years ago

nodbi - nodbi v0.9.8

• escaping newline character within a JSON value, in docdb_*() functions

- R
Published by rfhb over 2 years ago

nodbi - nodbi 0.9.7

• changed docdb_update() to directly use ndjson from file for duckdb
• regression error from not specifying top-level jq script
• corrected and improve field selection in docdb_query()
• corrected test exceptions for mongodb, updated GitHub Actions, expanded tests
• changed docdb_update() to directly use ndjson from file for duckdb
• cleaned up unnecessary code in docdb_create()
• no more using transactions with src_duckdb()

- R
Published by rfhb over 2 years ago

nodbi - nodbi 0.9.6

• corrected marginal case in docdb_query.src_duckdb()
• corrected minimum R version
• replaced in tests httpbin with webfakes
• removed explicit UTF-8 encoding reference
• speed up in docdb_query()
• switched to v2 GitHub r-lib/action for R CMD check

- R
Published by rfhb almost 3 years ago

nodbi - nodbi v0.9.5

• replaced a dependency, gained speed

- R
Published by rfhb about 3 years ago

nodbi - nodbi v0.9.4

• fix initialisation in docdbquery with srcduckdb

- R
Published by rfhb about 3 years ago

nodbi - nodbi v0.9.3

• docdb_update() now can do bulk updates when _id's are in 'value' (for SQLite, DuckDB, PostgreSQL, MongoDB; not yet for CouchDB and Elastic)
• fix tests for value parameter to be a file or an url

- R
Published by rfhb about 3 years ago

nodbi - nodbi v0.9.2

• srcduckdb(): handle when jsontype returns NULL for non-existing path (2023-02-18)
• src_sqlite(): handle when text includes double quotation marks (2023-03-25)
• added warning if DuckDB's JSON extension is not available; improve instructions; see also issue #45
• minor simplification of docdbexists() for srcmongo(), and of docdb_query() for SQL databases

- R
Published by rfhb about 3 years ago

nodbi - nodbi v0.9.1

• corrected closing connections to SQL database backends upon session restart
• improved provisions for parallel write access and corresponding tests
• capture marginal case of no rows in docdb_query()

- R
Published by rfhb over 3 years ago

nodbi - nodbi v0.9.0

• adding support for duckdb (R package version 0.6.0 or higher) as database backend

- R
Published by rfhb over 3 years ago

nodbi - nodbi v0.8.1

• replaced isa() as not available with R version 3.x

- R
Published by rfhb almost 4 years ago

nodbi - nodbi v0.8.0

2022-06-12 * refactored docdb_update.src_couchdb to use jqr * adapted docdb_create to accept jsonlite, jsonify, jqr JSON * added details to README * testing (unset LANG, relocate open code, better cleaning up)

- R
Published by rfhb about 4 years ago

nodbi - nodbi v0.7.1

BUG FIXES

• fixed docdb_query regression after SQLite 3.38.3 added quotation of labels (closes issue #44), test added
• made docdb_query work for PostgreSQL when a string used with the $in operator has a comma(s), test added

- R
Published by rfhb about 4 years ago

nodbi - nodbi v0.7.0

IMPROVEMENTS

• docdb_create now supports file names and http urls as argument value for importing data
• docdb_create (and thus docdb_update) now supports quantifiers (e.g., '[a-z]{2,3}') in regular expressions

BUG FIXES

• for SQLite, return FALSE like other backends when using docdb_delete for a non-existing container (table, in the case of SQLite)
• better handle special characters and encodings under Windows

- R
Published by rfhb over 4 years ago

nodbi - nodbi v0.6.0

IMPROVEMENTS

• full support for PostgreSQL (using jsonb)

BUG FIXES

• for SQLite add closing file references also on exit
• for SQLite under Windows ensure handling of special characters (avoiding encoding conversions with file operations that stream out / in NDJSON)

- R
Published by rfhb over 4 years ago

nodbi - nodbi v0.5.0

IMPROVEMENTS

• identical API for docdb_*() functions so that query and fields parameters can be used across database backends
• identical return values across database backends

UNDER THE HOOD

• re-factored recently added functions for (R)SQLite
• re-factored most functions to provide identical API
• performance (timing and memory use) profiled and optimised as far as possible

OTHER CHANGES

• testing now uses the same test file across databases
• currently, no more support for redis (no way was found to query and update specific documents in a container)
• docdb_list() added as function to list container in database

NOTES

• Support for complex queries not yet implemented for Elasticsearch
• Only root fields (no subitems) returned by Elasticsearch and CouchDB

- R
Published by rfhb over 4 years ago

nodbi - nodbi v0.4.3

MINOR IMPROVEMENTS

• docdb_query.src_sqlite() now handles JSON objects, returning nested lists (#40)
• src_sqlite() now uses transactions for relevant functions (#39)
• docdb_update.src_mongo() now returns the number of upserted or matched documents, irrespective of whether they were updated or not

BUG FIXES

• docdb_get() to not return 'id' field for `src{sqlite,mongo}` since already used for row names

OTHER

• change of maintainer agreed

- R
Published by rfhb almost 5 years ago

nodbi - nodbi v0.4.2

BUG FIXES

• fix for src_couchdb(): we were not setting user and password correctly internally, was causing issues in CouchDB v3 (#35) thanks to @drtagkim for the pull request

- R
Published by sckott over 5 years ago

nodbi - nodbi v0.4.0

MINOR IMPROVEMENTS

• in docdb_query() and docdb_get(), for sqlite source, use a connection instead of a regular file path to avoid certain errors on Windows (#33) work by @rfhb
• in docdb_query() and docdb_create() for sqlite source, fix to handle mixed values of different types (#34) work by @rfhb
• some Sys.sleep's added to Elasticserch eg's to make sure data is available after creation, and before a data request

- R
Published by sckott over 6 years ago

nodbi - nodbi v0.3.0

NEW FEATURES

• new author Ralf Herold, with contribution of new functions for working with SQLite/json1. new functions: src_sqlite, print.src_sqlite, docdb_create.src_sqlite, docdb_delete.src_sqlite, docdb_exists.src_sqlite, docdb_get.src_sqlite, docdb_query.src_sqlite, and docdb_update.src_sqlite. includes new dataset contacts (#25) (#27) (#28) (#29) (#30) (#31)
• docdb_update gains method for working with MongoDB, via (#27)

MINOR IMPROVEMENTS

• added .github files in the source repository to facilitate contributions
• src_mongo changes, improved behavior, via (#27)

DEFUNCT

• etcd (via the etseed package) integration has been removed from this package as etcd doesn't really fit the main goal of the pkg. functions now defunct are: src_etcd, docdb_create.src_etcd, docdb_delete.src_etcd, docdb_exists.src_etcd, docdb_get.src_etcd, and print.src_etcd (#26)

- R
Published by sckott over 6 years ago

nodbi - nodbi v0.2.0

NEW FEATURES

• docdb_get() gains limit parameter to do pagination, for CouchDB, Elasticsearch and MongoDB only (#17) (#23)
• gains function docdb_query() to send queries to each backend (#18) (#22)
• gains function docdb_exists() to check if a database or equivalent exists (#21) (#22)

MINOR IMPROVEMENTS

• Updated package for new version of elastic, which has slightly different setup for connecting to the Elasticsearch instance (#20)

- R
Published by sckott about 7 years ago

nodbi - nodbi v0.1.0

NEW FEATURES

• released to CRAN

- R
Published by sckott almost 8 years ago