vs.xml - v0.2.13

[!WARNING]
This release broke compatibility with the previous structure of nodes.
It has not been fully tested yet, so if you find yourself having strange regressions, use the same but two commits back.
Hopefully, once this project has proper tests, these notices will no longer be needed.

New release, fresh out of the oven.
Many of the improvements are not reported in here, as they are mostly infrastructural and will not impact the end user; please check commits if you want more details.
The main focus was to extend the current interface with useful utilities, remove even more dynamic allocations when not needed, and fix some downstream issues when linking this library.

Breaking changes

• DocBuilder renamed as DocumentBuilder to be more consistent in naming.
• Removed path functions from XML entities, leftovers from the very early versions;
  this functionality can now be trivially replaced by user-defined functions, since the rest of the interface is complete.
• The binary interface for trees and derived friends changed. Again. But it is for good reasons! We optimized away one of the biggest fields in nodes, saving a significant amount of memory.
  Technically this change prevents out of order nodes in the memory layout, but this was just a side-effect extra, not something intended.

New features

Introduced TreeRaw::visit and Tree::visit to implement a more flexible visitor pattern when compared to the one recently added iterator-based approach.
They are both based on private/(wrp-)visit.hpp, which is not publicly exposed (for now).

New print functions have been introduced for trees, based on the visitor pattern. It no longer uses std::print due to the awful overhead and additional memory allocations. fmt had no such issue to be honest.
The legacy version has been optimized as well: it is now called print_fast and still uses simple recursion to get a signifiant edge on performance; however, be mindful of stack overflows if working with stack intensive applications or badly nested trees.

- C++
Published by KaruroChori 11 months ago

vs.xml - v0.2.11

This one is just an intermediate release, don't expect much in terms of fancy new features!
The new queries are still cooking in the background.
Still, this one improves documentation coverage, adds some quality of life features and refactors several bad architectural choices which have been dragging since the very start of this project.

New features

• a new builder for Archive, avoiding most of the boilerplate needed before;
• TreeIterator to replace the current queries based on generators.

CLI utils for archives have been postponed to v0.2.13 due to parallelization concerns.

Breaking

• Builder have been split into separate files, so to have a clean separation between embeddable and not embeddable code.
• reserve for builders has been changed to be more extensible and descriptive.
• Constructor for leaf elements update to accept parent node.
• Binary files generated on v0.2.9 are not fully compatible as a bug was found in the tree builder. They are missing references to parent on leaf nodes, which prevents navigation. Binary files must be regenerated even if the binary format itself has been kept stable.

- C++
Published by KaruroChori 12 months ago

vs.xml - v0.2.9

Surprisingly little changed in this release for the final users, but many changes have been made to ensure this library is more suitable in embedded applications.
Trees, documents and archives no longer handle their memory directly, allowing for their external allocation.
Wrappers still owning memory are available in xml::stored under the same name as their original versions.
The namespace will be made unavailable when compiling for an embedded or offloaded target in a later release.

Next release will focus on documentation, examples, better handling of embedded/offloaded targets, and will start refactoring the code handling queries to match the current objectives.

Breaking changes

• The close function for Tree(Raw) and Document(Raw) changed signature to return the new Stored<T> objects, so that storage is separate from the base classes to better use code in embedded and offloaded contexts.
• The close_fragment and extract_symbols functions changed. The former now returns a data type more compatible with the revised Archive sections, while the latter has been extended to return both symbols and buffer at once.
• Significant updates in the binary format, with more offsets being relative and lengths in place of a second offset.
• Fixed signatures for from_binary removing stringviews in place of `std::span<const uint8t>, and fixed some return types which were not set toconst`.
• Other minor differences might be present due to the changes discussed prior, it is too hard to spot and report all of them, check the commit diffs to see the all.

- C++
Published by KaruroChori 12 months ago

vs.xml - v0.2.7

This release could not be postponed as the previous changes to the meson.build made the library not usable as a sub-dependency.
Hopefully this fixes that, but it means some other features are a bit under-baked or not tested.
Expect a corrective v0.2.9 Soon (TM).

New features

We now have full support for ArchiveRaw and Archive.
This means that multiple XML documents can share the same symbol table and be distributed as a single archive file.
Each file can be addressed with a name. Right now no checks are performed about uniqueness.

Also, reading or writing binary files is not hardened against attacks, and they will not be for quite a while.
If you want to send PR about that they are welcome.

We are almost ready to reach v0.3.x. In that new minor branch I will be refactoring the query system as outlined in https://github.com/lazy-eggplant/vs.xml/issues/4 and in the prior release notices.

Other minor features

• Additional checks for word sizes and endianess when reading binaries, to ensure binary compatibility.
• Work in the background to make the library more compatible with embedded targets and offloading.

Breaking

• The binary format is changing. Yet again. At this point it must be some kind of running joke.
  But really, with archives being done, I don't expect any further major revision in the nearby future.

- C++
Published by KaruroChori about 1 year ago

vs.xml - v0.2.5

This release is very dense release in terms of features!
Most of them have been under development for a while, but they just about reached the "untested but useful" state:

• Queries on the XML tree are now working! With shorthands and everything. They need a good documentation page as they are one of the core features to properly use this library.
• (breaking) changes in the binary format to support multi-document archives sharing symbols. No capability exposed to the library yet, but the binary format should be "final".
• The configuration flags used while configuring the project are now shipped with the library headers.
• A new text iterator to access all text and cdata sections of an element in one go. It is kind of slow compared to direct access of the string-view.
  

For some operations, filtering text/cdata children and iterate over their values will be a much better approach compared to the new text. Still, it can be good for queries or to implement searches.

Notice on future query redesign

TLDR: The overall high level syntax will be kept, but the underlying implementation will be replaced.

This first implementation of queries was very informative and technically functional... but also very annoying to work with.
It is plagued by issues linked to object lifetimes, which are not as easy to unravel due to the presence of co-routines in the mix.
By the way, coroutines in C++ are nice to use but quite bad in terms of memory footprint for this application, and while mitigations are possible, memory is going to be higher compared to an iterator-based approach.

Furthermore, we don't have properly serializable queries with simple hashing...
So, for the next release, it will be time to decide what was good about this first implementation and what to do so that the experience of the next is going to be better.

• Design queries to work with the query builder, not as an afterthought but since the very beginning. The design work being done in here must be extended.
• Drop co-routines. They were good to sketch a quick working solution, they are awful to harden against memory allocations and when handling issues related to their memory footprint.
• Keep the high level syntax the same, it is actually quite good I think.

- C++
Published by KaruroChori about 1 year ago

vs.xml - v0.2.1

Release to propagate (the minor) breaking changes downstream.
We don't have many new features visible, but a lot of work in the background to improve the code base and its scope.
For reference, v0.2.x should provide as main features:

• Queries
• Multiple documents sharing symbols (and related binary format)
• Helpers for annotations (but they might be rescheduled for v0.3.x based on how much feature creep is met as it relates to tree injection which is its own beast to handle)

As minor features:

• Support for iterators to visit text across text and cdata boundaries.
• Apply replacement for serialization functions which don't use memory allocations in the serialization code (already implemented, not used)

Several steps have been made to implement both, but they cannot be tested still (probably there will be queries mostly done in v0.2.3).

Changes

• Removed virtually all explicit exceptions.
• Removed memory allocations for a wide range of operations.
• Improvements to the binary format.
• Initial work to support shared symbol tables, annotations and queries.
• More entry-points for customization, like some containers or predefined reduced memory layouts.
• Structural improvements in the repo, code quality etc.

Breaking features

• The binary format changed.
• Several functions had some minor changes, like [[nodiscard]] being added, or std::expected in place of exceptions.

- C++
Published by KaruroChori about 1 year ago

vs.xml - v0.1.15

New release, freshly baked, so that I can use it downstream :smile: .
A lot of bugs have been fixed, some profiling to improve performance and to kill most memory allocations hidden to the sight.
I started working on some benchmarking figures and comparisons against pugixml, which I consider as baseline reference (at least for those operations which are comparable).

Feature-wise, this is where the v0.1.x minor release cycle ends.
Next one will be about documentation and testing, alongside two major missing features: - Support for tree injection, allowing to virtually attach a tree into an other without structural mutations involved. - Basic path model to navigate the tree and express queries more easily.

With that, I think this project would be done feature-wise; anything else is probably better left to downstream applications and libraries.

- C++
Published by KaruroChori about 1 year ago

vs.xml - v0.1.13

I said last release it would have been last before v0.2.1, but I lied. This one comes with some very important fixes for usability and to ensure forward compatibility:

• A more robust header for the binary format, introducing versioning. For now I will not fix minor or major revisions as the library is not stable enough, but when the binary format will reach its first major, compatibility will be ensured.
• Now standard C++ views/ranges will work with children and attrs, allowing to sort, filter, map things as one would do with stl containers. This way the library does not have to implement its own redundant functionality, but I will provide some helper filters at some point.

I also tested the library with datasets as big as half a gigabyte while fixing the CLI utils, and it seems to be working fine.
At some point I want to introduce some benchmarks to see how it compares, mainly with pugixml since this is what I want to replace in some downstream projects.

- C++
Published by KaruroChori about 1 year ago

vs.xml - v0.1.11

What is likely going to be the last pre-release. Only tree injection is now missing for v0.2.1. I also added several new features which were not initially planned, including some CLI utils distributed alongside the library.

- C++
Published by KaruroChori about 1 year ago

vs.xml - v0.1.9

A new preliminary version, mostly because the interface changed a lot and I want to fix the new version in some downstream projects already using it.
Before the first proper release, I want tree injection and Document properly implemented. And we are quite close to that.

Full Changelog: https://github.com/lazy-eggplant/vs.xml/compare/v0.1.7...v0.1.9

- C++
Published by KaruroChori about 1 year ago

vs.xml - v0.1.7

Initial release to test the CI/CD pipeline.
I mean, the library works to the extent it has been implemented, so feel free to have a look :).
But documentation is near to zero and some important features are still missing. Check the todo for more information.

- C++
Published by KaruroChori about 1 year ago