Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of qpbenchmark

qpbenchmark - v2.5.0

This release adds three QP solvers to the benchmark: jaxopt.OSQP, KVXOPT and QPAX.

Added

  • Add jaxopt.OSQP solver
  • Add KVXOPT solver with the same settings as CVXOPT
  • Add QPAX solver and document its choice of settings

Changed

  • CICD: Update checkout action to v4

- Python
Published by stephane-caron 10 months ago

qpbenchmark - v2.4.0

This release adds the ability to load and save test sets and results from Apache Parquet files. It also refactors the command-line tool with a progress bar (estimating the time remaining until completion) as well as a "very verbose" mode for per-call details.

console $ python mpc_qpbenchmark.py run [2024-12-16 10:27:18,154] [info] Loading existing results from '~/src/mpc_qpbenchmark/results/qpbenchmark_results.csv'... (results.py:56) [2024-12-16 10:27:18,160] [info] Loaded 3840 rows from '~/src/mpc_qpbenchmark/results/qpbenchmark_results.csv' (results.py:63) 23%|████████████▏ | 720/3072 [00:00<00:01, 1775.82it/s]

Added

  • Add ParquetTestSet class for test sets loaded from Parquet files
  • Add Problem.from_qpsolvers shorthand function
  • Add ProblemList class that can be saved to Parquet files
  • Allow strings or paths as arguments to the main function
  • CLI: Add list_problems command
  • New dependency: tqdm
  • Optionally add a GPU summary line to reports
  • Results can now be read and save from and to Parquet files
  • Results property to get the number of result rows
  • Support Python 3.12
  • Very-verbose mode for extra debug messages

Changed

  • Allow results file paths to be strings or pathlib paths
  • Don't save results more frequently than every 10 seconds
  • More informative solver exception warnings
  • Refactor results loading and saving
  • Save results to file after problem is done rather than at each solver call
  • Update skip logic in CPU info table

Fixed

  • CLI: Only allow --solver to be an available solver for the run command

- Python
Published by stephane-caron about 1 year ago

qpbenchmark - v2.3.0

This release improves subset handling for QP test sets that have both sparse and dense problems, allowing using the same results file for both.

Added

  • Add results_path argument to the main function
  • Results: handle loading/saving CSV files with out-of-test-set problems
  • Special handling to get CVXOPT version number properly

Changed

  • CLI: make results path an argument to all commands
  • Rename "include timeouts" argument to "rerun timeouts"

- Python
Published by stephane-caron over 1 year ago

qpbenchmark - v2.2.3

This patch release fixes reporting of solver versions when some solvers are excluded.

Fixed

  • Fix reporting of solver versions when some solvers are excluded

- Python
Published by stephane-caron over 1 year ago

qpbenchmark - v2.2.2

This patch version fixes handling of no-op conversions in test sets (dense to dense or sparse to sparse).

Added

  • Expose exceptions from top-level module

Fixed

  • Bump Python version to actual requirement of 3.9 or above
  • Handle no-op conversion from dense to dense properly
  • Handle no-op conversion from sparse to sparse properly

- Python
Published by stephane-caron over 1 year ago

qpbenchmark - v2.2.1

This patch release updates to qpsolvers v4.3.1 which fixes a bug in the Gurobi interface. Thanks @563925743 :+1:

Changed

  • Update qpsolvers to v4.3.1 to fix the Gurobi interface

- Python
Published by stephane-caron about 2 years ago

qpbenchmark - v2.2.0

The benchmark is now distributed on conda-forge:

console conda install qpbenchmark

This release also adds a main-function hook to make it easier to run test sets directly from their repository as python my_test_set.py.

Added

  • Distribute on conda-forge
  • Test set path argument to the main function of the command-line interface

Changed

  • Update environment file to install from conda-forge

- Python
Published by stephane-caron about 2 years ago

qpbenchmark - v2.1.1

This patch release fixes a bug in the handling of known solver issues.

Changed

  • Clarabel: set tol_gap_abs settings in set_eps_abs
  • Clarabel: set tol_gap_rel settings in set_eps_rel
  • Log a warning message when skipping known solver issue
  • PIQP: set eps_duality_gap_abs settings in set_eps_abs
  • PIQP: set eps_duality_gap_rel settings in set_eps_rel
  • ProxQP: set eps_duality_gap_abs settings in set_eps_abs
  • ProxQP: set eps_duality_gap_rel settings in set_eps_rel

Fixed

  • Handling of known solver issues

- Python
Published by stephane-caron about 2 years ago

qpbenchmark - v2.1.0

This release removes the cost error from benchmark metrics, and completes the spinning off of test sets to their own repositories. It also brings a few quality-of-use improvements.

Added

  • Expose Problem from top-level module
  • Link in report header to go to results tables directly
  • Number of problems in report header
  • Utility function to load a problem from file

Changed

  • Breaking: Remove the cost error from benchmark metrics
  • Include Python 3.8 in supported versions

Removed

- Python
Published by stephane-caron about 2 years ago

qpbenchmark - v2.0.0

This release renames the benchmark to qpbenchmark and adds the QPALM solver.

Added

  • New solver: QPALM

Changed

  • Breaking: Rename the project "qpbenchmark"
  • Don't install solvers by default from PyPI
  • Move solver issue and timeout listings to test-set themselves

Removed

  • Module-wide skip solver issue/timeout functions

- Python
Published by stephane-caron about 2 years ago

qpbenchmark - v1.2.0

This release adds a mid_accuracy set of settings to the QP solvers benchmark.

Added

  • Citation file to refer to the benchmark in scientific works
  • Medium accuracy settings
  • Note in the readme on CPU throttling

- Python
Published by stephane-caron over 2 years ago

qpbenchmark - v1.1.0

This release adds two solvers to the benchmark: HPIPM and PIQP. It also invites everyone to submit results on their machines to the Results discussions.

Added

  • Check consistency after loading results
  • More CPU information in reports
  • New solver: HPIPM
  • New solver: PIQP

Changed

  • Don't hard-wrap report lines, as it doesn't render well in Discussions
  • Improve reporting of shifted geometric mean errors
  • Make cpuinfo a proper dependency
  • Refactor results class to allow finer check_results sessions
  • Update to qpsolvers v4.0.0

Fixed

  • Correct None values to False in found column
  • Make sure found column has only boolean values

- Python
Published by stephane-caron over 2 years ago

qpbenchmark - v1.0.0

This is the first release of the QP solvers benchmark, which ships a qpsolvers_benchmark command-line tool contributed by @ZAKIAkram.

The goal of this benchmark is to help users compare and select QP solvers. Its methodology is open to discussions. The benchmark ships standard and community test sets, as well as a qpsolvers_benchmark command-line tool to run any test set directly on your machine. For instance:

console qpsolvers_benchmark maros_meszaros/maros_meszaros.py run

The outcome from running a test set is a standardized report evaluating all metrics of the benchmark across all available QP solvers. This repository also distributes results from running the benchmark on all test sets using the same computer.

New test sets are welcome. The benchmark is designed so that each test-set directory is standalone, so that the qpsolvers_benchmark command can be run on test sets from other repositories. Feel free to create ones that better represent the kind of problems you are working on.

Thanks to @aescande, @ottapav and @ZAKIAkram for contributing to this release :+1:

Added

  • Allow non-lowercase solver names in the command line (thanks to @ottapav)
  • Command-line tool and standalone test sets (thanks to @ZAKIAkram)

Changed

  • Plot: trim solutions that don't fulfill tolerance requirements
  • Rename hist command to plot
  • Update to qpsolvers v3.4.0

- Python
Published by stephane-caron over 2 years ago

qpbenchmark - v0.1.0-beta

This is the last pre-release of qpsolvers_benchmark, a benchmark for quadratic programming (QP) solvers available in Python. The main upgrade since alpha is that the benchmark now checks both residuals and the duality gap, i.e., full optimality conditions.

The goal of this pre-release is to request feedback from QP solver developers before the initial release. The initial release will in no way be the end: it will be advertised around, but the benchmark is designed to be iteratively refined by the community in the long run.

You can already help at this stage! Here are some suggestions:

Added

  • Check dual residual
  • Check duality gap
  • Document all benchmark functions
  • Main script: new hist plot command
  • ProblemNotFound exception
  • Results by settings in reports
  • Write benchmark version in reports

Changed

  • Benchmark script takes test set as first argument
  • Maros-Meszaros: empty equality constraints are now set to None
  • ProxQP: re-run benchmark with ProxQP 0.3.2
  • Refactor Report class and run function
  • Report encoding is now UTF-8
  • Switch to qpsolvers v2.7
  • Test set descriptions are now mandatory

Fixed

  • Conform to linter standards
  • Sparse matrix conversion

- Python
Published by stephane-caron about 3 years ago

qpbenchmark - v0.1.0-alpha

This is a pre-release of qpsolvers_benchmark, a benchmark for quadratic programming (QP) solvers available in Python. This benchmark is still a work in progress. The goal of this pre-release is to show the current status and gather feedback. In particular, feel free to:

Towards a tool to help us compare and select QP solvers!

- Python
Published by stephane-caron about 3 years ago