Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of https://github.com/bacalhau-project/bacalhau

https://github.com/bacalhau-project/bacalhau - v1.8.0

Bacalhau v1.8.0 Release Announcement

We're excited to announce Bacalhau v1.8.0: a groundbreaking release that transforms how you can approach distributed computing at the edge. This new release introduces:

  • Intelligent cost optimization through an advanced Splunk integration.
  • Enhanced daemon job orchestration for dynamic infrastructure.
  • Enterprise-grade improvements—docs, security, and more.
  • A completely reimagined job lifecycle management system.

In a nutshell, Bacalhau v1.8.0 delivers the integrations and capabilities that you need to reduce costs, improve operational efficiency, and unlock the full potential of your distributed infrastructure.

Let's dive into it to learn more!

Slash Your Splunk Bill: Intelligent Logging at the Edge

Let's talk about the elephant in the room: logging costs. For too long, you've faced a terrible choice: ship all your raw data to Splunk and watch your budget evaporate, or sample your data and risk missing the one critical event you need.

So, what if there could be an alternative solution? Bacalhau's new Splunk integration offers a third, much smarter path. Instead of moving mountains of data, Bacalhau lets you ship the computation to the data source. This fundamentally changes the cost equation.

With v1.8.0, Bacalhau's compute power can be integrated directly into Splunk so that you can:

  • Process data at the source: Deploy smart logging agents that filter, aggregate, and analyze data on the edge, sending only high-signal results to Splunk.
  • Fork your data stream: Send critical, processed alerts to Splunk for real-time monitoring while simultaneously archiving the full, compressed raw logs to cheap object storage.
  • Query live data without ingestion: Run queries directly on remote nodes to investigate issues in real-time, without moving the data or paying ingestion fees.
  • Replay history on demand: Need to analyze old logs? No problem. Use Bacalhau's compute power to pull data from your archive, process it, and analyze it in Splunk.

This isn't a minor tweak. It's a paradigm shift that can cut logging costs by 60-80% while giving you more analytical power. The best part? It all feels native. You use the Splunk dashboards and search queries you already know, while Bacalhau's distributed compute engine works transparently in the background.

Enhanced Daemon Jobs

Bacalhau v1.8.0 strengthens daemon job reliability with enhanced orchestration. The goal is to ensure comprehensive coverage across dynamic infrastructure.

Our developers have beefed up the entire system for the reliability that production environments demand:

  • Node discovery is faster.
  • Constraint checking is more robust.
  • Deployment tracking gives you a clear view of your job's footprint. When a new node spins up in your dynamic infrastructure, you can be confident your daemon job will deploy to it automatically and reliably.

These enhancements are particularly valuable for the Splunk integration, where logging agents must reliably deploy to new infrastructure automatically.

Enterprise-Ready by Default

Rounding out the release are features designed to make Bacalhau seamless in an enterprise environment:

  • Managed result storage: With Expanso Cloud, you no longer need to configure your own publisher or manage storage credentials. Job results are automatically and securely stored in our managed infrastructure. Just run your job and fetch your results.
  • Documentation overhaul: We've rebuilt our docs from the ground up with task-based guides, architectural deep dives, and real-world examples for things like distributed log processing and edge ML.
  • Hardened foundation: We've continued to strengthen security, authentication (with SSO), and reliability to ensure Bacalhau is ready for your most demanding production workloads.

Transformative Job Lifecycle Management

Bacalhau v1.8.0 introduces a fundamental transformation in how you manage computational workflows. We've moved beyond cryptic, auto-generated job IDs to a powerful name-based system that supports versioning, updates, and intelligent rerunning.

Goodbye j-f47ac10b-58cc-4372-a567-0e02b2c3d479

If you've ever felt the pain of managing workflows using nothing but UUIDs, this update is for you. We've completely reimagined job lifecycle management to be intuitive, version-controlled, and built for humans.

Jobs are now identified by meaningful names you define in your spec:

```bash

Before: Cryptic UUID management

bacalhau job describe j-f47ac10b-58cc-4372-a567-0e02b2c3d479

After: Intuitive name-based operations

bacalhau job describe monthly-sales-report-generator ```

Advanced Versioning and Dry-Run

Every job submission with an existing name creates a new version. The enhanced --dry-run functionality provides server-side diff previews, showing exactly what will change before applying updates:

```bash

Preview changes with detailed diff

bacalhau job run --dry-run updated-pipeline.yaml

Work with specific versions

bacalhau job logs data-analysis --version 2 bacalhau job describe ml-model-training --version 1 ```

Intelligent Job Rerunning

The new rerun command eliminates the need to resubmit jobs. This avoids you from creating new versions of existing jobs while preserving history:

```bash

Rerun the latest version

bacalhau job rerun data-processing-pipeline

Rerun a specific version

bacalhau job rerun ml-training-job --version 3 ```

Also, this versioning system is fully backward compatible: existing workflows using job IDs continue to work while new capabilities are available for adoption at your own pace.

What's next?

Bacalhau v1.8.0 is available now with full backward compatibility. Existing workflows continue to operate unchanged while new capabilities are available for immediate adoption:

  • For Splunk Users: The Expanso Splunk application is available through Splunkbase and can be deployed in minutes to start realizing immediate cost savings.
  • For Existing Users: Update to v1.8.0 to access enhanced daemon job orchestration, managed result storage, and the new job lifecycle management capabilities.
  • For New Users: Our completely rebuilt documentation provides clear pathways for getting started with distributed computing, whether for cost optimization, edge processing, or advanced analytics.

- Go
Published by wdbaruni 11 months ago

https://github.com/bacalhau-project/bacalhau - v1.7.0

Bacalhau v1.7.0 Release Notes

This release introduces enterprise support options, enhanced partitioning capabilities, improved networking, comprehensive authentication, and significant WebAssembly enhancements.

Enterprise Support through Expanso

  • Enterprise support now available through Expanso while maintaining Bacalhau's open source core
  • Free tier for everyone supporting up to 5 nodes with all core features
  • Enterprise licensing option for organizations requiring professional support

Expanso Cloud Enhancements

  • Comprehensive job submission capabilities through the Expanso Cloud web interface
  • Library of pre-defined job templates for common use cases:
    • Log processing workflows
    • Data analysis with DuckDB
    • Apache Iceberg operations
    • And more
  • Custom template creation capabilities with parameterization
  • Simplified complex distributed computing tasks while maintaining flexibility

Enhanced Job Execution with Partitioned Workloads

  • Distribution of work across compute nodes with built-in partition management
  • Horizontal scaling and parallel processing for improved performance
  • Granular failure recovery with automatic retry mechanisms for failed partitions
  • Independent partition execution with context through environment variables

S3 Storage Input Partitioning

  • Automatic data distribution across multiple executions with graceful failure handling
  • Multiple partitioning strategies:
    • Object-Based Distribution: Even distribution without specific grouping
    • Date-Based Partitioning: Parallel processing for time-series analysis
    • Regex Pattern Matching: Distribution by patterns in file paths
    • Substring-Based Partitioning: Organization by customer segments
  • Combined partitioned data with shared reference data support

S3 Publishing Enhancements

  • Plain encoding support for direct access to job outputs without compression
  • Individual file publishing rather than compressed archives
  • Efficient access to individual files with no decompression overhead
  • Default gzip encoding still available for complete result sets

Enhanced Networking Capabilities

  • Networking enabled by default (shifted from opt-in to opt-out)
  • Configuration option for restricting network access
  • Experimental port mapping support
  • Multiple network modes: bridge (default), host, and none

Comprehensive Authentication and Authorization

  • Redesigned authentication system with multiple methods:
    • Basic authentication
    • API tokens
    • Single Sign-On (SSO)
  • Fine-grained authorization controls
  • New CLI commands for authentication management

Host Environment Variable Forwarding

  • Access to specified environment variables from the host system
  • Secure credential passing through a controlled allowlist mechanism
  • Reference host variables using env: prefix in job specifications

Improved WebAssembly Support

  • Refactored WASM job specification with better separation of concerns
  • Remote modules defined as input sources for alignment with other executors
  • HTTP networking support for WebAssembly through new host function implementation
  • Configurable network access modes and security controls
  • TinyGo-compatible client library for HTTP functionality

Docker DinD Integration

  • Docker-in-Docker images for simplified containerized deployment
  • Two image variants:
    • Standard base image for client usage and orchestrator nodes
    • DinD image for compute nodes running Docker workloads
  • Pre-configured Docker daemon for simple integration with container ecosystems

Performance and Quality of Life Improvements

  • Enhanced resource management and more efficient state tracking
  • count field now defaults to 1 instead of 0
  • NetworkDefault type for automatic networking configuration

Documentation & Resources

Explore the complete release details including FAQs, implementation guides and enterprise licensing information in our comprehensive v1.7.0 documentation. The full documentation includes detailed answers to common questions about partitioning strategies, authentication methods, S3 integration, and enterprise support options.

New Contributors

  • @virajbhartiya made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4848
  • @haixuanTao made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4857
  • @Olg-K made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4873

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.0...v1.7.0

- Go
Published by wdbaruni about 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.7.0-rc2

What's Changed

  • Make Authentication Bypass test more robust by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4901
  • bacalhau auth info by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4902
  • Redact sensitive info from agent config command by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4903
  • Modify auth sso login syntax and add logout command by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4904
  • Add "bacalhau auth sso token" command by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4905
  • Fix SSO Login regression by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4907
  • fix regression in not setting log level by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4910
  • Avoid setting default network config when nil by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4911

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.7.0-rc1...v1.7.0-rc2

- Go
Published by wdbaruni about 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.7.0-rc1

What's Changed

  • deploy v1.6.0 to demo network by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4776
  • Update nclprotocol readme with seqNum exchange changes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4779
  • Fix TestConcurrentHealthCheckAndHeartbeat assertion by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4780
  • Add a linter to catch fmt.Print* by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4782
  • Show full nodeID and executionID with --wide flag by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4783
  • Replace NodeID field access with ID() by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4784
  • smaller job defaults requirements by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4785
  • build(deps): bump github.com/ipld/go-car/v2 from 2.13.1 to 2.14.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4723
  • build(deps): bump nanoid from 3.3.7 to 3.3.8 in /webui by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4737
  • build(deps): bump golang.org/x/crypto from 0.27.0 to 0.31.0 in /test_integration by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4760
  • build(deps): bump next from 14.2.10 to 14.2.15 in /webui by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4786
  • build(deps): bump jinja2 from 3.1.3 to 3.1.5 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4791
  • Updating git ignore by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4792
  • Add License Validation Mechanism using JWT and JWKS by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4799
  • Add Partition Support in Job Orchestrator by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4800
  • Execution and Partition-Level Environment Variables by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4801
  • Expose execution to storage providers by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4803
  • Add partitioning strategies for S3 storage source by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4805
  • remove unnecessary execution env vars by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4806
  • Remove kubectl template dependency with simplified implementation by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4807
  • Update partitioning.md by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4811
  • Support Local License Inspection Command by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4812
  • introducing new metrics recorder by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4815
  • add jobstore metrics by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4816
  • deploy v1.6.1 to demo network by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4818
  • add scheduler metrics by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4819
  • add planner metrics by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4820
  • build(deps): bump next from 14.2.15 to 14.2.21 in /webui by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4797
  • build(deps): bump virtualenv from 20.25.0 to 20.26.6 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4809
  • Manual Update for github.com/MicahParks/jwkset by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4821
  • Fix boltdb transaction panic on context cancellation by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4823
  • fix re-approving executions in the scheduler by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4824
  • fix devstack log format by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4825
  • Refactor LoggingPlanner for better debugging by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4826
  • devstack: support node joining existing networks and config passing by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4827
  • add ncl metrics by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4830
  • Add Agent license inspect command by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4813
  • increase ncl publish and subscribe timeouts by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4832
  • rate limit number of new executions per evaluation by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4831
  • Eng 565 messagehandler metrics by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4833
  • deploy v1.6.2 to demo network by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4834
  • build(deps): bump github.com/samber/lo from 1.47.0 to 1.49.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4828
  • publish docker-in-docker (dind) image by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4838
  • support plain encoding in s3 publisher by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4840
  • Fix race conditions when starting dind containers by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4841
  • Host Environment Variable Forwarding by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4842
  • fixing logo on readme by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4845
  • Enhance value column configuration with width constraints in config list by @virajbhartiya in https://github.com/bacalhau-project/bacalhau/pull/4848
  • fix golangci-lint workflow by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4854
  • install ca-certificates in docker images by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4853
  • Enable port mapping and network configuration for jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4855
  • Update Hello World job as the previous version seems to be outdated. by @haixuanTao in https://github.com/bacalhau-project/bacalhau/pull/4857
  • Fix Platform Compatibility Check for Locally Cached Docker Images by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4868
  • Count field defaults to 1 if not provided by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4869
  • add option to opt-out networking by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4871
  • More detailed instructions for local env by @Olg-K in https://github.com/bacalhau-project/bacalhau/pull/4873
  • cleanup dead config code by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4874
  • update readme and fix broken links by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4875
  • improve separation of concerns in WASM executor by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4882
  • fix backward compatibility with full network type by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4883
  • add HTTP host function for WebAssembly jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4884
  • make bacalhau logo adaptive to dark/light scheme by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4876
  • Fix deprecated webui flag usage in tests by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4890
  • build(deps): bump certifi from 2022.6.15 to 2024.7.4 in /ops by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4847
  • build(deps): bump virtualenv from 20.25.1 to 20.26.6 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4858
  • build(deps): bump golang.org/x/net from 0.29.0 to 0.36.0 in /test_integration by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4879
  • build(deps): bump golangci/golangci-lint-action from 6.5.0 to 6.5.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4880
  • build(deps): bump jinja2 from 3.1.5 to 3.1.6 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4885
  • add warnings when exceeding license claims by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4889
  • remove legacy apis and deprecated clis by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4895
  • Eng 689 memory limit exception points to 404 doc page by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4896
  • Bacalhau Auth v2: Basic Auth, API Keys, and SSO Support by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4888
  • Add NetworkDefault type for compute node autonomy by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4897
  • Add network configuration transformation for backward compatibility by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4898
  • run devstack with default ports by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4900
  • add docker image telemetry with privacy-first approach by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4899

New Contributors

  • @virajbhartiya made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4848
  • @haixuanTao made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4857
  • @Olg-K made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4873

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.0...v1.7.0-rc1

- Go
Published by wdbaruni about 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.5

Feature Enhancements and Fixes

Important Networking Changes

  • Networking Opt-Out Configuration: Added new JobAdmissionControl.RejectNetworkedJobs configuration to allow users to explicitly opt-out of networking. Currently, Bacalhau rejects networking by default and requires users to opt-in using JobAdmissionControl.AcceptNetworkedJobs=true. This new parameter is the first phase of a major change where networking will be enabled by default in v1.7 (planned for release in the coming weeks). Users who need to maintain disabled networking should begin transitioning to the new RejectNetworkedJobs=true configuration now to ensure a smooth upgrade path.

Job Configuration Enhancements

  • Default Count Parameter: Fixed a significant usability issue where batch/service jobs would silently fail when no count was specified. The system now automatically sets the count field to 1 when not explicitly provided, eliminating confusion for new users. Explicit count=0 settings are still respected for intentionally stopping jobs.

Docker Image Handling

  • Platform Compatibility Verification: Fixed the platform compatibility check for locally cached Docker images, ensuring that jobs correctly identify and utilize compatible images across different environments.

New Contributors

  • @haixuanTao made their first contribution in #4857

Full Changelog: v1.6.4...v1.6.5

- Go
Published by wdbaruni about 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.4

S3 Publisher Enhancements

  • Plain Encoding Support: Introduced an option to publish job results without gzip compression, facilitating more efficient data pipelines by allowing subsequent jobs to access individual files directly.

Environment Variable Management

  • Host Environment Variable Forwarding: Added the ability to securely forward host environment variables to job executions. This feature enables passing credentials and secrets from the host to jobs through a controlled allowlist mechanism.

Docker Image Improvements

  • CA Certificates Installation: Ensured that CA certificates are installed in our Docker images, enhancing security and enabling secure HTTPS communications within containerized environments.

These updates aim to enhance the functionality, security, and flexibility of Bacalhau.

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.3...v1.6.4

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.3

Docker Image Enhancements

  • New Docker-in-Docker (DinD) Image: Introduced a new Docker image variant specifically designed for compute nodes:

    • Added bacalhau:v1.6.3-dind and bacalhau:latest-dind images
    • Full Docker execution capability for compute nodes
    • Built-in Docker daemon support for container workloads
    • Requires privileged mode for Docker-in-Docker functionality

  • Image Variants Clarification:

    • Base image (bacalhau:v1.6.3, bacalhau:latest): Optimized for client usage, orchestrator nodes, and non-Docker compute workloads
    • DinD image (bacalhau:v1.6.3-dind, bacalhau:latest-dind): Designed for compute nodes requiring Docker execution capabilities

Operational Improvements

  • Docker Deployment:
    • Clearer separation of concerns between different node types
    • Improved documentation for Docker image usage

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.2...v1.6.3

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v.1.6.3

What's Changed

  • deploy v1.6.2 to demo network by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4834
  • build(deps): bump github.com/samber/lo from 1.47.0 to 1.49.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4828
  • publish docker-in-docker (dind) image by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4838

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.2...v.1.6.3

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.2

Performance and Stability Improvements

  • Enhanced Network Performance: Implemented batching and rate limiting of executions per scheduling cycle, ensuring all executions are processed while maintaining system stability when orchestrating jobs across hundreds of nodes
  • Improved Message Handling: Extended publish and subscribe timeout windows for more graceful network communication
  • Critical Bug Fixes:
    • Resolved boltdb transaction panic issues during context cancellation
    • Fixed execution re-approval logic in the scheduler
    • Enhanced transaction handling for better stability

Observability Enhancements

  • Comprehensive Metrics Coverage: Introduced OpenTelemetry metrics across multiple core components:
    • Job store metrics for storage monitoring
    • Scheduler metrics for job distribution insights
    • Planner metrics for execution planning visibility
    • NCL (Network Communication Layer) metrics
    • Message handler metrics for network communication monitoring

Operational Improvements

  • Development Environment:
    • Enhanced devstack logging format
    • Added support for node joining existing networks
    • Improved configuration passing mechanisms

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.1...v1.6.2

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.1

Major Improvements

  • Partitioned Execution Support: Added support for splitting jobs across multiple executions with automatic partition management. The feature includes:

    • Partition assignment and tracking
    • Independent execution progress monitoring
    • Granular failure handling with retry of only failed partitions
    • Each execution receives its partition details through environment variables, enabling partition-aware processing when needed

  • S3 Input Partitioning: Added automatic data distribution for S3 inputs across multiple executions using configurable strategies:

    • Multiple partitioning strategies: Users can choose between object-based distribution for even splitting, regex patterns for structured data, substring matching for fixed formats, or date-based partitioning for temporal data
    • Even distribution of data without requiring custom partition code
    • Support for shared data access through non-partitioned inputs
    • Automatic data subset assignment to each execution

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.0...v1.6.1

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.1-rc1

Major Improvements

  • Partitioned Execution Support: Added support for splitting jobs across multiple executions with automatic partition management. The feature includes:

    • Partition assignment and tracking
    • Independent execution progress monitoring
    • Granular failure handling with retry of only failed partitions
    • Each execution receives its partition details through environment variables, enabling partition-aware processing when needed

  • S3 Input Partitioning: Added automatic data distribution for S3 inputs across multiple executions using configurable strategies:

    • Multiple partitioning strategies: Users can choose between object-based distribution for even splitting, regex patterns for structured data, substring matching for fixed formats, or date-based partitioning for temporal data
    • Even distribution of data without requiring custom partition code
    • Support for shared data access through non-partitioned inputs
    • Automatic data subset assignment to each execution

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.0...v1.6.1-rc1

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.0

Bacalhau v1.6.0 Release Notes

We are excited to announce the release of Bacalhau v1.6.0, introducing a new communication architecture that significantly improves the reliability and resilience of distributed compute networks.

Key Features and Improvements

New NCL Protocol

At the heart of this release is the new NCL Protocol, a complete redesign of node communication that brings significant improvements to network reliability:

Key Benefits

  • Self-Healing Network: Compute nodes and orchestrators automatically reconnect and sync after network interruptions
  • Offline-First Operation: Compute nodes can start and operate even when disconnected from the orchestrator
  • Automatic State Recovery: When nodes reconnect, they automatically share all missed job execution information and results
  • Zero Data Loss: Ensures no job execution data or results are lost during network disruptions
  • Seamless Recovery: Network interruptions are handled transparently without requiring manual intervention

Technical Improvements

  • Reliable Message Delivery: Ordered, at-least-once message delivery between nodes
  • Automatic Recovery: Built-in failure detection and recovery mechanisms
  • Connection Health Monitoring: Proactive health checks and connection management
  • Event-Based Architecture: Decoupled event processing from message delivery
  • Efficient Checkpointing: Maintains system state for reliable recovery
  • Backward Compatibility: Maintains compatibility with v1.5 orchestrators

Enhanced Web UI Experience

  • Direct Result Downloads: Download job results directly from the interface
  • Simplified Configuration: Automatic request routing eliminates manual IP configuration
  • Improved Architecture: Streamlined backend setup while maintaining security

Operational Improvements

  • Reverse Proxy Support: Added capability to run orchestrator behind a reverse proxy
  • Agent Configuration: New bacalhau agent config command to inspect agent configuration
  • TLS Support: Added TLS encryption support for NATS communication
  • Better Logging: Implemented more human-readable logging patterns

Upgrade Notes and Backward Compatibility

Bacalhau v1.6.0 maintains backward compatibility while introducing the new NCL Protocol: - Compute nodes maintain compatibility with v1.5 orchestrators, and vice versa - Support for re-handshake from legacy clients

We're excited for you to experience the enhanced reliability and resilience provided by the NCL Protocol in Bacalhau v1.6.0. This release represents a significant architectural advancement in making distributed computing more robust and dependable.

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.0-rc4

What's Changed

  • Disable NCL tests as it is the default protocol by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4741
  • compute node backward compatibility with v1.5 orchestrators by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4761
  • improve orchestrator node self-registeration by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4762
  • more human readable logging pattern by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4763
  • fix lint failing due to sqlite3 missing by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4767
  • better exchange of starting seqNum during handshakes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4766
  • Add Shutdown Notice from Compute Nodes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4769
  • Fix race conditions resulting in multiple data planes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4771
  • Faster reconnect on handshake required response by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4772
  • Handle watchers with future sequence numbers gracefully by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4773
  • Propagate bacerrors over nclprotocol by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4774

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.0-rc3...v1.6.0-rc4

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.0-rc3

What's Changed

  • fix orchestrator data-plan startSeqNum by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4745

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.0-rc2...v1.6.0-rc3

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.0-rc2

What's Changed

  • always use orchestrator seqNum during handshake by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4740

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.0-rc1...v1.6.0-rc2

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.6.0-rc1

What's Changed

  • Release.1.5 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4594
  • Bump next from 14.2.7 to 14.2.10 in /webui by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4453
  • Improve API error handling by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4607
  • Support running Bacalhau in Docker compose by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4596
  • fix cspell and disable golangci-lint spellcheck by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4610
  • remove event emitter by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4601
  • remove pluggable executors by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4603
  • remove exec command and job translation by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4604
  • organize utils and remove unused ones by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4602
  • validate swagger.json by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4619
  • build(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.28.0 to 1.31.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4617
  • Support TestContainers PoC by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4627
  • Migrate BashTub Tests to TestContainers Based Tests by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4631
  • fix: default publishers not working by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4646
  • fix test flakiness by useing busybox instead of ubuntu images by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4655
  • fix: stop jobs with short ids by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4657
  • Improve startup time by optimizing IMDS access by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4649
  • improve routing of webui requests by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4645
  • Fix Go lint issues by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4647
  • Prevent False Positives in CI by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4642
  • Remove Depandabot Auto Merges and CircleCI References by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4660
  • build(deps): bump actions/checkout from 2 to 4 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4661
  • build(deps): bump peter-evans/repository-dispatch from 1 to 3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4662
  • build(deps): bump github.com/swaggo/swag from 1.16.2 to 1.16.4 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4632
  • release 1.5.1 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4668
  • build(deps): bump werkzeug from 2.2.3 to 3.0.6 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4666
  • build(deps): bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.16.5 to 1.17.34 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4667
  • Improve Github Actions Workflow and GoLint by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4672
  • integrate lib/watcher with job store by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4676
  • refactor execution store to use models.Execution by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4677
  • fix logstream test flakiness by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4678
  • speed up tests by using tagged images instead of latest by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4679
  • move messaging models from pkg/compute to pkg/models/messages by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4680
  • build(deps): bump go.opentelemetry.io/otel/sdk/metric from 1.28.0 to 1.31.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4673
  • feat: adds agent config command that returns the agent configuration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4671
  • async execution handling by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4683
  • add compute execution-flow doc and remove outdated docs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4685
  • Support NATS TLS Communication by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4686
  • skip flaky logstream tests by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4693
  • prevent port number reuse with TTL-based caching by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4689
  • ncl based async communication between nodes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4687
  • decouple envelope from ncl by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4695
  • fix swagger generation by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4699
  • Add missing execution state types to definitions by @markkuhn in https://github.com/bacalhau-project/bacalhau/pull/4698
  • Display full command for next results in job list pagination by @markkuhn in https://github.com/bacalhau-project/bacalhau/pull/4691
  • Add job result download option to WebUI by @markkuhn in https://github.com/bacalhau-project/bacalhau/pull/4702
  • build(deps): bump apache-airflow from 2.9.3 to 2.10.3 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4701
  • login docker hub access in buildkite by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4707
  • use new update checker endpoint by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4706
  • build(deps): bump aiohttp from 3.9.4 to 3.10.11 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4708
  • build(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /webui by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4709
  • build(deps): bump github.com/nats-io/nats-server/v2 from 2.10.20 to 2.10.22 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4710
  • fix nats server panic on shutdown by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4712
  • improve watcher creation pattern and manager naming by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4713
  • expose backoff.BackoffDuration method by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4714
  • build(deps): bump github.com/samber/lo from 1.39.0 to 1.47.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4711
  • build(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc from 1.28.0 to 1.32.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4715
  • ncl: introduce ordered publisher by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4717
  • build(deps): bump go.uber.org/mock from 0.4.0 to 0.5.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4718
  • Add Transport Package with Event Publishing Components by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4721
  • build(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp from 1.28.0 to 1.32.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4722
  • Support Running Orchestrator behind a Reverse Proxy by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4724
  • Switch to ubuntu 22.04 from ChainGaurd docker image by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4725
  • new node manager decoupled from transport layer by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4728
  • authenticate earthly actions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4729
  • enable ephemeral watchers by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4730
  • Introduce Request/Response Messaging in NCL by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4731
  • extract bprotocol transport layer from pkg/node by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4732
  • Add persisted sequence tracking and checkpoint support by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4733
  • Allow to pass URL as the API Host for CLI by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4726
  • NCL Protocol by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4734
  • Support re-handshake from legacy clients by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4739
  • backward compatible metadata keys by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4738

New Contributors

  • @jamlo made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4607
  • @markkuhn made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4698

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.5.0...v1.6.0-rc1

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.5.2

What's Changed

  • build(deps): bump werkzeug from 2.2.3 to 3.0.6 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4666
  • build(deps): bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.16.5 to 1.17.34 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4667
  • Improve Github Actions Workflow and GoLint by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4672
  • integrate lib/watcher with job store by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4676
  • refactor execution store to use models.Execution by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4677
  • fix logstream test flakiness by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4678
  • speed up tests by using tagged images instead of latest by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4679
  • move messaging models from pkg/compute to pkg/models/messages by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4680
  • build(deps): bump go.opentelemetry.io/otel/sdk/metric from 1.28.0 to 1.31.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4673
  • feat: adds agent config command that returns the agent configuration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4671
  • async execution handling by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4683
  • add compute execution-flow doc and remove outdated docs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4685
  • Support NATS TLS Communication by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4686
  • skip flaky logstream tests by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4693
  • prevent port number reuse with TTL-based caching by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4689
  • ncl based async communication between nodes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4687
  • decouple envelope from ncl by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4695
  • fix swagger generation by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4699
  • Add missing execution state types to definitions by @markkuhn in https://github.com/bacalhau-project/bacalhau/pull/4698
  • Display full command for next results in job list pagination by @markkuhn in https://github.com/bacalhau-project/bacalhau/pull/4691
  • Add job result download option to WebUI by @markkuhn in https://github.com/bacalhau-project/bacalhau/pull/4702
  • build(deps): bump apache-airflow from 2.9.3 to 2.10.3 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4701
  • login docker hub access in buildkite by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4707
  • use new update checker endpoint by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4706
  • build(deps): bump aiohttp from 3.9.4 to 3.10.11 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4708
  • build(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /webui by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4709
  • build(deps): bump github.com/nats-io/nats-server/v2 from 2.10.20 to 2.10.22 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4710
  • fix nats server panic on shutdown by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4712
  • improve watcher creation pattern and manager naming by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4713
  • expose backoff.BackoffDuration method by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4714
  • build(deps): bump github.com/samber/lo from 1.39.0 to 1.47.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4711
  • build(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc from 1.28.0 to 1.32.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4715
  • ncl: introduce ordered publisher by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4717
  • build(deps): bump go.uber.org/mock from 0.4.0 to 0.5.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4718
  • Add Transport Package with Event Publishing Components by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4721
  • build(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp from 1.28.0 to 1.32.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4722
  • Support Running Orchestrator behind a Reverse Proxy by @jamlo in https://github.com/bacalhau-project/bacalhau/pull/4724

New Contributors

  • @markkuhn made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4698

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.5.1...v1.5.2

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.5.1

Major Improvements

  • Enhanced Web UI Routing: Improved routing of Web UI requests without requiring backend address definition
  • Faster Startup: Dramatically reduced node startup time from ~9 seconds to ~1.5 seconds by optimizing IMDS access
  • Job Management: Added support for stopping jobs using short IDs
  • Bug Fix: Resolved issues with default publishers functionality

Breaking Changes

  • Removed exec command and job translation functionality

Additional Changes

  • Added Docker compose support
  • Improved API error handling

Links

  • Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.5.0...v1.5.1

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.5.1-rc1

Major Improvements

  • Enhanced Web UI Routing: Improved routing of Web UI requests without requiring backend address definition
  • Faster Startup: Dramatically reduced node startup time from ~9 seconds to ~1.5 seconds by optimizing IMDS access
  • Job Management: Added support for stopping jobs using short IDs
  • Bug Fix: Resolved issues with default publishers functionality

Breaking Changes

  • Removed exec command and job translation functionality

Additional Changes

  • Added Docker compose support
  • Improved API error handling

Links

  • Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.5.0...v1.5.1-rc1

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.5.0

Bacalhau v1.5 Release Notes

We're thrilled to announce the release of Bacalhau 1.5.0, a significant update that introduces powerful new features and enhancements. Building on the momentum from our previous releases, Bacalhau 1.5 focuses on simplifying configuration, improving visibility, and enhancing overall performance.

Key Features and Improvements

Simplified Configuration Management

  • New File-Based Configuration System: We've introduced a more intuitive file-based configuration system, replacing complex CLI flags. This change makes setting up and managing Bacalhau networks more straightforward and less error-prone.
  • Flexible Configuration Options: Users can now provide:
    • A single config file
    • Multiple config files that are merged
    • Key-value pairs directly via the -c - flag (e.g., -c key=value)
  • Decoupled Configuration: Configuration is now decoupled from the repo (now called data dir), allowing for more flexible setups.

Enhanced Data Directory Structure

  • Improved Organization: We've clearly separated compute and orchestrator related data, providing a cleaner structure.
  • Consolidated Metadata: System metadata is now consolidated into a single system_metadata.json file for easier management.

New WebUI

  • Embedded Management Interface: Introduced a comprehensive WebUI for easier management and monitoring of your Bacalhau network. This significant feature allows users to visualize and interact with their Bacalhau deployment without relying solely on the CLI.

Enhanced Job Visibility and Reporting

  • Granular Event Reporting: Improved reporting on job progress, including detailed scheduling actions, failures, and retries.
  • Better Error Messages: Enhanced error reporting system with meaningful messages and debugging hints.

API Enhancements

  • Pagination for Job History: Implemented pagination support for job history, improving the user experience when dealing with a large jobs and making it easier to navigate through job and execution history events.

Upgrade Notes and Backward Compatibility

While Bacalhau 1.5.0 introduces some breaking changes, we've ensured a smooth upgrade path:

  • Most CLI flags have been removed in favor of configuration files, but we gracefully handle deprecated flags for backward compatibility.
  • The structure of the data directory has changed, but we automatically handle the migration when you first run the new Bacalhau version.
  • Many old configuration options have been deprecated in favor of the new structure and config keys.

Please refer to our updated documentation for detailed instructions on upgrading to Bacalhau 1.5.0 and taking advantage of the new configuration system.

We're excited for you to explore the new features and enhancements in Bacalhau 1.5.0. Whether you're a seasoned Bacalhau user or just getting started, this update will empower you to build and run distributed compute networks more effectively than ever before.

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.5.0-alpha3

What's Changed

  • Update README.md by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4019
  • deprecate macOS intel tests by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4183
  • Frrist/tests/migrate to v2 client by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4147
  • refactor: migrate exec command to use models by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4155
  • Remove the model package by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4156
  • Bump github.com/hashicorp/go-retryablehttp from 0.7.5 to 0.7.7 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4135
  • Fix Python SDK by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4194
  • api/v1/orchestrator/jobs/{id}/executions endpoint, move sorting to job store rather then having it at endpoint level by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4193
  • Fix MakeFile for Releasing Python API Client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4203
  • fix makefile for release of python sdk by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4204
  • Fix a typo in python sdk by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4208
  • Fix docker release by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4212
  • Remove Un-necessary tries to mark execution complete when it has already been cancelled by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4199
  • Fix Bacalhau Airflow Build by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4240
  • Pagination Support for Job History by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4223
  • add Queued state and history event by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4115
  • Displaying Detailed Job Progress While Using Run Command by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4254
  • Move to Large Instances for our CI by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4256
  • Fix printing deprecated logs command by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4253
  • introduce nats client library (ncl) by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4257
  • Fix requiring even number of docker env vars by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4258
  • implement system_metadata.yaml replaces repo.version by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4237
  • Integration for Buildkite by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4294
  • Testsuite Integration for Buildkite by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4309
  • add persisted watcher library by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4307
  • Fix Test.sh Script by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4310
  • scale-down demo and staging networks by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4303
  • Bump github.com/docker/docker from 25.0.4+incompatible to 27.1.1+incompatible by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4295
  • Bump apache-airflow from 2.9.0 to 2.9.3 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4241
  • Bump go.opentelemetry.io/otel/sdk from 1.26.0 to 1.28.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4201
  • validate duplicate inputs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4315
  • Implement -c/--config flag by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4242
  • Release Pipeline Readiness for Go Binaries by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4326
  • Frrist/update golang1.23.0 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4332
  • drat: first pass at decoupling the repo and config paths by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4330
  • Execution store refactor by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4334
  • Unskip Tests by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4348
  • Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.51.0 to 1.61.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/4349
  • Fix Release CI Pipeline by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4358
  • fix pep440 build errors with releases with alpha, beta and dev suffixes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4360
  • feat: implement configv2, types, and tests by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4333
  • fix: node type trinary state by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4400
  • Introduce APIError for Sync Calls from CLI to Orchestrator Node by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4366
  • fix: remove env vars from config list output by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4399
  • do not print usage and error and infer error codes in custom error handler by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4405
  • fix migration, tests for more repo types by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4398
  • Enhanced WebUI by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4406
  • webui error handling + openapi gen update by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4413
  • Propogating centralized errors by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4412
  • chore: add node name flag to serve command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4410
  • Frrist/meta/instanceid by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4411

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.4.0...v1.5.0-alpha3

- Go
Published by wdbaruni over 1 year ago

https://github.com/bacalhau-project/bacalhau - v1.4.0

Announcing Bacalhau 1.4.0

We’re excited to announce the release of Bacalhau 1.4.0, a significant update that introduces powerful new features and enhancements. Building on the momentum from our previous releases this year (1.2.0, 1.3.0, 1.3.1, and 1.3.2), Bacalhau 1.4 strengthens our platform’s performance, scalability, and user experience, solidifying its position as a leading platform for building and running distributed compute networks.

In this release, we focused on three major efforts, with particular emphasis on those deploying Bacalhau at scale:

Performance and Scalability Enhancements

  • Extended Job Queuing: Bacalhau 1.4.0 introduces a more robust queuing system, improving job scheduling and execution efficiency, especially in high-demand or globally distributed networks. By intelligently managing job queues, Bacalhau ensures smoother operations and increased throughput, leading to higher success rates for your distributed compute tasks.

  • Migration to NATS, Deprecation of libp2p and Embedded IPFS Node: We’ve fully transitioned to NATS.io as Bacalhau’s communication backbone, moving away from libp2p and the embedded IPFS node. This change streamlines communication and reduces overhead, marking a significant step towards a more efficient and scalable network. IPFS integration remains available with external nodes for those who need it. 

Improved User Experience

  • Updated CLI and HTTP API: Bacalhau 1.4.0 introduces a revamped command-line interface (CLI) and HTTP API. These updates align the CLI commands with the new API structure and enhance overall usability. While most changes are seamless for existing users, some command adjustments have been made (e.g., bacalhau create becomes bacalhau job run). Our updated documentation will guide you through the transition smoothly. 

  • Job Spec Updates: We've introduced an updated Job Specification format while deprecating some features of the previous format. This change requires users to update their job specs but brings improved clarity and consistency.

  • Enhanced Error Reporting: Bacalhau 1.4.0 improves error reporting, making it easier to diagnose and troubleshoot issues. This enhancement contributes to a more stable and reliable experience, helping users quickly resolve any problems that arise. For detailed guidance, please consult our documentation on the new Job Spec requirements.

  • Introduction of Node Manager:  In Bacalhau 1.4.0, we’re introducing the Node Manager. This feature simplifies node operations, providing a clear view of all compute nodes and their status. You can approve, deny, or delete nodes as needed, making management straightforward. Heartbeats from nodes keep the Node Manager updated on their connectivity, enhancing overall stability and performance.

Smooth Transition for Existing Users

  • Error Handling and Guidance: We understand that transitioning to a new version can be challenging. To ease this process, we’ve implemented helpful error messages and guidance for those adjusting to the changes in CLI behavior and job specifications. We’ve also created a table to show how some of the Bacalhau API endpoints have been remapped. If you’re not ready to upgrade, you can continue using version 1.3.1 while maintaining your private Bacalhau cluster.

Join Us on the Journey

We’re excited for you to explore the new features and enhancements in Bacalhau 1.4.0. Over the next five days, we’ll dive deeper into each topic in our “5 Days of Bacalhau” blog series. Whether you’re a seasoned Bacalhau user or just getting started, this update will empower you to build and run distributed compute networks more effectively than ever before.

- Go
Published by wdbaruni almost 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.4.0-rc5

What's Changed

  • Update prod to v1.3.0. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3708
  • fix: update bacalhau install script to use go v1.21.8 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3675
  • Add authentication to the Web UI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3711
  • fix: re-use client TLS config for establishing ws connections by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3722
  • Add Basic Bashtub tests by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3669
  • Replace use of multierror libraries with now standardised errors.Join by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3744
  • Load WASM module dependencies if non-WASM input data is specified by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3747
  • Adds a node delete command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3716
  • Add error output to job selection exec hooks by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3745
  • feat: create a bacalhau user to run bacalhau process by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3733
  • refactor: bidder to simplfiy exposing errors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3680
  • Improve error reporting when job resources exceed capacity by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3749
  • Print ranking failures in the CLI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3752
  • Return failed bid when image does not exist by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3755
  • Generate certificates requester three by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3707
  • Just a bunch of spelling fixes by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3760
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp from 0.45.0 to 1.24.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3615
  • Improve ranking error output by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3763
  • Node heartbeats by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3709
  • Ensure compute node labels are published over LibP2P connections by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3769
  • Use RunE instead of Run for cobra commands by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3767
  • Optionally constrain to connected, approved compute nodes when selecting/ranking them by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3768
  • Print command line executable when displaying additional commands by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3770
  • Deprecates some IPFS flags by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3778
  • Make it easier to get in progress jobs of a certain type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3777
  • Fixes a panic in the wasm executor when entrypoint is not known by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3779
  • Structured events in place of comment strings by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3771
  • fix: remove Liveness and Acceptance from NodeInfo by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3785
  • Add issue templates by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3951
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3809
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3808
  • Bump idna from 3.3 to 3.7 in /ops by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3786
  • Bump idna from 3.6 to 3.7 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3788
  • Bump idna from 3.6 to 3.7 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3789
  • Bump sqlparse from 0.4.4 to 0.5.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3792
  • Bump gunicorn from 21.2.0 to 22.0.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3794
  • Bump aiohttp from 3.9.3 to 3.9.4 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3796
  • Bump apache-airflow from 2.8.1 to 2.9.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3798
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.23.1 to 1.26.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3955
  • Removing placeholders by @MichaelHoepler in https://github.com/bacalhau-project/bacalhau/pull/3795
  • adding ok-to-test functionality by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3960
  • 3444 adding all fake data by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3445
  • feat: premit update check configuration w/ envvar by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3967
  • fix: prevent panic in test due to nil pointer dereference by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3977
  • fix: require connected and approved nodes for scheduling by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3957
  • fix: allow timeouts in the client methods that make API call by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3964
  • refactor: add deprecation message to libp2p & ipfs flags by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3966
  • Support MP deployments with self-generated TLS certificate by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3721
  • fix: update checker doesn't require server connection to work by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3775
  • ExecutionTimeout to fail executions instead of jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3974
  • adding update checker test env variable by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3997
  • Fix Repo Clone by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3973
  • urldownload GetVolumeSize check for content length by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4006
  • add atomic job store transactions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3996
  • compute nodes no longer reject bids based on queued capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4002
  • fix txContext auto-cancellation of context by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4012
  • single requester node in demo networks by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3673
  • fixing circle ci commenting by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4022
  • run demo network on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4009
  • add evaluation watcher and enable atomic enqueues by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3998
  • make job name optional by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4005
  • fix: implement NodeStore migration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4029
  • Splitting to Consumer and Producer Client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4027
  • refactor: define config instance. config is no longer global by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3959
  • ops: update prod cluster vars to reflect current state by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4034
  • introduce job queueing when no nodes were found by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4049
  • filter out nodes with high queue capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4051
  • Update canaries to 1.3.1 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4039
  • fail jobs exceeding TotalTimeout by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4055
  • fix canary after config changes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4056
  • Migrate docker CLI to API v2 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4020
  • refactor: migrate wasm CLI to v2 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4060
  • fully remove embedded ipfs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4061
  • refactor: deprecate and remove cancel command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4066
  • refactor: deprecate and remove create command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4064
  • Aronchick remove canaries by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4076
  • Frrist/cli/move v1 cli by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4071
  • remove libp2p by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4073
  • added get to job type command by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4079
  • circleci cache ipfs + previous PR fixes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4069
  • CircleCI fixes to allow for external kickoff of CircleCI by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4077
  • Debugging GH Secrets by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4087
  • fix ipfs storage test flakiness by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4086
  • refactor: move get command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4072
  • fix: ipfs client HasCID performes offline check by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4088
  • Add .circleci/config.yml by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4091
  • Adding fork to upstream workflow by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4090
  • merge by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4093
  • Add .circleci/config.yml by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4098
  • Aronchick adding execution to script by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4100
  • cli: print get command after job run by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4089
  • Optimize locking in producer and consumer NATS streaming client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4068
  • add --queue-timeout flag by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4080
  • feat: implement UnmarshalJob method with clear errors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4103
  • Removing flyte by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4107
  • Remove docs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4112
  • refactor: deprecate requester api endpoint by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4109
  • refactor: migrate List*Response names to Items by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4111
  • refactor: move validate command to job validate by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4108
  • fix: a follow on to #4109 that wasn't caught in CI by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4120
  • fix: allow unlimited reconnect attempts on compute nats client by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4116
  • add job Queued state by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4121
  • Update README.md by @jmsingleton in https://github.com/bacalhau-project/bacalhau/pull/3982
  • test flakiness improvements by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4122
  • set minimum client version on http server by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4117

New Contributors

  • @udsamani made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4006
  • @jmsingleton made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/3982

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.3.0...v1.4.0-rc5

- Go
Published by wdbaruni almost 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.4.0-rc1

What's Changed

  • Update prod to v1.3.0. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3708
  • fix: update bacalhau install script to use go v1.21.8 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3675
  • Add authentication to the Web UI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3711
  • fix: re-use client TLS config for establishing ws connections by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3722
  • Add Basic Bashtub tests by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3669
  • Replace use of multierror libraries with now standardised errors.Join by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3744
  • Load WASM module dependencies if non-WASM input data is specified by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3747
  • Adds a node delete command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3716
  • Add error output to job selection exec hooks by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3745
  • feat: create a bacalhau user to run bacalhau process by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3733
  • refactor: bidder to simplfiy exposing errors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3680
  • Improve error reporting when job resources exceed capacity by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3749
  • Print ranking failures in the CLI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3752
  • Return failed bid when image does not exist by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3755
  • Generate certificates requester three by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3707
  • Just a bunch of spelling fixes by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3760
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp from 0.45.0 to 1.24.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3615
  • Improve ranking error output by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3763
  • Node heartbeats by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3709
  • Ensure compute node labels are published over LibP2P connections by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3769
  • Use RunE instead of Run for cobra commands by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3767
  • Optionally constrain to connected, approved compute nodes when selecting/ranking them by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3768
  • Print command line executable when displaying additional commands by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3770
  • Deprecates some IPFS flags by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3778
  • Make it easier to get in progress jobs of a certain type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3777
  • Fixes a panic in the wasm executor when entrypoint is not known by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3779
  • Structured events in place of comment strings by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3771
  • fix: remove Liveness and Acceptance from NodeInfo by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3785
  • Add issue templates by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3951
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3809
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3808
  • Bump idna from 3.3 to 3.7 in /ops by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3786
  • Bump idna from 3.6 to 3.7 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3788
  • Bump idna from 3.6 to 3.7 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3789
  • Bump sqlparse from 0.4.4 to 0.5.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3792
  • Bump gunicorn from 21.2.0 to 22.0.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3794
  • Bump aiohttp from 3.9.3 to 3.9.4 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3796
  • Bump apache-airflow from 2.8.1 to 2.9.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3798
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.23.1 to 1.26.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3955
  • Removing placeholders by @MichaelHoepler in https://github.com/bacalhau-project/bacalhau/pull/3795
  • adding ok-to-test functionality by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3960
  • 3444 adding all fake data by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3445
  • feat: premit update check configuration w/ envvar by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3967
  • fix: prevent panic in test due to nil pointer dereference by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3977
  • fix: require connected and approved nodes for scheduling by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3957
  • fix: allow timeouts in the client methods that make API call by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3964
  • refactor: add deprecation message to libp2p & ipfs flags by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3966
  • Support MP deployments with self-generated TLS certificate by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3721
  • fix: update checker doesn't require server connection to work by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3775
  • ExecutionTimeout to fail executions instead of jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3974
  • adding update checker test env variable by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3997
  • Fix Repo Clone by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3973
  • urldownload GetVolumeSize check for content length by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4006
  • add atomic job store transactions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3996
  • compute nodes no longer reject bids based on queued capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4002
  • fix txContext auto-cancellation of context by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4012
  • single requester node in demo networks by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3673
  • fixing circle ci commenting by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4022
  • run demo network on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4009
  • add evaluation watcher and enable atomic enqueues by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3998
  • make job name optional by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4005
  • fix: implement NodeStore migration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4029
  • Splitting to Consumer and Producer Client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4027
  • refactor: define config instance. config is no longer global by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3959
  • ops: update prod cluster vars to reflect current state by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4034
  • introduce job queueing when no nodes were found by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4049
  • filter out nodes with high queue capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4051
  • Update canaries to 1.3.1 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4039
  • fail jobs exceeding TotalTimeout by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4055
  • fix canary after config changes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4056
  • Migrate docker CLI to API v2 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4020
  • refactor: migrate wasm CLI to v2 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4060
  • fully remove embedded ipfs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4061
  • refactor: deprecate and remove cancel command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4066
  • refactor: deprecate and remove create command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4064
  • Aronchick remove canaries by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4076
  • Frrist/cli/move v1 cli by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4071
  • remove libp2p by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4073
  • added get to job type command by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4079
  • circleci cache ipfs + previous PR fixes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4069
  • CircleCI fixes to allow for external kickoff of CircleCI by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4077
  • Debugging GH Secrets by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4087
  • fix ipfs storage test flakiness by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4086
  • refactor: move get command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4072
  • fix: ipfs client HasCID performes offline check by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4088
  • Add .circleci/config.yml by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4091
  • Adding fork to upstream workflow by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4090
  • merge by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4093
  • Add .circleci/config.yml by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4098
  • Aronchick adding execution to script by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4100
  • cli: print get command after job run by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4089
  • Optimize locking in producer and consumer NATS streaming client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4068
  • add --queue-timeout flag by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4080
  • feat: implement UnmarshalJob method with clear errors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4103
  • Removing flyte by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4107
  • Remove docs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4112
  • refactor: deprecate requester api endpoint by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4109
  • refactor: migrate List*Response names to Items by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4111
  • refactor: move validate command to job validate by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4108
  • fix: a follow on to #4109 that wasn't caught in CI by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4120
  • fix: allow unlimited reconnect attempts on compute nats client by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4116
  • add job Queued state by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4121
  • Update README.md by @jmsingleton in https://github.com/bacalhau-project/bacalhau/pull/3982
  • test flakiness improvements by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4122
  • set minimum client version on http server by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4117

New Contributors

  • @udsamani made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4006
  • @jmsingleton made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/3982

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.3.0...v1.4.0-rc1

- Go
Published by wdbaruni almost 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.4.0

What's Changed

  • Update prod to v1.3.0. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3708
  • fix: update bacalhau install script to use go v1.21.8 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3675
  • Add authentication to the Web UI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3711
  • fix: re-use client TLS config for establishing ws connections by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3722
  • Add Basic Bashtub tests by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3669
  • Replace use of multierror libraries with now standardised errors.Join by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3744
  • Load WASM module dependencies if non-WASM input data is specified by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3747
  • Adds a node delete command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3716
  • Add error output to job selection exec hooks by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3745
  • feat: create a bacalhau user to run bacalhau process by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3733
  • refactor: bidder to simplfiy exposing errors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3680
  • Improve error reporting when job resources exceed capacity by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3749
  • Print ranking failures in the CLI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3752
  • Return failed bid when image does not exist by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3755
  • Generate certificates requester three by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3707
  • Just a bunch of spelling fixes by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3760
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp from 0.45.0 to 1.24.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3615
  • Improve ranking error output by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3763
  • Node heartbeats by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3709
  • Ensure compute node labels are published over LibP2P connections by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3769
  • Use RunE instead of Run for cobra commands by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3767
  • Optionally constrain to connected, approved compute nodes when selecting/ranking them by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3768
  • Print command line executable when displaying additional commands by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3770
  • Deprecates some IPFS flags by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3778
  • Make it easier to get in progress jobs of a certain type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3777
  • Fixes a panic in the wasm executor when entrypoint is not known by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3779
  • Structured events in place of comment strings by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3771
  • fix: remove Liveness and Acceptance from NodeInfo by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3785
  • Add issue templates by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3951
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3809
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3808
  • Bump idna from 3.3 to 3.7 in /ops by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3786
  • Bump idna from 3.6 to 3.7 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3788
  • Bump idna from 3.6 to 3.7 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3789
  • Bump sqlparse from 0.4.4 to 0.5.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3792
  • Bump gunicorn from 21.2.0 to 22.0.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3794
  • Bump aiohttp from 3.9.3 to 3.9.4 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3796
  • Bump apache-airflow from 2.8.1 to 2.9.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3798
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.23.1 to 1.26.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3955
  • Removing placeholders by @MichaelHoepler in https://github.com/bacalhau-project/bacalhau/pull/3795
  • adding ok-to-test functionality by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3960
  • 3444 adding all fake data by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3445
  • feat: premit update check configuration w/ envvar by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3967
  • fix: prevent panic in test due to nil pointer dereference by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3977
  • fix: require connected and approved nodes for scheduling by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3957
  • fix: allow timeouts in the client methods that make API call by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3964
  • refactor: add deprecation message to libp2p & ipfs flags by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3966
  • Support MP deployments with self-generated TLS certificate by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3721
  • fix: update checker doesn't require server connection to work by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3775
  • ExecutionTimeout to fail executions instead of jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3974
  • adding update checker test env variable by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3997
  • Fix Repo Clone by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3973
  • urldownload GetVolumeSize check for content length by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4006
  • add atomic job store transactions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3996
  • compute nodes no longer reject bids based on queued capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4002
  • fix txContext auto-cancellation of context by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4012
  • single requester node in demo networks by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3673
  • fixing circle ci commenting by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4022
  • run demo network on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4009
  • add evaluation watcher and enable atomic enqueues by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3998
  • make job name optional by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4005
  • fix: implement NodeStore migration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4029
  • Splitting to Consumer and Producer Client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4027
  • refactor: define config instance. config is no longer global by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3959
  • ops: update prod cluster vars to reflect current state by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4034
  • introduce job queueing when no nodes were found by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4049
  • filter out nodes with high queue capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4051
  • Update canaries to 1.3.1 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4039
  • fail jobs exceeding TotalTimeout by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4055
  • fix canary after config changes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4056
  • Migrate docker CLI to API v2 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4020
  • refactor: migrate wasm CLI to v2 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4060
  • fully remove embedded ipfs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4061
  • refactor: deprecate and remove cancel command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4066
  • refactor: deprecate and remove create command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4064
  • Aronchick remove canaries by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4076
  • Frrist/cli/move v1 cli by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4071
  • remove libp2p by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4073
  • added get to job type command by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4079
  • circleci cache ipfs + previous PR fixes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4069
  • CircleCI fixes to allow for external kickoff of CircleCI by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4077
  • Debugging GH Secrets by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4087
  • fix ipfs storage test flakiness by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4086
  • refactor: move get command by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4072
  • fix: ipfs client HasCID performes offline check by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4088
  • Add .circleci/config.yml by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4091
  • Adding fork to upstream workflow by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4090
  • merge by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4093
  • Add .circleci/config.yml by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4098
  • Aronchick adding execution to script by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4100
  • cli: print get command after job run by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4089
  • Optimize locking in producer and consumer NATS streaming client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4068
  • add --queue-timeout flag by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4080
  • feat: implement UnmarshalJob method with clear errors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4103
  • Removing flyte by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4107
  • Remove docs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4112
  • refactor: deprecate requester api endpoint by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4109
  • refactor: migrate List*Response names to Items by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4111
  • refactor: move validate command to job validate by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4108
  • fix: a follow on to #4109 that wasn't caught in CI by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4120
  • fix: allow unlimited reconnect attempts on compute nats client by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4116
  • add job Queued state by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4121
  • Update README.md by @jmsingleton in https://github.com/bacalhau-project/bacalhau/pull/3982
  • test flakiness improvements by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4122
  • set minimum client version on http server by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4117

New Contributors

  • @udsamani made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4006
  • @jmsingleton made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/3982

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.3.0...v1.4.0

- Go
Published by wdbaruni almost 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.2

What's Changed

  • Splitting to Consumer and Producer Client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4027
  • refactor: define config instance. config is no longer global by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3959
  • ops: update prod cluster vars to reflect current state by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4034
  • introduce job queueing when no nodes were found by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4049
  • filter out nodes with high queue capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4051
  • Update canaries to 1.3.1 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4039

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.3.1...v1.3.2

- Go
Published by wdbaruni almost 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.2-rc2

What's Changed

  • Splitting to Consumer and Producer Client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4027

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.3.1...v1.3.2-rc2

- Go
Published by wdbaruni about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.2-rc1

What's Changed

  • Splitting to Consumer and Producer Client by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4027

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.3.1...v1.3.2-rc1

- Go
Published by wdbaruni about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.1

What's Changed

  • Update prod to v1.3.0. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3708
  • fix: update bacalhau install script to use go v1.21.8 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3675
  • Add authentication to the Web UI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3711
  • fix: re-use client TLS config for establishing ws connections by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3722
  • Add Basic Bashtub tests by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3669
  • Replace use of multierror libraries with now standardised errors.Join by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3744
  • Load WASM module dependencies if non-WASM input data is specified by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3747
  • Adds a node delete command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3716
  • Add error output to job selection exec hooks by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3745
  • feat: create a bacalhau user to run bacalhau process by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3733
  • refactor: bidder to simplfiy exposing errors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3680
  • Improve error reporting when job resources exceed capacity by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3749
  • Print ranking failures in the CLI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3752
  • Return failed bid when image does not exist by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3755
  • Generate certificates requester three by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3707
  • Just a bunch of spelling fixes by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3760
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp from 0.45.0 to 1.24.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3615
  • Improve ranking error output by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3763
  • Node heartbeats by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3709
  • Ensure compute node labels are published over LibP2P connections by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3769
  • Use RunE instead of Run for cobra commands by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3767
  • Optionally constrain to connected, approved compute nodes when selecting/ranking them by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3768
  • Print command line executable when displaying additional commands by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3770
  • Deprecates some IPFS flags by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3778
  • Make it easier to get in progress jobs of a certain type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3777
  • Fixes a panic in the wasm executor when entrypoint is not known by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3779
  • Structured events in place of comment strings by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3771
  • fix: remove Liveness and Acceptance from NodeInfo by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3785
  • Add issue templates by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3951
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3809
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3808
  • Bump idna from 3.3 to 3.7 in /ops by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3786
  • Bump idna from 3.6 to 3.7 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3788
  • Bump idna from 3.6 to 3.7 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3789
  • Bump sqlparse from 0.4.4 to 0.5.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3792
  • Bump gunicorn from 21.2.0 to 22.0.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3794
  • Bump aiohttp from 3.9.3 to 3.9.4 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3796
  • Bump apache-airflow from 2.8.1 to 2.9.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3798
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.23.1 to 1.26.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3955
  • Removing placeholders by @MichaelHoepler in https://github.com/bacalhau-project/bacalhau/pull/3795
  • adding ok-to-test functionality by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3960
  • 3444 adding all fake data by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3445
  • feat: premit update check configuration w/ envvar by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3967
  • fix: prevent panic in test due to nil pointer dereference by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3977
  • fix: require connected and approved nodes for scheduling by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3957
  • fix: allow timeouts in the client methods that make API call by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3964
  • refactor: add deprecation message to libp2p & ipfs flags by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3966
  • Support MP deployments with self-generated TLS certificate by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3721
  • fix: update checker doesn't require server connection to work by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3775
  • ExecutionTimeout to fail executions instead of jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3974
  • adding update checker test env variable by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3997
  • Fix Repo Clone by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3973
  • urldownload GetVolumeSize check for content length by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4006
  • add atomic job store transactions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3996
  • compute nodes no longer reject bids based on queued capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4002
  • fix txContext auto-cancellation of context by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4012
  • single requester node in demo networks by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3673
  • fixing circle ci commenting by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4022
  • run demo network on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4009
  • add evaluation watcher and enable atomic enqueues by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3998
  • make job name optional by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4005
  • fix: implement NodeStore migration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4029

New Contributors

  • @udsamani made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4006

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.3.0...v1.3.1

- Go
Published by frrist about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.1-rc2

What's Changed

  • Update prod to v1.3.0. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3708
  • fix: update bacalhau install script to use go v1.21.8 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3675
  • Add authentication to the Web UI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3711
  • fix: re-use client TLS config for establishing ws connections by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3722
  • Add Basic Bashtub tests by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3669
  • Replace use of multierror libraries with now standardised errors.Join by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3744
  • Load WASM module dependencies if non-WASM input data is specified by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3747
  • Adds a node delete command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3716
  • Add error output to job selection exec hooks by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3745
  • feat: create a bacalhau user to run bacalhau process by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3733
  • refactor: bidder to simplfiy exposing errors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3680
  • Improve error reporting when job resources exceed capacity by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3749
  • Print ranking failures in the CLI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3752
  • Return failed bid when image does not exist by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3755
  • Generate certificates requester three by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3707
  • Just a bunch of spelling fixes by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3760
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp from 0.45.0 to 1.24.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3615
  • Improve ranking error output by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3763
  • Node heartbeats by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3709
  • Ensure compute node labels are published over LibP2P connections by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3769
  • Use RunE instead of Run for cobra commands by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3767
  • Optionally constrain to connected, approved compute nodes when selecting/ranking them by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3768
  • Print command line executable when displaying additional commands by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3770
  • Deprecates some IPFS flags by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3778
  • Make it easier to get in progress jobs of a certain type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3777
  • Fixes a panic in the wasm executor when entrypoint is not known by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3779
  • Structured events in place of comment strings by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3771
  • fix: remove Liveness and Acceptance from NodeInfo by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3785
  • Add issue templates by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3951
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3809
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3808
  • Bump idna from 3.3 to 3.7 in /ops by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3786
  • Bump idna from 3.6 to 3.7 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3788
  • Bump idna from 3.6 to 3.7 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3789
  • Bump sqlparse from 0.4.4 to 0.5.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3792
  • Bump gunicorn from 21.2.0 to 22.0.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3794
  • Bump aiohttp from 3.9.3 to 3.9.4 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3796
  • Bump apache-airflow from 2.8.1 to 2.9.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3798
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.23.1 to 1.26.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3955
  • Removing placeholders by @MichaelHoepler in https://github.com/bacalhau-project/bacalhau/pull/3795
  • adding ok-to-test functionality by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3960
  • 3444 adding all fake data by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3445
  • feat: premit update check configuration w/ envvar by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3967
  • fix: prevent panic in test due to nil pointer dereference by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3977
  • fix: require connected and approved nodes for scheduling by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3957
  • fix: allow timeouts in the client methods that make API call by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3964
  • refactor: add deprecation message to libp2p & ipfs flags by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3966
  • Support MP deployments with self-generated TLS certificate by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3721
  • fix: update checker doesn't require server connection to work by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3775
  • ExecutionTimeout to fail executions instead of jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3974
  • adding update checker test env variable by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3997
  • Fix Repo Clone by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3973
  • urldownload GetVolumeSize check for content length by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4006
  • add atomic job store transactions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3996
  • compute nodes no longer reject bids based on queued capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4002
  • fix txContext auto-cancellation of context by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4012
  • single requester node in demo networks by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3673
  • fixing circle ci commenting by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4022
  • run demo network on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4009
  • add evaluation watcher and enable atomic enqueues by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3998
  • make job name optional by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4005
  • fix: implement NodeStore migration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/4029

New Contributors

  • @udsamani made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4006

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.3.0...v1.3.1-rc2

- Go
Published by frrist about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.1-rc1

What's Changed

  • Update prod to v1.3.0. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3708
  • fix: update bacalhau install script to use go v1.21.8 by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3675
  • Add authentication to the Web UI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3711
  • fix: re-use client TLS config for establishing ws connections by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3722
  • Add Basic Bashtub tests by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3669
  • Replace use of multierror libraries with now standardised errors.Join by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3744
  • Load WASM module dependencies if non-WASM input data is specified by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3747
  • Adds a node delete command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3716
  • Add error output to job selection exec hooks by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3745
  • feat: create a bacalhau user to run bacalhau process by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3733
  • refactor: bidder to simplfiy exposing errors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3680
  • Improve error reporting when job resources exceed capacity by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3749
  • Print ranking failures in the CLI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3752
  • Return failed bid when image does not exist by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3755
  • Generate certificates requester three by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3707
  • Just a bunch of spelling fixes by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3760
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp from 0.45.0 to 1.24.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3615
  • Improve ranking error output by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3763
  • Node heartbeats by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3709
  • Ensure compute node labels are published over LibP2P connections by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3769
  • Use RunE instead of Run for cobra commands by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3767
  • Optionally constrain to connected, approved compute nodes when selecting/ranking them by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3768
  • Print command line executable when displaying additional commands by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3770
  • Deprecates some IPFS flags by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3778
  • Make it easier to get in progress jobs of a certain type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3777
  • Fixes a panic in the wasm executor when entrypoint is not known by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3779
  • Structured events in place of comment strings by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3771
  • fix: remove Liveness and Acceptance from NodeInfo by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3785
  • Add issue templates by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3951
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3809
  • Bump golang.org/x/net from 0.21.0 to 0.23.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3808
  • Bump idna from 3.3 to 3.7 in /ops by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3786
  • Bump idna from 3.6 to 3.7 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3788
  • Bump idna from 3.6 to 3.7 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3789
  • Bump sqlparse from 0.4.4 to 0.5.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3792
  • Bump gunicorn from 21.2.0 to 22.0.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3794
  • Bump aiohttp from 3.9.3 to 3.9.4 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3796
  • Bump apache-airflow from 2.8.1 to 2.9.0 in /integration/airflow by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3798
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.23.1 to 1.26.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3955
  • Removing placeholders by @MichaelHoepler in https://github.com/bacalhau-project/bacalhau/pull/3795
  • adding ok-to-test functionality by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3960
  • 3444 adding all fake data by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3445
  • feat: premit update check configuration w/ envvar by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3967
  • fix: prevent panic in test due to nil pointer dereference by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3977
  • fix: require connected and approved nodes for scheduling by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3957
  • fix: allow timeouts in the client methods that make API call by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3964
  • refactor: add deprecation message to libp2p & ipfs flags by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3966
  • Support MP deployments with self-generated TLS certificate by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3721
  • fix: update checker doesn't require server connection to work by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3775
  • ExecutionTimeout to fail executions instead of jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3974
  • adding update checker test env variable by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3997
  • Fix Repo Clone by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/3973
  • urldownload GetVolumeSize check for content length by @udsamani in https://github.com/bacalhau-project/bacalhau/pull/4006
  • add atomic job store transactions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3996
  • compute nodes no longer reject bids based on queued capacity by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4002
  • fix txContext auto-cancellation of context by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4012
  • single requester node in demo networks by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3673
  • fixing circle ci commenting by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/4022
  • run demo network on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4009
  • add evaluation watcher and enable atomic enqueues by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3998
  • make job name optional by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/4005

New Contributors

  • @udsamani made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4006

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.3.0...v1.3.1-rc1

- Go
Published by frrist about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.0

We are thrilled to announce the release of Bacalhau v1.3.0, a significant milestone in our quest for helping organizations of all sizes deal with the world of distributed compute. Packed with exciting new features like user access control, local results publishing, and TLS support, this release is built to address the needs of even the largest organizations without the complexity of traditional distributed platforms! 🚀

Without further ado, let’s dive in!

New Features

User access control

Bacalhau v1.3.0 now supports authentication and authorization of individual users with a flexible and customizable auth system that remains simple for single-node clusters but scales up well to wide enterprise deployments.

Bacalhau auth integrates well with whatever auth systems users already have. Bacalhau can use private keys, shared secrets, usernames, and passwords and 2FA. Additionally, Bacalhau offers OAuth2/OIDC for authentication and can apply access control to single users, groups, teams and can use RBAC or ABAC mechanisms as desired.

The default behaviour is unchanged. Users will be authenticated based on their private key and authorized to submit and control their own jobs and read-only information about the cluster will be available with authentication.

To start using user authentication, check out the auth docs and install a custom policy to control user access and their permissions.

Publishing and serving results on local disks

In Bacalhau v1.3.0 we are introducing a new publisher type that lets users publish to the local disk of the compute node. This will streamline the process of testing the publisher functionality without the need for a remote storage service. This is especially handy for those who are just getting started with Bacalhau.

The local publisher is composed of two parts: the publisher that compresses and moves job outputs to a specified location, and an HTTP server that delivers the content back to the user.

By default, the HTTP server listens on port 6001, but this can be modified using the --local-publisher-port flag. The server will deliver content from the directory specified by the local-publisher-directory flag, or, if not set, from a subdirectory of the configured Bacalhau storage directory. The --local-publisher-address flag can be used to set the address that the HTTP server listens on. Default values for this vary by environment (e.g., localhost for test and development environments, public for production environments), but users can set these values in the config if the defaults are not suitable.

We should stress that managing the storage is still the administrator’s responsibility. Because local storage necessarily means storing on a single node, thinking through clean up, persistence, etc. are things you should think through before moving into production!

NATS-based networking

In the Bacalhau v1.3.0 release, we are introducing a new transport layer to improve inter-node connectivity. This new layer utilizes NATS, a robust messaging system, instead of the existing libp2p transport.

With the introduction of NATS, we are simplifying the network requirements for Compute nodes. Now, only Orchestrator nodes (also known as Requester nodes) need to be publicly reachable. As a result, Compute nodes only need to know the address of a single Orchestrator node, and they can learn about and connect to other Orchestrators at runtime. This change not only simplifies the setup process but also enhances resilience as it allows Compute nodes to failover and reconnect to other Orchestrators when necessary. This change only affects inter-node communication, and the Bacalhau HTTP API is unchanged.

We acknowledge that adapting to new technologies takes time. In recognition of this, libp2p will continue to be supported as an alternative during this transition period. This ensures that you have the flexibility to migrate at your own pace. Users who wish to continue using libp2p need to specify the Node.Network.Type config option or --network flag as libp2p explicitly when running their network.

Persistent memory of connected nodes

The Bacalhau v1.3.0 release introduces a significant upgrade ensuring the persistence of node information across requester node restarts. This addresses a shortcoming of the previous in-memory store, which would lose all knowledge of compute nodes upon a restart. The new persistent store is a major advancement towards maintaining more accurate node information and tracking compute nodes that may be temporarily inaccessible to the cluster.

The new persistent store is used automatically when NATS-based networking is used.

TLS support for Bacalhau CLI

Bacalhau v1.3.0 now supports TLS requests to the requester node for all CLI commands. While the default communication remains HTTP, users can activate TLS calls using the command line flag --tls , setting the Node.ClientAPI.ClientTLS.UseTLS config option to true or by exporting the BACALHAU_API_TLS=1 environment variable.

For self-signed certificates, users can either accept insecure requests or provide a CA certificate file. The Node.ClientAPI.ClientTLS.CACert config option, BACALHAU_API_CACERT environment variable and --cacert flag can be used to verify the certificate with a provided CA certificate file. Alternatively, the Node.ClientAPI.ClientTLS.Insecure config option, --insecure flag or BACALHAU_API_INSECURE environment variable can be used to make API requests without certificate verification.

Customizable node names

In the Bacalhau v1.3.0 release, we've introduced a new feature that allows users to set their own nodeID. This addition gives users the flexibility to tailor their node names according to their preferences and needs.

Users have the option to manually set the node name, or they can opt for automatic generation using various providers. These providers include puuid (which is the default option), uuid, hostname, aws, and gcp.

The puuid option generates a node name using the n-{uuid} pattern, such as n-f1bab231-68ad-4c72-bab6-580cd49bf521. The uuid option generates a uuid as a node name. The hostname option uses the hostname as the node id, replacing any . with - to ensure compatibility with NATS. The aws option uses the EC2 instance name if the node is deployed on AWS, and the gcp option uses the VM's id if the node is deployed on GCP.

It's important to note that these providers will only be called into action if no existing node name is found in config.yaml, the CLI --name flag, or environment variables. Once a node name is generated, it will be persisted in config.yaml, ensuring that the node names are consistent across sessions.

To set the node name manually:

bash bacalhau serve --name my-custom-name

To use a puuid as the node name (which is the default):

bash bacalhau serve

To use the hostname as the node name:

bash bacalhau serve --name-provider hostname

This new feature is aimed at enhancing user customization and control, making Bacalhau even more user-friendly and adaptable to different user needs.

Improved telemetry and metrics

Bacalhau Telemetry Suite

In this update we have introduced a docker-compose based telemetry suite complete with open-telemetry, Prometheus, Grafana, and Jaeger containers for collecting and inspecting telemetry data emitted from bacalhau nodes. For details on running the suite see the respective README.md

Improved Visibility via New Metrics

In this update we have added new metrics to improve the observability of bacalhau nodes. These metrics include:

  • job_submitted: Number of jobs submitted to the Bacalhau node.
  • job_publish_duration_milliseconds: Duration of publishing a job on the compute node in milliseconds.
  • job_storage_upload_duration_milliseconds: Duration of uploading job storage input on the compute node in milliseconds.
  • job_storage_prepare_duration_milliseconds: Duration of preparing job storage input on the compute node in milliseconds.
  • job_storage_cleanup_duration_milliseconds: Duration of job storage input cleanup on the compute node in milliseconds.
  • job_duration_milliseconds: Duration of a job on the compute node in milliseconds.
  • docker_active_executions: Number of active docker executions on the compute node.
  • wasm_active_executions: Number of active WASM executions on the compute node.
  • bacalhau_node_info: A static metric with labels describing the bacalhau node.
    • node_id: ID of bacalhau node emitting metric
    • node_network_transport: bacalhau node network transport type (libp2p or NATs)
    • node_is_compute: true if the node is accepting compute jobs
    • node_is_requester: true if the node is serving as a requester node
    • node_engines: list of engines the node supports.
    • node_publishers: list of publishers the node supports.
    • node_storages: list of storages the node supports

Improved Out of Memory handling for Docker jobs

The Bacalhau CLI will now explain when Docker jobs run out of memory and include links to the Bacalhau documentation showing how to increase the memory limit for a job.

Improved configuration for IPFS

In this update, we have allowed for the embedded IPFS nodes gateway, API, and swarm listening multi-addresses to be configured, providing users with more control and determinism, particularly when configuring firewall rules.

This update also introduces changes when the --ipfs-serve-path flag is set, now preserving the content of the embedded IPFS nodes repo across Bacalhau restarts, maintaining any data the embedded IPFS node stored as well as its identity.

Furthermore, we've added a new flag, --ipfs-profile, to configure the embedded IPFS nodes configuration profile.

New experimental features

Custom job types

We're delighted to announce the introduction of a new exec command, designed to facilitate the submission of custom job types. This marks a significant milestone in our push for greater customization and flexibility. In this initial version, we are offering support for both duckdb and python job types.

This command allows users to run simpler higher-level commands that will be familiar from the command line but without worrying about how Bacalhau safely orchestrates and executes their code.

bash bacalhau exec python -p local -- -c 'print("hello world!")'

To accommodate job types that require code to run, we have introduced a --code flag. This allows users to specify either a single file or a folder containing multiple files. If the total size of the files does not exceed 10Mb, the contents will be added as an inline storage to the job specification.

Up next

For the next release, Bacalhau's development will focus on delivering a superior user experience. We'll improve job visibility, error reporting, and progress updates. Look for a fully managed orchestrator with multi-tenancy support and a web UI, easier network bootstrapping, and availability in major cloud marketplaces. We're also committed to unwavering reliability, including better job orchestration, failed job detection, compute node failover, and a seamless "fire and forget" experience.

Have something else in mind? Let us know!

- Go
Published by rossjones about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.0-rc3

What's Changed

  • Clarifies error message when bacalhau encounters a newer repo version by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3638
  • Approve/Reject node membership from CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3594
  • Fix low cleanup frequency in cache tests by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3636
  • Add more auth documentation. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3642
  • release v1.2.3 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3650
  • Default local publisher to binding to all addresses by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3655
  • Configure IP for local publisher address by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3659
  • Upgrades the codebase to Go 1.21 by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3640
  • Enable dark mode system-pref and switcher for docs by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3661
  • Remove custom yaml tag on TLS ServerKey by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3639
  • Local publisher address tf by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3660
  • Publisher processing for exec command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3656
  • Default for NetworkType is NATS by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3628
  • Use local node in Bash tests when starting hybrid node by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3667
  • stop auto-generating auth tokens by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3670
  • run staging on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3671
  • Allow nodes to call register on startup, even if previously registered by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3672

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.3...v1.3.0-rc3

- Go
Published by rossjones about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.0-rc2

What's Changed

  • Clarifies error message when bacalhau encounters a newer repo version by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3638
  • Approve/Reject node membership from CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3594
  • Fix low cleanup frequency in cache tests by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3636
  • Add more auth documentation. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3642
  • release v1.2.3 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3650
  • Default local publisher to binding to all addresses by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3655
  • Configure IP for local publisher address by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3659
  • Upgrades the codebase to Go 1.21 by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3640
  • Enable dark mode system-pref and switcher for docs by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3661
  • Remove custom yaml tag on TLS ServerKey by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3639
  • Local publisher address tf by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3660
  • Publisher processing for exec command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3656
  • Default for NetworkType is NATS by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3628
  • Use local node in Bash tests when starting hybrid node by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3667
  • stop auto-generating auth tokens by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3670

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.3...v1.3.0-rc2

- Go
Published by rossjones about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.3.0-rc1

What's Changed

  • Clarifies error message when bacalhau encounters a newer repo version by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3638
  • Approve/Reject node membership from CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3594
  • Fix low cleanup frequency in cache tests by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3636
  • Add more auth documentation. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3642
  • Default local publisher to binding to all addresses by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3655
  • Configure IP for local publisher address by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3659
  • Upgrades the codebase to Go 1.21 by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3640
  • Enable dark mode system-pref and switcher for docs by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3661
  • Remove custom yaml tag on TLS ServerKey by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3639
  • Local publisher address tf by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3660
  • Publisher processing for exec command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3656
  • Default for NetworkType is NATS by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3628
  • Use local node in Bash tests when starting hybrid node by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3667

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.3...v1.3.0-rc1

- Go
Published by rossjones about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.3

What's Changed

  • Bump github.com/invopop/jsonschema from 0.11.0 to 0.12.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3350
  • Improve testing for tls by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3345
  • Moving spelling file to its own directory (for vscode integration) and skipping draft PRs by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3280
  • Project level changes for improving webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3356
  • updating circle config by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3366
  • Configuration changes for webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3357
  • Fix webui linting by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3363
  • 3358 - Adding MSW and mocks into webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3359
  • decouple logsstream from transport layer by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3354
  • Docupdating a1qa into main by @MichaelHoepler in https://github.com/bacalhau-project/bacalhau/pull/3227
  • 3317 - Moving to YARN and including the reactSVG code by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3372
  • Adding example ask authn policy that uses a shared secret by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3364
  • Add initial docs for exec command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3173
  • Include supported job type options in help text by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3391
  • 3367 - more spelling fixes by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3373
  • 3368 - adding svgs in static dir and tests by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3374
  • 3371 - Adding storybook stories by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3387
  • feat: add metrics to publisher and input storage by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3360
  • Fix some issues in makefile. by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3399
  • fix: auth docs and config set for Auth by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3349
  • External CLI tests and subsequent authn/z fixes by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3406
  • fix python ci build by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3414
  • NATS based LogStream by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3407
  • Improve pagination and filtering of job list command. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3344
  • Make publisher optional in CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3417
  • Bump cryptography from 41.0.6 to 42.0.0 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3413
  • Select network address based on type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3394
  • Remove inmemory jobstore by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3422
  • Removes the compute node's inmemory store by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3424
  • First Storyboard web ui tools for JobTable by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3404
  • Support anonymous mode authorization for V1 APIs by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3426
  • Add vite env to tsconfig by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3435
  • Install webui dependencies when missing by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3433
  • Fix to my (accidental) reversion of this code by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3437
  • Require NATS clients to be authenticated with a server secret token by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3425
  • Fix too-speedy shutdown in devstack by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3434
  • Adds support for specifying a default publisher on the server by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3430
  • Updates the out of date mocks by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3439
  • 3427 move from webpack to vite by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3442
  • fix nats logstream by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3449
  • Generate TLS server certificates, keys and CA by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3443
  • feat: GA trigger deployment to development cluster by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3440
  • Fix Python building in project (was: Bump github.com/aws/aws-sdk-go-v2 from 1.21.1 to 1.24.1) by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3405
  • Verify signatures of JWTs when authorizing by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3452
  • no job spec with executions by default by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3453
  • Improve job describe output by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3459
  • s3 only publish compressed results by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3464
  • Adds a 'local' publisher by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3431
  • Replace freeport dependency with modified version by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3470
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.22.0 to 1.23.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3455
  • run staging on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3463
  • Enable access to local publisher content on demo network by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3486
  • Devstack cancellations by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3487
  • persist store paths, and migrate repo to v3 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3429
  • nodeID generation not based on libp2p by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3450
  • Updates the experimental slices package by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3502
  • Bump github.com/multiformats/go-multiaddr from 0.9.0 to 0.12.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3473
  • fix: set EvalID in execution by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3500
  • Bump ip from 2.0.0 to 2.0.1 in /webui by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3501
  • Remove invalid or expired authorization tokens by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3506
  • Update AWS SDK packages by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3511
  • Bump cryptography from 42.0.2 to 42.0.4 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3504
  • 3454 use earthly for building webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3460
  • 3461 push the image after building it ci by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3482
  • [cleanup] staticcheck cleanup by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3397
  • fix build from source by installing earthly by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3541
  • fix v3 repo migration when store config is present, but empty by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3542
  • Make it easier to build just the binary by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3536
  • Persistent node store by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3503
  • Adds support for filtering of the List command in NodeInfoStores by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3545
  • fix expected repoclone, but received: repoCloneLFS error by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3532
  • Hide NATS auth token if the user supplied it by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3539
  • Adds a node approval state to the node info by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3515
  • Move job utils to temporary legacy package by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3457
  • Fix CI and enable releases to happen by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3548
  • Bump github.com/nats-io/nats-server/v2 from 2.10.7 to 2.10.11 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3507
  • Excludes local address from TF cluster settings by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3551
  • Release v1.2.2 by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3561
  • Ensure compute node does not open jobstore by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3565
  • 3538 make build is broken by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3568
  • Address type resolution in publisher not server by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3555
  • Remove local addresses from cluster settings by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3553
  • Node registration when network=nats by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3535
  • Regularly updates node info when using NATS by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3573
  • Fix python flyte tests by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/3563
  • 3462 build python using earthly by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3483
  • Live compute-node resource tracking by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3556
  • Cancel authn input if the user presses Ctrl+C by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3579
  • More documentation for WASM loader. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3584
  • Fix error on reading an empty tokens file by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3589
  • Fixes a race condition starting server for local publisher by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3587
  • Two fixes for more upfront validation by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3593
  • Show node approval status in CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3585
  • Bump github.com/lestrrat-go/jwx from 1.2.28 to 1.2.29 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3582
  • Upgrade docker and fix deprecations by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3597
  • deps: remove job info consumer by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3400
  • Manually cleanup repo directory to work around flaky window test. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3602
  • 3491 move plugins to earthly by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3495
  • Implement new and improved terraform-base deployment by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3089
  • fix: add token_valid check to authz mp-tf by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3611
  • Store registration sentinel file alongside executions db by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3596
  • Disables the swagger doc generation by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3625
  • fix nodeIDs when using libp2p by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3624
  • Updates timeout for TestNack_TimeoutReset by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3618
  • Revert "Store registration sentinel file alongside executions db" by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3629
  • Fix job logs following to work with auth by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3613
  • Fix documentation not being built or deployed. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3634
  • Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3631
  • Simplify labels for compute node by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3637

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.1...v1.2.3

- Go
Published by wdbaruni about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.3-rc2

What's Changed

  • Release v1.2.2 by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3561
  • Ensure compute node does not open jobstore by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3565
  • 3538 make build is broken by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3568
  • Address type resolution in publisher not server by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3555
  • Remove local addresses from cluster settings by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3553
  • Node registration when network=nats by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3535
  • Regularly updates node info when using NATS by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3573
  • Fix python flyte tests by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/3563
  • 3462 build python using earthly by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3483
  • Live compute-node resource tracking by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3556
  • Cancel authn input if the user presses Ctrl+C by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3579
  • More documentation for WASM loader. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3584
  • Fix error on reading an empty tokens file by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3589
  • Fixes a race condition starting server for local publisher by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3587
  • Two fixes for more upfront validation by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3593
  • Show node approval status in CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3585
  • Bump github.com/lestrrat-go/jwx from 1.2.28 to 1.2.29 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3582
  • Upgrade docker and fix deprecations by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3597
  • deps: remove job info consumer by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3400
  • Manually cleanup repo directory to work around flaky window test. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3602
  • 3491 move plugins to earthly by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3495
  • Implement new and improved terraform-base deployment by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3089
  • fix: add token_valid check to authz mp-tf by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3611
  • Store registration sentinel file alongside executions db by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3596
  • Disables the swagger doc generation by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3625
  • fix nodeIDs when using libp2p by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3624
  • Updates timeout for TestNack_TimeoutReset by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3618
  • Revert "Store registration sentinel file alongside executions db" by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3629
  • Fix job logs following to work with auth by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3613
  • Fix documentation not being built or deployed. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3634
  • Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3631
  • Simplify labels for compute node by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3637

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.2...v1.2.3-rc2

- Go
Published by wdbaruni about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.3-rc1

What's Changed

  • Fix CI and enable releases to happen by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3548
  • Bump github.com/nats-io/nats-server/v2 from 2.10.7 to 2.10.11 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3507
  • Excludes local address from TF cluster settings by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3551
  • Release v1.2.2 by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3561
  • Ensure compute node does not open jobstore by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3565
  • 3538 make build is broken by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3568
  • Address type resolution in publisher not server by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3555
  • Remove local addresses from cluster settings by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3553
  • Node registration when network=nats by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3535
  • Regularly updates node info when using NATS by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3573
  • Fix python flyte tests by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/3563
  • 3462 build python using earthly by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3483
  • Live compute-node resource tracking by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3556
  • Cancel authn input if the user presses Ctrl+C by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3579
  • More documentation for WASM loader. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3584
  • Fix error on reading an empty tokens file by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3589
  • Fixes a race condition starting server for local publisher by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3587
  • Two fixes for more upfront validation by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3593
  • Show node approval status in CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3585
  • Bump github.com/lestrrat-go/jwx from 1.2.28 to 1.2.29 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3582
  • Upgrade docker and fix deprecations by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3597
  • deps: remove job info consumer by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3400
  • Manually cleanup repo directory to work around flaky window test. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3602
  • 3491 move plugins to earthly by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3495
  • Implement new and improved terraform-base deployment by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3089
  • fix: add token_valid check to authz mp-tf by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3611
  • Store registration sentinel file alongside executions db by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3596
  • Disables the swagger doc generation by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3625
  • fix nodeIDs when using libp2p by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3624
  • Updates timeout for TestNack_TimeoutReset by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3618
  • Revert "Store registration sentinel file alongside executions db" by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3629
  • Fix job logs following to work with auth by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3613
  • Fix documentation not being built or deployed. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3634

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.2-rc2...v1.2.3-rc1

- Go
Published by wdbaruni about 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.2-rc3

What's Changed

  • Improve testing for tls by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3345
  • Moving spelling file to its own directory (for vscode integration) and skipping draft PRs by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3280
  • Project level changes for improving webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3356
  • updating circle config by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3366
  • Configuration changes for webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3357
  • Fix webui linting by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3363
  • 3358 - Adding MSW and mocks into webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3359
  • decouple logsstream from transport layer by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3354
  • Docupdating a1qa into main by @MichaelHoepler in https://github.com/bacalhau-project/bacalhau/pull/3227
  • 3317 - Moving to YARN and including the reactSVG code by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3372
  • Adding example ask authn policy that uses a shared secret by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3364
  • Add initial docs for exec command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3173
  • Include supported job type options in help text by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3391
  • 3367 - more spelling fixes by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3373
  • 3368 - adding svgs in static dir and tests by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3374
  • 3371 - Adding storybook stories by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3387
  • feat: add metrics to publisher and input storage by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3360
  • Fix some issues in makefile. by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3399
  • fix: auth docs and config set for Auth by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3349
  • External CLI tests and subsequent authn/z fixes by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3406
  • fix python ci build by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3414
  • NATS based LogStream by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3407
  • Improve pagination and filtering of job list command. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3344
  • Make publisher optional in CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3417
  • Select network address based on type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3394
  • Remove inmemory jobstore by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3422
  • Removes the compute node's inmemory store by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3424
  • First Storyboard web ui tools for JobTable by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3404
  • Support anonymous mode authorization for V1 APIs by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3426
  • Add vite env to tsconfig by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3435
  • Install webui dependencies when missing by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3433
  • Fix to my (accidental) reversion of this code by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3437
  • Require NATS clients to be authenticated with a server secret token by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3425
  • Fix too-speedy shutdown in devstack by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3434
  • Adds support for specifying a default publisher on the server by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3430
  • Updates the out of date mocks by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3439
  • 3427 move from webpack to vite by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3442
  • fix nats logstream by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3449
  • Generate TLS server certificates, keys and CA by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3443
  • feat: GA trigger deployment to development cluster by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3440
  • Fix Python building in project (was: Bump github.com/aws/aws-sdk-go-v2 from 1.21.1 to 1.24.1) by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3405
  • Verify signatures of JWTs when authorizing by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3452
  • no job spec with executions by default by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3453
  • Improve job describe output by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3459
  • s3 only publish compressed results by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3464
  • Adds a 'local' publisher by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3431
  • Replace freeport dependency with modified version by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3470
  • run staging on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3463
  • Enable access to local publisher content on demo network by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3486
  • Devstack cancellations by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3487
  • persist store paths, and migrate repo to v3 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3429
  • nodeID generation not based on libp2p by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3450
  • Updates the experimental slices package by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3502
  • fix: set EvalID in execution by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3500
  • Remove invalid or expired authorization tokens by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3506
  • Update AWS SDK packages by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3511
  • Bump cryptography from 42.0.2 to 42.0.4 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3504
  • 3454 use earthly for building webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3460
  • 3461 push the image after building it ci by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3482
  • [cleanup] staticcheck cleanup by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3397
  • fix build from source by installing earthly by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3541
  • fix v3 repo migration when store config is present, but empty by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3542
  • Make it easier to build just the binary by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3536
  • Persistent node store by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3503
  • Adds support for filtering of the List command in NodeInfoStores by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3545
  • fix expected repoclone, but received: repoCloneLFS error by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3532
  • Hide NATS auth token if the user supplied it by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3539
  • Adds a node approval state to the node info by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3515
  • Move job utils to temporary legacy package by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3457
  • Fix CI and enable releases to happen by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3548

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.1...v1.2.2-rc3

- Go
Published by simonwo over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.2-rc2

What's Changed

  • Improve testing for tls by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3345
  • Moving spelling file to its own directory (for vscode integration) and skipping draft PRs by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3280
  • Project level changes for improving webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3356
  • updating circle config by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3366
  • Configuration changes for webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3357
  • Fix webui linting by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3363
  • 3358 - Adding MSW and mocks into webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3359
  • decouple logsstream from transport layer by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3354
  • Docupdating a1qa into main by @MichaelHoepler in https://github.com/bacalhau-project/bacalhau/pull/3227
  • 3317 - Moving to YARN and including the reactSVG code by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3372
  • Adding example ask authn policy that uses a shared secret by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3364
  • Add initial docs for exec command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3173
  • Include supported job type options in help text by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3391
  • 3367 - more spelling fixes by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3373
  • 3368 - adding svgs in static dir and tests by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3374
  • 3371 - Adding storybook stories by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3387
  • feat: add metrics to publisher and input storage by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3360
  • Fix some issues in makefile. by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3399
  • fix: auth docs and config set for Auth by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3349
  • External CLI tests and subsequent authn/z fixes by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3406
  • fix python ci build by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3414
  • NATS based LogStream by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3407
  • Improve pagination and filtering of job list command. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3344
  • Make publisher optional in CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3417
  • Select network address based on type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3394
  • Remove inmemory jobstore by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3422
  • Removes the compute node's inmemory store by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3424
  • First Storyboard web ui tools for JobTable by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3404
  • Support anonymous mode authorization for V1 APIs by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3426
  • Add vite env to tsconfig by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3435
  • Install webui dependencies when missing by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3433
  • Fix to my (accidental) reversion of this code by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3437
  • Require NATS clients to be authenticated with a server secret token by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3425
  • Fix too-speedy shutdown in devstack by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3434
  • Adds support for specifying a default publisher on the server by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3430
  • Updates the out of date mocks by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3439
  • 3427 move from webpack to vite by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3442
  • fix nats logstream by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3449
  • Generate TLS server certificates, keys and CA by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3443
  • feat: GA trigger deployment to development cluster by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3440
  • Fix Python building in project (was: Bump github.com/aws/aws-sdk-go-v2 from 1.21.1 to 1.24.1) by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3405
  • Verify signatures of JWTs when authorizing by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3452
  • no job spec with executions by default by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3453
  • Improve job describe output by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3459
  • s3 only publish compressed results by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3464
  • Adds a 'local' publisher by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3431
  • Replace freeport dependency with modified version by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3470
  • run staging on nats by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3463
  • Enable access to local publisher content on demo network by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3486
  • Devstack cancellations by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3487
  • persist store paths, and migrate repo to v3 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3429
  • nodeID generation not based on libp2p by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3450
  • Updates the experimental slices package by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3502
  • fix: set EvalID in execution by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3500
  • Remove invalid or expired authorization tokens by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3506
  • Update AWS SDK packages by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3511
  • 3454 use earthly for building webui by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3460
  • 3461 push the image after building it ci by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3482
  • [cleanup] staticcheck cleanup by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3397
  • fix build from source by installing earthly by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3541
  • fix v3 repo migration when store config is present, but empty by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3542
  • Make it easier to build just the binary by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3536
  • Persistent node store by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3503
  • Adds support for filtering of the List command in NodeInfoStores by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3545
  • fix expected repoclone, but received: repoCloneLFS error by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3532
  • Hide NATS auth token if the user supplied it by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3539
  • Adds a node approval state to the node info by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3515
  • Move job utils to temporary legacy package by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3457

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.1...v1.2.2-rc2

- Go
Published by simonwo over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.2-rc1

What's Changed

  • WebUI Enhancements: A series of improvements have been made to the WebUI, including better linting, the addition of Mock Service Worker (MSW) for better frontend testing, storybook integration for UI component testing, and a move from webpack to Vite for a faster development experience. These changes aim to improve the development workflow and user interaction with Bacalhau's WebUI.
  • feat: add metrics to publisher and input storage by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3360
  • External CLI tests and subsequent authn/z fixes by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3406
  • NATS based LogStream by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3407
  • Improve pagination and filtering of job list command. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3344
  • Make publisher optional in CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3417
  • Select network address based on type by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3394
  • Remove inmemory jobstore by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3422
  • Removes the compute node's inmemory store by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3424
  • Support anonymous mode authorization for V1 APIs by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3426
  • Require NATS clients to be authenticated with a server secret token by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3425
  • Adds support for specifying a default publisher on the server by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3430
  • Generate TLS server certificates, keys and CA by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3443
  • Verify signatures of JWTs when authorizing by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3452
  • no job spec with executions by default by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3453
  • Improve job describe output by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3459

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.1...v1.2.2-rc1

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.1

What's Changed

  • Remove building Bacalhau as part of Docker image build by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3092
  • Docs migration by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3070
  • Custom job type docker images by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3093
  • Fix error message when missing plugin path by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3109
  • fix: require teststack connected before test by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3111
  • fix: helpful error when IPFS fails to get content by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3101
  • refactor: allow node info store ttl configuration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3102
  • Fix GPU labels for nodes with multiple GPU vendors by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3116
  • Speed up CI a little bit by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3119
  • Don't auto-generate Go-based docs using Swagger anymore by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3115
  • Fix non-working test to save 30s by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3124
  • Speed up the failing ipfs downloader test by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3123
  • feat: further configuration of embedded ipfs daemon by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3096
  • Webui fix job name and by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3098
  • Run code generation by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3133
  • Appeasing linter by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3122
  • Provide flag to set webui port by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3132
  • Translation converting custom task types to docker tasks by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3108
  • Build and release on tags by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3136
  • Only release Bacalhau artifacts from the workspace. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3140
  • Shunt GPU tooling error messages to DEBUG by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3153
  • Enable experimental job translation by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3139
  • Make logs wait for execution to be running by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3138
  • Show error message when nodes don't support Docker by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3155
  • adding error information to boltdb not running by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3156
  • 3151 fix tests when s3 credentials are valid but dont have access to the test bucket by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3154
  • Fixing many spelling errors and adding codespell check to CI by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3161
  • Support extra flags for exec command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3162
  • 200 for api root by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3158
  • Update the duckdb executor image to put binary in path by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3172
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3175
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3176
  • Add authorization middleware by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3170
  • Create Bacalhau repo with correct permission from Python SDK by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3186
  • fix: filter out requester nodes in rankAndFilterNodes by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3216
  • feat: allow clients to submit docker/wasm jobs with mismatched versions by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3207
  • adding testing bacalhau running script by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3157
  • Enable TLS support for CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2854
  • Update docker executor for oom by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3230
  • Add authentication API endpoints and default using private keys by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3220
  • feat: implement server/client version notify middleware by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3221
  • Swagger PRs - 1 - MVP by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3224
  • Store source job spec in metadata when translating by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3257
  • Refactor how we configure TLS in devstack by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3259
  • Removes mypy from airflow plugin by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3273
  • cleanup: Rename NewIPFSPublishers by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3263
  • Remove estuary by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3270
  • Some easy cleanup with spell checking by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3276
  • Add authentication to the CLI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3261
  • Some small spelling fixes and adding a configuration file for prettier by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/3277
  • introduce nats transport by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3188
  • docs: Fix dev links in main README.md by @adamac in https://github.com/bacalhau-project/bacalhau/pull/3291
  • Make ExecuteTestCobraCommand safer to use by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3294
  • Allow authn and authz methods to be configured by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3299
  • Improve config of docker manifest caching by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2516
  • Minor doc update by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3307
  • Move linters to github actions by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3289
  • feat: implement local Telemetry suite by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3302
  • Add authentication that supports usernames and passwords by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3311
  • instument basic metrics by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3309
  • Update the deprecated linux image for CI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3272
  • refactor v2 client to be more idiomatic by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3318
  • Ensures requests as json do not include headers by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3341
  • Stops linters from 'fixing' code by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3346

New Contributors

  • @adamac made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/3291

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.0...v1.2.1

- Go
Published by simonwo over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.1-rc2

What's Changed

  • Bump github.com/jedib0t/go-pretty/v6 from 6.4.7 to 6.4.9 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3091
  • Remove building Bacalhau as part of Docker image build by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3092
  • Docs migration by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3070
  • Bump github.com/go-git/go-git/v5 from 5.8.1 to 5.11.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3097
  • Bump github.com/hashicorp/go-hclog from 1.5.0 to 1.6.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3105
  • Custom job type docker images by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3093
  • Fix error message when missing plugin path by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3109
  • fix: require teststack connected before test by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3111
  • fix: helpful error when IPFS fails to get content by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3101
  • refactor: allow node info store ttl configuration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3102
  • Fix GPU labels for nodes with multiple GPU vendors by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3116
  • Speed up CI a little bit by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3119
  • Don't auto-generate Go-based docs using Swagger anymore by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3115
  • Bump github.com/labstack/echo/v4 from 4.11.1 to 4.11.3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3117
  • Fix non-working test to save 30s by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3124
  • Speed up the failing ipfs downloader test by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3123
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.18.0 to 1.21.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3126
  • feat: further configuration of embedded ipfs daemon by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3096
  • Bump golang.org/x/crypto from 0.16.0 to 0.17.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3129
  • Webui fix job name and by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3098
  • Run code generation by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3133
  • Bump golang.org/x/time from 0.3.0 to 0.5.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3134
  • Appeasing linter by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3122
  • Provide flag to set webui port by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3132
  • Translation converting custom task types to docker tasks by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3108
  • Build and release on tags by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3136

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.0...v1.2.1-rc2

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.1-rc1

What's Changed

  • Bump github.com/jedib0t/go-pretty/v6 from 6.4.7 to 6.4.9 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3091
  • Remove building Bacalhau as part of Docker image build by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3092
  • Docs migration by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3070
  • Bump github.com/go-git/go-git/v5 from 5.8.1 to 5.11.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3097
  • Bump github.com/hashicorp/go-hclog from 1.5.0 to 1.6.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3105
  • Custom job type docker images by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3093
  • Fix error message when missing plugin path by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3109
  • fix: require teststack connected before test by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3111
  • fix: helpful error when IPFS fails to get content by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3101
  • refactor: allow node info store ttl configuration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3102
  • Fix GPU labels for nodes with multiple GPU vendors by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3116
  • Speed up CI a little bit by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3119
  • Don't auto-generate Go-based docs using Swagger anymore by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3115
  • Bump github.com/labstack/echo/v4 from 4.11.1 to 4.11.3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3117
  • Fix non-working test to save 30s by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3124
  • Speed up the failing ipfs downloader test by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3123
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace from 1.18.0 to 1.21.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3126
  • feat: further configuration of embedded ipfs daemon by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3096
  • Bump golang.org/x/crypto from 0.16.0 to 0.17.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3129
  • Webui fix job name and by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3098
  • Run code generation by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3133
  • Bump golang.org/x/time from 0.3.0 to 0.5.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3134
  • Appeasing linter by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3122
  • Provide flag to set webui port by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3132
  • Translation converting custom task types to docker tasks by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3108

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.2.0...v1.2.1-rc1

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.2.0

Announcing Bacalhau v1.2

📢 We are extremely proud to announce the release of Bacalhau 1.2! Since we announced the release of Bacalhau 1.1 in September 2023, we have been the community has stepped forward to release a number of novel ways to use Bacalhau. For example:

We are also excited to announce that the U.S. Navy Chose Bacalhau to Manage Predictive Maintenance Workloads!

But that’s just the start - Bacalhau 1.2 is packed with new features, enhancements, and bug fixes to improve your workflow and overall user experience.

Read on to learn more about these exciting updates! Or install straight away.

Job templates

Users can now create and customize job templates to streamline their workflow by creating a large number of similar jobs more easily.

A job template contains named placeholders:

yaml Name: docker job Type: batch Count: 1 Tasks: - Name: main Engine: Type: docker Params: Image: ubuntu:latest Entrypoint: - /bin/bash Parameters: - -c - echo {{.greeting}} {{.name}}

The placeholders will be filled in during a call to bacalhau job run with environment variables or command-line flags:

bash export greeting=Hello bacalhau job run job.yaml --template-vars "name=World" --template-envs "*"

The templating functionality is based on the Go text/template package. This robust library offers a wide range of features for manipulating and formatting text based on template definitions and input variables. For more information about the Go text/template library and its syntax, you can refer to the official documentation: Go text/template Package.

Telemetry from inside WASM jobs with Dylibso integration

This release introduces the ability to collect telemetry data from within WebAssembly (WASM) jobs via integration with the Dylibso Observe SDK.

Now, WebAssembly modules that have been automatically or manually instrumented will pass tracing information to any OTEL endpoints configured for Bacalhau to use. This happens automatically if OTEL is configured and the WASM is instrumented. Uninstrumented WASM continues to be run as normal with no required changes.

This allows WASM jobs to pass telemetry into Jaeger or any other configured OTEL client:

279524347-2e2b2226-4e24-49b6-b838-046d7ed63c98

Simplified node bootstrapping

Bacalhau nodes now no longer require peers to be supplied with a LibP2P key, which simplifies configuration of private Bacalhau clusters. Now, Bacalhau only requires an IP address and port number to establish peering with other nodes.

Whereas previously, Bacalhau required a public key to be specified with peer information:

bash /ip4/35.245.161.250/tcp/1235/p2p/QmbxGSsM6saCTyKkiWSxhJCt6Fgj7M9cns1vzYtfDbB5Ws

A simpler version with automatic key lookup is now supported:

bash /ip4/35.245.161.250/tcp/1235/http

This means in private environments, node set-up can be considerably simplified, as only the IP address and port need to be known in advance of establishing the cluster.

New browser-based dashboard

We have introduced a modern and intuitive browser-based dashboard, providing web users with a fresh experience. Users can now see both job execution lists and node information:

Screen_Shot_2023-12-04_at_17 26 12

Screen_Shot_2023-12-04_at_17 27 36

For the demonstration network, you can now visit http://bootstrap.production.bacalhau.org to see the dashboard in action.

It is simple to add the web UI to your own private cluster – just pass --web-ui to your bacalhau serve command or set Node.WebUI: yes in your config file. Once set Bacalhau will serve the web UI on port 80 automatically with no further configuration required.

Support for AMD and Intel GPUs

Users can now take advantage of the power of AMD and Intel GPUs for their computational tasks, in addition to our existing support for Nvidia GPUs.

Now, if a compute node has the AMD utlity rocm-smi or Intel utility xpu-smi installed, Bacalhau will automatically detect their GPUs at boot and make them available for use in Docker jobs.

Further, details on available GPUs are now available from the Nodes API. Calls to /api/v1/orchestrator/nodes now return extended GPU information that identifies the GPU’s vendor, available VRAM, and model name.

Support for multiple GPUs in Docker jobs

We now support the utilization of multiple GPUs in Docker jobs, enabling workload splitting and parallel processing.

Users can take advantage of multiple GPUs by passing the --gpu=... flag to bacalhau docker run or by specifying the number of GPUs in the Resources section of the job spec. The job will wait for the requested number of GPUs to become available at which point the job will have exclusive use of the GPUs for as long as it is executing.

Results download from S3-compatible buckets

Users can now download results directly from S3-compatible buckets using bacalhau get, simplifying the data retrieval process.

The S3 Publisher will now generate pre-signed URLs on-demand for use by Bacalhau clients downloading job results without needing to provide credentials to the bucket itself. This requires the requester node to have appropriate IAM permissions for reading buckets.

Support for Google Cloud buckets

Users can now seamlessly integrate Google Cloud Storage (GCS) buckets into their jobs, allowing for easy storage and retrieval of data.

To use GCS buckets, request data from a GCS storage endpoint or configure an S3 publisher using a [storage.googleapis.com](http://storage.googleapis.com) endpoint.

Programmatic API for manipulating config

Developers can now programmatically manipulate configuration settings using the new bacalhau config command, providing more flexibility and automation in managing Bacalhau nodes.

  • Use bacalhau config list to show the current state of configuration variables including defaults.
  • Use bacalhau config set <key> <value> to permanently set a configuration option.
  • Use bacalhau config default to generate a Bacalhau configuration file in YAML format without any user-specific overrides in place.
  • Use bacalhau config auto-resources to detect available system capacity (CPU, RAM, Disk, GPUs, etc.) and write the capacity into the config file. Users can specify a number from 0-100 to limit the capacity to that percentage of total capacity. A number over 100 can also be used to overcommit on capacity, which is useful mainly for setting queue capacities.

Improvements and Bug Fixes

  • Sanity checking of WebAssembly resource usage: we now check that WebAssembly jobs have requested less than the maximum possible RAM size of 4GB.
  • Helpful messages when jobs fail to find nodes: improved error messaging provides users with the reason each node rejected or was not suitable for the job.
  • Improvements to long-running jobs: long-running jobs now are not subject to execution timeouts and won’t be waited for when submitted on the CLI.
  • Automatic notification of new software versions: wsers will now receive automatic notifications when new software versions are available, keeping them up to date with the latest features and improvements.

What’s Coming Next?

We have lots of new features coming in the next quarter. A selection of these items includes

  • Easier bootstrapping - More easily establish a full network of Bacalhau nodes running across multiple regions and clouds.
  • Native Python executor - A pluggable executor that can run raw Python against a Bacalhau network.
  • Network queues - Jobs are queued (up to indefinitely) at the network level and scheduled only when resources become available. Fully supports all current scheduling constructs including all node metadata.
  • Managed services - In a single action via our API, users can deploy a cluster to GCP, AWS, or Azure according to their zone/region criteria (new nodes, not existing ones).
  • Rich, periodic node metadata - Nodes can provide custom metadata (maybe as labels, or maybe as something more structured) that can be configured to refresh to some custom schedule or as the result of a job.

Or better yet, you tell us!

5 Days of Bacalhau 1.2 Blog Series

If you’re interested in exploring these features more in depth, check back tomorrow for our 5 Days of Bacalhau.

  • Day 1 - Job Templates
  • Day 2 - Streamlined Node Bootstrap
  • Day 3 - Multi-GPU Jobs with AMD and Intel GPU Support
  • Day 4 - Seamless S3 Downloads with Bacalhau
  • Day 5 - Instrumenting WebAssembly: Enhanced Telemetry with Dylibso Observe SDK

How to Get Involved

We're looking for help in several areas. If you're interested in helping, there are several ways to contribute. Please reach out to us at any of the following locations.

As always, thank you for reading, and onward!

Your humble Bacalhau team.

- Go
Published by simonwo over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.7-rc1

What's Changed

  • Layout of plugins directory by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3026
  • Fix exec probe rejection message. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3035
  • Small Repo Cleanup by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3037
  • support GCP buckets by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3040
  • WebUI with ability to spin up using the serve command by @XandraMcC in https://github.com/bacalhau-project/bacalhau/pull/3044
  • Ensure commands using the Client APIs are cancellable by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3048
  • Run Web UI in deployed Bacalhau clusters. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3047
  • Temporarily hide "View" button until Job Detail and Node Detail pages are available by @XandraMcC in https://github.com/bacalhau-project/bacalhau/pull/3051
  • Web UI: design updates and tweaks by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3052
  • CSS more accurately represents design document by @XandraMcC in https://github.com/bacalhau-project/bacalhau/pull/3056
  • Remove PNGs and replace with SVGs. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3054
  • feat: add a command to generate a default config and list keys by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3012
  • Integrate observe-sdk to get tracing information during Wasm execution by @zshipko in https://github.com/bacalhau-project/bacalhau/pull/2971
  • Support multiple AMD and Nvidia GPUs for Docker jobs by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3061
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/3064
  • More gracefully handle bad tool output by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3065
  • Webui fix job labels to display properly by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/3067
  • Tell StorageProviders where to store data by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3053
  • fix: correctly parse resource config by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3072
  • Bulk update OpenTelemetry dependencies. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3069
  • Webui job detail by @XandraMcC in https://github.com/bacalhau-project/bacalhau/pull/3066
  • Copy large inline sources to IPFS by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3073

New Contributors

  • @zshipko made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2971

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.6...v1.1.7-rc1

- Go
Published by frrist over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.6-rc1

What's Changed

  • Allow IPFS to run in devstack when ./plugins exists by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3024
  • Add Templating Support to bacalhau job run by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3020
  • s3 storage pattern filtering by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3031

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.5...v1.1.6-rc1

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.6

What's Changed

  • Allow IPFS to run in devstack when ./plugins exists by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3024
  • Add Templating Support to bacalhau job run by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3020
  • s3 storage pattern filtering by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3031

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.5...v1.1.6

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.6-dev

Dev release for job templating feature

What's Changed

  • Bump github.com/gorilla/websocket from 1.5.0 to 1.5.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/3016
  • Fix canary pipeline by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3005
  • Temporarily removes CI step uploading of python build metadata by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3028
  • Allow IPFS to run in devstack when ./plugins exists by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/3024
  • Add Templating Support to bacalhau job run by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/3020

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.5...v1.1.6-dev

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.5

What's Changed

  • change to orchestrator endoint file to allow cross origin comms by @XandraMcC in https://github.com/bacalhau-project/bacalhau/pull/2998
  • Improve bid strategy error messages by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3006
  • Show node ranking failures in the CLI (again) by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3008
  • Fix Python linting and ensure packages are built on PRs. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3007
  • Bugfix: invalid wasm memory request by @Nasfame in https://github.com/bacalhau-project/bacalhau/pull/2992
  • refactor: no longer write a config file by default by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3009

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.4...v1.1.5

- Go
Published by simonwo over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.5-rc1

What's Changed

  • change to orchestrator endoint file to allow cross origin comms by @XandraMcC in https://github.com/bacalhau-project/bacalhau/pull/2998
  • Improve bid strategy error messages by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3006
  • Show node ranking failures in the CLI (again) by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3008
  • Fix Python linting and ensure packages are built on PRs. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/3007
  • Bugfix: invalid wasm memory request by @Nasfame in https://github.com/bacalhau-project/bacalhau/pull/2992
  • refactor: no longer write a config file by default by @frrist in https://github.com/bacalhau-project/bacalhau/pull/3009

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.4...v1.1.5-rc1

- Go
Published by simonwo over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.4

What's Changed

  • allow describing nodes with their short IDs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2986
  • support shortIDs for GetHistory and GetExecutions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2990
  • fixes: image manifest cache by @Nasfame in https://github.com/bacalhau-project/bacalhau/pull/2993
  • Choose better log stream addresses internally by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2996
  • enable downloading s3 results by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2995
  • change prod bootstrapping peers by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2999

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.3...v1.1.4

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.4-rc2

What's Changed

  • allow describing nodes with their short IDs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2986
  • support shortIDs for GetHistory and GetExecutions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2990
  • fixes: image manifest cache by @Nasfame in https://github.com/bacalhau-project/bacalhau/pull/2993
  • Choose better log stream addresses internally by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2996
  • enable downloading s3 results by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2995
  • change prod bootstrapping peers by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2999

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.3...v1.1.4-rc2

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.3

What's Changed

  • Fix auto-adding GPU labels to nodes. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2951
  • UserID to updates,create/set userID,rename userID by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2943
  • Increase the default job resource limits by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2956
  • Provide option to turn on HTTPS for SDK by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2853
  • Fix: return a reason when a node rejects a job due to capacity. by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2964
  • Fix bug in date formatting during test assertion by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2975
  • Adds SourcePath in conversion to legacy spec by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2974
  • Add job Running state by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2977
  • Don't wait for long running jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2978
  • Fix only one job is restarted during startup by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2979
  • Middleware change to enable CORS by @XandraMcC in https://github.com/bacalhau-project/bacalhau/pull/2976

New Contributors

  • @XandraMcC made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2976

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.2...v1.1.3

- Go
Published by rossjones over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.3-rc1

What's Changed

  • Fix auto-adding GPU labels to nodes. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2951
  • userID to updates,create/set userID,rename userID by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2943
  • Increase the default job resource limits by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2956
  • Provide option to turn on HTTPS for SDK by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2853
  • fix: return a reason when a node rejects a job due to capacity. by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2964
  • Fix bug in date formatting during test assertion by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2975
  • Adds SourcePath in conversion to legacy spec by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2974
  • add job Running state by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2977
  • don't wait for long running jobs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2978
  • fix only one job is restarted during startup by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2979
  • Middleware change to add CORS support by @XandraMcC in https://github.com/bacalhau-project/bacalhau/pull/2976

New Contributors

  • @XandraMcC made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2976

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.2...v1.1.3-rc1

- Go
Published by rossjones over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.2

:warning: Breaking Change

The CLI flag --max-timeout for the bacalhau serve command has been renamed to --max-job-execution-timeout. This new flag more accurately describes its functionality: setting the maximum job execution timeout that the node can accept. If you have any automation or scripts that use bacalhau serve, please update them to use this new flag name.

Main Changes

  • Enforce Normalization in Marshaller Implementations by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2890
  • Flyte integration by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2761
  • whitelist: python package manager by @Nasfame in https://github.com/bacalhau-project/bacalhau/pull/2875
  • Use non-p2p multiaddress for peer by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2897
  • Run update check when any client command is executed by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2909
  • fix migration of new jobs submitted with local directory by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2900
  • Fix time/date substitutions for S3 urls by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2922
  • Limit update checks to once per day by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2918
  • Changes the Publisher interface to take an Execution rather than an ExecutionID by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2924
  • fix job selection policy flag by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2936
  • Add update checking to Bacalhau nodes by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2939
  • fix: allow compute timeouts to be configured by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2931

New Contributors

  • @criadoperez made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2881
  • @Nasfame made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2875

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.1...v1.1.2

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.2

:warning: Breaking Change

The CLI flag --max-timeout for the bacalhau serve command has been renamed to --max-job-execution-timeout. This new flag more accurately describes its functionality: setting the maximum job execution timeout that the node can accept. If you have any automation or scripts that use bacalhau serve, please update them to use this new flag name.

Main Changes

  • Enforce Normalization in Marshaller Implementations by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2890
  • Flyte integration by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2761
  • whitelist: python package manager by @Nasfame in https://github.com/bacalhau-project/bacalhau/pull/2875
  • Use non-p2p multiaddress for peer by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2897
  • Run update check when any client command is executed by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2909
  • fix migration of new jobs submitted with local directory by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2900
  • Fix time/date substitutions for S3 urls by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2922
  • Limit update checks to once per day by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2918
  • Changes the Publisher interface to take an Execution rather than an ExecutionID by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2924
  • fix job selection policy flag by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2936
  • Add update checking to Bacalhau nodes by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2939
  • fix: allow compute timeouts to be configured by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2931

New Contributors

  • @criadoperez made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2881
  • @Nasfame made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2875

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.1...v1.1.2

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.1

What's Changed

  • release v1.1.0 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2857
  • Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.11.83 to 1.11.86 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2859
  • Convert job spec from lowercase to PascalCase by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2856
  • ENG-167 - Flaky test: Allow different target states by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2863
  • Fix builds for OSX by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2823
  • Bump github.com/ipfs/go-unixfsnode from 1.7.4 to 1.9.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2862
  • Bump github.com/jedib0t/go-pretty/v6 from 6.4.6 to 6.4.7 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2867
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2869
  • Remove unused schema site by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2860
  • Bump github.com/ipld/go-car/v2 from 2.11.0 to 2.13.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2871
  • Simple Version Check and Update Notification by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2832
  • Clarify existence of --wait=false by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2873
  • Bump k8s.io/kubectl from 0.28.1 to 0.28.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2874
  • Remove tracing mutex by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2872
  • Bump github.com/invopop/jsonschema from 0.9.0 to 0.11.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2876
  • Ensure GPU tools are present in Bacalhau GPU image by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2880
  • use title-case for version spec by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2883
  • make name optional in job spec and use jobID by default by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2884

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.1.0...v1.1.1

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.1.0

v1.1.0 Release Notes

📢 Introducing Bacalhau v1.1.0 - Unleash the Power!

We are thrilled to announce the release of Bacalhau v1.1.0, a significant milestone in our quest for unparalleled computing capabilities. Packed with exciting new features like Full Fleet Targeting, Configurable Compute Timeouts, persistent storage, integration with private data swarms and API TLS support, this release is sure to take your computational experience to new heights! 🚀

But that's not all! We invite you to explore the experimental features of this release, such as Long-Running Jobs, as we continue to push the boundaries of computational possibilities.

So, what are you waiting for? Upgrade to Bacalhau v1.1.0 and unlock a world of infinite possibilities in distributed computing! 🌟

bash curl https://get.bacalhau.org/install.sh | bash

New features

Full Fleet Targeting

Jobs can now target all nodes in a network simultaneously, allowing for more efficient and parallel operations jobs that need to query or modify an entire fleet.

Full fleet jobs are perfect for fleet management, allowing an operator to quickly understand the state of all of their nodes at once with a single command.

Full fleet jobs will only succeed if all known nodes in a network can be reached and can execute the job successfully. Jobs can still be targeted at a subset of the fleet by using labels or resource requirements.

Pass the --target=all parameter to any Bacalhau job command or set Deal.TargetAll: true in an existing Bacalhau job spec.

New node CLI and APIs

New CLI and APIs have been introduced allowing users to easily list nodes in a network and see what compute resources that are available.

Use the new command bacalhau node list to get a tabular output of all known nodes:

Bacalhau node list output table

You can then use bacalhau node describe to get in-depth output about a specific node.

Configurable Timeouts

Jobs can now last for days or weeks, enabling the execution of big computations that require longer processing times.

By default, compute nodes now do not enforce an execution timeout and jobs default to the longest allowed timeout. Job submitters can still request a timeout using the --timeout flag or the Timeout field in their job spec.

Node operators can still choose to limit the maximum timeout allowed by passing the --max-timeout flag to the serve command or by specifying the new Node.Compute.Capacity.JobTimeouts.MaxJobExecutionTimeout property in their config file.

Richer Node Configuration

We're excited to unveil enhanced configuration options in Bacalhau v1.1.0! With a heightened focus on flexibility, we've expanded the ways you can configure Bacalhau, whether it be via a configuration file, command-line flags, or environment variables.

The new release introduces a persistent configuration file that provides more flexibility and control over node configurations. Read the documentation for how to get started with configuration files.

Key Changes from v1.0.3 to v1.1.0:

  • The enriched config.yaml now has a trove of default configuration values, an improvement from the empty version in v1.0.3.
  • Event and Libp2p tracing is no longer activated by default. Enable this by specifying paths for via EventTracerPath and Libp2PTracerPath in config.yaml.
  • The node’s private key is no longer calledprivate_key.1235 and is named libp2p_private_keyby default. Configure its path with Libp2PKeyPath in config.yaml.
  • user_id.pem remains consistent. Direct its location using KeyPath in config.yaml.
  • Directory name has changed from execution-state-<NODE_ID> to <NODE_ID>-compute, and now, apart from jobStats.json, it also includes executions.db using BoltDB when using persistent storage mode. Define its path using ExecutionStore.Path in config.yaml.
  • New directories include <NODE_ID>-requester (stores the state for the requester node using BoltDB), executor_storages (hosts data for Bacalhau storage types), and plugins (houses executor plugin binaries). Configure their paths respectively via JobStore.Path, ComputeStoragePath, and ExecutorPluginPath in config.yaml.

⚠️Note: there are optional migration steps for existing Bacalhau users who want to keep their previous configuration. See the end of this note for how to migrate.

Support for TLS on public APIs

TLS certificates for serving client-facing APIs are now supported, ensuring secure and encrypted communication between Bacalhau clients and requester nodes.

To use a TLS certificate to encrypt communication, you can:

  • Configure automatic certificates from Let’s Encrypt by passing --autocert=<your-hostname> and ensuring the Bacalhau binary can respond to challenges by running sudo setcap CAP_NET_BIND_SERVICE+ep $(which bacalhau).
  • Pass a certificate to --tlscert and the corresponding private key to --tlskey.

By default, if none of the above options are used, the server will continue to serve its API endpoints over HTTP.

Persistent Storage of Jobs and Executions

Compute and requester nodes now support persistent storage, ensuring data integrity and allowing for long-term job and execution audit records. This feature is now switched on by default and records are persisted to the Bacalhau repository.

See the documentation for how to configure persistence.

Improved Error Messages

Clearer error messages are now displayed when no node is available to run a job, making troubleshooting easier and more efficient.

Instead of receiving ‘not enough nodes to run the job’, users will now get more specific help messages, such as ‘Docker image does not exist or repo is inaccessible’ or ‘job timeout exceeds the maximum allowed’.

Fine-Grained Control Over Image Entrypoint and Parameters

Users now have finer control over the entrypoint and parameters passed to a Docker image. Previously, Bacalhau would ignore the default entrypoint to the image and replace it with the first argument after bacalhau docker run <image>. Now, the default entrypoint in the image is used and all of the positional arguments are passed as the command to that entrypoint.

The entrypoint can still be explicitly overriden by using the --entrypoint flag or by setting the Entrypoint field in a Docker job spec.

GPU Support Inside Docker Containers

Bacalhau now has the capability to automatically utilize GPUs when the Bacalhau node is running inside a Docker container. Ensure that the Bacalhau node is started with a GPU capability by passing --gpus=all to docker run, and Bacalhau nodes will automatically detect GPUs running on the host machine.

Submit a job to a node running inside Docker using bacalhau docker run --gpu=1 to run the job in a new GPU-enabled container on the host.

Support for Private IPFS Clusters

Integration with private IPFS clusters has been added, providing enhanced security and control over data storage and retrieval.

To connect to a private swarm, pass the path to a swarm key to --ipfs-swarm-key, set the BACALHAU_IPFS_SWARM_KEY environment variable or configure the Node.IPFS.SwarmKeyPath configuration property.

When connecting to a private swarm, Bacalhau will no longer bootstrap using or connect to public peers and will rely on the swarm for all data retrieval.

These steps are also necessary on clients who use bacalhau get to download from a private IPFS swarm.

Note that these steps are not necessary if using the --ipfs-connect flag, which already can connect to IPFS nodes running a private swarm.

New Experimental Features

All of these features are experimental, meaning that their APIs are liable to change in an upcoming release. You are encouraged to try out these features and provide feedback or bug reports on Bacalhau Slack.

Long-Running Jobs

Bacalhau jobs can now run indefinitely and will automatically restart when nodes come back online, allowing for continuous and uninterrupted processing.

Long-running jobs allow compute workloads to process data that arrives continuously, and is perfect for tasks such as pre-filtering logs, processing real-time analytics, or working with edge sensors.

With the introduction of long-running jobs, ML inference tasks can now operate in a "warm-boot" environment. This means that the necessary resources and dependencies are already loaded, significantly reducing the time taken to run an inference job.

With this experimental feature, you can now unleash the power of Bacalhau to handle dynamic and ever-changing data streams, ensuring continuous and uninterrupted processing of your computational workloads.

Deprecated Features

Estuary

The Estuary publisher is no longer supported in this release. Compute nodes will now reject jobs that require the Estuary publisher.

Verification

The Verifiers feature is no longer supported in this release. Compute nodes will silently ignore verification requirements on jobs.

⚠️ Migration steps

Users who wish to continue using their previous Bacalhau private key or their previous Bacalhau Client ID as their identity will need to either:

  • Rename private_key.1235 to libp2p_private_key
  • Modify the config.yaml to use the previous key by editing the value of Libp2PKeyPath to point to its path.

Up Next

These upcoming features aim to provide users with increased flexibility and convenience in their computational workflows while maintaining a focus on privacy and security.

User-definable executor plugins

In the next release, users will have the opportunity to experiment with pluggable executors, which will allow them to run jobs without the need to worry about Docker images. The first executor that we will make available will be for Python, and it will be able to execute Python srcipts by using the command bacalhau run python script.py. This feature aims to provide a more seamless and convenient experience for running jobs.

Cluster bootstrapping and private data

Additionally, on the roadmap for future releases, we are planning to introduce easier bootstrapping of Bacalhau clusters. This will simplify the process of setting up and configuring Bacalhau clusters, making it more accessible for users. Furthermore, we are also working on adding support for private data and jobs, ensuring enhanced security and control over sensitive information.

- Go
Published by simonwo over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.0.4

What's Changed

  • Bump vite from 2.9.15 to 2.9.16 in /dashboard/frontend by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2548
  • Pin the curl docker image used in tests by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2558
  • Update http-domain-allowlist.txt by @hakymulla in https://github.com/bacalhau-project/bacalhau/pull/2547
  • Add a node selection algorithm to target all nodes by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2518
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2563
  • Disappeared or maximally retried nodes should be job failures by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2564
  • Bump github.com/docker/docker from 23.0.3+incompatible to 24.0.2+incompatible by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2567
  • Bump golang.org/x/sync from 0.1.0 to 0.2.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2569
  • Bump github.com/imdario/mergo from 0.3.15 to 0.3.16 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2570
  • Bump github.com/aws/aws-sdk-go-v2/config from 1.18.19 to 1.18.26 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2571
  • Fix jobs targeting all nodes being unable to retry by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2572
  • Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.31.1 to 1.34.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2573
  • Add flag for job submitters to control node targeting by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2574
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2575
  • Deprecate v1alpha1 and v1beta1 models, correct API version, & hiberante dashboard by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2550
  • Bump github.com/ipfs/interface-go-ipfs-core from 0.11.1 to 0.11.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2578
  • [ux] Fix display of Inputs/Outputs/Path in describe command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2568
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2579
  • Job Info Event consumer by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2576
  • fix TestEnvWriter by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2577
  • Bump aws-cdk-lib from 2.41.0 to 2.80.0 in /ops/aws/canary by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2582
  • cleanup dead event types by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2584
  • remove lotus filecoin storage and simplify publishers by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2556
  • Remove python and pythonwasm executors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2552
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2586
  • Show node ranking failures in the CLI by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2581
  • Bump github.com/hashicorp/go-retryablehttp from 0.7.2 to 0.7.4 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2583
  • Bump cryptography from 39.0.1 to 41.0.0 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2525
  • Bump github.com/stretchr/testify from 1.8.2 to 1.8.4 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2590
  • Bump github.com/multiformats/go-multicodec from 0.8.1 to 0.9.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2591
  • refactor CLI to align with design guidelines of cobra by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2560
  • fix: add missing error checking and fix linting by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2595
  • Add bacalhau nodes to output details of known nodes. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2599
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2598
  • Fix swagger build and ensure it runs as part of testing. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2602
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2603
  • fix: update py sdk readme example by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2601
  • Implement evaluation broker by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2600
  • use math.Max/Min instead of system.Max/Min by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2609
  • Fix tests when estuary timing out by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2611
  • Update README.md by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/2606
  • Bump github.com/tetratelabs/wazero from 1.0.1 to 1.2.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2610
  • Set cobra's error output to stderr by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2612
  • Ensure prefixes in S3 are unique by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2615
  • Fix container entry points by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/2302
  • implement evaluation worker by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2616
  • Bump github.com/jedib0t/go-pretty/v6 from 6.4.4 to 6.4.6 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2613
  • update metrics to stable otel v1.16 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2620
  • Bump github.com/spf13/viper from 1.15.0 to 1.16.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2621
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2624
  • Bump golang.org/x/sync from 0.2.0 to 0.3.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2626
  • Compute Object Store by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2597
  • implement MetricRecorder by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2627
  • Bump github.com/multiformats/go-multihash from 0.2.2 to 0.2.3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2628
  • Bump github.com/ipfs/go-unixfsnode from 1.7.0 to 1.7.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2641
  • Bump tough-cookie from 4.1.2 to 4.1.3 in /ops/aws/canary by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2640
  • implement batch job scheduler by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2643
  • chore: update Kubo to v0.21 by @Jorropo in https://github.com/bacalhau-project/bacalhau/pull/2642
  • remove MinBids feature by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2647
  • Bump github.com/aws/aws-sdk-go-v2/config from 1.18.26 to 1.18.27 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2649
  • Bump github.com/rs/zerolog from 1.29.0 to 1.29.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2655
  • Switch to maintained fork of gomock by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2656
  • Ensure tools required for go generate are installed by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2654
  • deprecate job verification by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2648
  • remove simulator by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2652
  • speed up node discovery during startup by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2653
  • Pin chainguard image in dockerfile by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2634
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2664
  • Bump github.com/docker/docker from 24.0.2+incompatible to 24.0.4+incompatible by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2663
  • Fix assignment of reference in nodeinfo lookup by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2665
  • Preserve storage path and name when copying by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2667
  • chore(docs): add pre-release instructions by @philwinder in https://github.com/bacalhau-project/bacalhau/pull/2208
  • Bump github.com/aws/aws-sdk-go-v2 from 1.18.1 to 1.19.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2671
  • Transport log cmd errors to the client in main message struct by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2670
  • Bump cryptography from 41.0.0 to 41.0.2 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2673
  • Bump go.ptx.dk/multierrgroup from 0.0.2 to 0.0.3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2674
  • Bump k8s.io/apimachinery from 0.27.2 to 0.27.3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2676
  • refactor: executor interface and storage dependency by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2675
  • Bump github.com/ipfs/go-unixfsnode from 1.7.1 to 1.7.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2677
  • Bump word-wrap from 1.2.3 to 1.2.5 in /ops/aws/canary by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2682
  • introduce scheduler plan executors + migrate requester by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2681
  • Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.11.60 to 1.11.72 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2685
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2684
  • bug: Does this manifest a bug in CI? by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2680
  • Bump github.com/aws/aws-sdk-go-v2/config from 1.18.28 to 1.18.29 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2691
  • Fix all outstanding/broken lints by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2687
  • Bump certifi from 2022.12.7 to 2023.7.22 in /ops by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2689
  • Bump k8s.io/apimachinery from 0.27.3 to 0.27.4 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2693
  • Bump github.com/aws/aws-sdk-go-v2 from 1.19.0 to 1.19.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2694
  • Adds cancel function to Python SDK by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2697
  • introduce ops job type, a batch job that targets all matching nodes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2690
  • Persistent Jobstore by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2650
  • Implement pluggable executors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2637
  • Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.37.0 to 1.38.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2703
  • Bump google.golang.org/grpc from 1.56.1 to 1.57.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2707
  • Disable log client id checking by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2704
  • Bump github.com/tetratelabs/wazero from 1.2.1 to 1.3.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2711
  • Bump cryptography from 41.0.2 to 41.0.3 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2708
  • Bump github.com/rs/zerolog from 1.29.1 to 1.30.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2715
  • Allow maximum job execution time to be configured by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2705
  • Fix linter errors preventing main builds from passing. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2720
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2721
  • Bump github.com/hashicorp/go-hclog from 1.2.0 to 1.5.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2722
  • Bump github.com/aws/aws-sdk-go-v2/config from 1.18.29 to 1.18.33 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2725
  • Bump commonmarker from 0.23.9 to 0.23.10 in /schema.bacalhau.org by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2724
  • [Py SDK] Remove coupling of host and api env vars by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2717
  • Bump github.com/libp2p/go-libp2p from 0.27.7 to 0.27.8 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2729
  • Bump github.com/libp2p/go-libp2p from 0.26.3 to 0.27.8 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2727
  • Bump github.com/libp2p/go-libp2p from 0.27.7 to 0.27.8 in /apps/job-info-consumer/consumer by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2728
  • Add EngineSpec to Job Spec by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2706
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2732
  • Update poetry deps (certifi/poetry) by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2731
  • Bump github.com/tetratelabs/wazero from 1.3.1 to 1.4.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2733
  • Bump github.com/go-git/go-git/v5 from 5.7.0 to 5.8.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2735
  • Add Simon's and John's client IDs to the demo network trusted list. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2736
  • Remove estuary storage/publisher by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2730
  • Bump github.com/ipld/go-ipld-prime from 0.20.0 to 0.21.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2737
  • Bump github.com/docker/docker from 24.0.4+incompatible to 24.0.5+incompatible by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2741
  • Bump go.uber.org/zap from 1.24.0 to 1.25.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2747
  • Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.11.72 to 1.11.77 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2749
  • Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.38.2 to 1.38.3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2753
  • Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.11.77 to 1.11.79 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2756
  • New model types by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2719
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2760
  • Bump k8s.io/kubectl from 0.27.0 to 0.28.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2764
  • Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.11.79 to 1.11.81 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2766
  • Use priority field for compute node execute buffer by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2765
  • Bump github.com/tetratelabs/wazero from 1.4.0 to 1.5.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2770
  • Bump github.com/ipld/go-car/v2 from 2.9.1-0.20230325062757-fff0e4397a3d to 2.11.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2771
  • Bump go.opentelemetry.io/otel/trace from 1.16.0 to 1.17.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2773
  • Restart long running jobs after compute node visibility change by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2734
  • Detect system GPUs using nvidia-smi binary by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2777
  • refactor: implement configuration by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2740
  • Fix lint by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2781
  • Use a temporary folder for the devstack repo by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2780
  • Default to persistent stores by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2779
  • flaky: Improve live execution test for Windows by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2782
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc from 0.39.0 to 0.40.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2774
  • use go-chi api router + refactor api package structure by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2772
  • use echo web framework by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2775
  • Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from 0.42.0 to 0.43.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2783
  • Bump k8s.io/apimachinery from 0.28.0 to 0.28.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2786
  • Auto cert for TLS certificates by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2785
  • Frrist/fix api envvar by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2788
  • Update README.md by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/2790
  • Add option to specify location of devstack config by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2784
  • New Job APIs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2791
  • Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.11.81 to 1.11.83 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2789
  • Bump google.golang.org/grpc from 1.57.0 to 1.58.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2793
  • Support IPFS private swarm keys by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2792
  • Bump k8s.io/kubectl from 0.28.0 to 0.28.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2795
  • fix: print correct env var names when running serve by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2794
  • Async Compute Executors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2778
  • Remove shellcheck binary from repo by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2798
  • Adds manual TLS certificate support to requester by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2800
  • New job, node and agent CLIs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2802
  • Remove using cancel when restarting long-running jobs by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2796
  • fix failing to start when HOME is ureachable by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2805
  • rebuild staging in us-east4 with 1 gpu node by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2807

New Contributors

  • @hakymulla made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2547

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.0.3...v1.0.4

- Go
Published by wdbaruni over 2 years ago

https://github.com/bacalhau-project/bacalhau - v1.0.3

What's Changed

  • release v1.0.0 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2458
  • Bump markdown-it-py from 2.1.0 to 2.2.0 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2454
  • Bump github.com/ipfs/go-cid from 0.3.2 to 0.4.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2457
  • Bump github.com/docker/distribution from 2.8.1+incompatible to 2.8.2+incompatible by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2462
  • Bump github.com/ipfs/kubo from 0.18.2-0.20230214211946-0ff406170dd9 to 0.19.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2460
  • Bump github.com/cloudflare/circl from 1.1.0 to 1.3.3 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2464
  • Bump github.com/docker/distribution from 2.8.1+incompatible to 2.8.2+incompatible in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2465
  • Bump github.com/ipfs/kubo from 0.18.2-0.20230214211946-0ff406170dd9 to 0.19.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2461
  • Bump github.com/cloudflare/circl from 1.1.0 to 1.3.3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2463
  • Strip debug symbols from wasm builds by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2471
  • Skip S3 tests if no credentials are present by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2473
  • Fix examples for 1.0.0 Python SDK by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2474
  • Bump github.com/ipfs/go-unixfsnode from 1.5.2 to 1.7.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2477
  • Insulated jobs: post-job results moderation by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2468
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2478
  • Storage cleanup of inputs by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2449
  • Resolve docker digests from supplied image ids by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2410
  • Print errors if no nodes complete the job successfully by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2487
  • Bump requests from 2.28.2 to 2.31.0 in /python by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2490
  • Bump github.com/lib/pq from 1.10.7 to 1.10.9 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2488
  • Bump requests from 2.28.1 to 2.31.0 in /ops by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2489
  • Bump github.com/go-git/go-git/v5 from 5.6.1 to 5.7.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2493
  • Add docker secrets to terraform by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2417
  • Record Canary Jobs in Postgres Database by @OwainJones in https://github.com/bacalhau-project/bacalhau/pull/2494
  • chore: remove python and wasm_python executors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2373
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2495
  • Cache docker manifests for compute node bidding by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2491
  • Extend deadline for serve tests by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2497
  • migrate to expanso's grafana by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2501
  • Bump k8s.io/apimachinery from 0.27.0 to 0.27.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2496
  • enable mounting writable local paths by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2499
  • update to libp2p v0.27.4 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2506
  • Bump github.com/multiformats/go-multihash from 0.2.1 to 0.2.2 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2509
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2510
  • Disable requester node docker caching until we can address issues by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2511
  • 2482 output peering information to run by @aronchick in https://github.com/bacalhau-project/bacalhau/pull/2485
  • Revert removal of python and wasm_python executors by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2513
  • Bump github.com/samber/lo from 1.36.0 to 1.38.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2515
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2517

New Contributors

  • @OwainJones made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2494

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.0.0...v1.0.3

- Go
Published by wdbaruni about 3 years ago

https://github.com/bacalhau-project/bacalhau - v1.0.0

Bacalhau 1.0 Release: Featuring Private Clusters, Octostore, and Federated Learning

Today marks the launch of Bacalhau 1.0, the general availability (GA) release of the open source distributed compute platform. The project’s mission is to revolutionize the way organizations and developers harness the power of collaborative computing, and the GA release marks an important milestone towards that goal. Since launching our beta release in November, the project has seen more than 3,000 commits from more than 30 contributors and a release every two weeks. Additionally, customers like the New Atlantis Foundation, the City of Las Vegas, and the University of Maryland are executing hundreds of thousands of jobs every month on the public network. To read more about Bacalhau, and try it out for yourself, go to https://bacalhau.org/.

Background

​​Distributed computing has long been recognized as a powerful approach for tackling large-scale, complex problems by harnessing the collective power of devices everywhere. However, developers face significant challenges in adopting it, including inefficient resource allocation, communication bottlenecks, and high barriers to entry for non-expert users.

But the time to address the issues is now. By 2025, IDC believes that we will have generated more than 175 zettabytes of data, 50 times more data than we do today. Yet critical insights to make better decisions are hidden behind distributed devices and storage.

(Re-)Introducing the Bacalhau Project

Bacalhau was created to address these challenges head-on through a platform designed from the ground up for the distributed world. Built by core members of the Kubernetes, Kubeflow, Amazon Kinesis communities and employees from Google, AWS, and Microsoft, Bacalhau provides a new way to build and use globally deployed applications and data that is familiar, high scale, and efficient. Further, because Bacalhau is open source and Apache2/MIT licensed, the community is built to foster collaboration and innovation, allowing developers from around the world to contribute their expertise and continually improve upon the platform.

General Availability Release of Bacalhau

The GA release of Bacalhau includes the following features:

Long Term Mission

Our long term goal is to transform the way that developers can interact with the breadth of computing and data resources out there. Some of the features we have on the horizon include: A fully distributed computation platform that can run on any device, anywhere A declarative pipeline that can both run the data processing and also record the lineage of the data A highly resilient system that can schedule across latency boundaries and deliver the reliability a global deployment needs, even over spotty network connectivity Secure and verifiable results that can be used to confirm the integrity and reproducibility of the results forever

But you tell us! We'd love to hear about new directions we may need to include.

How to Get Involved

We're looking for help in several areas. If you're interested in helping out, please reach out to us at any of the following locations:

- Go
Published by wdbaruni about 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.29

What's Changed

  • Show in the CLI when a job is queued by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2370
  • add s3 secrets to terraform by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2372
  • feat: functions to relate job inputs & outputs by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2304
  • Changes to repo provider logging by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2384
  • Make log-follower wait for container to exist by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2380
  • Logging for wasm by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2301
  • minor typo by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2386
  • Allows bprotocol to process errors with responses by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2374
  • Enable download of single file from output by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2345
  • Allow the simulator to return errors more gracefully too by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2388
  • Node operators can disable features with CLI flags by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2389
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2390
  • Update the version of coverage used in dev requirements by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2392
  • Improve performance of wasm logging by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2391
  • Ensure feature tests pass when features not disabled. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2400
  • Show Client ID in bacalhau id output by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2404
  • Ensure all nodes have announced before running devstack tests by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2408
  • Only print compute node API URL if node is a compute node. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2409
  • Add dashes in name and memory of the GPUs by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2396
  • Use content-disposition for downloads where available by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2416
  • Bump golang.org/x/crypto from 0.7.0 to 0.8.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2425
  • Bump github.com/filecoin-project/go-state-types from 0.10.0 to 0.11.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2428
  • Increase the size of the VM's data disk by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2432
  • Reset the disk size EU only allows 1TB by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2435
  • fix: network config validation by @frrist in https://github.com/bacalhau-project/bacalhau/pull/2418
  • enable local directory storage provider by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2436

New Contributors

  • @frrist made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2304

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v0.3.28...v0.3.29

- Go
Published by wdbaruni about 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.28

What's Changed

  • add canary favouring non-bacalhau nodes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2220
  • Stream docker logs to the CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2161
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2223
  • fix(ci): python sdk release by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2221
  • Let wasm run have it's own example in CLI doc. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2228
  • Fix error logs devstack by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/2226
  • Repo mount by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2232
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2233
  • Return error message in execution failure events. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2235
  • [serve cmd] defaults for private clusters + improve docs by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2196
  • Enforce lints in CI rather than locally by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2237
  • Automatically resolve WebAssembly module imports by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2242
  • use executionID instead of jobID to uniquely identify executions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2240
  • Allow fetching of events/history 'since' a specific time by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2238
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2245
  • Fix typo by @fakela in https://github.com/bacalhau-project/bacalhau/pull/2252
  • Rate-limiting fix for compute node docker pulls by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2243
  • Enables filtering of job history by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2247
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2256
  • Update README.md by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2257
  • Update py sdk README.md by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2258
  • Expose port 1000 for dashboard to avoid nginx proxies that have hardc… by @lukemarsden in https://github.com/bacalhau-project/bacalhau/pull/2260
  • Added Install GIt and Git-lfs function in install-node.sh by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2244
  • deps: updates wazero to 1.0.1 by @codefromthecrypt in https://github.com/bacalhau-project/bacalhau/pull/2262
  • Remove unused type from logs endpoint by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2263
  • Adds a wasm module for testing wasm logging code by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2259
  • add s3 storage provider by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2264
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2265
  • fix closing downloaded s3 files by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2266
  • Add URI example by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2278
  • generic cli storage inputs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2285
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2286
  • Bump modernc.org/sqlite from 1.20.2 to 1.21.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2275
  • Support human pre-moderation of job execution by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2288
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2294
  • Fix storage cli string representation by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2295
  • Fixes to repo mount by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2299
  • Fixes race condition with spinner when retry fails by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2236
  • Removes the result folder immediately after publish by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2309
  • Allow compute nodes to receive async job authorizations by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2307
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2310
  • Bump golang.org/x/crypto from 0.6.0 to 0.7.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2276
  • Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.11.59 to 1.11.60 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2274
  • Bump github.com/joho/godotenv from 1.4.0 to 1.5.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2270
  • Bump github.com/mattn/go-isatty from 0.0.17 to 0.0.18 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2269
  • Bump github.com/docker/docker from 23.0.1+incompatible to 23.0.3+incompatible in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2311
  • Bump github.com/docker/docker from 23.0.1+incompatible to 23.0.3+incompatible by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2312
  • Bump golang.org/x/mod from 0.8.0 to 0.10.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2313
  • Automatic update and merge of dependabot requests by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2336
  • Make sure auto action can author and push commits. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2338
  • Apply job selection policies in the dashboard by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2340
  • Correctly detect identity. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2339
  • Dependabot: check out a branch to push commits to. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2341
  • add s3 publisher by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2333
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2344
  • Dependabot: push directly to branch. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2347
  • Bump nokogiri from 1.14.1 to 1.14.3 in /schema.bacalhau.org by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2343
  • Bump commonmarker from 0.23.8 to 0.23.9 in /schema.bacalhau.org by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2342
  • Dependabot: check out the correct branch to push back to by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2348
  • Bump github.com/libp2p/go-libp2p-pubsub from 0.9.1 to 0.9.3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2273
  • Bump github.com/swaggo/http-swagger from 1.3.3 to 1.3.4 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2272
  • Bump k8s.io/kubectl from 0.26.1 to 0.27.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2349
  • Bump github.com/multiformats/go-multiaddr from 0.8.0 to 0.9.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2271
  • Only try and commit docs changes when updating Swagger. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2350
  • Ensure the minimum number of bids is reached before execution starts by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2346
  • feat: serve default node type is requester - fixes #2305 by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2308
  • Update path when generating swagger-docs in CI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2353
  • Fixes a yaml typo by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2354
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2355
  • fix(ci): build and publish py sdk in two separate steps by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2356
  • Update bac version in terraform for deployment by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2357
  • Update bacalhau version for canary by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2358
  • set MinBacalhauVersion to v0.3.26 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2359
  • Allow network traffic to RPC Endpoints (Polygon, Ethereum, Binance, and Filecoin) by @kelindi in https://github.com/bacalhau-project/bacalhau/pull/2352
  • fix: identity node discoverer only discovers nodes with addrs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2362

New Contributors

  • @kelindi made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2352

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v0.3.25...v0.3.28

- Go
Published by wdbaruni about 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.27

What's Changed

  • Update bac version in terraform for deployment by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2357
  • Update bacalhau version for canary by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2358
  • set MinBacalhauVersion to v0.3.26 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2359

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v0.3.26...v0.3.27

- Go
Published by wdbaruni about 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.26

What's Changed

  • add canary favouring non-bacalhau nodes by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2220
  • Stream docker logs to the CLI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2161
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2223
  • fix(ci): python sdk release by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2221
  • Let wasm run have it's own example in CLI doc. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2228
  • Fix error logs devstack by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/2226
  • Repo mount by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2232
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2233
  • Return error message in execution failure events. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2235
  • [serve cmd] defaults for private clusters + improve docs by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2196
  • Enforce lints in CI rather than locally by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2237
  • Automatically resolve WebAssembly module imports by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2242
  • use executionID instead of jobID to uniquely identify executions by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2240
  • Allow fetching of events/history 'since' a specific time by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2238
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2245
  • Fix typo by @fakela in https://github.com/bacalhau-project/bacalhau/pull/2252
  • Rate-limiting fix for compute node docker pulls by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2243
  • Enables filtering of job history by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2247
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2256
  • Update README.md by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2257
  • Update py sdk README.md by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2258
  • Expose port 1000 for dashboard to avoid nginx proxies that have hardc… by @lukemarsden in https://github.com/bacalhau-project/bacalhau/pull/2260
  • Added Install GIt and Git-lfs function in install-node.sh by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2244
  • deps: updates wazero to 1.0.1 by @codefromthecrypt in https://github.com/bacalhau-project/bacalhau/pull/2262
  • Remove unused type from logs endpoint by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2263
  • Adds a wasm module for testing wasm logging code by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2259
  • add s3 storage provider by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2264
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2265
  • fix closing downloaded s3 files by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2266
  • Add URI example by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2278
  • generic cli storage inputs by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2285
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2286
  • Bump modernc.org/sqlite from 1.20.2 to 1.21.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2275
  • Support human pre-moderation of job execution by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2288
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2294
  • Fix storage cli string representation by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2295
  • Fixes to repo mount by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2299
  • Fixes race condition with spinner when retry fails by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2236
  • Removes the result folder immediately after publish by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2309
  • Allow compute nodes to receive async job authorizations by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2307
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2310
  • Bump golang.org/x/crypto from 0.6.0 to 0.7.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2276
  • Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.11.59 to 1.11.60 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2274
  • Bump github.com/joho/godotenv from 1.4.0 to 1.5.1 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2270
  • Bump github.com/mattn/go-isatty from 0.0.17 to 0.0.18 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2269
  • Bump github.com/docker/docker from 23.0.1+incompatible to 23.0.3+incompatible in /ops/aws/canary/lambda by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2311
  • Bump github.com/docker/docker from 23.0.1+incompatible to 23.0.3+incompatible by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2312
  • Bump golang.org/x/mod from 0.8.0 to 0.10.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2313
  • Automatic update and merge of dependabot requests by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2336
  • Make sure auto action can author and push commits. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2338
  • Apply job selection policies in the dashboard by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2340
  • Correctly detect identity. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2339
  • Dependabot: check out a branch to push commits to. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2341
  • add s3 publisher by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2333
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2344
  • Dependabot: push directly to branch. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2347
  • Bump nokogiri from 1.14.1 to 1.14.3 in /schema.bacalhau.org by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2343
  • Bump commonmarker from 0.23.8 to 0.23.9 in /schema.bacalhau.org by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2342
  • Dependabot: check out the correct branch to push back to by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2348
  • Bump github.com/libp2p/go-libp2p-pubsub from 0.9.1 to 0.9.3 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2273
  • Bump github.com/swaggo/http-swagger from 1.3.3 to 1.3.4 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2272
  • Bump k8s.io/kubectl from 0.26.1 to 0.27.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2349
  • Bump github.com/multiformats/go-multiaddr from 0.8.0 to 0.9.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2271
  • Only try and commit docs changes when updating Swagger. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2350
  • Ensure the minimum number of bids is reached before execution starts by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2346
  • feat: serve default node type is requester - fixes #2305 by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2308
  • Update path when generating swagger-docs in CI by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2353
  • Fixes a yaml typo by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2354
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2355
  • fix(ci): build and publish py sdk in two separate steps by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2356

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v0.3.25...v0.3.26

- Go
Published by rossjones about 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.25

What's Changed

  • Airflow readme + worked example by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2133
  • Update README.md by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2138
  • Update Airflow README.md by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2139
  • instructions to deploy and test on staging by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2140
  • Mark a job as completed when executions finish by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2137
  • Avoid multiple addresses when connecting nodes by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2143
  • Don't use relative path for the caller for deps. by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2147
  • Update to Go 1.20 by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2148
  • Lookup the latest version of Go by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2150
  • Allow in-process IPFS to find a random port by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2153
  • Avoid concurrent map write in the simulator by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2156
  • Don't fetch GOPATH from env. var. by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2157
  • Implement a way to get output from a running job by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2159
  • Attempt to avoid timing out when downloading by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2162
  • Add simple documentation for run python by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2165
  • Add tracing around WASM execution by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2169
  • fix(ci): only build containers on real releases by @philwinder in https://github.com/bacalhau-project/bacalhau/pull/2170
  • Build & release to PyPi CI pipeline + align package versions with same release tag by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2154
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2171
  • Better handling of pypi version scheme by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2172
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2173
  • makefile pypi version sed fix by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2174
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2175
  • Uri storage spec and constraint job scheduling using labels by @js-ts in https://github.com/bacalhau-project/bacalhau/pull/2155
  • Add job count to persistent local store by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/2193
  • enable retries with pluggable retry strategies by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2205
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2206
  • Minor lints by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2207
  • Add log message describing where to find debug API by @olgibbons in https://github.com/bacalhau-project/bacalhau/pull/2209
  • polish readme by @it09 in https://github.com/bacalhau-project/bacalhau/pull/2204
  • Simplify client interface for setting host and port by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2212
  • deps: updates wazero to 1.0.0-rc.2 by @codefromthecrypt in https://github.com/bacalhau-project/bacalhau/pull/2195
  • enable downloading raw results by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2217
  • release v0.3.25 by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2219

New Contributors

  • @it09 made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/2204

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v0.3.24...v0.3.25

- Go
Published by wdbaruni about 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.23

What's Changed

  • Add support for passing a context when cleaning up by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2028
  • Use context for more logging statements by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2034
  • Apply Job selection policies in the Requestor node by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2036
  • Fixes incorrect state using list command by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2016
  • Update Bacalhau to v0.3.22 by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2027
  • fix(ci): disable performance tests because we don't have a runner by @philwinder in https://github.com/bacalhau-project/bacalhau/pull/2053
  • Migrate filecoin-project to bacalhau-project. by @simonwo in https://github.com/bacalhau-project/bacalhau/pull/2013
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2055
  • [Ready] chore(dependencies): update kubo and related libraries by @philwinder in https://github.com/bacalhau-project/bacalhau/pull/2030
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc from 1.12.0 to 1.13.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2049
  • Bump golang.org/x/crypto from 0.5.0 to 0.6.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2041
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp from 1.7.0 to 1.13.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2040
  • Bump go.opentelemetry.io/otel/bridge/opencensus from 0.35.0 to 0.36.0 by @dependabot in https://github.com/bacalhau-project/bacalhau/pull/2048
  • a go.work.sum update by @lukemarsden in https://github.com/bacalhau-project/bacalhau/pull/2062
  • dashboard: expose postgres port by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2063
  • ignore false positives from go.work.sum by @lukemarsden in https://github.com/bacalhau-project/bacalhau/pull/2061
  • chore(deps): updating ipfs related deps by @philwinder in https://github.com/bacalhau-project/bacalhau/pull/2058
  • filter compute nodes with old bacalhau version by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2066
  • fix scheduler race condition by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2067
  • Use the latest version of russross/blackfriday dep by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2070
  • Moves the telemetry DetachedContext to util by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2060
  • Update workspace sum file for blackfriday/heredoc by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2072
  • Expose metrics for libp2p using OpenTelemetry by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2074
  • canary build from new repo by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2068
  • fix DevstackConcurrencySuite flakiness by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2077
  • set BACALHAU_ENVIRONMENT to production by default by @wdbaruni in https://github.com/bacalhau-project/bacalhau/pull/2078
  • Adds a frontend to the cancel command. by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2037
  • Removes cancelled task from logging sensor by @rossjones in https://github.com/bacalhau-project/bacalhau/pull/2075
  • Don't trace local activity when DONOTTRACK by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2079
  • Change test case to verify truncation by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2080
  • Make sure Docker pull errors appear to the user by @wjam in https://github.com/bacalhau-project/bacalhau/pull/2081
  • Fix naming bac airflow by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2025
  • [readme] py sdk install command by @enricorotundo in https://github.com/bacalhau-project/bacalhau/pull/2084

Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v0.3.22...v0.3.23

- Go
Published by wdbaruni over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.22

What's Changed

  • Update Bacalhau to v0.3.21. by @simonwo in https://github.com/filecoin-project/bacalhau/pull/2006
  • Bump golang.org/x/net from 0.5.0 to 0.7.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/2011
  • Bump golang.org/x/net from 0.5.0 to 0.7.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/filecoin-project/bacalhau/pull/2012
  • Airflow operator by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1924
  • Log all messages in the correct format by @wjam in https://github.com/filecoin-project/bacalhau/pull/2017
  • Clarifies function doing output of job events for submit by @rossjones in https://github.com/filecoin-project/bacalhau/pull/2020
  • Update setup.py by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/2021
  • Ensure successful publishing is favoured over errors by @simonwo in https://github.com/filecoin-project/bacalhau/pull/2024
  • Move SystemManager & span handling to root command by @wjam in https://github.com/filecoin-project/bacalhau/pull/2026

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.21...v0.3.22

- Go
Published by enricorotundo over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.21

What's Changed

  • dashboard: remove annotations box by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1998
  • Add the span, trace & job IDs to log statements by @wjam in https://github.com/filecoin-project/bacalhau/pull/2000
  • Adds a cancel command to the CLI by @rossjones in https://github.com/filecoin-project/bacalhau/pull/1979
  • Don't forget to release Darwin ARM64 builds. by @simonwo in https://github.com/filecoin-project/bacalhau/pull/2003
  • chore(docs): add link to PL curriculum by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1999

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.20...v0.3.21

- Go
Published by simonwo over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.20

What's Changed

  • release v0.3.19 by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1919
  • Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from 0.38.0 to 0.39.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1921
  • Add missing v0.3.19 JSONSchema. by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1923
  • Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from 0.38.0 to 0.39.0 in /ops/aws/canary/lambda by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1920
  • Bump cryptography from 38.0.4 to 39.0.1 in /python by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1915
  • Implement job cancellation for all executors by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1952
  • Update README.md by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1953
  • Add 'Station Mode' to logging options for integration with Station by @olgibbons in https://github.com/filecoin-project/bacalhau/pull/1956
  • Build for ARMv6 and ARMv7 platforms by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1960
  • Improve the naming of the distributed traces by @wjam in https://github.com/filecoin-project/bacalhau/pull/1957
  • Ensures makefile build task sees main.go changes by @rossjones in https://github.com/filecoin-project/bacalhau/pull/1959
  • Fail with more helpful error when jq is not installed. by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1966
  • Trace SQL commands automatically by @wjam in https://github.com/filecoin-project/bacalhau/pull/1967
  • Avoid aws_key precommit false positives with import statements by @rossjones in https://github.com/filecoin-project/bacalhau/pull/1968
  • dashboard consume from gossipsub instead of websocket by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1970
  • new job store with fsm support, and explicit job state by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1972
  • upgrade canary to go 1.19 by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1974
  • update deployment script to install go1.19 by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1975
  • deps: updates wazero to 1.0.0-pre.8 by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1976
  • Stop the spinner when the command is interupted by @rossjones in https://github.com/filecoin-project/bacalhau/pull/1969
  • Trace Lotus & Estuary communication by @wjam in https://github.com/filecoin-project/bacalhau/pull/1977
  • Make sure files are written to the correct dir by @wjam in https://github.com/filecoin-project/bacalhau/pull/1980
  • Shares devstack environment vars via a dotenv file by @rossjones in https://github.com/filecoin-project/bacalhau/pull/1958
  • Add tracing to the docker client by @wjam in https://github.com/filecoin-project/bacalhau/pull/1984
  • Use Station Environment variable as config dir by @olgibbons in https://github.com/filecoin-project/bacalhau/pull/1983
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1985
  • Trace remote IPFS queries by @wjam in https://github.com/filecoin-project/bacalhau/pull/1988
  • Give more context to single span traces by @wjam in https://github.com/filecoin-project/bacalhau/pull/1989
  • update tempo user id by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1994
  • fix missing job specs in published events by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1995

New Contributors

  • @rossjones made their first contribution in https://github.com/filecoin-project/bacalhau/pull/1959

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.19...v0.3.20

- Go
Published by wdbaruni over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.19

What's Changed

  • add SubmitWithConcurrencyOwnedNodes canary by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1796
  • deploy v0.3.18 by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1798
  • Avoid concurrent map writes in provider caches by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1799
  • Start IPFS node when ipfs-connect is not defined by @wjam in https://github.com/filecoin-project/bacalhau/pull/1746
  • migrate staging to us-west4 by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1806
  • Don't throw an error when not running docker by @olgibbons in https://github.com/filecoin-project/bacalhau/pull/1805
  • Dashboard stable diffusion endpoint and generic run endpoint by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1697
  • Make it easy to start a private Bacalhau network by @wjam in https://github.com/filecoin-project/bacalhau/pull/1803
  • Bump contrib.go.opencensus.io/exporter/prometheus from 0.4.0 to 0.4.2 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1831
  • Simplify schema generation to remove security vulnerabilities by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1837
  • Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc from 1.11.2 to 1.12.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1828
  • Bump github.com/swaggo/swag from 1.8.9 to 1.8.10 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1830
  • Bump github.com/ipfs/go-merkledag from 0.8.1 to 0.9.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1712
  • fix(ci): remove unused circleci job (updatecli) by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1842
  • Improve IPFS logging output by @wjam in https://github.com/filecoin-project/bacalhau/pull/1840
  • fix(docker): run binary from location on path by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1841
  • Bump go.opentelemetry.io/otel/exporters/stdout/stdouttrace from 1.11.2 to 1.12.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1829
  • Support relative paths for WASM files by @wjam in https://github.com/filecoin-project/bacalhau/pull/1844
  • Bump github.com/ipfs/go-block-format from 0.0.3 to 0.1.1 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1827
  • Bump github.com/multiformats/go-multicodec from 0.7.0 to 0.8.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1832
  • Closes #1843 by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1845
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1846
  • Fix broken py sdk tests - missing pypi package by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1853
  • 1849 uplift python sdk + ci releases bacalhau apiclient to pypi by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1850
  • Stop logging the text and the bytes of the text by @wjam in https://github.com/filecoin-project/bacalhau/pull/1854
  • Clean up IPFS FUSE storage provider by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1847
  • Output the command to run a private node by @wjam in https://github.com/filecoin-project/bacalhau/pull/1856
  • split circle ci tests into separate workflows by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1865
  • migrate metrics to otel with a configurable exporter by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1864
  • improve stability of NodeSelectionSuite by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1866
  • enable deploying bacalhau from source by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1867
  • update canary to use bacalhau with new staging swarm nodes by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1868
  • fix IPFS canary timeout values by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1870
  • Output command to use private requester by @wjam in https://github.com/filecoin-project/bacalhau/pull/1873
  • deps: updates wazero to 1.0.0-pre.8 by @evacchi in https://github.com/filecoin-project/bacalhau/pull/1848
  • Add a test for local node not auto-discovering by @wjam in https://github.com/filecoin-project/bacalhau/pull/1874
  • Stop Docker tests crashing by @wjam in https://github.com/filecoin-project/bacalhau/pull/1879
  • python sdk should generate a key-pair by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1857
  • Update README.md by @fakela in https://github.com/filecoin-project/bacalhau/pull/1839
  • Fix resource leak with IPFS download nodes by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1890
  • disable submitDockerIPFSJobAndGet alarm by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1891
  • use shorter systemd service names, and set default bacalhau_branch value by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1892
  • Make CPU & memory profiling optional by @wjam in https://github.com/filecoin-project/bacalhau/pull/1894
  • #1607 Fixed by @KalindaC-muSpace in https://github.com/filecoin-project/bacalhau/pull/1872
  • Lotus test fails too often in CI so skipping by @wjam in https://github.com/filecoin-project/bacalhau/pull/1898
  • canary local test Readme by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1900
  • Fix/look into flaky tests by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1899
  • 1764 python sdk should be able to generate a key pair by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1887
  • chore(canary): update to 1.19 and use bac nodes only by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1903
  • Update README.md by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1904
  • Revert "chore(canary): upgrade to 1.19" by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1905
  • add a pre-prod manual approval step in the canary by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1906
  • improve stability of canary_check owned nodes by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1907
  • Ipfs peering by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1908
  • Add unit test for Lotus Filecoin publisher by @wjam in https://github.com/filecoin-project/bacalhau/pull/1911
  • increase IPFS canary's cadance from 3 to 5 minutes by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1913
  • reduce metric cardinality, and deploy staging from main by default by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1914
  • Remove 5 minute IPFS timeout by @wjam in https://github.com/filecoin-project/bacalhau/pull/1917

New Contributors

  • @olgibbons made their first contribution in https://github.com/filecoin-project/bacalhau/pull/1805
  • @evacchi made their first contribution in https://github.com/filecoin-project/bacalhau/pull/1848
  • @KalindaC-muSpace made their first contribution in https://github.com/filecoin-project/bacalhau/pull/1872

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.18...v0.3.19

- Go
Published by wdbaruni over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.18

What's Changed

  • +x apply-http-allowlist.sh by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1750
  • update terraform to v0.3.17 by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1753
  • update canary to v0.3.17 by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1754
  • canary pipeline integration test by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1755
  • Update HTTP gateway image to the latest version by @wjam in https://github.com/filecoin-project/bacalhau/pull/1759
  • Stop TestDockerNetworkingAppendsHTTPHeader flaking by @wjam in https://github.com/filecoin-project/bacalhau/pull/1760
  • Check storage and publishers before bidding on job by @wjam in https://github.com/filecoin-project/bacalhau/pull/1761
  • Add text feedback when user doesnt specify an IMAGE tag/digest [new] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1763
  • Python SDK by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1519
  • Allow network traffic to two BOINC projects by @wjam in https://github.com/filecoin-project/bacalhau/pull/1769
  • feat(docker): add bacalhau Docker container by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1748
  • Remove broken workflow by @wjam in https://github.com/filecoin-project/bacalhau/pull/1771
  • V0.3.17 fixes by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1775
  • [circleci] Build jsonschema - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1777
  • Staging canary by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1778
  • refactor: Use inmemory Kubo API if the local node is used. by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1615
  • refactor(ci): remove updatecli check by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1781
  • local directory storage driver by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/1782
  • close bprotocol sessions when done by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1783
  • [circleci] Build jsonschema - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1784
  • Inform Iryna on new releases by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1787
  • enable node-info store for both compute and requester nodes by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1789
  • add custom libp2p resource limits by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1790
  • fail fast in unable to reach --ipfs-connect by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1791
  • Modifies board metrics tool to export relevant events to CSV by @ricmk in https://github.com/filecoin-project/bacalhau/pull/1793
  • A number of quality of life improvements by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1788
  • publish libp2p resource manager metrics by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1795

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.17...v0.3.18

- Go
Published by wdbaruni over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.17

What's Changed

  • Deploy dashboard by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/1568
  • 1593 responsive dashboard by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/1599
  • bumped canary timeout in getIPFSDownloadSettings from 60 to 300 secs by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1645
  • canary check for number of completed jobs regardless if some nodes fail by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1646
  • Bump prod version from 0.3.15 to 0.3.16 - and increase disks 500GB -> 1TB by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1670
  • 1149 centralized logging loki by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1662
  • Fix typo in Loki/promtail config file by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1696
  • Merge dev and main by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1703
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1704
  • rollback canary to pre-requester client by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1714
  • Update terminal gif by @fakela in https://github.com/filecoin-project/bacalhau/pull/1715
  • rollback canary even further by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1716
  • fix swagger godoc after splitting requester APIs by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1721
  • change check_canary to call prod endpoint by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1722
  • Stop Bacalhau from crashing when Docker times out by @wjam in https://github.com/filecoin-project/bacalhau/pull/1718
  • Avoid timeouts so large they become negative by @wjam in https://github.com/filecoin-project/bacalhau/pull/1719
  • Fix flaky TestDownloaderSuite tests by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1724
  • Allow HTTP job networking using an allowlist by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1679
  • Stop Gateway container from spamming the logs by @wjam in https://github.com/filecoin-project/bacalhau/pull/1720
  • Bump github.com/jedib0t/go-pretty/v6 from 6.4.3 to 6.4.4 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1706
  • Bump golang.org/x/crypto from 0.4.0 to 0.5.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1709
  • Bump modernc.org/sqlite from 1.19.4 to 1.20.2 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1713
  • Bump k8s.io/kubectl from 0.25.4 to 0.26.1 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1717
  • Bump github.com/spf13/viper from 1.14.0 to 1.15.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1728
  • Bump github.com/docker/docker from 20.10.22+incompatible to 20.10.23+incompatible by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1727
  • Allow traffic to IP addresses by @wjam in https://github.com/filecoin-project/bacalhau/pull/1734
  • Add multiple downloader implementations by @pyropy in https://github.com/filecoin-project/bacalhau/pull/1527
  • Initial draft of tool for deriving daily board metrics (#1699) by @ricmk in https://github.com/filecoin-project/bacalhau/pull/1723
  • add node labels and job node selectors by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1737
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1738
  • [circleci] Build jsonschema - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1739
  • publish job events async by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1741

New Contributors

  • @ricmk made their first contribution in https://github.com/filecoin-project/bacalhau/pull/1723

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.16...v0.3.17

- Go
Published by wdbaruni over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.16

What's Changed

  • release v0.3.15 by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1496
  • Bump certifi from 2022.6.15 to 2022.12.7 in /ops by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1478
  • Hide canaries by default from list command by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1500
  • [updatecli] Ops Bacalhau Dependency by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1497
  • remove testground by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1505
  • Support submitting UCAN Invocations as jobs by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1492
  • 658 implement protocol by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1488
  • more graceful DockerIPFS canary by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1553
  • Dashboard merged by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/1567

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.15...v0.3.16

- Go
Published by binocarlos over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.15

What's Changed

  • Terraform + Canary v0.3.14 by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1489
  • [updatecli] Ops Bacalhau Dependency by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1490
  • Add tag inclusion and exclusion list filters by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1493
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1494
  • Address canary timeouts by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1495

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.14...v0.3.15

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.14

What's Changed

  • bumped canary version by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1216
  • Terraform production version bump by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1217
  • Log the correct error variable by @wjam in https://github.com/filecoin-project/bacalhau/pull/1266
  • Enable tests on Linux arm64 machines by @wjam in https://github.com/filecoin-project/bacalhau/pull/1269
  • Avoid common test flake of IPFS failing to start by @wjam in https://github.com/filecoin-project/bacalhau/pull/1273
  • Test Go packages in parallel by @wjam in https://github.com/filecoin-project/bacalhau/pull/1275
  • Un-skip TestRunDeterministicVerifier by @wjam in https://github.com/filecoin-project/bacalhau/pull/1277
  • 1215 add swagger docs generation to ci by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1279
  • Stop test flake when logging after test finished by @wjam in https://github.com/filecoin-project/bacalhau/pull/1282
  • CI for Swagger docs by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1286
  • ci build swagger docs by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1288
  • Clarify timeout error, point user to where they can change it by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1297
  • Build Swagger docs: Automatic commit from CircleCI [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1300
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1301
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1303
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1304
  • Make unit test suite runnable without Docker or internet by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1284
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1306
  • Fix swagger docs title by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1305
  • Fix missing golangci-lint + use Golang 19 in swagger CI by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1307
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1309
  • Make sure HTTP client responses are drained by @wjam in https://github.com/filecoin-project/bacalhau/pull/1308
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1310
  • Avoid attempting to access the network by @wjam in https://github.com/filecoin-project/bacalhau/pull/1312
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1313
  • Websockets by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1283
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1361
  • Update running_locally.md by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1360
  • ci build swagger docs by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1368
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1369
  • Pinned swag version in swagger CI workflow by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1367
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1370
  • ci build swagger docs by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1371
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1372
  • Support trace endpoints other than Honeycomb by @wjam in https://github.com/filecoin-project/bacalhau/pull/1365
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1373
  • Initial trivial simulator framework & "smart contract" mock by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1378
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1380
  • Documentation for the Scenario test framework by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1381
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1382
  • refactor compute into frontend, backend and execution store by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1401
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1407
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1410
  • CI swagger docs: new gh PAT + PR branch name is unique (generated aut… by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1411
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1412
  • Fix typo in comment by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1413
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1419
  • CI swagger docs: new gh PAT + PR branch name is unique (generated aut… by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1420
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1421
  • Fix Python WASM tests by @wjam in https://github.com/filecoin-project/bacalhau/pull/1422
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1424
  • add mutex by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1423
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1426
  • IPFS client should respect timeouts by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1432
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1446
  • deps: updates wazero to 1.0.0-pre.4 by @codefromthecrypt in https://github.com/filecoin-project/bacalhau/pull/1442
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1448
  • Bump github.com/ipfs/kubo from 0.16.0 to 0.17.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1433
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1449
  • Bump go.uber.org/zap from 1.23.0 to 1.24.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1435
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1450
  • Bump github.com/pelletier/go-toml/v2 from 2.0.5 to 2.0.6 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1434
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1451
  • Delete codeql.yml by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1452
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1453
  • Bump github.com/multiformats/go-multiaddr from 0.7.0 to 0.8.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1437
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1454
  • Bump k8s.io/kubectl from 0.25.3 to 0.25.4 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1440
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1455
  • Bump github.com/jedib0t/go-pretty/v6 from 6.4.2 to 6.4.3 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1436
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1456
  • Bump google.golang.org/grpc from 1.50.1 to 1.51.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1441
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1457
  • Bump github.com/ipfs/go-merkledag from 0.8.0 to 0.8.1 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1438
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1458
  • feat(canary): add scenario that leverages IPFS by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1425
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1460
  • Include CID returned from Estuary in job results by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1461
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1462
  • Hide profiling issues unless the user is in debug mode by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1463
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1464
  • Avoid Python install to fix performance job by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1465
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1466
  • fix(canary): increase ipfs job timeout to 10min by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1467
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1468
  • fix(canary): increase download timeout on Bacalhau job by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1469
  • [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1470
  • Build swagger on all commits and only open a PR when there are changes. by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1471
  • fix(canary): add sync command to flush data to outputs directory by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1472
  • fix canary ephemeralStorageSize and increase memory of IPFSJobAndGet by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1476
  • Expose flag for enabling public IPFS node by @pyropy in https://github.com/filecoin-project/bacalhau/pull/1473
  • [DAG] Refactor model.Job and /submit payload + make canary tests work locally by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1430

New Contributors

  • @pyropy made their first contribution in https://github.com/filecoin-project/bacalhau/pull/1473 🎈 Thanks a lot @pyropy, you're awesome 🎈

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.13...v0.3.14

- Go
Published by enricorotundo over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.13

What's Changed

  • [updatecli] Ops Bacalhau Dependency by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1167
  • Aggregate coverage across all test runs by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1168
  • Change zone for European GPU node to europe-west4-a by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1170
  • extract bidding strategies to separate types by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1169
  • Run Linux arm64 builds on a real ARM box by @wjam in https://github.com/filecoin-project/bacalhau/pull/1205
  • when telling the user what command to run, tell them the correct executable on their system by @SvenDowideit in https://github.com/filecoin-project/bacalhau/pull/1171
  • Ensure Docker is running for any test that needs it. by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1208
  • Ensure errors from Estuary publisher are not passed to the user by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1210
  • Test all canary scenarios as part of CI by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1211
  • chore(docs): update to remind people that dev/staging need love too by @philwinder in https://github.com/filecoin-project/bacalhau/pull/1212
  • Only log container start failure when there was actually a failure by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1213
  • Add Swagger/OpenAPI endpoint + docs by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/1214

New Contributors

  • @SvenDowideit made their first contribution in https://github.com/filecoin-project/bacalhau/pull/1171, thank you @SvenDowideit 🎉 !

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.12...v0.3.13

- Go
Published by enricorotundo over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.12

UX and core features

  • Allow WASM jobs to have environment variables by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1026
  • Don't overwrite files with hardlinks when downloading by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/985
  • Support volumes with slashes for WASM jobs by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1027
  • Fix running locally docs to use binary name by @iand in https://github.com/filecoin-project/bacalhau/pull/1038
  • Always use a random port of IPFS API server by @wjam in https://github.com/filecoin-project/bacalhau/pull/1006
  • Put WASM env vars in a consistent order by @wjam in https://github.com/filecoin-project/bacalhau/pull/1051
  • Add job timeout by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1061
  • add --timeout cli option by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1065
  • moving to beta by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1067
  • 1015 fix CVE by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1019
  • Load WASM import modules from IPFS (#880) by @lsp1138 in https://github.com/filecoin-project/bacalhau/pull/1144
  • Revert libp2p withdirectpeers by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1154
  • More robust startup by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1159
  • Pin results to Estuary after publishing to IPFS by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1002

Developer experience

  • Fix canary to look in correct results folder by @simonwo in https://github.com/filecoin-project/bacalhau/pull/987
  • Expose piggybacked publisher information by @wjam in https://github.com/filecoin-project/bacalhau/pull/997
  • Fetch build info from debug.BuildInfo struct by @wjam in https://github.com/filecoin-project/bacalhau/pull/999
  • Ensure temp files are cleaned up by @wjam in https://github.com/filecoin-project/bacalhau/pull/1005
  • Update dependencies and enable dependabot by @wjam in https://github.com/filecoin-project/bacalhau/pull/1028
  • Suppress logging output in tests by @iand in https://github.com/filecoin-project/bacalhau/pull/1039
  • Turn down some log messages that show up on every single test in CI by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1041
  • Make GPU capacitymanager tests work when you have a GPU installed by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1042
  • Remove test as it is a duplicate by @wjam in https://github.com/filecoin-project/bacalhau/pull/1044
  • Tweaks to make it easier to diagnose CI issues by @wjam in https://github.com/filecoin-project/bacalhau/pull/1052
  • First pass at removing Docker from tests which don't test it by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1054
  • Don't run Docker tests on Windows or Mac by @wjam in https://github.com/filecoin-project/bacalhau/pull/1066
  • remove dead function GetSystemDirectory by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/1072
  • Associate the job id with both parts of the trace by @wjam in https://github.com/filecoin-project/bacalhau/pull/1076
  • have CircleCI notice the unittests.xml file by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1078
  • Use Scenario-based tests to remove Docker and streamline test code by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1069
  • Split tests out into unit vs integration, and run each type in parallel by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/1083
  • Clean up a directory related to testing by @wjam in https://github.com/filecoin-project/bacalhau/pull/1128
  • Expose test coverage by @wjam in https://github.com/filecoin-project/bacalhau/pull/1123
  • Use Noop executor in scenario-based tests by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1122
  • Avoid flaky port collisions in tests by assign node ports up front by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1142
  • Build canary against latest code by @simonwo in https://github.com/filecoin-project/bacalhau/pull/1156
  • Improve CI speed by @wjam in https://github.com/filecoin-project/bacalhau/pull/1160
  • Create CNAME by @aronchick in https://github.com/filecoin-project/bacalhau/pull/1162
  • Download Go on Windows directly by @wjam in https://github.com/filecoin-project/bacalhau/pull/1164

Dependency updates

  • [updatecli] Ops Bacalhau Dependency by @philwinder in https://github.com/filecoin-project/bacalhau/pull/984
  • deps: updates wazero to 1.0.0-pre.3 by @codefromthecrypt in https://github.com/filecoin-project/bacalhau/pull/989
  • chore: Update Kubo to v0.16.0 by @Jorropo in https://github.com/filecoin-project/bacalhau/pull/949
  • Update Go version to 1.19 by @wjam in https://github.com/filecoin-project/bacalhau/pull/1014
  • Bump github.com/prometheus/client_golang from 1.13.1 to 1.14.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1049
  • Bump github.com/filecoin-project/go-jsonrpc from 0.1.8 to 0.1.9 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1048
  • Bump github.com/bmatcuk/doublestar/v4 from 4.3.2 to 4.4.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1047
  • Bump github.com/invopop/jsonschema from 0.6.0 to 0.7.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1046
  • Bump golang.org/x/net from 0.1.0 to 0.2.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1055
  • Bump github.com/filecoin-project/go-state-types from 0.9.8 to 0.9.9 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1057
  • Bump golang.org/x/mod from 0.6.0 to 0.7.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1059
  • Bump github.com/ipfs/go-ipfs-files from 0.1.1 to 0.2.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1058
  • Bump github.com/filecoin-project/go-address from 1.0.0 to 1.1.0 by @dependabot in https://github.com/filecoin-project/bacalhau/pull/1056

New Contributors

  • @codefromthecrypt made their first contribution in https://github.com/filecoin-project/bacalhau/pull/989
  • @Jorropo made their first contribution in https://github.com/filecoin-project/bacalhau/pull/949
  • @iand made their first contribution in https://github.com/filecoin-project/bacalhau/pull/1039
  • @lsp1138 made their first contribution in https://github.com/filecoin-project/bacalhau/pull/1144

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.11...v0.3.12

- Go
Published by simonwo over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.11

What's Changed

  • Revert FloodPublish change as now we're seeing high network latencies and flakiness

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.10...v0.3.11

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.10

What's Changed

  • Disable FloodPublish to see if that's what's causing shards to show up as 'Verifying' when they are in fact completed verifying (VerificationResult.Complete = true)

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.9...v0.3.10

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.9

What's Changed

  • add /debug api and periodically log compute node active jobs by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/978
  • add requester node info to /debug api by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/979
  • Allow slashes to be in branch names by @wjam in https://github.com/filecoin-project/bacalhau/pull/980
  • 876 Improvements to downloader by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/953

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.8...v0.3.9

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.8

What's Changed

  • Write all handled event to a JSON log file by @simonwo in https://github.com/filecoin-project/bacalhau/pull/972

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.7...v0.3.8

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.7

What's Changed

  • add info for up to 3 nodes for each stack and bootstrap nodes against… by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/944
  • inter-connect nodes with internal ips by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/954
  • Prefer t.TempDir for temporary directories by @wjam in https://github.com/filecoin-project/bacalhau/pull/960
  • Connect logging when running tests with the tests by @wjam in https://github.com/filecoin-project/bacalhau/pull/964
  • Input arguments and running from an IPFS CID for WASM jobs by @simonwo in https://github.com/filecoin-project/bacalhau/pull/959
  • Connect up the Filecoin publisher by @wjam in https://github.com/filecoin-project/bacalhau/pull/966
  • Fix concurrent map writes in MappedStorageProvider by @wjam in https://github.com/filecoin-project/bacalhau/pull/968
  • reserve 20% of physical resources for the system by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/969
  • Timeout bids by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/970
  • Aggressive reconnect by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/973

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.6...v0.3.7

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.6

What's Changed

  • Populate Estuary API keys when deploying from Terraform by @simonwo in https://github.com/filecoin-project/bacalhau/pull/923
  • Change the Lotus publisher to discover the miner by @wjam in https://github.com/filecoin-project/bacalhau/pull/925
  • Return stdout and stderr from WASM jobs even if they fail. by @simonwo in https://github.com/filecoin-project/bacalhau/pull/930
  • Avoid failing if profiling fails by @wjam in https://github.com/filecoin-project/bacalhau/pull/931
  • Make CLI commands be less nosiy by @simonwo in https://github.com/filecoin-project/bacalhau/pull/932
  • Fix disconnected network by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/938

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.5...v0.3.6

- Go
Published by binocarlos over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.5

What's Changed

  • Validate size limit on all context files, not just the compressed blob by @simonwo in https://github.com/filecoin-project/bacalhau/pull/917
  • Update the Lotus publisher to use the API by @wjam in https://github.com/filecoin-project/bacalhau/pull/912
  • Fix compute nodes stuck at bidding by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/921

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.4...v0.3.5

- Go
Published by philwinder over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.4

What's Changed

  • 889 adding some precommit hooks by @aronchick in https://github.com/filecoin-project/bacalhau/pull/891
  • Avoid reusing cancelled context in cleanup by @wjam in https://github.com/filecoin-project/bacalhau/pull/894
  • 411 parallel flaky url downloads by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/903
  • Add resilience to Estuary publishing by @simonwo in https://github.com/filecoin-project/bacalhau/pull/904
  • Make Language jobs use Estuary publisher by default by @simonwo in https://github.com/filecoin-project/bacalhau/pull/910
  • 899 test for concurrent gpus by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/913

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.3...v0.3.4

- Go
Published by binocarlos over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.3

What's Changed

  • add mutex for the publishers installed cache by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/863
  • 624 publish on error by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/864
  • wait flags by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/869
  • Remove cgo by @simonwo in https://github.com/filecoin-project/bacalhau/pull/870
  • Better environment checking by @aronchick in https://github.com/filecoin-project/bacalhau/pull/877
  • Use t.Cleanup rather than defer TeardownTest by @wjam in https://github.com/filecoin-project/bacalhau/pull/882
  • Clean up usage of Name fields in job spec by @simonwo in https://github.com/filecoin-project/bacalhau/pull/884
  • Support inputs and outputs to WASM jobs that use WASI by @simonwo in https://github.com/filecoin-project/bacalhau/pull/874
  • Add an option to start Lotus with the devstack by @wjam in https://github.com/filecoin-project/bacalhau/pull/879
  • Use Docker API to pull images by @wjam in https://github.com/filecoin-project/bacalhau/pull/892
  • Typography and documentation fixes by @joshklop in https://github.com/filecoin-project/bacalhau/pull/854

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.2...v0.3.3

- Go
Published by simonwo over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.2

What's Changed

  • 855 fix mounting urls by @philwinder in https://github.com/filecoin-project/bacalhau/pull/857
  • [updatecli] Ops Bacalhau Dependency by @philwinder in https://github.com/filecoin-project/bacalhau/pull/853

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.1...v0.3.2

- Go
Published by philwinder over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.1

What's Changed

  • Build statically on Linux to avoid libc versioning issues. by @simonwo in https://github.com/filecoin-project/bacalhau/pull/852

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.3.0...v0.3.1

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.3.0

What's Changed

  • fix --id-only --wait by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/849

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.9...v0.3.0

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.2.9

  • fix nil pointer panic on bacalhau docker run in the CLI

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.8...v0.2.9

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.2.8

What's Changed

  • Native execution of very simple WASM jobs by @simonwo in https://github.com/filecoin-project/bacalhau/pull/816
  • 707 More CLI Cleanup by @aronchick in https://github.com/filecoin-project/bacalhau/pull/846

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.7...v0.2.8

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.2.7

What's Changed

  • Ensure download timeouts are consistent across commands. by @simonwo in https://github.com/filecoin-project/bacalhau/pull/799
  • 610 multiple cid mounts by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/805
  • 586 multiple urls by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/806
  • add test that combines an IPFS input with a URL by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/807
  • Downloads should not fail if stdout or stderr are not present (#803) by @simonwo in https://github.com/filecoin-project/bacalhau/pull/808
  • Fix waitUntilCompleted to no longer check for cancelled state by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/809
  • graceful cancel jobs before a bid is sent by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/815
  • Build Bacalhau under CGO by @simonwo in https://github.com/filecoin-project/bacalhau/pull/814
  • 782 Progressive Display for CLI by @aronchick in https://github.com/filecoin-project/bacalhau/pull/817

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.6...v0.2.7

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.2.6

What's Changed

  • Phil circle rename by @philwinder in https://github.com/filecoin-project/bacalhau/pull/680
  • #673 Native Windows support by @simonwo in https://github.com/filecoin-project/bacalhau/pull/681
  • [updatecli] Ops Bacalhau Dependency by @philwinder in https://github.com/filecoin-project/bacalhau/pull/687
  • chore(ci): ci to check terraform bacalhau version is now working on e… by @philwinder in https://github.com/filecoin-project/bacalhau/pull/688
  • replace controller with separate event handlers by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/689
  • 679 Better Marshaling of Enum Types by @aronchick in https://github.com/filecoin-project/bacalhau/pull/698
  • Whole bunch of UX Fixes by @aronchick in https://github.com/filecoin-project/bacalhau/pull/763
  • Improve error passing between server and client by @aronchick in https://github.com/filecoin-project/bacalhau/pull/767
  • introduce requesterNode state machine, and dependency providers by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/768
  • mount disk on reboot as well as initial install/upgrade by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/772
  • add API access logs and timeouts by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/785
  • Add a newline after bacalhau describe <id> by @wjam in https://github.com/filecoin-project/bacalhau/pull/790
  • [updatecli] Ops Bacalhau Dependency by @philwinder in https://github.com/filecoin-project/bacalhau/pull/789
  • revert canary to v0.2.4 by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/792
  • add event handler logs by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/793
  • fix in-memory db deadlock, and add NodeID to logging context by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/796

New Contributors

  • @simonwo made their first contribution in https://github.com/filecoin-project/bacalhau/pull/681 🔥 🔥 🔥
  • @wjam made their first contribution in https://github.com/filecoin-project/bacalhau/pull/790 🔥 🔥 🔥

Welcome to the project!

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.5...v0.2.6

- Go
Published by enricorotundo over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.2.5

What's Changed

  • bacalhau validate by @js-ts in https://github.com/filecoin-project/bacalhau/pull/657
  • 602 Implement DryRun, fix Describe and Improve List Functionality by @aronchick in https://github.com/filecoin-project/bacalhau/pull/678
  • bacalhau output jobspec, create template by @js-ts in https://github.com/filecoin-project/bacalhau/pull/602
  • 611 Make List command work well/faster by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/675
  • Bump protobuf from 4.21.5 to 4.21.6 in /ops by @dependabot in https://github.com/filecoin-project/bacalhau/pull/669

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.4...v0.2.5

- Go
Published by enricorotundo over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.2.4

What's Changed

  • Rename state of finished job by @enricorotundo in https://github.com/filecoin-project/bacalhau/pull/587
  • Bacalhau monitoring canary by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/612
  • Simple fixes to the "Describe" verb by @aronchick in https://github.com/filecoin-project/bacalhau/pull/638
  • correcting some spans and variable names by @aronchick in https://github.com/filecoin-project/bacalhau/pull/639
  • Bump github.com/moby/moby from 20.10.17+incompatible to 20.10.18+incompatible by @dependabot in https://github.com/filecoin-project/bacalhau/pull/633
  • finished refactoring from err to RunOutput by @aronchick in https://github.com/filecoin-project/bacalhau/pull/601
  • Add RunOutput to describe cmd by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/648
  • 636 id command by @binocarlos in https://github.com/filecoin-project/bacalhau/pull/649

New Contributors

  • @dependabot made their first contribution in https://github.com/filecoin-project/bacalhau/pull/633

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.3...v0.2.4

- Go
Published by philwinder over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.2.3

What's Changed

  • testground initial onboarding by @wdbaruni in https://github.com/filecoin-project/bacalhau/pull/583
  • Fix estuary get by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/584

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.2...v0.2.3

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.2.2

What's Changed

  • 546 adding otel for lots more (all client and some server) by @aronchick in https://github.com/filecoin-project/bacalhau/pull/571
  • Rate limiting by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/552
  • Min bids by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/573
  • Estuary publisher by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/574
  • switch the default publisher to use estuary by @lukemarsden in https://github.com/filecoin-project/bacalhau/pull/575

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.1...v0.2.2

- Go
Published by lukemarsden over 3 years ago

https://github.com/bacalhau-project/bacalhau - v0.2.1

Try to fix release being broken because some random github employee's GPG key expired.

Full Changelog: https://github.com/filecoin-project/bacalhau/compare/v0.2.0...v0.2.1

- Go
Published by lukemarsden over 3 years ago