spark-submit - spark-submit 1.4.0

Features: - Report if framework not detected by spark_submit.system_info

Bug Fixes - Fix quotes in spark-sumbit binary path in spark_submit.system_info

Misc/Internal - Better way to construct the spark-submit command

- Python
Published by PApostol almost 3 years ago

spark-submit - spark-submit 1.3.0

Features: - Addition of spark_submit.SparkJob.get_id() method to obtain the Spark submission ID - Use all environment variables from os.environ if use_env_vars=True in spark_submit.SparkJob.submit() - Additional environment variables can be used in spark_submit.SparkJob.submit() with extra_env_vars argument

Misc/Internal - spark_submit.SparkJob.submit() now uses keyword arguments only - Argument await_result is changed to poll_time in spark_submit.SparkJob.submit() - No default environment variables to be used with spark_submit.SparkJob.submit() - Code standards improvements and general refactoring

- Python
Published by PApostol over 3 years ago

spark-submit - spark-submit 1.2.1

Misc/Internal - Refactor to comply with pylint, isort and mypy checks - Minor logic improvements

- Python
Published by PApostol about 4 years ago

spark-submit - spark-submit 1.2.0

Features - Added optional timeout argument for spark_submit.SparkJob.submit()

Improvements - Improve support for Yarn and Kubernetes clusters on polling and killing submitted tasks

Misc/Internal - Use dedicated file for default arguments and Spark states - Use consistent path separators where possible

- Python
Published by PApostol about 4 years ago

spark-submit - spark-submit 1.1.0

Features - Addition of spark_submit.SparkJob.get_submit_cmd() method - Introduction of spark_submit.SparkJob.concluded attribute to keep track of submitted job

Improvements - Improve spark_submit.system_info() functionality - More generic status polling - Minor refactoring

- Python
Published by PApostol over 4 years ago

spark-submit - spark-submit 1.0.1

Bug Fixes - Fix return code bug in spark_submit.SparkJob.get_code()

Improvements - Change argument order in spark_submit.SparkJob.submit() - Minor improvements

- Python
Published by PApostol over 4 years ago

spark-submit - spark-submit 1.0.0

First release

- Python
Published by PApostol over 4 years ago