Releases | Open Source Science

pypickle - 2.0.1

Some security checks, such as builtinsor os, can introduce many false positive detections. I fixed this by making some modules more specific.

Checks of builtins, os , sys and subprocess modules are more specific.

- Python
Published by erdogant about 1 year ago

This is a large update where many security mechanisms are incorporated in the loading and saving of pickle files. Updating to this version can break your pipeline! It is recommended to first look at the changes in saving and loading over here and here. Overall, save and load still work in the same manner, but with restrictions.

The new updates are as following: Save * Home and Temp directories are safe locations to save your pickle file and are allowed by default * To save outside the home and temp directory, you need to set the parameter: allow_external=True * It is not allowed to save in system-critical paths. See here for more information. * The extension of the pickle file must be one of these: .pkl, .pickle, .pklz, .pbz2 * Path traversal detection is implemented to detect and block path traversal attempts

Load * Loading pickle files now contains a validation step that checks for high-risk modules before loading. See here for the list. * To load risky modules, it must be explicitly added via the validate parameter or disable the validation using validate=False (not recommended). * Before loading pickle files, the modules stored in the pickle file can be checked using the validate_modules function without loading them. See here for more details.

General fixes * More logging * Code refactoring * Code cleaning * Updating docstrings * This update will solve issues #2 #3

- Python
Published by erdogant about 1 year ago