STEMIA

Scripts and Tools for Electron Microscopy Image Analysis.

This is a simple personal collection of (sometimes...) useful scripts and tools for cryoem/cryoet.

Installation

bash pip install stemia

You can quickly list all the available tools with

stemia -l

Completion

You can enable completion for your bash shell by running:

eval "$(_STEMIA_COMPLETE=bash_source stemia)"

See the click docs for how to do it for other shells.

Tools

Everything is accessible through the main command line interface stemia.

Try stemia -h for help, or stemia -l for the command tree:

.stemia ├── aretomo: A collection of AreTomo-related tools and scripts. │ ├── aln2xf: Convert AreTomo `aln` file to imod `xf` format. │ └── batch: Run AreTomo on a full directory. ├── cryosparc: A collection of Cryosparc-related tools and scripts. │ ├── csplot: Read a cryosparc job directory and plot interactively any column. │ ├── fix_filament_ids: Replace cryosparc filament ids with small unique integers. │ ├── generate_tilt_angles: Generate angle priors for a tilted dataset. │ ├── merge_defects_gainref: Merge serialEM defects and gainref for cryosparc usage. │ └── time_wasted: Print the total amount of time wasted on a project. ├── image: Simple image manipulation and processing. │ ├── center_filament: Center an mrc image (stack) containing filament(s). │ ├── classify_densities: Do hierarchical classification of particle stacks based on densities. │ ├── create_mask: Create a mask for INPUT. │ ├── extract_z_snapshots: Grab z slices at regular intervals from a tomogram as jpg images. │ ├── flip_z: Flip the z axis for particles in a RELION star file. │ ├── fourier_crop: Bin mrc images to the specified pixel size using fourier cropping. │ ├── project_profiles: Project re-extracted and straightened membranes and get some stats. │ │ ├── prepare: Generate and select 2D chunked projections for the input data. │ │ ├── compute: Take the outputs from prepare and compute statistics and plots. │ │ └── aggregate: Aggregate the generated data into general stats about given subsets. │ └── rescale: Rescale an mrc image to the specified pixel size. ├── imod: A collection of IMOD-related tools and scripts. │ └── find_NAD_params: Test a range of k and iteration values for nad_eed_3d. ├── relion: A collection of Relion-related tools and scripts. │ ├── align_filament_particles: Fix filament PsiPriors so they are consistent within a filament. │ └── edit_star: Simple search-replace utility for star files. └── warp: A collection of Warp-related tools and scripts. ├── fix_mdoc: Fix mdoc files to point to the right data and follow warp format. ├── offset_angle: Offset tilt angles in warp xml files. ├── parse_xml: Parse a warp xml file and print its content. ├── prepare_isonet: Update an isonet starfile with preprocessing data from warp. ├── spoof_mdoc: Create dummy mdocs for warp. ├── summarize: Summarize the state of a Warp project. └── preprocess_serialem: Prepare and unpack data from sterialEM for Warp.

stemia aretomo aln2xf

``` Usage: stemia aretomo aln2xf [OPTIONS] ALN_FILE

Convert AreTomo aln file to imod xf format.

Options: -f, --overwrite overwrite existing output --help Show this message and exit. ```

stemia aretomo batch

``` Usage: stemia aretomo batch [OPTIONS]

Run AreTomo on a full directory.

Options: --help Show this message and exit. ```

stemia cryosparc csplot

``` Usage: stemia cryosparc csplot [OPTIONS] JOB_DIR

Read a cryosparc job directory and plot interactively any column.

All the related data from parent jobs will also be loaded. An interactive ipython shell will be opened with data loaded into a pandas dataframe.

JOB_DIR: a cryosparc job directory.

Options: --drop-na drop rows that contain NaN values (e.g: micrographs with no particles) --no-particles do not read particles data --no-micrographs do not read micrographs data --help Show this message and exit. ```

stemia cryosparc fixfilamentids

``` Usage: stemia cryosparc fixfilamentids [OPTIONS] STAR_FILE

Replace cryosparc filament ids with small unique integers.

Relion will fail with cryosparc IDs because of overflows.

Options: -o, --star-output FILE where to put the updated version of the star file [default: fixedid.star] -f, --overwrite overwrite output if exists --help Show this message and exit. ```

stemia cryosparc generatetiltangles

``` Usage: stemia cryosparc generatetiltangles [OPTIONS] STARFILE TILTANGLE TILT_AXIS

Generate angle priors for a tilted dataset.

Read a Relion STARFILE with in-plane angles and generate priors for rot and tilt angles based on a TILTANGLE around a TILT_AXIS.

Options: -r, --radians Provide angles in radians instead of degrees -o, --star-output FILE where to put the updated version of the star file [default: _tilted.star] -f, --overwrite overwrite output if exists --help Show this message and exit. ```

stemia cryosparc mergedefectsgainref

``` Usage: stemia cryosparc mergedefectsgainref [OPTIONS] DEFECTS GAINREF

Merge serialEM defects and gainref for cryosparc usage.

requires active sbrgrid.

Options: -d, --output-defects FILE -o, --output-gainref FILE -f, --overwrite overwrite output if exists --help Show this message and exit. ```

stemia cryosparc time_wasted

``` Usage: stemia cryosparc time_wasted [OPTIONS] [PROJECT_DIRS]...

Print the total amount of time wasted on a project.

Options: -u, --useful_jobs TEXT ID of job that gave useful results. Its running time and that of its parents will be used to calculate useful time. Can be passed multiple times. --help Show this message and exit. ```

stemia image center_filament

``` Usage: stemia image center_filament [OPTIONS] INPUT [OUTPUT]

Center an mrc image (stack) containing filament(s).

Can update particles in a RELION .star file accordingly. If OUTPUT is not given, default to INPUT_centered.mrc

Options: -s, --update-star FILE a RELION .star file to update with new particle positions -o, --star-output FILE where to put the updated version of the star file. Only used if -s is passed [default: STARFILE_centered.star] --update-by [class|particle] whether to update particle positions by classes or 1 by 1. Only used if -s is passed [default: class] -f, --overwrite overwrite output if exists -n, --n-filaments INTEGER number of filaments on the image [default: 2] -p, --percentile INTEGER percentile for binarisation [default: 85] --help Show this message and exit. ```

stemia image classify_densities

``` Usage: stemia image classify_densities [OPTIONS] [STACKS]...

Do hierarchical classification of particle stacks based on densities.

Options: -c, --max-classes INTEGER --help Show this message and exit. ```

stemia image create_mask

``` Usage: stemia image create_mask [OPTIONS] INPUT OUTPUT

Create a mask for INPUT.

Axis order is zyx!

Options: -t, --mask-type [sphere|cylinder|threshold] -c, --center TEXT center of the mask (comma-separated floats) -a, --axis INTEGER main symmetry axis (for cylinder) -r, --radius FLOAT radius of the mask. If thresholding, equivalent to "hard padding" [required] -i, --inner-radius FLOAT inner radius of the mask (if any) -p, --padding FLOAT smooth padding --ang / --px whether the radius and padding are in angstrom or pixels --threshold FLOAT threshold for binarization of the input map -f, --overwrite overwrite output if exists --help Show this message and exit. ```

stemia image extractzsnapshots

``` Usage: stemia image extractzsnapshots [OPTIONS] [INPUTS]...

Grab z slices at regular intervals from a tomogram as jpg images.

INPUTS: any number of paths of volume images

Options: -o, --output-dir PATH --mrc also output mrc files -n, --n-slices INTEGER number of equidistant slices to extract --keep-extrema whether to keep slices at z=0 and z=-1 (if false, slices is reduced by 2) -a, --average INTEGER number of slices to average over -s, --size TEXT size of final image (X,Y) -r, --range TEXT range of slices to image (A,B) --axis INTEGER axis along which to do the slicing --help Show this message and exit. ```

stemia image flip_z

``` Usage: stemia image flipz [OPTIONS] STARPATH

Flip the z axis for particles in a RELION star file.

STAR_PATH: star file to flip along z

Assumes all tomograms have the same shape.

Options: -o, --output FILE -m, --mrcpath FILE --starpixelsize FLOAT --mrcpixelsize FLOAT --zshape INTEGER --help Show this message and exit. ```

stemia image fourier_crop

``` Usage: stemia image fourier_crop [OPTIONS] [INPUTS]...

Bin mrc images to the specified pixel size using fourier cropping.

Options: -b, --binning FLOAT binning amount [required] -f, --overwrite overwrite output if exists --help Show this message and exit. ```

stemia image project_profiles prepare

``` Usage: stemia image project_profiles prepare [OPTIONS] [PATHS]...

Generate and select 2D chunked projections for the input data.

Options: -o, --output PATH [required] -s, --chunk-size INTEGER -f, --overwrite --help Show this message and exit. ```

stemia image project_profiles compute

``` Usage: stemia image projectprofiles compute [OPTIONS] PROJDIR

Take the outputs from prepare and compute statistics and plots.

Options: -f, --overwrite --help Show this message and exit. ```

stemia image project_profiles aggregate

``` Usage: stemia image project_profiles aggregate [OPTIONS] [INPUTS]...

Aggregate the generated data into general stats about given subsets.

Inputs are subdirectories of the project_dir from compute.

Options: -o, --output-name TEXT Title/filename given to the aggregated outputs. --help Show this message and exit. ```

stemia image rescale

``` Usage: stemia image rescale [OPTIONS] INPUT OUTPUT TARGETPIXELSIZE

Rescale an mrc image to the specified pixel size.

TARGETPIXELSIZE: target pixel size in Angstrom

Options: --input-pixel-size FLOAT force input pizel size and ignore mrc header -f, --overwrite overwrite output if exists --help Show this message and exit. ```

stemia imod findNADparams

``` Usage: stemia imod findNADparams [OPTIONS] INPUT

Test a range of k and iteration values for nadeed3d.

Options: -k, --k-values TEXT -i, --iterations TEXT -s, --std TEXT --help Show this message and exit. ```

stemia relion alignfilamentparticles

``` Usage: stemia relion alignfilamentparticles [OPTIONS] STAR_FILE

Fix filament PsiPriors so they are consistent within a filament.

Read a Relion STAR_FILE with in-plane angles and filament info and flip any particle that's not consistent with the rest of the filament.

If a consensus cannot be reached, or the filament has too few particles, discard the whole filament.

Options: -o, --star-output FILE where to put the updated version of the star file [default: _aligned.star] -t, --tolerance FLOAT angle in degrees within which neighbouring particles are considered aligned -c, --consensus-threshold FLOAT require an angle consensus at least higher than this to use a filament. -d, --drop-below INTEGER drop filaments if they have fewer than this number of particles -r, --rotate-bad-particles rotate bad particles to match the rest of the filament -f, --overwrite overwrite output if exists --help Show this message and exit. ```

stemia relion edit_star

``` Usage: stemia relion edit_star [OPTIONS] [STAR_FILES]...

Simple search-replace utility for star files.

Full regex functionality works (e.g: reusing groups in output)

Options: -s, --suffix-output TEXT suffix added to the output files before extension -c, --column TEXT column(s) to modify -i, --regex-in TEXT regex sed-like search pattern(s) -o, --regex-out TEXT regex sed-like substitution to apply to the column(s) -f, --overwrite overwrite output if exists --help Show this message and exit. ```

stemia warp fix_mdoc

``` Usage: stemia warp fixmdoc [OPTIONS] MDOCDIR

Fix mdoc files to point to the right data and follow warp format.

Options: -d, --data-dir PATH --dates fix date format --paths fix image paths --help Show this message and exit. ```

stemia warp offset_angle

``` Usage: stemia warp offset_angle [OPTIONS] [WARP_DIR]

Offset tilt angles in warp xml files.

Options: --help Show this message and exit. ```

stemia warp parse_xml

``` Usage: stemia warp parsexml [OPTIONS] XMLFILE

Parse a warp xml file and print its content.

Options: --help Show this message and exit. ```

stemia warp prepare_isonet

``` Usage: stemia warp prepareisonet [OPTIONS] WARPDIR ISO_STAR

Update an isonet starfile with preprocessing data from warp.

Options: --help Show this message and exit. ```

stemia warp spoof_mdoc

``` Usage: stemia warp spoof_mdoc [OPTIONS] [RAWTLT_FILES]...

Create dummy mdocs for warp.

RAWTLT_FILES: simple file with one tilt angle per line. Order should match sorted filenames.

Options: -d, --dose-per-image FLOAT electron dose per tilt image (or per frame if inputs are movies) [required] -p, --pixel-size FLOAT -e, --extension [tif|mrc] -f, --overwrite --help Show this message and exit. ```

stemia warp summarize

``` Usage: stemia warp summarize [OPTIONS] [WARP_DIR]

Summarize the state of a Warp project.

Reports for each tilt series: - discarded: number of discarded tilts - total: total number oftilts in raw data - stacked: number of image slices in imod output directory - mismatch: whether stacked != (total - discarded) - resolution: estimated resolution if processed

Options: --help Show this message and exit. ```

stemia warp preprocess_serialem

``` Usage: stemia warp preprocessserialem [OPTIONS] RAWDATA_DIR

Prepare and unpack data from sterialEM for Warp.

You must be in a new directory for this to work; new files will be placed there with the same name as the original tifs.

RAWDATADIR: the directory containing the raw data

Options: --help Show this message and exit. ```