Supported by the HHMI Janelia Open Science Software Initative
This package allows you to run compute-intense parts of BigStitcher distributed on your workstation, a cluster or the cloud using Apache Spark. The following modules are currently available in BigStitcher-Spark listed as JavaClassName/cmd-line-tool-name (you can find documentation below, but a good start is also to just check out the cmd-line arguments, they mostly follow the BigStitcher GUI; each module takes an existing XML):
SparkResaveN5/resave(resave an XML dataset you defined in BigStitcher - use virtual loading only - into N5 for processing)SparkInterestPointDetection/detect-interestpoints(detect interest points for alignment, Difference-of-Gaussian or scale space)FilterInterestPoints/filter-interestpoints(filtered labels of detected interest points: by response, intensity, sigma or a fixed number per view, without detecting again)SparkInterestPointEmbedding/embed-interestpoints(learned embeddings of the interest points of a label, pcmatch's ONNX model evaluated in Java, formatch-interestpoints -m EMBEDDINGS)SparkGeometricDescriptorMatching/match-interestpoints(perform pair-wise interest point matching)SparkPairwiseStitching/stitching(run pairwise stitching between overlapping tiles)Solver/solver(perform the global solve, works with interest points and stitching)CreateFusionContainer/create-fusion-container(set up the container into which the affine fusion will write it's data, now supports OME-ZARR)SparkAffineFusion/affine-fusion(fuse the aligned dataset using interpolated affine/rigid/translation models)SparkNonRigidFusion/nonrigid-fusion(fuse the aligned dataset using non-rigid models)
Additonally there are some utility methods:
SparkDownsample/downsample(perform downsampling of existing volumes)ClearInterestPoints/clear-interestpoints(clears or repairs interest points and correspondences, for all or selected views)ClearRegistrations/clear-registrations(clears registrations, for all or selected views)
Note: BigStitcher-Spark is designed to work hand-in-hand with BigStitcher. You can always verify the results of each step BigStitcher-Spark step interactively using BigStitcher by simply opening the XML. You can of course also run certain steps in BigStitcher, and others in BigStitcher-Spark. Not all functionality is 100% identical between BigStitcher and BigStitcher-Spark; important differences in terms of capabilities is described in the respective module documentation below (typically BigStitcher-Spark supports a specific feature that was hard to implement in BigStitcher and vice-versa).
- Prerequisite: Java 21 (17 also works; Spark 4 dropped Java 8 and 11), e.g. Zulu or Temurin. Check with
java -version.
Install Script Download and run the install-release script. It fetches the latest release and creates one executable per module (resave, detect-interestpoints, ..., plus bigstitcher-spark for everything), preset with the given number of cores and memory in GB, without cloning or building anything:
curl -fsSLO https://raw.githubusercontent.com/JaneliaSciComp/BigStitcher-Spark/main/install-release
./install-release # options: -t threads, -m GB, -v <version>, -d <dir>, -f
then run for example ./resave -x ~/dataset.xml.
Manually instead: download BigStitcher-Spark-<version>-local.jar from the latest release (Spark is bundled, nothing else to install; a stable link to the current version is https://github.com/JaneliaSciComp/BigStitcher-Spark/releases/latest/download/BigStitcher-Spark-local.jar) and run any module as a subcommand:
java -Xmx64g -jar BigStitcher-Spark-local.jar resave -x ~/dataset.xml
java -Xmx64g -jar BigStitcher-Spark-local.jar --help # lists all commands
-Xmx is the memory available to the local Spark. By default all cores are used; limit them with -Dspark.master='local[8]' (quote it, the brackets are shell globs). No further JVM flags are required, the jar's manifest carries the module-access settings Spark 4 needs on Java 17/21.
Building from source instead (developers): install Apache Maven (mvn -v; JAVA_HOME must point at the JDK), clone the repo, cd BigStitcher-Spark and run ./install -t <num-cores> -m <mem-in-GB>. This builds the project and creates one executable per module in the working directory (resave, detect-interestpoints, filter-interestpoints, embed-interestpoints, match-interestpoints, stitching, solver, match-intensities, solve-intensities, create-fusion-container, fusion, nonrigid-fusion, split-images, filter-views, downsample, clear-interestpoints, clear-registrations, transform-points, overlay-landmarks, define-dataset, apply-transformation, screenshot), each preset with the given cores and memory. The executables contain absolute paths into your local Maven repository, so they only work on the machine that built them.
If you run the code directly from your IDE, you will need to add JVM parameters for the local Spark execution (e.g. 8 cores, 50GB RAM), plus the --add-opens flags listed in the install script:
-Dspark.master=local[8] -Xmx50G
Use BigStitcher-Spark-<version>-cluster.jar from the latest release (Spark is not bundled, the cluster's Spark 4.1 / Java 21 provides it), or build it yourself with mvn clean package -Pfatjar (target/BigStitcher-Spark-<version>-cluster.jar). Submit either with spark-submit --class net.preibisch.bigstitcher.spark.<JavaClassName> ... BigStitcher-Spark-<version>-cluster.jar <arguments> or, using the jar's entry point, spark-submit ... BigStitcher-Spark-<version>-cluster.jar <cmd-line-tool-name> <arguments>.
Important: if you use HDF5 as input data in a distributed scenario, you need to set a common path for extracting the HDF5 binaries (see solved issue here), e.g.
--conf spark.executor.extraJavaOptions=-Dnative.libpath.jhdf5=/groups/spruston/home/moharb/libjhdf5.so
Please ask your sysadmin for help how to run it on your cluster, below are hopefully helpful tutorials for different kinds of clusters. They can be helpful to transfer the knowledge to your home institution.
- HHMI Janelia (LSF cluster): Tutorial on YouTube by @trautmane
- MDC Berlin (SGE cluster): Google doc explaining the steps by @bellonet
- We are currently developing a generic NextFlow-based pipeline for submitting BigStitcher-Spark jobs to a cluster
Use the same BigStitcher-Spark-<version>-cluster.jar as for clusters (see above).
BigStitcher-Spark is now fully "cloud-native". For running the fatjar on the cloud check out services such as Amazon EMR and Google Serverless Batches. Note that some modules support prefetching --prefetch, which is important for cloud execution due to its delays as it pre-loads all image blocks in parallel before processing. We will soon add detailled information on how to run the examples on both cloud platforms (it works - if you need help now, please contact @StephanPreibisch).
We provide two example datasets (one for interest-point based registration, one that works well with Stitching), which are available for download several times with increasing level of reconstruction so you can test different modules of BigStitcher-Spark directly. The datasets are again linked throughout the documentation for the individual modules. If you would like to test the entire pipeline we suggest to start with RAW datasets and run the entire pipeline. Here is an overview of the two datasets at different stages:
-
Dataset for Stitching:
- As TIFF (unaligned, no BigStitcher project defined)
- As TIFF/XML (unaligned)
- As N5/XML (unaligned)
- As N5/XML containing pairwise stitching results (unaligned)
- As N5/XML (aligned)
-
Dataset for Interest Points:
For this tutorial I extracted the Stitching dataset into ~/SparkTest/Stitching and the dataset for experimenting with interest points into ~/SparkTest/IP. There are two cloud-based copies (read only) of the Stitching dataset available on AWS S3 (s3://janelia-bigstitcher-spark/Stitching/dataset.xml) and Google Storage (gs://janelia-spark-test/I2K-test/dataset.xml).
When working with BigStitcher the first step is to define a dataset, where the goal is to provide sufficient meta-data in order to allow BigStitcher (and BigDataViewer) to load your images. This step is typically done in the BigStitcher GUI, but some people have written scripts to automatically generate XML's for their datasets (e.g. here). If you want to start testing an entire BigStitcher(-Spark) pipeline from scratch, please use this dataset for stitching or this one using interest points.
After the dataset is defined one usually re-saved the input data (TIFF, CZI, ...) into a multi-resolution format that makes it possible to interactively display and work with the data the image as various resolution levels, and is essential for distributed processing. Right now, we use the N5 format for temporary storage of the input data. This resaving process can take substantial amounts of time if your input is large and can be distributed using Spark. Importantly, you need to define your dataset using the Automatic Loader (Bioformats based) and select to Load data directly and load data virtually.
For testing the re-saving (and multi-resolution pyramid creation) with Spark use your defined dataset(s), or download this dataset for stitching or this dataset for interest points
The command for resaving the stitching dataset could look like this and will overwrite the input XML, a backup XML will be automatically created:
./resave -x ~/SparkTest/Stitching/dataset.xml -xo ~/SparkTest/Stitching/dataset.xml
It is analogous for the interest point dataset:
./resave -x ~/SparkTest/IP/dataset.xml -xo ~/SparkTest/IP/dataset.xml
Please run resave without parameters to get help for all command line arguments. Using --blockSize you can change the blocksize of the N5, and --blockScale defines how many blocks at once will be processed by a Spark job. With -ds you can define your own downsampling steps if the automatic ones are not well suited. Every Spark task loads its compute block (--blockSize x --blockScale) into memory before writing it, which needs that much memory per task; --no-preloadComputeBlocks streams the blocks instead, which is very slow for the virtual Bio-Formats loader (every plane is re-read for every block). Datasets of the views that already exist in the container, e.g. from an interrupted run, stop the resaving; --overwriteExistingDatasets deletes those views' datasets first and keeps the others.
Note: --dryRun allows the user to test the functionality without writing any data. The Spark implementation parallelizes over user-defined blocks across all input images at once, so also few, very big images will be processed efficiently.
To perform classical stitching (translation only), first pair-wise stitching between all overlapping tiles needs to be computed. So far we only support standard grouping where all channels and illuminations of a specific Tile will be grouped together as one image and stitching is performed individually per Timepoint and Angle. To run the stitching with default paramters you can run the following command on this example dataset:
./stitching -x ~/SparkTest/Stitching/dataset.xml
The results will be written in to the XML, in order to compute transformation models and apply them to the images you need to run the solver that computes a global optimization next.
Please run stitching without parameters to get help for all command line arguments. -ds sets the resolution at which cross correlation is performed; 2,2,1 is the default and usually superior to 1,1,1 due to suppressed noise, even higher resolution levels typically work well too since by default the peaks are located with subpixel accuracy. --disableSubpixelResolution disables subpixel accurate shifts. -p sets the number of phase correlation peaks that are checked with cross-correlation (incrementing this number can help with stitching issues). --minR and --maxR are filters that specify the accepted range for cross correlation for any pair of overlapping tiles, reducing --minR may be useful to accept more pair and excluding a --maxR of 1 may be useful too if you get wrong links with r=1.0. --maxShiftX/Y/Z and --maxShiftTotal set the maximal allowed shift between any pair of images relative to their current location; limiting it if the current position is close to the correct one might be useful. If your dataset contains multiple channels or illumination directions per Tile, you can select how they will be combined for the pair-wise stitching process using --channelCombine and --illumCombine, which can be either AVERAGE or PICK_BRIGHTEST.
You can choose which Tiles --tileId, Channels --channelId, Iluminations --illuminationId, Angles --angleId and Timepoints --timepointId will be processed, a typical choice could be --timepointId 18 --tileId 0,1,2,3,6,7,8 to only process the timepoint 18 and select Tiles. If you would like to choose Views more fine-grained, you can specify their ViewIds directly, e.g. -vi '0,0' -vi '0,1' -vi '1,1' to process ViewId 0 & 1 of Timepoint 0 and ViewId 1 of Timepoint 1. By default, everything will be processed.
Note: --dryRun allows the user to test the functionality without writing any data. The Spark implementation parallelizes over pairs of images.
Interest-point based registration is generally more reliable and faster than stitching while supporting various transformation models including (regularized) Translation, (regularized) Rigid, (regularized) Affine, and Non-Rigid. At the same time parameter selection is more involved. The first step is to detect interest points in the images. A typical command line call that works well on this example dataset looks as follows:
./detect-interestpoints -x ~/SparkTest/IP/dataset.xml -l beads -s 1.8 -t 0.008 -dsxy 2 --minIntensity 0 --maxIntensity 255
The results will be written in the XML and the interest point store interestpoints.zarr next to it, in order to compute transformation models and apply them to the images you need to run matching followed by the solver that computes a global optimization. If you want to inspect the interest points you can open the XML in BigStitcher, go to multiview mode, right click, start the Interest Point Explorer and click on the number of detections that will then be overlaid onto the images (see screenshot below).
Interest points of projects written before October 2026 live in a per-view interestpoints.n5; they are read as they are and can be moved into the store once with java -cp <the fat jar> net.preibisch.mvrecon.fiji.spimdata.interestpoints.InterestPointsN5ToZarr dataset.xml (one group per label and timepoint instead of thousands of small files; nothing changes in the XML).
Please run detect-interestpoints without parameters to get help for all command line arguments. -s and -t define the sigma and threshold of the Difference-of-Gaussian, respectively and -l specifies the label for the interest points. --minIntensity and --maxIntensity set the intensity range in which all processed blocks are normalized to [0...1]; these values are mandatory since each individual Spark job is unable to figure out correct min/max values of the images. You could find good guesses for all these values by starting the interactive interest point detection in BigStitcher. -dsxy and -dsz define the downsampling at which interest point detection is performed. Using --localization you can specify the type of subpixel localization, either NONE or QUADRATIC. --type allows to set which type of intensity peaks should be identified; MIN, MAX or BOTH. Finally, --blockSize sets the blocksize that will be processed in each Spark job.
--overlappingOnly only identifies interest points in the regions of each image that overlap other selected images by the current registrations (the same rule as the BigStitcher GUI's "Only in regions overlapping other views": one detection per region, combined, points closer than 0.5 px to one of another region dropped); --onlyCompareOverlapTiles counts only other volumes of the same timepoint as partners, i.e. other tiles or angles, not other channels or illuminations of the same volume (which overlap it fully). --prefetch will use parallel threads to pre-load all image data blocks ahead of the computation, which is desirable for cloud execution.
Every detection stores three attributes next to its position, exactly as the BigStitcher GUI does: the DoG response, the sigma (in pixels at full resolution) and the image intensity at the detection (linearly interpolated, in the raw units of the image at the resolution of the detection). They are stored with every point (the points array of a label in interestpoints.zarr holds x, y, z, response, sigma and intensity as float32) and are what filter-interestpoints and the interactive filter of the BigStitcher GUI work with (the former --storeIntensities is gone, intensities are always stored). --maxSpots keeps the brightest points per view by that intensity; prefer a filtered label (filter-interestpoints --keepPoints), which keeps the full detection.
--method SCALE_SPACE runs the scale-space Difference-of-Gaussian of BigStitcher instead of the single-scale one: starting at -s (the initial blur, e.g. 1.6) it finds structures of all sizes and stores the sigma of each, so beads and nuclei of different sizes come out of one run and can be separated by filter-interestpoints --sigma. --steps (per octave, default 4), --octaves (default -1 = all the image allows) and --no-finestLevel (keep scale extrema only; by default all spatial extrema of the finest level are kept, since sub-resolution structures like beads are no scale extrema) are its parameters, -dsxy/-dsz is the resolution it starts at.
--anisotropy gives the ratio z voxel / x voxel of the raw data; the Gaussians are then isotropic in physical units (the BigStitcher GUI suggests 0.8 x the calibration ratio, since the PSF blurs z more than the calibration says). Without it the Difference-of-Gaussian blurs isotropically in pixels as it always did, the scale space uses the calibration ratio.
--noiseFactor replaces -t: the threshold becomes the factor times the noise floor of the data, the robust standard deviation of the finest Difference-of-Gaussian level, estimated in 16 random blocks of every view (one Spark job per view) and pooled as the highest of the views, as "Estimate from noise floor" in the BigStitcher GUI does (2.5 is its default). The resolved threshold is printed and stored with the label, nothing is estimated at detection time:
./detect-interestpoints -x ~/SparkTest/IP/dataset.xml -l beads -s 1.8 --noiseFactor 2.5 -dsxy 2 --minIntensity 0 --maxIntensity 255
You can choose which Tiles --tileId, Channels --channelId, Iluminations --illuminationId, Angles --angleId and Timepoints --timepointId will be processed, a typical choice could be --timepointId 18 --tileId 0,1,2,3,6,7,8 to only process the timepoint 18 and select Tiles. If you would like to choose Views more fine-grained, you can specify their ViewIds directly, e.g. -vi '0,0' -vi '0,1' -vi '1,1' to process ViewId 0 & 1 of Timepoint 0 and ViewId 1 of Timepoint 1. By default, everything will be processed.
Note: --dryRun allows the user to test the functionality without writing any data. The Spark implementation parallelizes over user-defined block across all processed images at once.
A filtered label keeps no points of its own: it filters an existing label (its base) on the fly by ranges of the |DoG response|, the image intensity and the sigma of the detections and/or keeps a fixed number of points per view, has its own correspondences, and follows a re-detection of its base. It is the way to try several thresholds without detecting again, to split the structures a scale-space detection found by their size, or to keep the strongest n points per view (this is what "Filter/Manage Interest Points" of the BigStitcher GUI creates, and the Interest Point Explorer shows filtered labels in italics). A typical call:
./filter-interestpoints -x ~/SparkTest/IP/dataset.xml -l beads --response 0.016
creates beads_r0.016, the detections of beads with |response| >= 0.016. -l is the label to filter, --response, --intensity and --sigma take a range each (a keeps >= a, a-b keeps a..b; the intensity in the raw units of the image at the resolution of the detection, the sigma in pixels at full resolution); several entries separated by , create one label per entry, e.g. --response 0.008,0.016,0.032 is a threshold ramp. --keepPoints 3000 keeps a fixed number of points per view (after the ranges), ranked by --keepBy (response, intensity or sigma, default response) and --keepMode (highest, median or lowest, default highest). The label is named base label plus suffix (e.g. beads_r0.016, beads_rh3000, beads_s1.5-3) unless --newLabel says otherwise; --copy stores the points that pass as a plain label instead of a filtered one. The view-selection flags apply, --dryRun only prints what would be created. It runs on the driver only, no Spark job is needed.
Removing a label with clear-interestpoints --clearMode REMOVE_LABEL also removes the labels filtered from it, and FIX_INTERESTPOINTS drops filtered labels whose base is gone.
embed-interestpoints computes a learned embedding (a unit vector, 64-D for the current model) for every interest point of a label and stores it with the points in interestpoints.zarr, as pcmatch's embed does; match-interestpoints -m EMBEDDINGS then finds the correspondences by the similarity of these vectors instead of the geometric descriptors. The model is pcmatch's ONNX export (pcmatch export-onnx, a .onnx file with its .onnx.json sidecar next to it, which carries the model's fingerprint and the cube geometry); it is evaluated in Java, no ONNX Runtime or GPU is needed. A typical call:
./embed-interestpoints -x ~/SparkTest/IP/dataset.xml -l beads --model hf://JaneliaSciComp/bigstitcher-embedding-translation-stack64
takes the model from the Hugging Face Hub (hf://<namespace>/<name>[@<revision>][/<file>]; the revision defaults to main and is pinned to its commit, which the action history records; every executor downloads the .onnx and its sidecar once into --modelCache, default ~/.bigstitcher/models on each node or the system property bigstitcher.models, verified by sha256). On a cluster without internet access download the two files once and pass the .onnx as --model /models/translation-stack.onnx (a path every executor can read, the sidecar next to it). The command runs one Spark task per view: it samples five isotropic cubes per point from the image pyramid (edges of 5 to 30 detection sigmas, never finer than the level the detection ran at, NaN outside the image), runs the network block by block (--blockSize, default 128 points per call) and writes the view's rows into the array --arrayName (default embedding) of the label. A filtered label is embedded through its base label: all points of the base, stored under the base, so the filtered labels of that base read them too. An array of another model under the same name is replaced as a whole, since rows of two models cannot be matched against each other. The label needs the detection sigma from its parameters string (a DoG or scale-space detection); for a scale-space label every point's own sigma scales its cubes. The view-selection flags apply, --dryRun computes without writing. Speed: about 500 points per second and core.
After interest points are detected they are pair-wise matching between all views/images (Note: this also works for Stitching, try it out). Several point cloud matching methods and ways how views can be grouped are supported, which will be explained below. Importantly, matching & solving can be performed once or iteratively; typical workflows that match & solve more than once are 1) to first align each timepoint of a series using affine models individually followed by registration across time using translation models or 2) to first align using geometric descriptor matching to then subsequently refine the result using Iterative Closest Point (ICP) that only works once the current transformation is very good (Note: ICP alignment creates many corresponding interest points that might be desirable for Non-Rigid fusion where all corresponding interest points are perfectly matched on top of each other).
A typical, simple command line call to register each timepoint alignment individually using this example looks like:
./match-interestpoints -x ~/SparkTest/IP/dataset.xml -l beads -m FAST_ROTATION --clearCorrespondences
Please run match-interestpoints without parameters to get help for all command line arguments. -l defines the label of the detected interest points used for matching. -tm specifies the transformation model to be used (TRANSLATION, RIGID or (default)AFFINE), -rm defines the regularization model (NONE, IDENTITY, TRANSLATION, (default)RIGID or AFFINE) and --lambda [0..1] is the lambda for the regularization model, which is set to 0.1 by default. -vr defines which views/images will be matched; OVERLAPPING_ONLY or ALL_AGAINST_ALL. --clearCorrespondence removes potentially existing, stored matches between views, if it is not called the identified matches will be added to the existing ones.
-m defines the matching method; FAST_ROTATION, PRECISE_TRANSLATION, ICP, EMBEDDINGS, or LOAD_CANDIDATES (precomputed candidates from an N5 store, --candidatesPath; only RANSAC runs).
FAST_ROTATIONis a rotation invariant method that uses geometric hashing and can find corresponding constellation of points even if they are significantly rotated relative to each other.-sdefines the significance during descriptor matching, to establish a correspondence the best matching descriptor has to bestimes better than the second best matching descriptor.-ris the level of redundancy during descriptor matching, it adds extra neighbors to each descriptor and tests all combinations of neighboring points.
PRECISE_TRANSLATIONis a translation invariant method that uses exhaustive search to find corresponding constellation of points irrespective of their location in the image. It tolerates small rotatation of up to a few degrees.- supports the same parameters
-r,-sas FAST_ROTATION above, and additionally support-nto specify the number of neighboring points that are used to build geometric descriptors,-srto limit the search to a radius around each point, and--descriptorSearchto pick the descriptor comparison (AUTOchooses per view pair between a flat KD-tree and a blocked brute force sweep;LEGACYis the original exhaustive loop)
- supports the same parameters
ICPis a method that iteratively assignes closest pairs of points between two images until convergence and can be used for fine alignment.-imeis the ICP maximum allowed error,-iitdefines the number of ICP iterations and--icpUseRANSACenables RANSAC at every ICP iteration
EMBEDDINGSmatches on the vectorspcmatch embedstored with the interest points (the per-point arrayembeddingofinterestpoints.zarr, one unit vector per point); no images are read. Partners within-sr/--searchRadius(optional, as for PRECISE_TRANSLATION: without it every point of the other view is a possible partner, which works well; a radius restricts the partners to the registration's uncertainty in global px, about 20 for a well pre-registered dataset, 300-500 for large translations) are compared by cosine, the best one is a candidate if(1 - best) < ratio x (1 - second best), or if it is alone within the radius.--descriptorRatiois that ratio (default 0.85, smaller is stricter),--minSimilaritythe cosine similarity a candidate needs at least (default 0.3),--arrayNamepicks the array when a label has more than one (otherwise the one it has is used). A filtered label uses the vectors of its base label (the array must hold one row per point of the base label); a label that was detected again since the vectors were written fails with "run pcmatch embed again"; grouped views are not supported. The BigStitcher GUI offers the same matcher as "Embeddings".
All methods use RANSAC to robustly identify a set of corresponding points in the set or correspondence candidates (optional for ICP). -rit defines the number of RANSAC iterations (increasing might help to find more correspondences), -rme the maximum error (epsilon) for RANSAC (increasing might help to find more correspondences), -rmir the minimum inlier ratio (setting to 0.0 might help to find more correspondences), and -rmif defines the minimum inlier factor for RANSAC (i.e. how many times the minimal number of inliers required by the transformation model need to be identified so a pair is valid).
By default, all views/images are matched individually. However, under certain conditions it may be useful to group certain views together (see illustration below). --splitTimepoints groups all angles/channels/illums/tiles that belong to the same timepoint as one single View, e.g. for stabilization across time. --groupChannels groups all channels that belong to the same angle/illumination/tile/timepoint together as one view, e.g. to register all channels together as one. --groupTiles groups all tiles that belong to the same angle/channel/illumination/timepoint together as one view, e.g. to align across angles. --groupIllums groups all illumination directions that belong to the same angle/channel/tile/timepoint together as one view, e.g. to register illumation directions together. Importantly, interest points in overlapping areas of grouped views need to be merged; --interestPointMergeDistance allows to set the merge distance. Note: You do not need to group views for interest point matching in order to group views during solving, these are two independent operations. However, it (usually) is not advisable to only group during matching.
You can choose which Tiles --tileId, Channels --channelId, Iluminations --illuminationId, Angles --angleId and Timepoints --timepointId will be processed, a typical choice could be --timepointId 18 --tileId 0,1,2,3,6,7,8 to only process the timepoint 18 and select Tiles. If you would like to choose Views more fine-grained, you can specify their ViewIds directly, e.g. -vi '0,0' -vi '0,1' -vi '1,1' to process ViewId 0 & 1 of Timepoint 0 and ViewId 1 of Timepoint 1. By default, everything will be processed.
When performing timeseries alignment, grouping is often a good choice (--splitTimepoints) and further details regarding matching across time need to be specified. Important: if you are running a second (or third) round of matching, you always need to solve in between to bake in the resulting transformations. -rtp defines the type of time series registration; TIMEPOINTS_INDIVIDUALLY (i.e. no registration across time), TO_REFERENCE_TIMEPOINT, ALL_TO_ALL or ALL_TO_ALL_WITH_RANGE. Depending on your choice you may need to define the range of timepoints --rangeTP or the reference timepoint --referenceTP. Below is an example command line call for aligning all views against all across time:
./match-interestpoints -x ~/SparkTest/IP/dataset.xml -l beads -m FAST_ROTATION --clearCorrespondences -rtp ALL_TO_ALL --splitTimepoints
Note: --dryRun allows the user to test the functionality without writing any data. The Spark implementation parallelizes over pairs of images.
The Solver computes a globally optimized result (one transformation per view/image) using all pairwise matches (interest points or stitching), specifically by minimizing the distance between all corresponding points (paiwise stitching is also expressed as a set of corresponding points) across all images/views. A typical call for running the solver on stitching results is (e.g. this dataset):
./solver -x ~/SparkTest/Stitching/dataset.xml -s STITCHING
and when using matched interestpoints individually per timepoint it is (e.g. this dataset):
./solver -x ~/SparkTest/IP/dataset.xml -s IP -l beads
Please run solver without parameters to get help for all command line arguments. -s switches between STITCHING and IP (interest points) mode, -l defines the interest point label in the latter case. By default the first view of each timepoint will be fixed, -fv allows to specify certain views to be fixed and --disableFixedViews will not fix any views (in this case make sure to not use plain affine models). Without fixed views, --enableMapbackViews maps the result back onto one reference view per timepoint (--mapbackViews '0,0', default: first view) using a RIGID or TRANSLATION --mapbackModel, exactly like the GUI's "Map back views" option. -tm specifies the transformation model to be used (TRANSLATION, RIGID or (default)AFFINE), -rm defines the regularization model (NONE, IDENTITY, TRANSLATION, (default)RIGID or AFFINE) and --lambda [0..1] is the lambda for the regularization model, which is set to 0.1 by default.
--maxError sets the maximum allowed error for the solve (it will iterate at least until it is under that value), --maxIterations defines the maximum number of iterations, and --maxPlateauwidth defines the number of iterations that are used to estimate if the solve converged (and is thus also the minimal number of iterations).
There are several types of solvers available; --method allows to choose ONE_ROUND_SIMPLE, ONE_ROUND_ITERATIVE, TWO_ROUND_SIMPLE or TWO_ROUND_ITERATIVE. Two round handles unconnected tiles (it moves them into their approximate right location using metadata using a second solve where all views that are connected are grouped together), while iterative tries to identify and remove wrong/inconsistent links between pairs of views. The iterative strategies are parameterized by --relativeThreshold (relative error threshold, how many times worse than the average error a link between a pair of views needs to be) and the --absoluteThreshold (absoluted error threshold to drop a link between a pair of views - in pixels). The error is computed as the difference between the pairwise alignment of two views and their alignment after running the solve.
You can choose which Tiles --tileId, Channels --channelId, Iluminations --illuminationId, Angles --angleId and Timepoints --timepointId will be processed, a typical choice could be --timepointId 18 --tileId 0,1,2,3,6,7,8 to only process the timepoint 18 and select Tiles. If you would like to choose Views more fine-grained, you can specify their ViewIds directly, e.g. -vi '0,0' -vi '0,1' -vi '1,1' to process ViewId 0 & 1 of Timepoint 0 and ViewId 1 of Timepoint 1. By default, everything will be processed.
By default, all views/images are matched individually. However, under certain conditions it may be useful to group certain views together (see illustration above). --splitTimepoints groups all angles/channels/illums/tiles that belong to the same timepoint as one single View, e.g. for stabilization across time. --groupChannels groups all channels that belong to the same angle/illumination/tile/timepoint together as one view, e.g. to register all channels together as one. --groupTiles groups all tiles that belong to the same angle/channel/illumination/timepoint together as one view, e.g. to align across angles. --groupIllums groups all illumination directions that belong to the same angle/channel/tile/timepoint together as one view, e.g. to register illumation directions together.
When performing timeseries alignment, grouping is usually a good choice (--splitTimepoints) and further details regarding matching across time need to be specified. Important: if you are running a second (or third) round of matching, you usually need to match interest points again. -rtp defines the type of time series registration; TIMEPOINTS_INDIVIDUALLY (i.e. no registration across time), TO_REFERENCE_TIMEPOINT, ALL_TO_ALL or ALL_TO_ALL_WITH_RANGE. Depending on your choice you may need to define the range of timepoints --rangeTP or the reference timepoint --referenceTP. Below is an example command line call for aligning all views against all across time:
When using interestpoints (for timeseries alignment with grouping all views of a timepoint together) a tyical call could look like that:
./solver -x ~/SparkTest/IP/dataset.xml -s IP -l beads -rtp ALL_TO_ALL_WITH_RANGE --splitTimepoints
Note: --dryRun allows the user to test the functionality without writing any data. The solver currently only runs multi-threaded.
Two utilities for editing the interest-point data and the registrations stored in a project without opening BigStitcher. Both accept the same view-selection flags as the other commands (--tileId, --channelId, --illuminationId, --angleId, --timepointId, or -vi '0,0' -vi '0,1' for individual ViewIds); without them every view is processed. --dryRun prints what would happen without writing anything.
./clear-interestpoints -x ~/SparkTest/IP/dataset.xml --clearMode REMOVE_LABEL --label beads --tileId 3,4
./clear-registrations -x ~/SparkTest/IP/dataset.xml --remove 1 -vi '0,0'
clear-interestpoints modes (--clearMode): CLEAR_EVERYTHING (default) removes all interest points and correspondences of the selected views; without a view selection it deletes the whole interest point store (interestpoints.zarr, and a legacy interestpoints.n5). CLEAR_ALL_CORRESPONDENCES keeps the detections and drops the correspondence links. ADD_LABEL / REMOVE_LABEL add an empty interest-point list with the given --label, or remove that label (with the filtered labels of it), from the selected views. FIX_INTERESTPOINTS repairs a project whose views were removed from the XML (drops orphaned entries, legacy N5 groups and correspondences, and filtered labels whose base label is gone); it always covers the whole dataset and ignores the view selection.
Note: whenever data is removed from selected views, correspondences in unselected views that point into the selection are removed as well, so the project stays consistent. --silent skips the per-view listing, which otherwise loads all selected interest points just to print counts (recommended for large or cloud-hosted datasets).
clear-registrations strips transformations from the selected views, exactly one of: --keep N keeps only the first N transformations (in order of application, e.g. calibration first), --remove N removes the last N, --index 1,3 removes the ones at these positions (0 = the first applied; every view's chain is printed with its indices, so a --dryRun shows them), --name 'Mirror *' removes the ones whose name matches the pattern (* any text, ? one character; repeat for several patterns).
Two utilities that create a project and set up its initial transformations without BigStitcher's dialogs, e.g. for a deskewed diSPIM acquisition whose geometry (slice spacing, reversed scan direction, arm rotation) is known beforehand, so that only the interest point detection and the registration remain.
./define-dataset -x ~/data/dataset.xml -i viewA.tif -i viewB.tif -i viewC.tif --voxelSize 0.0534,0.0534,0.549 --voxelSize 0.0534,0.0534,0.836 --voxelSize 0.0534,0.0534,0.5 --unit um
define-dataset is the command-line counterpart of "Define Multi-View Dataset" for the plain case of one image file per view: every -i file (anything Bio-Formats reads; each channel of a file becomes a view with that channel id, only the first series is used) becomes a view setup with the image size read from the file, the voxel size from --voxelSize x,y,z (one per image or one for all; default: the file's metadata, which ImageJ TIFFs often get wrong) and --unit, and the attributes --angle, --tile, --illumination (one id per image or one for all; default: the image index as the angle, tile and illumination 0) and --timepoint. The XML gets the virtual Bio-Formats image loader (spimreconstruction.filemap2, planes are read on demand) and the calibration transforms (voxel size over the smallest voxel size) and is refused if it exists, unless --overwrite. --dryRun prints the setups and writes nothing. Resave to N5/OME-Zarr afterwards with resave for fast access.
./apply-transformation -x ~/data/dataset.xml --mirror z --angleId 1
./apply-transformation -x ~/data/dataset.xml --rotate y,-56.7 --angleId 1
./apply-transformation -x ~/data/dataset.xml --centerOnView 0,2 --angleId 0,1
apply-transformation is "Apply Transformations" as a command: it adds one view transform to the selected views (the usual selection flags; all views without them), after the existing transforms like the dialog, or before them (in the image's pixel coordinates) with --applyFirst. Exactly one of --rotate AXIS,DEGREES (the dialog's right-handed rotation about x, y or z), --translate dx,dy,dz, --scale sx,sy,sz, --mirror x|y|z, --affine m00,...,m23 (12 row-major values as in the XML) or --centerOnView TIMEPOINT,SETUP (the translation that puts the centre of each selected view's image, through its current transforms, onto the centre of the reference view: a starting point for the registration). --name sets the transform's name in the XML, --dryRun only prints. Transforms are removed again with clear-registrations (by index or name). A shear is not offered yet, since BigDataViewer's interpolation of a sheared view breaks down (future work).
Headless BigDataViewer renderings of a project: the selected views under their current registrations, composited with per-view colours and display ranges, written as PNG files without a window (works over ssh with DISPLAY unset, e.g. on a cluster node). Made for judging an alignment by eye or by a script after solver: a view pair in magenta and green turns white where the two views agree and shows coloured doubles where they do not. Read-only: the XML is never written.
./screenshot -x ~/SparkTest/IP/dataset.xml -o ~/SparkTest/IP/shots --pairs overlapping --planes xy,xz --auto --json
./screenshot -x ~/SparkTest/IP/dataset.xml -o ~/SparkTest/IP/shots --overview --single --planes xy,xz,yz --render slice --width 2048
What (at least one): --pairs overlapping renders one image set per pair of selected views whose registered bounding boxes intersect (the OVERLAPPING_ONLY rule of the registration), --pairs 'A,B;C,D' the given setup pairs; the camera is centred on the intersection of the two boxes, view A is magenta and view B green (--colors), files pair_tp<T>_s<A>_s<B>_<plane>.png. --overview shows all selected views together, centred on their union box, one colour per view (--palette colorstream = BigStitcher's per-view colours from --paletteStart, fixed = magenta, green, cyan, yellow, orange, blue cycling, or an explicit list with exactly one colour per selected view), files overview_<plane>.png. --single renders every selected view alone in grey, files view_tp<T>_s<S>_<plane>.png. --transform FILE.json renders the selected views once per viewer transform in the file (the JSON BigStitcher's BDV fly-through saves: an array of 12-value affines, global to screen pixels, applied as-is to --width x --height), files transform<i>.png. The usual view-selection flags apply; without a timepoint selection the first timepoint is rendered, and one call renders one timepoint.
Camera: --planes xy,xz,yz (default xy) are the viewing axes of the global frame: xy looks along z with x right and y down (BigDataViewer's default), xz has x right and z down, yz has y right and z down. --render mip (default) is a maximum projection over --depth global pixels centred on the box (default: the whole box for pairs and single views, a quarter of it for the overview), in steps of --step global pixels (default: the largest of one voxel of the shown views along that axis, one screen pixel and depth/256, so a projection costs at most 257 frames); --render slice is one plane through the box centre. --width (1024), --height (from the box aspect, 16..4096), --margin (0.1 of the box on every side), --maxScale (4): a box smaller than the image, e.g. the overlap strip of two neighbouring tiles, is drawn at most at this magnification into a correspondingly smaller image rather than blown up to the full width. --level L pins the resolution level of every view instead of letting the renderer choose it from the screen scale.
Display: --auto (the default) takes the 0.1 / 99.9 percentiles of every view at a coarse resolution level, for a pair image inside the pair's overlap box so that both views show with comparable brightness where they overlap, --min a --max b one range for all views, --range S=a:b a range per setup. --json [path] writes a manifest (default <outputDir>/screenshots.json) with, per image, the file, kind, views, plane, render mode, depth, step, frames, the box, the viewer transform (12 values, the same layout as a --transform file), size, scale, colours and ranges, so a script can tell what every picture shows. --dryRun prints the planned images and writes nothing.
Note: to check that the pictures react to a misalignment, shift one view with ./apply-transformation -x dataset.xml --translate 20,0,0 -vi 0,3 on a copy of the project: its pair images then show magenta/green doubles 20 pixels apart. Overlays of interest points and bounding boxes are not implemented yet.
Performs fusion using affine transformation models computed by the solve (also supports translations, rigid, interpolated models) that are stored in the XML (Warning: not tested on 2D). By default the affine fusion will create an output image that encompasses all transformed input views/images. While this is good in some cases such as tiled stitching tasks, the output volume can be unnecessarily large for e.g. multi-view datasets. Thus, prior to running the fusion it might be useful to define a custom bounding box in BigStitcher.
The first step for fusing a dataset is to create an empty output container that also contains all metadata and empty multi-resolution pyramids. By default an OME-ZARR is created, N5 and HDF5 are also supported (note that HDF5 only works if Spark is run multi-threaded on a local computer, i.e. not on a cluster or the cloud). By default, this will create an output container that contains a 3D volume for each channel and timepoint present in the dataset (i.e. numChannels x numTimepoints 3D volumes). In the case of OME-ZARR, it is represented as a single 5D volume, for N5 and HDF5 it is a series of 3D volumes. Note: if you do NOT want to export the entire project, or want to specify fusion assignments (which views/images are fused into which volume), please check the details below. In short, you can specify the dimensions of the output container here, and the fusion assignments in the affine-fusion step below.
A typical call for creating an output container for e.g. the stitching dataset is (e.g. this dataset):
./create-fusion-container -x ~/SparkTest/Stitching/dataset.xml -o ~/SparkTest/Stitching/Stitching/fused.zarr --preserveAnisotropy --multiRes -d UINT8
The output container for the dataset that was aligned using interest points can be created in the same way, except that we choose to use the bounding box embryo that was specified using BigStitcher and we choose to save as an BDV/BigStitcher project using N5 as underlying export data format:
./create-fusion-container -x ~/SparkTest/IP/dataset.xml -o ~/SparkTest/IP/fused.n5 -xo ~/SparkTest/IP/dataset-fused.xml -s N5 -b embryo --bdv --multiRes -d UINT8
Running create-fusion-container without parameters lists help for all command line arguments. -x specifies the XML, -o defines the output volume location and -s the output type OME-ZARR, N5, or HDF5 (latter only when running on a local computer). --multiRes will create multiresolution pyramids of the fused image(s), -ds allows to optionally specify the downsampling steps for the multiresolution pyramid manually. --bdv will create fused volumes together with an XML that can be directly opened by BigStitcher or BigDataViewer, where -xo defines the location of the XML for the fused dataset.
You can specify the datatype for the fused image using -d, UINT8 [0..255], UINT16 [0..65535] or FLOAT32 (default) are supported. When selecting UINT8 or UINT16 you can set --minIntensity and --maxIntensity, which define the range of input intensities that will be mapped to [0..255] for UINT8 or [0..65535] for UINT16, respectively. The default values are [0..255] for UINT8 or [0..65535] for UINT16, which works well if input and output datatype are the same.
You can specify a bounding box using -b. --preserveAnisotropy will preserve the anisotropy of the input dataset, which is a recommended setting if all views/images are taken in the same orientation, e.g. when processing a tiled dataset. The factor will be computed from the data by default, or can be specified using --anisotropyFactor.
-c allows to specify the compression method, -cl set the compression level if supported by the compression method. --blockSize defaults to 128x128x128, which you might want to reduce when using HDF5.
--numChannels allows you to override the number of channels in the fused output dataset, by default it will be as many channels as present in the BigStitcher project/XML. Similarly, --numTimepoints allows you to override the number of timepoint in the fused noutput dataset, by default it will be as many timepoints as present in the BigStitcher project/XML. *Note: If (and only if) you specify --numChannels or --numTimepoints you MUST specify in the affine-fusion which ViewIds are fused into which 3D volume, since the assignment deviates from the BigStitcher project/XML. By default, all tiles/angles/illuminations for each channel and timepoint will be fused together.
--s3Region allows to specify a specifc AWS region when saving to AWS S3.
Note: creating the container for fusion is NOT Spark code (i.e. not distributed), just plain Java code, the fusion itself below is Spark code.
This is not updated yet, in short you just need to run e.g. ./affine-fusion -o ~/SparkTest/Stitching/Stitching/fused.zarr that points to the container you created before.
Importantly, one can fuse several volumes into the same N5, ZARR or HDF5 container by running the fusion consecutively and specifying different folders or BDV ViewIds.
You can choose which Tiles --tileId, Channels --channelId, Iluminations --illuminationId, Angles --angleId and Timepoints --timepointId will be processed. For fusion one normally chooses a specific timepoint and channel, e.g. --timepointId 18 --channelId 0 to only fuse timepoint 18 and Channel 0 into a single volume. If you would like to choose Views more fine-grained, you can specify their ViewIds directly, e.g. -vi '0,0' -vi '0,1' to process ViewId 0 & 1 of Timepoint 0.
--blockScale defines how many blocks to fuse in a single processing step, e.g. 4,4,1 means for blockSize of 128,128,64 that each spark thread processes 512,512,64 blocks.
Note: --dryRun allows the user to test the functionality without writing any data. It scales to large datasets as it tests for each block that is written which images are overlapping. For cloud execution one can additionally pre-fetch all input data for each compute block in parallel. You need to specify the XML of a BigSticher project and decide which channels, timepoints, etc. to fuse.
OLD DOCUMENTATION:
A typical set of calls (because it is three channels) for affine fusion into a multi-resolution ZARR using only translations on the stitching dataset is (e.g. this dataset):
./affine-fusion -x ~/SparkTest/Stitching/dataset.xml -o ~/SparkTest/Stitching/fused.zarr -d /ch0/s0 -s ZARR --multiRes --preserveAnisotropy --UINT8 --minIntensity 0 --maxIntensity 255 --channelId 0
./affine-fusion -x ~/SparkTest/Stitching/dataset.xml -o ~/SparkTest/Stitching/fused.zarr -d /ch1/s0 -s ZARR --multiRes --preserveAnisotropy --UINT8 --minIntensity 0 --maxIntensity 255 --channelId 1
./affine-fusion -x ~/SparkTest/Stitching/dataset.xml -o ~/SparkTest/Stitching/fused.zarr -d /ch2/s0 -s ZARR --multiRes --preserveAnisotropy --UINT8 --minIntensity 0 --maxIntensity 255 --channelId 2
You can open the ZARR in Fiji (File > Import > HDF5/N5/ZARR/OME-NGFF or Plugins > BigDataViewer > HDF5/N5/ZARR/OME-NGFF), using n5-view in the n5-utils package (./n5-view -i ~/SparkTest/Stitching/fused.zarr -d /ch0) or in Napari (simply drag&drop e.g. the ch0 or a s0 folder).
The dataset that was aligned using interest points can be fused in a similar way, except that here we choose to use the bounding box embryo that was specified using BigStitcher and we choose to save as an BDV/BigStitcher project using N5 as underlying export data format:
./affine-fusion -x ~/SparkTest/IP/dataset.xml -o ~/SparkTest/IP/fused.n5 -xo ~/SparkTest/IP/dataset-fused.xml -s N5 -b embryo --bdv 18,0 --multiRes --UINT8 --minIntensity 0 --maxIntensity 255 --timepointId 18
./affine-fusion -x ~/SparkTest/IP/dataset.xml -o ~/SparkTest/IP/fused.n5 -xo ~/SparkTest/IP/dataset-fused.xml -s N5 -b embryo --bdv 30,0 --multiRes --UINT8 --minIntensity 0 --maxIntensity 255 --timepointId 30
In additon to the opening methods mentioned above, you can also directly open the dataset-fused.xml in BigStitcher or BigDataViewer; unfortunately opening of N5's in a vanilla Napari is not supported.
Note: since both acquisitions have more than one channel or timepoint it is important to fuse these into seperate output volumes, respectively.
Note: --dryRun allows the user to test the functionality without writing any data. It scales to large datasets as it tests for each block that is written which images are overlapping. For cloud execution one can additionally pre-fetch all input data for each compute block in parallel. You need to specify the XML of a BigSticher project and decide which channels, timepoints, etc. to fuse.
nonrigid-fusion performs non-rigid distributed fusion using net.preibisch.bigstitcher.spark.SparkNonRigidFusion. The arguments are identical to the Affine Fusion, and one needs to additionally define the corresponding interest points, e.g. -ip beads that will be used to compute the non-rigid transformation.



