# Scripts Reference¶

Command-line scripts are available, both Ptychography and CDI scripts.

## Ptychography¶

These are instructions to run the command-line scripts such as pynx-cxipty.py, pynx-id01pty.py, pynx-id13pty.py and pynx-id16apty.py.

This help text can be simply obtained by typing the script without any parameter.

### Generic instructions¶

Here are generic instructions applying to all the scripts:

generic (not beamline-specific) command-line arguments: (all keywords are case-insensitive)
scan=56: scan number (e.g. in specfile) [mandatory, unless cxifile is used as input].
Alternatively a list or range of scans can be given:
scan=12,23,45 or scan="range(12,25)" (note the necessary quotes when using range)

maxframe=128: limit the total number of frames used to the first N
[default=None, all frames are used]

maxsize=256: the frames are automatically cropped to the largest possible size while keeping
the center of gravity of diffraction in the frame center. Use this to limit the
maximum frame size, for tests or limiting memory use [default=512]

moduloframe=n1,n2: instead of using all sequential frames of the scan, only take one in n1.
if n2<n1 is also given, then the frame numbers taken will be the n for which
n % n1 == n2. This can be used to perform two reconstructions with half of
the frames, and then analyse the resolution using Fourier ring correlation.
If both moduloframe and maxframe are given, the total number of frames taken
is still maxframe.
[default: take all frames]

algorithm=ML**50,DM**100,probe=1: algorithm for the optimization: [default='ML**50,DM**100']

The algorithm used is:
- divided in 'steps' separated by commas, e.g: ML**50,DM**100
- interpreted from right to left, as a mathematical operator to an object on the
right-hand side
- should not contain any space, unless it is given between quotes ('')

- the first type of commands can change basic parameters or perform some analysis:
- probe=1 or 0: activate or deactivate the probe optimisation (by default only the
object is optimised)
- object=1 or 0: activate or deactivate the object optimisation
- background=1 or 0: activate or deactivate the background optimisation
(only works with AP)
- position=1 or 0: activate or deactivate position optimisation (for AP or ML, needs some prior convergence)
- nbprobe=3: change the number of modes for the probe (can go up or down)
- regularization=1e-4: setting the regularization parameter for the object, to penalize
local variations in ML runs and smooth the solution
- obj_smooth=1.5 or probe_smooth=1.0: these parameters will partially smooth the object and probe inertia,
softening the resulting arrays. This applies to DM and AP algorithms.
- obj_inertia=0.01 or obj_probe=0.001: these parameters set the inertia of the object and/or probe update,
yielding more stable result. This applies to DM and AP algorithms.
- ortho: will perform orthogonalisation of the probe modes. The modes are sorted by
decreasing intensity.
- analysis: perform an analysis of the probe (propagation, modes). Useful combined
with 'saveplot' to save the analysis plots

- the second type are operators which will be applied to the Ptycho object:
- AP: alternate projections. Slow but converging algorithm
- DM: difference map. Fast early convergence, oscillating after.
- ML: maximum likelihood conjugate gradient (Poisson-noise). Robust, converging,
for final optimization.
These operators can be combined mathematically, e.g.:
- DM**100: corresponds to 100 cycles of difference map
- ML**40*DM**100: 100 cycles of DM followed by 40 cycles of ML (note the order)

Example algorithms chains:
- algorithm=ML**40,DM**100,probe=1: activate probe optimisation,
then 100 DM and 40 ML (quick)
- algorithm=ML**100,DM**200,nbprobe=3,ML**40,DM**100,probe=1,DM**100: first DM with
object update only,  then 100 DM also updating the probe, then use 3 probe modes
and do 100 DM followed by 40 ML
- algorithm=ML**100*AP**200*DM**200,probe=1: 200 DM then 200 AP then 100 ML (one step)
- algorithm='(ML**10*DM**20)**5,probe=1':
repeat 5 times [20 cycles of DM followed by 5 cycles of ML]
(note the quotes necessary for the parenthesis)

nbrun=10: number of optimizations to perform [default=1]

run0=10: number for the first run (can be used to overwrite previous run results)
[default: after previous results or 1]

liveplot: liveplot during optimization [default: no display]

saveplot: will save plot at the end of the optimization (png file) [default= not saved].
Optionally this can also specify if only the object phase should be plotted, e.g.:
saveplot=object_phase: will display the object phase
saveplot=object_rgba: will use RGBA to display both amplitude and phase.

saveprefix=ResultsScan%04d/Run%04d: prefix to save the optimized object and probe
(as a .cxi or .npz file) and optionally image (png) [default='ResultsScan%04d/Run%04d']

output_format='cxi': choose the output format for the final object and support.
Possible choices: 'cxi', 'npz'
[Default='cxi']

save=all: either 'final' or 'all' this keyword will activate saving after each optimization
step (ptycho, ML) of the algorithm in any given run [default=final]

load=Results0057/Run0001.cxi (or .npz): load object and probe from previous optimization. Note that the
object and probe will be scaled if the number of pixels is different for the probe.
[default: start from a random object, simulate probe]

[default: simulate probe]

(and possibly object). If the pixel size is different,
the loaded arrays will be scaled to match the new pixel size.
calculated from the size of the probe array, assuming same detector
distance and pixel size]

probe=focus,60e-6x200e-6,0.09: define the starting probe, either using:
focus,60e-6x200e-6,0.09: slits size (horizontal x vertical),
focal distance (all in meters)
focus,200e-6,0.09: radius of the circular aperture,
focal distance (all in meters)
gaussian,100e-9x200e-9: gaussian type with horizontal x vertical
FWHM, both given in meters
disc,100e-9: disc-shape, with diameter given in meters

defocus=1e-6: defocused position (+: towards detector). The probe is propagated by this distance
before being used. This is true both for calculated probes (using 'probe=...')
and for probes loaded from a previous file.

rotate=30: rotate the probe (either simulated or loaded) by X degrees [default: no rotation]

object=random,0.9,1,0,6: specify the original object values. The object will be initialised
over the entire area using random values: random,0-1,0-6.28 : random
amplitudes between 0 and 1, random phases between 0 and 6.28.
For high energy small-angle ptycho (i.e. high transmission),
recommended value is: random,0.9,1,0,0
[default: random,0,1,0,6.28]

verbose=20: print evolution of llk (and display plot if 'liveplot' is set) every N cycle
[default=50]

data2cxi: if set, the raw data will be saved in CXI format (http://cxidb.org/cxi.html),
will all the required information for a ptychography experiment (energy, detector
distance, scan number, translation axis are all required). if 'data2cxi=crop'
is used, the data will be saved after centering and cropping (default is to save
the raw data). If this keyword is present, the processing stops after exporting the data.

shape as the raw detector data.
This should be a boolean or integer array with good pixels=0 and bad ones>0
(values are expected to follow the CXI convention)
Acceptable formats:
2D array. 'hdf5' is also accepted as extension.
- 'maxipix': if this special name is entered, the masked pixels will be rows
and columns multiples of 258+/-3

roi=xmin,xmax,ymin,ymax: the region-of-interest to be used for actual inversion. The area is taken
with python conventions, i.e. pixels with indices xmin<= x < xmax and
ymin<= y < ymax.
Additionally, the shape of the area must be square, and
n=xmax-xmin=ymax-ymin must also be a suitable integer number
for OpenCL or CUDA FFT, i.e. it must be a multiple of 2 and the largest number in
its prime factor decomposition must be less or equal to the largest value
acceptable by clFFT (<=13 as of November 2016) or cuFFT (<=7).
If n does not fulfill these constraints,
it will be reduced using the largest possible integer smaller than n.
This option supersedes 'maxsize' unless roi='auto'.
Other possible values:
- 'auto': automatically selects the roi from the center of mass
and the maximum possible size. [default]
- 'all' or 'full': use the entire, uncentered frames. Only useful for pre-processed
data. Cropping may still be performed to get a square and
FFT-friendly size.

rebin=2: the experimental images can be rebinned (i.e. a group of n x n pixels is replaced by a
single one whose intensity is equal to the sum of all the pixels). This 'rebin' is
performed last: the ROI, mask, background, pixel size should all correspond to
full (non-rebinned) frames.
[default: no rebin]

autocenter=0: by default, the object and probe are re-centered automatically after each
optimisation step, to avoid drifts. This can be used to deactivate this behaviour
[default=True]

center_probe_n, center_probe_max_shift: during DM, the probe can shift. The probe center of mass  will be checked
every center_probe_n cycles, and the object and probe position will be
corrected if the center deviates by more than center_probe_max_shift pixels.
This is ignored for near field ptycho.
[default: center_probe_n=5, center_probe_max_shift=5]

dm_loop_obj_probe=3: during DM, when both object and probe are updated, it can be more stable to loop the object
and probe update for a more stable optimisation, but slower. [default: 2]

detector_orientation=1,0,0: three flags which, if True, will do in this order:
array transpose (x/y exchange), flipud, fliplr [default: no change]
The changes also apply to the mask.

xy=y,x: order and expression to be used for the XY positions (e.g. '-x,y',...). Mathematical
operations can also be used, e.g.: xy=0.5*x+0.732*y,0.732*x-0.5*y
[default: None- may be superseded in some scripts e.g. for ptypy]

flatfield=flat.npy: the flatfield correction to be applied to the detector data. The array must
have the same shape as the frames, which will be multiplied by this
correction.
Acceptable formats:
- flat.npy, flat.npz (the first data array will be used)
- flat.edf or flat.edf.gz (a single 2D array is expected)
- "flat.h5:/entry_1/path/to/flat" hdf5 format with the full path to the
2D array. 'hdf5' is also accepted as extension.
- flat.mat: from a matlab file. The first array found is loaded
[default: no flatfield correction is applied]

dark=dark.npy: the dark correction to be subtracted from the detector data. The array must have
the same shape as the frames, which will be substracted by this correction.
Acceptable formats:
- dark.npy, dark.npz (the first data array will be used)
- dark.edf or dark.edf.gz (a single 2D array is expected)
- "dark.h5:/entry_1/path/to/dark" hdf5 format with the full path to the
2D array. 'hdf5' is also accepted as extension.
- dark.mat: from a matlab file. The first array found is loaded
[default: no dark correction is applied]

orientation_round_robin: will test all possible combinations of xy and detector_orientation
to find the correct detector configuration.


### Specific instructions¶

Instructions for beamline-dedicated scripts can be fully obtained by executing the script without any parameter.

#### pynx-cxipty.py:¶

Script to perform a ptychography analysis on data recorded in CXI format (http://cxidb.org/cxi.html)

Examples:
pynx-cxipty.py data=data.cxi probe=focus,60e-6x200e-6,0.09
algorithm=analysis,ML**100,DM**200,nbprobe=3,probe=1 verbose=10 save=all saveplot liveplot

command-line arguments:
cxifile=/some/dir/to/data.cxi or data=some/dir/to/data.cxi: path to CXI data [mandatory]

scan=56: scan number [optional: this is used to name the output directory, otherwise 0].
Several scans can also processed by giving a generic cxi name and scan numbers,
e.g.: data=data%03d.cx scan=13,45,78

gpu=Titan: GPU name for OpenCL calculation
[default = will be auto-selected from available GPUs]

xyrange=-2e-6,2e-6,-5e-6,-2e-6: range where the data points will be taken into
account (xmin, xmax, ymin, ymax). All scan positions outside this range are ignored.
This must be given in orginal coordinates, in meters.
[default=No spatial restrictions]


#### pynx-id01pty.py:¶

Script to perform a ptychography analysis on data from id01@ESRF

Example:
pynx-id01pty.py specfile=siemens.spec scan=57 detectordistance=1.3 ptychomotors=pix,piz,-x,y
probe=60e-6x200e-6,0.09 algorithm=analysis,ML**100,DM**200,nbprobe=3,probe=1

command-line arguments:
specfile=/some/dir/to/specfile.spec: path to specfile [mandatory]

scan=56: scan number in specfile [mandatory].
Alternatively a list or range of scans can be given:
scan=12,23,45 or scan="range(12,25)" (note the quotes)

imgcounter=mpx4inr: spec counter name for image number
[default=auto, will use mpx4inr or ei2minr]

[default: will be extracted from ULIMA_mpx4 or ULIMA_eiger2M the spec scan header]

nrj=8: energy in keV [default: will be extracted from UMONO spec scan header,  mononrj=x.xxkeV]

detectordistance=1.3: detector distance in meters
[mandatory, unless given in spec header UDETCALIB]

pixelsize=55e-6: pixel size on detector in meters [default:55e-6]

ptychomotors=pix,piz,-x,y: name of the two motors used for ptychography, optionally followed
by a mathematical expression to be used to calculate the actual
motor positions (axis convention, angle..). Values will be extracted
from the spec files, and are assumed to be in microns.
Example 1: ptychomotors=pix,piz
Example 2: ptychomotors=pix,piz,-x,y
Note that if the 'xy=-y,x' command-line argument is used,
it is applied after this, using 'ptychomotors=pix,piz,-x,y'
is equivalent to 'ptychomotors=pix,piz xy=-x,y'
[Mandatory]

monitor=opt2: spec name for the monitor counter. The frames will be normalized by the ratio
of the counter value divided by the median value of the counter over the entire
scan (so as to remain close to Poisson statistics). A monitor intensity lower
than 10% of the median value will be interpreted as an image taken without beam
and will be skipped.
[default = None]

kmapfile=detector/kmap/kmap_00000.edf.gz: if images are saved in a multiframe data file
This superseeds imgname=...

gpu=Titan: GPU name for OpenCL calculation [default = Titan]

livescan: if set, this keyword will trigger the analysis of all scans of a given type
(by default 'spiralscan') in the supplied spec file. If scan=NN is also given,
the analysis will start after scan #NN.
Once all scans are processed, the script will wait for new scans to arrive.


#### pynx-id13pty.py:¶

Script to perform a ptychography analysis on data from id13@ESRF

Example:
pynx-id13pty.py h5file=pty_41_61_master.h5 specfile=scan.dat scan=41 detectordistance=1.3
ptychomotors=pix,piz,-x,y probe=60e-6x200e-6,0.09
algorithm=analysis,ML**100,DM**200,nbprobe=3,probe=1 verbose=10 save=all saveplot liveplot

Command-line arguments (beamline-specific):
specfile=/some/dir/to/specfile.spec: path to specfile [mandatory]

scan=56: scan number in specfile [mandatory].
Alternatively a list or range of scans can be given:
scan=12,23,45 or scan="range(12,25)" (note the quotes)

h5data=entry/data/data_%6d: generic hdf5 path to stack of 2d images inside hdf5 file
[default=entry/data/data_%06d]

nrj=8: energy in keV [default: will be extracted from the HDF% file,
entry/instrument/beam/incident_wavelength]

detectordistance=1.3: detector distance in meters [mandatory]

pixelsize=75e-6: pixel size on detectorin meters
[default: will read from hdf5 data: entry/instrument/detector/x_pixel_size]

ptychomotors=nnp2,nnp3,-x,y: name of the two motors used for ptychography, optionally followed
by a mathematical expression to be used to calculate the actual
motor positions (axis convention, angle..). Values will be
extracted from the edf files, and are assumed to be in microns.
Example 1: ptychomotors=nnp2,nnp3
Example 2: ptychomotors=nnp2,nnp3,-x,y
Note that if the 'xy=-y,x' command-line argument is used, it is
applied after this, using 'ptychomotors=nnp2,nnp3,-x,y' is
equivalent to 'ptychomotors=nnp2,nnp3 xy=-x,y'
[Mandatory]

monitor=opt2: spec name for the monitor counter. The frames will be normalised by the ratio of
the counter value divided by the median value of the counter over the entire scan
(so as to remain close to Poisson statistics). A monitor intensity lower than 10%
of the median value will be interpreted as an image taken without beam and will be
skipped.
[default = None]

kmapfile=detector/kmap/kmap_00000.edf.gz: if images are saved in a multiframe data file.
This superseeds imgname=...

gpu=Titan: GPU name for OpenCL calculation [default = Titan]


#### pynx-id16apty.py:¶

Script to perform a ptychography analysis on data from id16(a)@ESRF

Example:
pynx-id16pty.py h5meta=data.h5 probe=60e-6x60e-6,0.09
algorithm=analysis,ML**100,DM**200,nbprobe=3,probe=1

Command-line arguments (beamline-specific):
h5meta=data.h5: hdf5 filename with the scan metadata. The scan number, energy,
detector distance will be extracted
[mandatory - if a single *.h5 file is in the working directory,
it will be used by default]

pixelsize=55e-6: pixel size on detector in meters

from 0 or 1, until no more frames are found, or maxframe
has been reached.
To process several scans, several % fields can be added, the last
will be interpreted as the image number and the others replaced by the
scan number.
Examples:
/dir/to/scan_%04d/prefix_scan%04d_img%05d.edf
/dir/to/scan_%04d/prefix__img%05d.edf
/dir/to/images/prefix_img%05d.edf
[mandatory if h5data is not supplied]

h5data=/path/to/hdf5/data/file.nxs: path to nexus data file (lambda detector).
If one or several % field are included in the name, they will be replaced by
the scan number.
Examples:
h5data=/path/to/hdf5/data/scan%d/scan%d.nxs
h5data=/path/to/hdf5/data/scan%d.nxs
h5data=/path/to/hdf5/data/data.nxs
[mandatory unless imgname is given, will default
to the only *.nxs file in directory]

ptychomotors=spy,spz,-x,y: name of the two motors used for ptychography, or the motor
positions file, optionally followed by a mathematical expression to
be used to calculate the actual motor positions (axis convention, angle..).
Values are either:
- extracted from the edf files, and are assumed to be in mm.
- or extracted from the 'mot_pos.txt' file, and assumed to
be in microns..
Example 1: ptychomotors=spy,spz
Example 2: ptychomotors=spy,spz,-x,y
Example 3: ptychomotors=mot_pos.txt
Note that if the 'xy=-y,x' command-line argument is used, it is
applied after this, using 'ptychomotors=spy,spz,-x,y'
is equivalent to 'ptychomotors=spy,spz xy=-x,y'
[Mandatory]

detectordistance=1.3: detector distance in meters [default: will be read from hdf5 metadata]

gpu=Titan: GPU name for OpenCL calculation [Mandatory]


## CDI¶

These are instructions to run the command-line scripts: pynx-id10cdi.py and pynx-id01cdi.py. They can also be used for generic data not from these beamlines.

This help text can be simply obtained by typing the script without any parameter.

### Generic instructions¶

Here are generic instructions applying to all the scripts:

generic (not beamline-specific) command-line/file parameters arguments: (all case-insensitive)

data=data.npy: name of the data file including the 3D observed intensity.
recognized formats include .npy, .npz (if several arrays are included iobs,
should be named 'data' or 'iobs'), .tif or .tiff
(assumes a multiframe tiff image), or .cxi (hdf5).
[mandatory unless another beamline-specific method to import data is used]

detector_distance=0.7: detector distance in meters

pixel_size_detector=55e-6: pixel size of the supplied data (detector pixel size)

wavelength=1.5e-10: wavelength in meters

verbose=20: the run will print and optionally plot every 'verbose' cycles

live_plot: if used as keyword (or live_plot=True in a parameters file), a live plot
will be shown  every 'verbose' cycle. If an integer number N is given,
display every N cycle.

gpu=Titan: name of the gpu to use [optional, by default the fastest available will be used]

auto_center_resize: if used (command-line keyword) or =True, the input data will be centered
and cropped  so that the size of the array is compatible with the (GPU)
FFT library used. If 'roi' is used, centering is based on ROI.
[default=False]

roi=0,120,0,235,0,270: set the region-of-interest to be used, with min,max given along each
dimension in python fashion (min included, max excluded), for the 2 or 3
axis. ROI coordinates should be indicated before any rebin is done.
Note that if 'auto_center_resize' is used, the ROI may still be shrunk
to obtain an array size compatible with the FFT library used. Similarly
it will be shrunk if 'max_size' is used but ROI size is larger.
Other example: roi=0,-1,300-356,300+256,500-256,500+256
[default=None]

nb_run=1: number of times to run the optimization [default=1]

nb_run_keep: number of best run results to keep, according to likelihood statistics. This is only useful
associated with nb_run [default: keep all run results]

data2cxi: if used as keyword (or data2cxi=True in a parameters file), convert the original
data to CXI(HDF5)  format. Will be saved to file 'data.cxi', or if a data file
has been supplied (e.g. data57.npz), to the same file with extension .cxi.

output_format='cxi': choose the output format for the final object and support.
Other possible choice: 'npz', 'none'
[Default='cxi']

note='This dataset was measure... Citation: Journal of coherent imaging (2018), 729...':
Optional text note which will be saved as a note in the output CXI file
(and also for data2cxi).

instrument='ESRF-idxx': the name of the beamline/instrument used for data collection
[default: depends on the script actually called]

sample_name='GaN nanowire': optional name for the sample

mask=zero: mask for the diffraction data. If 'zero', all pixels with iobs <= 0 will be masked.
If 'negative', all pixels with iobs < 0 will be masked.
If 'maxipix', the maxipix gaps will be masked.
If 'maxipix1', the maxipix gaps will be masked, except for the large pixel one ach side of the gap,
for which the intensity will be divided by 3.
If 'maxipix6', the maxipix gaps will be linearly interpolated between the gap large pixels.
Other possibilities: give a filename for the mask file and  import mask from .npy, .npz, .edf, .mat.
(the first available array will be used if multiple are present) file.
Pixels = 0 are valid, > 0 are masked. If the mask is 2D
and the data 3D, the mask is repeated for all frames along the first dimension (depth).

randomly initialised, but this can be used to load a mask from a file for
consistency between runs. It can be loaded from a .npy, .edf, .tif, .npz or
.mat file - if .npz or .mat are used, the first array matching the iobs shape is used.
A .cxi filename can also be used as input, in which case it is assumed that this
is the result of a previous optimisation, and that the mask can be loaded from

iobs_saturation=1e6: saturation value for the observed intensity. Any pixel above this intensity will be masked
[default: no saturation value]

zero_mask: by default masked pixels are free and keep the calculated intensities during HIO, RAAR, ER and CF cycles.
Setting this flag will force all masked pixels to zero intensity. This can be more stable with a large
number of masked pixels and low intensity diffraction data.
If a value is supplied the following options can be used:
zero_mask=auto: this is only meaningful when using a 'standard' algorithm below. The masked pixels will
be set to zero during the first 60% of the HIO/RAAR cycles, and will be free during the
last 40% and ER, ML ones.

object=obj.npy: starting object. Import object from .npy, .npz, .mat (the first available array
will  be used if multiple are present), or CXI file.
[default=None, object will be defined as random values inside the support area]

support=sup.npy: starting support. Import support from .npy, .npz, .edf, .mat (the first
available array will be used if multiple are present) file.  Pixels > 0 are in
the support, 0 outside. if 'auto', support will be estimated using the intensity
auto-correlation. If 'circle' or 'square', the suppport will be initialized to a
circle (sphere in 3d), or a square (cube). Ignored if support_formula is given.
[default='auto', support will be defined otherwise]

support_size=50: size (radius or half-size) for the initial support, to be used in
combination with 'support_type'. The size is given in pixel units.
Alternatively one value can be given for each dimension, i.e.
support_size=20,40 for 2D data, and support_size=20,40,60 for 3D data.
This will result in an initial support which is a rectangle/parallelepiped
or ellipsis/ellipsoid.  Ignored if support_formula is given.
[if not given, this will trigger the use of auto-correlation
to estimate the initial support]

support_formula: formula to compute the support. This should be an equation using ix,iy,iz and ir pixel coordinates
(all centered on the array) which can be evaluated to produce an initial support (1/True inside,
0/False outside). Mathematical functions should use the np. prefix (np.sqrt, ..). Examples:
support_formula="ir<100": sphere or circle of radius 100 pixels
support_formula="(np.sqrt(ix**2+iy**2)<50)*(np.abs(iz)<100)": cylinder of radius 50 and height 200

support_autocorrelation_threshold=0.1: if no support is given, it will be estimated
from the intensity autocorrelation, with this relative
threshold.
[default value: 0.1]

support_threshold=0.25: threshold for the support update. Alternatively two values can be given, and the threshold
will be randomly chosen in the interval given by two values: support_threshold=0.20,0.28.
This is mostly useful in combination with nb_run.
[default=0.25]

support_threshold_method=max: method used to determine the absolute threshold for the
support update. Either:'max' or 'average' or 'rms' (the default) values,
taken over the support area/volume, after smoothing.
Note that 'rms' and 'average' use the modulus or root-mean-square value normalised
over the support area, so when the support shrinks, the threshold tends to increase.
In contrast, the 'max' value tends to diminish as the optimisation
progresses. rms is varying more slowly than average and is thus more stable.
In practice, using rms or average with a too low threshold can lead to the divergence
of the support (too large), if the initial support

support_only_shrink: if set or support_only_shrink=True, the support will only shrink
(default: the support can grow again)

support_smooth_width_begin=2
support_smooth_width_end=0.25: during support update, the object amplitude is convoluted by a
gaussian with a size
(sigma) exponentially decreasing from support_smooth_width_begin
to support_smooth_width_end from the first to the last RAAR or
HIO cycle.
[default values: 2 and 0.5]

support_smooth_width_relax_n: the number of cycles over which the support smooth width will
exponentially decrease from support_smooth_width_begin to
support_smooth_width_end, and then stay constant.
This is ignored if nb_hio, nb_raar, nb_er are used,
and the number of cycles used
is the total number of HIO+RAAR cycles [default:500]

support_post_expand=1: after the support has been updated using a threshold,  it can be shrunk
or expanded by a few pixels, either one or multiple times, e.g. in order
to 'clean' the support:
- support_post_expand=1 will expand the support by 1 pixel
- support_post_expand=-1 will shrink the support by 1 pixel
- support_post_expand=-1,1 will shrink and then expand the support
by 1 pixel
- support_post_expand=-2,3 will shrink and then expand the support
by 2 and 3 pixels
- support_post_expand=2,-4,2 will expand/shrink/expand the support
by 2, 4 and 2 pixels
- etc..
[default=None, no shrinking or expansion]

support_update_border_n: if > 0, the only pixels affected by the support updated lie within +/- N pixels around the
outer border of the support.

support_update_period=50: during RAAR/HIO, update support every N cycles.
If 0, support is never updated.

positivity: if set or positivity=True, the algorithms will be biased towards a real, positive
object. Object is still complex-valued, but random start will begin with real
values. [default=False]

beta=0.9: beta value for the HIO/RAAR algorithm [default=0.9]

crop_output=0: if >0 (default:4), the output data will be cropped around the final
support plus 'crop_output' pixels. If 0, no cropping is performed.

rebin=2: the experimental data can be rebinned (i.e. a group of n x n (x n) pixels is
replaced by a single one whose intensity is equal to the sum of all the pixels).
Both iobs and mask (if any) will be rebinned, but the support (if any) should
correspond to the new size. The supplied pixel_size_detector should correspond
to the original size. The rebin factor can also be supplied as one value per
dimension, e.g. "rebin=4,1,2".
[default: no rebin]

max_size=256: maximum size for the array used for analysis, along all dimensions. The data
will be cropped to this value after centering. [default: no maximum size]

user_config*=*: this can be used to store a custom configuration parameter which will be ignored by the
algorithm, but will be stored among configuration parameters in the CXI file (data and output).
e.g.: user_config_temperature=268K  user_config_comment="Vibrations during measurement" etc...

############# ALGORITHMS: standard version, using RAAR, then HIO, then ER and ML

nb_raar=600: number of relaxed averaged alternating reflections cycles, which the
algorithm will use first. During RAAR and HIO, the support is updated regularly

nb_hio=0: number of hybrid input/output cycles, which the algorithm will use after RAAR.
During RAAR and HIO, the support is updated regularly

nb_er=200: number of error reduction cycles, performed after HIO, without support update

nb_ml=20: number of maximum-likelihood conjugate gradient to perform after ER

detwin: if set (command-line) or if detwin=True (parameters file), 10 cycles will be performed
at 25% of the total number of RAAR or HIO cycles, with a support cut in half to bias
towards one twin image

############# ALGORITHMS: customized version

algorithm="ER**50,(Sup*ER**5*HIO**50)**10": give a specific sequence of algorithms and/or
parameters to be
used for the optimisation (note: this string is case-insensitive).
Important:
1) when supplied from the command line, there should be NO SPACE in the expression !
And if there are parenthesis in the expression, quotes are required around the
algorithm string
2) the string and operators are applied from right to left

Valid changes of individual parameters include (see detailed description above):
positivity = 0 or 1
support_only_shrink = 0 or 1
beta = 0.7
live_plot = 0 (no display) or an integer number N to trigger plotting every N cycle
support_update_period = 0 (no update) or a positive integer number
support_smooth_width_begin = 2.0
support_smooth_width_end = 0.5
support_smooth_width_relax_n = 500
support_threshold = 0.25
support_threshold_mult = 0.7  (multiply the threshold by 0.7à
support_threshold_method=max or average or rms
support_post_expand=-1#2 (in this case the commas are replaced by # for parsing)
verbose=20
fig_num=1: change the figure number for plotting

Valid basic operators include:
ER: Error Reduction
HIO: Hybrid Input/Output
RAAR: Relaxed Averaged Alternating Reflections
DetwinHIO: HIO with a half-support (along first dimension)
DetwinHIO1: HIO with a half-support (along second dimension)
DetwinHIO2: HIO with a half-support (along third dimension)
DetwinRAAR: RAAR with a half-support (along first dimension)
DetwinRAAR1: RAAR with a half-support (along second dimension)
DetwinRAAR2: RAAR with a half-support (along third dimension)
CF: Charge Flipping
ML: Maximum Likelihood conjugate gradient (incompatible with partial coherence PSF)
PSF or EstimatePSF: calculate partial coherence point-spread function
with 50 cycles of Richardson-Lucy
Sup or SupportUpdate: update the support according to the support_* parameters
ShowCDI: display of the object and calculated/observed intensity. This can be used
to trigger this plot at specific steps, instead of regularly using
live_plot=N. This is thus best used using live_plot=0

Examples of algorithm strings, where steps are separated with commas (and NO SPACE!),
and are applied from right to left. Operations in a given step will be applied
mathematically, also from right to left, and **N means repeating N tymes (N cycles)
the  operation on the left of the exponent:
algorithm=HIO : single HIO cycle

algorithm=ER**100 : 100 cycles of HIO

algorithm=ER**50,HIO**100 : 100 cycles of HIO, followed by 50 cycles of ER

algorithm=ER**50*HIO**100 : 100 cycles of HIO, followed by 50 cycles of ER

algorithm="ER**50,(Sup*ER**5*HIO**50)**10" :
10 times [50 HIO + 5 ER + Support update], followed by 50 ER

algorithm="ER**50,verbose=1,(Sup*ER**5*HIO**50)**10,verbose=100,HIO**100":
change the periodicity of verbose output

algorithm="ER**50,(Sup*ER**5*HIO**50)**10,support_post_expand=1,
(Sup*ER**5*HIO**50)**10,support_post_expand=-1#2,HIO**100"
same but change the post-expand (wrap) method

algorithm="ER**50,(Sup*PSF*ER**5*HIO**50)**5,(Sup*ER**5*HIO**50)**10,HIO**100"
activate partial correlation after a first series of algorithms

algorithm="ER**50,(Sup*PSF*HIO**50)**4,(Sup*HIO**50)**8"
typical algorithm steps with partial coherence

algorithm="ER**50,(Sup*HIO**50)**4,(Sup*HIO**50)**4,positivity=0,
(Sup*HIO**50)**8,positivity=1"
same as previous but starting with positivity constraint, removed at the end.

[default: use nb_raar, nb_hio, nb_er and nb_ml to perform the sequence of algorithms]

save=all: either 'final' or 'all' this keyword will activate saving after each optimisation
step (comma-separated) of the algorithm in any given run [default=final]


### Specific instructions¶

Instructions for beamline-dedicated scripts can be fully obtained by executing the script without any parameter.

#### pynx-id10cdi.py:¶

Script to perform a CDI reconstruction of data from id10@ESRF.
command-line/file parameters arguments: (all keywords are case-insensitive):

Specific defaults for this script:
auto_center_resize=True
detwin = False
nb_raar = 600
nb_hio = 0
nb_er = 200
nb_ml = 0
support_size = 50
support_type = circle
positivity = True


#### pynx-id01cdi.py:¶

Script to perform a CDI reconstruction of data from id01@ESRF.
command-line/file parameters arguments: (all keywords are case-insensitive):

specfile=/some/dir/to/specfile.spec: path to specfile [mandatory, unless data= is used instead]

scan=56: scan number in specfile [mandatory].
Alternatively a list or range of scans can be given:
scan=12,23,45 or scan="range(12,25)" (note the quotes)

imgcounter=mpx4inr: spec counter name for image number
[default='auto', will use either 'mpx4inr' or 'ei2mint']

imgname=/dir/to/images/prefix%05d.edf.gz: images location with prefix
[default: will be extracted from the ULIMA_mpx4 entry in the spec scan header]

Specific defaults for this script:
auto_center_resize = True
detwin = True
nb_raar = 600
nb_hio = 0
nb_er = 200
nb_ml = 0
support_size = None