PyNX: Python tools for Nano-structures Xtallography

NEWS

  • 2023/02/03: PyNX 2023.1 is available. See the full Changelog

  • 2022/02/06: PyNX 2022.1 is available.

  • 2020/12/12: PyNX 2020.2.2 is available with a few fixes.

  • 2020/10/23: PyNX 2020.2 is available, notably introducing distributed computing on multiple GPU

  • 2020/09: the PyNX article: PyNX: high-performance computing toolkit for coherent X-ray imaging based on operators is out: J. Appl. Cryst. 53 (2020), 1404, also available as arXiv:2008.11511

  • 2020/02/02: version 2020.1 is out !

  • 2019/06/19: version 2019.2.6 is out. Note that 2019.2.x (x<6) versions had incorrect Ptycho CUDA scaling, preventing correct ML minimisation

  • 2018/06: all coherent imaging modules have now been converted to use an operator-based API

  • 2016/09: overhaul of the library structure - pynx.gpu has now become pynx.scattering, with pynx.scattering.gid as a sub-module.

Introduction

PyNX stands for Python tools for Nano-structures Xtallography. It can be used for:

  • Coherent X-ray imaging simulation and analysis: coherent diffraction imaging (CDI), Ptychography, Wavefront propagation, near field and far field techniques…

  • Fast scattering calculations from large number of atoms and reciprocal space positions.

PyNX is fully optimised to use Graphical Processing Units, using either CUDA or OpenCL, to provide fast calculations with 1 to 3 orders of magnitude speedup compared to standard processor calculations.

PyNX scripts

PyNX can be used simply with command-line scripts for some applications (2D/3D CDI and 2D Ptychography). These can take generic files as input, such as CXI files (http://cxidb.org), or can analyse data directly from beamlines.

PyNX as a python toolkit

PyNX can be used as a python library with the following main modules:

  1. pynx.scattering: X-ray scattering computing using graphical processing units, allowing up to 2.5x10^11 reflections/atoms/seconds (single nVidia Titan X). The sub-module``pynx.scattering.gid`` can be used for Grazing Incidence Diffraction calculations, using the Distorted Wave Born Approximation

  2. pynx.ptycho : simulation and analysis of experiments using the ptychography technique, using GPU (OpenCL). Examples are available in the pynx/Examples directory. Scripts for analysis of raw data from beamlines are also available, as well as using or producing ptychography data sets in CXI (Coherent X-ray Imaging) format.

  3. pynx.wavefront: X-ray wavefront propagation in the near, far field, or continuous (examples available at the end of wavefront.py). Also provided are sub-modules for Fresnel propagation and simulation of the illumination from a Fresnel Zone Plate, both using OpenCL for high performance computing.

  4. pynx.cdi: Coherent Diffraction Imaging reconstruction algorithms using GPU for Coherent Diffraction Imaging, in 2D or 3D, for small-angle or Bragg diffraction data. This uses either CUDA or OpenCL, but CUDA is strongly recommended for 3D data (significant speedup).

Download

PyNX is available from:

Installation

The simplest way to install PyNX (usually in an existing virtual or conda environment) is to use (in this case with both cuda and OpenCL backends):

curl -O http://ftp.esrf.fr/pub/scisoft/PyNX/pynx-latest.tar.bz2
pip install pynx-latest.tar.bz2[cuda]

For more installation options, see the detailed installation instructions (preferably using a python virtual environment)

Changelog

See the full Changelog:

Version 2023.1 (2023-02-03)

Major changes

  • command-line scripts have been renamed to be more standard and without a .py extension, e.g. pynx-ptycho-cxi instead of pynx-cxiptycho.py

  • Ptycho (near field): added Paganin operator to estimate the initial object, using a reference frame with the direct beam

  • Ptycho: if a reference frame (direct beam) is given, it can be also used for optimisation, yielding an absolute reference for the probe.

  • Ptycho (API): the intensity mask should not be supplied anymore, but the iobs data should have masked data as negative values. For near field, padded data must be extrapolated and stored as -1-I_extrapolated.

  • Ptycho: add cxi_output command-line option to e.g. only save the object phase and save space.

  • CDI: correct CUDA random initialisation of object which was not-so-random

Mailing list, git & issue tracker

There is a mailing list for PyNX: to subscribe, send an email to pynx-subscribe@esrf.fr

To access the git repository and the issue tracker, you can create an account on http://gitlab.esrf and ask the developers for access to the PyNX project at http://gitlab.esrf.fr/favre/PyNX

Citation & Bibliography

If you use PyNX for scientific work, please consider including a citation:

  • If you use PyNX for coherent X-ray Imaging including CDI and ptychography:

  • If you use PyNX for GPU scattering calculations:

  • Cite the first PyNX article: J. Appl. Cryst. 44(2011), 635-640. A preprint version is also available on ArXiv:1010.2641

  • Give a link to the project: http://ftp.esrf.fr/pub/scisoft/PyNX/

PyNX re-uses or was inspired by features described in the following articles and open-source software packages:
  • PtyPy: 1. B. Enders and P. Thibault, “A computational framework for ptychographic reconstructions”, Proc Math Phys Eng Sci 472(2196), (2016).

  • M. Odstrčil, A. Menzel, and M. Guizar-Sicairos, “Iterative least-squares solver for generalized maximum-likelihood ptychography,” Optics Express 26(3), 3108 (2018).

  • S. Marchesini, A. Schirotzek, C. Yang, H. Wu, and F. Maia, “Augmented projections for ptychographic imaging,” Inverse Problems 29(11), 115009 (2013).

    1. Thibault and A. Menzel, “Reconstructing state mixtures from diffraction measurements,” Nature 494(7435), 68–71 (2013).

    1. Thibault and M. Guizar-Sicairos, “Maximum-likelihood refinement for coherent diffractive imaging,” New J. Phys. 14(6), 063004 (2012).

  • J. N. Clark, X. Huang, R. Harder, and I. K. Robinson, “High-resolution three-dimensional partially coherent diffraction imaging,” Nat Commun 3, 993 (2012).

  • S. Marchesini, “A unified evaluation of iterative projection algorithms for phase retrieval,” Review of Scientific Instruments 78(1), 011301 (2007).

License

The PyNX library is distributed with a CeCILL-B license (an open-source license similar to the FreeBSD one). See http://www.cecill.info/licences/Licence_CeCILL-B_V1-en.html

Note that CPU computing of the pynx.scattering module uses the sse_mathfun.h header, which is distributed under the zlib license. See http://gruntthepeon.free.fr/ssemath/

See http://ftp.esrf.fr/pub/scisoft/PyNX/README.txt for more details about the license, copyright, as well as other possible issues regarding ptychography.

Automated testing

To automatically test PyNX after installation, you can run the pynx-test script, which will run a series of tests and can help diagnose issues specific to GPU languages (OpenCL, CUDA), dependencies or applications (CDI, Ptycho..). Alternatively you can run pytest (if installed) from the root of the PyNX source directory.

Beginner tutorials

To begin using PyNX, you can read the following Tutorials:

Command-line scripts

Scripts Reference

Documentation of scripts included in PyNX

API Documentation

API Reference

Documentation of modules included in PyNX

Indices and tables