X-ray data integration

Data Integration and Analysis for Synchrotron and FEL crystallography

Structural biology and in particular the productivity of X-ray crystallography is developing at an increasingly rapid pace. High brilliance beamlines at 3rd generation synchrotron sources coupled to high frame-rate pixel array detectors and increasing levels of automation in both hardware and software are transforming macromolecular crystallography and allowing researchers to address ever more challenging problems [1, 2]. Furthermore the recent exciting demonstration of FEL nano-crystallography [3] has opened the door to a new era of structural biology at large facilities. These advances and the incredibly fast rate of development provide new challenges and opportunities in the analysis of raw diffraction data generated at both synchrotrons and FELs. Our joint research activity in Work Package 6 is aimed at addressing these challenges and opening up new doors to future collaboration between FELs and synchrotrons.

In this work package we will produce data analysis software that is forward compatible and can keep pace with the changes in the way we measure and analyse data. We will develop a diffraction data integration program that will be optimised for modern pixel array detectors, will work at very high speeds in order to keep pace with the high rates of data collection, will handle data from crystal samples with very high mosaicity and be able to deal with overlapping diffraction spots. The handling of coherence effects in the case of FEL data will be explored.

We will attempt, where possible, to unify the analysis of synchrotron and FEL data and develop tools to manage and handle very large numbers of diffraction images from large numbers of samples.
The software will be open source and flexible enough to enable straightforward development in the future by contributors from multiple sites. The new software will be designed and programmed using a modular, extensible architecture suitable for implementation within fully or semi automated pipelines. The new software will be available at all MX beamlines under this project as well as at researchers’ home institutions, and as such will enable easy and effective transnational use of the BioStruct-X project facilities.

[1] Evans, G., Axford, D., Waterman, D. & Owen, R. L. (2011). Cryst. Reviews 17, 105 - 142.
[2] Winter, G. & McAuley, K. E. (2011). Methods, (In press).
[3] Chapman, H. N. et al. (2011). Nature 470, 73-77.

Objectives summary:

  • To develop new MX data integration software optimized for next generation pixel array detectors working at very high data acquisition speeds.
  • To highlight and nurture common activities between Synchrotron and FEL crystallography via a common MX data analysis platform.
  • To develop new algorithms for parameterized profile fitting, spot deconvolution and handling of highly mosaic crystals.
  • To develop a modular extensible architecture suitable for implementation within fully automated pipelines together with tools for interactive and diagnostic analysis of problem cases.
  • To implement these developments at BioStruct-X MX beamline facilities.

Main tasks:

  1. Development of common frameworks for data organisation, integration and results inspection for both synchrotron and FEL crystallography
  2. Development of key algorithms modules for new integration program optimised for PAD detectors and challenging crystallographic cases
  3. Development of data simulation software and visualisation tools for both raw data inspection and results inspection. Systematic count rate optimisation studies and new hybrid detector technology tests
  4. Development of integration pipeline to assist in communication with up and down stream analysis programs (DIAMOND).
  5. Release of integration program to all partner sites for testing


Diamond, CFEL, XFEL, EMBL-Grenoble, PSI, Dectris, BESSY-HZB, EMBL-Hamburg, ESRF, ALBA, SOLEIL, MAX-Lab, Elettra

In addition, we are collaborating closely with Software developers from the Collaborative Computational Project No. 4 (CCP4) at the Rutherford Appleton Laboratory, UK and from the MRC Laboratory of Molecular Biology in Cambridge (MRCLMB).

Home | Disclaimer | Contact

BioStruct-X is part of the Capacities Specific Programme Research Infrastructures, Project Number 283570
BioStruct-X contact: biostructx(at)embl-hamburg.de, I. Custic, ph: +49 - 40 89902 124, fax: +49 - 40 89902 149

eu fp7 Collaborator: instruct