Tristan—A 10 Million Pixel Large Area Time Resolved Detector for Synchrotron Use

This article describes the development of the Tristan 10 M detector for time-resolved synchrotron experiments. Tristan 10 M has an unprecedented time resolution (ns time scale) over long-duration continuous acquisition (days). The detector is constructed from an array of 160 Timepix3 readout application-specified integrated circuit (ASIC; about 10 million pixels) flip chip bonded to ten monolithic silicon sensors, which enable it to cover an area large enough to effectively carry out crystallography experiments. The large array of ASICs resulted in a number of severe technical challenges that had to be overcome during the development of the detector. The minimization of the dead area between sensors required the development of a very challenging mechanical and electronic packaging. Such a packaging had to be able to route the large number of data and power lines within the footprint of a sensor, had to effectively sink the heat generated by the ASICs, and had to be able to position the sensors accurately. In addition, the packaging of the detector was designed to be scalable in consideration of possible future larger versions of this detector, which added a further challenge. The data-driven nature of Timepix3 and the sheer data volume produced by the array of ASICs required us to devise a dedicated hardware, firmware, and software data acquisition architecture. This architecture proved very effective during the commissioning of Tristan 10 M when time-resolved crystallography experiments were carried out.


I. INTRODUCTION
H YBRID photon counting detectors (HPCDs) have become a widely deployed technology at synchrotrons and are used in a wide variety of experiments. A recent review [1] described the transformative effect that the introduction of this technology had on synchrotron experiments. One of the major reasons for this transformation is the much higher frame rate that HPCD can offer with respect to other detector technologies, such as, for instance, scintillating plates coupled to charge-coupled device (CCD) light sensors. Certain HPCD can nowadays achieve frame rates in excess of 20 kHz [2]. This enabled effective studying of the dynamics of phenomena happening in the order of milliseconds to be initiated.
In order to be able to measure dynamics with time scales which are considerably shorter than a millisecond, the detector can be gated. That means that the counters are enabled only when an appropriate logic signal (the gate) is high. The gate width can be of the order of several tens of nanoseconds. A typical pump and probe experiment can exploit the gating of the detector to measure the dynamics of the order of a microsecond and below by scanning the gate delay with respect to a reference signal [3]. For example, the time variation of the characteristics of a sample excited with a laser pulse (the pump) can be measured with this technique by measuring the X-ray diffraction patterns (the probe). However, this mode of operation comes to the expenses of efficiency because the detector is not active for most of the time resulting in photons produced by the synchrotron being wasted. This has a big impact on the practical duration of the experiment; however, this method can also make an experiment unviable if the samples are subject to radiation damage and the dose exceeds what the samples can withstand. Finally, the gating technique cannot work if the measurement to be carried out is not cyclic as it is in pump and probe experiments, but it is a one-off.
In order to be able to efficiently study dynamics of the order of a microsecond or shorter, a different concept needs to be applied to HPCD, and this concept is offered by the Timepix3 application-specified integrated circuit (ASIC) [4]. Timepix3 was designed for particle tracking where data are This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ sparse. The read-out channels (pixels) of Timepix3 are organized in a square matrix of 256 × 256 pixels with 55 µm pitch. Timepix3 timestamps the time of arrival of an event with a notional accuracy of 1.5625 ns. When an event is detected, it is timestamped, and the timestamp and location (pixel coordinates) of the event are immediately output. Sparse timestamped data allow the Timepix3 to supply data where it exists without the need to readout a frame of largely zero counts. This mode of operation enables the building of a large area HPCD capable of efficiently measuring very fast dynamics. The effective time resolution of Timepix3 bonded to a silicon sensor was measured a few years ago by the detector group at diamond light source (DLS) [5], and the best time resolution measured was 8-ns standard deviation.
The results on the time resolution of Timepix3 on a demonstration system [6], [7], [8] convinced DLS to fund the development of Tristan, a large area detector for time-resolved experiments. The first application that DLS decided to target is time-resolved crystallography. To obtain an area large enough for the use case 160, Timepix3 ASICs have to be tiled. In this article, we describe the detector architecture together with the challenges and the outcome of such a development project.

A. Sensors and Hybrid Fabrication
The sensors used in the Tristan project are 500-µm-thick silicon sensors already used in the Excalibur project [9]. This was possible because Timepix3 was designed with the same pixel pitch and dimensions as Medipix3, which is the ASIC used by Excalibur. Thus, 16 Timepix3 ASICs are bump bonded to an individual sensor in an 8 × 2 matrix, giving a total of 2048 × 512 pixels, that is approximately one million pixels per sensor.
In order to match the pitch of Timepix3, the size of the pixels of the sensors is 55 × 55 µm 2 ; however, the pixels at the boundary between two ASICs are 2.5× larger leading to pixel dimensions of 137.5 × 55 µm 2 ; the corner pixels between four ASICs have also the other dimension extended 2.5× giving a pixel size of 137.5 × 137.5 µm 2 . Fig. 1 shows the details of the boundary pixels. In practice, this means that every inter-ASIC gap adds the length of three standard pixels to the sensitive area. The sensitive area is then equivalent to 2069 × 515 standard pixels that is a size of 113.795 × 28.325 mm 2 . The dimensions of the silicon sensor dice are approximately 115.7 × 30.2 mm 2 as they include space for the guard rings and a safety margin for the dicing streets.
Timepix3 has two sets of wire bonding pads: an external set, which is rather extended, and an internal set, which is smaller and closer to the chip periphery. When the sensor was originally used with Medipix3, it covered the internal set, but Timepix3 has slightly larger periphery electronics, so we found that the internal wire bonding pads just protrude the edge of our sensors. We decided to use these pads for wire bonding the ASICs. The external wire bonding pads are 0.87 mm long. By removing them, we could reduce the module to module gap. The Tristan hybrids are fabricated with ASICs without   2. Tristan sensor assembly sketch shows the hybrid glued on the cold finger plate and the two separate PCBs, which are recessed to fit round the cold finger. Clamps hold the PCBs in place and prevent delamination of the assembly should it need to be unplugged. the external wire bonding pads, which are removed when the ASICs are diced.
These sensor modules are designed to be mounted co-planar to give a flat detection surface. The roof tile approach, which could eliminate intermodule gaps as used in detectors such as Widepix [10], was rejected. Roof tiling requires the detector to be assembled in sequence. Our prior experience has shown us that this leads to additional effort in assembly, test, and maintenance. The sensor was already available, so we chose to stay with this approach.

B. Module Front-End
Each Timepix3 ASIC can dissipate up to 2 W, and therefore, an effective method to remove the heat is necessary to guarantee the stability of the operating parameters. Thus, the hybrids are glued directly on an aluminum cold finger, whose function is both to remove the heat by conduction to the following cold plates and to act as an accurate positioner for the sensors. The cold finger is T-shaped with a plate connected to a stem, which is mounted to the back end of the module (see Fig. 2). The plate dimensions are 115.6 mm × 30.84 mm just enough to cover the dimensions of the sensor and the portion of the ASICs, which protrudes the sensor. The thickness of the plate is 1.5 mm, which is calculated to guarantee both sufficient stiffness and heat conduction capability. Two accurately machined dowels on the back of the plate plug onto a mechanical framework with an accuracy of a few micrometers.
During the module fabrication, the sensors are accurately positioned on the cold finger surface by a robot controlled by an optical system. The accuracy in the positioning is achieved by using fiducials on the surface of the cold finger. The fiducials are machined with the accuracy of a few micrometers with respect to the dowels, which were mentioned before.
The printed circuit boards that route the power and data lines of the ASICs to the following electronics need to be glued on the back surface of the plate. However, it is very difficult to connect the pads on the ASICs by wire bonds to the pads on a board, which is located over 2 mm (thickness of base plate plus the thickness of the ASIC) below, without excessively extending the lateral dimension of the board. Instead of fabricating a planar board, L-shaped boards were fabricated by milling out a considerable section of the board. This enabled the bonding pads on the board to be brought close to the level of the pads on the ASICs (see Fig. 2).
The bonding pad pattern of a test board for a single ASIC is shown in Fig. 3. Eight identical patterns are necessary for the boards of a module. The pads for the data lines are round because they are easier to etch accurately, and they are arranged in a hexagonal matrix to optimize the packing [11]. The resultant pad structure had no room to track out to through hole vias, so micro via in pad structures was used. The micro vias connect to conventional vias to route the electrical lines to the connector (Samtec BSH-150-01-F-D-A) on the back of the board. Both the design and the fabrication of the boards were very challenging, and we had to make several attempts with more than one manufacturer before one of them succeeded in delivering boards, which met the specification with a good yield. This is a testament to the challenge of these boards.
Wire bonding between the ASIC and the PCB required the use of 20-µm wire rather than the standard 25-µm wire. This was primarily due to the use of the inner bond pads on the Timepix3 ASIC. These pads, being smaller than the extended pads, means that there is no scope for rework.

C. Module Back-End, Mechanical Structure, and Heat Sink
Timepix3 needs a series of regulated voltages, and the sensor needs high-voltage bias. It is impossible to accommodate the voltage regulators and other electronic parts on the boards on the cold finger; therefore, two point of load (POL) regulator cards were developed. In addition to voltage regulators, these boards have fan-out buffers for signals such as clocks, which reduce the required connection count. A high-voltage module was also included to avoid the complication of routing high voltage within the detector.
Each of the two chip boards is connected to a POL board, and the two POL boards share a common front-end design with the difference being that one plugs directly to the data acquisition card on the back, while the second plugs into the first. The POL boards route all the 128 low-voltage differential signaling (LVDS) data lines coming from the 16 Timepix3 ASICs (eight per ASIC) plus clock and control lines to the back-end connector.
The two POL boards are mounted together on a frame that also holds the sensor assembly cold finger, as shown in Fig. 4. This frame acts as the primary heat transfer path for the sensor and the regulators. The resultant sensor module slides in between water cooled plates mounted in the chassis. The heat is extracted from the side of the POL frame. It was found in simulation that drawing heat back from the sensor cold finger into the POL frame and then drawing heat sideways gave a good even heat distribution across the sensor (see Fig. 5). If heat was drawn too close to the cold finger, then there was a higher thermal variation across the sensor area.
The necessary accuracy in the module positioning cannot be provided by the cold plates; thus, the sensors are aligned mechanically to a precision frame using dowels into the cold fingers. With this arrangement, the modules are positioned with a vertical pitch of 34.76 mm (632 pixels) and a horizontal pitch of 116.27 mm (2114 pixels). The gap between two  contiguous sensitive areas is then 117 pixels (6.435 mm) vertically and 45 pixels (2.475 mm) horizontally.
With this structure, ten modules arranged in a 5 × 2 matrix enable the tiling of the required 160 Timepix3 ASICs. The front-end of the Tristan detector is shown in Fig. 6.

D. Data Acquisition Cards, Clock and Trigger Electronics, Power Electronics, and IT Equipment
The data acquisition card immediately on the back of the two POL boards was designed by Science and Technology Facilities Council (STFC), and it is known as FEM-II. The FEM-II provides a Virtex7 690T field-programmable gate array (FPGA) with DDR3 and a Zynq-7020 system on chip as an on-board control processor. It has a high count of LVDS differential IO, which enable to process all the data lines coming from the Timepix3 ASICs in parallel. Two Quad Small Form Factor Pluggable (QSFP) sockets for fiber optic transceivers are used as network interfaces. Each QSFP can support either 40-Gb/s Ethernet or four 10-Gb/s Ethernet links. The current version of the Tristan detector uses one QSFP socket with four 10-Gb/s Ethernet links. Finally, an Ethernet 1-Gbit/s connector provides access for controls and diagnostic data. Each FEM-II card connected to a module is an independent detector. This enabled building two prototypes with a single module to test the various aspects of the development before committing to the production of the modules necessary to populate the full detector.
The FEM-II is connected to the module through a connector (FCI Zipline), and it is designed, so as its dimensions do not exceed the cross section of the module. In this way, the cross section of the detector is limited only by the size of the modules. The modules can be then stacked in arbitrary numbers in both vertical and horizontal directions.
A card developed specifically for the Tristan project and known as a clock and trigger card (CTC) provides the common clock to all the FEM-II cards to maintain multiple acquisition subsystems in lockstep. In addition, the CTC provides the function to zero the detector time base to a common point and it timestamps external signals. The CTC uses an ovencontrolled crystal oscillator to provide a highly accurate clock, and it has a Zynq module to provide the programmable logic. There is the provision to connect an external clock source that, for instance, enables the use of the synchrotron beam clock. External triggers are first synchronized to the clock trigger card internal clock and presented synchronously to all FEM-IIs. This is important to ensure that all sensor modules are perfectly in sync. Not synchronizing at a single common point could lead to individual FEM-IIs resolving the asynchronous trigger on different clock edges. Other inputs are one for the signal to zero the detector and one input for signals generated by external events, which can be timestamped. The time zero signal can also be generated internally by the programmable logic and triggered by a software command.
The detector is fed with 48 V, allowing the use of standard off the shelf mains to 48-V power supplies housed externally. The 48 V powers up five intermediate bus converter (dc to dc converters), which are carried by two specific boards known as power boards. A power board provides the low voltages needed to power up all the subsystems; 12 V is generated to power up the FEM-II cards and the POLs, and this was adopted for most internal power distribution; 12 V is a good balance of low voltage, with the ability to deliver the required power without excessive current requirements for the interconnect.
The four 10-Gb/s Ethernet link of each FEM-II card are used to transmit the data with a User Datagram Protocol (UDP) protocol to an array of ten Linux servers each with a 40-G Ethernet network card (Fig. 7 shows an outline of the system). These servers run Odin data acquisition software, which is described later. A network switch supporting both 40 and 10 Gb/s Ethernet ports is used between the detector and the servers. This provides flexibility in the configuration but adds a significant cost and might be eliminated in certain circumstances. In the current 10 M configuration it takes 40 and 10 Gb/s Ethernet links from the detector and distributes data to servers using 40 Gb/s Ethernet links. How the data are addressed to the servers is explained in the firmware section. The Linux servers are then connected to a General Parallel File System (GPFS) file system through an InfiniBand connection.
An independent control network uses 1 Gb/s copper Ethernet to link the FEM-II to the CTC and the CTC to the Odin control plane. Small off the shelf network switches are fit within the detector. This cuts the external control network down to one single standard patch cable which was considered better than a relatively bulky cable and an external switch.

III. FIRMWARE ARCHITECTURE
The firmware is divided between the CTC card and the FEM-II cards. Each has a programmable logic part written in VHDL and a control part providing the application programming interface (API) and control program written in C ++ running on the Zynq Arm A9 processors. All the hard real time processing is done in programmable logic, which eases the requirements on the C ++ code. Embedded Linux is used as the operating system on the Zynq as it has a good TCP stack for the control API and supports multi-threading. When designing the CTC, the same Zynq module used on FEM-II was chosen as it allowed a single Linux port to be used across the detector components.
The firmware in the FEM-II cards provides the following functions: configuration and sequencing of the Timepix3 ASICs, acquisition of data from the ASICs, data format and extension of the time base and pixel address, and transmission to the data acquisition PC. The firmware in the CTC provides the overall acquisition control for the detector as well as synchronizing and timestamping trigger signals. It has no direct role in the acquisition of data.

A. Timepix3 Data Acquisition and Processing Firmware in FEM-II
The FEM-II Virtex7 690T FPGA provides the core of the data acquisition function. In principle the task given to the firmware is simple: collect data from sixteen Timepix3 ASICs, combine the data and transmit it to external servers. In practice the aggregate data rates involved make this quite challenging. Sixteen Timepix3 chips with eight data lines each running at 640 Mb/s 8 B 10 B encoded data yields a potential payload of 56 Gb/s once decoded, and taking into account the comma symbols that can be immediately removed. This is an upper figure as it represents the maximum possible data output of all chips. Timepix3 uses Linear Feedback Shift Register (LFSR) and Gray code counters internally, so these are converted to binary in the FPGA firmware.
Before we combine the data from multiple Timepix3, there are two important details that we must consider, both of which lead to an increase in the data size. Firstly, the Timepix3 internal timestamp counter (including course and fine parts totalling 18-bits) operating at 640 MHz rolls over every 409.6 µs. For the detector to be able to acquire for an arbitrary time the firmware extends this field to 52 bits allowing for 81 days of continuous measurement. There is a variable delay from a packet being generated in a pixel to it emerging from the Timepix3 which is dependent on the pixel location and on the amount and location of other data generated in response to photon hit events in other pixels.
A state machine (see Fig. 8) tracks how the packet delay changes over time. The 409.6 µs timestamp of the Timepix3 is divided into four time-windows using the upper two bits of the Timepix3 counter. These are compared against the FPGA's own time base to work out how delayed the packet is and determine the correct extension to add. The states indicate if the emerging packet is aligned with the current time-window, one, two or three windows behind the current clock, or if the link is idle with no data. Any transition to three time-windows behind will trigger a warning. A transition from three behind to aligned must be treated as an error as it will be impossible to tell if the packet has been held up to four time-windows behind. No data is discarded, and the packet will be time extended as aligned. It is for the data analysis to decide how to process the warning and error messages and resultant data. The presence of any warning is an indication that the experimental conditions are close to the limits of the detector.
Secondly, each Timpix3 event packet contains the pixel coordinates within the single ASIC so we must identify which chip originated the packet. This is a much easier task than extending the timestamp and with a little more work we can even remap the pixels of those chips that are rotated 180 • so the whole image has the same orientation. This also accommodates ASICs rotated 90 • or 270 • , something we have not used yet but might do in future. By adding arbitrary offsets, rather than a simple extension we can account for the three-pixel gaps between ASICs within a sensor and the larger gaps between modules. Pixels can thus appear in their correct relative position without requiring software remapping.
With every packet correctly timed and located, it is now possible to combine data streams from different ASICs. Sixteen ASICs are combined into eight streams. This is a compromise to ensure that we keep within the operating frequency of the FPGA logic while not duplicating more logic than necessary.

B. Data Routing and Network Output Firmware on FEM-II
Processing within the FPGA aims to do the data manipulation that is easy within logic, but expensive within software. We can maintain an arbitrary bit width for the data packets in the FPGA, but we need to consider standard data sizes when we transfer this to software. Here we have the option of rounding up to the next 2^n size 128 bits, or round down to 64 bits. We chose 64 bits as the best compromise, not expanding the data too much. This size accommodates the extended pixel coordinates, but not the extended time stamp. Truncating the time sounds like a retrograde step having just extended it, but we are now able to control the data stream so that reapplying the extension is not such a difficult process. Data is time sliced and presented to the server as sets of data belonging to discrete time intervals, or time slices (see Fig. 9). Within each time slice that same time extension can be applied, and this is provided with each block of data. The time slice is not related to any experimental parameter, but it does provide a limit on the disorder of data.
Each time slice collects its own queue of data and assembles full UDP datagrams before transmission to its selected Odin Frame receiver process, the details of which will be given in the software section below. Before a time slice is needed again, any remaining data are flushed in a short datagram, and then, a new frame receiver is selected from a software configurable network routing table, so that data can be shared across frame receiver instances.
UDP Jumbo frames are used to reduce network overhead. Taking account of the hardware implementation details, it is easiest to restrict the datagram to 8192 bytes, including all header information. Source module and sequence numbers are included in every frame and data start with a time extension word, so that the full time stamp of every data packet can be reconstructed. Trigger and shutter event packets follow the same format as the time extension word and may appear at any point. As soon as a frame buffer is full, it is queued for transmission. This does mean that data may appear on the network interface out of time order as the last unfilled datagram in any time slice is held to catch any straggling event packets. This does not matter as time slices are routed to different Odin frame receivers.
The logic design needs to cope with multiple Ethernet links. Data from any time slice can be routed in a round robin fashion between any combination of links. Each time slice will target one Odin frame receiver, identified by server MAC address, IP address, and UDP port number, until the time interval roll over when it will take the next entry from the routing table. The number of time slices and the number of frame receivers are decoupled, but there must be at least as many frame receivers as time slices.
While it is possible to implement a 40-G Ethernet core in Virtex7, in practice, this was not as readily available as the 10-G Ethernet core, so the implementation proceeded with the latter. This increased the number of links that need separate support and resource usage limited us to four links. This is sufficient for the primary use on the small molecule single crystal diffraction beamline I19 but will require further firmware optimization to fit the second interface, which will be required for experiments such as diffracted X-ray tracking (DXT) with white beam. This is an area being considered for improvement.

C. Setup and Control Firmware on FEM-II and CTC
During experimental setup and operation, it is useful to see the image that the detector sees. However, in the case of Tristan, the data are not in a convenient image format. Converting the data in real time would be computationally expensive in software and might interfere with the process of data capture. The FPGA can easily assemble a low frame rate image from the incoming data. These viewfinder data are passed to the FEM-II Zynq processor and transmitted via the control channel. The CTC Zynq collects all the image segments and makes complete images available on a dedicated ZeroMQ channel over the control network.
Both the FEM-II and CTC have firmware to sequence the detector's acquisition. The FEM-IIs are synchronized to the CTC. The CTC can be programmed to operate with internal timing, but it can also be synchronized to external equipment. Trigger inputs are processed in the CTC firmware, where they are timestamped. The CTC has no direct output to the data acquisition system, so these timestamps are encoded with the trigger signal that is sent to the FEM-IIs. The FEM-II firmware adds the timestamps to the data stream.
The control interface was based on a restful API with JSON over ZeroMQ. It was found that the structure of JSON fitted the modular structure of the detector very well and could describe the internal configuration. This worked so well that the same protocol is used between the CTC and the FEM-II.
The FEM-II designers designated certain IO to be a geographic address, so that FEM-II in a multimodule system could identify their position. In Tristan, this idea is extended, turning the lower two address bits into an I2C interface to an EEPROM mounted in the chassis. Not only does this extend beyond the nominal sixteen modules allowed by the original scheme, but it allows the inclusion of arbitrary configuration, such as network configuration. This is again achieved using JSON, so that the EEPROMs can use the same interpreter as all other configuration data.

IV. SOFTWARE ARCHITECTURE
The requirements of large high-speed detectors have led Diamond to initiate the development of a new high-speed framework for imaging detectors-Odin-which has been used for Percival [12], an update to Excalibur [13] and Eiger detectors. Odin provides a modular framework with detector specific plug-ins. A network switch connects the detector to the Linux servers. The servers are then connected via InfiniBand to a high-performance GPFS file system.
Odin is divided into Odin Control, a python-based framework, and Odin Data, which is itself divided into frame receiver and frame processor C ++ applications. These Odin Data applications each have a generic core component and a detector specific plug-in. A separate meta writer process completes the Odin framework. This is shown in Fig. 10. The frame receiver is responsible for receiving and reordering UDP datagrams from the detector. These are placed in a shared memory buffer ready for the frame processor. The frame processor converts the data and saves it as HDF5 files. The frame processors also pass metadata about the time slice boundaries in the data to the meta writer, which records this information.
The frame processor is the most CPU intensive part and runs slower than the frame receivers, so the server memory, and the shared memory buffer it contains, must be sized to avoid a data overrun during a typical experiment. During development of the Tristan plug-ins, the nature of the data challenged key assumptions in the design of Odin. Odin was designed for fast frame-based detectors, where the size of data frames is predictable and constant. Tristan data do not conform to this model, so certain changes to the core parts of Odin were also necessary.
There is one Odin rack mount server per detector module, and each server has been optimized with ten Odin frame receiver/frame processor pairs, which was found to give the best performance under high load.
The HDF5 single writer multiple reader (SWMR) extensions [14] are used. Without this, the HDF5 file is unreadable until closed when the final index data are written. With long acquisitions, there is a danger that any interruption to the file writing will leave the whole dataset unusable. SWMR eliminates this problem.
Diamond has experimented with two technologies for combining HDF5 data from multiple sources into a single dataset. The first was parallel HDF5 [15], but this had performance issues, so a new approach was taken, by writing multiple files and combining the data in a HDF5 virtual dataset (VDS) [16]. With this, the HDF5 libraries handle the complexity of multiple files and the application code sees the data as one large dataset. The HDF5 VDS was tried with Tristan. Each frame processor writes its own file and maintains metadata giving indexes to Tristan time-slice boundaries. The metadata from all frame processors on one sever is sent to a meta writer process that creates a metadata file. Once the main HDF5 files are closed, the metadata is written, and then, lastly the VDS is created using information in the metadata files. Due to the potential nonuniform distribution of Tristan data across SWMR files, the VDS could not be used to access the data until after the end of the acquisitions. This was not the main problem; the VDS might work efficiently with frame-based data, where all blocks are a constant size, but with the variable sized Tristan data, the performance was poor and this approach was abandoned. Now, the metadata files are used as an index to the individual HDF5 files.
Odin uses a representational state transfer (REST) [17] JSON over http interface as its main API to the outside world. Most instrumentation at diamond is controlled with EPICS [1], so to provide a standard interface Odin is controlled via ADOdin, an EPICS Area Detector implementation. Overall control of I19 beamline experiments is handled by generic data acquisition (GDA) [18]. GDA coordinates all parts of the experiment, including the detector.
Experimental datasets are saved with a Nexus [19] data structure to include all experimental metadata from the detector and the beamline, for example, beam energy, goniometer position. GDA collects the necessary information, including extra detector metadata gathered via ADOdin. The Nexus file includes references to all the Odin HDF5 files and metafiles.

V. EXPERIMENTAL RESULTS
During development, the prototype 1 and 10 M systems have been trialed on user experiments for pump probe [20] using the Tristan 1 M readout with a CERN Timepix3 chip board and DXT [21] using the 1 M. Other experiments, including those using the 10 M, are yet to be published. Characterization results have been published separately [22].
The Tristan 10 M detector has been used in multiple timeresolved crystallographic experiment on beamline I19, DLS. A crystal that undergoes a reversible photo-induced transformation upon illumination is repeatedly subjected to laser pump (PORTO femtosecond laser) running at 50 kHz. A crystal is continuously rotated at 0.05 deg/s over a 360 • range. The slow rotation is required to ensure significant signal after the data have been split and each reflection is measured multiple times across the profile. A pump-probe rotation dataset is obtained in 2 h.
The signal from the diffractometer position compare indicating the start and finish of the scan is recorded in the data stream by the Tristan LVDS input. The X-ray fast shutter and laser fast shutter is opened/closed 20 ms before/after the scan. The transistor-transistor logic (TTL) signal from the PORTO laser is combined (AND) with a signal from the laser fast shutter and fed into the Tristan detector (TTL), which is also recorded in the data stream.
The Tristan detector continuously records data with timestamps of 25-ns time resolution. This means that the entire time-resolved series is recorded in one experiment, which results in better comparison between delay times (the same crystal, temperature, radiation damage, and laser excitation). How the data are split into discrete time bins can be determined post experiment. For example, for a laser repetition rate of 50 kHz (20 µs), data can be split into 20 time bins, where each time bin will be averaged over 1 µs. If there is significant Fig. 11. Diffraction data from a [Pd(Bu4dien)(NO2)]BPh4 sample using a beam energy at the Ag edge (0.4859 A), captured on the Tristan 10M. signal to noise, the user may want to attempt splitting the data into 40 time bins, thus giving a 500-ns time resolution.
The excitation and decay of the photoactive compound [Pd(Bu4dien)(NO2)][BPh4] was investigated using the Tristan 10M detector. During a continuous rotation diffraction timeresolved experiment, data from the first time bin after the laser excitation have been accumulated into 0.2 • wedges over a 350 • scan. An example of this 0.2 wedge from the first time bin can be observed in Fig. 11. The specifics of the experiment and processing are beyond the scope of this article.
Once the data are split, each of the datasets within the series is processed using the xia2 processing suite producing structural shelx files ready for refinement. These processing routines take about 30 min. The quick processing is crucial for optimizing the experiment.

VI. CONCLUSION
The Tristan detector provides a new and flexible way to capture time-resolved data. It has advantages over frame-based detectors in providing zero readout dead time at high frame rates. The ability to capture "always on" during an experiment can ease synchronization requirements. The fully timestamped data mean that if the analysis does not yield good results, the data can be reprocessed at a different frame rate, rather than requiring the experiment to be repeated with a new sample.
They would like to remember the late Brian Willis, who gave an essential contribution in developing the mechanical concept of Tristan in the early stage of the project and the late Kirk Arndt who helped in devising the mounting procedure of the hybrids on the cold finger.