Ethernet-based timing system for accelerator facilities: The IFMIF-DONES case

Abstract

This article presents the design of a timing system for accelerator facilities, which relies on a general networking approach based on standard Ethernet protocols that keeps all the devices synchronized to a common time reference. The case of the IFMIF-DONES infrastructure is studied in detail, providing a framework for the implementation of the timing system. The network time protocol (NTP) with software timestamping and the precision time protocol (PTP) with hardware timestamping are used to synchronize devices with sub-millisecond and sub-microsecond accuracy requirements, respectively. The design also considers the utilization of IEEE 1588 high accuracy default PTP profile (PTP-HA) to provide sub-nanosecond accuracy for the most demanding components. Three different solutions for the design of the timing system are discussed in detail. The first solution considers the deployment of one time-dedicated network for each synchronization protocol, while the second one proposes the integration of the synchronization data of NTP and PTP into the networks of the facility. The third solution relies on the single distribution of PTP-HA to all the systems. The final design aims to be fully based on standard technologies and to be cost-efficient, seeking for interoperability and scalability, and minimizing the impact on other systems in the facility. An experimental setup has been implemented to evaluate and discuss the suitability of the solutions for the timing system by studying the synchronization accuracy obtained with NTP, PTP and PTP-HA under different network conditions. It includes a timing evaluation platform that tries to resemble the network architecture foreseen in the facility. The measured results revealed that PTP is the most limiting protocol for the second solution. Using the default PTP configuration, it tolerates less than 20% of maximum bandwidth utilization for symmetric bidirectional flows, and around 30% in the case of unidirectional flows (server to client or client to server), with the current setup and using switches without enabled timing support. This case study provides a better understanding of the trade-off between bandwidth utilization, synchronization accuracy and cost in these kinds of facilities.