System analyzes atomic collision before blinking

Scientists at Oak Ridge and Berkeley National Laboratory have developed a revolutionary data flow system that could fundamentally change the way nuclear physics experiments are conducted. The software (called Deleria) creates a high-speed data pipeline that can analyze large amounts of experimental data almost immediately, allowing researchers to make real-time adjustments to the experiment instead of waiting for hours or days to get results.

This innovative approach connects advanced scientific instruments directly to the most powerful supercomputers in the United States through ultra-fast networks, potentially eliminating the need for expensive local computing infrastructure on research facilities. The system is currently being tested in the Greta, an advanced nuclear physics detector under construction at Berkeley Lab, which will eventually be installed in a facility at Michigan State University’s Rare Isotope Beam (FRIB).

But what happens when you need to analyze hundreds of thousands of particles to collide when the data has to travel more than 2,000 miles? This is a challenge that researchers must overcome with some creative computing solutions.

Chasing the speed of light in the country

Deleria – abbreviation for distributed event-level experimental readings and integrated analysis – enables Greta to transfer data directly to computing centers across the country for near-inherited processing. The system transmits information from Berkeley Laboratory in California to Oak Ridge National Laboratory in Tennessee through the Energy Science Network (ESNET).

“Our main goal is to build a data pipeline for future experimental needs without having to build a local computing infrastructure. The pipeline will also allow us to increase the amount of analyses performed online.”

In the current test bench, the researchers reached a data transfer rate of 35 gigabits per second, which exceeded Greta’s requirements and exceeded ESNET6’s capabilities, which provided multiple 400 GBPS paths between laboratories.

How the system works

Greta itself is a miracle of nuclear physics, a spherical array of 120 supercyanite crystals that can record photon interactions when experimental samples are bombarded by charged particles. These interactions produce a large amount of data that must be processed quickly to be useful to scientists.

“The crystals inside the detector are connected to them,” Jansen explained. “We need to figure out where the photons hit in the crystal – x, y, z coordinates. We need to do this within 10 seconds of collision. This is what the pipeline we designed to do. ”

This processing occurs at ORNL’s Defiant, a 36-node GPU-accelerated compute cluster. Beyond speed, Deleria also provides an impressive efficiency boost by reducing data storage requirements by 97.5% or 40 times.

Key Benefits of Deleria Pipeline

• Process approximately 480,000 photon collision events per second
• Reduce data storage requirements by 97.5%
• Enable real-time experimental tuning during data collection
• Eliminate the need for expensive on-premises computing infrastructure
• Can adapt to other scientific facilities other than Greta
• Transfer data at 35 gigabits per second in over 2,000 miles

Physics to overcome the incubation period

The biggest challenge for this system is not the raw bandwidth, but the latency, but the inevitable delay caused by physical distance data. Even at the speed of light, the data takes about 120 milliseconds (0.12 seconds) to make the round trip from Berkeley to Oak Ridge and return.

This latency is 24 times longer than the 5 milliseconds required to handle a single photon collision event. To overcome this limitation, the team developed clever parallel processing techniques.

“Of course, you can bypass the speed of light only in science fiction,” Jansen said. “What we are working on right now is to get 10 times faster by cheating on latency. Let’s say. The solution is to run an event in parallel so that we can handle one event, while others are through transfers. Finding the right balance, we can make sure that the compute cluster is always busy. Otherwise, our analysis will take 10 times the time. Otherwise, our analysis will take 10 times.”

When it comes to speeds faster than light, “only Gustav can do that,” Tom Beck joked that the head of science major Tom Beck had scientific involvement at the ORNL National Center for Computational Sciences.

Models of future scientific computing

Deleria represents a groundbreaking approach in the Department of Energy’s Integrated Research Infrastructure (IRI) program, which aims to connect major research facilities with leadership-level computing resources.

“It’s really a groundbreaking work on the IRI topic, and it’s a standard for how we build how we build a national leading research center. The more use cases we have, the better we will get,” Baker said. “At one case at a time. That’s really the first case I know so far and working on this mature level, which means we’re getting a really good start.”

These implications go far beyond nuclear physics. Jansen highlights the broad potential of this approach: “We also intend to show that Deleria can be extended to other research facilities and wider scientific applications.”

As scientific instruments become more complex and produce larger and larger data sets, this distributed computing model may be crucial to the field from climate science to materials research. With GRETA scheduled to be installed in 2026, researchers have time to further refine the system, which could push for higher data processing speeds and establish new paradigms for how big science can connect with large computing.