DoE invests $24 million in data management R&D – Blocks and Files

The US Department of Energy is investing $24 million in research into “next-generation” data management and visualization.

Organizations large and small grapple with the problem of managing large amounts of data, as well as visualizing what it is actually telling them. However, the challenge is particularly acute for the DoE, given its role in overseeing some of the United States’ most highly regarded science labs and associated supercomputers.

The DoE will therefore allocate $23.9 million for research on the “next data management”. Barbara Helland, DoE Associate Director of Science for Advanced Scientific Computing Research, said in a statement, “These efforts will allow data to be processed and stored at higher rates across edge, cloud, and desktop computing environments. high performance, and to develop new visualization methods to explore this data, formulate hypotheses and communicate conclusions to a wide range of audiences.

The agency said improvements in data management would “facilitate discovery in a wide range of areas”, including climate modelling, developing clean energy, increasing energy consumption and reducing energy consumption.

Part of that will be through “optimizing” the handling of data that needs to be moved around and analyzed, “using sophisticated mathematical techniques, including machine learning.”

The projects are also likely to “advance innovative techniques that harness smart storage and networking hardware that can provide breakthroughs that address the data challenges facing scientists and engineers.”

The research findings will likely ripple out to the rest of us over time, both in terms of better tools and techniques for storing and managing data, as well as in terms of breakthroughs in creating, managing and energy efficiency.

Argonne National Laboratory and Los Alamos National Laboratory will all initiate projects on “A Compositional Approach to Harnessing Smart Devices in Elastic Data Services.”

Sandia National Laboratories, Illinois Institute of Technology, and Oak Ridge National Laboratory will examine “Coeus: Accelerating Scientific Insights Using Enriched Metadata.”

Brookhaven National Laboratory, University of Texas, and Argonne will research “Scalable Provenance and Metadata Services for Reproducible Hybrid Workflows.”

Lawrence Berkley National Lab, Northeastern University, and the University of Illinois will conduct research on “End-to-End Object-Driven Software-Defined Data Management for Science.”

Penn State will research “intelligent file system interfaces for computer storage devices.”

The rest of the pot will go towards visualization research “on new techniques and theories needed to aid in the development of informative and interactive visualization of complex scientific data relevant to the DoE mission space.”

The explosion of data creation is itself a power consumption problem. The energy consumption of buildings housing computers means that some areas struggle to accommodate buildings housing people.

Last year, the DoE announced $5 million in funding for research into data reduction techniques and algorithms “to facilitate more efficient analysis and use of massive datasets produced by observations , experiments and simulations”.

Comments are closed.