In a significant development for environmental research, a data analytics platform called terrabyte has been introduced. The platform will empower scientists in Germany to harness the potential of satellite and earth observation data. These can be used for simulations and evaluation, utilising artificial intelligence methods.
—
About terrabyte
A joint initiative by the German Aerospace Center (DLR) and the Leibniz Supercomputing Centre (LRZ), terrabyte boasts a heterogeneous system comprising multiple CPUs, accelerators, and GPUs. The core of the platform consists of Lenovo ThinkSystem SD650-N V2 servers and DSS-G memory. Featuring 61 CPU nodes and 15 GPU nodes, terrabyte is equipped with nearly 50 petabytes of additional online storage. Data processing speeds can reach an impressive 320GB/s with terrabyte’s InfiniBand HDR. Data is seamlessly transferred from the DLR’s German Satellite Data Archive to the terrabyte platform at the LRZ. The transfer is done via direct network connection running at 100GB/s.
According to a statement by the LRZ, the high-performance equipment of the terrabyte platform enables it to achieve a remarkable performance of up to 1.3 petaflops. Furthermore, the LRZ anticipates doubling this performance by autumn as part of its ongoing expansion efforts.
Satellite data also contributes to the evaluation and analysis of offshore wind farm locations. It then assesses their impact on national power supply. Terrabyte users have the flexibility to utilise their own algorithms or train machine learning applications on the platform. In addition it assesses a range of open-source software, tools, and algorithms developed by the Earth Observation Centre and other DLR institutes.
Previously, limitations in computing power and scalability confined certain geoscientific analyses to commercial platforms. However, terrabyte now offers a solution to overcome these barriers.
—
About the organisations behind this
The DLR, renowned for its vast collection of historical data spanning 50 years of Earth observation, along with current radar and multispectral images, serves as the foundation for various research projects. Notable among them is the DLR’s “World Settlement Footprint,” which has been instrumental in tracking global settlement development since 1985. This initiative has provided invaluable support to urban and transportation planners, particularly in megacities such as Cairo, Mumbai, and Rio de Janeiro.
The development of terrabyte has been made possible through an investment of approximately €19 million in German federal funds from the Bavarian State Ministry of Science and the Arts and the Federal Ministry of Economics and Climate Action. While the DLR spearheads the construction and expansion of terrabyte, the LRZ oversees platform operations, ongoing costs, and staffing.
With the Leibniz Supercomputing Centre, we have found the ideal cooperation partner for this challenging project. With terrabyte, we have an autonomous and secure platform with comprehensive global datasets from Earth observation that we can access together with our partners.
Stefan Dech, director of the German Remote Sensing Data Centre at DLR
—
If you are looking to push the boundaries of your data analytics, we can help. Get in touch with Pl Talents to discuss your hiring requirements.