Verne Global transitions its Finnish underground data center to liquid cooling

The transition to direct liquid cooling is in response to growing data demands from AI, machine learning, and high-performance computing.
6 December 2023

‘The Rock’ data center in Pori, Finland. Source: Verne Global

  • ‘The Rock’ data center is situated in tunnels under Pori, Finland.
  • It is transitioning from row coolers and CRAHs to liquid cooling.
  • This could decrease the energy it uses on cooling by 10%.

Verne Global has introduced liquid cooling to its underground data center campus in Finland in response to growing data demands from AI, machine learning, and high-performance computing. These intensive applications result in data systems producing significant amounts of heat that can no longer be cooled sufficiently with air.

The data facility, known as ‘The Rock’, is located in ex-military tunnels running through the bedrock underneath Ulvilia, a small town near the city of Pori. The tunnels were quarried in the 1960s by the Finnish Defense Forces but were never used for their intended purpose of a defensive vault. Instead, they were bought by Ficolo in 2011 and repurposed into a data center, taking advantage of both their in-built security and cool temperature.

Ficolo was bought by investment firm Digital 9 in 2022 and later merged with Verne and Volta, two of its other acquisitions. This month, the resulting conglomerate, Verne Global, was sold to Ardian.

Liquid cooling of a server, seen from the back.

The DLC3000 is a stand-alone rack with an in-built cooling distribution unit designed for a small deployment. Coolant, in the form of water, is directed to the units in the rack and exchanges heat from the rack to the facility water supply. Source: Verne Global

The Rock is comprised of nine separate tunnels and has a total area of 8,500 m2 (90,000 sqft). Its infrastructure is powered by 100% renewable energy from nearby wind farms and its own 2,600 m2 solar plant. According to a Verne Global blog post, the average data center must commit 40% of its energy to cooling.

Verne Global is installing Dell’s DLC3000 solutions, which provide up to 80 kW of cooling capacity, and 4th Gen Intel Xeon scalable processors to “maximize performance and cooling efficiency.”

According to Kim Gunnelius, Verne Global Finland’s managing director, “the surge in AI, machine learning, and high-performance computing is amplifying the need for robust data center cooling solutions.” This is what inspired the company’s transition from conventional air cooling to direct liquid cooling (DLC).

He told TechHQ: “In the last few years, we have directed the demand towards our the Air data center, which opened in Q2 2020. We can use indirect free air cooling up to 35 or 40°C (95 or 104°F) there and have very tall racks. With the use of liquid cooling, we expect demand to increase at The Rock.”

The DLC3000, combined with liquid cooling, should meet new levels of compute need in the Rock data center.

DLC3000 front. Source: Dell

An old Ficolo environmental policy optimistically explained that The Rock’s racks only need to be actively cooled a few days a year when the above-ground temperature exceeds 77°F (25°C), as the tunnels under Pori have an average temperature of 46°F (8°C). However, this didn’t turn out to be the case.

Mr Gunnelius told TechHQ: “The initial theory was that we could utilize underground cooling. In practice this is only feasible for low density racks. Any sizeable HPC implementation will need additional cooling. The naturally cool environment does of course decrease the amount of cooling needed.”

The Rock therefore currently uses row coolers and computer room air handlers (CRAHs) to direct air chilled to just below 77°F (25°C) onto the racks. The facility’s total energy use is around 10 MW when all nine tunnels are fully built out. Mr Gunnelius says that the current plan is to transition a portion of the campus’ operations to DLC, but that this could increase to around 80% in time.

“This is also driven by customer demand; many customers take time to move to newer technologies,” he told TechHQ. “As a colocation provider, we cater to customer’s requirements.

“We have been talking to our customers about their transition to DLC over at least five years, looking also at immersion, but customers have chosen to use air cooling, because it is more standard and works well for lower densities. As we move to ever higher densities, you can add on things like rear door cooling, but at some point, you need to provide cooling to the chip.”

Solar panels above The Rock DC in finland. Source: YouTube/Verne Global Finland.

Solar panels above The Rock data center in Finland. Source: YouTube/Verne Global Finland.

A rack at the Rock data center in Finland, before liquid cooling.

A rack at The Rock, currently using CRAHs rather than liquid cooling. Source: YouTube/Verne Global Finland

Liquid cooling is more efficient than air cooling because water has a higher heat capacity, meaning more efficient heat transfer to the surrounding environment or dedicated cooling systems. According to Dell, the rear fan power requirement for a 270 W CPU at 10% load would be 1076.8 W if air-cooled, while it would be 111.7W if liquid-cooled with DLC modules.

The total energy usage of The Rock will be significantly larger if rack density increases to warrant 80% DLC. “Perhaps even 20 to 25 MW,” according to Mr Gunnelius, and this will still be covered by the local solar and wind power sources. However, the proportion of energy required for cooling will decrease.

“Assuming a full tunnel at high capacity moving from air cooling to liquid cooling, we expect that cooling power usage would go down from around 20% of IT power to 10% of IT power,” Mr Gunnelius told TechHQ

He added: “There are several types of liquid cooling solutions, such as single or dual phase and immersion with special liquids and submerged systems. There’s also the refrigeration system for quantum computing, for which most of the visible part is for cooling. The technology is moving fast, so we expect to see new solutions in the future, possibly designed by AI.”