How a Data Center Is Installed & Maintained
What comes to mind when you think “Data Center?” Is it a fluffy white cloud — “it’s in the cloud?” Or maybe countless rows of massive machines whirring and beeping in an eerie blue light, à la The Matrix or Transcendence?
Perhaps you shoot back to the 1940s and ENIAC (Electronic Numerical Integrator and Computer) with its banks of terminals, cables and connections? Data Centers have come a long way since then.
In 2022 our computers are minuscule compared to those giants of the ’40s and even the PCs of the ’90s.
We carry devices in our pockets and on our wrists and access information 24/7 without ever thinking about where it’s coming from.
But regardless which device you use, most of that information has been stored in a data center somewhere along the way.
So, how do WBE teams install, maintain and repair the intricate maze of machinery that makes up a modern data center? And what can you, our customer, expect?
We asked Paul Gigliello, VP of our Telecom division and Tom Lauchenauer, VP Electrical, “What happens behind the scenes to get these massive facilities up and running?”
Gigliello is a twenty-year veteran with WBE, while Lauchenauer joined the company nearly three years ago. But they both know the trade inside out.
They started as apprentices and worked their way up through the ranks — journeyman, foreman, superintendent etc.— and both gathered extensive experience with data centers along the way. Indeed, Paul Gigliello was the lead foreman for one of WBE’s big data center projects in the early 2000s.
As vice presidents of their respective divisions, they see their role as supporting the Telcom and Electrical project managers, electricians, field techs and customers with their knowledge and experience to install the complex infrastructure on time and within budget.
Both agree that technology has changed a lot in their 20+ years of working in the industry. Twenty years ago, everyone used copper wiring, transmitting information as electricity.
Now, copper’s outdated and slow. Modern data centers use fiber optics which send information as light. Speeds have skyrocketed, too.
“Where ten gigs used to be a big deal, now the speeds are 200, 400 gig,” says Gigliello.
Fiber has changed, too. Ribbon fiber is a big deal because you pack it into a much smaller trunk cable. Why does that matter in data center installation?
“15 years ago when I did a data center, we’d run a 96-strand fiber and it would be an inch and a half in diameter. That same 96-strand fiber nowadays is just under a half inch. So the data centers are getting more dense and complex.”
Source: Unsplash
What is a data center?
This is a trickier question than it might seem. The answer’s different depending on who you talk to.
Lauchenauer sees data centers as “a managed distribution system,” His Electrical Division’s priority is to ensure that the power is clean and balanced. “Typically,” he says, “it’s a dedicated space for those critical loads.”
From a telcom viewpoint, Gigliello says it’s all about information storage, a mainbrain if you will. His team handles every aspect of a Data Center install, from the design to laying fiber optic cables and other infrastructure.
Wikipedia defines a data center as “a building or dedicated space within a building, or a group of buildings used to house computer systems and associated components such as telecommunications and storage systems.”
Data centers may belong to a single company — Google, for example, has at least 21 data center locations worldwide.
Or they may be colocations — vast spaces shared by many clients. ‘Colos’ are typically arranged in caged cabinet groups, with each cage owned or leased by a different organization.
What’s involved in a WBE data center project?
Each project is engineered based on the needs of the facility. It’s rarely just as easy as pipe and wire.
Project managers must ensure that everything goes in the correct sequence. That takes a lot of pre-planning and onsite coordination before WBE teams even step foot in the facility.
PMs consider hundreds of details:
- Is the floor at ground level or raised? A raised floor allows room to install cables and wiring underneath the cabinets rather than overhead.
- What’s the cooling system — HVAC or CRAC? Cooling is critical — all the equipment produces a tremendous amount of heat. If left unchecked, heat can shorten the unit’s working life or cause it to fail altogether. Raised flooring may include an HVAC (Heating, Ventilation & Air Con) system to combat the heat. Alternatively, there could be CRAC (computer room air conditioning) units to monitor and maintain the air temperature and humidity.
- Is there a Track Busway — an overhead power distribution system in sizes ranging from 40 right up to 1250 amps. Starline busways are helpful because they have a continuous access slot, making it easy to insert plug-in units anywhere in the system.
- Do you have the structural bases to support the UPSs — Uninterruptible Power Supplies — and the PDUs — Power Distribution Units?
- How will the power get to and from the equipment? Are you using pipe and wire, bussing or maybe tray cable?
Both agree, “there’s a lot involved with the data center builds; a lot more than just your standard Tenant Improvement space, that’s for sure.”
Safety and security are paramount
Fire
Imagine this — a UPS fails, and a battery cabinet overheats beside it. Flames lick the nearby cabinets — they spark and flare. Fire alarms blare, and thick black smoke fills the building. It’s already too late! Within minutes the entire building is destroyed.
That’s the scenario that played out when OVHcloud lost one of its data centers in Strasbourg, France.
It’s one that data centers around the world desperately want to avoid.
So WBE always considers the EPO (Emergency Power Off) and fire safety systems in data center contracts. You can’t use water on a bank of computers, so they have to use a non-water system like a clean foaming agent or inert gas that won’t damage the equipment.
Seismic
Earthquakes are another hazard, especially in the San Francisco Bay Area.
As Lauchenauer says, “The seismic and structural requirements can be strict in our area since we are in an active earthquake fault zone. The goal of the seismic requirements is that all equipment will be installed seismically and structurally to withstand the strongest of earthquakes and to protect all people that are working in the facility.”
Backups
Then there’s the redundancy issue. It’s crucial for the safety and security of supply that you have more than one pathway for electricity to enter the building.
As Gigliello points out,
“I have seen it where two different carriers are coming into the same main point of entry. And if something happens there, the whole building goes down. Even though they have two carriers, there’s no redundancy. So, you’ve got to make sure that one carrier is coming in on the east side of the building, and then there’s a pathway on the west side of the building with another carrier coming in so that there’s no potential for that cross to happen. You want total redundancy throughout.”
In other words, if A goes down, B works; if B goes down, A still works. Even better are the facilities that include a “plus one redundancy,” which gives them an additional backup.”
Surveillance
Security can be incredibly tight. Some facilities go to the extent of having card readers on cabinets so that individuals can’t access a cabinet unless they have badge access.
“There’s usually tons of surveillance— cameras down every row. There’s a lot of liability in these data centers as well. So if someone’s walking by and opens a cabinet just, let’s say unplugged a fiber cord and maybe a doctor is in the operating rooms using images at the time.”
That could be disastrous. WBE can install all the security infrastructure needed, including cameras and access control.
Unique complications in a data center build
We asked what the tricky issues in data center installations are. The answers came back thick and fast.
The dense pathways for trunk cables or cross-connects
“I’ve actually been on a couple of data center projects where they’ve run out of tray space, you know, halfway through the project because they just didn’t anticipate how many cables were going in there.”
Hot and cold aisles for energy efficiency and cooling
“I was in one where you couldn’t penetrate anything through a cold aisle to a hot aisle and back to a cold aisle. So you have to really plan and coordinate your pathways to avoid penetrating through areas determined to be a no fly zone.
The cabinet installs, especially if they’re in a raised floor environment.
“There is a lot of seismic that we have to deal with; unistrut and drilling through all the raised floor panels and attaching strut to the below deck.
I did one facility a while ago, and it was a four-foot raised floor with HVAC under there as well. We were running struts at 45 degrees, this all has to be figured out by a seismic engineer.
Future-proofing
Lauchenauer says, “from the electrical standpoint, it’s important to make sure that it is designed correctly from the beginning and that it will be built for any future known expansion. What we have seen when we come in on a remodel is that there may have already been multiple expansions to the system over time and the facility and equipment might already be maxed out with no additional capacity for increased power requirements.”
That presents the owners with a difficult choice. Will you build an entirely new system or rebuild the one you have with more energy-efficient equipment?
The permit process
Getting everything approved can be frustrating and time-consuming, too. And in earthquake-prone regions, there are additional requirements.
“You might be required to bring in a special testing agency to do pull anchor testing to make sure that everything is seismic appropriate.”
Source: Unsplash
Supply woes and tight schedules
There are incredibly long lead times to obtain much of the equipment you need in a data center installation, and they’re getting worse by the day. That puts the schedule under pressure.
In today’s world, suppliers are struggling to hit their lead times. So, WBE might have two to three weeks for a submittal process, but then wait 22 weeks or more for the equipment to arrive, instead of the usual ten.
To ensure that they meet all the deadlines, the WBE teams plan to have all the other infrastructure done and ready so that when the final piece shows up, they can install it and have things ready for action. Of course, the same complication applies to fiber optics, cabinets and almost every other thing you need.
Gigliello tells us he just ordered some cabinets, and the delivery estimate was 68 weeks! Lauchenauer agrees. “We have seen the standard lead times of 8-12 weeks grow to 22-24 weeks or even longer.”
WBE leads with coordination and cooperation
As we’ve seen, data center installations are vastly complicated and require extensive coordination between all the different companies on the build. That’s where WBE can alleviate stress and time-wasting by taking on the whole project.
“There’s definitely benefits to having both communications and electrical on the same job. I mean, it can help with the coordination for sure. It’s nice that we can offer all that to one one client or customer.”
It often makes sense for WBE to be the prime contractor in data center work. When you involve Electrical, Telcom and Security, you alleviate many of the communication pain points that arise in the pre-planning phase.
Lauchenauer explains:
“One example… the rebuild WBE did on a facility that had a fire. We originally thought it was just going to be the replacement of one piece of equipment but it turned into a full system upgrade and a project that lasted about six months. We were the prime contractor on the project managing the structural steel contractor, rigging subs and working with various equipment providers to expedite the equipment.”
In that job, the facility was still up and running, although the B side of the UPS was working, so it was very vulnerable to disruption. WBE brought out temporary UPS trailers as a backup while working on the damaged side of the UPS.
WBE provides critical maintenance
When you’re looking for continuity on your data facility maintenance, WBE provides outstanding service in a cut-throat market. We’re in it for the long haul with our customers, and that includes keeping your data centers up and running.
For example, fifteen years ago, WBE installed the data center for a major health provider. Today we’re still there, keeping it working at top capacity.
Every day a WBE team of eight to twelve staff works full time on maintenance tasks and upgrades like adding cabinets, servers and cables. As you can imagine, with hundreds, even thousands of cabinets already in that one defined area, fitting in extra cabinets and cables can be a mission,
Digital DAS
Finally, digital DAS is a new type of technology that operates within the Common Public Radio Interface (CPRI). At a basic level, it makes use of a Base Band Unit (BBU), which relays the signal through fiber optic or ethernet cable to a master unit. It doesn’t require the signal conversion that other DAS types do.
Due to fewer components, digital DAS is quick and affordable to install. However, it’s still in the early adoption phase and hasn’t yet been widely used.
What sets WBE apart from other contractors?
“What comes to mind, says Gigliello, “is our experience.
Our Telcom division’s done quite a few data centers, from ginormous ones to super small. So we’ve got the whole dynamic range of what we’re capable of doing successfully.
We know the service part of it because we’re currently doing that, as well. So we can do a new build and add that value. We know how it’s supposed to start, how it should finish, and then we know how to handle the service part of it after it’s built and up and running.”
Lauchenauer adds that customer service is also a massive part of the equation. WBE’s attention to the end-users and the client’s needs is second to none.
“That’s our whole company motto. We’re not there to just do the quick job and get out of there. We want to maintain that relationship. When you’re talking about the uniqueness of WBE, think how many clients we’ve had for 20, 30 years. There’s a lot of them.”
The fact that WBE has sustained so many relationships for that long — there’s no better example of customer service than that.