Preventing Vaccine Production Stoppages with HPE Synergy Redundant Infrastructure

Medical science moves at such a rapid pace that it can seem miraculous. Often, however, we only see the end result: A loved one beats cancer. A friend with Crohn’s Disease goes months between flare-ups. A partner with HIV has an undetectable viral load. In general, people are living longer and staying healthier.

Researchers around the world dedicate their careers to finding treatments that can improve and save millions of lives. But lab work is only half the story. Once any therapy has been tested and approved, it still has to make its way to doctors and patients.

The Road from Research to Market

There are many stops on the road from research to market. Along the way, medications go through several rounds of clinical trials and a rigorous approval process. When they eventually enter production, their manufacturing and distribution channels are highly regulated and tightly monitored.

Researchers around the world are currently racing for ways to combat and halt through the latest challenges, but the need for other drugs and therapies remains. Pharmaceutical companies and drug manufacturers are constantly working to ensure that patients around the world continue to have access to the medicines they need.

Like almost all other businesses, pharmaceutical companies employ experts in dozens of fields—including science and medicine, law, packaging, and IT—to bring their products to market. But the public only sees the benefits of our end products and not all the hard work that goes into treating their ailments.

A Company Takes on the World

Our company—a large pharmaceutical company—has grown globally over the years. We now employ over 50,000 at facilities across the world. 

I’m a manufacturing systems engineer at our vaccine plant. I’ve worked here for 18 years, remaining at the plant after our company was acquired. That previous company was purely a European entity. Working for a multinational changed the scope of my responsibilities. My IT team remains focused on local concerns and on onsite operations, but we also report to our global IT team so we can align our goals with the company’s worldwide ambitions.

We broke ground on a new plant in 2016 that opened its doors in 2019. As we were beginning the construction of the new plant, we also started building a new data center to power existing and future production and research facilities. At the time, the plant's IT infrastructure comprised 11 HPE servers that powered both our manufacturing floor and the adjacent labs. These servers also housed our local archives. 

For regulatory purposes, pharmaceutical companies have to hold on to data for 10 to 15 years, and so we require massive amounts of easily accessible storage that is fully backed up. Even though we upgrade everything every five years, we hold on to aging servers that store older data and run legacy applications. Because we need to hold on to data for so long, you’ll often find obsolete equipment at companies such as ours. 

As part of the upgrade, we also needed redundant IT infrastructure for our production facility. We, therefore, set out to build a data recovery center that mirrored our primary site, but which was housed in another building. In the event of system failure, we could spin up this secondary site and resume the manufacturing of vaccines within a couple of hours.

Redundancy is crucial. Under ordinary circumstances, we must adhere to manufacturing standards and production quotas. But in the event of an emergency, even the smallest interruption in vaccine production can jeopardize thousands of lives.

An Initial Misstep Leads to a Perfect Solution

In 2016, we made an initial attempt to modernize our infrastructure using HPE StoreVirtual VSA technology. We assumed that virtualizing our existing 11 servers would bring them up to speed and give us room to grow. But this solution wasn’t sufficiently scalable and lacked the power and flexibility to run the new factory we were building.

Two years later, we sought out a local IT partner and assembled an architecture team to blueprint our new data centers. Our global IT team provided some guidance but also gave us the latitude to explore the solutions that best met our specific needs without imposing an external set of criteria.

After looking at several new options, the architecture team recommended HPE Synergy. The idea clicked with us right away. HPE was a trusted vendor that provided the core of our existing IT infrastructure. HPE Synergy is a composable infrastructure product that combines storage, compute, and networking hardware in a single chassis. It uses built-in storage intelligence, auto-discovery tools, and fluid resource pools to create flexible system configurations that run physical and virtual applications.

Think of it as a cloud-like solution that runs on physical hardware. HPE Synergy provisions your onsite infrastructure’s physical layer using the same flexible parameters that typically power software-defined assets that reside on the cloud. For example, if you run out of storage or compute on one rack, HPE Synergy can channel the needed resources from another. It gives you the flexibility of the cloud and the speed and security of an on-prem solution.

We put our blueprint and our budget together and then sent our specifications back to our global IT architecture team for approval. Once we were given the go-ahead, we started to set up and document our new data centers.

Fully Redundant from Day One

In pharmaceutical IT, we have to document every data center device and its configuration. It can take at least a day to set up a server, and another to write it up. We have to show who is responsible for maintaining each server, what department it belongs to, who manages the business process it runs, and so forth.

There was no trial period. The setup and documentation process took three months, and then we went live. We did all our testing—including failover tests—during the setup period. When we launched, our primary and backup data centers were ready to go. We had a fully redundant High Availability platform from day one.

We started spec’ing the system in 2018 and rolled everything out in March 2019. We now have a total of 12 HPE Synergy 12000 frames, six each at our primary and secondary data centers. They house about 80 of the 480 Gen10 compute modules and roughly 60 HPE 3PAR storage modules. If anything in the production environment at our new building fails, we can switch over to the backup very quickly.

This new infrastructure has increased speed, stability, and availability. Some of the manufacturing and the archiving processes at the older buildings run on legacy servers, which are maintained by external contractors. There is no automated backup for these machines, but should they fail we have maintenance contracts in place to ensure their return to service. We are already working on plans to replace these aging servers and hope to have every process at the factory running on HPE Synergy within two years.

Another production facility is also in the works, and we’re applying all the lessons we learned during our first HPE Synergy deployment to equipping its new IT infrastructure.

Seeking Solutions in This Time of Crisis

I’m proud of the work we do at our company. Vaccine production is the final step in a lengthy process that begins with a simple question: How do we keep people from getting sick?

On top of supplying existing medications in this time of crisis, our company is putting our resources into finding therapies to combat diseases. Millions of people around the world rely on us to help them stay healthy and improve the quality of their lives. 

With HPE Synergy, we have a faster, more reliable way to keep our production moving, no matter what. Medical advances and cutting-edge IT go hand in hand, and we’re more relaxed knowing we have the right platform to deliver treatments to the doctors and patients who need them most.