Emergency Services for the Next Generation: Building California's NextGen 9-1-1 System

It's been drilled into us since we were kids: When there's an emergency, grab a phone and dial 9-1-1. Those three magic numbers ensure that first responders—including police officers, firefighters, and paramedics—come to our rescue.

Unfortunately, most 9-1-1 networks across the country use decades-old infrastructure that relies on the circuit-switching, analog technology that does not support today's needs. They're simply not equipped for today's mobile and digital communications tools, and that means a lot of people may not get help as quickly as they need it. 

A Much-Needed Upgrade

California's 9-1-1 infrastructure has needed an upgrade for years. The state put together a framework to transition to the Next Generation 9-1-1 (NG9-1-1) standard in 2017, but the funding, regulations, and contracts to do so were not in place. We began to work with the California Legislature, the California Public Utilities Commission (CPUC), The California Department of Technology (CDT), and the vendor community to build out NG9-1-1 in California. 

Meanwhile, the situation was getting desperate. Our statewide 9-1-1 system experienced 10,000 to 20,000 minutes of downtime a month, stemming from two types of outages. The first is what we call community isolation. That's when 9-1-1 is up and running, but the local phone system, such as landlines, cell towers, or a VoIP provider, can't connect to it. The other type of outage is when the local phone system has the ability to make the call, but can’t connect to the local PSAP (public safety answering point). Add the dramatic rise in natural disasters, including California's recent wildfires, and it was clear that we needed to act.

We also had limited fallback capability. When a local PSAP fails, we can route calls to another nearby answer location. But if the outage also impacts the alternative answer location, then the call can't be routed properly unless someone manually reroutes the call. In the instance that a central office or router that controlled the original and fallback PSAPs is offline, there is no way to redirect the call. These points of failure posed a problem because there was no mitigation strategy. The 9-1-1 call couldn’t go through, putting Californians at great risk. 

Coordinating Dozens of Moving Parts

I've worked for the California Governor's Office of Emergency Services (Cal OES) since 2011 and have served as our Statewide Interoperability Coordinator (SWIC) since 2014. I am in charge of California Emergency Function 2, which supports communications during emergencies, and have developed a number of statewide catastrophe and emergency response plans. 

I am also the Lead Program Manager of the State's Broadband Services Division and oversee California's public safety broadband infrastructure. In 2016, I was named California's 9-1-1 Emergency Communications Branch Manager, where I oversee our 9-1-1 network comprising 438 PSAPs with an annual volume of 27 million calls. 

As you might expect, there are dozens of moving parts involved in these efforts. I have to make sure they all work together to provide seamless and always-available 9-1-1 services for the most populous state in the country. Upgrading our system required this coordination on a much larger scale.

By 2018, the regulatory, funding and procurement pieces were beginning to come together and we had a clear NG9-1-1 strategy. Cal OES, working with CDT,  prepared an RFP for our NG9-1-1 system that included the requirement to submit tariffs to the CPUC that aligned with the RFP. To ensure the ability to facilitate collaboration, we released a draft version of the proposal and collected feedback from select vendors. After this initial consultation process, we released the full RFP and held one-on-one meetings with all interested vendors, including those who helped us clarify our needs during the pre-procurement phase.

Unfortunately, that initial RFP wasn't funded and the process had to be put on hold until California passed legislation to provide the necessary funding. We didn’t have to start entirely from scratch, though; this time around, we returned to our shortlist of vendors, including Atos Public Safety. We sat down with Atos and all interested vendors again, but this time, we went deeper into the project specifics. We worked out a very detailed proposal that refined the statement of work and the technical requirements to fully align with the state's 9-1-1 needs. This process was part of the selection criteria, and ultimately, Atos Public Safety put in the best offer and earned the highest number of points during the final evaluation to provide services as the Prime Network Service Provider.

An Emergency Network Based on National Standards

Our NG9-1-1 network is based on the i3 standard developed by the National Emergency Number Association (NENA). NENA i3 introduces the concept of an Emergency Services IP network (ESInet). It uses IP-based signaling and media tailored to the modern devices and services—like smartphones and text messaging—we now use to dial 9-1-1. Among other things, ESInet-compliant systems use Session Initiation Protocol (SIP) headers to route calls to a specific PSAP and deliver location and callback information to emergency responders. 

To ensure that California's NG9-1-1 network can handle a high volume of calls while maintaining high availability and operability, we have built redundancy into every aspect of the NG9-1-1 system. Each PSAP will have two NG9-1-1 providers to ensure redundancy. Atos is our primary partner and will have a presence at every location. A second vendor will provide fallback equipment. Atos Public Safety also developed the interface control document that set out our technical requirements, allowing us to choose the right equipment and create a roadmap to migrate the entire state to our new NG9-1-1 infrastructure. 

However, the NENA i3 standard is written assuming every telecommunication carrier can interface with NG9-1-1 via SIP, but this is not the case. As California implements the NG9-1-1 system, many carriers will require a gateway that converts the incoming signal to SIP.  In addition, because a carrier can only direct 9-1-1 traffic to a single location, we have implemented a staggered rollout of our NG9-1-1 system. By staggering the rollout we are able to go live in each PSAP as the carriers and PSAPs are ready instead of waiting six months to a year for every PSAP and carrier to be ready.

Our Just-In-Time Approach

We planned our "just-in-time" migration strategy by looking at the state's 45 selective routers and choosing the three that were simplest to convert to NG9-1-1. Our two vendors at each PSAP worked separately to install the equipment at each PSAP. Once both vendors have completed the install and the go-live testing, the PSAP is ready to receive a call in the NG9-1-1 environment. We then looked at the PSAPs that the go-live PSAP could transfer to and ensured that all transfer PSAPs had also completed go-live testing. At this point, the PSAP is ready for NG9-1-1. We then designed a test to verify whether call processing equipment could receive a call across legacy and NextGen 9-1-1 infrastructure simultaneously, because during the transition PSAPs will continue to receive some 9-1-1 calls from carriers over the selective routers, while others are received over NG9-1-1. 

We worked the carriers connected to each selective router to determine a transition plan for each carrier. We then directed the vendors to build the network connections to NG9-1-1 and test calls were made for each carrier to the PSAP prior to go-live. When directed by Cal OES, the carriers stopped sending 9-1-1 calls into the selective router and started sending them into the NG9-1-1 system for that PSAP. We then repeat the process for the entire state and transition every PSAP. Using this approach, we plan to migrate all 438 PSAPs to our NENA i3-compliant network by July 2021.

The immediate benefits are tremendous. We're bringing a secure IP connection into every single PSAP, and that network has built in redundancy and multiple paths into every single PSAP. We can dynamically route calls based on network conditions and transfer them to any PSAP in the state, and every PSAP has a built-in fallback. We will have better location accuracy, and much-improved reliability.

It will be the first time this kind of network exists anywhere in the state of California. With the NG9-1-1 system in place, we can do other types of IP integration into the PSAP. We couldn’t do any of this with our old equipment. Next Gen 9-1-1 is what makes it all possible. 

Looking Beyond NG9-1-1

We've already started our next major project, which is moving our on-premise call processing equipment (CPE) to the cloud. Once we've completed our NG9-1-1 rollout, we plan to update CPE at each PSAP with cloud-based CPE. 

We also want to consider integrating our NextGen network with a statewide computer-aided dispatch (CAD) database in compliance with the new AB9-1-1 legislation. Finally, we will incorporate additional 9-1-1 related information and hope to eventually route OnStar in-vehicle security calls, medical alerts, crisis center text messages, and suicide hotline calls to our PSAPs.

While the move to NG9-1-1 is mainly a technology transition, it also requires a shift in mindset. Public safety officials and PSAP operators are often hesitant to adopt new technologies because so much is at stake. We also have specific paradigms and processes in place at the local level and can't disrupt workflows by introducing new technologies abruptly.

I worked with Scott Fischer and Olivier Cost at Atos Public Safety to ensure Atos leadership understood that we needed to integrate this new technology into our PSAPs without disrupting operations or bombarding our people with information. One of the key success factors in this partnership is the willingness to listen to specific needs, and then being flexible enough to ensure the right technology aligns with those needs and is rolled out in a way that the PSAP can manage implementation.

Atos and Cal OES have created an open environment to have these conversations. Working within our specific contract and state rules, we have gotten to a point where we can say we need X, and Atos can tell us what’s possible, what’s not possible, and how we can go about filling the gap between the two. Once those pieces fall into place, a workable solution emerges that makes everyone happy.

Leading So Others Can Follow

We partnered with Atos to renew a critical piece of infrastructure. Our digital-ready NG9-1-1 network will help the state save lives and deal with emergencies for years to come. But we’re not just in it for ourselves. 

California is very active in NENA and NASNA (National Administration of State 9-1-1 Administrators) to establish the groundwork for fully compliant end-to-end i3 and NG9-1-1 solutions. We have worked to vet as many vendors as possible and to develop easy-to-follow frameworks that can be used by smaller states that may lack the technical resources to bear much of this burden on their own. We have also been very open about our procurement process and offer support to other state 9-1-1 administrators so they can make standards-based decisions informed by best practices. 

As this year's wildfires and COVID crisis have shown, we need to be prepared for anything. Cal OES is paving the way for future 9-1-1 administrators. We hope that other states and jurisdictions around the world will follow our lead.