Blue Sky Thinking in Big Sky Country

Montana is Big Sky Country. It’s home to breathtaking vistas that are a feast for the eyes and a balm for the soul. Every year, millions of tourists come to enjoy iconic scenery, hiking trails, campgrounds and fishing. If you love the outdoors, our state is paradise. But when it comes to the great indoors, Montana is also home to world-class high-speed internet service providers who bridge the distance separating our sparse population.

I'm a senior network engineer at Mid-Rivers Communications. We provide internet, telephone, and cable TV services to residential customers across the state. We also offer enterprise class telecommunications and managed data services, including POTS lines and SIP phones, to businesses in Montana and neighboring states. One of our most popular innovations is our Wide Open with managed Wi-Fi home internet package. We offer 1GB speeds for a low monthly price and pay-per-use bandwidth. We no longer bother with data caps or different speed tiers. All of our customers get ultra-high-speed fiber and cable modem connectivity at a price that rivals our competitors' DSL offerings. 

I started here 15 years ago and have seen our customer base grow to 24,000 home and business users in that time. I've also weathered many upheavals on the technology front. Back then, smartphones and managed services were just hitting the mainstream, nobody was streaming, and the cloud was in its infancy. Up until last year, I worked as an information systems analyst. I did a lot of customer-facing tech support, helping residential and enterprise users with their computer and network device problems. I also configured our internal routers and switches and did troubleshooting on our network. 

At the end of 2019, I stepped into a role as a network and switching engineer. After all my years of frontline support and configuration duties, I gladly moved away from the implementation side of the company and started to focus on network design, planning, and testing. It is now my job to explore new technologies, discover their features, find ways to use them, and then pass them on to my colleagues for deployment. 

Mid-Rivers has been in business for a long time, and when I started some of our equipment was older. We had about 150 routers and switches in operation. Some of them were 10–15 years old and way past end of life. There was no support for them, and no way to update their drivers. Over time, this older equipment has been replaced and has made us secure and improved our MTTR if issues do arise.

In my previous position, I spent a lot of time maintaining our aging equipment, and it cost an arm and a leg. When something went wrong, I couldn't even call in a support ticket; I had to find a solution online or through trial and error. It wasn't very efficient, but replacing every single switch would have been prohibitively expensive, and configuring and sending out new ones to replace failing units took forever. 

Then there was the matter of disaster recovery. Several years ago, it was non-existent. We had DS3s and 45 Mb/s backhaul and a T1 behind that. If our 45 Mb/s connection failed, we switched to the T1, but that didn't give us enough bandwidth to meet our customers' needs. 

About five years ago, we hired a new general manager who was more willing to put time and money into updating our infrastructure. That came as a welcome change and a breath of fresh air at the time, but I only started to appreciate it fully when I took on my new role. 

The Ace Up My Sleeve

I’m used to being in the trenches, not leading the way. Fortunately, I had an ace up my sleeve: Cisco. I’m the company’s resident Cisco expert. I have years of experience with their products, and I understand how everything fits together. My familiarity and my expertise with the networking giant's technology give me the confidence and the authority to pitch projects to my superiors and to guide our implementation teams once these infrastructure initiatives were approved. 

Recently, we upgraded to Cisco Modeling Labs v 2.0. It's a network simulation platform that uses virtual versions of real-world Cisco hardware to model networks. You can see how your infrastructure will perform without having to touch any of your actual equipment. 

Cisco Modeling Labs is a great way to plan an upgrade or work out an entirely new topology. I'm using it to architect the upgrade of our backhaul using Cisco ASR 9000 series routers. We're also adding Multi-Protocol Label Switching (MPLS) and Virtual Private LAN Service (VPLS) to boost our service offering to business customers who operate geographically dispersed networks. I've never worked on a project of this scale before, but I know that Cisco has my back. 

On the compute and storage side, I recently oversaw our transition from our old Dell bare metal servers to virtual machines running on Cisco UCS architecture. The process started after a year of negotiations, during which Cisco stepped in and helped convince our manager. We had a Cisco partner assist us with the initial design of our primary data center. We have since increased the number of blades and mirrored this setup at our disaster recovery site. We now have a geographically-diverse location that will kick into gear if our primary site fails. 

We couldn't have done this with our old hardware, because it takes a long time to reboot a bare metal server to get back to a running state. Our VMs take only seconds to come back online, even a reboot on a server that was running some heavy-duty processes. The UCS is that fast, and the result is improved stability for our customers. 

Digging Deep? No Problem

I've done a lot of work behind the scenes to make sure that everything runs smoothly for Mid-Rivers' customers and our internal teams. To them, our newly adopted Cisco technologies are practically invisible because they work so well. There are far fewer configuration headaches and service disruptions, and people notice. 

A few weeks ago, a road work crew accidentally dug up some of our fiber optic and copper cabling. When our new manager came on board several years ago, one of the first things he approved was the creation of redundant fiber rings. We ran simulations to ensure that our failover capacity worked—and everything worked perfectly then—but now we were able to test our failover capacity in a real-world emergency.

The network switched to the backup ring and our customers experienced zero downtime. The network latency was higher due to running on the longer second loop, but the impact on our users was minimal. 

Every successful Cisco deployment means I can push for another one of the company's innovations, be it a new piece of hardware, or the adoption of a new or unused feature on a product we already use. I can ask for more because Cisco has already met or exceeded every one of our expectations. 

Imagining the Possibilities

We use Cisco Firepower on the front end of our Cisco hardware for easy management, but another big change I’m helping to plan is our transition to out new Cisco Firepower to ensure resiliency, make hands-on changes, and manage it ourselves using Cisco Firepower's single pane of glass interface.

We are also looking at adding Cisco DNA Center to our infrastructure. Firepower and UCS have shown us the potential of dashboard-based, single pane of glass infrastructure configuration, management, and automation, and we want to apply it to our entire network. 

With Cisco DNA Center, we’ll be able to configure all of our routers and switches from a central location, replacing switches as they near end of life with new ones. It's so much easier than dispatching a technician to every site to configure each switch manually. 

Cisco DNA Center also simplifies and automates security. In the past, we had to change the passwords on thousands of switches one at a time when someone left the company. With Cisco DNA, we can do it with a single keystroke, and we no longer have to worry that somebody onsite typed in the wrong password when setting up a switch. The reduction and elimination of human error are one of the most crucial benefits of automation. 

Once we’ve accomplished these upgrades, we'll be able to take full advantage of Cisco's entire suite of products. Apart from turbocharging network automation and configuration with Cisco DNA, we’d like to streamline our identity and access control with Cisco ISE. 

I also want to pursue additional Cisco certification for myself. My CCNA Routing and Switching credentials helped me move into my current position, and with more training I can get a deeper knowledge of the tools that Cisco has to offer. This will help me contribute even more to Mid-Rivers' continued success.

A Partner with a Heart

Cisco Support is one of the best things about the company. When I call support, I'm not speaking to an operator, but to a fellow engineer. It makes a difference knowing that I'm speaking with a peer when I'm dealing with a crisis. Cisco support will walk me through a problem, and I can even arrange a callback so that we can work on an issue during a scheduled maintenance window. I especially appreciate the fact that their people respect me. When they realize that I know my stuff, we get to work. 

I also feel a personal connection to Cisco. It's not just another faceless conglomerate. Their customer community, The Gateway, is an active part of the organization. There, I can reach out and network with peers around the world and ask for technical advice from fellow engineers and Cisco enthusiasts.

Every year at Cisco Live, the company hosts a community involvement zone where attendees help with everything from assembling meal kits to building furniture. That wasn't possible this year because Cisco Live was online. But when COVID-19 became a global pandemic, the company donated Webex DX80 units to healthcare facilities so healthcare workers and patients could communicate and collaborate remotely while staying safe. Those types of efforts mean a lot to me. 

I'm proud to partner with Cisco because the company understands that the real power of technology does not lie in spec sheets or feature sets but in its ability to transform the lives of everyone it touches. Here in Montana, it might not seem like I’m doing life-changing work, but our partnership with Cisco makes a difference to our customers—and to me—each and every day.