Futureproofing a Cloud-Agnostic Platform with Instana

As a sysadmin, part of my job is predicting the future and preventing the worst from happening. With technology changing so fast, that's getting harder to do. But if you start with a solid set of tools and methodologies, you can futureproof your platform and create a workflow that will continue to serve you, even if you change infrastructure a few years down the road.

I'm a DevOps engineer at PathMotion. We're a cloud-based talent acquisition and employer branding platform that connects potential hires to employees within an organization. Applicants use our platform to learn more about a particular business and its corporate culture during the recruitment process. This creates a positive candidate experience that attracts the right talent. 

Even though I’ve only been with the company for less than a year, in that time we’ve gone through some major changes.

Exploring New Observability Options

My role involves managing our infrastructure, which is currently hosted on Google Cloud Platform (GCP). I am also responsible for maintaining CI/CD pipelines that help our developers accelerate our release and deployment schedule. We used to do everything on-prem, but about three years ago, we moved our production and staging environments to the cloud. Around the same time, we started monitoring system performance with New Relic.

A few months after I joined the company, however, our contract with New Relic was set to expire and our CTO asked me to explore other Observability and Monitoring options. A lot had changed since we'd first negotiated our deal with New Relic. Of course there were new vendors out there, but monitoring systems themselves have evolved over the last few years. 

In the past, when production environments and platforms ran on physical hardware, monitoring solutions measured things like processor cycles, bandwidth allocation, and storage usage. But cloud-based infrastructure, software-defined networks, and containerized software have drastically altered the landscape. Now, you can spin-up capacity as needed and scale to add new functions or new resources with the click of a button. As a result, today's monitoring solutions measure how well your microservices communicate. Your bottlenecks are no longer based on hardware, but the connections between all the components of your platform.

Our internal focus had shifted, too. On top of monitoring application performance, we also wanted to monitor all end user-facing performance to determine whether we were fulfilling our service level agreements (SLAs). We needed a tool that would track our service level objectives (SLOs) and our service level indicators (SLIs). These metrics measure the customer experience. They tell us how users interact with our platform and help us determine any pain points that they may encounter.

And then there’s the issue of human resources. Over the past few years, PathMotion has gone from being a monolithic platform to a series of complex intertwined microservices. There's a lot to monitor, and the entire responsibility has fallen upon my shoulders. There's no way I can manually keep track of everything as closely as I need to with all the changes happening constantly. We needed a monitoring solution that’s not only easy to use, and easy to understand, but easy to set up, automate, and use right away.

Easy to Configure and Understand

After looking at our options, Instana stood out to me as a platform that would meet our needs. Founded in 2015, Instana is one of the newer kids on the block. The company uses AI/ML and 3D visualizations to monitor the performance of microservices architecture and software running on container orchestration platforms like Kubernetes.

One of the reasons I went with Instana is that it's practically a turnkey solution. Sure, you have to do some simple configuration, but it's relatively straightforward, and you start seeing metrics and collecting actionable data right away. 

Another attraction was pricing. Some vendors offer a confusing à la carte menu, where you get a certain number of features at the basic service tier, but you have to pay extra if you want to level up. Instana gives us everything we need at a low monthly price per host; simple and predictable. There are no surprises, and the company's stellar technical support is included in that price. There are no variable costs. We get unlimited users, open standards, and more than 200 supported technologies for easy integrations. 

I recommended it to our CTO, and he approved my selection. We were ready to see how this could change our monitoring for the better.

A Growing Company That Keeps Improving

One of the benefits of working with a young, innovative and fast growing company like Instana is their willingness to keep improving, which means a constant stream of new and improved features. Some of the more established vendors put out a product and then only provide minor updates between major releases, but Instana is always adding something new. 

A few months ago, they rolled out a set of features for tracking SLIs and SLOs, and we signed up as beta testers. It was a great opportunity for us to get an early glimpse into these new features. These new tools went live recently for all users, and we are slowly integrating them into our technical and business workflows.

Redefining Underused Cloud Resources

Instana has helped us identify underused resources in our Kubernetes clusters.  Because we can see how our microservices interact, we have optimized our Kubernetes nodes and are only running the virtual machines we need to keep our production environment operating at peak performance. As a result, we have eliminated about 5–10% of our VMs. The reduced number of nodes helps us deploy more resources elsewhere. Should we need more capacity in the future, Instana can warn us if we are straining against our limits, and we can spin up new servers in seconds. 

And then there’s the team. When I talk to the team at Instana, I get timely and topical tech support instead of a sales pitch. They're easy to reach, and their documentation is stellar. Their business model relies on providing outstanding customer service instead of trying to upsell additional features or higher subscription tiers, and that means everything to a small company like ours.

A Cloud-Agnostic Future

I mentioned that we use GCP for our infrastructure, which currently has the best combination of features and pricing for our present needs. But everything changes so quickly, and we’re well aware that our infrastructure platform is likely to change as well. We may want to duplicate our resources to AWS, or any other cloud provider in the future as PathMotion will grow. 

Thanks to containerized microservices running on Kubernetes clusters, PathMotion is cloud-agnostic. We can move it to a new environment with the push of a button. Instana is also cloud-agnostic. It provides data about the way that our components interact with one another instead of only relaying how they use resources. If everything works in Kubernetes on GCP, it will work the same way if we migrate to AWS or another orchestrator. We can even port PathMotion to a physical server and get it to work on-prem. 

Computer scientists are notorious about following trends. Kubernetes may be hot today, but they may think it's passé in a couple of years. It’s great to have found a partner that can stay with us and continue to provide value, wherever we decide to go and whatever we decide to use.

Part of my job is to look at current trends and adopt the ones best matched to our goals, but I also have to worry about what's coming around the corner. Observability and automation are two of the biggest concepts right now, and they are also helping us prepare for the future, no matter what platform we choose in the coming years. 

Focusing on our Developers

We’ve simplified and automated our Observability, and have optimized our resources—all in a very short period of time. The result is better performance for our clients and more time to work with our developers. 

Freed of the need to continually maintain our GCP infrastructure, I can devote my time to improving our CI/CD pipelines. I can now focus on giving my coders the tools they need to accelerate the development and deployment of patches, performance improvements, and new PathMotion features. This gets them excited about their work, and they’re more eager to flex their skills for the benefit of the company.

Instana helps PathMotion transform talent acquisition, employer branding, and the candidate experience, and keeps our developers happy in the process.