Using CI/CD to Increase Velocity and Wow Your Customers

These days, every customer—whether an individual or a large company—expects both quality and speed. When they come to you with a problem, they need it resolved now. If you tell that customer, “Just wait a quarter,” good luck come renewal time.

But if you can ship that feature in a week, that’s a moment your customer will remember. By moving quickly, you show that you listened and were receptive to their needs. This isn’t just about bug fixes—the same goes for shipping new features: time to market is king. To move at a speed that will delight your customers, it all starts with the right infrastructure. 

Testing Shouldn’t Be Painful

At mabl, we’re all about building velocity into your business. We do that by fixing the endemic issues of automated testing that cause unnecessary delays. Tests that used to be difficult to write? We make easy. Tests that used to be brittle? We make robust. Things you used to have to manually fix yourself? We use machine-learning algorithms to fix them automatically. 

With easy, automated testing, you can release faster. Most companies are still in a quarterly, monthly, or maybe even—for the fastest of them—weekly release cycle. The reason for the delay in releases is they don’t have confidence in their code. They think there are bugs. Actually, they know there are bugs.

But with robust testing, without manually creating and fixing tests, developers can release in smaller and smaller increments. Every time they reduce the increment size, the cost for each release decreases, because there’s less weight. You see some companies having to test constantly because they put everything into one release. This means higher risk. The smaller the commits, the lower the risk. The result is greater velocity and faster time to market. 

Obviously, we need to think about how we can provide velocity for our customers’ customers, and for mabl's customers as well. The challenge, though, is we have to balance that with quality. We have 16 engineers committing code all day every day, and we push that out as rapidly as we can across dozens of repositories. That’s why continuous integration and continuous deployment (CI/CD) is a critical part of our infrastructure.

The Move to NoOps

Before we get into our CI/CD implementation, it’s important to talk about our tech stack at mabl. We’re largely based on Google Cloud Platform. In addition, we strive to have a serverless infrastructure. That is, even though on a given day many hundreds of machines are running in the cloud for mabl, we don’t own a single machine. No person has to stand up and support a machine by hand.

In other words, we’re a NoOps shop. Who here works on operations full time? No one. Everything is a service provided by vendors like Google Cloud because that way, no one here has to worry about setting up infrastructure and managing ops. All of us here have worked at other places where we had to set up machines. We know how much time you lose trying to keep them running. By using only SaaS and PaaS providers, we increase our velocity. We rely on key partners to help us move at the speed our customers demand. 

Container-First—The Only Way Forward 

From our first days as an organization, the issue we had to solve was that the cloud revolves around containers. As I said, we don’t want people sitting around hand-tuning custom servers. With the offerings now available, we can simply take a container that runs on your laptop, ship it and run 500 instances a few minutes later in Kubernetes. Kubernetes is a core underpinning of all our test-running clusters at mabl, so we knew we were going to run lots of containers. To give you some context, last year alone we deployed millions of containers.

We needed a way to speed up our development process that didn’t just work well with containers—it had to be container-centric. A lot of legacy providers were trying to catch up, but containers aren’t a first-class citizen for them. That narrowed our search to true container-building products. We chose CodeShip. For us, the deciding factor was CodeShip's great Google Cloud Platform integration.

We loved how every build is its own little island. You set up your build and everything works within it. Many legacy tools had byzantine configuration screens. With a confusing UI, it becomes unnecessarily complex when someone else on the team has to reconfigure the build. A simple, self-contained build model and strong cloud vendor integration is superior for onboarding and team productivity.  

With the simplicity of CodeShip, along with Jet tool and YAML configs, onboarding is a trivial process. Someone can just clone the repo, change the name to make a new product and they just automatically inherit all the configuration and pipelines. CodeShip did exactly what we needed.

Plus, it didn’t hurt that CodeShip is local here in Boston and they made the effort to visit our office a few times to build that closer connection. We were actually able to sit down and talk with them, which is something you don’t usually get to experience with most vendors. 

A Straightforward Development Pipeline

I have to admit, it was a bit of a mind warp thinking about how to implement all of this. It’s a chicken inside an egg inside a chicken: We needed a container to build my container to deploy my container. The team spent a day thinking this through and reviewing the CodeShip docs, which are quite thorough and only a few pages long.

Our initial setup with CodeShip was straightforward. We have a simple YAML file that goes through the respective steps of building our containers, running our tests and pushing containers out to Google Container Registry. 

We’ve also set up exclusion flags for most files. So if someone is working on a feature branch, it will perform a particular series of steps. If someone has just merged to master after their pull request, it will automatically deploy itself to our integration environment. And if we use a specific git tag, it will then promote itself to production.

For our developers, this was a seamless integration. All they had to do was commit their code and push it to GitHub and, automatically, their feature branches get built and tested. If they just merged their pull request, their code automatically went into the integration environment. From there, we use the GitHub integration with CodeShip so we can easily see which tests passed during a code review. The developer can then push their code to production. It’s an incredibly straightforward pipeline that everyone found easy to understand.  

Skyrocketing Stats, with Practically Zero Overhead

After implementing CodeShip, we noticed right away that our stats skyrocketed. Our small team of 16 engineers was doing 1,000 builds a month. Our commit counts and pull requests for the year after implementation of CodeShip were well above 10,000. 

Our team started to move incredibly fast. CI/CD and CodeShip have reduced our time to market from weeks or days down to hours and minutes. Even our UI designer can make a few tweaks and it goes to production. 

The CI/CD environment has a big impact on two groups of customers: current and future. 

For current customers, we send out a weekly email about all of the new features we’ve released. They get to see the constant innovation of our team and how quickly we roll out new features based on their requests. If we can ship that feature within a few days, to have it included in the next week’s email, that has a major impact on retaining or gaining that customer’s business. There’s a shock-and-awe aspect to it, because customers aren’t used to tech companies responding so quickly. This rapid deployment process also helps us quickly integrate new features that major enterprise customers need. This helps us close major deals with prospects, which our sales team loves.

For our development team, the operational overhead is practically zero. That’s definitely where the final impact is for us with CI/CD. The cost of doing all this is low and things move incredibly quickly, so we are able to maintain that high velocity, while at the same time our product is updated constantly with new deployments. 

Worry-Free Support

Throughout this whole process, CodeShip has been receptive to our needs and feedback. The handful of times we’ve used CodeShip support, they got back to us quickly and were quite thorough. I think it says a lot about the quality of the product, though, that we rarely have to use their support.

One aspect of their support I truly value is the CodeShip Status Twitter account. We all know that any major cloud provider will have ups and downs. Things might get a little glitchy for a few minutes in the cloud, and then it gets fixed. It happens. 

What bothers me is that some companies are not transparent about what’s happening. When things get a little weird for a few minutes and I don’t know whether it’s us or something in the cloud, that puts our team in a state of paralysis. We’re happy with the fact that whenever there’s a hitch in CodeShip, they always provide a status notice, so we know exactly what’s going on. I appreciate that transparency and candor, because it takes away the worry.

When Every Company Is a Tech Company, Velocity Matters

CodeShip has let us streamline and simplify our containerization. We now build test and deployment infrastructure in such a scalable, automatic fashion that we sometimes forget we even have CI/CD. Today, it just comes naturally. It’s not something we have to think about on a daily basis. We don’t spend much time or effort setting it up or running it. It’s a foundation that we as developers can depend upon. 

The most exciting thing for mabl is that we now have a great foundation to build upon. With the right infrastructure and the platform in place, we can focus on higher-level features. We can move on from standard features to building “delighters”—the awesome features that customers didn’t even know could be done. 

While this has been a huge success for mabl—we’re not a unique case. Every company out there needs to consider how to maximize their velocity and understand the operational overhead weight on their development cycle. Because the most nimble players will implement these sorts of pipelines to pull away from the rest of the pack. 

But CI/CD isn’t just for SaaS companies. Every company is a technology company. So the question becomes: Do you want to play catch-up, close that gap, and keep pace? Or do you want to be the company that sets the pace? For mabl, the answer is obvious and we're a leader in the pack.