Automation: How To Shift An IT Department's Culture

At the company I work for, we make memory. We’re one of the premier manufacturers of memory worldwide. We create DRAM, flash, DRAM modules, and SSD drives just to name a few. Anything that uses DRAM or Flash memory likely has some of our products in it. Everything we do is directed towards creating these products, reducing costs, or innovating. To be the best.

As a senior software engineer in the IT department, I’m focused on tools used by IT. Simply put: I develop software to make IT faster and more efficient. “Efficient” being the key word, yet it seems whenever we discuss efficiency, we encounter roadblocks, especially, it seems, when it comes to automation. 

Traditionally, roadblocks to automating processes come in the form of a mindset that replies to questions about efficiency with the adage: “We’ve always done it that way.” Over the years, I’ve found that others in my position struggle with innovation and cling to a “stay-the-course” culture. I also find many employees talk theoretically about what automation might look like, but are not always successful in the implementation. Because of this, the ability to deliver automated services has been challenging from my corner of the world in IT. 

For nearly 20 years, this mindset has puzzled me. Employees read about automated delivery benchmarks, think it’s great and remark, "Wouldn’t it be wonderful if we did this?" Yet when it comes down to making it happen, many companies fail or don't start, even when it can result in massive time and cost savings. Why? 

It comes down to a culture of hands-on control. Some employees believe they personally need to ensure every detail of a process is correct. When asked to quantify the conditions of those details, they often have a difficult time doing so. The true value of automation is best demonstrated by example to help others visualize what the end results look like. People need that “aha” moment and the realization: “Okay, manual is inefficient, error prone and requires additional personnel.”

When the value of automation is illustrated and the ability to implement the process is defined, employees recognize the waste of having someone at a keyboard performing these low value tasks. The realization is they want a self-service, automated system, from beginning to end; that’s when the floodgates open with new process automation requests.

Creating the “Aha” Moment: Coercing an IT Department’s Culture

Back in 2015, to provision a VM workstation the requestor would submit a request through the Service Request Management application and then wait for up to a week to have an IT resource manually complete it. I thought this was insane, because the execution time to provision the VM takes minutes. I knew as an IT department we could do better.

I approached my manager and the rest of our team to outline what I thought the best approach would be going forward. Everyone on the team was very supportive, as they could easily see the time and personnel savings that this automation could achieve.

The current process used BMC Remedy ITSM Service Request Management as the user-facing request mechanism, and it seemed a logical choice to keep this in place.  The Service Request Management application would create an incident in the BMC Remedy ITSM Incident Management application and route it to a support team to complete the fulfillment of the request manually. This is where I saw the opportunity for automation.  

To accomplish this automation, we used Microsoft System Center Orchestrator, a visual workflow engine. A runbook within Orchestrator would query the Remedy Incident Management application, searching for new VM workstation incidents waiting for fulfillment. Upon finding appropriate requests, Orchestrator runbooks would step through the process, maintaining state by updating the incident with the state changes in the process. This would also help with debugging failed requests.

This process used to take a week of duration and approximately 90 minutes in staff time. The automation of this process condensed the duration to an average of 15 minutes. That’s a huge time savings for the requester and the IT staff fulfilling the request, who are now free to perform higher value tasks.

Automating this process was just the beginning—we needed to shift the culture in order to spur adoption. We needed to raise awareness to really drive automation home. That’s where necessity came in.

Trial by Fire

Right about the time I was getting ready to beta test the VM workstation automation project, my wife began working at my company as an IT project manager. Being highly experienced and a seasoned veteran, the IT department recognized her expertise and immediately assigned her to a high visibility, large scale project in the big data group.

To source this big data project, she had to bring in contractor resources. At that time she became acquainted with the onboarding process. The contractors were on the other side of the globe, offsite, and required remote access to our environment and a VM workstation. This was in addition to all the other normal onboarding processes for a new contractor. Due to the additional requirements of granting permissions to the big data hosts etc., 20+ individual Remedy Service Management requests were submitted for each contractor. Factoring in 10 contractors that needed to be onboarded, well, you can do the math. This was completely overwhelming, especially while trying to perform her role as project manager at the same time!  

The contractors were ready to go—on the clock—and they had no way to perform work. No workstation, nothing. She knew I could connect with the right people to get things done, so she sent me a message for help. It was really more along the lines of HELP!!! I gave her the contacts for requesting remote access and helped her to expedite (since that time, remote access has been requested to be automated). For the VM workstations, I told her I could make that happen before the end of the day.

Two hours later, I had all the VMs ready for use. She was absolutely delighted and relieved because the clock was running and no progress had been made up to that point. She could not believe it, especially when other IT resources were telling her it was going to take about a week or two. 

The speed at which we could spin up the workstations using Remedy and the automation scripts saved the company a significant amount of money, enabled this highly visible project to stay on schedule and ultimately be a great success. I was the hero for a day, which felt great, but better yet, it provided a glimpse to my peers and stakeholders of what was possible.

That day was one of the catalysts for a shift in our IT culture. It was at that point that stakeholders began to understand the value and necessity of automation.

Going Forward

Now, with the IT culture in full support of automation, my goal is to consumerize the customer experience and industrialize the service fulfillment. I’m working towards creating a self-service IT portal in order to automate the delivery of IT services that are currently manual. 

As the floodgate for IT automation requests widen, IT is also adopting Agile, DevOps and private cloud. Now, more than ever, IT teams are asking: How do we deliver infrastructure faster? How do we connect operations in-line? How do we industrialize the back-end so we can deliver more services on the front-end? All of this fits into an automation mindset—and everyone sees the value. Sometimes, it takes tenacity to illustrate value.  In the end, if you have the right tools and processes in place, the culture will shift and you will be a catalyst for innovation.