I have been in and around this industry for more than 30 years. Having been a backup admin at the very beginning of my career, a data protection consulting toward the middle of my career, and on the manufacturer’s side currently in my career, has given me a good sense of the challenges IT still faces today.
First, let’s talk about the title of this blog, or at least some of the choice words I have used. “Business-Centric”, that one always seems to get people talking. What does business-centric actually mean? Don’t we already focus on the business? Potentially, yes, but as I talk with more and more IT leaders, it is a concept or philosophy that hasn’t trickled down to all parts of an organization. When you create a business-centric approach, you are focusing on what is important to the business itself, not just what is important to IT success.
For example, if your company is in the hospitality industry, you probably depend significantly on online bookings, whether from your own website or partner & affiliate websites. You also depend heavily on your point of sale system at the various properties under management for check-ins, guest billing, service orders, maintenance, etc. To the business, having those systems up, running and available to the customers, and employees define “business as usual”. However, if we just look at it from an IT perspective, we typically see lots of VMs, data, and databases, and because we are IT, we would normally take on the responsibility for making sure all the VMs are up and running and the data is protected. As you know, just because a machine, or virtual machine, is up and running, doesn’t mean it is actually doing its job. The application may not be running, the database may not be in an operable state, we may have network issues, etc. Or, to the business, “we are down”.
I remember a time when I was consulting for a very large telecom manufacturer when we had a significant engineering server go down. Since I was the one performing the health-check on the backup environment, I was asked to remain on to assist the team. It was Friday (naturally) at 4:30pm when it was discovered by the SysAdmin that the server was non-responsive. Turns out one of the disks suffered a catastrophic failure and had to be replaced and fully recovered. Once we had recovered all of the volumes on that disk, the sysadmin and the backup admin felt the job was done, but the application admin still had issues.
just because the hardware is there and everything “seems” to be fine, doesn’t necessarily mean that is what your customers see.
This happened to be the server where the engineers were checking in and checking out their code for a new product. So, while the server was up and the data was restored, the application and productivity of this Global company were not. The IT organization did not have a business-centric view of the situation, it was their own view of the situation. Hardware replaced, disk partitioned and formatted, and data recovered, from the night before. Had they looked at it from the business side, they may have had a whole different perspective.
Remember, just because the hardware is there and everything “seems” to be fine, doesn’t necessarily mean that is what your customers see.
One of the items on my audit list was the fact that they did not have a DR plan for those systems, the company didn’t see the value (at the time) in spending countless hours creating a plan and putting it in place. After all, it was just the engineering servers and not the front line servers supporting the customers and distribution. However, what “up” meant to IT, didn’t align with what the Business defined as “up”.
This particular part of the business was down for more than 7 hours, and in anyone’s book that is a disaster. To make matters worse, the engineering project on that server was one of the most competitive projects the company was working on in order to maintain an edge in the market. The end result was all of the code check-ins from that day were lost because of the traditional legacy approach to backup. The backup window was 6pm-6am M-F, and they had just missed it by 90 minutes. This loss represented 1000s of hours of engineering work. It also meant that the release date for this new product would be delayed, allowing its primary competition an opportunity to release first. It affected the company on Wall Street once it was announced that they were delayed, it affected customer confidence as they were heavily marketing this new product and its release date, and it naturally affected the actual sales compared to the projected sales. Had they taken a business-centric approach to this situation before all of this happened the disaster may have been averted altogether.
Building a Business-Centric Approach to IT Recovery
I believe one of the ruts we get in as IT professionals is we tend to treat all data and systems equally. Maybe not at first, but over time, in the fast pace hustle of mergers and acquisitions, new lines of business, etc. we may find ourselves doing just that, treating all data equally. While it is true that we do have certain procedures for data and systems we know is critical to the company’s business, I think we tend not to review those as often as we probably should. So, it stands to reason that if you want to view the business’ digital assets in a way that reflects the various levels of criticality, then you should conduct some type of review of these assets to understand the value each hold for the company. In short, this is a Business Impact Analysis. What does it mean to the company when X service or application is unavailable? What does it cost when it is unavailable for an hour, two hours, three hours or more? If those questions cannot be answered, then it starts to become very difficult to truly prioritize during a recovery and more importantly, the frequency of the protection schedule based on those answers.
Enlist Executive Sponsorship
When you embark on this journey, you may quickly find that some of the questions you are about to ask of the business unit manager(s) may be more difficult for them to answer than it is for you to ask. I call this the “Storm before the Calm”, and it was the title of my consulting document I used to train my clients on how to conduct a BIA.
You see, leaning in on specific aspects of the business may expose something the BU never thought of or imagined. It may also expose to you an area where the level of protection for this particular BU is less than what it should be for the given value of the service/application within the BU. This is why it is vitally important to gain an executive sponsor within the company. This does a number of things for the entire organization, firstly it shows a commitment from executive management the important nature of this project and fully supports the transparency required to deliver a high valued asset from this BIA. It also expresses the need for both IT and the Business to come together to help solve problems together which in turn shows a commitment to up-leveling its service to its main constituents, chiefly the customers. In my experience, when you begin asking the questions required to build a BIA for the various business units within the company, certain areas that were once thought to be super critical to the company are reduced a few levels, while others are bumped up a few levels. This is such a great way to really understand the core of your business, it also provides IT a ‘seat at the table’ to engage with the business and ultimately show how IT may be able to help the business in a number of other ways and avoid being viewed as just “overhead”.
There is so much more to cover, but I believe I offered a good overview. I have given several workshops, training, intro presentations, etc. on this topic over the last 20 years, and the amazing thing I have found is the relevance of it today is the same if not more as it was 20 years ago when I conducted my first workshop.
One of the biggest “a-ha” moments many of my clients had was when the business-centric IT plan was complete, it looked very much like a business plan. Complete with an Executive Summary, Gap analysis, Financial Assessment, Risk Mitigation, etc. Something the business leaders could read, understand, and more importantly, support. Within this plan, based on the BIA, you will have the real keys to build a better, more business-centric approach to your recovery strategies.
This is the foundation of The CTE Group, my new company. Be sure to follow me there, http://www.thectegroup.net.