What is Success for a BI Project?
Hello and welcome to this new blog post. Today we are going to talk about one of the most important ingredients of guaranteeing success of a Data Warehousing Project. Namely, being able to define what success looks like so that you have a much better chance of being successful.
If you ask the people on a Data Warehousing Project or “Big Data” project a question like this.
“What does it mean for this project to be successful? How is success defined for this project?”
Almost inevitably you will get answers along the lines of:
- If we make our date that is successful.
- If we have all the data we said we would have that is successful.
- If the users are happy with what we deliver that is successful.
- If we deliver on or under budget that is successful.
Think about that question yourself. On all the data warehousing projects you have been on, or done, or seen, what would people call “being successful”?
The reason success is seen as a function of “budget, timeline, dates, functionality, and happy users” is because this is how IT people perceive “success” for IT projects. I have seen this from day 1 I started in IT. I started in IT in 1982 and “success” was “we go live with this system over the Christmas break”. That is what we were heading for. A three day outage of the plant over the Christmas break and us installing the new computer terminals and installing the new system in those three days. I worked in a steel mill at the time and we were introducing an upgrade of a computer system.
Yes. We did it. We got the system in and running over the Christmas break. And it was called “successful”. It is a VERY rare IT person who looks past these very narrow definitions of success because the USAGE of the system is responsibility of the business users, not IT. So IT does not think in business terms for success.
Now. What is the danger in this approach? It is pretty clear when you look at it. I have seen many data warehousing projects suffer from this danger. The Business People do not have a solid grasp of what they see as successful and the IT people limit their view of “success” to the items mentioned. Budgets, functionality, timelines etc.
When there is this mismatch the likelihood of “success” is usually highly compromised.
Whenever I am asked to review a data warehousing project and give a report on the health of the data warehousing project I always ask to see the Project Charter, Business Requirements and Project Objectives as a minimum. I also ask to see what has been written about the exploitation projects that will follow on from the delivery of the data warehouse.
You might be amazed how often these documents are entirely aimed at a “technology deliverable” and not at a “business deliverable”.
Back in the 90s when we were first promoting the idea of data warehousing our prospective clients had one overwhelming desire as a “requirement” for their data warehousing projects.
“Reduce costs”.
It got so bad that I eventually had this great comeback which was along the lines of
“the best way to cut costs is to close down, is that what you want to do?”
I would persuade my prospects that what they really wanted was sustainable, long term, profit growth so that the company would be solid for many years to come. It would attract quality investors. It would provide good jobs and new job opportunities. It would produce great products and services at competitive prices to win a good portion of clients and retain them over the longer period.
Many of my IT prospects bought in to that idea. And my business prospects bought into it even faster!
In the 90s we were selling the idea of:
“Sure, lets optimise the business to make it run as efficiently as possible, and then lets invest the money we find that is wasted back in to marketing to sell more great products and services to people who need them or want them.”
So when I read these things like Project Charter, Business Requirements, and Project Objectives I can usually tell if the project is going to be a success just from those documents alone. If they are too focussed on budgets, cost reductions, timelines and dates, features and functions, it is likely that the project will “fail” in that it does not contribute to the profitability of the company.
Now. Over the years I have been able to tell “success stories” in sales cycles without naming the clients. Now that so much time has passed and everyone has moved on from some of my best “success stories” I can repeat these stories without concern for the those involved becoming at all upset about me talking about them. They have long moved on from the competitive advantage they gained from my contributions. These stories are so long ago that there is no one there now who was there then.
But these “success stories” are timeless in that the same issues exist today and the same remedies apply. Further, these success stories will point to how and why my clients have a 100% success rate when they take my advice.
Firstly, what is defined as success on my projects?
Almost always “success” is defined as
“the profit impact of the data warehouse pays for itself in the first year of operation.”
Now. It is true that not every client of mine defined success of my projects in that way. Sometimes we were building infrastructure that was absolutely necessary and if there was a positive financial benefit to it then that was a “bonus”. But in general the “success” we talked about to our clients was “you should be able to point to profit impacts that were impossible without the data warehouse, and they should be more than 100% of the cost of the data warehouse in the first year.”
Now, we said first year, but I was usually aiming at 90 days for break even. By placing the focus on profit we were able to more actively engage the business users in the project and get them focussed on the management decision making cycle and the investments they were making.
So let me give you two sales stories where there was remarkable success where others had tried and failed before us.
Australian Consolidated Press (ACP)
ACP was the largest publisher of magazines in Australia in the 90s. It published the countries two most popular womens magazines, Womens Weekly and Womans Day. It had 37% market share. It was owned by Kerry Packer who was the richest man in the country.
The Magazine publishing industry was highly regulated and it was highly competitive. It was also very mature and was coming under attack from the internet. Revenues and profits were falling and ACP was doing more and more poorly as the second half of the 90s rolled on.
I first tried to sell to ACP when I was working at Hitachi Data Systems in 1997. We lost out to “we will do that internally, no need to hire you expensive consultants.”
I had another crack at ACP when I was working for PriceWaterhouseCoopers. We were privy to some of their internal data and we developed a proposal for a data warehousing program for two years which was the standard sort of offer from PwC. We lost again to “our ERP vendor tells us they can do a data warehouse for us and we are going with our ERP vendor.”
By the time that project had failed I had moved on to Ardent, the makers of DataStage, as the Professional Services Manager. We had to negotiate a tortuous sales process because senior figures in the Packer empire wanted Axciom (part owned by Packer in Australia) to do the work. They did not want to allow the work to go to a third party that had no other relationships with the Packer empire. Axciom were making all sorts of promises and they were a “sister company”. It was a very tough fight but eventually we got the deal because we had the best sales stories of making money for our clients.
By now this had been nearly three years I had been selling to ACP. Getting a foot into the door of the Packer Empire was something worth working hard on for that length of time.
The timeframe we had to deliver version 1.0? Yep. 90 days. Three years of selling to get a 90 day project.
At the time the fortunes of ACP were poor. Kerry Packer himself called the company “a dog with fleas”. The guy who ran ACP was the best friend of James Packer, Kerrys son, and they had grown up together. Kerry personally intervened and fired the CEO of ACP due to the poor performance of the company. The new CEO, John Alexander, was very dubious about being given ACP as a challenge. Everyone thought that might be something of a poison chalice, and reasonably so. When I met John face to face to talk about the project one of his comments was “I sure hope you know what you are doing because we really do need some help around here.”
Of course, I had NO IDEA what happened inside a publisher because I had never worked on one before. So I had NO IDEA how one might go about improving the fortunes of the countries largest publisher. I was especially worried because it was clear no one at ACP had any idea about how to turn around their fortunes either.
Kerry Packer was the richest man in the country and he was not able to find anyone to turn around this, the largest publishing house in the country? Why would I imagine that I could figure out a problem that had, seemingly, evaded the best publishing minds in the country?
In those days I had a unique way (in Australia) of approaching the Business Requirements workshops that we used to hold. It was technique that I was taught by Metaphor Computer Systems which included envisioning what the client would call “Outstanding Success”. This was a very powerful technique that allowed us to get from the clients what would be called “Outstanding Success” two years in the future if they were able to achieve it. This would put a “stake in the ground” to aim at in terms of business success.
At one of these workshops one of the publishers made the following comment. She was asking me, as a consultant to her company, did I know this. She said words close to the effect:
“Do you know that we do not just publish issues of the same magazine? Do you understand that our product is different each week?”
I said
“Well, no, I don’t understand that. Can you please explain that idea to me more?”
She said:
“Sure. It’s like this. We put different people on the cover each week. And we write different articles inside the magazine to suite the audience attracted by the cover. Sure, we cycle through these themes as weekly and monthly magazines. But we might take some months to cycle back to the same type of people.”
And I said:
“That’s interesting, can you give me some examples to help me understand better?”
She said:
“Sure. Lets look at the royals. The royals appeal to a certain set of people. We can put the queen on the cover. We used to put Charles and Diana on the cover a lot. We can put Andrew and Fergie on the cover. You know. Any of the royals on the cover appeals to people who are traditional, who are monarchists , who support the Queen, that sort of thing.
And then we put stories in the magazine about the royals. We put in opinion pieces saying we should stay as part of the commonwealth with the queen as head of state. We get advertising that is royal friendly. The whole magazine will be aimed at people who are royal friendly.”
“And then we have Tom and Nicole. (For those who don’t know Tom Cruise was married to Nicole Kidman and they were an incredibly popular couple). Tom and Nicole represent “new and modern”. They are movie stars, not royals. They have a different base of support. Different fans if you like. Much younger. Mostly less concerned with politics.
So when we put Tom and Nicole on the front we will have opinion pieces about the republic debate, we will talk about the movies that are out now that are suited to young people. It will be a “modern and new” look to the magazine that month.
We do this because if we hit the same sort of demographic month after month we alienate those who are not in that demographic. So even though people who like Tom and Nicole are less likely to buy a “Royals” issue and those who are “Royals” are less likely to buy a Tom and Nicole issue. By presenting different issues with different themes we get more sales than if we pushed just one or the other. Does this make sense?”
And I said:
“Sure. That makes a lot of sense. Your product is so different from week to week that you consider it a new product even if it gets recycled to an extend a few months later. Is that what you are telling me?”
And it was.
This conversation was noted in among all the other conversations that were had. I then had a meeting with the guys who were the experts on the ERP. When I got to this conversation I asked them.
“Please tell me how the number of copies to send to each of the 7,000 locations for each magazine is calculated in the ERP.”
As I suspected the number of copies were calculated based on a straight history of sales to that location. This method left ACP with about 25% of copies unsold of their #1 magazine each week. It also left them with many “sell outs” of their magazines. A “sell out” meaning the retail outlet had no copies left at the end of the magazines life. This is not desirable. You still want a few copies on the shelf so that you keep the shelf space and so not allow a competitor to get that shelf space.
So we added one more idea on the agenda for testing with the data warehouse when we got it. Analysing the data warehouse sales to determine if the cover of the magazine showed correlations to sales in the outlets.
The theory was that if “Royals” vs “Tom and Nicole” truly did appeal to different people then perhaps these people bought their magazines from different outlets because they live in different places. If this was so then the ERP would not be able to know this and it could not adjust the copies allocated to each of the 7,000 retail outlets to match the forecast sales for this cover and this content.
When the data warehouse went live and we had done all the MUST HAVE reports we decided to dive in to this question because sell outs and over-supplies were hugely expensive for ACP. As you might have guessed from telling this story. We found very high levels of correlation between the “theme” of the magazine and the sales at the 7,000 outlets. We got the publishers to tell us all the different “themes” they ran with and we profiled each of the themes.
We then developed a forecasting mechanism for the ERP in the data warehouse and we fed the ERP with our custom forecasting mechanism. Now, we expected some improvement. (I was gone from the project by now so this is what was reported to me later.) But the improvement was MASSIVE. It was not only that we were under supplying outlets where the magazine would be popular. We were over-supplying outlets where the issue would be less popular.
The team were able to learn over a period of about 2 years how to recalculate and recalibrate the forecasting module for number of copies to send to each of the 7,000 outlets based on the profiles and the 10 years of historical sales data we had available. This recalibration reduced the oversupplies. But more importantly, it reduced sell outs as well. So people were no longer buying a competitive magazine so often.
There were other things the team did. This was not the only change. But it was THE BIG CHANGE that led to a massive performance improvement by ACP. Inside 2 years ACP had doubled its gross profit. Market share went from 37% to 42% in a mature, saturated marketplace. Kerry Packer himself called ACP “The Jewel in the Corporate Crown” just 2 years after calling it a “dog with fleas”.
John Alexander, for being willing to take the risks necessary, was rewarded and moved up the Packer Hierarchy, eventually becoming CEO of the group. Kerry Packer himself mandated all his companies have a data warehouse just like ACP. The consultant I put in as the Business Analyst was named as the signatory and responsible for all Packer Company BI Solutions. All Packer companies had to have his blessing on their projects. He called me ever so pleased and said “I have died and gone to consultant heaven!”
That is what I mean by “Successful Data Warehousing Project”. You double the gross profit, you garner 5% market share, and the richest man in the country says that your client has gone from a “dog with fleas” to “the jewel in the corporate crown” in just two short years. After Kerrys death James sold ACP and it was rated as one of the world top 10 publishing houses when sold off.
Just in case you are wondering how long it took to get the investment back? ACP put AUD300,000 in to the version 1.0 data warehouse with later funding to be dependent on success of the project.
So the target was to impact the profit line by AUD300,000 inside the first year.
While we were building the data warehouse the Managing Director of another Packer company came to us looking for ideas. He ran the distribution company. This company delivered ACP magazines to the 7,000 outlets. It also delivered competitive magazines and other products. Their carriers went past all 7,000 outlets at least once a week. Many of the carries went past some outlets 5 days a week Sunday to Friday. The distribution data was inside our ERP which, of course, meant we had access tot competitive information about other magazines which was not allowed to be passed back to ACP publishers.
We started the meeting and the Managing Director of the distribution company explained his problem. They were bidding for the “Readers Digest” distribution deal for the next 5 years. This was the contract to distribute all copies of “Readers Digest” to all 7,000 outlets for the next 5 years. A VERY substantial contract.
He explained that he had been personally informed that his company would not be getting the deal. When informed he begged for one more meeting and he had been granted this concession by the senior management of Readers Digest. He was here looking for ideas. What could he sell to them that he had not sold so far?
We were brainstorming and kicking ideas around. No idea was “too outrageous” because of the size of the deal. He encouraged us to throw even the wildest ideas out there. So the consultant I had placed there says “Why not promise them that our data warehouse will have their sales numbers on their desktops in their offices on a daily basis. 9am in the morning. They will have their numbers.”
He MD looked at him and said: “Can we actually do that?”
The consultant said: “How the hell would I know? Besides, it doesn’t matter if they don’t buy, right?”
The MD said: “I like the guy! Let’s add that to the presentation! It sounds like a great idea!”
About two weeks later the CIO asks the consultant and I to come in to his office. He had a voice message on the phone from the MD of Distribution. They were obviously having a HUGE party. The MD was screaming “We got the deal! We got the deal! We got Readers Digest! You guys have to deliver now!”
Now. I, as a consultant, was never told the value of the deal. But it was well in excess of AUD300,000 that is for sure. Delivering Readers Digest to 7,000 outlets for 5 years would be worth millions of dollars, that is for sure. So, here we were. We still had a few weeks to run on our project even to get it into production and the CIO was shaking our hands saying “Gentlemen, congratulations, I have never seen anything like this in my professional life before. Well done.”
When your data warehouse becomes an essential competitive weapon that wins you multi million dollar deals? You know you are doing something right!
This is what success of a data warehousing project looks like. This is one of my best sales stories because the company in question was in such a mature and competitive industry that was also being pressured by one of the largest technological changes in the last 20 years. The internet. To turn around such a company based on the data inside their ERP and other systems was an amazing feat.
And in case you are wondering? This is not the only example of this sort of thing I have. It is just one of the few that I can so easily talk about since the company has been sold twice since then and no one from that project is still with the company.
Qantas Cargo – Different Techniques Leads to Success
Everyone knows Qantas, the national carrier of Australia. They have an international cargo division that sells cargo space on the Qantas flights and this is a very profitable part of the overall Qantas business. The system that was used to manage cargo was from Unisys and it was not very good. It certainly did not have a data warehouse that would help Qantas Cargo optimise their business.
In the late 90s Qantas Cargo decided they would try and build a data warehouse to optimise the Qantas Cargo business. As I recall it was explained to me that the prior three projects had failed. The new project manager had been told that if this effort failed then he should resign as there was no way forward for him in the company. The man, lets call him Fred, had been in airlines for 20 years so he was very concerned that this project would be a success. He later told me he called 10 or so of his most trusted colleagues and asked them “If I have a really tough data warehouse project that HAS to work who should I hire?”
Fred told me much later that 6 of the 10 or so gave my name and no one individual got more than one mention. Fred had decided that with that sort of reputation I was the man for the job. Fred still ran us through the hoops of a “tender” to squeeze us on price etc. But we were, apparently, destined to get the job as long as I presented as his colleagues had claimed I would.
By this time I had been “Constructively Demoted” back to a consulting role and I decided that Qantas Cargo was so important to us that I would to lead the project myself. Fred was right. The Unisys system was a nightmare and this would be a very tough project. It was no surprise to me the three previous efforts had failed. The reason for the importance was that Qantas had bought in to an Enterprise License deal for DataStage and this would be the first large DataStage project at Qantas. So the future of DataStage also hinged on this project.
As I said above we ran a unique workshop process. However at Qantas, for reasons that I can not recall, we decided to run the process as a series of interviews. So I set about interviewing the 20 most senior people at Qantas Cargo. The schedule was 3 or 4 interviews a day. No one can do more than 4 interviews a day at that sort of level.
We also did not record these interviews as there was sensitive information to discuss about personnel. It is often my habit to record interviews on tape. But when personnel issues are being discussed recording will stifle discussion because no man will ever believe that a recording can never find its way in to the public whereas “conversation” can easily be dismissed as “a misunderstanding”. When you do not record interviews the work of the interview is much harder because you have no backup tape to go to. I just wanted to set that as the scene.
So I ran our “Outstanding Success” technique that is so incredibly successful for all the clients that we use it on. But it is not what people expect. And we ALWAYS get complaints about it.
Qantas Cargo was no different. About 2 weeks in to the project Fred and I were summoned to the CEOs office. The conversation, summarised, went like this.
CEO:
“My people are complaining that you do not know what you are doing? Can you tell me what this is all about?”
Peter:
“Sure. I bet they are saying things like “He is not asking us the question we are expecting. He is not asking us what data we use, what reports we use, what reports we want, those sorts of things.” Would I be correct?”
CEO:
“That’s quite remarkable because that is very much the flavour of what they are asking. They tell me you know nothing about Cargo, about our business, and you are not asking them questions that would lead you to know about our business and our reporting needs. Quite frankly, they don’t trust you or believe you. You have a real problem here.”
Peter:
“So may I ask you a question?”
CEO:
“Sure”.
Peter:
“The project has failed three times before we were asked to come in. I have the documents from the failures. Those documents define the data your managers say they need. They define the reports they say they need. The interviews run and the documentation produced is exactly what your managers think I should be asking about, correct?”
CEO:
“Correct”
Peter:
“Have you ever head the saying that the definition of insanity is to do the same things over again expecting a different result? Your projects failed precisely because people asked those questions. Those sets of questions are not sufficient, or even necessary, to make a data warehouse project successful. We take a different approach because we developed it over a long time and we improved it and it works. We improve it each time over.
Fred here hired us because we are the best in the country. And the reason we are the best in the country is precisely because we do not do what anyone else does. We do things differently. And because we do things differently we can guarantee the success of your project if you do as we advise you to do. You will see this when we show you the prototype in 6 weeks time.
So. Will you take my word that the reason we are not doing what your senior staff expect, what they have done three times before, is because we know that does not work? Will you take my word that we are doing things differently because that is how we guarantee success of our clients projects?”
CEO:
“Very well put young man. I look forward to the prototype demonstration.”
He shook my hand and I went on my way. Fred was VERY impressed at the meeting. We laboured long and hard on the prototype.
When it came the day of the prototype I did the talking and one of our team ran the demonstration.
The result was stunning. I have to say the team had done a remarkable job in just the 8 weeks we had to do the requirements and build the prototype to ask for the money for the build. Every team member deserves the greatest of praise for their dedication, commitment, hard work and just plain brilliance at their jobs. It was a pleasure to present the prototype. It was one of the best I had ever been involved in building.
Those who had been “complaining” just 6 weeks earlier were amazed at what they saw presented. They were making comments like:
“It is like they are mind readers, this is what I need, but I never once said this is what I need. How the hell do these guys know what we need when we didn’t even tell them?”
Eventually, over all the excitement, the CEO raised his hand and hushed the very excited crowd.
CEO:
“I have to say, that is one of the most remarkable demonstrations I have ever seen in my life. You told me 6 weeks ago you have never worked in Cargo, is that true?”
Peter:
“Yes. I have never worked in cargo. I have not even worked in anything like it before. Not ever.”
CEO:
“The presentation you just gave demonstrates a level of understanding of our business that I would be impressed by if someone who had been here 10 years gave it. You understand our business better than many people who have been here for a decade. I have to ask. How did you do that?”
Peter:
“Sure. You will remember 6 weeks ago you invited me to your office to discuss the complaints of the people in this room that I was not doing what they expected.”
CEO:
“Yes, I remember.”
Peter:
“You will remember that I told you we do not ask the same questions as everyone else. You will remember I said that we have a unique technique and a unique way we go about our job and it is because of this we get different results. You will remember I told you that we are the best there is at this?”
CEO:
“Yes, I remember.”
Peter:
“Well? This is what that looks like. The way we approach our projects allows us to develop a very deep understanding of the drivers of our clients business very quickly which means we can then build the charts, graphs and reports we know you need to generate outstanding success over the next 2 years. In short, we can figure out what you need better than you can figure our what you need because we use a different approach.”
CEO:
“Ok. I have to admit. This is impressive. I think I speak for all my senior staff when I say what we have seen today as a prototype? This is what we need.”
Peter:
“Great. Now there is this issue of money for the project. I am sure Fred has some paperwork for you and your staff to sign off on to approve the budget. May I presume that is all ok with you? That we can approve the budget today?”
CEO:
“Yep. You have your money. Come back on Monday and get this thing to production as fast as you can.”
Now…there were plenty of bumps in the road ahead for that project. Not that we need to go in to them in this blog entry. But by the end of the next year the CEO declared the project was “the most successful project ever run in Qantas Cargo”.
I am not personally privy to the financial aspects of the results produced as, again, I had moved on. It is sufficient to say that the project team I left behind remained working there for about 5 years.
Our technique of “Envision Outstanding Success” two years into the future was integral to making Qantas Cargo such an outstanding success of a project. The business managers were absolutely signed up and committed to making those visions come true. All we really did was give them the graphs, charts, and reports, they would need to look at on a daily, weekly, monthly basis to track the key business drivers that would indicate they were on the right path, or, indeed, deviating from that path.
The team later built all sorts of applications to detect how to optimise various areas of the business processes and optimise use of space/weight allocations inside the aircraft. But it was the “Envision Outstanding Success” two years into the future technique that allowed the senior managers to clearly and accurately put a stake in the ground as to where they wanted their department to be in 2 years time that allowed us to build one of the best prototypes I have ever done.
Of course, in those days we had no “prepackaged” models and so we still had to run these Business Requirements interviews before we built the prototypes. We now delay the whole “Business Requirements” process until we have analysed the data that is available so we know what is possible and not possible as we interview. This has been a rather radical change in our procedures which has been enabled by SeETL.
In Summary
Too often “success” in a data warehousing project is linked to budget, features, functions, timelines, delivery dates.
This takes the focus off what should be the REAL measure of success. Profit contribution. More specifically, “can we get a financial payback for the data warehouse in the first 12 months of operation?”
By focusing in return on investment, by focusing on achieving what can be agreed as “outstanding success”, you look beyond the end of the data warehouse build and look into the realm of the exploitation projects that will be run out of the data warehouse after its implementation. That keeps the business people far more active and far more engaged than they might otherwise be. In short, it keeps their attention. It gets them ready to use the data warehouse for exploitation when it comes into production.
Where this connection to the exploitation projects is weak or lacking it is that much harder to actually make the business process changes or run the business projects that will actually make the money you need to make from the data warehouse to pay for it.
I used two old but still very relevant examples of how I have done this in the past to deliver massive value to the clients that were on the receiving end.
If you got this far? Thank you for reading and/or listening. I hope this was useful to you.
Best Regards
Peter
