Requirements for a Successful Data Warehouse Project

Data Warehousing – The Basics
October 26, 2016
Data Warehouse Design Methodologies
November 9, 2016
In order to have a successful data warehousing project you will need the following five challenges addressed:

#1. Executive Sponsorship

All projects need executive sponsorship. Without senior executive leadership, most corporate projects will likely fail. Data warehouse projects are no different, in fact, these projects are even more in need of executive sponsorship than most business initiatives. It is imperative that the organization’s corporate leaders infuse the enterprise with excitement concerning the success of a data warehousing project. These executives must also tie the success of the project to each and every one of their direct reports. This will ensure that all of the business units of the organization are linked to the success of the Data Warehousing project. Organizational commitment from the top down will eliminate data fiefdoms, those individuals/departments who refuse to share their data or information with the larger organization for fear of loss of power or influence. These fiefdoms often work subtly to sabotage the success of the warehouse project. They do this out of fear, afraid that sharing their information may cause them to lose stature or influence. After all, today’s society teaches us that “data is king and information is power.” Anything less than an all-in commitment by senior management will place the success of the data warehousing project in jeopardy.

 

#2. Well Defined Business Requirements

Strong, stable requirements are critical to the success of the data warehousing project. Requirements for data warehousing projects must be aligned with the performance measures defined in the organization’s strategic plan. No project, especially a data warehousing/business intelligence (DW/BI) project, should proceed without strong requirements that align to the corporate strategy. These requirements should clearly define what is being measured and how that measurement supports one or more Key Performance Indicators (KPIs) which measure the health and success of the organization. If the business requirements for a DW/BI project are “fluid” then there is a high probability that this project will not be successful or, even worse, provide the senior leaders with erroneous information causing poor strategic decisions. Remember the old adage, “Measure twice, cut once.” Take time to define the corporate KPIs and their corresponding measures before moving forward with the project.

 

#3. Robust Infrastructure

Now that you have executive sponsorship and well-defined business requirements it is time to architect and purchase the infrastructure for the project. This step is often taken for granted but a poorly architected infrastructure will cost the organization in both money and time for response. Over-architecting the infrastructure needs of the organization leaves the organization with underutilized hardware and network resources and unused disk space. Under architecting the infrastructure causes delay in the processing of data because CPU, disk and memory resources are scarce. It takes time and talent to right-size a Data Warehouse / Business Intelligence solution, especially one that will grow with your organization as your informational needs increase.

 

#4. Data Warehouse Design

This step can take place at the same time as the previous step as both of these impact the other step. At this point, you know the KPIs and corresponding measures, the amount of data you will need to process and its year-over-year growth rate, the source data location and the transfer methodology of this source data and, finally, what software and hardware you will be using to process and display this information. All of these inputs are critical to the design of the data warehouse. These essential ingredients influence the methodology a Data Architect will use to store the data for consumption by the end user. Having produced many data warehousing solutions for a variety of customers I can tell you that there is no single methodology which is better than another, just one which is more appropriate to answer the business questions accurately and efficiently, with the correct level of detail to meet the customer’s business requirements. The design can be broken into releases based on available data, subject area or some other user defined grouping so as to start the project and thereby quickly show the results of the efforts up to this point. These releases should occur every 60 to 120 days, optionally every 90 days. If you project a release to go longer than 120 days, then you should resize the release so as to ensure the deliverable is complete in under 120 days. The reason is the customer needs to see progress and experience the value of the data warehouse. Those projects which have release schedules longer thank 120 days have often been seen as unresponsive to the business needs and therefore considered a failure by the enterprise.

 

#5. Business Intelligence & Analytics Technology

Now that the data warehouse is designed, and data is flowing into the warehouse it is now time to leverage this data via business intelligence and analytics tools. These software packages are becoming easier and easier for non-technical individuals to use. Because of this ease-of-use by non-IT workers the real power of the data warehouse is coming to fruition and information, which used to take weeks, is available to the business user within a day and, sometimes, within an hour. This increased speed of information allows companies to be extremely agile and quickly address issues before they negatively impact the corporate bottom line or enhance successes ensuring corporate growth.

4 Comments

  1. Hi Jim!
    Have been studying all of your articles, moving forward. You said “It takes time and talent to right-size a Data Warehouse / Business Intelligence solution”
    How can we measure the sources needs in terms of Hardware?

    • Jim McHugh says:

      Muhammed, unfortunately, there are no magic formulas I can provide you to measure the hardware resource needs of your DW project. Some rules of thumb that I use is 1) Make sure you have adequate disk space purchased for your warehouse. I find the disk space is quickly used long before the initial three-year growth projection by the business. For disks, I purchase 2x of what the business projects. I purchase the disks yearly, as it is needed, unless the vendor gives me a great deal to make a bulk purchase early. 2) Memory – I try to get as much as I can. The more dimensions that I can pin in memory, the better my application will perform. 3) Virtualization – I virtualize everything so that I can easily upgrade servers as better ones come available. In the end, it is experience that will help you make these decisions as every DW project is unique. I hope this helps you as you move forward in your career.

  2. Bhanukumar says:

    Hi Jim,

    Your article is really good. Congratulations!!

    As a Business Analyst, I have been assigned the task to design the company’s Dataware house. Do you have the template which will be important to set clear and concise business requirements? Since I will be working on DW for the first time, I do not want to miss any requirement while business meetings/sessions.

    Your guidance would be highly appreciated! Thanks in advance