Project Management Estimation (DWH)

Estimation is a project management activity that is used to predict the amount of work and money required to complete the project. We perform two types of estimations in DWH namely Top-Down estimates and Bottom-Up estimates for the effort required to complete the project. The Top-Down estimates are used during the initial phase of the project based on the scope defined in the project definition phase. The Bottom-Up estimates are used after the WBS and the activity list have been created. Top-Down Estimates The top-down estimates are based on organization's effort and cost for past similar projects already implemented. The similarity of projects can be based many factors. Some of them are as follows Domain (Retail/Telecommunication/Banking etc). Types and number of source systems. Number of end-users. Processing complexity (obtained from the objective and scope definition)o Do we require integration of disparate sources. o Are sources geographically distributed. o Is the scope suggesting EDW or a Data Mart. o History data required. o Functional complexity. o Types of business contexts involved (Dimensions). o Types of metrics required (facts). o Levels of hierarchies required. o Level of user and data security required. Number of user reports (canned and ad hoc). Type of DWH project o Building a DWH from the scratch. o Re-Architecting an existing system. o Maintenance Project.

Please note that all of the above data may not be available in the scope definition. In that case, you may have to go back and review the above factors with the end user before you can perform the top-down estimate. Once we know that the new project is similar to one or more projects, we take the project which has most common factors to the new project, and get an idea of the total effort in person days and the cost of development involved. We also get the productivity of a Simple, a Medium and a Complex data flow (Job/Mapping) for such a project. Simple data flow (job/mapping) could be defined as having 1 to 3 sources, 1 to 5 transformations, 1 to 5 lookups, 1 to 3 targets. Medium data flow (job/mapping) could be defined as having 4 to 5 sources, 6 to 9 transformations, 6 to 8 lookups, 4 to 8 targets.

Complex data flow (job/mapping) could be defined as having more than 6 sources, more than 10 transformations, more than 9 lookups, more than 9 targets. We also get the project phase split of effort based on the construction effort. For example, Design = 31% of Construction. Testing = 42 % of Construction. Analysis = 18% of Construction. Implementation = 9 % of Construction. Based on the above information, we predict the number of process flows, data flows and interfaces. Then we estimate the total effort for developing and unit testing the data flows for each process flows.

Taking an example of an EDW for Retail operations, we get the above mentioned project phase split for effort (Design=31% of Construction, Testing = 42% of Construction etc) and the productivity of 1.24 Person Days for simple data flow (job/mapping), 2.0 Person Days for a medium data flow (job/mapping) and 3.4 Person Days for a complex data flow (job/mapping) from a similar retail EDW done in the past.

We have identified 3 Process flows for the new project, and have predicted the number of data flows (jobs/mappings). Based on the productivity and the predicted number of data flows (jobs/mappings) we get the construction effort. Based on the phase wise split, we get the effort for all the phase.

Bottom-Up Estimate The bottom-up estimates are also based on organization's effort and cost for past similar projects already implemented. In the bottom-up estimates, we estimate the tasks at a very detail level, unlike at process flow level (in the top-down estimate example above). We create the activity list from the WBS, and estimate individual activities based on productivity information from the organizations history data for DWH projects.

In the above example, we take the tasks from the activity list and assign the effort based on productivity from organizations past projects and our expert judgment.