OPSO-SAP BW Out bound interface and SAP BW Reporting

-by Hema Subhash 23rd April 2012

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 1

Introduction to SAP BW Reporting and OPSOs role in it

A set of 12 SAP BW Report were developed for use by Group 1 countries of MY/SG, TH, ID, CN and VN. These reports were developed by SAP BW teams, based out of Istanbul, Turkey. As of now, the SAP BW Support team, based out of Pune oversees issues related to this track. Data for these reports is sourced from Opsolight+ Staging DB, post completion of the daily replication job. Data from a set of 22 tables (interchangeably called as Interfaces) from the OPSO Staging Layer is interfaced with SAP BW via SAP PI at 2 00 AM SGT everyday.

Report Name Budget Details Budget Report Commitment Phasing Report Category Brand By Objectives Customer Brand Performance Report Category Brand by Mechanics Category ScoreCard PromotionMatrix Budget Cube Promo & Budget Cube ROI Cube Promotion KPI Cube

Budget Reports

Technical names for the BW Reports ZOS_M001_Q006_70 ROI Cube ZOS_M001_Q004_70 Category Scorecard ZOS_M001_Q003_70 Performance Report ZOS_M001_Q001_70 Objective Report ZOS_M001_Q002_70 Mechanics Report ZOS_M002_Q001_70 budget report ZOS_M002_Q003_70 Budget Cube ZOS_M002_Q002_70 Budget Details ZOS_M003_Q001_70 Commitment Phasing ZOS_M004_Q001_70 Cube Budget& Promotion ZOS_M005_Q001_70 KPI & Evaluation Cube
2010 MindTree Limited Slide 2

CONFIDENTIAL: For limited circulation only

Cube

Promotion Reports

What role OPSO plays in SAP BW Reporting

The only role we have to play in SAP BW Reporting:

Sending correct delta data to SAP BW via SAP PI.

Data that is being sent to SAP BW, should be in agreement with the data in the transaction databases. Data for 22 tables listed below are interfaced from OPSOLight_Staging_group1 DB:
KPIMaster BudgetHolder BudgetMaster PhasedBudgetMaster AccountBudgetHolderMapping Bundle Promotion PromotionKPI PromotionMechanicsDetails PromotionAccountAllocation PromotionInvestmentDetails PromotionInvestmentDetailProducts ROIBaselineStaging PromotionPlanningSalesTarget PromotionPlanningTimeLine PromotionQualitativeEvaluationResults ChildPromotion ChildPromotionAccountAllocation PostEvaluationScanData PromotionBudgetMapping PhasedPromotionBudgetMapping PromotionActuals

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 3

Delta Data logic implemented by Opso

1. OPSO would be generating the delta record set with the status "I" & "D" only.

2. Record with status "I" means, these records are either updated or newly created.

3. OPSO Team would be providing the list of columns for each table to identify a unique record. 4. SAP BW has to identify the unique record based on this key combination and if the record already exists delete and re insert the whole record else only insert the record.
5. SAP BW has to delete records based on the key combinations to identify unique record and with status "D"

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 4

Delta Data logic implemented by Opso

ROIBaselineStaging is taken as an example to explain the same.
PromotionID, AccountID, ProductID - Combination of these columns would help in identifying a unique record. (Check Unique Key Identifiers for more details)

1. Scenario - Initial load ID

307931 307933 307934 307935 307936 307937 307938 307939 307940 307941 307942

PromotionID BrandProductID
10041156 10041156 10041156 10041156 10041156 10041156 10041156 10041156 10041156 10041156 10041156 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20

AccountID ProductID
E5786:20 E5796:20 E5798:20 E5800:20 E5801:20 E5802:20 E5803:20 E5811:20 E5812:20 E5814:20 E5996:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20 441638036A50321:20

AdjustedBaseLineDuring
755.5000000 1500.0000000 1149.7500000 966.0000000 956.4000000 533.6500000 696.6000000 1173.1500000 947.6000000 959.4500000 553.7500000

Status I I I There is an increase/decrease in I value. OPSO would be interfacing the latest values. I I SAP BW has to identify whether it is an update or insert based by key I combination(in this case Promotion, I AccountID, ProductID), to identify I unique record. I In this example, data at the BW end I would be updated or overwirtten
with the new data sent from OPSO.

2. Scenario - Update on some records

ID
307931 307933 307934

PromotionID BrandProductID
10041156 10041156 10041156 441638036A50321:20 441638036A50321:20 441638036A50321:20

AccountID ProductID
E5786:20 E5796:20 E5798:20 441638036A50321:20 441638036A50321:20 441638036A50321:20

AdjustedBaseLineDuring
760.5000000 1600.0000000 1000.7500000

Status I I I
Slide 5

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Delta Data logic implemented by Opso

3. Scenario - Delete of some records

ID
307940 307941

PromotionID BrandProductID
10041156 10041156 441638036A50321:20 441638036A50321:20

AccountID ProductID
E5812:20 E5814:20 441638036A50321:20 441638036A50321:20

AdjustedBaseLineDuring
947.6000000 959.4500000

Status D D

Based on the key combination (to identify unique record), in this case PromotionID, AccoutID, ProductID the record has to be deleted.

Application Table

Application table deleted

Job: OpsoSAPBWDelta_Job

Description : Calls the SSIS package that identifies delta data and inserts the same into BWStaging tables.
Frequency : Daily, 2 00 AM SGT.
BW staging table

Mode of Data movement : SSIS packages

Data Moved : Incremental data governed by Lastextracteddate in the Extractcontrolmaster.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 6

Extractcontrolmaster

As the name suggests, the extractcontrolmaster, controls the extract that will be pushed during that instance of job execution.
The lastextracteddate is updated by the package to indicate the date till which delta data has been pushed to the BWstaging tables (the start of current date). Key columns: Interfacename: Lists names of the tables from which data will be interfaced Delta Push CCID: List of country company ids for which delta data will be interfaced. Lastextracteddate: Specifies the date from which the delta data will be generated. Isactive: If Y then delta for that interface will be pushed into the corresponding Bwstaging table, else delta wont be interfaced.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 7

Reading data from SAPBWControltable

The sample records from the SAPBWControltable, indicates the following: 1. In the Countrycompanymasterbwstaging table, there are 34 records with the GUID DT20110523-INTF1-EXT1-FC1, that need to be processed by SAP PI. 2. This delta data corresponds to delta starting from 1/1/2007 till 5/23/2011 3. The extract was created on 5/23/2011

InterfaceID InterfaceName CountryCompanyMaster DT20110523-INTF1-EXT1FC1

GUINumber
RecordsCount Status DeltaStartDate DeltaEndDate

34 N 1/1/2007 5/23/2011

ExtractStartTime
ExtractEndTime CreatedDate

5/23/2011
5/23/2011 5/23/2011

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 8

On-going support required

1/ Ensuring PI picks up data from OPSO Staging At least once a week the query given below should be executed in Opsolightplus_staging_Group1
select interfacename, GUIs [Date], status, count(*) [Number of GUIs not picked by SAP PI] from ( select interfacename,left(guinumber,10) GUIs, status from sapbwcontroltable where status in ('N','I') and interfacename in ('PromotionPlanningTimeLine','BudgetHolder','PromotionMechanicsDetails','AccountBudgetHolderMapping', 'BudgetMaster', 'ChildPromotionAccountAllocation', 'ChildPromotion', 'PromotionActuals', 'PromotionAccountAllocation', 'PromotionBudgetMapping', 'ROIBaseLineStaging', 'BudgetHolderAttribute', 'Bundle', 'KPIMaster', 'PhasedBudgetMaster', 'PhasedPromotionBudgetMapping', 'PostEvaluationScanData', 'Promotion', 'PromotionMechanicsDetail', 'PromotionTimeLine', 'PromotionInvestmentDetailProducts', 'PromotionInvestmentDetails', 'PromotionPlanningSalesTarget', 'PromotionKPI', 'PromotionQualitativeEvaluationResults') )a group by interfacename, GUIs, status order by interfacename , GUIs, status

During business hours this query shouldnt return any values, if it does please let the PI team know and perform a re-push of data if the status I and N records are loads before the last completed load.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 9

On-going support required

2/ Ensuring the OPSO ETL doesnt generate status F and S records and monitoring the OPSO_SAPBW_DeltaPush job on SGPSAPPW064 server. Check the Delta data job for any failures at least once a week. At least once a week the query given below should be executed in Opsolightplus_staging_Group1
select interfacename, GUIs [Date], status, count(*) [Number of GUIs not picked by SAP PI] from ( select interfacename,left(guinumber,10) GUIs, status from sapbwcontroltable where status in (F',S') and interfacename in ('PromotionPlanningTimeLine','BudgetHolder','PromotionMechanicsDetails','AccountBudgetHolderMapping', 'BudgetMaster', 'ChildPromotionAccountAllocation', 'ChildPromotion', 'PromotionActuals', 'PromotionAccountAllocation', 'PromotionBudgetMapping', 'ROIBaseLineStaging', 'BudgetHolderAttribute', 'Bundle', 'KPIMaster', 'PhasedBudgetMaster', 'PhasedPromotionBudgetMapping', 'PostEvaluationScanData', 'Promotion', 'PromotionMechanicsDetail', 'PromotionTimeLine', 'PromotionInvestmentDetailProducts', 'PromotionInvestmentDetails', 'PromotionPlanningSalesTarget', 'PromotionKPI', 'PromotionQualitativeEvaluationResults') )a group by interfacename, GUIs, status order by interfacename , GUIs, status

If this query returns any values then, re-push of data starting from the delta date on which F and S records are found.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 10

On-going support required

4/ Apart from ensuring delta data loads have been successful, the data in PostEvaluationdata, Evaluationdata and ROIBS and PA have to be synchronized as per Primary Sales processing process followed in OPSO. The agreed process as of now, is that post PS processing (45 days after end of a month), the Postevaluationdata should be updated with the PS and Actual values received. If this process is followed diligently, then incidents due to data mismatches between OPSO and BW reports can be minimized.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 12

Commonly encountered issues and their fixes

1/ Data mismatches between OPSO and SAP BW Reports

Out of the 12 SAP BW reports, the ROI Cube report and the ROI Cube based reports are compared with the OPSO Promotion measures reports. If mismatches occur between these BW reports and the OPSO Promotion measures report then the following checks need to be done in the order specified:

Step 1. As the first level of analysis, the SAP BW Support team should confirm that the mismatch is not being caused due to an issue at the BW end. (Such as, Process chains failing at the BW end) Step 2. As an outcome of Step 1, if the BW support team, confirms that the issue isnt being caused due to unexpected behavior at their end, then at the OPSO end, firstly check if the delta data for the promotion(s) with the mismatch has been flowing correctly. In other words, the delta pushed for the promotion should match the data in the transaction tables for this promotion. * For mismatches between ROI Cube based reports and the Promotion measures report, check the ROIBAselinestaging table. * For mismatches in other reports, consult the BW team for the source tables that feed into the respective BW report.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 13

Commonly encountered issues and their fixes

Step 3. If the mismatch is not being caused due to delta data issues, then check if the data is synchronized between the Postevaluationdata and the ROIBaselinestaging tables.
This is done by comparing the KPIs in the two tables.

For e.g. ActualGSV in Postevaluationdata is calculated as the sum(GSVBefore+GSVDuring+GSVPost) from the ROIBS table in the SP: usp_getinsertpostevaluationdata. Therefore run the following queries for the promotionid in error: Opsolightplus_Group1: Select sum(GSVBefore+GSVDuring+GSVPost) from ROIBAselinestaging Where promotionid= <<>>
Select GSVActaul from Postevaluationdata Where promotionid=<<>>

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 14

Commonly encountered issues and their fixes

Step3 explained in detail.. 1. When mismatches are reported for a certain BW measure, then ask for the OPSO measure that it is being compared with. 2. Validate if it is logically correct to compare that BW measure with the OPSO measure reported. 3. If yes, then understand the way the OPSO measure is derived at by looking at script for usp_promotionmeasures 4. Then if the measure is derived from the Postevaluationdata or evaluationdata tables then, understand how these measures are arrived at from the ROIBS and PA tables, by looking at the usp_getinsertpostevaluationdata and usp_Getinsertevalutiondata SPs. 5. Then once the entire formula for the mismatching measure is understood, then try to see if the measure in the Postevaluationdata is as expected.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 15

Commonly encountered issues and their fixes

2/ Non-ROI promotions in SAP BW ROI Cube reports As per the BW report design, Non-ROI Promotions are also present in the ROI Cube report and other related reports. Mismatches are expected for these promotions, since we dont perform Postevaluation processing for these promotions.

3/ Modified Date in transaction tables not getting updated when data in it is updated. This could happen due to two reasons: 1. The SP that updates the modifieddate in the transaction tables is not updating the modified date as expected. 2. Due to changes in the data from the back-end without diligent changes being made to the modified date. As a solution to this, the whenever data is being modified in the 22 transaction tables in concern here, the modified date should also be updated so that the right data flows to the downstream systems.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 16

Commonly encountered issues and their fixes

4/ Incomplete processing by SAP PI team. If there are any records that remain to be processed in either status N or I after the scheduled PI loads, then we would need to do a re-push starting from the date on which GUIs in these statuses were identified.

5/ Null or 0 conversion factors in ROIBaselinestaging, causing divide by zero error at the BW end due to which data will not be uploaded to the BW targets, thus causing mismatches.

As a workaround, these NULL or 0 conversion factors in the ROIBaselinestaging table are updated to 1. And the modifieddate of all the updated records are updated to getdate() so that the updated information flows to BW during the successive delta load. If conversion factors have not been shared by SAP ECC for SKU with this issue then the concern needs to be raised with the SAP ECC team.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 17

Commonly encountered issues and their fixes

6/ Data across multiple days being processed at once by the PI team, causing data mismatches. For e.g.
The payload that was processed on 23 Oct 2011, 00:50:00 IST contains data from the 19 th of October till the 22nd of October. In this case there will be duplicates in the payload since the same budget ids could have undergone modifications between the dates of 19th till 22nd October. In order to avoid this from recurring the data that is pushed from Opso (on a daily basis) has to be loaded daily so that the duplicates are not present in a single payload. The screen shot below shows that the single payload has data for the 19th of October

22th of October

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 18

Commonly encountered issues and their fixes

7/ Previously, data mismatches have occurred due to mismatches in key combinations between OPSO and BW, due to which data is not updated or deleted as expected. The solution for this is a design change at the BW end, to ensure that the key combinations for the interfaces are as expected. 8/ Data being missed out in the OPSO Delta loads, due to incorrect join conditions in the SP: usp_SAPBWReportingGenerateDeltaExtractQueries (in Opsolightplus_staging_Group1 Db). This query is responsible for creating the delta data for all the interfaces. 9/ Deleted records not flowing into the Applicationdeleted tables in the transaction DB. As a fix for this issue a job Job_RP_EnableServiceBrokerDeleteQueue was created and scheduled to run at 11 00 PM SGT before the start of the replication job. This job ensures that the Service queues that were disabled due to poison messages are reenabled and the deleted messages that are buffered are sent to the application deleted tables before they are replicated into the staging layer.

CONFIDENTIAL: For limited circulation only

2010 MindTree Limited

Slide 19