You are on page 1of 102

1

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Oracle Database 12c Optimizer Overview


Maria Colgan Master Product Manager

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Agenda
Adaptive Query Optimization Optimizer Statistics enhancements SQL Plan Management enhancements

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Query Optimization


Overview Adaptive Plans
Join methods Parallel distribution methods

Adaptive Query Optimization

Adaptive Statistics
Discovered at compile time Discovered at run time

Adaptive Plans

Adaptive Statistics

Join Methods

Parallel distribution Methods

At compile time

At run time

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Adapt join methods
Query: Find all of the products with a unit price of 15 that we have sold more that 1 of Two possible join methods for this query Nested Loops Hash Join

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Adapt join methods
Alternative sub-plans are pre-computed
NESTED LOOPS HASH JOIN

Rows coming out of order_items table are buffered up to a point If row count is less than the threshold use nested Loops otherwise switch to hash join

Sub-plans stored in the cursor

Stats collect inserted before join


Rows buffered until final decision is made

Statistics Collector

Table scan Order _items

Index Scan Prod_info_ind

Table scan Prod_info

Default Plan is a nested loops join


6 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Adapt join methods
Number of rows seen in statistics collector exceeds threshold Plan switches to hash join
Statistics collect disabled Plan resolved on first execution & remains the same for subsequent executions
Statistics Collector

Statistics collector disabled after decision is made and becomes a pass through operation

NESTED LOOPS

HASH JOIN

Table scan Order _items

Index Scan Prod_info_ind

Table scan Prod_info

Final Plan is a hash join


7 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Displaying the default plan
Explain plan command always shows default plan Example shows a nested loops join as default plan No statistics collector shown in plan

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Displaying the final plan
After the statement has completed use
DBMS_XPLAN.DISPLAY_CURSOR

to see the final plan selected Example shows that hash join picked at execution time Again the statistics collector is not visible in the plan

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Displaying the full adaptive plan

Full adaptive plan displayed when format parameter +adaptive is set Example shows both the nested loops and hash join in the plan

10

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Displaying plan with +adaptive & +report formats

Additional information displayed on why operations are inactive can be seen with format parameter +report

11

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Indicator in V$SQL
New column in V$SQL
IS_RESOLVED_ADAPTIVE_PLAN

Indicates statement had an adaptive plan which was resolved on first execution Resolved plan is used for subsequent executions Statistics collectors and buffering is disabled
12 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Reporting mode
Adaptive plans are enabled by default Can be put in reporting mode
OPTIMIZER_ADAPTIVE_REPORTING_ONLY

Reporting mode shows what would have happened during execution in the plan

13

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Adaptive join methods
Join method decision deferred until runtime
Default plan is computed using available statistics Alternate sub-plans are pre-computed and stored in the cursor Statistic collectors are inserted at key points in the plan

Final decision is based on statistics collected during execution


Default plan and sub-plans have a valid range for statistics collected If statistics prove to be out of range, the sub-plans will be swapped Requires buffering near the swap point to avoid returning rows to user

Possible sub-plans are nested loop joins or hash joins and vice versa
14 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Adaptive parallel data distribution
User connects to the database Shadow process is spawned When user issues a parallel SQL statement the background process becomes the Query Coordinator QC gets parallel servers from global pool and distributes the work to them Parallel servers individual sessions that perform work in parallel

User
Parallel servers communicate among themselves & the QC using messages that are passed via memory buffers in the shared pool How the data is distributed among the processes is based on the operation being done & number of rows expected

15

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Distribution Methods


Hybrid-HASH Join
Cardinality based distribution skew common scenario
Crucial for parallel join of very small data sets with very large data sets

Distribution method decision based on expected number of rows New adaptive distribution method HYBRID-HASH
Statistic collectors inserted in front of PX process on the left hand side of the join
If actual number of rows less than threshold, switch from HASH to Broadcast Threshold number of total rows < 2x DOP

Enabled by default

16

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Execution Plans


Adaptive parallel data distribution
Hybrid hash join between EMP and DEPT Distribution method based on runtime stats Statistics collector inserted in front of PX processes scanning DEPT
Query coordinator # rows returned less than threshold so rows distributed via Broadcast P3 P4

P1

P2

Statistics Collector threshold 2X DOP

P5 P6

DEPT
P7 P8

EMP

17

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Distribution Methods


Hybrid hash join between EMP and DEPT Distribution method based on runtime stats Statistics collector inserted in front of PX processes scanning DEPT

18

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Distribution Methods


Broadcast/Round Robin

Hybrid hash join between EMP and DEPT Distribution method based on runtime stats If DEPT uses Broadcast - EMP uses Round-Robin DOP used is 6 Number of rows returned is 4 Broadcast used because 24 rows distributed (6 X 4)

19

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Distribution Methods


Broadcast/Round Robin

Hybrid hash join between EMP and DEPT Distribution method based on runtime stats If DEPT uses Broadcast - EMP uses Round-Robin If DEPT used HASH - EMP uses HASH DOP used is 2 Hash used because only 4 rows distributed
20 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Hash/Hash

Adaptive Query Optimization


Overview
Adaptive Query Optimization Adaptive Plans
Join methods Parallel distribution methods

Adaptive Statistics
Discovered at compile time Discovered at run time

Adaptive Plans

Adaptive Statistics

Join Methods

Parallel distribution Methods

At compile time

At run time

21

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Dynamic Statistics
During compilation optimizer decides if statistics are sufficient to

generate a good plan or not Dynamic statistics are used to compensate for missing, stale, or incomplete statistics They can be used for table scans, index access, joins and group bys One type of dynamic statistics is dynamic sampling

22

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Dynamic Statistics
Dynamic Sampling
Dynamic sampling has a new level 11(AUTO) Decision to use dynamic sampling depends on the complexity of predicate, existing statistics and total execution time Dynamic statistics shared among queries

23

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Dynamic Statistics
Select * From product_information Where list_price-min_price=29 And category_id not in (11,22) And prod_name like Smart%;

SGA
SHARED DYNAMIC STATISTICS
Table cardinality estimate for prod_info

Optimizer determines plan using dynamic statistics

SHARED_POOL
Cursor 0: Select * from product_information

SQL statement is submitted

Optimizer checks for existing statistics in data dictionary

Dynamic sampling occurs on small number of blocks from table Resulting dynamic statistics are stored in cache

DATA DICTIONARY

Statistics found but need to be augmented due to complex predicates

24

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Dynamic Statistics
Select supplier_id, prod_name From product_information Where list_price-min_price=29 And category_id not in (11,22) And prod_name like Smart%;

SGA
SHARED DYNAMIC STATISTICS
Table cardinality estimate for prod_info

10 Optimizer determines
plan using dynamic statistics

SHARED_POOL
Cursor 0: Select supplier_id, prod_name

Different SQL statement is submitted with same predicates

Necessary Dynamic statistics found in shared cache

Optimizer checks for existing statistics in data dictionary

DATA DICTIONARY

Statistics found but need to be augmented due to complex predicates

25

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Statistics
Re-optimization
During execution optimizer estimates are compared to execution statistics

If statistics vary significantly then a new plan will be chosen for

subsequent executions based on execution statistics


Re-optimization uses statistics gathered from previous executions

First introduced as Cardinality Feedback in 11.2

26

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Statistics
Cardinality Feedback pre 12c
Statistics gathered about data volume and data type seen during execution

If execution statistics vary significantly statement will be hard parsed on

the next execution using the execution statistics instead Statements are only monitored once if they dont show significant differences initially they wont change in the future Only individual table cardinalities and group by estimates examined Information is stored in the cursor only and is lost if cursor ages out

27

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Statistics
Cardinality Feedback pre 12c

Initial execution of a query is monitored


Example shows initial plan is hash join between sales and customers Cardinality estimates 8X off
Initial Cardinality estimates are more than 8X off

28

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Statistics
Cardinality Feedback pre 12c
Execution Plan after the second execution

Execution statistics used to reparse the statement on the second execution New plan shows correct cardinality estimates and a new join order Information learnt is stored in the cursor only and is lost if cursor ages out

Estimates are now correct

29

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Statistics
Re-optimization in 12c
Join statistics are also monitored

Works with adaptive cursor sharing for statement with binds


New Column in V$SQL IS_REOPTIMIZABLE Information found at execution time is persisted as SQL Plan Directives

30

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Statistics
Re-optimization indicator in V$SQL

New column in V$SQL


IS_REOPTIMIZABLE

Indicates that the statement will be re-parsed on the next execution

31

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Statistics
Adaptive Plans & Re-optimization working together
Adapt join method example showed the join change from nested loops to hash join But cursor is also marked IS_REOPTIMIZABLE

Why?
Re-optimization will occur on second execution because cardinality estimates off
32 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Initial Cardinality estimates are off

Adaptive Statistics
Adaptive Plans & Re-optimization working together
On second execution statement is re-parsed using execution statistics New child cursor created with new plan New plan changes the join order

33

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Statistics
SQL Plan Directives
Directives are additional information used during optimization to

generate a better plan


For example, when table T1 is joined to T2 use dynamic statistics to get

accurate cardinality estimate


Directives are collected on query expressions not at a statement level
Allows for directives to be used for multiple statements

Persisted on disk in the SYSAUX tablespace Directives will be automatically maintained

Managed using the new package DBMS_SPD

34

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Directives


Select * FROM customers where state =CA AND country=USA;

SGA
DIRECTIVE CACHE
Directive: Use DS for customers table when column city, country are used

2
SQL statement is submitted

Optimizer determines plan

SHARED_POOL
Cursor 0: Select * from customers .. IS_REOPTIMIZABLE = Y

During execution cardinality estimate discovered to be wrong and directive created

Sysaux Tablespace

35

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Directives


Select * FROM customers where state=CA AND country=USA;

SGA
DIRECTIVE CACHE
Directive: Use DS for customer table when column city, country are used

Optimizer determines new plan using execution statistics

SHARED_POOL
Cursor 0: Select * from customers .. IS_REOPTIMIZABLE = Y Cursor 1: Select * from customers .. IS_REOPTIMIZABLE = N

Same SQL statement is submitted again

This execution and subsequent execution use child cursor 1

Sysaux Tablespace
Extension Binds Stats Sqlid

SQL Directives SQL Directives

Directives periodically persisted to disk in the sysaux tablespace

36

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Directives


Select email FROM customers where state=MA AND country=USA;

SGA
DIRECTIVE CACHE

Optimizer determines plan with help of directive

SHARED_POOL
Cursor 0: Select email from customers

New SQL statement is submitted on customers table

Optimizer adds column group creation for state & country columns to next statistics gather on customer table

10

Optimizer checks for directives on customers table and finds one on the columns state and country

Sysaux Tablespace
Extension Binds Stats Sqlid

SQL Directives

37

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Directives


In use
Number of SQL Plan Directives used for a statement is displayed in the notes section of the execution plan In this case one SQL Plan Directive was used

38

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Directives


Monitoring
Auto flushed every 15 minutes

39

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Directives


Monitoring
Auto flushed every 15 minutes

40

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Directives


Gather statistics to allow automatic creation of missing statistics
SQL Plan Directive state was MISSING_STATS Indicates Oracle can automatically gather statistics to improve the original mis-estimation In this case a column group on STATE and COUNTRY will address the problem
41 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Directives


Monitoring

42

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Directives


Final plan no longer needs the directive
Correct statistics in place Optimizer uses statistics instead of directives Accurate cardinality estimate seen in the plan

Statement not marked for re-optimization

43

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive Query Optimization


Overview Adaptive Plans
Join methods Parallel distribution methods

Adaptive Query Optimization

Adaptive Statistics
Discovered at compile time Discovered at run time

Adaptive Plans

Adaptive Statistics

Join Methods

Parallel distribution Methods

At compile time

At run time

44

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Summary
Optimizer begins to learn from its mistakes
Adaptive Plans Adapt join methods Adapt parallel distribution methods Adaptive Statistics Dynamic statistics Re-optimization SQL Plan Directives

And lots of other great features!


45 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Agenda
Adaptive Query Optimization Optimizer Statistics enhancements SQL Plan Management enhancements

46

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Statistics Enhancements

New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics

Concurrent statistics gathering


Automatic detection of column groups

Statistic gathering reporting


47 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Histograms
Histograms tell Optimizer about the data distribution in a Column

Creation controlled by METHOD_OPT parameter


Default create histogram on any column that has been used in the

WHERE clause or GROUP BY of a statement AND has a data skew Relies on column usage information gathered at compilation time and stored in SYS.COL_USAGE$ Four types of histograms
Frequency Top-Frequency
48 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Height balanced

Hybrid
Oracle Confidential

Histograms
Frequency Histograms (FREQUENCY)
A frequency histogram is only created if the number of distinct values in

a column (NDV) is less than 254 values

Frequency histogram
49 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Oracle Confidential

Histograms
Top Frequency (TOP-FREQUENCY)
Traditionally a frequency histogram is only created if NDV < 254

But if a small number of values occupies most of the rows (>99% rows)
Creating a frequency histograms on that small set of values is very

useful even though NDV is greater than 254


Ignores the unpopular values to create a better quality histogram for

popular values Built using the same technique used for frequency histograms Only created with AUTO_SAMPLE_SIZE

50

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Top Frequency Histogram Example


Table PRODUCT_SALES contains information on Christmas ornament sales It has 1.78 million rows There are 620 distinct TIME_IDs But 99.9% of the rows have less than 254 distinct TIME_IDs
TIME_ID column perfect candidate for top-frequency histogram

51

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Histograms
Height Balanced Histograms (HEIGHT BALANCED)
A height balanced histogram is created if the number of distinct values

in a column (NDV) is greater than 254 values

Height balanced histogram


52 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Histograms
Hybrid Histograms (HYBRID)
A height balanced histogram is created if the number of distinct values

in a column (NDV) is greater than 254 values

Hybrid histogram
53 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Histograms
Hybrid Histograms (HYBRID)
Similar to height balanced histogram as created if the NDV >254

Store the actual frequencies of bucket endpoints in histograms


No values are allowed to spill over multiple buckets More endpoint values can be squeezed in a histogram

Achieves the same effect as increasing the # of buckets


Only created with AUTO_SAMPLE_SIZE

54

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Height-balanced versus Hybrid Histogram

Oracle Database 11g


55 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Oracle Database 12c

Height-balanced Histogram Example


Step 1: SELECT row_num, time_id FROM sales ORDER BY 2;
ROWNUM
1 2 3 4 5 6 7 8 9 10 :
56

TIME_ID
02-JAN-98 03-JAN-98 05-JAN-98 05-JAN-98 06-JAN-98 09-JAN-98 09-JAN-98 09-JAN-98 10-JAN-98 10-JAN-98 :

With a traditional height based histogram an even number of rows goes in each bucket Multiple buckets can have the same endpoint There are 960 rows in the sales table Automatically created histograms have 254 buckets That is 3.77 rows per bucket

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Height-balanced Histogram Example


Step 2: Assign roughly an equal number of rows per bucket
ROWNUM
1 2 3 4 5 6 7 8 9 10 :
57

TIME_ID
02-JAN-98 03-JAN-98 05-JAN-98 05-JAN-98 06-JAN-98 09-JAN-98 09-JAN-98 09-JAN-98 10-JAN-98 10-JAN-98 :

BUCKET INFORMATION
Bucket 0 has end point 02-JAN-98

Step 3: Bucket 0 added for min value

Bucket 1 has end point 05-JAN-98

Bucket 2 has end point 10-JAN-98

Step 4: Buckets with the same endpoint are compressed

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Height-balance Histogram Example

Multiple buckets have the same endpoint Max number of popular values that could have been recorded 96

Missing bucket numbers are buckets that have been compressed

58

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Height-balanced versus Hybrid Histogram

Oracle Database 11g


59 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Oracle Database 12c

Hybrid Histogram Example


Step 1: SELECT row_num, time_id FROM sales ORDER BY 2;
ROWNUM
1 2 3 4 5 6 7 8 9 10 :
60

TIME_ID
02-JAN-98 03-JAN-98 05-JAN-98 05-JAN-98 06-JAN-98 09-JAN-98 09-JAN-98 09-JAN-98 10-JAN-98 10-JAN-98 :

As with traditional height based histogram want an even number of rows in each bucket But no two buckets have the same endpoint There are 960 rows in the sales table Automatically created histograms have 254 buckets That is 3.77 rows per bucket

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Hybrid Histogram Example


Step 2: Assign roughly an equal number of rows per bucket
ROWNUM
1 2 3 4 5 6 7 8 9 10 :
61

TIME_ID
02-JAN-98 03-JAN-98 05-JAN-98 05-JAN-98 06-JAN-98 09-JAN-98 09-JAN-98 09-JAN-98 10-JAN-98 10-JAN-98 :

BUCKET INFORMATION
Bucket 1 has end point 02-JAN-98

FREQUENCY OF ENDPOINT
1

Bucket 5 has end point 06-JAN-98

Bucket 10 has end point 10-JAN-98

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Hybrid Histogram Example


No two buckets have the same endpoint

Frequency of endpoint values recorded in new column called endpoint repeat count
Potential to recorded 254 popular values :

Frequency at which an endpoint values occurs in the sample is recorded

You can manual increase the number of buckets to 2048


62 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Statistics Enhancements

New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics

Concurrent statistics gathering


Automatic detection of column groups

Statistic gathering reporting


63 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Online Statistics Gathering


Statistics gathered as part of the direct path load operations
Create Table As Select or Insert As Select commands

Statistics available directly after load No additional table scan required to gather statistics

All internal maintenance operations that use CTAS benefit from this
Note only occurs on IAS if table is empty

64

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Online Statistics Gathering Example


New table SALES2 is created using a CTAS command Both table and column level statistics are available immediately after table has been created
Statistics immediately available

65

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Online Statistics Gathering Example


Histogram & index statistics not gathered To gather these statistics without re-gathering basic statistics set option parameter of GATHER_TABLE_STATS to GATHER AUTO

66

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Online Statistics Gathering Example

Histogram & index statistics gathered using new GATHER AUTO option without regathering base column statistics

67

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Statistics Enhancements

New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics

Concurrent statistics gathering


Automatic detection of column groups

Statistic gathering reporting


68 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Session private statistics for GTT


Overview
Traditionally statistics gathered on GTT were shared by all sessions

Share statistics are not always optimal


Now each session can have its own version of statistics for GTT Controlled by new preference GLOBAL_TEMP_TABLE_STATS

Default value is SESSION (non shared)


To force sharing (as in 11g) set table preference to SHARED

69

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Session private statistics for GTT


Statistics gathered on a GTT are no longer shared by all sessions
By default statistics on GGT are session-private

To restore shared statistics change the table preference


GLOBAL_TEMP_TABLE_STATS to SHARED

70

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Statistics Enhancements

New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics

Concurrent statistics gathering


Automatic detection of column groups

Statistic gathering reporting


71 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Enhanced Incremental Statistics


Incremental statistics allows global level statistics to accurately

generated from partition level statistics NDV statistics can now be accurately aggregated by the introduction of the synopsis The synopses are stored in the Sysaux tablespace In 12c reduced the space required to store synopses on disk

72

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Enhanced Incremental Statistics


Sales Table

1. Partition level stats are


gathered & synopsis created 2. Global stats generated by aggregating partition level statistics and synopsis

NDV statistics can now be accurately aggregated The synopses are stored in the Sysaux tablespace 12c reduces the space required to store synopses on disk

May 18th 2012

May 19th 2012

May 20th 2012

May 21st 2012

May 22nd 2012

May 23rd 2012

Sysaux Tablespace
73 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Enhanced Incremental Statistics for Partition Exchange


Sales Table

DBA
1. Create external table for flat files

May 18th 2012

6. Global stats generated


by aggregating partition level statistics for existing partition with stats on new partition

May 19th 2012

May 20th 2012

2. Use CTAS command to create non-partitioned


table TMP_SALES

May 21st 2012

3. Set INCREMENTAL to true &


TMP_SALES

May 22nd 2012

INCREMENTAL_LEVEL

to TABLE
May 23rd 2012

4. Gather Statistics
5. Alter table Sales exchange partition May_24_2012 with table tmp_sales
74 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

May 24th 2012

Sysaux Tablespace

Enhanced Incremental Statistics Staleness Tolerance


During data load some rows going to the older partitions
In 11g any DML on older partitions triggered partition statistics to be re-gathered New DBMS_STATS preference
INCREMENTAL_STALENESS

Sales Table
May 18th 2012

1. Partition level stats are


gathered & synopsis created 2. Global stats generated by aggregating partition level statistics and synopsis

May 19th 2012

May 20th 2012

May 21st 2012

When set to
USE_STALE_PERCENT DML on

May 22nd 2012

less than 10% of rows in older partitions will not trigger regather
75 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

May 23rd 2012

Sysaux Tablespace

Statistics Enhancements

New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics

Concurrent statistics gathering


Automatic detection of column groups

Statistic gathering reporting


76 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Statistics Enhancements
Concurrent Statistics Gathering Original introduced in 11.2
Gather statistics on multiple objects at the same time

Controlled by DBMS_STATS preference, CONCURRENT


Uses Database Scheduler and Advanced Queuing Number of concurrent gather operations controlled by

job_queue_processes parameter
Each gather operation can still operate in parallel

77

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Concurrent Statistics Gathering for SH Schema


A statistics gathering job is created for each table and partition in the schema Level 1 contain statistics gathering jobs for all nonpartitioned tables and a coordinating job for each partitioned table
Exec DBMS_STATS.GATHER_SCHEMA_STATS(SH);

Level 2 contain statistics gathering jobs for each partition in the partitioned tables
78 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Concurrent Statistics Gathering on Partition Table


Exec DBMS_STATS.GATHER_TABLE_STATS(SH,SALES);

The number of concurrent gathers is controlled by the parameter job_queue_processes In this example it is set to 3

Sales Table
Partition 1 May 18th 2012

Job1

Partition 2

Remember each concurrent gather operates in parallel In this example parallel degree is 4

May 19th 2012

Job2

Partition 3 May 20th 2012

Job3

79

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Statistics Enhancements
Multiple partitioned table support in 12c

Batch manager to batch smaller jobs together to reduce scheduling

over head Auto statistics gather job can now use concurrency Cap to limit the resource available to the job via resource manager

80

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Statistics Enhancements

New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics

Concurrent statistics gathering


Automatic detection of column groups

Statistic gathering reporting


81 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Extended Statistics
Two types of Extended Statistics
Column groups statistics
Column group statistics useful when multiple column from the same table are

used in where clause predicates or group by clause


Expression statistics
Expression statistics useful when a column is used as part of a complex

expression in where clause predicate

Automatically maintained when statistics are gathered on the table

Candidates for columns groups can be manually or automatically

determined
82 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Automatic Column Group Detection


Oracle can automatically detect column group candidates based on an STS or by monitoring a workload Uses DBMS_STATS procedure SEED_COL_USAGE If the first two arguments are set to NULL the current workload will be monitored The third argument is the time limit in seconds
83 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

1.

Start column group usage capture

Automatic Column Group Detection


Actual number of rows returned by this query is 932 Optimizer under-estimates the cardinality as it assumes each where clause predicate will reduce number of rows returned Optimizer is not aware of real-world relations between city, state, & country
2.

Run your workload

84

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Automatic Column Group Detection


2.

Run your workload

Actual number of rows returned by this query is 145 Optimizer over-estimates the cardinality as it is not aware of the real-world relations between state & country

85

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Automatic Column Group Detection


3.

Check column usage information recorded for our table


SELECT dbms_stats.report_col_usage(user, 'customers') FROM dual;
EQ means column was used in equality predicate in query 1

COLUMN USAGE REPORT FOR SH.CUSTOMERS FILTER means columns used together 1. COUNTRY_ID : EQ as filter predicates rather than join etc. 2. CUST_CITY : EQ Comes from query 1 3. CUST_STATE_PROVINCE : EQ 4. (CUST_CITY, CUST_STATE_PROVINCE, COUNTRY_ID) : FILTER 5. (CUST_STATE_PROVINCE, COUNTRY_ID) : GROUP_BY
GROUP_BY columns used in group by expression in query 2

86

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Automatic Column Group Detection


4.

Create extended stats for customers based on usage

SELECT dbms_stats.create_extended_stats(user, 'customers') FROM dual;

EXTENSIONS FOR SH.CUSTOMERS


1.

(CUST_CITY, CUST_STATE_PROVINCE, COUNTRY_ID): SYS_STUMZ$C3AIHLPBROI#SKA58H_N created

2.

(CUST_STATE_PROVINCE, COUNTRY_ID) : SYS_STU#S#WF25Z#QAHIHE#MOFFMM_ created

Column group statistics will now be automatically maintained every time you gather statistics on this table
87 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Statistics Enhancements

New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics

Concurrent statistics gathering


Automatic detection of column groups

Statistic gathering reporting


88 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

DBMS_STATS.REPORT_STATS_OPERATIONS

89

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

DBMS_STATS.REPORT_GATHER_SCHEMA_STATS

90

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Agenda
Adaptive Query Optimization Optimizer Statistics enhancements SQL Plan Management enhancements

91

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Management


How it works
GB

Plan Acceptable
Execute

Users

Parse Plan history Plan baseline


GB HJ HJ

HJ HJ

Actual execution plans stored in SQL plan baseline in Oracle Database 12c

92

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Management


How it works
GB

Users

Parse Plan history Plan baseline

NL NL

GB NL NL GB HJ HJ

Note you do not need to be in autocapture mode to have a new plan added to an existing SQL plan baseline

Note additional fields such as fetches, row processed etc. are not populated because new plan never executed

93

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SQL Plan Management


How it works
GB

Plan Acceptable
Execute

Users

Parse Plan history Plan baseline

HJ HJ

GB NL NL GB HJ HJ

94

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive SQL Plan Management


SPM Evolve Advisor
New evolve auto task running in the maintenance window
Ranks all non-accepted plans and runs evolve process for them Newly found plans are ranked the highest If new plan performs better than existing plan it is automatically accepted If new plan performs worse than existing plan it will remain unaccepted Poor performing plans will not be retried for 30 days and then only if the

statement is active

95

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Automatic Plan Evolution


Plan history
1

Nightly verification task checks for unaccepted plans

Plan baseline
GB NL NL HJ GB HJ

Plan history Plan baseline


GB
GB NL GB NL NL

It then test execute to check if new plan is as good as or better than old plan
2

Plans which dont perform as good as the original plan stay in the L plan history & are marked unaccepted but last_verified updated
96 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

HJ NL HJ

Plans which perform better than original plan are recommended to be added to the plan baseline

Adaptive SQL Plan Management

SPM Evolve Advisor


New nightly auto task called
SYS_AUTO_SPM_EVOLVE_TASK

Information on task found in DBA_ADVISOR_TASKS


You can review what happening during the task using DBMS_SPM. REPORT_AUTO_EVOLVE_TASK

97

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Reviewing the evolve report


Reports are persisted and can be review any time Clearer findings section summaries the results of the evolve task
Same comparison criteria as in 11g

New recommendation section that clearly states how to implement the recommendations proposed

98

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Adaptive SQL Plan Management


Manually Invoking evolve task
Each task is named to allow it to be re-executed any time Each execution is names to allow the results to be persisted Results can be viewed at any time using the task_id and exec_id
99 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Summary
Optimizer begins to learn from its experiences

New types of statistics


More automatic statistics gathering techniques Faster statistics gathering

Multi-Table Left Outer Join


Partial Join Evaluation (PJE) New Transformations
Null-Accepting Semi Join
Scalar Subquery Unnesting
100 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

More Information
White papers
Optimizer with Oracle Database 12c

SQL Plan Management with Oracle Database 12c


Understanding Optimizer Statistics with Oracle Database 12c

Optimizer Blog
http://blogs.oracle.com/optimizer

Oracle.com
http://www.oracle.com/technetwork/database/focus-areas/bi-

datawarehousing/dbbi-tech-info-optmztn-092214.html
101 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

102

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

You might also like