DRAFT

INTOSAI Public Debt Committee

Guidance on Using Substantive Tests in Audits of Public Debt

Final Report

June 2006

INTOSAIs Public Debt Committee

Page 1 of 16

Guidance on Substantive Tests

DRAFT

Public Debt Committee Chairman Arturo Gonzlez de Aragn, P.C.A. Superior Auditor of Mexico Members
Argentina Canada Gabn Korea Lithuania USA Mexico Portugal United Kingdom Russian Federation Sweden Zambia Yemen -

Collaborators
Chile Jordan Egypt

This publication has been prepared for official publication in separate English, Spanish, (________, and ________) versions by the Public Debt Committee of the International Organization of Supreme Audit Institutions (INTOSAI).

INTOSAIs Public Debt Committee

Page 2 of 16

Guidance on Substantive Tests

DRAFT
INDEX INTRODUCTION PART 1 Purposes and definitions of substantive audit procedures PART 2 Analytical Procedures PART 3 Substantive Audit Procedures for Derivatives PART 4 Substantive Audit Procedures for Unrecorded Contingencies PART 5 Sampling Procedures

INTOSAIs Public Debt Committee

Page 3 of 16

Guidance on Substantive Tests

DRAFT
INTRODUCTION 1. Under the terms of reference laid down by the Governing Board of INTOSAI the Public Debt Committee (PDC) was given the task of publishing guidelines and other information for use by Supreme Audit Institutions (SAIs) to encourage the proper reporting and sound management of public debt. In recent years the PDC has discharged this responsibility by publishing guides on the definition, disclosure and reporting of public debt, a guide on auditing internal controls, as well as research studies related to financial commitments, contingencies and fiscal risks that affect the management and audit of public debt. 2. This guide covers the following issues related to substantive tests in public debt audits: (1) Purposes and definitions of substantive audit procedures (2) Analytical procedures (3) Substantive audit procedures for derivatives (4) Substantive audit procedures for unrecorded contingencies (5) Audit sampling techniques 3. This guide is based on documents presented and discussed at PDC meetings in past two years. This guide incorporates the feedback received at those meetings, and is based mainly on two documents, namely, The Audit of Public Debt Application of INTOSAI Auditing Standards (1st Draft), prepared by UK National Audit Office, and Guidance for Using Substantive Tests in Audits of Public Debt, prepared by US Government Accountability Office.

INTOSAIs Public Debt Committee

Page 4 of 16

Guidance on Substantive Tests

DRAFT
PART 1 Purposes and Definitions of Substantive Audit Procedures 1. Substantive audit procedures are performed after auditors have completed the planning and internal control steps of a public debt audit. Substantive audit procedures help auditors to meet the INTOSAI field standard requirement: 2. Competent, relevant and reasonable evidence should be obtained to support the auditors judgment and conclusions regarding the organization, program, activity or function under audit. 3. The audit findings, conclusions and recommendations must be based on evidence. 4. Auditors should have a sound understanding of techniques and procedures such as inspection, observation, inquiry and confirmation, to collect audit evidence. 5. In choosing approaches and procedures, consideration should be given to the quality of evidence. 6. In evaluating the nature and extent of use of substantive audit procedures, auditors consider the main objective of their audit, namely, to determine whether public debt elements are recorded correctly with respect to accounts, amounts and periods, authorized by law and executive regulations, and disclosed to policymakers and the public in a timely, consistent and transparent manner. 7. In order to collect sufficient evidence to form an opinion, auditors examine public debt transactions, such as borrowings, debt servicing, voluntary and forced debt restructurings. Auditors also examine public debt balances, including the amounts of main debt categories (domestic and foreign, short- and long-term, marketable and non-marketable). In addition to examining debt balances and borrowing transactions, auditors use analytical procedures to compare actual and expected relations between key variables, such as the relationship between public debt levels and interest expenditures. 8. In performing tests of debt balances and borrowings, the auditor is concerned with overstatement or understatement of the line item in the financial statement. The tests make use of the inherent properties of double-entry accounting systems, that is, a test of one side of a public debt transaction simultaneously tests the other side of the transaction (e.g. cash account). Accounting entries that are normally examined include cash borrowings, debt rollovers, interest payments and extraordinary debt transactions, such as advanced debt purchases at market prices, and debt renegotiations with official and private creditors.

INTOSAIs Public Debt Committee

Page 5 of 16

Guidance on Substantive Tests

DRAFT
9. The levels of public debt and related borrowing transactions may be described in terms of their characteristics: Existence whether a liability exists at a given date. For example, the debt securities reported in the financial statements of the government exist at the end of the reporting period. Rights and obligations whether the liability pertains to the reporting entity. For example, a government has rights and obligations associated with the debt securities, money market instruments or derivative instruments reporting in the governments financial statements. Occurrence whether a transaction or event took place which pertains to the borrower during the reporting period. For example, the transaction that gave rise to a derivative, or to a profit or loss on the disposal of a debt security, occurred during the reporting period. Completeness whether there are no unrecorded debt instruments or borrowings, or undisclosed items. For example, all of the governments debt securities, deposits, money market instruments or derivatives are reported in the balance sheet and disclosed in footnotes. Valuation the amount of assets and liabilities is recorded at an appropriate carrying amount. For example, the values of debt securities or derivatives reported in the financial statements were determined in accordance with relevant legislation, regulations and applicable accounting standards. Measurement a borrowing event is recorded at the proper amount and expense is allocated to the proper period. For example, interest outlays are properly accrued or recorded; profits or losses on sales of debt securities are correctly calculated and attributed to the correct accounting period. Presentation and disclosure an item is disclosed, classified and described in accordance with the applicable reporting framework, including relevant legislation and applicable accounting standards and regulations. In order to determine if any of the above characteristics are materially misstated, the auditor would design and apply substantive tests, but only after performing and evaluating the results of the tests of internal controls. Based on the level of expected overall audit assurance the auditor determined in the planning phase of the audit, the auditor establishes the minimum levels of substantive assurance for each level of combined audit risk. For example, GAOs audit risk model provides that, based on a desired overall audit assurance of 95 percent, the following minimum

INTOSAIs Public Debt Committee

Page 6 of 16

Guidance on Substantive Tests

DRAFT
levels of substantive assurance for each level of combined risk would be appropriate: (a) if combined risk is low - 63%, (b) if combined risk is moderate - 86%, and (c) if combined risk is high - 95%. In other words, the higher the combined risk, the more substantive assurance would be required.1 Auditors use a combination of the following substantive procedures to obtain the evidence required to provide them with the appropriate assurance. Inspection examining records, documents and tangible assets. Three major categories of evidence are: 1) Evidence created and provided to auditors by third parties, including such items as bank statements for cash or custodian statements of debt security holdings. 2) Evidence created by third parties and held by the borrower, including the results of counterparty transactions. For example, outstanding balances on repurchase agreements, that is, collateralized borrowings. 3) Evidence created and held by the borrower, including the debt schedules, records and reconciliations underpinning the financing statements. Observation looking at borrowing transactions and management actions performed by debt management staff, in particular those that leave no audit trail. For example, attending a debt auction and observing a debt strategy committee meeting. This technique helps to ensure that all procedures are properly adhered to and monitored. Inquiry seeking information from knowledgeable persons inside and outside the debt management office. Inquiries may range from formal written inquiries to third parties, to informal oral inquiries to persons inside the debt management office. Responses to inquiries can provide audito2rs with information not available previously or serve as corroborative evidence. Key stakeholders that the auditor may consult in a public debt audit include primary dealers and debt management office staff in front, middle and back offices.
1

Substantive assurance is the auditor's judgment that all of the substantive tests will detect misstatements that in total exceed materiality threshold. Substantive assurance is related to the entire audit and is directly related to level of combined risk; it is not the same as the confidence level of an individual sample. VOUCHING consists in selecting sample items from an account and going backward through the accounting system to find the source documentation that supports the item selected. TRACING consists in selecting sample items from basic source documents and proceeds forward through the accounting system to find the final recording of the transactions (e.g., in the general ledger).

INTOSAIs Public Debt Committee

Page 7 of 16

Guidance on Substantive Tests

DRAFT
Confirmation responses to inquiries designed to corroborate information in accounting records. For example, the auditor may seek direct confirmation of debt amounts by communicating with creditors. Recalculation checking the mathematical accuracy of debt records by footing or crossfooting or by recomputing amounts and tracing journal postings, subsidiary ledger balances, and other details to the corresponding general ledger accounts. For example, the auditor can recalculate payments in the interest payable list, foot the list, and trace the total to the general ledge interest payable amount. 10. In general, the reliability of the evidence is influenced by its source internal or external, and by its nature visual, documentary, or oral. Evidence obtained and verified directly by auditors is more reliable than that obtained by third parties. For example, visual observation of public debt operations and computations by auditor are more reliable than observations and computations done by third parties. Written documents are the second most reliable form of audit evidence. Original documents are more reliable than photocopies and facsimiles. Oral interviews are the least reliable audit evidence. Oral interviews with third parties can be more reliable than interviews with debt management staff, but less reliable than written documents.

INTOSAIs Public Debt Committee

Page 8 of 16

Guidance on Substantive Tests

DRAFT
11. The following table provides common examples of substantive procedures in public debt audits. The procedures are grouped by type of assertion; in practice the same procedure can be used to test several assertions. Assertion Existence and Occurrence Rights and Obligations Audit Procedure Confirmation with the holder of the debt security or fiscal agent or custodian of debt records or trustee Inspection of underlying debt agreements and other supporting documents, confirmations received by creditor, in paper or electronic form, for amounts reported Inspection of supporting documents for subsequent realization or settlement after the end of the reporting time period Observation of primary auctions and underwritings

Assertion Rights and Obligations

Audit Procedure Confirmation with the holder of the debt security or fiscal agent or custodian of debt records or trustee Inspection of underlying debt agreements and other supporting documents, confirmations received by creditor, in paper or electronic form, for amounts reported

Assertion Audit Procedure Comple- Review of all counterparty transactions. When requesting details from the teness counterparty, consider which part is responding, and whether this represents all relevant aspects of its dealing with the borrower Send zero-balance confirmations to potential debt holders or counterparties Review primary dealers statements for the existence of transactions and holdings of public debt Use computer-aided techniques to extract aggregate trading data for agreement with general ledger and financial statement report Perform sampling tests of individual trades for counterparty confirmations and after-date receipts Review accounting records before and after the year end for unusual transactions Review counterparty confirmations received but not matched to transaction records Review unresolved reconciliation items in reports Inspect debt agreements for embedded derivatives Review minutes of debt committee and related papers Perform calculation for proper accrual and recognition of debt expense

INTOSAIs Public Debt Committee

Page 9 of 16

Guidance on Substantive Tests

DRAFT
Assertion Valuation and Measurement Audit Procedure Inspect documents to verify cash receipts in borrowing Confirm the nominal value of debt amounts with fiscal agent or trustee Re-calculate mark-to-market calculations for a sample of high value debt instruments Check the accuracy of translation of book and market value of debt securities denominated in foreign currencies Use quoted market prices to verify values disclosed of debt securities, money market instruments and derivatives

Assertion Presentation & Disclosure

Audit Procedure Verify that accounting principles selected and applied are in conformity with legislation, regulations and applicable accounting standards, and are appropriate for the debt management office Verify that the financial statements and related footnotes provide sufficient disclosure that is neither too detailed nor too condensed Verify that the financial statements provide information on matters that may affect their use, understanding and interpretation Verify that the financial statements reflect transactions in a manner that present the debt levels, results of borrowings and interest payments, and cash flows within a range of acceptable limits Review the classification of debt securities to ensure it is in agreement with the legislation, regulations and practices

INTOSAIs Public Debt Committee

Page 10 of 16

Guidance on Substantive Tests

DRAFT
PART 2 Analytical Procedures 1. Analytical procedures consist of comparing recorded account balances with the auditor's expectations. The auditor develops an expectation or estimate of what the recorded amount should be based on an analysis and understanding of relationships between the recorded amounts and other data. This estimate is then used to form a conclusion on the recorded amount. A basic premise underlying analytical procedures is that plausible relationships among data may reasonably be expected to continue unless conditions are known that would change the relationship. 2. Analytical procedures generally rely on aggregate data rather than unit values, which makes them more effective and efficient than tests of individual transactions. Common analytical procedures involve the use of ratios, trends and variance analysis. More sophisticated analytical procedures use econometric analysis, including regression, simulations, stress-testing and large-scale economic models. 3. An Illustration Explaining the Difference between Expected and Actual Interest Expense. Suppose the auditor estimates that the interest expense for the current period is $80 million. The auditor obtains this estimate based on a $1 billion public debt average balance times 8 percent, the average annual interest rate. The materiality limit for analytical procedures is $5 million. The auditor finds that the the actual amount of interest expense is $94.5 million. The difference - $14.5 million - exceeds the test materiality by $9.5 million. Auditors ask debt managers and their explanation is that we borrowed more money and interest rates are higher than last year. The auditor needs to corroborate this explanation. For example, auditors can find that interest rates increased during the year and then fell, and were computed to average 9 percent based on a monthly average instead of 8 percent. Additionally, loan statements from lenders indicate that $100 million was borrowed and repaid during the year, and the additional borrowings were outstanding for 6 months. Thus, the average loan balance was actually $50 million higher and the average interest rate was 1 percent higher than the figures used in the auditor's original estimate. 4. If substantive analytical procedures are used, the auditor should perform steps a. through l. below. a. Determine the amount of the limit. The limit is the amount of difference between the auditors expectation and the recorded amount that the auditor will accept without investigation. The determination of the limit is a matter of the auditor's judgment. b. Identify a plausible, predictable relationship and develop a model to calculate an expectation of the recorded amount. Consider the type of

INTOSAIs Public Debt Committee

Page 11 of 16

Guidance on Substantive Tests

DRAFT
misstatements that could occur and how those misstatements would be detected by the model. c. Gather data for developing the expectation, and perform appropriate procedures to establish the reliability of the data. The reliability of these base data is subject to the auditor's judgment. d. Develop the expectation of the recorded amount using the information obtained during the previous steps. The preciseness of the expectation is subject to the auditor's judgment. e. Compare the expectation with the recorded amount, and note the difference. f. Obtain explanations for differences that exceed the limit, since they are considered significant. g. Corroborate explanations for significant differences. h. Determine whether the explanations and corroborating evidence provide sufficient evidence for the desired level of substantive assurance. If unable to obtain a sufficient level of substantive assurance from analytical procedures, perform additional procedures and consider whether the difference represents a misstatement. i. Consider whether the assessment of combined risk remains appropriate, particularly in light of any misstatements identified. Revise the assessment of combined risk, if necessary, and consider the effects on the extent of detail tests. j. Document the amount of any misstatements detected by substantive analytical procedures and their estimated effects. The limit -- the amount of the difference between the recorded amount and the expectation that does not require explanation -- is not considered a known or likely misstatement and is not included in the list of possible audit adjustments. k. Conclude on the fair presentation of the recorded amount. l. Include documentation of work performed, results, and conclusions in the workpapers.

INTOSAIs Public Debt Committee

Page 12 of 16

Guidance on Substantive Tests

DRAFT
PART 3 Substantive Audit Procedures for Derivatives 1. Derivatives have become commonly used in public debt transactions. Major industrialized countries with established capital markets routinely use combinations of swaps and long-term debt borrowing to shorten the duration of their debt profile, reduce their cost of borrowing, and provide liquidity to their benchmark securities across the yield curve. Other governments find that traditional public debt arrangements, like loans from multilateral financial institutions, are routinely offered with derivatives features that require special audit and management skills. Examples of the latter category include World Bank loans with collars, different rates and currencies. 2. Common substantive audit procedures applied to derivative instruments include the following. Assertions: Completeness and Existence Confirm significant terms with the holder of, or counterparty to, the derivative Inspect underlying agreements and other forms of supporting documentation, in paper or electronic form Ask holder of or counterparty to the derivative to provide details of all derivatives and transactions with the debt management office. Send zero-balance confirmations to potential holders or counterparties to derivatives to test completeness Review brokers statements for existence of derivative transactions and positions held Review counterparty confirmations received but not matched to transactions records Review unresolved reconciliation items Inspect agreements, such as loan or equity agreements, for embedded derivatives

Assertions: Valuation and Effectiveness Assess of the reasonableness of models, variables and assumptions used to value derivatives Gather market prices to assess valuation that are firm and valid. Assess the sensitivity of valuation to changes in variables and assumptions Inspect supporting documents for settlement of the derivative transactions after end of the reporting period Use proprietary models or the debt management offices internally developed models to assess valuation when no market prices exist Use analytical procedures to evaluate risk management policies, including compliance with credit limits.

INTOSAIs Public Debt Committee

Page 13 of 16

Guidance on Substantive Tests

DRAFT
Assess the effectiveness of hedging activities (e.g., to minimize large gains or losses)

Assertion: Derivatives Used for Hedging In the case of derivatives used as hedges, auditors should gather sufficient evidence to determine effectiveness of the hedging relationship. This entails the following procedures: Assess whether the derivative was designated as a hedge at the inception of the transaction Determine what was the nature of the hedge Determine what was the risk management objective for undertaking the hedge Determine what was the debt managers assessment of the hedges effectiveness If the derivative was hedging a future debt transaction, determine what was the debt managers assessment of the likelihood of the future event Assess the extent of disclosures of derivatives used as hedges, and extent of compliance with laws and regulations that require disclosure of derivative transactions, including notional and fair value, number and credit quality of counterparties, value at risk, stress test results, etc.

INTOSAIs Public Debt Committee

Page 14 of 16

Guidance on Substantive Tests

DRAFT
PART 4 Substantive Audit Procedures for Unrecorded Contingencies 1. Large unrecorded contingencies and commitments are a major financial challenge in both OECD and developing countries. Escalating health care costs combined with an aging population have created an unsustainable fiscal risk in the United States, Japan and other OECD countries. Some countries spend scarce budget resources to subsidize government-owned enterprises (e.g. government-owned railways). Most countries provide explicit and implicit guarantees on deposits of financial institutions. These unrecorded contingencies and commitments are not fully accrued and reported in the governments financial statements. 2. Common substantive audit procedures applied to unrecorded contingencies and commitments include the following. Examine approved budget for all outstanding government guarantees, subsidies and commitments Review the governments expenditures and compare actual outlays with budget amounts Examine long-term lease agreements for possible capital lease recognition Examine government pension agreements for benefit vesting provisions and funded status Examine privatization agreements of former government-owned enterprises for minimum revenue government guarantees

INTOSAIs Public Debt Committee

Page 15 of 16

Guidance on Substantive Tests

DRAFT
PART 5 Audit Sampling Techniques 1. Auditors can apply substantive audit procedures to all items that comprise a debt account balance, to a non-representative small group of the items, or to a representative sample of the items. All debt items would be generally selected if their total number is small. The option to select a nonrepresentative group is appropriate when the auditor knows enough about the population and is able to identify the specific items of interest, usually because they are likely to be misstated or have a high risk. In this case, the results of the application of substantive tests and the effects of any misstatements must not be projected to the items that were not tested. 2. Finally, a representative selection of items is appropriate when there are many items and it is not possible to obtain sufficient assurance by examining only a nonrepresentative group. Selecting a representative sample of debt items allows auditors to project the results of their substantive tests on the sample to the population. Auditors generally find that it is efficient to combine the above three selection methods. For example, an auditor can select all debt items with a large book amount and a statistical sample of the balance of the other debt items. 3. In the process to select a representative sample the auditor must first define the item to be sampled. In dollar-unit sampling (DUS), each dollar of debt has an equal chance of being selected. In classical variables estimation sampling, each item in a stratum has an equal chance of being selected. 4. Probability-proportional-to-size (PPS) sampling or dollar-unit sampling (DUS). This technique is used to give larger debt items proportionally more opportunity to be selected than units with smaller debt amounts. Auditors use this technique when they expect to find a small number of large misstatements (this is also known as the needles in haystack challenge). This technique usually results in smaller sample size and can be used in conjunction with stratified selection techniques. 5. Stratified Selection. This technique allows auditor to group together homogeneous debt transactions and instruments. A stratified selection allows auditor to apply different audit techniques to each debt classification, and improves efficiency of sample design, achieving a smaller sample size. By stratifying the population, the auditor can give greater representation to the larger recorded debt transactions that are small in number.

INTOSAIs Public Debt Committee

Page 16 of 16

Guidance on Substantive Tests