Use of Computer-Assisted Audit Techniques (CAATs)

1. BACKGROUND

1.1 Linkage to Standards

1.1.1 Standard S6 Performance of Audit Work states "During the course of the audit, the IS auditor should obtain sufficient, reliable and relevant evidence to achieve the audit objectives. The audit findings and conclusions are to be supported by appropriate analysis and interpretation of this evidence."

1.1.2 Standard S5 Planning states "The IS auditor should plan the information systems audit coverage to address the audit objectives and to comply with applicable laws and professional auditing standards."

1.1.3 Standard S3 Professional Ethics and Standards states "The IS auditor should exercise due professional care, including observance of applicable professional auditing standards."

1.2 Need for Guideline

1.2.1 Computer Assisted Audit Techniques (CAATs) are important tools for the IS auditor in performing audits.

1.2.2 CAATs include many types of tools and techniques, such as generalised audit software, utility software, test data, application software tracing and mapping, and audit expert systems.

1.2.3 CAATs may be used in performing various audit procedures including:

  • Tests of details of transactions and balances
  • Analytical review procedures
  • Compliance tests of IS general controls
  • Compliance tests of IS application controls
  • Penetration testing

1.2.4 CAATs may produce a large proportion of the audit evidence developed on IS audits and, as a result, the IS auditor should carefully plan for and exhibit due professional care in the use of CAATs.

1.2.5 This Guideline provides guidance in applying IS auditing standards. The IS auditor should consider it in determining how to achieve implementation of the above Standards, use professional judgment in its application and be prepared to justify any departure.

1.2.6 This guidance should be applied in using CAATs regardless of whether the auditor concerned is an IS auditor .

2. PLANNING

2.1 Decision Factors for Using CAATs

2.1.1 When planning the audit, the IS auditor should consider an appropriate combination of manual techniques and CAATs. In determining whether to use CAATs, the factors to be considered include:

  • Computer knowledge, expertise, and experience of the IS auditor
  • Availability of suitable CAATs and IS facilities
  • Efficiency and effectiveness of using CAATs over manual techniques
  • Time constraints
  • Integrity of the information system and IT environment
  • Level of audit risk

2.2 CAATs Planning Steps

2.2.1 The major steps to be undertaken by the IS auditor in preparing for the application of the selected CAATs are:

  • Set the audit objectives of the CAATs
  • Determine the accessibility and availability of the organisation's IS facilities, programs/system and data
  • Define the procedures to be undertaken (e.g., statistical sampling, recalculation, confirmation, etc.)
  • Define output requirements
  • Determine resource requirements, i.e., personnel, CAATs, processing environment (organisation's IS facilities or audit IS facilities)
  • Obtain access to the organisation's IS facilities, programs/system, and data, including file definitions
  • Document CAATs to be used, including objectives, high-level flowcharts, and run instructions

2.3 Arrangements with the Auditee

2.3.1 Data files, such as detailed transaction files, are often only retained for a short period of time; therefore, the IS auditor should make arrangements for the retention of the data covering the appropriate audit time frame.

2.3.2 Access to the organisation's IS facilities, programs/system, and data, should be arranged for well in advance of the needed time period in order to minimise the effect on the organisation's production environment.

2.3.3 The IS auditor should assess the effect that changes to the production programs/system may have on the use of the CAATs. In doing so, the IS auditor should consider the effect of these changes on the integrity and usefulness of the CAATs, as well as the integrity of the programs/system and data used by the IS auditor .

2.4 Testing the CAATs

2.4.1 The IS auditor should obtain reasonable assurance of the integrity, reliability, usefulness, and security of the CAATs through appropriate planning, design, testing, processing and review of documentation. This should be done before reliance is placed upon the CAATs. The nature, timing and extent of testing is dependent on the commercial availability and stability of the CAATs.

2.5 Security of Data and CAATs

2.5.1 Where CAATs are used to extract information for data analysis the IS auditor should verify the integrity of the information system and IT environment from which the data are extracted.

2.5.2 CAATs can be used to extract sensitive program/system information and production data that should be kept confidential. The IS auditor should safeguard the program/system information and production data with an appropriate level of confidentiality and security. In doing so, the IS auditor should consider the level of confidentiality and security required by the organisation owning the data and any relevant legislation.

2.5.3 The IS auditor should use and document the results of appropriate procedures to provide for the ongoing integrity, reliability, usefulness, and security of the CAATs. For example, this should include a review of program maintenance and program change controls over embedded audit software to determine that only authorised changes were made to the CAATs.

2.5.4 When the CAATs reside in an environment not under the control of the IS auditor, an appropriate level of control should be in effect to identify changes to the CAATs. When the CAATs are changed, the IS auditor should obtain assurance of their integrity, reliability, usefulness, and security through appropriate planning, design, testing, processing and review of documentation before reliance is placed on the CAATs.

3. PERFORMANCE OF AUDIT WORK

3.1 Gathering Audit Evidence

3.1.1 The use of CAATs should be controlled by the IS auditor to provide reasonable assurance that the audit objectives and the detailed specifications of the CAATs have been met. The IS auditor should:

  • Perform a reconciliation of control totals if appropriate
  • Review output for reasonableness
  • Perform a review of the logic, parameters or other characteristics of the CAATs
  • Review the organisation's general IS controls which may contribute to the integrity of the CAATs (e.g., program change controls and access to system, program, and/or data files)

3.2 Generalised Audit Software

3.2.1 When using generalised audit software to access the production data, the IS auditor should take appropriate steps to protect the integrity of the organisation's data. With embedded audit software, the IS auditor should be involved in system design and the techniques will have to be developed and maintained within the organisation's application programs/systems.

3.3 Utility Software

3.3.1 When using utility software, the IS auditor should confirm that no unplanned interventions have taken place during processing and that the utility software has been obtained from the appropriate system library. The IS auditor should also take appropriate steps to protect the integrity of the organisation's system and files since these utilities can easily damage the system and its files.

3.4 Test Data

3.4.1 When using test data, the IS auditor should be aware that test data only point out the potential for erroneous processing; this technique does not evaluate actual production data. The IS auditor also should be aware that test data analysis can be extremely complex and time consuming, depending on the number of transactions processed, the number of programs tested, and the complexity of the programs/system. Before using test data the IS auditor should verify that the test data will not permanently affect the live system.

3.5 Application Software Tracing and Mapping

3.5.1 When using application software tracing and mapping, the IS auditor should confirm that the source code being evaluated generated the object program currently being used in production. The IS auditor should be aware that application software tracing and mapping only points out the potential for erroneous processing; it does not evaluate actual production data.

3.6 Audit Expert Systems

3.6.1 When using audit expert systems, the IS auditor should be thoroughly knowledgeable of the operations of the system to confirm that the decision paths followed are appropriate to the given audit environment/situation.

4. CAATs DOCUMENTATION

4.1 Workpapers

4.1.1 The step-by-step CAATs process should be sufficiently documented to provide adequate audit evidence.

4.1.2 Specifically, the audit workpapers should contain sufficient documentation to describe the CAATs application, including the details set out in the following sections.

4.2 Planning

4.2.1 Documentation should include:

  • CAATs objectives
  • CAATs to be used
  • Controls to be exercised
  • Staffing and timing

4.3 Execution

4.3.1 Documentation should include:

  • CAATs preparation and testing procedures and controls
  • Details of the tests performed by the CAATs
  • Details of inputs (e.g., data used, file layouts), processing (e.g., CAATs high-level flowcharts, logic) and outputs (e.g., log files, reports)
  • Listing of relevant parameters or source code

4.4 Audit Evidence

4.4.1 Documentation should include:

  • Output produced
  • Description of the audit analysis work performed on the output
  • Audit findings
  • Audit conclusions
  • Audit recommendations

5. REPORTING

5.1 Description of CAATs

5.1.1 The objectives, scope and methodology section of the report should contain a clear description of the CAATs used. This description should not be overly detailed, but it should provide a good overview for the reader.

5.1.2 The description of the CAATs used should also be included in the body of the report, where the specific finding relating to the use of the CAATs is discussed.

5.1.3 If the description of the CAATs used is applicable to several findings, or is too detailed, it should be discussed briefly in the objectives, scope and methodology section of the report and the reader referred to an appendix with a more detailed description.

6. EFFECTIVE DATE

6.1 This guideline is effective for all information systems audits beginning on or after 1 December 1998.