An investigation of alternative report formats for communicating moderate levels of assurance.

INTRODUCTION

In recent years, there has been an expansion of the traditional audit function to a wide range of subject matter, such as prospective financial information, environmental performance and internal control (Elliott 1998, Elliott and Palais 1997), and at different levels of assurance.

   in theory, it is possible to provide an infinite range of assurance
   from a very low level to an absolute level of assurance. In practice
   it is not ordinarily practicable to provide such fine graduations of
   assurance or to communicate the level of assurance in a clear and
   unambiguous manner ... Therefore professional accountants ordinarily
   undertake engagements to provide one of only two distinct levels: a
   high level and a moderate level

The current regulatory framework provides guidance for high level assurance engagements and guidance is expected to be provided for moderate level assurance engagements.

Moderate assurance refers to engagements where the professional accountant obtains sufficient appropriate evidence to be satisfied that the subject matter is plausible in the circumstances (ISAE No. 100, para. 30). These engagements are intended to provide a lower level of assurance than traditional high assurance engagements, and are expected to constitute a continuing area of growth for the profession. This increasing interest in moderate assurance is evidenced in the recent calls for research in this area, as well as the recent monograph commissioned by the International Federation of Accountants (IFAC 2002). This monograph states: "The accurate determination and effective communication of levels of assurance provided in assurance engagements are critical issues for the well-being of the profession and the future development of assurance services" (IFAC 2002, i).

One of the issues addressed in the IFAC monograph is whether the various national auditing institutes believe that users understand the different levels of assurance that are being conveyed in assurance reports issued by their members. The results indicate that many member institutions have concerns related to potential perception problems associated with the different forms of reporting required for financial statement audits, where the positive form of expression is used to communicate high assurance and the negative form is commonly used to communicate moderate assurance. In particular, concerns were raised about the negative form of expression that uses the phrase "nothing has come to my attention." From the preparers' perspective, the best way to ensure the ability to provide a clean opinion that nothing has come to their attention is to do nothing. From the users' perspective, concerns were raised that a high level of assurance could be attributed to a report that states that nothing has come to the assurer's attention.

These concerns regarding effective communication associated with the negative form of opinion are not new. However, they are potentially exacerbated for "other" assurance services where, internationally, the reporting form for moderate assurance engagements is not mandated. The IFAC (2002) report includes a survey of moderate assurance engagements and identifies a wide range of subject matter on which assurance is currently provided and a number of different reporting formats used in practice. The present IFAC framework does not include reporting considerations for moderate assurance engagements other than those involving financial statements, but this issue is on their current work agenda. Different countries provide various forms of guidance on how to determine and communicate the level of assurance for such engagements. The United States has adopted the approach of issuing attestation standards that define the types of engagements for which professional accountants can provide assurance, the levels of assurance that can be provided, and the form of reporting. Most other countries either provide conceptual guidance on the elements of an engagement that will affect assurance (similar to those reported in ISAE No. 100, para. 28), or provide no guidance at all.

As highlighted in the IFAC monograph, there is a scarcity of empirical evidence to inform the standard-setting process, with very few studies examining the communicative effectiveness of alternative reporting formats. Research undertaken in this area will be of significant benefit to both standard setters and readers of assurance reports. This study contributes to knowledge in this field by focusing on two specific issues. First, it compares a high assurance report with current forms of moderate assurance reports used in practice to determine whether users do perceive a lower level of assurance. Second, it examines these different types of moderate assurance report formats to see if differences in opinion formats have an impact on the perceived level of assurance.

We begin with a review of the existing research relating to the expectations gap, communication issues, and assurance reports. This is followed by the development of the research questions, and then an outline of the methodology. Empirical results are presented, and the final section discusses the implications and limitations of the findings.

THEORY AND PRIOR RESEARCH

The significance of the communication issue in auditing has traditionally been expressed in the literature in terms of the expectations gap, i.e., significant differences between what the public expects from an audit and what the profession understands the objectives to be. Humphrey et al. (1992) suggest that the nature and meaning of assurance report messages is one of the key elements in the expectations gap debate. Empirical studies (e.g., Nair and Rittenberg 1987; Yardley 1989; Bartlett 1991; Gay et al. 1998) have shown that significant differences in perception exist between users and preparers of audit, review, and compilation reports.

Given that assurance reporting is a communication process, theories from the communications literature (drawing on fields as diverse as psychology, linguistics, and mathematics) may be useful in analyzing and enhancing the communication associated with assurance reports. Fiske (1990) categorizes this literature into two main schools of thought: the process and the semiotic views of communication.

The process school depicts a linear model of communication, where information flows from a source, through a channel, to a receiver (Shannon and Weaver 1949). This body of thought suggests that communication is enhanced by focusing on the channel, which in the assurance reporting context is the assurance report itself. This has been the implicit rationale underlying studies that examine the effect of changing report wording (for example, Hatherly et al. 1991; Houghton and Messier 1991; Monroe and Woodliff 1994). Such studies have found that users are sensitive to changes in terminology used (Bailey et al. 1983; Kelly and Mohrweis 1989) and to an increase in the amount of information provided (Innes et al. 1997).

The semiotic school adopts a broader view of the communication process, emphasizing the interaction between the reader and the message in the generation of meaning and therefore the process of communication (Ogden and Richards 1949; Barthes 1968). Thus, effective communication is contingent on an individual's capacity to understand and interpret the meanings of words and sentences (Korzybski 1958). For example, Schandl (1978) argues that one of the main obstacles in communication is predicting the effect of the data on the receiver. He proposes that the communicator should use "symbols" (standard forms of reporting) bearing in mind the connotations they may evoke in the reader's schemata. This is perhaps best conveyed by his suggestion: "If you want to say ... something new, you should start with something the [receiver] already knows" (Schandl 1978, 66).

The communication literature thus suggests that effective communication can be achieved by focusing on the channel of communication, as well as the interaction between the reader and the message conveyed by the channel. This study incorporates both of these approaches by examining the impact of alternatively worded reports on user perceptions. In doing so, it also derives feedback on the reporting forms examined, noting Schandl's (1978) proposition that obtaining feedback is one of the elements of effective communication.

The assurance reports of interest in this study pertain to those providing moderate levels of assurance. A number of previous studies have focused on the communication of assurance on financial statements at levels other than high. Earlier work found that users could not distinguish between the level of assurance provided in an audit (positive assurance) and a review engagement (negative assurance) (Pany and Smith 1982; Pillsbury 1985; Nair and Rittenberg 1987). Subsequent research, however, indicated that users perceived the audit report provided greater assurance than the review report, suggesting that increasing familiarity with review reports may have enhanced their communicative ability over time (Brown et al. 1993; Gay et al. 1998). These studies examined moderate assurance only in the context of financial statement reviews and did not consider the broader range of possible assurance engagements. Arnold et al. (2000) examined communication effectiveness for the assurance service of software product reliability and challenged the view that assurance can be effectively communicated at only two levels. In their experimental market study, where it was possible to summarize and communicate the assurance provided with one word (e.g., high, moderate, or low), they found that the market was better informed (less confused and chaotic) when the assurance provider made assurance available at three levels (high/medium/low) than when assurance was provided at only two levels (high/low).

An examination of the determination and communication of levels of assurance in the context of newly developing assurance services was undertaken by IFAC (2002). The study included two questionnaire surveys. The first questionnaire surveyed 12 national institutes of countries to identify what they considered to be the determinants of a moderate level of assurance and the guidance that they offered on report format. The second questionnaire was sent to the Big 5 and five non-Big 5 firms in each of the 12 countries whose national institutes had participated in the first questionnaire. It asked each firm to identify the determinants of the level of assurance provided and how this was communicated. They were also asked to provide examples of moderate assurance reports issued by their firms, with an emphasis on any innovative reporting formats they may be using. The surveys identified a range of factors that determined the level of assurance, broadly in accordance with the factors contained in ISAE No. 100, and a range of reporting formats used in practice to communicate moderate assurance.

One of the assurance services that was found to represent a significant proportion of moderate assurance engagements was environmental and sustainability reporting. Appendix 2 of the IFAC report (2002) highlights that one of the major issues in communicating a moderate level of assurance is the variety of opinion formats being used. While all of these reports claimed to provide moderate assurance, the IFAC study did not examine whether this communication was effective from a user perspective. This is the focus of the current study.

DEVELOPMENT OF RESEARCH QUESTIONS

This study uses moderate assurance reports on environmental and sustainability reporting. Environmental and sustainability information is an important issue for shareholders' decision making and reporting on such matters has become an important issue for corporations. Groups such as the Global Reporting Initiative have emerged, (2) producing guidance such as the Sustainability Reporting Guidelines (Global Reporting Initiative 2000). Providing assurance on such reports is a relevant issue for the profession. A study conducted by KPMG (2002) revealed that 45 percent of the top 250 companies of the Global Fortune 500 companies, and 28 percent of the top 100 companies for each of 19 countries issued environmental, social, and sustainability reports. A quarter of these reports were independently assured, with the Big 5 accounting firms providing 65 percent of these assurance activities. (3) This assurance service has also drawn the attention of policy makers (e.g., Federation des Experts Comptables Europeens 2002). Therefore from an investor, corporate, practitioner, and regulatory perspective, environmental assurance represents a significant area in relation to moderate assurance engagements.

As outlined earlier, the ISAE No. 100 framework on assurance engagements permits assurance to be provided at two levels: high and moderate. The initial question is whether users perceive that moderate assurance reports provide assurance at a level distinctly lower than high. Prior empirical research discussed above indicates that users did not perceive significant differences between high and moderate assurance reports when financial statement reviews were a relatively new assurance product. If various moderate assurance reporting formats are effective, they should each communicate a level of assurance significantly below that contained in an assurance report intended to communicate a high level of assurance. This issue is explored with Research Question 1:

RQ1: Do shareholders perceive a lower level of assurance from a moderate assurance report compared to a high assurance report?

There is a wide range of reporting styles utilized in practice for moderate assurance reports. The IFAC monograph (2002, Appendix 2) highlighted that various report (opinion) formats are used. However, the existing international assurance reporting framework suggests that irrespective of the diversity of moderate assurance report formats currently used in practice, they should still provide the same (moderate) level of assurance to users (ISAE No. 100, para. 28). The reports examined in this study (with the exception of the high assurance report) were deemed by the assurance provider to provide moderate assurance. While this is the intention of the assurance provider, there is no evidence to date that the user will infer a similar (moderate) level of assurance from each of these report formats. This is examined in the following research question:

RQ2: Are alternative report (opinion) formats in moderate assurance reports perceived to provide significantly different levels of assurance?

RESEARCH METHOD

Development of Appropriate Moderate assurance reporting Formats

In order to determine a set of moderate assurance reports for further investigation, current moderate assurance reporting techniques were examined. This approach was adopted to ensure that reports developed would be reflective of practice and usable by the practitioners who have issued them. The moderate assurance reports considered in detail were those relating to environmental information submitted for the IFAC (2002) study. (4) Of the 56 respondents to the survey of assurance firms, 27 indicated that they provided assurance with respect to environmental information (24 indicated they provided this assurance at a moderate level), and 18 examples were provided of reports issued on such engagements. These reports were examined as a sample of the variety of innovative reporting forms being used in practice at present. (5)

The opinion paragraph in the 18 moderate assurance reporting forms was worded in various ways. Two researchers independently coded these opinion paragraphs and separately identified four broad categories. These four categories were agreed upon, and then the 18 reports were independently coded into these categories. A 100 percent inter-rater agreement was achieved. These categories were labeled opinion on procedures, negative assurance, positive assurance, and positive assurance with a limitations paragraph.

Seven of the reports examined were in the form of what was classified as an opinion on procedures format. This form of opinion refers back to the objectives of the engagement outlined in the assurance report and phrases the opinion in terms of these objectives including reference to procedures performed. This is similar to agreed-upon procedures engagements where the conclusion is structured as a listing of the findings of the assurance provider relating to procedures performed. However, the conclusion contains an opinion, rather than statement of findings. It is this expression of opinion that brings this form of reporting within the international assurance framework. (6)

Five reports used a negative assurance opinion, which informs the user that nothing has come to the assurance provider's attention that causes them to believe that the information is not presented in accordance with the appropriate framework. It does not confirm that material errors are absent, and only indirectly expresses the intended assurance, with the double negative wording emphasizing the limited nature of the assurance. This form of opinion is one of the more common ways of expressing a moderate level of assurance in practice. The advantage of using this format for moderate assurance engagements is that users are already familiar with it from its use in financial statement audits, thus making it a benchmark for reports associated with a lower level of assurance.

Four expressed the opinion in the form of positive assurance, which refers to an expression of an opinion that the information is presented in accordance with the specified framework. This is similar to current forms of reporting used for high assurance engagements (including audits in the international standards and examinations in the U.S. attestation standards). However, assurance is limited by the report including phrases associated with moderate assurance such as "review," and a description of a reduced work effort compared to that necessary to provide a high assurance level. An interesting issue to be tested is whether this opinion will lead users to infer the intended lower level of assurance.

Two reports provided positive assurance with a limitations paragraph, which involves the inclusion of an additional paragraph outlining the inherent limitations associated with providing assurance on environmental reports. These limitations refer to the difficulties associated with providing assurance on environmental information, given its often qualitative nature. A similar paragraph is often used in assurance reports on prospective financial information and internal control.

Based on this analysis four report formats were developed. They were identical in all aspects except for the different forms of expressing the opinion. They were titled Independent Review Report and were explicitly stated to provide a moderate level of assurance based on the review undertaken. The purposes and scope paragraph highlighted that the scope of the engagement was not as extensive as that of an audit and the description of work effort undertaken was consistent with the level of work effort that was described in the moderate assurance reports examined. The basic format of the moderate assurance report is shown in the Appendix, with the different wordings used for the opinion paragraph presented in the first section of Table 1. To ensure the reports were appropriate for distribution, they were reviewed for their validity and realism by a staff member of IFAC, who was a technical expert with significant experience in the area of assurance reporting.

Development of High assurance reporting Format

In order to examine RQ1, a report intended to communicate a high level of assurance was developed as a "control" treatment. It was used to assess whether users correctly inferred a lower level of assurance for the four moderate assurance reports. The high assurance report was primarily based on the example report contained in the appendix to the draft standard Assurance Engagements on Environmental Reports (International Auditing Practices Committee [IAPC] 2001a). It was adapted to ensure consistency with the four moderate assurance reports. The form of opinion for the high assurance report is presented in Table 1. Other differences in the report were that the purposes and scope paragraph contained no discussion of limitation of scope, and the description of work performed was consistent with the level of work effort required to provide a high level of assurance.

Development of the Research Instrument

A draft of the questionnaire was shown to technical specialists and the executive officer of the Australian Shareholders' Association in order to obtain feedback on the clarity, validity, and appropriateness of the questions and questionnaire design. This review covered both technical matters (such as understandability and the wording of the questions) as well as practical issues (such as length, visual appearance, and layout). As a result, minor amendments were made to clarify the wording and presentation of the document.

The final version of the questionnaire consisted of three parts. The first section provided background information and elicited respondents' views on environmental assurance services generally. The background information consisted of a short description of the Environmental Report of a hypothetical company. This paragraph stated that the report was issued with the company's annual report each year and outlined the various types of information, qualitative and quantitative, that was contained in this report. This description was based on an extensive review of environmental reports issued by companies at present, as well as environmental guidelines produced by the IFAC and the Global Reporting Initiative. This ensured that it was reflective of environmental reports that shareholders would have encountered in their experience. (7) Following this introduction, subjects were required to respond to two questions. The first related to whether environmental assurance is a subject on which shareholders require assurance, while the second focused on who the most appropriate parties are to provide this assurance: accountants or other independent professionals. This inquiry into stakeholder expectations was justified by the lack of empirical evidence existing on this issue (Wallage 2000).

The second part required each participant to examine two (of the five) randomly assigned assurance reports (8) and complete a questionnaire on their perceptions. The participants were asked to answer ten questions consisting of bipolar adjectival statements, separated by a seven-point scale on which to anchor responses. This method is a derivative of the semantic differential technique developed by Osgood et al. (1957) to measure the connotative meaning of a concept. The technique has been adapted by accounting researchers seeking to ascertain the "meaning" users attribute to various assurance reports (Gay et al. 1998; Roebuck et al. 2000). The questions were drawn from prior studies on the expectations gap related to either the elements or nature of the report. The elements examined were purpose, work performed, judgment required of the assurance provider, and the likelihood of the assurance provider discovering a material misstatement or significant omission. The questions regarding nature of the report focused on whether it provided too little or too much information, the degree to which it was understandable, whether it conveyed the degree of assurance required by users, and whether it enhanced the credibility of the environmental information. The final issue examined whether users would read such a report if it was provided to them.

The main question of interest for this study asked the participants to indicate the level of assurance they perceived from the report on a seven-point scale, anchored from zero (1) to absolute (7). Responses to this question constituted the main dependent variable of interest. Prior studies use similar measures to gauge perceived assurance levels. (See for example, Bartlett 1991; Monroe and Woodliff 1994; Gay et al. 1998; Roebuck et al. 2000.)

The final section asked them to indicate their familiarity with annual reports, environmental reports, and assurance reports, as well as to provide various demographic details.

Participants

Shareholders were used as participants in this study because they represent one of the main user groups of assurance reports. They are an important group to study when investigating communication forms that are understandable to the broadest cross-section of the public. The population surveyed consisted of the 2,850 New South Wales members of the Australian Shareholders' Association (ASA). Australia has one of the highest levels of retail share ownership in the world, (9) and the ASA is the primary shareholder representation body in the country. It represents a significant user group of assurance reports and characterizes the heterogeneous readership at which assurance reports are aimed. Information obtained from the demographic data included with the questionnaire indicated that 66 percent of the respondents were familiar with company annual reports, 49.3 percent were familiar with audit reports, and 28.1 percent were familiar with corporate environmental disclosures. (10)

RESULTS

Descriptive Results

The research instrument was mailed to participants, together with a letter of endorsement from the executive officer of the ASA. The overall response rate was 28.5 percent (812 responses). (11) Twenty of these responses were unusable (refer to footnote a in Table 2), thus 792 responses (27.8 percent) were available for the subsequent analysis (Panel A of Table 2). Panel B of Table 2 shows the numbers of responses received in relation to each of the ten combinations of reports. Response rates in each cell ranged from 23.2 percent to 35.8 percent. A Chi-square test of proportions (p = 0.254) indicated there was no significant difference in response rates across the ten possible combinations of two reports.

Table 3 summarizes the demographic information for the respondents. The average level of work experience for this group was 36.93 years, with 81.6 percent possessing a tertiary qualification. The proportion of respondents who were accountants or were otherwise involved in account preparation was 15.5 percent. (12) The demographic profile of the respondents (age, gender, qualifications) is consistent with the overall demographics of ASA members as reported in the most recent ASA Membership Survey, conducted in 1998.

An analysis of the responses to the ten questions on the elements and nature of the report revealed no significant differences between the aggregate responses to the five reports on these dimensions. Of note, in relation to the general nature of the assurance report, respondents felt the format was understandable (with a mean response of 3.10 on a seven-point scale). This may be attributable partly to the familiar structure of the reports examined that were closely linked to existing reporting forms. In terms of information content, a mean response of 3.17 suggests that users believe these reports convey too little information.

Perceptions of Usefulness of Assurance on Environmental Information

The majority of respondents indicated that environmental performance was a subject matter on which shareholders require assurance. Seventy-five percent of respondents felt that shareholders do require assurance on this type of information, checking one of the three categories closest to this statement. In contrast, only 4.7 percent of respondents to this question felt that assurance was not required by shareholders in relation to environmental information, anchoring their response on this extreme of the scale. However, there was overwhelming support for the statement that other independent professionals (for example, environmental engineers) were most appropriate for this type of assurance service, with 53.2 percent of responses being on this extreme of the scale (i.e., a response of 7). In all, 86.2 percent of all respondents felt that other independent parties were best suited to provide this service, with only 5.1 percent of respondents believing that accountants were suitable (8.7 percent indicated a neutral response). It is apparent that environmental assurance is a product demanded by the market. However, the accounting profession needs to convince users of its ability and expertise in this area in order to be perceived as a credible and reputable provider of such assurance services.

Test of Research Questions

The research utilized an incomplete block design (Kirk 1968), where each participant was presented with two of the five report types. The repeated-measures aspect of the design yielded "paired" observations for each participant. One-way ANOVAs were conducted for the responses to each of the five reports to test whether this repeated-measures design introduced any bias, i.e., whether the assurance perceived for a particular report was associated with which particular other report the participant saw. All five ANOVAs yielded insignificant results, indicating that the particular combination of reports a respondent received did not affect their perceptions of each report. Since the assurance perceived by a respondent is considered to be independent of the other report presented, the analysis below pools the responses received for each report. Between-group differences in perceived assurance for the five reports are therefore examined, with the two responses obtained from each participant considered as independent observations.

Test of RQ1

RQ1 asked if the moderate level assurance reports are perceived as providing lower levels of assurance than the high assurance report. Table 4 presents the results of the test of the research question. Panel A provides a descriptive summary of the responses to each report with regard to the dependent variable of interest: the assurance perceived. A one-way ANOVA revealed that significant (p = 0.000) differences existed between the mean assurance perceived from the five reports. To identify the source of these differences, post hoc multiple comparisons were undertaken.

Given the inclusion of the high assurance control group (Report 5), a Dunnett two-tailed test (13) was used for comparison to each of the moderate assurance reports (Keppel 1982). The results of the pairwise comparisons (Panel C of Table 4) indicate that the assurance perceived from the high assurance report (Report 5) (14) was significantly higher than Reports 2, 3, and 4 (negative assurance, positive assurance, and positive assurance with a limitations paragraph, respectively) as expected. However, the difference between the assurance perceived from the high assurance report and that inferred from the opinion on procedures paragraph (Report 1) was only marginally significant (p = 0.087) and not significant according to the results of other tests. (15) Since this form of reporting was identified as the one used most widely in practice, this suggests that users are attributing a higher level of assurance to this format of opinion than the profession intends. However, for the other moderate assurance opinion formats, it is clear that users are inferring a level of assurance that is significantly lower than that conveyed by a high level of assurance report.

Test of RQ2

RQ2 asked whether the alternative moderate assurance report formats are perceived to provide significantly different levels of assurance. To test this, further post hoc comparisons were conducted. Panel C of Table 4 shows that while most of the pairwise comparisons revealed no significant differences, Report 1 (opinion on procedures) was found to convey a significantly higher perceived level of assurance (p = 0.1301) than Report 4 (statement of positive assurance with a limitations paragraph). No significant differences were found between the means of Reports 2, 3, and 4, indicating that these variations in wording have little effect on the assurance inferred by users. (16)

DISCUSSION AND CONCLUSION

The primary focus of this research was to investigate whether alternative report formats used for reporting moderate assurance affect user perceptions of the level of assurance. The results show that, in general, the moderate assurance reports being used for environmental reporting engagements are conveying a lower level of assurance than the corresponding high assurance report. For three of the moderate assurance reports examined (negative assurance, positive assurance, and positive assurance together with an inherent limitations paragraph), users perceived a significantly lower mean level of assurance than they did for the high assurance report. This suggests that there are two distinct levels of assurance that are being communicated by current reporting formats. This provides some reassurance to standard setters and practitioners involved in engagements based on the current international framework for assurance engagements.

The moderate assurance report format that uses an opinion on procedures, and which is similar to the agreed-upon procedures reports in the U.S., was not found to be significantly different from the high assurance report. A possible explanation for this is that the opinion on procedures format may imply assurance on a series of stated objectives. The user may therefore interpret a higher level of assurance than would be provided by a general statement of assurance.

Further analysis revealed no significant differences between user perceptions relating to the other three moderate assurance formats. The inclusion of a separate paragraph of inherent limitations in the opinion was found to not decrease the level of assurance perceived by users. Perhaps this reflects that the respondents were already aware of these inherent limitations for environmental assurance engagements, making this aspect of the opinion "redundant" in the terms of Shannon and Weaver (1949).

Interestingly, there was no difference between the assurance inferred from an expression of negative assurance compared to a statement of positive assurance. This is despite arguments from several groups against the use of the positive format of opinion for expressing levels of assurance less than high. (17) It shows that when combined with other aspects associated with a lower level of assurance (e.g., subject matter, description of work effort), a positive form of opinion can be an effective mechanism for providing less than high assurance. This provides evidence that users do not necessarily use the positive form of opinion as a symbol for high assurance, i.e., it may also convey moderate assurance in specific circumstances. Further, the appropriate report format for moderate assurance is not necessarily restricted to the negative form of opinion.

The findings of this study have implications for practitioners and standard setters. For practitioners involved in engagements providing a moderate level of assurance, it provides guidance in formulating the appropriate wording for assurance reports. Being aware of the assurance levels conveyed by various types of opinion will allow the profession to better communicate the appropriate level of assurance to users. For standard setters, information about the meaning that users attribute to alternatively worded opinions can inform the development of principles on the reporting forms appropriate for various engagements. The results suggest that some caution should be exercised with regard to the "opinions on procedures" format, but that standard setters should not be as concerned with whether the moderate assurance is expressed in positive or negative form, or whether a limitations paragraph regarding the subject matter be included. Regardless of the form, all of these reports, with their description of limited procedures undertaken and statements of reduced reliance, were perceived by shareholders to communicate the same level of assurance, and a level of assurance significantly lower than that communicated by high assurance reports.

The findings must be considered in light of the following limitations. First, this research examined only whether differences in perceived assurance were appropriately communicated. It did not examine whether these perceptions accurately reflect the difference between high and moderate assurance. The high assurance report format scored 4.27 on a seven-point scale, while the moderate assurance report formats of negative assurance, positive assurance, and positive assurance with a limitations paragraph scored between 3.45 and 3.70 on the seven-point scale. The research cannot identify, for the subject matter and description of work effort, whether these are appropriate scores on such a scale to reflect high and moderate assurance. Second, the research cannot identify whether these differences are significant enough to translate into action and impact on report users' decision-making processes. We can only emphasize that the report formats used in this study are those that are commonly used in practice to communicate high and moderate assurance and are therefore intended to have resource implications by communicating a different level of assurance in the subject matter obtained by the assurance provider. Third, it is not known whether the results of this study are generalizable to "other" subject matter. In particular, it may be that the inherent limitations of providing assurance on environmental information are well known, and this may be the reason why the average level of assurance for the high assurance report format scored only 4.27 on a seven-point scale, and why the inherent limitations paragraph for this form of assurance engagement was perceived to have no effect of assurance provided.

Several avenues of future research arise from the conclusions of this study. The investigation could be extended to users of environmental assurance information such as institutional investors, environmentalists, and representatives of government agencies. Another fruitful area of study would be to examine the perceptions of the preparers of such reports to determine if their interpretations of the assurance intended to be conveyed by alternative opinion formats are aligned with those of users. Future researchers could also examine elements of the assurance report, other than the opinion paragraph, that may affect the assurance perceived. A greater understanding of these issues will enable the profession to adapt its communications in order to minimize the expectations gap and effectively convey the desired level of assurance.

APPENDIX

EXAMPLE OF A MODERATE ASSURANCE REPORT

INDEPENDENT REVIEW REPORT

We have reviewed the accompanying Environmental Performance Report 20XX ('the Report') of ABC Company. The responsibility for the accuracy, completeness and adequacy of the statements made in the Report lies exclusively with ABC's management. Our responsibility is to provide a moderate level of assurance on the Report based on our review.

Purpose and scope

The objective of our review is to provide stakeholders with an independent opinion on whether the Report presents a sound, balanced and objective view of ABC's environmental performance. This involved examining:

* whether the Report presents the significant and relevant environmental impacts in relation to important stakeholder groups

* whether the information in the Report has been presented in accordance with (specify environmental guidelines)

* whether the procedures for gathering and processing environmental information have been planned and implemented in an appropriate way.

The scope of this engagement is not as extensive as that of an audit and accordingly, does not provide the user of the report with the same level of assurance in respect of the information in the Report and ABC's underlying environmental performance.

Procedures performed

Our work has been conducted in accordance with assurance standards generally accepted in (name of country). Our work has included, based on an assessment of materiality and risk:

* reviewing the reasonableness of the procedures for collecting and collating the information contained in the Report

* discussions with and inquiries of, Company management and staff who have responsibilities for the presentation of the Report

* examining, on a test basis, evidence supporting the quantitative data presented

* comparing the information presented in the Report with the respective supporting documents and testing the accuracy of the calculations.

We believe that our review provides a reasonable basis for our opinion.

Opinion

[The different opinions used to create the four versions of moderate assurance reports are outlined in Table 1.]

PQR Auditors

Date/Address

TABLE 1
Alternative Report Formats in Sample Assurance Reports Developed

Report Formal             Wording

Moderate Assurance

Opinion on procedures     As a result of the procedures performed, it
                          is our opinion that the Report presents the
                          significant and relevant environmental
                          impacts in relation to important stakeholder
                          groups. Furthermore, in our opinion, the
                          information presented in the Report has been
                          presented in accordance with (specify
                          environmental guidelines). The procedures
                          for gathering and processing environmental
                          information have, in our opinion, been
                          planned and implemented in an appropriate
                          way.

Negative assurance        As a result of the procedures performed,
                          nothing has come to our attention that
                          causes us to believe that the Report does
                          not present, in all material respects, a
                          sound, balanced, and objective view of ABC's
                          environmental performance.

Positive assurance        As a result of the procedures performed,
                          it is our opinion that the Report fairly
                          presents, in all material respects, a sound,
                          balanced, and objective view of ABC's
                          environmental performance.

Positive assurance with   As a result of the procedures performed, it
a limitations paragraph   is our opinion that the Report fairly
                          presents, in all material respects, a sound,
                          balanced, and objective view of ABC's
                          environmental performance.

                          A significant proportion of environmental
                          information is qualitative in nature. Such
                          information often requires judgments that
                          are subjective, complex, or may require
                          assumptions about the effects of future
                          events. Given the subject matter of the
                          Report and the extent of the work per formed
                          on the engagement, we are not in a position
                          to express a high level of assurance.

Control Group

High assurance            As a result of the procedures performed, we
                          conclude, with a high level of assurance,
                          that the information in the Report conforms,
                          in all material respects, with (specify
                          environmental guidelines) and presents a
                          sound, balanced, and objective view of ABC's
                          environmental performance.

TABLE 2
Response Statistics

Panel A: Overall Response Rate

Treatment Group (b)        Number     %

Total mailed                 2850   100.0
- returned letter             (4)   (0.1)
Responses received            812   28.5
- nonusable (a)              (20)   (0.7)
Usable responses              792   27.8

Panel B: Usable Responses by Cell

Treatment Group (b)        Number   % (c)

Reports 1 and 2 (d)            84    29.5
Reports 1 and 3                69    24.2
Reports 1 and 4                74    26.0
Reports 1 and 5                66    23.2
Reports 2 and 3                83    29.1
Reports 2 and 4                76    26.7
Reports 2 and 5                79    27.7
Reports 3 and 4               102    35.8
Reports 3 and 5                83    29.1
Reports 4 and 5                76    26.7
                              792

(a) A number of respondents returned the questionnaire incomplete,
citing time pressure and health factors as reasons why they could
not complete the research instrument. Some returned the research
instrument accompanied by comments that they believed accountants
were not well placed to perform such engagements and therefore felt
that answering questions on the reports would be meaningless.

(b) This refers to the combination of reports that each participant
received.

(c) 285 questionnaires were mailed to each treatment group. This
column indicates the proportion of the 285 research instruments
that were usable.

(d) Within each group, half the respondents received the two reports
in reverse order to counterbalance any order effects. A series of
independent sample t-tests were conducted on the two respondent
subgroups in each of the ten groups. These indicated there was no
interaction between report format and ordinal position. Since there
were no significant differences in subject response based on the
sequence in which they saw the two reports, the two subgroups were
considered as one cell for the purposes of subsequent analysis.

TABLE 3
Demographic Profile of Respondents

Highest formal
qualification                    n        %

Secondary school                127     16.0
Tertiary qualification (a)      647     81.6
Undisclosed                      18      2.4
                                       100.0

Occupation                       n        %

Accountant                       40      5.0
Others involved in               82     10.4
 accounts preparation (b)
Professionals                   155     19.6
Other                           121     15.3
Retired                         377     47.6
Undisclosed                      17      2.1
                                       100.0

Work experience               Mean    (s.d.)

Number of years               36.93     11.3

Age                              n        %

Below 35                         10      1.3
35-45                            44      5.6
45-55                           143     18.1
55-65                           230     29.0
Above 65                        352     44.4
Undisclosed                      13      1.6
                                       100.0

Gender                           n        %

Male                            605     76.4
Female                          136     17.2
Undisclosed                      51      6.4
                                       100.0

(a) Tertiary qualification includes diplomas awarded from
the tertiary and further education (TAFE) sector, university
(college) degrees, and postgraduate qualifications.

(b) These include corporate executives, company secretaries,
and company directors--occupations that involve familiarity
with financial statements and assurance reports.

TABLE 4
Effect of Report Type (Moderate and High Assurance)
on Assurance Perceived

Panel A: Descriptive Statistics

                                 Report 1 (a)     Report 2     Report 3
                                 (n = 278) (c)    (n = 306)   (n = 320)

The assurance report conveys
a zero/absolute level of
assurance. (b)

Mean                                 3.96           3.63         3.70
Std deviation                        1.53           1.63         1.61

                                   Report 4       Report 5
                                   (n = 308)      (n = 289)

The assurance report conveys
a zero/absolute level of
assurance. (b)

Mean                                 3.45           4.26
Std deviation                        1.53           1.53

Panel B: Analysis of Variance

                                    Sum of                        Mean
Source of variation                 Squares          df          Square

Between Groups                      115.703           4          28.926
Within Groups                      3674.859         1496          2.456
Total                              3790.562         1500

                                                Significance
Source of variation                    F            of F

Between Groups                      11.775          0.000
Within Groups
Total

Panel C: Multiple Comparison
(Dunnett) Tests

Moderate Assurance versus            Mean
High Assurance Report             Difference    Significance

Report 1 vs. Report 5                -0.29         0.087
Report 2 vs. Report 5                -0.62         0.000 *
Report 3 vs. Report 5                -0.55         0.000 *
Report 4 vs. Report 5                -0.80         0.000 *

Comparisons within the Moderate      Mean
Assurance Reports                 Difference    Significance

Report 1 vs. Report 2                 0.33          0.111
Report 1 vs. Report 3                 0.26          0.425
Report 1 vs. Report 4                 0.51          0.001 *
Report 2 vs. Report 3                -0.07          1.000
Report 2 vs. Report 4                 0.18          1.000
Report 3 vs. Report 4                 0.25          0.471

* Significant at the 5 percent level.

(a) The report formats are as follows:

Report 1: opinion on procedures
Report 2: statement of negative assurance
Report 3: statement of positive assurance
Report 4: statement of positive assurance with a limitations
paragraph
Report 5: statement conveying a high level of assurance

(b) Responses were anchored on a seven-point scale. For the purpose
of the analysis, a zero level of assurance was assigned a value of
1 while an absolute level of assurance was assigned a value of 7.

(c) The maximum total number of reports was 1,584 (792 respondents
each evaluating two report formats). Of these, 83 completed the
question on level of assurance conveyed by the report for only one
report format, resulting in the 1,501 responses reported in this table.

The authors appreciate the assistance of Michael Nugent and the members and executive of the Australian Shareholders' Association, as well as the comments of Peter Luckett, Ken Trotman, and Arnold Wright. The financial support of The Accounting and Assurance Services Centre at The University of New South Wales is greatly appreciated.

(1) The International Standards on Assurance Engagements are comparable to the U.S. Attestation Standards.

(2) The Global Reporting Initiative is a joint undertaking by the Coalition for Environmentally Responsible Economies (CERES) and the United Nations Environment Programme (UNEP). Its aim is to develop and disseminate internationally applicable sustainability reporting guidelines for voluntary use by organizations reporting on the economic, environmental, and social dimensions of their activities.

(3) The one country where there was a low level of providing assurance on environmental and sustainability reports was the U.S. While the environmental and sustainability reporting was commonplace in the U.S. (36 of the top 100 companies), providing assurance on such reports was rare (only one instance).

(4) A review of moderate assurance reports relating to other subject matter (such as internal control, intellectual capital, and prospective financial information) was also undertaken to ensure that the issues raised in the analysis of the environmental assurance reports were applicable in these types of report. This assessment confirmed that, in general, these reports were broadly similar in form to the environmental assurance reports analyzed.

(5) Other environmental assurance reporting was also identified and considered. These included (among others) the Novo Group Environmental and Social Report 2000, ING in Society Report 2000, Statoirs HSE 2000 report, and the Shell Report 2000. This was done to maximize the variety in the reports examined and ensure that potentially effective forms of reporting were not overlooked. None of these assurance reports claimed to communicate a moderate level of assurance.

(6) Agreed-upon procedures engagements are expressly excluded as an assurance engagement within the international framework (ISAE No. 100, para. 6), although they are included as part of the attestation standards in the U.S. (AT 201). In both jurisdictions, they contain no opinion, and factually report procedures and findings.

(7) The full text of an environmental report was not presented with the assurance reports because this would have unnecessarily increased the length of the questionnaire, and distracted the participants from the focus of the study. Prior studies relating to financial statement audits have used this approach (Yardley 1989; Bartlett 1991) and reported that users did not have difficulty completing the task in a context-free frame of reference.

(8) The within-subjects design creates little risk of demand effects in this instance, as it would have been almost impossible for subjects to guess the nature of the hypotheses. Potential order effects were addressed by counterbalancing the sequence in which the reports were presented. Further, the possibility of report order playing a confounding role was considered in the data analysis and found to be of no effect.

(9) According to the results of a survey conducted by the Australian Stock Exchange (2000), 41 percent of Australians held a direct investment in shares. This level of ownership was higher than those in other countries studied: United Kingdom (30 percent), United States (32 percent), New Zealand (31 percent), Canada (26 percent), and Germany (13 percent).

(10) Participants ranked their familiarity with such information on a seven-point scale, with a response of 1 indicating that they were "not at all familiar" and 7 indicating that they were "very familiar." For the purposes of calculating these percentages, participants were taken to be "familiar" if they anchored their response between 5 and 7.

(11) Despite the overall acceptable response rate for the method employed, a recognized limitation of information gathered through surveys is the possibility of nonresponse bias. This was considered in the analysis by comparing the responses of participants who replied within the ten days given in the questionnaire to the responses of the later respondents. No statistically significant difference was found between "early" and "late" respondents, suggesting that a nonresponse bias is unlikely and the responses received are a representative sample of the population surveyed.

(12) Further analysis of demographic details revealed that there were no statistically significant differences in respondent characteristics between the five report groups, ruling out the possibility that respondent perceptions of the reports were driven by factors such as their occupation. This was further tested by partitioning respondents based on their self-rated familiarity with assurance reports. There were no significant changes regarding the results in relation to the research questions. Separate analyses were also conducted on respondent groups based on whether they stated they would read such an assurance report if it was provided to them. Despite these two groups having ostensibly different levels of motivation to read the reports, no systematic differences were found in their responses.

(13) Dunnett t-tests treat one group as a control, and compare all other groups against it.

(14) The level of assurance for the high assurance report was 4.26 on a seven-point Likert scale tall moderate assurance reports had a mean of less than 4). This mean may be considered low, indicating that the level of perceived assurance for environmental reporting engagements may not be very high, given the subject matter, suitability of criteria, or description of work effort, which was contained in our high assurance environmental report.

(15) These comparisons were also conducted using alternative tests, including Bonferroni (Kirk 1968), Sidak, and the Tukey test (Keppel 1982). The results were qualitatively similar to those reported in the main analysis. In comparing reports and 5, Bonferroni yielded a significance level of 0.267, Sidak 0.237, and Tukey 0.173.

(16) To test the robustness of the findings, the analysis was repeated using nonparametric tests. The results of Kruskal-Wallis tests indicated effects qualitatively similar to those reported in the main analysis.

(17) This was revealed by comments received through the exposure process to the Exposure Draft, Assurance Engagements (IAPC 1999). A similar view was earlier advanced by Milburn (1980).

REFERENCES

Arnold, V., J. C. Lampe, J. J. Maselli, and S. G. Sutton. 2000. An analysis of the market for systems reliability assurance services. Journal of Information Systems 14 (Supplement): 65-82.

Australian Stock Exchange. 2000. 2000 Australian Shareownership Study. Sydney, Australia: Australian Stock Exchange Limited.

Bailey, K. E., J. H. Bylinski, and M. D. Shields. 1983. Audit report wording changes. Journal of Accounting Research 21: 355-370.

Barthes, R. 1968. Elements of Semiology. London, U.K.: Cape.

Bartlett, R. W. 1991. Perceived levels of assurance of bankers and CPAs: A comparison. Advances in Accounting 9: 205-225.

Brown, T., D. Hatherly, and, J. Innes. 1993. The review report: An empirical investigation. Accounting and Business Research 24:11-18.

Elliott, R. K. 1998. Assurance services and the audit heritage. Auditing: A Journal of Practice & Theory 17 (Supplement): 1-7.

Elliott, R. K., and D. M. Pallais. 1997. Are you ready for new assurance services? Journal of Accountancy 183 (6): 47-51.

Federation des Experts Comptables Europeens (FEE). 2002. Providing Assurance on Sustainability Reports. Discussion paper, Brussels, Belgium.

Fiske, J. 1990. Introduction to Communication Studies. Second edition. London, U.K.: Routledge.

Gay, G. E., P. Schelluch, and A. Baines. 1998. Perceptions of messages conveyed by review and audit reports. Accounting, Auditing and Accountability Journal 11 (4): 472-494.

Global Reporting Initiative. 2000. Sustainability Reporting Guidelines on Economic, Environmental and Social Performance. Boston, MA. Available at http://www.globalreporting.org.

Hatherly D., J. Innes, and T. Brown. 1991. The expanded audit report--An empirical investigation. Accounting and Business Research 21:311-319.

Houghton, K. A., and W. F. Messier. 1991. The wording of audit reports: Its impact on the meaning of the message communicated. In Accounting, Communication and Monitoring, edited by S. Moriarty, 85-106. Norman, OK: University of Oklahoma.

Humphrey C., P. Moizer, and S. Turley. 1992. The audit expectations gap.--Plus ca change, plus c'est la meme chose? Critical Perspectives in Accounting 3 (2): 137-161.

Innes, J., T. Brown, and D. Hatherly. 1997. The expanded audit report--A research study within the development of SAS No. 600. Accounting, Auditing & Accountability Journal 10 (5): 702-717.

International Auditing Practices Committee (IAPC). 1999. Assurance Engagements. Exposure draft. New York, NY: International Federation of Accountants.

--. 2001a. Assurance Engagements on Environmental Reports. Draft standard. New York, NY: International Federation of Accountants.

--. 2001b. International Standard on Assurance Engagements. ISAE No. 100. New York, NY: International Federation of Accountants.

International Federation of Accountants (IFAC). 2002. The Determination and Communication of Levels of Assurance Other Than High. New York, NY: International Federation of Accountants.

Kelly, A. S., and L. C. Mohrweis. 1989. Bankers' and investors' perceptions of the auditor's role in financial statement reporting: The impact of SAS No. 58. Auditing: A Journal of Practice & Theory 9 (Fall): 87-97.

Keppel, G. 1982. Design and Analysis: A Researcher's Handbook. Second edition. Upper Saddle River, N J: Prentice Hall, Inc.

Kirk, R. E. 1968. Experimental Design: Procedures for the Behavioral Sciences. Belmont, CA: Wadsworth Publishing Company, Inc.

Korzybski, A. 1958. Science and Sanity. Fourth edition. Lakeville, CT: The International Non-Aristotelian Library Publishing Co.

KPMG. 2002. International Survey of Corporate Sustainability Reporting 2002. Amsterdam, The Netherlands: KPMG.

Milburn, J. A. 1980. Limited Audit Engagements and the Expression of Negative Assurance. Ontario, Canada: Canadian Institute of Chartered Accountants.

Monroe, G. S., and D. Woodliff. 1994. An empirical investigation of the audit expectation gap: Australian evidence. Accounting and Finance (May): 47-74.

Nair, R. D., and L. E. Rittenberg. 1987. Messages perceived from audit, review and compilation reports: Extension to more diverse groups. Auditing: A Journal of Practice & Theory 7 (Fall): 15-38.

Ogden, C., and I. A. Richards. 1949. The Meaning of Meaning. London, U.K.: Routledge & Kegan Paul.

Osgood, C. E., G. J. Suci, and P. H. Tannenbaum. 1957. The Measurement of Meaning. Urbana, IL: University of Illinois Press.

Pany, K., and C. H. Smith. 1982. Auditor association with quarterly financial information: An empirical test. Journal of Accounting Research 20: 472-481.

Pillsbury, C. M. 1985. Limited assurance engagements. Auditing: A Journal of Practice & Theory 4 (Spring): 63-79.

Roebuck, P., R. Simnett, and H. L. Ho. 2000. Understanding assurance service reports: A user perspective. Accounting and Finance 40 (3): 211-232.

Schandl, C. W. 1978. Theory of Auditing: Evaluation, Investigation and Judgment. Houston, TX: Scholars Book Co.

Shannon, C., and W. Weaver. 1949. The Mathematical Theory of Communication. Urbana, IL: University of Illinois Press.

Wallage, P. 2000. Assurance on sustainability reporting: An auditor's view. Auditing: A Journal of Practice & Theory 19 (Supplement): 53-65.

Yardley, J. A. 1989. Lenders' and CPAs' perceptions of the assurance provided by prescribed procedures.

Submitted: April 2002

Accepted: March 2003

Mahreen Hasan is an Associate Lecturer, Peter J. Roebuck is a Senior Lecturer, and Roger Simnett is a Professor, all at The University of New South Wales, Australia.