1
www.intosaijournal.org summer 2019 edition
The International Journal of Government Auditing is
published quarterly in Arabic, English, French, German
and Spanish on behalf of INTOSAI (International
Organization of Supreme Audit Institutions). The Journal,
which is an ofcial organ of INTOSAI, is dedicated to
the advancement of government auditing procedures
and techniques. Opinions and beliefs expressed are
those of individual contributors and do not necessarily
reect the views or policies of the organization.
The editors invite submissions of articles, special reports,
and news items, which should be sent to the editorial
ofces at:
U.S. Government Accountability Ofce
441 G Street, NW, Room 7814
Washington, D.C. 20548
U.S.A.
(Phone: 202-512-4707; Fax: 202-512-4021; E-mail:
intosaijournal@gao.gov).
Given the Journal’s use as a teaching tool, articles most
likely to be accepted are those that deal with pragmatic
aspects of public sector auditing. These include case
studies, ideas on new audit methodologies, or details
on audit training programs. Articles that deal primarily
with theory would not be appropriate. Submission
guidelines are located at http://intosaijournal.org/our-
submissions/.
The Journal is distributed electronically to INTOSAI
members and other interested parties at no cost. It is
also available electronically at www.intosaijournal.org
or www.intosai.org and by contacting the Journal at
intosaijournal@gao.gov.
Summer 2019
Vol. 46, No. 3
Board of Editors
Margit Kraker, President, Rechnungshof, Austria
Sylvain Ricard, Interim Auditor General, Canada
Nejib Gtari, Premier Président, Cour des Comptes, Tunisia
Gene L. Dodaro, Comptroller General, United States of America
Manuel E. Galindo Ballesteros, Comptroller General, Venezuela
President
James-Christian Blockwood (U.S.A.)
Vice President
Michael Hix (U.S.A.)
Editor
Heather Santos (U.S.A.)
Special Contributors
Kristie Conserve (U.S.A.)
Brighton Nyanga (Norway)
Allen Parker (Cook Islands)
Chris Stone (U.S.A.)
Tytti Yli-Viikari (Finland)
Laurel Voloder (U.S.A.)
Administration and Operations
Peter Knopes (U.S.A.)
Associate Editors
AFROSAI Secretariat
ARABOSAI Secretariat
ASOSAI Secretariat
CAROSAI Secretariat
EUROSAI Secretariat
OLACEFS Secretariat
PASAI Secretariat
INTOSAI General Secretariat
Ofce of the Auditor General, Canada
Ofce of the Auditor General, Tunisia
Ofce of the Auditor General, Venezuela
U.S. Government Accountability Ofce
TH
SSUE
S
Editorial
News in Brief
Special Contribution
"Turning a Bad News Headline into a Good News Story"—How the
Cook Islands Audit Ofce Successfully Engaged Stakeholders to
Address Audit Effectiveness, Efciency
Feature Stories
Quantitative and Qualitative Methods: The Optimal Combination for
Effective Audits
Determinants Affecting Audit Quality
Improving Quality Through Effective Management Frameworks
A Model for Examining Audit Effectiveness—Nationally and Globally
Inside INTOSAI
Spotlight on Capacity Building
4
7
9
12
21
30
by Mr. Karol Mitrík, President, Supreme Audit Institution of
Slovakia
A king placed a large boulder on the road then hid and
watched to see if anyone would remove the huge rock. A
few royal, wealthy merchants and courtiers encountering
the obstacle simply bypassed it. Many whose paths were
impeded by the boulder loudly blamed the king for not
keeping the roads clean. Yet, nobody did anything to get
the boulder out of the way.
Along came a peasant carrying a hefty load of vegetables.
When he came to the boulder, he placed his load on the
ground and tried to push the enormous rock to the road’s
edge. After a great push and much strain, he nally
succeeded.
As he returned to retrieve his load of vegetables, he noticed
a pouch lying on the ground where the boulder originally
rested. Inside the pouch—gold coins and a message from
the king conveying that the coins were intended for the
person who removed the boulder from the path.
The moral of the story is that every obstacle is an opportunity
to improve our situation.
ACHIEVING AUDIT QUALITY: A PATH LINED
WITH OBSTACLES, AS WELL AS TREASURES
International Journal of Government Auditing—Summer 2019
4
EDITORIAL
"Quality is a philosophy, a key
priority and the cornerstone of SAI
work. It is a component of strategy,
culture, rules and practices, and,
given these characteristics, quality is
(as it should be) rst and foremost."
Ensuring the highest audit quality is a permanent, endless
road lined with seemingly never-ending obstacles.
Quality is a philosophy, a key priority and the cornerstone
of Supreme Audit Institution (SAI) work. It is a component
of strategy, culture, rules and practices, and, given these
characteristics, quality is rst and
foremost.
Achieving audit quality requires
an appropriate legal framework
and institutional independence. The
Supreme Audit Ofce (SAO) of the
Slovak Republic (SR), our nation’s
SAI, possesses these prerequisites.
As such, we are committed to
fullling our mandate and mission while performing to
the utmost level and seeking the highest quality and
professionalism possible.
Only then can we be a credible institution that promotes
useful public sector changes. Only then can we truly bring
value and benets to the lives of citizens.
Audit Quality—Our Path Lined with Landmark Achievements
In our pursuit of the highest quality, we will, inevitably,
encounter obstacles. Along the way, as we overcome the
challenges put before us, we will uncover great treasures,
particularly in the form of improved public resource
management.
Over the past four years, SAI Slovakia has made signicant
progress in ensuring audit quality, and it must be stressed
that these improvements have been possible, in large part,
through our high-performing staff and exemplary work
in providing analytic, strategic, communicative and legal
support.
We have achieved some important milestones in our positive
audit quality shift, particularly in human resource expertise
and professionalism; audit planning; analysis and strategy;
methodology; audit quality assessment; and communications
(internal and external).
Human Resource Expertise and Professionalism
In every organization, people are a decisive factor in
organizational progress and success. SAI Slovakia pays
considerable attention to staff professional knowledge and
skills through various education and training programs—
nationally and internationally.
At present, we are running a national
project, "Building and Developing
Professional Capacities to Improve
SAO SR Audit Activities Quality," which
is aimed at staff development and
quality improvement.
The program, co-nanced from
European sources and led by top
experts from Slovakia and abroad, is
designed for analysts, as well as nancial and performance
audit specialists and includes professional seminars and
internships, Information Technology training courses and
specialized English language instruction.
Audit Planning
Long-term strategic goals emphasize new approaches to
audit topics and risk- and value-based audits. We endeavor
to apply the “Value for Money” concept in ex-post audits
of public nance, seek to transform audit philosophies and
focus on public policy assessments.
Analysis and Strategy
We created a strong analytical team responsible for risk
analysis, applying new strategic approaches and supporting
audits and auditors alike.
Methodology
Updating our approaches, implementing the International
Standards of Supreme Audit Institutions (ISSAIs) and ensuring
audit procedure professionalization are some of the
mechanisms employed to enhance our audit methodology.
Audit Quality Assessment
We introduced new procedures, improved, augmented
transversal quality assessments and established senates to
deliberate audit reports.
www.intosaijournal.org
5
EDITORIAL
"In our pursuit of the highest quality,
we will, inevitably, encounter
obstacles. Along the way, as we
overcome the challenges put before us,
we will uncover great treasures."
International Journal of Government Auditing—Summer 2019
6
EDITORIAL
Internal and External Communication
Today, there are new demands on what we share and how
we share it. To exceed these demands, we have modied
our communications strategy, strengthened our specialized
communications and public relations departments and have
created new staff positions.
We are now applying new approaches
to stakeholder engagement, which
includes leveraging our website and
social networks and developing print,
radio and television products for
broadcast media.
Our ofce is the rst state institution in
Slovakia that has chosen such a broadly
structured approach to informing stakeholders (at all levels)
about who we are, what we do, and the key activities of
interest.
Audit Quality—Our Path Toward Continuous Improvement
In addition to our landmark achievements, we continuously
pursue effective tools and support aimed at improving
the quality of our business, including self-assessments,
independent peer reviews, and collaboration with the
International Organization of Supreme Audit Institutions
(INTOSAI) and its regions.
SAI PMF Self-Assessment
Our ofce champions self-assessments through the SAI
Performance Measurement Framework (PMF) methodology.
With INTOSAI Development Initiative (IDI) assistance, the
SAO SR conducted a pilot self-assessment in 2013 illustrating
how the SAI PMF is a useful tool in identifying organizational
performance strengths and weaknesses, as well as their root
causes.
In 2018, we repeated this process. This time, we veried
the results through an external group of peers, which
demonstrates our willingness to be an open, transparent
entity seeking to constantly improve. The self-assessment
resulted in recommendations the SAO SR is subsequently
implementing.
Independent Peer Review
We also underwent an independent, international peer
review in 2010-2011. Led by the United Kingdom National
Audit Ofce, the evaluation, which
included experts from the SAIs
of Estonia, Poland and Slovenia,
focused on audit activities and
audit quality, human resource
development and public relations
efforts.
We will be subject to an additional
independent peer review between
2019 and 2020, where experts from the U.S. Government
Accountability Ofce (lead) and the SAIs of Finland,
Hungary and Poland will assess our planning processes,
audit quality, transparency and communication.
As chair of the INTOSAI Subcommittee on Peer Reviews, we
recognize the importance of these assessments and remain
steadfast in our commitment to quality.
Support within the INTOSAI Family
INTOSAI and its ability to bring independent national
audit authorities together—to collaborate, cooperate and
support one another—is admirable and, dare I say, unique.
The focus on exchanging ideas, experiences and best
practices makes audit quality all the more important and
gives us a true appreciation for the brilliantly applicable
INTOSAI motto.
"Mutual Experience Benets All"—it certainly does!
"INTOSAI and its ability to
bring independent national audit
authorities together—to collaborate,
cooperate support one another—is
admirable and, dare I say, unique."
www.intosaijournal.org
7
NEWS IN BRIEF
NEWS FROM
AROUND THE WORLD
Click on the labeled buttons to discover the latest SAI news from that nation
PERU
MALTA
KUWAIT
EGYPT
SERBIA
BHUTAN
MONGOLIA
MYANMAR
UAE
PAKISTAN
CYPRUS
ALGERIA
GUATEMALA
LITHUANIA
SPECIAL THEME: AUDIT QUALITY AND EFFECTIVENESS
SPECIAL CONTRIBUTION
"Turning a Bad News Headline into a Good News Story"—How the Cook Islands Audit
Ofce Successfully Engaged Stakeholders to Address Audit Effectiveness, Efciency
FEATURE STORIES
Quantitative and Qualitative Methods: The Optimal Combination for Effective Audits
Determinants Affecting Audit Quality
Improving Quality Through Effective Management Frameworks
A Model for Examining Audit Effectiveness—Nationally and Globally
SPECIAL THEME: AUDIT QUALITY AND EFFECTIVENESS
www.intosaijournal.org
9
SPECIAL CONTRIBUTION
TURNING A BAD
NEWS HEADLINE
INTO A GOOD
NEWS STORY
How the Cook Islands Audit Ofce
Successfully Engaged Stakeholders to
Address Audit Effectiveness, Efciency
It was the type of newspaper coverage most
Supreme Audit Institutions (SAI) would dread.
“Unaudited Accounts Cause Issues,” declared a
September 2018 Cook Island News headline.
The article, which went on to detail how the
Cook Islands has a backlog of unaudited
public accounts that could "potentially give the
country a bad nancial rating,” was sparked by
a statement during a national Public Accounts
Committee (PAC) induction workshop.
The statement called into question the Cook
Islands Audit Ofce's (CIAO) role in reporting to
the PAC and its ability to hold the government
accountable and piqued stakeholder interest
(including the national media), who now wanted
to know, “Where are the reports? Why is there
a backlog?”
Quick on the heels of the original news story,
CIAO Director of Audit, Allen Parker, addressed
these questions. In an article appearing in
the same news outlet four days later, Parker
explained how two technical auditing practices
were primarily behind the shortcomings.
Allen Parker, Director of Audit,
Cook Islands Audit Ofce
International Journal of Government Auditing—Summer 2019
10
SPECIAL CONTRIBUTION
First, the Cook Islands is one of few small countries in the
Pacic region to have adopted full accrual accounting
(most employ cash accounting or modied cash accounting
methods when preparing public accounts).
The second, and perhaps more far-reaching explanation,
stems from the consolidation of all government entities,
including State Owned Entities (SOE), into the whole of
government accounts to provide a more informative set of
Financial Statements of Government
(FSG) accounts.
This approach, which requires the CIAO
to individually audit nancial statements
of each government entity, SOE and
government parent prior to incorporation
into the government’s nancial statements,
proved quite challenging on a number of
fronts.
To begin with, parliament sittings (where reports are tabled)
had been irregular, and for three years (2014-2016),
roughly 65% of government entities were well behind the
recommended six-month time frame in providing nancial
statements to the audit ofce.
Parker claried that the CIAO could not perform the whole
of government account audit until 44 individual government
entities and three SOEs (47 in total) are completed. A
further challenge is the timely preparation and availability
of auditable consolidated nancial statements from the
Ministry of Finance and Economic Management (MFEM).
The CIAO is a member of the Pacic Association of Supreme
Audit Institutions (PASAI), and the PASAI Chief Executive has,
over the past few years, focused on monitoring backlogs of
FSG accounts, particularly for the smaller island states within
the region.
This emphasis coincides with PASAI’s strategic priorities
to achieve timely and effective audits; provide regional
capacity building programs; and deliver technical support
through cooperative regional approaches, such as the Sub-
Regional Audit Support (SAS) program.
PASAI, with assistance from the European Union, will supply
additional support over the next two years to the region’s
SAIs still facing hefty backlogs with designs on ensuring all
SAIs in the Pacic are up-to-date with auditing FSGs by
2024—an ambitious, but achievable, goal.
For most SAIs, the initial phase of auditing core government
entities to clear backlogs will be followed by a second
phase (auditing SOEs). For the CIAO, the two phases have
occurred simultaneously.
The CIAO has, in fact, achieved
timely and effective audits by
devoting its limited resources to
reducing the number of unaudited
nancial statements for government
entities and SOEs. The CIAO also
established a plan that included
completing the 2014-2016 FSG
statements by June 30, 2019. It is supported in this process
by New Zealand's Ofce of the Auditor General—the
CIAO's twinning partner.
That said, several key developments have emerged from this
very open, very visible engagement with stakeholders—
PAC support and contributions to audit effectiveness and
efciency; increased SAI visibility, transparency and
accountability; and a broader discussion on accounting
practices.
PAC Support and Contributions to
Audit Effectiveness and Efciency
Parker has always believed the MFEM proved itself willing
to share information on government spending. Legislative
changes will enhance this position. These changes include
ensuring audit reports are available to the public upon
submission to the Speaker, thus negating the need for
parliament to be in session.
The PAC has also passed a resolution enabling the
CIAO to identify agencies not in compliance with audit
requirements, and PAC Chairman, Tingika Elikana, believes
"Several key developments have
emerged from this very open and very
visible stakeholder engagement."
www.intosaijournal.org
11
SPECIAL CONTRIBUTION
timely information will lead to better-informed decisions
by the PAC and government and greater accountability for
all entities.
SAI Visibility, Transparency and Accountability
Through Media, Communication
The public debate allowed the CIAO to describe its work
and obstacles to efciency—to the government, the PAC
and the nation’s citizens. Proactive, clear and objective
messaging allowed the SAI to compellingly and concisely
clarify complex, and potentially inammatory, government
issues. This clarity and openness speaks to the CIAO’s
determination to be transparent and inclusive.
"A key part of our strategic plan includes stakeholder
engagement. While many of our objectives in this area
are in draft form, we have already commenced with
establishing a process to discuss our audit reports with the
PAC and, of course, the media."
Broader Discussion on Accounting Practices
This dialogue also allowed the CIAO to raise citizen
awareness about national accounting practices, as well
as FSG and the whole of government approaches to the
scrutiny of accounts.
The Cook Islands experience is similar to that of other SAIs.
The timely completion of accounts remains a national issue—
one critical to an audit organization's strategic planning
efforts, especially in developing and maintaining necessary
capacity, capability and resourcing to prepare staff for
challenges (such as transitioning to auditing accrual-based
and fully consolidated accounts).
For Parker, the dialogue has already led to demonstrable
improvements in auditing practices.
“It is satisfying to note that 90% of our audits were
completed within six months of receipt for the 2017 nancial
statements,” he noted.
For the Cook Islands, the discussion, hopefully, ceases the
notion that the country is threatened by poor nancial ratings.
For SAIs worldwide, the debate, perhaps, illustrates an
ability to overcome, and capitalize on, seemingly negative
news.
In this instance, a negative headline provided a stage
for proactive, positive stakeholder engagement, and the
ensuing inquiries allowed the SAI to be open about current
failures and equally open (and optimistic) about future
successes.
News Story Links
"Unaudited Accounts Raise Issues" by Rashneel Kumar, Cook
Island News, September 15, 2018.
"Audit Ofce Explains Delays" by Rashneel Kumar, Cook
Island News, September 19, 2018.
"PAC Crackdown on Ministries" by Rashneel Kumar, Cook
Island News, February 1, 2019.
Emphasis on Stakeholder Engagement, Audit Quality
Reected Across PASAI Region
The importance of stakeholder engagement, which is
vital to audit impact and quality, is reected across
the PASAI region in SAI communication strategies. As
recently as May 2019, Fiji's Auditor General, Ajay
Nand, engaged with local media to raise awareness
on the lack of action taken against government
departments, statutory bodies and councils (provincial
and municipal) that do not submit draft nancial
statements on time.
In the article, "Auditor General Calls for Sterner Action
on Non-Compliance," published by the Fiji Broadcasting
Commission, Nand demonstrates how employing
media can enhance transparency, accountability and
inclusion.
International Journal of Government Auditing—Summer 2019
12
FEATURE STORIES
The creation of the International Organization of Supreme
Audit Institutions (INTOSAI) Working Group on Big Data
speaks to the increased attention to enhanced data analytics.
While much consideration has focused on quantitative
techniques, this article highlights qualitative analysis, its
relevance and ability to complement quantitative methods
used in Supreme Audit Institution (SAI) work.
Though they contribute differently to an audit (broadly
dened to also include evaluation), quantitative and
qualitative methods share a common purpose—to produce
actionable information addressing the audit’s objective.
Research nds the difference is more a matter of degree
than type.
Qualitative methods, such as content review, interviews,
focus groups and case studies, generally seek to develop
an understanding and assessment of the audit’s conceptual
and behavioral context. As such, they score highly on
validity and inductive reasoning (moving from observation
to hypothesis).
Quantitative methods, including systematic data analyses, aim
for reliability and hypothesis testing. Thus, qualitative and
quantitative methods are best viewed as complements in the
audit process. Examining existing studies shows complementarity
occurring at three levels in that qualitative analysis (1) is
an essential preliminary to quantitative research, as it
enables a contextual description and understanding that is
instrumental in designing the appropriate audit scope; (2)
supplements quantitative work (and vice-versa) by adding
validation to empirical ndings; and (3) can explore areas
not amenable to quantitative research, such as internal
control processes within an agency or program.
Together, qualitative and quantitative methods form a range
of approaches that, when optimally combined, enable a
SAI to decipher concepts, content and data that may exist in
an audit situation. The optimal combination is case-specic,
as it depends upon facts that characterize each audit.
Assessing the right methodological mix demands expertise.
The U.S. Government Accountability Ofce (GAO) established
a team—Applied Research and Methods (ARM)—of about
135 specialists with advanced degrees in 14 disciplines across
the social, physical, and computing sciences who contribute
independent expertise in qualitative and quantitative methods
to GAO’s audit teams. ARM’s specialists work closely with GAO’s
analysts to combine qualitative and quantitative techniques in
by Oliver M. Richard, Chief Economist and Director, Center for
Economics, U.S. Government Accountability Ofce
The Optimal Combination
for Effective Audits
GAO reports. GAO’s report, “Social Security Disability:
Additional Measures and Evaluation Needed to Enhance
Accuracy and Consistency of Hearings Decisions,” provides
a good example of employing a combination of qualitative
and quantitative tactics.
Background
The U.S. Social Security Administration (SSA) manages
disability benet programs that cover roughly 16 million
Americans who receive an estimated $200 billion in benets
annually. For adult eligibility, a person must have a medically
determinable physical or mental impairment that (1) has
lasted (or is expected to last) for a continuous period of at
least one year or result in death, and (2) prevents the person
from engaging in any substantial gainful activity. The SSA
received over 2.5 million disability claims in scal year 2016.
When a person applies, SSA makes an
initial determination. Typically, when a
claim is denied, the claimant may request
a hearing before an Administrative
Law Judge (judge). In scal year 2016,
claimants appealed 698,000 decisions to
judges, who may deny or allow a claim at
a hearing. The judge’s allowance rate is
the percentage of claims for which a judge
allows benets.
Stakeholders noted a variance in allowance
rates across judges, and GAO was asked
by the U.S. Congress to examine the extent
to which allowance rates vary across
judges (and what factors are associated
with this variation); and the extent to which
SSA processes monitor accuracy and
consistency of hearings decisions.
The Analysis and Findings
GAO relied on qualitative methods to
better understand SSA’s process, such as conducting a
content review of applicable SSA policies and procedures;
interviewing SSA ofcials; holding semi-structured interviews
with regional chief judges; and attending hearings in selected
ofces (not generalizable) to conceptualize the hearings
process in practice.
GAO learned, in particular, that SSA randomly assigns cases
to a judge based on the claimant’s location. This practice
increased the chance that factors possibly explaining
differences in allowance rates across judges were similar
across cases and thereby identiable using a quantitative
analysis of allowance decisions.
Quantitatively, GAO compiled information from several
SSA administrative systems, federal personnel and payroll
systems, and economic data (spanning scal years 2007-
2015 that included an estimated 3.3 million claims records).
Using this data, an annual allowance rate was computed for
each judge.
GAO rst examined the allowance rate data to substantiate
concerns that motivated Congress’ request. Figure 1 shows
that allowance rates vary substantially across judges,
even as the average allowance rate across judges fell 15
percentage points from 2008 to 2015.
GAO engaged with SSA ofcials, who noted the variation is
not surprising due to case complexity and judges' decisional
independence.
SSA ofcials also cited an expansion in quality assurance
reviews and electronic data usage to monitor judges’
decisions, which should have helped lessen variation in
allowance rates across judges over time. These qualitative
insights raised the hypothesis that allowance rates could
vary across judges due to case dissimilarity.
www.intosaijournal.org
13
FEATURE STORIES
Figure 1: Variation in Social Security Administration (SSA) Disability Allowance Rates across
Administrative Law Judges, Fiscal Years 2007-2015.
International Journal of Government Auditing—Summer 2019
14
FEATURE STORIES
To test this hypothesis, GAO used individual claims data
to develop and estimate a quantitative statistical model
identifying factors associated with a judge’s decision to
allow or deny a claim. In consultation with SSA ofcials, and
based on a qualitative review of relevant literature, GAO
controlled for certain factors, such as claimant characteristics
(type of claim and demographics); judge making the decision
(appointment year, prior experience); location; presence of
representatives at the hearing (attorney, medical, vocational
expert); and local economic conditions (unemployment and
poverty rates).
As illustrated in Figure 2, GAO estimated the allowance
rate varied by 46 percentage points for a typical claim
depending upon the judge who heard the case. The
quantitative model identied multiple factors associated
with a judge’s decision, and GAO’s interpretation of these
ndings was supplemented by qualitative evidence. For
instance, the model demonstrated claimant age mattered,
as a 55 year-old claimant was allowed benets at a rate
4.3 times higher than a typical 35 year-old claimant. SSA
ofcials informed that this nding was consistent with SSAs
vocational guidelines, which are generally more lenient for
older claimants.
Similarly, the model found that claimants who had a
representative were allowed benets at a rate 2.9 times
higher than a typical claimant with no representative. SSA
ofcials said that, under SSA’s fee structure, representatives
are only paid if the claimant is awarded benets. As a
result, representatives may tend to take cases they believe
will be successful. Thus, GAO’s quantitative ndings were
interpreted in conjunction with qualitative evidence—further
enhancing credibility. Note: A typical claim had average
values on all factors GAO controlled for in its quantitative
statistical model (factors related to the claimant, judge, other
participants in the process, hearing ofce, and
economic characteristics).
Finally, GAO examined to what extent SSA
had processes to monitor the accuracy and
consistency of hearings decisions. While this
line of inquiry was an internal controls audit
relying on qualitative methods, the empirical
evidence developed with GAO’s quantitative
model underscored the need for SSA to
measure and hold itself accountable for
accuracy and consistency. GAO ultimately
found that SSA had not systematically
considered how each of its quality assurance
reviews helped the agency meet its strategic
plan’s objective to improve quality, consistency
and timeliness of hearings-level decisions.
Conclusion
Even as advanced data techniques become
more prevalent in audit work, SAIs must
maintain an audience awareness and ensure
messaging is both credible and accessible.
Developing qualitative evidence that grounds quantitative
ndings within the audit’s conceptual and behavioral situation
adds validity to audit ndings and enhances the likelihood
that recommendations will be implemented.
Sources
Catherine Pope and Nick Mays, “Reaching the parts other methods cannot reach: an
introduction to qualitative methods in health and health services research”, The BMJ,
Volume 311, 1 July 1995, 42-45.
Nicholas Mays and Catherine Pope, “Rigour and qualitative research”, The BMJ,
Volume 311, 8 July 1995, 109-112.
INTOSAI Working Group on IT Audit, “Big Data Management and Data
Analytics”, Paper A1-14, available at http://www.intosaiitaudit.org/working group_
paper_lists/20
Figure 2: Estimated Variation in Social Security Administration (SSA) Allowance Rates
across Judges for Typical Claims, Fiscal Years 2007-2015.
www.intosaijournal.org
15
FEATURE STORIES
by Dr. Nguyen Manh Cuong, Deputy Director General,
Department of Personnel and Organization, State Audit Ofce
of Vietnam and Dr. Tran Manh Dung, National Economics
University
Introduction
The State Audit Ofce of Vietnam (SAV) issues audit reports
at every audit's conclusion, yet judging and comparing quality
among various audits is a difcult task. Some research cites
audit quality as the ability for auditors to detect and report
material misstatements during the audit process. This ability is
highly dependent on auditor prociency and independence,
particularly when misstatements may be derived from fraud.
While several studies have focused on providing solutions
to improve audit quality, these solutions do not facilitate
identifying determinants potentially affecting audit quality.
This article discusses a SAV study performed to identify
factors that can affect audit quality at the planning,
implementation (process) or reporting phase.
Audit Quality
Audit quality reects several elements and has different
meanings depending on the audit’s phase and/or product
(plan, process, report).
Audit plan quality indicates planning is conducted within
prescribed guidelines. For the SAV, these guidelines include
clearly dened audit objectives, content, scope, materiality,
risks, methods, time and personnel. Current guidance also
calls for content to be reasonably explained, relevant and
evidential.
Audit process quality refers to the ability to detect and report
material misstatements, revealing auditor compliance with
the audit’s objectives and contents. Here, quality is achieved
by ensuring audit documents are developed and recorded
in accordance with pertinent regulations; audit evidence has
been adequately and appropriately gathered; and the
audit team has complied with all organizational regulations
and standards.
Audit report quality calls for consistency with applicable
standards and regulations, ensuring accuracy, clarity and
timeliness. Any errors, fraud and gaps should be properly
considered and resolved, and comments, assessments and
conclusions should be based on complete, accurate and
reliable evidence. Audit recommendations should be lawful,
fair, objective and feasible.
DETERMINANTS AFFECTING AUDIT QUALITY
International Journal of Government Auditing—Summer 2019
16
FEATURE STORIES
Determinants
Several determinants, both internal and external, can affect
audit quality, including auditor professional knowledge and
skills; skepticism; standards compliance; working conditions;
audit duration and quality control.
Methodology
Two hundred questionnaires were
sent to audit department heads, audit
team leaders and individual auditors
Responses were received from 175
participants. Invalid responses (seven
in total) were excluded from the
study.
A regression model was developed
that incorporated independent and dependent variables
(including internal and external factors)—legal framework,
auditee, knowledge and skill, experience, professional
skepticism, compliance, working conditions, audit time, quality
control measures—as determinants potentially affecting
audit quality.
Empirical Results and Conclusions
Regression analysis revealed two external determinants
having the most impact on audit quality: auditees (strongest
inuencer) and legal framework.
Both determinants demonstrated direct correlation to quality:
if the legal framework includes proper and adequate state
audit laws, auditing standards, processes and guidelines,
audit quality improved.
Analysis indicated decreasing levels of inuence associated
with audit time, knowledge and skill, professional skepticism,
and compliance.
Working conditions and quality control were statistically
insignicant, revealing these factors have very little to no
effect on audit quality. While quality control showed no
evidence it affected audit quality, this result does not coincide
with research associated with non-state audit rms. The
result could stem from the SAV synchronously implementing
audit quality control measures at several levels using strict
regulations, which resulted in overall audit improvement with
very few signicant errors.
Working conditions, also bearing a low signicance factor
can perhaps be attributed to auditor experience, as well
as the ability to perform timely onsite audits using remote
technology.
Regression results exposed an
“R-square” gure of 0.982, denoting
that a 98.2 percent variation in audit
quality can be explained through the
regression model, with the remaining
estimated two percent in variation
caused by other factors not included
in the model.
Additionally, applying the Durbin-Watson test (to detect the
correlation in prediction errors) resulted in an approximate
score of two, signifying that the likelihood of model self-
correlation is low. Additionally, multicollinearity tests showed
Variance Ination Factors (VIF) between 0.1-2, conrming
no collinearity occurrence between variables in the model.
This study, which focused on determinants affecting audit
quality, exposes the need for policy makers and auditors to
emphasize those factors having signicant impact on audit
quality, particularly as concerns intensify about effective
and efcient public resource use.
For full references and to see the complete regression model
results, email the authors at cuongsav@yahoo.com and
dungtm@neu.edu.vn.
"Regression analysis revealed two
external determinants having the
most impact on audit quality:
auditees and legal framework."
www.intosaijournal.org
17
FEATURE STORIES
IMPROVING QUALITY
THROUGH EFFECTIVE
MANAGEMENT
FRAMEWORKS
by Hasan Masud, Pakistan Audit and Accounts Service,
Ofce of the Auditor General of Pakistan
Quality—An Introduction
"A major challenge facing all Supreme Audit Institutions
(SAIs) is to consistently deliver high quality audits and
other work." —International Standards of Supreme Audit
Institutions (ISSAI) 40, "Quality Control for SAIs."
Quality, the degree to which inherent characteristics—
clarity, effectiveness, efciency, objectivity, relevance,
reliability, signicance and timeliness—of an audit fullls
requirements, must be viewed as a continuous process, one
that occurs throughout the audit cycle.
Controlling and assuring quality can be challenging,
particularly in environments where resources are limited, since
quality involves systems and processes aimed at ensuring
SAIs issue reports that are appropriate and in accordance
with applicable laws and regulations. Creating a central,
dedicated quality ofce is essential—it allows a SAI
to focus on the public expenditure cycle in its entirety,
improve nal reports prior to issuance, and creates a
more effective and efcient framework.
Managing Quality—SAI Pakistan’s Experience
Pakistan’s SAI is at the heart of the nation’s
accountability process. Led by an Auditor
General serving a xed, four-year, non-
extendable term and staffed with more than
ve thousand employees in 30 eld audit ofces
across the country, SAI Pakistan performs nearly
nine thousand audits annually.
The audits and nancial recovery efforts have proven
successful, in part, due to a strong quality management
system based on strengthened ethics, standards, guides
and frameworks.
SAI Pakistan has adopted the International Organization
of Supreme Audit Institutions (INTOSAI) code of ethics,
and SAI Pakistan’s Financial Audit Manual (FAM), Quality
Management Framework (QMF) and public sector audit
guidelines are consistent with the ISSAIs.
The FAM. The FAM provides auditors with a set of modern,
ISSAI-based standards, concepts, techniques and quality
assurance arrangements for auditing government entities
in Pakistan. A comprehensive document spanning the audit
cycle, the FAM requires the SAI to pay particular attention
to quality assurance programs as a means to improve audit
performance and results.
Establishing systems and procedures to conrm integral
quality assurance processes have operated satisfactorily;
ensure audit report quality; and secure improvements
while avoiding repetition of weaknesses are part of the
FAM’s provisions. The SAI has implemented a system of
quality assurance checks and balances that are objective
and consultative and include periodic reporting to top
management.
International Journal of Government Auditing—Summer 2019
18
FEATURE STORIES
The QMF. In Pakistan, an audit report tabled before the
Public Accounts Committee (PAC) passes through several
quality control and assurance stages, which includes team
observations and eld ofce report re-examination prior to
sending forward to the Auditor General, who also provides
a thorough analysis prior to report nalization.
The QMF, originally implemented in 2011, was developed
using INTOSAI and Asian Organization of Supreme Audit
Institutions (ASOSAI) standards and guidelines.
The framework provides three broad mechanisms to ensure
quality:
Quality Assurance (QA) is implemented by the head of
a particular eld audit ofce and focuses on quality
throughout the audit (at the onset of audit planning through
audit execution to audit report production).
Quality Control (QC) is the application of external quality
checks on an audit ofce. It takes place in two stages: (1)
sample testing an audit ofce’s previously quality-assured
assignments; and (2) examining all Quality Control
Committee (QCC) nal audit reports.
Quality Improvement includes creating and implementing
corrective actions based on quality assurance and quality
control reviews.
To ensure QMF compliance, SAI Pakistan established two
committees—an internal QCC (headed by a Deputy
Auditor General (DAG) in the audit’s department) and
an external QCC, led by a DAG outside the department
performing the audit.
Managing Quality—Some Challenges
Today, SAIs operate in a constantly changing world with
an ever-rising tide of expectations for transparency and
accountability. Quality Management (QM) must be viewed
as a dynamic process aimed at responding to these changing
societal demands. Some current challenges faced by SAI
Pakistan include:
Treating QM as a rotational assignment;
Developing a proper Information Technology (IT) support
system;
Missing linkages with Civil Society Organizations (CSOs);
Developing and managing relationships with auditees;
Managing organizational change;
Sustaining reforms;
QM training;
Retaining talent; and
Expanding areas of operation.
Managing Quality—Some Recommendations
Focusing on the Complete Audit Cycle. Creating high
standards for reports is key, as is emphasizing overall
quality. To do this, the quality cycle must begin at the pre-
audit stage and end when nal recommendations are
implemented at the PAC level.
Making Quality Management a Dedicated Function. An
independent ofce dedicated to managing and assuring
quality provides more consistent, institutional approaches
to quality. This notion can be further strengthened through
training that makes quality management an automatic,
continual part of the audit process.
Developing National Standards. SAI Pakistan is nalizing a
Performance Measurement Framework (PMF) draft report,
which represents a critical rst step in consolidating the
SAI’s various frameworks while also conforming to ISSAI
guidelines. This effort can lead to future national public-
sector auditing standards development.
Parliamentary Support and Ownership. SAI Pakistan and
its quality framework operate within limits dened by
parliament. It is important that parliament and the PAC
ensure no nancial, administrative or legal issues limit the
SAI’s scope in producing high-quality reports.
Employing an Effective Management Information System
(MIS). An MIS supports QM frameworks by providing a
platform for report centralization, control, ltering and
processing. For example, a robust MIS can facilitate report
prioritization by agging important issues and sensitive
observations.
Incorporating QM into Pre-Report Issuance. The QM
intervention must be a pre-report issuance inspection, not
a post-report inspection step. Adequate, competent staff
across elds (audit, IT, administrative) is imperative.
Creating a Strong Internal Audit System. Establishing an
independent internal audit department (and improving
coordination with external audit divisions) at the federal
and provincial levels is vital to broadening audit quality.
www.intosaijournal.org
19
FEATURE STORIES
A MODEL TO EXAMINE
AUDIT EFFECTIVENESS:
NATIONALLY, GLOBALLY
by Dr. Amir Seri, Commissioner of State Audit, Regulation and Marketing
Group, Israel Electric Cooperation, and Lecturer at Bar Ilan University-Israel
Supreme Audit Institutions (SAIs) represent important
mechanisms in safeguarding public resources, and
the broad authorities of the Comptroller of Israel
are intended to, among other things, strengthen
good governance. However, evaluating public
audit effectiveness and its ability to enhance good
governance is complex and often difcult, if not
impossible.
This article explores modeling to examine audit
effectiveness, including an examination of inuential
factors and a comparison that uncovers the most
impactful report type.
The Proposed Model
Creating a benchmark denition for effectiveness
was the rst step. For this model, effectiveness (E)
is the rate of deciency correction—how many
corrective measures were implemented based on
audit ndings and recommended actions.
Next, dening variables that potentially impact
audit effectiveness were identied and categorized
(categories were based on research, which often
cited grouping factors into four broad categories):
Category A: Reciprocal relationship between audit
and public and media opinion;
Category B: Reciprocal relationship between audit
and legislative authority;
Category C: Reciprocal relationship between audit
and executive authority; and
Category D: Audit ofce work patterns.
Given these variables, measuring audit effectiveness
can be calculated using the formula:
E=f (A, B, C, D) or
i= Ai+ Bi +Ci +Di, where i
represents the report type—regular (r) or special (s).
Regular reports are those published at times dened
by law, while special reports are initiated by the
comptroller and not included in the annual work
plan.
International Journal of Government Auditing—Summer 2019
20
FEATURE STORIES
The resulting "E" value falls within a range between 0 and
1 (0-100%), where a value of “1” indicates all deciencies
have been corrected, and a value of “0" means no corrections
have been implemented.
Putting the Model Into Practice
This model was used to specically evaluate SAI Israel's audit
effectiveness, and an equal amount of completed regular
and special audit reports (13 each) were examined and
compared to provide the basis for the “E” value.
Additionally, 15 interviews were
conducted, which included sessions
with three recent Comptrollers;
the former Chairman of the State
Audit Affairs Committee; previous
managers of the State Control
Inspector General Division in the
Prime Minister’s ofce; and relevant
internal staff (both managerial and
technical).
To create quantitative measures
from qualitative information for
each category, the following data were used:
Category A was analyzed using the number of articles
published in the press on a specic report;
Category B was calculated by examining the extent of
discussions in designated parliamentary committees;
Category C was checked by analyzing activities associated
with correcting the audit’s identied deciencies; and
Category D scoring included analyzing and comparing
various parameters within the audit ofce that may
affect audit effectiveness, such as audit work timing,
audit frequency and monitoring deciency corrections.
Performing interviews (in-depth, semi-structured) with
relevant personnel, both internal and external, helped
validate and strengthen ndings.
Results
Values calculated for Category A concluded that, on average,
interactions between SAI Israel and the media increase special
report effectiveness when compared to regular reports;
Analyzing Category B results indicate interactions between
SAI Israel and the legislative branch also increase the
effectiveness associated with special reports more so than
that of regular reports;
On the other hand, Category C results suggest the interactions
between the SAI and executive authorities increase regular
report effectiveness in comparison to special reports; and
Category D analysis showed that, as a rule, there is no
difference (in the context of effectiveness) in how SAI Israel
operational patterns impact regular
and special reports.
Conclusions
Findings suggest that effectiveness
(faults correction rate) coinciding
with special reports will generally
be higher than that of regular
reports.
Special reports tend to generate a
greater amount of real-time value
and relevancy given the nature in which they are initiated
and the topics they tend to cover. This is also in-keeping
with available literature suggesting such factors (value and
relevancy) increase the probability of an event becoming
sensationalized in the media.
Research also indicates the media's ability to raise awareness
on more publicly stimulating topics leads to increased
coverage of special reports and boosts topic recognition,
which, in turn, can result in heightened pressure on auditees
to correct identied faults, thus enhancing audit effectiveness.
The uniqueness of this model provides a foundation to
examine audit effectiveness and enables measuring specic
factors that can affect an audit’s impact.
While this study focused on SAI Israel's audit effectiveness,
the model may be applicable to other SAIs.
Contact the author at amir.seri@iec.co.il for a full list of
references and to learn more about the model.
"This unique model provides a foundation
to examine audit effectiveness and enables
measuring specic factors that can affect an
audit's impact. While this study focused on
SAI Israel's audit effectiveness, the model
may be applicable to other SAIs."
www.intosaijournal.org
21
FEATURE STORIES
International Journal of Government Auditing—Summer 2019
22
INSIDE INTOSAI
GOAL CHAIR MEETINGS UNITE DELEGATES TO
DISCUSS INTOSAI PROGRESS, WAY FORWARD
Click on each INTOSAI goal icon to access video stories from 2019 Goal Chair meetings
Support SAIs in developing capacity to maximize
their value and benets to society by: promoting the
development of capabilities and professional capacities
of independent SAIs, regional organizations; informing
INTOSAI decision-making on SAI capacity development
matters; and cooperating with donors, relevant partners,
and stakeholders.
GOAL TWO: CAPACITY DEVELOPMENT
Promote strong, independent, multidisciplinary SAIs
and encourage good governance by: advocating for,
providing, and maintaining International Standards of
Supreme Audit Institutions (ISSAI); and contributing to
the development and adoption of appropriate and
effective professional standards.
GOAL ONE: PROFESSIONAL STANDARDS
Encourage SAI cooperation, collaboration, and continuous
improvement through knowledge development, sharing,
and services, including: producing and revising INTOSAI
products; providing benchmarks and operating a
community portal; and conducting best practice studies
and performing research on issues of mutual interest
and concern.
GOAL THREE: KNOWLEDGE SHARING
IRCP PROVIDES FORUM TO DISCUSS
REGIONAL, GLOBAL COMMUNITY SYNERGIES
“To evolve, you have to become involved.
Pamela Monroe-Ellis, Auditor General of Jamaica and
General Secretary for the Caribbean Organization of
Supreme Audit Institutions (CAROSAI), succinctly captured
the notion that we must be active contributors if we are to
transform—a concept thoroughly embraced at this year’s
INTOSAI-Regions Coordination Platform (IRCP) hosted by
South Africa's Supreme Audit Institution (SAI) in Cape Town.
The three-day International Organization of Supreme Audit
Institutions (INTOSAI) event united global bodies and regions
to explore synergies, address the INTOSAI value chain and
crosscutting priorities, and foster an integrative approach in
developing solutions and aligning efforts.
The meeting kicked off with a new approach—parallel
regional and global meetings. The Organization of Latin
American and Caribbean Supreme Audit Institutions
(OLACEFS) led the regional session, which highlighted
innovation and its signicance in audit work, while the global
gathering focused on overarching concepts most affecting
the broader INTOSAI community.
Einar Gørrissen, INTOSAI Development Initiative (IDI)
Director General, noted independence remains the biggest
issue facing SAIs today and stressed the need to reconrm
its importance, particularly as "there is a close connection
between SAI independence and the ability to add value
and benets to the lives of citizens."
Adding value and benets to society is a key driver in the
work SAIs perform and intertwined into the Sustainable
Development Goals (SDGs). As the countdown continues to
meet the 2030 deadline in achieving the global goals, IRCP
participants shared experiences, challenges, requirements
and future outlooks regarding SDG auditing initiatives and
contributing to national SDG implementation.
Several delegates noted difculty in performing SDG
audits, primarily in dening what constitutes an SDG audit.
An abundance of models and approaches were also cited
as challenges.
www.intosaijournal.org
23
INSIDE INTOSAI
International Journal of Government Auditing—Summer 2019
24
INSIDE INTOSAI
While establishing a common understanding and a unied
approach will require additional research and practice,
delegates concurred that mapping and streamlining
collective efforts are vital to future SDG work.
Archana Shirsat, IDI Deputy Director General, Professional
and Relevant SAIs Department, believes aligning and
integrating "will lead to one voice we can stand behind."
“If you want to walk fast, walk alone. If you want to walk far
walk together. But, how do we walk far and fast together?”
Such unity and the required collaborative measures it beckons
requires communication—a heavily debated, engaging topic
among participants.
Michael Hix, International Relations Director for the U. S.
Government Accountability Ofce and Vice President of
the INTOSAI Journal, shared insight on communications and
stakeholder engagement and highlighted current internal
and external INTOSAI efforts.
Hix asked attendees, who worked in small groups, to tackle
questions on information availability, accessibility and
alternate approaches. The dialogue identied opportunities
for enhanced coordination and awareness in sharing
information.
Silke Steiner, representing the INTOSAI General Secretariat,
indicated the revamped INTOSAI website (www.intosai.org),
tentatively scheduled for launch prior to the XXIII Congress
in Moscow, will enrich communication efforts, as the new site
will prominently display essential information and links.
The INTOSAI Community Portal is also undergoing a facelift,
and the Knowledge Sharing Committee's representative,
Vishnukanth P.B., informed delegates that the new portal
(www.intosaicommunity.net) aims to increase data sharing and
engagement opportunities.
While the freshly designed websites will better facilitate
knowledge sharing, delegates also suggested establishing a
forum designed, in part, to develop a single, global INTOSAI
communications strategy.
As the INTOSAI community continues to think strategically,
the concepts of leadership, evolution and innovation rose to
the forefront.
"It is important that leaders be daring—that they dare to
make change and be willing to shift the boundaries," stressed
Tsakani Ratsela, Deputy Auditor General, SAI South Africa.
Active dialogue continued on initiatives supporting standards
quality and value and moving toward more effective
coordination and consultation. Participants also offered
perspectives into successful efforts—peer-to-peer projects,
blended learning and twinning arrangements—and
provided ideas on innovative techniques, such as tailored
or customized programs and creating a regional pool of
quality reviewers, that can be employed.
Improving performance and enhancing capacity were also
signicant topics discussed during this year's IRCP. During
his presentation on performance measuring and monitoring,
Tiolusi Tiueti, Chief Executive, Pacic Association of
Supreme Audit Institutions (PASAI), asked a key question of
participants—what do we really mean by SAI performance?
Tiueti explained that dening performance, perhaps in such a
way that focuses more so on quality versus quantity, can lead
to more accurate measurement, better informed decision-
making and greater sustainability.
As INTOSAI continues forging a path toward progress in
achieving its strategic goals, the sentiments Kimi Makwetu—
Auditor General of South Africa and Chair of the Capacity
Building Committee—shared in his opening remarks remind
us that meetings like the IRCP are greatly needed, as they
unite delegates on a global scale to share ideas that "take
INTOSAI to the next level and beyond."
Watch the IRCP video story on the Journal's YouTube
channel.
WGPD HOLDS MEETING
IN AZERBAIJAN
The Republic of Azerbaijan's Chamber of Accounts (COA)
hosted the 2019 International Organization of Supreme
Audit Institutions (INTOSAI) Working Group on Public Debt
(WGPD) annual meeting from May 23-25, 2019.
Fifty-three participants from 27 Supreme Audit Institutions
(SAIs) attended the meeting, which included guest speakers
from the World Bank, Azerbaijan Diplomatic Academy
University and the United Nations Conference on Trade and
Development (UNCTAD).
Vugar Gulmammadov, COA Chairperson, welcomed guests
and noted the signicance of cooperation and knowledge
in sharing public sector audit information among states—a
major WGPD objective.
A fundamental WGPD role is to strengthen public debt
management audits by preparing and publishing specic
guidance and best practices to inform and encourage
sound management and proper reporting,” noted Michael
G. Aguinaldo, Commission on Audit of the Republic of the
Philippines and INTOSAI WGPD Chairman.
As part of the event's key speeches and presentations, Vugar
Ibrahimov, Zaur Valiyev and Elvin Hajiyev, COA public debt
experts, jointly presented perspectives on debt and scal
sustainability; Elchin Rashidov, ADA University, highlighted
sustainability, solvency and liquidity; the World Bank’s
Lea Hakim offered insight into strengthening public debt
monitoring; and Stephanie Blankenburg, UNCTAD, provided
participants with a view into debt transparency initiatives
and effective macroeconomic debt strategies for sustainable
development.
Aguinaldo shared WGPD recent accomplishments, including
the approval and adoption of the group’s Terms of Reference
by the INTOSAI Governing Board in November 2018. He also
noted the “Guidance on the Audit of Public Debt” exposure
draft's availability for comment (www.issai.org).
Jaisankar Dhandapani, SAI India, updated participants on the
INTOSAI Community Portal, promoting the site as an effective
and efcient platform for knowledge sharing, particularly
given the geographical dispersion of working group members.
The event's agenda also included a session in which SAIs
Shared experiences in auditing public debt, as well as
dialogue on prioritizing activities for the group's Work Plan
2020-2022.
SAI Qatar was welcomed as the newest member of the
WGPD, which is now 35 members strong. SAI Bhutan
conrmed hosting the 2020 WGPD meeting, and the SAIs of
Indonesia, Kuwait, and Qatar expressed interest in hosting
the WGPD event in 2021.
For more information about the WGPD, visit https://www.
coa.gov.ph/wgpd/.
www.intosaijournal.org
25
INSIDE INTOSAI
International Journal of Government Auditing—Summer 2019
26
INSIDE INTOSAI
NEWS FROM THE OLACEFS REGION
RESEARCH COMPETITION TO STRENGTHEN AUDITING, CONTROL
The Presidency and the Legal Advisor Committee (CAJ) of the Organization of Latin
American and Caribbean Supreme Audit Institutions (OLACEFS) are carrying out
the 2019 Regional Research Competition “Strengthening Auditing and Government
Control to Benet Latin America and the Caribbean” for OLACEFS full members and
associates. The competition aims to enrich the exchange of knowledge and experience
within OLACEFS to improve the quality of audits. Read the full story here.
OLACEFS MARKS 54TH ANNIVERSARY
On April 9, 2019, the Organization of Latin American and Caribbean Supreme Audit
Institutions (OLACEFS) marked its 54th anniversary as a space for research, study,
training, specialization, technical assistance and the exchange of ideas and experiences.
OLACEFS aims to strengthen the institutional capacity of SAIs in the region and enhance
collaborative efforts. Learn more about OLACEFS membership and presidency here.
OLACEFS HOLDS SUCCESSFUL GOVERNING BOARD MEETING
OLACEFS held a successful 69th Governing Board Ordinary Meeting in Lima, Peru,
that brought together the Presidency, Executive Secretariat, Governing Board (GB),
Supreme Audit Institution (SAI) representatives, and the German Society for International
Cooperation (GIZ). During the event, the GB approved the creation of two new working
groups related to ghting transnational corruption and auditing the management of
disasters. Read the full story here.
OLACEFS IS NOW ON FACEBOOK, TWITTER AND INSTAGRAM
The Organization of Latin America and Caribbean Supreme Audit Institutions (OLACEFS)
is now on Facebook, Twitter and Instagram to expand dialogue and participation within
the region and enhance communication around the world. The new social media presence
complements current OLACEFS platforms—YouTube, Flickr, and LinkedIn.
Twitter: @OLACEFS_ocial
Facebook: @olacefs
Instagram: @olacefs
www.intosaijournal.org
27
INSIDE INTOSAI
REGIONAL COMPETITION ON GOOD GOVERNANCE UNDERWAY
The Organization of Latin American and Caribbean Supreme Audit Institutions (OLACEFS)
Technical Commission of Practices on Good Governance launched an annual competition
focusing on “Supreme Audit Institutions (SAIs) and the Use of New Technologies to Detect
Financial Fraud.” All professionals working in an OLACEFS-member SAI can participate.
Contestants can upload relevant work at www.contraloria.gob.gt through August 19,
2019, at 15:00 (Guatemala time). For more information, visit http://bit.ly/2wuOBPj.
OLACEFS HOLDS INNOVATION WORKSHOP AT IRCP IN SOUTH AFRICA
A regional roundtable was held as part of the International Organization of Supreme
Audit Institutions (INTOSAI)-Regions Coordination Platform. The Organization of Latin
American and Caribbean Supreme Audit Institutions (OLACEFS), represented by Chile’s
Ofce of the Comptroller General, Executive Secretariat, led the roundtable, which took
place in Cape Town, South Africa, in May. Click here to learn more about the session
and the toolkit designed to provide creative solutions to challenges SAIs currently face.
OLACEFS CONTRIBUTES TO P4R NETWORK MEETING
The Organization of Latin American and Caribbean Supreme Audit Institutions (OLACEFS),
represented by the Presidency and the Commissions of Good Governance (Supreme
Audit Institution (SAI) Argentina) and Citizen Engagement (SAI Peru), participated in the
United Nations (UN) Partners for Review (P4R) sixth network meeting (P4R) held from
May 21-23, 2019, in Oaxaca, Mexico. Access the full story here.
MEMBERS PARTICIPATE IN INTOSAI COURSE ON COOPERATIVE AUDITS
The INTOSAI Subcommittee on Cooperative Audits offered a self-guided virtual course
"Guidelines for the Execution of Cooperative Audits " from April 1 to May 2, 2019, and
86 professionals from 14 Supreme Audit Institutions (SAIs) from the Organization of
Latin American and Caribbean Organization of Supreme Audit Institutions (OLACEFS)
participated. Learn more about the course, participants and next steps here.
International Journal of Government Auditing—Summer 2019
28
INSIDE INTOSAI
PEER REVIEW
SUBCOMMITTEE
SHARES 2019 GLOBAL
SURVEY RESULTS
The International Organization
of Supreme Audit Institutions
(INTOSAI) Subcommittee on
Peer Reviews implements
an annual global survey
on various peer review
aspects within the INTOSAI
community.
The 2019 survey, available
in English, French and Spanish,
was distributed to nearly 480 delegates.
While there was success in connecting with such a broad and
diverse audience, including Supreme Audit Institutions (SAIs)
in more remote locations, the response rate was only 10%
(lowest in survey history).
As of March 2019, statistical analysis revealed a total
of 2,080 downloads of peer review-related documents
currently available on the CBC web-based library (www.
intosaicbc.org). SAI Hungary’s 2016 report and the European
Court of Auditors 2014 report logged the most downloads.
Typically, if a SAI conducts two peer reviews, there is a
signicant break (8-10 years) between them. Additionally,
peer review follow-ups, as suggested in International
Standards of Supreme Audit Institutions 5600, are very
seldom.
The survey also measured interest in attending a 2020
workshop in Bratislava, Slovakia, to focus practical aspects
and experiences in the peer review process. Twenty-ve
SAIs expressed preliminary interest in such a seminar.
At the 2018 INTOSAI Capacity Building Committee (CBC)
meeting in Kuwait, the subcommittee posed adjusting
the survey interval from annual to bi-annual in an effort
to capture more responses in the future, as seeking input
annually may be too frequent.
Visit https://www.intosaicbc.org/subcommittee-3-on-peer-
reviews-2/ to learn more about the Subcommittee on Peer
Reviews.
PSC SEEKS TO STAFF
NEW TECHNICAL
SUPPORT FUNCTION
Are you ready for new challenges?
The International Organization of Supreme Audit Institutions
(INTOSAI) Professional Standards Committee (PSC) seeks
to staff its newly established Technical Support Function
(TSF).
Currently, the PSC is recruiting two TSF ofcers and one
TSF team manager to support INTOSAI standard-setting
activities.
To apply, candidates will need to submit a (1) nomination
form and (2) brief letter explaining why the candidate
would like to become a part of the TSF team and how his/
her background and professional experience can better
contribute to TSF work.
To learn more about the TSF, positions and the application
process, visit the PSC website.
www.intosaijournal.org
29
INSIDE INTOSAI
INDEPENDENT REVIEWS
VITAL TO QUALITY,
CREDIBILITY
by Brighton Nyanga, INTOSAI Development Initiative SAI
Governance Department Manager
Quality management is vital to
Supreme Audit Institutions (SAIs)—in
audit work, as well as performance
reports. Over the past few years,
we have made great strides in
increasing performance reporting
efforts through the SAI Performance
Measurement Framework (SAI PMF).
By incorporating International Standards of Supreme Audit
Institutions (ISSAIs) along with global best practices, the SAI
PMF is a mechanism for self, peer and external assessments
that enables SAIs to examine institutional, organizational
and professional capacities in a holistic and evidence-
based manner.
But, how can internal and external users of a SAI Performance
Report be certain the report is credible? Through independent
reviews.
Independent reviews (1) help ensure SAI PMF assessment
indicators and scores are applied correctly and based on
sufcient and appropriate evidence; and (2) substantiate
that the ndings support an analysis that leads to valid
conclusions.
As the global operational lead in implementing the SAI
PMF, the INTOSAI Development Initiative (IDI), can conduct
(or arrange for other SAI PMF experts to conduct) an
independent review of draft SAI performance reports for
SAIs recently completing SAI PMF assessments.
This independent review, provided at no cost to the SAI,
results in a “Statement of Independent Review” issued upon
the review’s completion, and seeks to answer the following
questions:
Is the report factually correct?
Has the SAI PMF methodology been adhered to as it relates
to the process for conducting the assessment, scoring of
indicators, content and structure of the assessment report?
Is there relevant and sufcient evidence to justify the
indicator scores?
Does the assessment report provide a meaningful analysis
of the evidence and identify root causes of performance
across the different domains and indicators?
According to surveys conducted by the IDI SAI PMF team,
most assessment results are used by SAIs to guide strategic
planning and to identify opportunities for enhancing capacity
development efforts.
Assessments have also been used in benchmarking a SAI’s
performance against the ISSAIs, demonstrating SAI value
and benets to society, and as an aid in preparing annual
reports.
With an accurate understanding of the SAI’s position, SAI
leadership is better positioned to implement appropriate and
timely interventions to improve and maintain organizational
performance.
These interventions can only be as good as the quality of
the ndings presented, which makes an independent review
a vital step in the SAI PMF assessment process.
For more information about IDI independent reviews of
SAI PMF assessments, visit http://www.idi.no/en/idi-cpd/
sai-pmf.
International Journal of Government Auditing—Summer 2019
30
SPOTLIGHT ON CAPACITY BUILDING
GROWING SAI YOUNG
LEADERS: THE IDI STORY
“I saw the garden as a metaphor for certain aspects of my
life. A leader must also tend his garden; he, too, plants seeds
and then watches, cultivates, and harvests the results.
—Nelson Mandela
When Nelson Mandela was imprisoned, he tended his garden
and spoke of throwing seeds on the ground, watering them
and watching them grow. Similarly, the INTOSAI Development
Initiative (IDI) has observed and nurtured young leaders from
Supreme Audit Institutions (SAI) worldwide through the SAI
Young Leader (SYL) initiative.
As these future leaders take the journey from seedling to
blossoming tree, the SYL initiative, now in its second iteration,
provides nourishment through enriching exchanges, as well
as leveraging work and support from the pilot program.
Preparing the Ground
Before gardeners sprinkle seeds upon the ground, they must
survey the land. Time and again, IDI’s experience shows
that leadership drives SAI transformation and performance
enhancement. While IDI and other partners can certainly
play supporting roles, leadership remains at the forefront
of change.
Seedlings
Once the ground has been prepared, we must plant the seeds.
The SYL program selection process is designed to identify
candidates with the highest potential to grow and succeed.
Selection also includes examining each candidate's change
strategy project (an integral component of the SYL program)
having the highest likelihood for SAI implementation.
These criteria meant ensuring a SAI environment conducive
to change and possessing leadership fully dedicated to
the young leader, as well as the program as a whole. The
IDI fullled both objectives through a two-stage process—
assessing change strategy projects then interviewing
nominated candidates.
Watering
As spring began in March 2018, IDI and the participating
SAIs began nurturing and growing the select young leaders.
Through an intense interactive, engaging program that
included workshops, communication platforms, support from
SYL coaches, and innovative evaluation methods, candidates
continued learning leadership skills and perfecting their
change strategy projects.
Cultivating
Individual responsibility combined with INTOSAI community
support further cultivated young leader transformation. The
many gardeners tending to the seedlings provided valuable
inputs—SAI-coordinated and hosted interactions, specialists
and leaders sharing personal journeys, coaches providing
extensive support beyond the call of duty, and, perhaps most
signicantly, a network that created a climate necessary for
encouragement and growth.
The inaugural young leaders established ve core values,
identied behaviors best reecting those values, and
endeavored to hold one another accountable for them.
They also introduced the “SYL Digital Yearbook” to capture
experiences, memories and achievements on the personal
journeys of discovery and change.
Blossoming Trees
As autumn nears, a gardener takes stock of progress and
seeks to uncover key ingredients to his ourishing crop. As
IDI reects on the SYL initiative and the young leaders born
from the pilot program, there is an enormous sense of pride
at the blossoming trees that have taken root.
Twenty young leaders completed the rst iteration of the
SYL program and developed change strategy proposals on
a wide variety of topics—including communications, digital
solutions, data analytics employment, value chain reporting—
that have already impacted SAI capacity and performance.
www.intosaijournal.org
31
SPOTLIGHT ON CAPACITY BUILDING
PAST, FUTURE SAI
YOUNG LEADERS
SHARE PERSPECTIVES
The SYL Program, one of the best development programs
I have experienced, provides a holistic approach that
includes all facets of leadership.
Through the program, I completed a project from concept
to implementation and was able to share it with other
teams. It was an empowering, eye-opening experience,
as the challenges I faced helped me exercise knowledge
and skills and helped me grow in my leadership journey.
—Seolebaleng Nkhisang, SAI Botswana
2018 SAI Young Leader Graduate
There is a Chinese idiom that weaves the tale of a group
of blind men touching an elephant for the rst time. Each
touch results in a completely different answer in describing
an elephant. Before this project began, I was like one of
the tale's blind men—not knowing what I would learn and
experience.
Though the young leaders are from different countries,
different backgrounds and different cultures, we have
similar expectations—seeking growth and change. As we
met for the rst time in Cape Town, perhaps we hoped to
touch the elephant exactly as we envisioned. Yet, much
like the tale, communicating, cooperating and coordinating
with others was key, particularly as personal observations
tend to have limited data and biased interpretations.
I am condent to continue the SYL journey with my peers
and with the help and guidance from IDI and inspirational
INTOSAI leaders to further explore the "elephant" (change
and transformation).
—Boyuan Su, SAI China
2019 SAI Young Leader Candidate
Some ideas and experiences have also been published (SAI
Malta Unveils Audit Smart; SAI Estonia Shares Stakeholder
Engagement Strategies), enabling broader outreach to the
global accountability community.
Rising Above the Canopy
To motivate SYLs in change strategy implementation, the
IDI will present the “Best Change Initiative” award at the
XXIII INTOSAI Congress in Moscow to the young leader
who demonstrated the highest level of innovation, impact,
quality, inclusiveness, personal growth and community
values.
Fallon Stephany Arias Calero from SAI Costa Rica earned
this year’s award for her innovative change strategy on
design thinking, which has profoundly impacted the SAI—
Fallon developed design thinking methodology, trained a
design thinking team, and facilitated a design thinking audit
approach while displaying signicant leadership growth.
Planting a New Crop in Spring
A gardener can never rest, and when spring comes, it is
time to plant a new garden. Twenty-ve seedlings were
selected in March 2019 to represent the new crop of SAI
young leaders.
They began the SYL journey in Cape Town, South Africa,
in May, and are already beneting from the fruits of the
pilot program, as previous graduates extend support and
provide valuable lessons learned.
SYL participants "explore the elephant."
International Journal of Government Auditing—Summer 2019
32
SPOTLIGHT ON CAPACITY BUILDING
SAI YOUNG LEADERS
LEAD US TO THE FUTURE
by Tytti Yli-Viikari, Auditor General, SAI Finland, and SAI
Young Leader Coach, Mentor
Just before the turmoil and havoc, the sea was translucent
and calm. Soft sounds of oars diving in the water—pushing
a wooden boat smoothly forward—were in unison as a team
of men rowed toward the shy promise of sunrise. Seconds
later, the Sampo, a magical machine that could create
riches, was lost forever as it slipped into the sea that now
rolled and roared.
This story, "The Stealing of the Sampo," from the epic tale
"Kalevala," has left a cultural footprint on Finnish society.
Perhaps the Sampo could allow today’s young auditor to
easily master challenging, emerging topics—employing
data analytics, ensuring Sustainable Development Goal
(SDG) policy coherence, applying design-thinking to
audits, enhancing stakeholder relations, ensuring audit
work independence and quality. However, no such magical
machine exists, so we must come together as a professional
community and build new competencies to tackle such issues.
I had the honor to evoke the Sampo with the rst cohort of IDI
SAI Young Leader candidates, where I observed a journey
that required determination, courage and a strong belief in
one’s ability to turn challenges into opportunities. I witnessed
a group of bright minds, warm hearts and inspired audit
professionals grow—as persons, as well as a network.
As the one-year program came to its end last October, I
was touched by the magnitude of a common path and its
impact—the young leaders had been transformed, and
they exuded empowerment, positive self-awareness and joy.
They supported each other, valuing the chance to reect on
leadership skills and openly share lessons learned.
The change projects they were charged with writing as
part of the program compelled their respective institutions
to discover and share new ways of thinking and working.
Coaches encouraged similar participation and outreach
efforts in what was clearly a change-making journey.
The SYL program contributes to my strong belief in the
future of public audit. The operating environment, methods
and stakeholder expectations are rapidly changing, but
our young leaders give us a great source of inspiration.
Personal leadership journeys link peers to experimentation
and change, and such exposure to global challenges brings
about a wider understanding and incorporates enthusiasm
from a community of change leaders.
Let us welcome the next generation of IDI SAI young leaders,
who cultivate the future of public external audit by working
together, discovering the INTOSAI community, and sharing
insights in transforming challenges into opportunities.
The treasure they gain will not disappear when storms
come—it carries on in their hearts and minds and is shared
by alumni eager to cheer them on!