Monitoring the effectiveness of the Australian aid program

What is aid effectiveness?

The purpose of aid is to contribute to tangible improvements in the lives of the world’s poorest people. The aid effectiveness agenda, enshrined in the Paris Declaration on Aid Effectiveness (2005) and more recently through the Accra Agenda for Action (AAA) (2008), aims to improve the quality of the delivery, management, and use of official development assistance (ODA) in order to maximise its development impacts.

The Paris Declaration on Aid Effectiveness represents a significant step towards formalising and focusing international efforts to improve the effectiveness of aid and its contribution to development. It is premised on a partnership between donors and partner countries and defines the principles and commitments by which they intend to ensure that aid is as effective as possible. Its five basic principles are as follows:

Ownership: Partner countries exercise effective leadership over their development policies and strategies and co-ordinate development actions;
Alignment: Donors base their overall support on partner countries' national development strategies, institutions and procedures;
Harmonisation: Donors' actions are more transparent, collectively effective and harmonised with each other;
Managing for Results: Managing resources and improving decision-making with a focus on results;
Mutual Accountability: Donors and partners are accountable for development results.

The Paris Declaration is grounded in good practices identified over many years and reflects international consensus on reforming official aid delivery and management. It is based on the important assumption that improved aid effectiveness will increase the impact aid has in reducing poverty and inequality, increasing growth, building capacity and accelerating achievement of the Millennium Development Goals. The aid effectiveness agenda seeks to transform aid relationships to achieve these goals. For donors, it means radical shifts in approaches to aid management and delivery. For partner countries it requires strong leadership of development interventions, including through strengthening development planning, public financial management and donor coordination.

Australia is a signatory to the Paris Declaration and more recently to the AAA, which commits donors, partner governments and multilateral development organisations to undertake actions to accelerate progress towards the goals of the Paris Declaration. Australia’s commitment to the aid effectiveness agenda is given in the policy framework that guides the management of the Australian aid program; a stronger Operations and Policy Branch; the strengthening of the aid program’s performance and quality policies, guidelines and tools; and the establishment of the Office of Development Effectiveness (ODE) within AusAID.

Office of Development Effectiveness

Office of Development Effectiveness Website

The Australian Government established the ODE in 2006 to monitor the quality and evaluate the impact of the Australian aid program.

The ODE answers directly to the Director General of AusAID. It is guided by the Development Effectiveness Steering Committee which includes deputy secretaries from the following Australian Government departments: Prime Minister and Cabinet, Foreign Affairs and Trade, Treasury and Finance and Administration.

Terms of Reference: ODE [PDF 112KB]

Annual Review of Development Effectiveness

The ODE publishes an Annual Review of Development Effectiveness (ARDE) drawing on the breadth of its work, including cross cutting reviews and evaluations, and on the experiences of all Australian agencies delivering Official Development Assistance. The ARDE contributes to the transparency and accountability of the Australian aid program and provides a link between increasing budget allocations and increased aid effectiveness.

The first ARDE was released in March 2008. The ARDE complements AusAID’s Annual Reports and reporting by other aid-delivering Australian government agencies.

Strategic evaluations

Recent years have seen an international debate about the need for evaluation of aid programs increasing and as well as advances in evaluation methodologies. With growing community interest in aid effectiveness, aid agencies are increasingly seeking credible evaluations of the long term impact of aid initiatives, using rigorous social sciences methods. ODE ensures that Australia uses the best in evaluation methodology and development thinking.

The ODE conducts evaluations or reviews of major country programs when a new strategy is to be developed. The ODE also manages a program of flagship evaluations, usually led by senior, internationally recognised figures. ODE's evaluation program is based on the following criteria:

Relevance to policy or budget priorities
Whole of government interest
Cross program relevance
Trialling of new evaluation methodologies

Managing for better performance in the aid program

Country strategies

In each country where the Australian Government delivers aid, there is a strategy to set out the plan for aid over the next three to ten years. The country strategy identifies where Australia can make the most difference in contributing to the country's development outcomes. It defines what priority sectors Australia's aid will be targeted to, why those priority sectors have been chosen, and how aid objectives will be achieved.

Country strategies are key documents supporting Australia's aid effectiveness. They provide a framework to guide policy dialogue, aid activities, partnerships and other elements that directly contribute to achieving development objectives. They are also the basis for assessing the effectiveness of Australian aid, by setting out
what Australian aid is intended to achieve (its objectives), and how progress towards those objectives will be measured and managed.

Country strategies are shaped by the key principles of aid effectiveness. They are aligned to the priorities and needs of the partner country, harmonised with the activities of other donors in the country, and encourage mutual accountability between governments. They promote a focus on managing for results.

Pacific Partnerships for Development will initially focus on new commitments. In line with the Prime Minister of Australia’s Port Moresby Declaration (March 2008), the Pacific Partnerships represent a new way of engaging with our partners based on mutual respect and mutual responsibility. They will incrementally expand to include the entire country strategy and aid relationships.

New country strategies will be informed by reviews of previous strategies with the Office of Development Effectiveness (ODE) undertaking the reviews for the major partner countries.

Country strategies are complemented by regional strategies (for example, strategies for the Pacific or Asia or Africa). The regional strategy defines the Australian Government’s approach where regional-level interventions are necessary to achieve shared development outcomes for multiple countries in a particular region.

Performance Management and Evaluation Policy

The Australian aid program is committed to strengthening its performance orientation to help managers improve development effectiveness and account for results. The Performance Management and Evaluation Policy [PDF 106kb] sets out expectations for measuring the performance of Australian aid, including a number of principles that apply to all types of reporting.

The policy sets out the minimum expectations for measuring performance at strategy or program level and at activity level. There are three types of reporting processes: annual performance reporting, the quality reporting system, and evaluations.

Annual performance reporting

A major part of performance assessment is annual performance reporting. It describes the performance of the program over the year, and rates progress against strategy objectives. The information is then used in making decisions about future program planning and budgeting.

Annual performance reporting includes reporting against country and regional strategies and selected thematic areas.

Guidelines on quality reporting, independent completion reports and evaluations are linked to the Performance Management and Evaluation Policy [PDF 106kb].

Aid quality

AusAID’s Quality Reporting System (QRS) provides for ongoing measurement of the quality of activities to support managing for results and reporting on the effectiveness of the aid program. The system helps to ensure reliable, valid and robust information is available to assist ongoing improvements of program management, and to meet external accountability through AusAID’s Annual Report and Annual Review of Development Effectiveness (ARDE). The QRS is a major source of information for Annual Program Reports.

The QRS is mandatory for all monitored aid activities. A monitored aid activity is where the expected Australian Government funding over the life of the activity is greater than $3 million, or where the value is less than $3 million but the activity has strategic importance.

The QRS monitors the quality of aid activities at the three stages: design (or entry), implementation and completion. At each of those phases, it reports on the degree to which activities apply internationally recognised characteristics of good aid practice. Reports are subject to peer contestability of findings.

The Quality at Entry process comprises three steps: a concept peer review, an independent appraisal of the design, and an appraisal peer review. A Quality at Entry report, produced following the appraisal peer review, provides ratings of the quality of the design and identifies any actions needed to improve the design before proceeding to implementation.

The Quality at Implementation report is completed at least once a year, typically in preparation for the Annual Performance Report. It provides assessment and ratings on the quality of activity implementation.

The Quality at Completion process is fulfilled through an Independent Completion Report.

Good quality monitoring and evaluation underpins an effective QRS. AusAID staff, contractors, NGOs and partners develop monitoring and evaluation arrangements for each activity.

Activity level evaluations

Evaluations are an important part of learning and accountability about aid effectiveness. Good evaluations consider the following elements of aid activity:

relevance
effectiveness
efficiency
impact
sustainability of benefits and systems
promotion of gender equality

All monitored aid activities are required to conduct regular independent evaluations to complement the QRS. The two primary types of independent evaluations are:

The Independent Progress Report, conducted at the mid-point of implementation for shorter activities, or at regular intervals during implementation of longer activities.
The Independent Completion Report, conducted at the end of the activity implementation.