Monitoring and Evaluation for Results: Lessons From Uganda

Arild O. Hauge, Consultant, Operations Evaluation Department (OED), and Keith Mackay, Senior Evaluation Officer, OED

Uganda has achieved a strong rate of growth and a decrease in absolute poverty in the past decade. With GDP growing at nearly 7 percent a year, the proportion of people living below the national absolute poverty line decreased from 56 percent to 35 percent between 1991–92 and 1999–2000.

Despite remarkable progress in economic management and poverty reduction, however, public service delivery is showing signs of limited effectiveness and value for money.  For instance, between 1995 and 2000, infant mortality increased (from 81 to 88 deaths per 1,000 live births), whereas maternal mortality only fell marginally (from 527 to 505 per 100,000 live births), and immunization coverage declined, despite a major increase in government health sector spending. Similarly, although the total amount of spending on the water sector tripled between 1998 and 2001, the annual number of new water points dropped by 8 percent.

The Government of Uganda (GOU) has recognized that M&E can be critical to achieving a results orientation in its poverty eradication efforts. The national poverty eradication action plan (PEAP), which serves as its poverty reduction strategy paper (PRSP), has as its goal reducing the incidence of poverty to 10 percent by 2017. To coordinate data collection on progress in fighting poverty, the Poverty Monitoring and Analysis Unit was established in the Ministry of Finance, Planning, and Economic Development (MOFPED). This unit has also coordinated formulation of the national Poverty Monitoring and Evaluation Strategy ( MOFPED, 2002). As a result, Uganda now has a fairly robust demographic and household poverty data set, which allows monitoring of national progress against Millennium Development Goals and other development indicators. The major source of data is the regular schedule of surveys by the Uganda Bureau of Statistics, which conducts household income and expenditure surveys every two years, demographic and health surveys every five years, and censuses every decade. This bureau also maintains a district resource endowment profile data base on, for example, topographical characteristics, natural resources, and land use in districts. The Uganda Participatory Poverty Assessment Programme, first undertaken in 1999–2000, provides a qualitative dimension to analysis of poverty by bringing the perspective of the poor into planning. Client satisfaction with public services has been surveyed through a national service delivery survey, for which the bureau is now responsible. 

Government institutions and donors have put much time and effort into developing M&E systems and practices in Uganda. This has culminated in creation of new M&E systems in the areas of accountability, governance, public sector management, and financial management. Attention to M&E has aimed at bringing greater rationality to public finances and development management by providing a more evidence based foundation for policy, budgeting, and operational management. Increased production and public availability of M&E information is also allowing civil society and Parliament to assume a more meaningful role in public affairs. Government has, in fact, managed to bring expenditures under control and maintain macroeconomic stability. By enabling a closer monitoring of expenditures, M&E has also helped facilitate a shift in resource priority given to the social sectors. The sharp rise in the share of the government’s Poverty Action Fund (PAF) expenditures (related to Heavily Indebted Poor Countries, HIPC, debt relief savings) within the national budget from 18 to 37 percent between 1997–98 and 2002–03 has resulted in a major reorientation of national expenditure. In addition, M&E-related initiatives have deepened the performance orientation of PEAP goals by sharpening the focus on objectives and targets.

Several diagnostic reviews of M&E in Uganda— commissioned by the government and a notable illustration of its priority for M&E—have, however, revealed a downside to the growing emphasis on M&E: a proliferation of uncoordinated M&E systems and processes (Hauge, 2001 and 2003, Hauge and others, 2002). These currently include:

Poverty Status Report, PEAP revisions, and PRSP progress reports

Medium Term Expenditure Framework (MTEF) budget planning, commitment control system, and budget performance reports

Twenty-three PAF work planning, and progress reporting formats

Local Government Information and Communication System

Sector working group/Sector-Wide Approach (SWAP) joint reviews

Separate ministerial management information systems

Twenty-five different donor agreements with associated reporting requirements

Development project reporting (Office of the Prime Minister)

Ministry and district physical inspections Accountability investigations: auditor general, inspector general, and treasury inspectorate

Civil society, and community-based monitoring

Ad hoc studies, and researchIntegrated Personnel and Payroll Management System

Results-oriented management implementation reporting.

This proliferation has resulted in duplication, waste, and deterioration of data quality. For example, a review of ministry data collection systems in the health, education, and water and sanitation sectors revealed that Uganda’s district administrations are required to report on more than 2,000 different performance indicators with an average monthly reporting volume of more than 300,000 data entries. Some data records, for example, on program expenditures, are manually entered several times into different systems.

In the health sector, there are complaints that data collection, inspection, and reporting diverts attention from productive service delivery. To follow the National Supervision Guideline of Uganda’s Ministry of Health would involve 1,400 person-years of health professionals’ time. To the extent that capacity development efforts have been tied to donors’ individual rulebooks and requirements, donors have been part of the M&E problem, rather than its solution. Although the tendency of donors to move toward budget support, reducing government transaction costs for donor liaison, is a positive trend, in 2002 forty-two donors still had 524 active projects and entered into 825 separate agreements with GOU.

Existing M&E systems also largely focus on measurement and reporting of expenditures and activities and on poverty, but produce relatively less systematic data on “middle” levels of outputs and outcomes, such as client satisfaction. M&E systems are thus not geared to understanding causality and attribution, that is, the results that follow from individual government or donor development interventions. In-depth reviews or evaluations are rare and usually undertaken by donors in connection with their own projects.

A final point is that incentives are often tied to nominal compliance with formal reporting requirements, rather than underlying performance revealed by M&E. Managers are, in effect, rewarded for good paperwork, not contribution to poverty eradication. Moreover, much inspection activity appears motivated by collection of allowances (Hauge and others, 2002). Mid-level GOU managers can double their income by undertaking frequent “site inspection visits.” Findings of misconduct, however, are rarely sanctioned, and little feedback is provided on information collected through ministerial inspection.

On the positive side, Uganda’s experience with public expenditure tracking studies (PETS) has shown how transparency can be a powerful vehicle for improving service delivery quality. A PETS conducted in 1996 (at a cost of $60,000) showed that only 13 percent of government annual nonsalary expenditures for primary education (capitation grants) reached schools. Since then, public notices on funds allocations in newspapers and radio broadcasts and on notice boards have helped encourage managers to become more responsive to constituency needs and satisfaction. A repeat PETS five years later showed that at least 80 percent of this funding—an increase of more than $18 million a year—reached the intended schools. The government is now conducting PETS on a regular basis and other countries (Chad,

Ghana, Honduras, Mozambique, Papua New Guinea, Peru, Rwanda, Senegal, Tanzania, and Zambia) have also undertaken PETS (World Bank, 2003).

Civil society organizations (CSOs) are also playing an increasingly effective role in planning and review of the budget process, demonstrating the importance of tapping communities in continuous monitoring of public expenditures for proper service delivery. For example, the Uganda Debt Network has established a PAF monitoring committee in each of seventeen districts to monitor progress on activities funded to address poverty reduction. This has contributed to consistent adherence to budget allocations, so resources are used for their intended purposes and in a manner yielding value for public money. Public dialogue meetings have also helped improve community awareness on government decisions on budget resource allocation.

Several major conclusions have emerged from the various diagnostic studies commissioned by the government:

Increased M&E will not by itself lead to a results orientation; what Uganda needs is better, not more, M&E.

Information on the status of poverty and progress with Millennium Development Goals, although important, is not sufficient for understanding the government’s own performance, that is, knowing which of its development interventions do or do not work well and the reasons why.

Civil society can handle some M&E functions better and at lower cost than government, for example, inspection of school facilities by local parentteacher associations.

A key challenge for M&E in Uganda is to link what the country wants to achieve—its national development objectives—with what government actually does, that is, the operational environment of public functions and service delivery. Having recognized that the foundation of results-oriented M&E is clarity of goals, GOU and its civil society and external partners are currently reviewing and revising the PEAP with attention to refining intermediate-term goals and targets at the output and outcome levels that apply to Uganda’s poverty eradication efforts. This will, in turn, provide a basis for further strengthening the results orientation of budget allocations and ministries’ work planning.

To clarify responsibilities in the national M&E arena, Uganda’s Cabinet recently adopted a framework of arrangements for coordination and monitoring within government as a whole. The government has thus recognized the need for addressing M&E in a strategic manner, in particular in terms of the decision making environment that data collection and information management are meant to serve. The key responsibility for interministerial coordination of M&E has been assigned to the Office of the Prime Minister.

At the same time, the PRSC process has become the vehicle for a systems approach to M&E capacity development in Uganda, supporting M&E in three main ways:

Inducing greater reliance on the government’s own M&E systems and practices, rather than multiple donor M&E rules and requirements

Identifying M&E, through the agreed PRSC policy matrix, as a priority in cross-cutting reform, with related changes in civil service pay, budget management, accountability and anticorruption, and engagement with civil society

Allowing policy-level M&E seminars and debates among officials from Uganda’s central and sector ministries, CSOs, and donors to encourage convergence in plans and expectations for strengthening M&E systems. 

In terms of capacity enhancement, the entire PRSC process has emphasized facilitating debate on options and priorities for M&E, especially coordination and rationalization of existing M&E systems and activities within government, rather than provision of technical skills or promotion of additional data collection systems. The PRSC policy matrix has thus become an agenda for results orientation and reform—a “rolling plan” of issues and specific actions to improve national M&E. By April 2003 these actions included:

Establishing an integrated national M&E system with a structure of responsibilities for implementing cross-cutting public management reforms under the entire PRSC Continuing to refine PEAP sector goals and performance targets

Building a common platform for local-level information collection and data entry Consolidating inspection into multidisciplinary teams and have CSOs undertake some tasks Establishing a mechanism for setting an agenda for evaluations, reviews, and ad hoc research, include ing expenditure tracking and value-for-money studies

Adopting findings of the 2003 national service delivery survey to establish baseline performance measures, gauge targeting effectiveness of government service delivery, and enable benchmarking comparisons of performance at different locations and over time

Designing facility-level “client scorecards” managed by CSOs.

An M&E working group comprising representatives of the Office of the Prime Minister and ministries of finance and planning, public service, and local government, is coordinating implementation of PRSC M&E actions.

The Uganda experience dispels the notion that increased M&E activities by themselves will lead to improvements in results orientation. What matters is not so much the data collected or the facts available, but how they are used to inform choices. In Uganda, attention is slowly shifting from activity monitoring to service delivery targeting and outcomes. Cost-effectiveness questions are beginning to be raised and discussed, even if not yet resolved. Growing trans

* This CE Brief was based on OED’s evaluation capacity development work in Uganda.

parency and public participation bring a degree of assurance that government actions are congruent with citizens’ perceptions of need. Bringing managerial attention to poverty outcomes is an ongoing exercise achieving important progress. The government fully appreciates that much unfinished business remains and only sustained effort will strengthen its M&E systems. Its strong commitment to tackle this task, however, provides good grounds for optimism that it will succeed.

Peer Reviewers: Jit Bahadur S. Gill, Sector

Manager, AFTPR, and William Eckert, Senior

Evaluation Officer, WBIEG

Reading

Hauge, Arild. 2001. Strengthening Capacity for Monitoring and Evaluation in Uganda: A Results-Based Management Perspective. ECD Working Paper Series No.8. OED. Washington, D.C.: World Bank.

———. 2003. The Development of Monitoring and Evaluation

Capacities to Improve Government Performance in Uganda. ECD Working Paper Series No.10. OED. Washington, D.C.: World Bank.

Hauge, A., W. Kisembo, M. Makumbi, and O. Omongin. 2002. Effectiveness of Uganda’s PEAP: The Role of M&E in the Health, Education and Water Sectors. Kampala: MOFPED.

Mackay, K. 1999. Evaluation Capacity Development: A Diagnostic Guide and Action Framework. ECD Working Paper No. 6. OED. Washington, D.C.: World Bank.

Ministry of Finance, Planning, and Economic Development.

2002. Poverty Monitoring and Evaluation Strategy. Kampala.

World Bank. 2002. 2002  Annual Report on Evaluation Capacity Development. OED. Washington, D.C.

———. 2003. 2004  Making Services Work for Poor People. World Development Report. Washington, D.C.

Uganda has achieved a strong rate of growth and a decrease in absolute poverty in the past decade. With GDP growing at nearly 7 percent a year, the proportion of people living below the national absolute poverty line decreased from 56 percent to 35 percent between 1991–92 and 1999–2000.

Despite remarkable progress in economic management and poverty reduction, however, public service delivery is showing signs of limited effectiveness and value for money.  For instance, between 1995 and 2000, infant mortality increased (from 81 to 88 deaths per 1,000 live births), whereas maternal mortality only fell marginally (from 527 to 505 per 100,000 live births), and immunization coverage declined, despite a major increase in government health sector spending. Similarly, although the total amount of spending on the water sector tripled between 1998 and 2001, the annual number of new water points dropped by 8 percent.

The Government of Uganda (GOU) has recognized that M&E can be critical to achieving a results orientation in its poverty eradication efforts. The national poverty eradication action plan (PEAP), which serves as its poverty reduction strategy paper (PRSP), has as its goal reducing the incidence of poverty to 10 percent by 2017. To coordinate data collection on progress in fighting poverty, the Poverty Monitoring and Analysis Unit was established in the Ministry of Finance, Planning, and Economic Development (MOFPED). This unit has also coordinated formulation of the national Poverty Monitoring and Evaluation Strategy ( MOFPED, 2002). As a result, Uganda now has a fairly robust demographic and household poverty data set, which allows monitoring of national progress against Millennium Development Goals and other development indicators. The major source of data is the regular schedule of surveys by the Uganda Bureau of Statistics, which conducts household income and expenditure surveys every two years, demographic and health surveys every five years, and censuses every decade. This bureau also maintains a district resource endowment profile data base on, for example, topographical characteristics, natural resources, and land use in districts. The Uganda Participatory Poverty Assessment Programme, first undertaken in 1999–2000, provides a qualitative dimension to analysis of poverty by bringing the perspective of the poor into planning. Client satisfaction with public services has been surveyed through a national service delivery survey, for which the bureau is now responsible. 

Government institutions and donors have put much time and effort into developing M&E systems and practices in Uganda. This has culminated in creation of new M&E systems in the areas of accountability, governance, public sector management, and financial management. Attention to M&E has aimed at bringing greater rationality to public finances and development management by providing a more evidence based foundation for policy, budgeting, and operational management. Increased production and public availability of M&E information is also allowing civil society and Parliament to assume a more meaningful role in public affairs. Government has, in fact, managed to bring expenditures under control and maintain macroeconomic stability. By enabling a closer monitoring of expenditures, M&E has also helped facilitate a shift in resource priority given to the social sectors. The sharp rise in the share of the government’s Poverty Action Fund (PAF) expenditures (related to Heavily Indebted Poor Countries, HIPC, debt relief savings) within the national budget from 18 to 37 percent between 1997–98 and 2002–03 has resulted in a major reorientation of national expenditure. In addition, M&E-related initiatives have deepened the performance orientation of PEAP goals by sharpening the focus on objectives and targets.

Several diagnostic reviews of M&E in Uganda— commissioned by the government and a notable illustration of its priority for M&E—have, however, revealed a downside to the growing emphasis on M&E: a proliferation of uncoordinated M&E systems and processes (Hauge, 2001 and 2003, Hauge and others, 2002). These currently include:

Poverty Status Report, PEAP revisions, and PRSP progress reports

Medium Term Expenditure Framework (MTEF) budget planning, commitment control system, and budget performance reports

Twenty-three PAF work planning, and progress reporting formats

Local Government Information and Communication System

Sector working group/Sector-Wide Approach (SWAP) joint reviews

Separate ministerial management information systems

Twenty-five different donor agreements with associated reporting requirements

Development project reporting (Office of the Prime Minister)

Ministry and district physical inspections Accountability investigations: auditor general, inspector general, and treasury inspectorate

Civil society, and community-based monitoring

Ad hoc studies, and researchIntegrated Personnel and Payroll Management System

Results-oriented management implementation reporting.

This proliferation has resulted in duplication, waste, and deterioration of data quality. For example, a review of ministry data collection systems in the health, education, and water and sanitation sectors revealed that Uganda’s district administrations are required to report on more than 2,000 different performance indicators with an average monthly reporting volume of more than 300,000 data entries. Some data records, for example, on program expenditures, are manually entered several times into different systems.

In the health sector, there are complaints that data collection, inspection, and reporting diverts attention from productive service delivery. To follow the National Supervision Guideline of Uganda’s Ministry of Health would involve 1,400 person-years of health professionals’ time. To the extent that capacity development efforts have been tied to donors’ individual rulebooks and requirements, donors have been part of the M&E problem, rather than its solution. Although the tendency of donors to move toward budget support, reducing government transaction costs for donor liaison, is a positive trend, in 2002 forty-two donors still had 524 active projects and entered into 825 separate agreements with GOU.

Existing M&E systems also largely focus on measurement and reporting of expenditures and activities and on poverty, but produce relatively less systematic data on “middle” levels of outputs and outcomes, such as client satisfaction. M&E systems are thus not geared to understanding causality and attribution, that is, the results that follow from individual government or donor development interventions. In-depth reviews or evaluations are rare and usually undertaken by donors in connection with their own projects.

A final point is that incentives are often tied to nominal compliance with formal reporting requirements, rather than underlying performance revealed by M&E. Managers are, in effect, rewarded for good paperwork, not contribution to poverty eradication. Moreover, much inspection activity appears motivated by collection of allowances (Hauge and others, 2002). Mid-level GOU managers can double their income by undertaking frequent “site inspection visits.” Findings of misconduct, however, are rarely sanctioned, and little feedback is provided on information collected through ministerial inspection.

On the positive side, Uganda’s experience with public expenditure tracking studies (PETS) has shown how transparency can be a powerful vehicle for improving service delivery quality. A PETS conducted in 1996 (at a cost of $60,000) showed that only 13 percent of government annual nonsalary expenditures for primary education (capitation grants) reached schools. Since then, public notices on funds allocations in newspapers and radio broadcasts and on notice boards have helped encourage managers to become more responsive to constituency needs and satisfaction. A repeat PETS five years later showed that at least 80 percent of this funding—an increase of more than $18 million a year—reached the intended schools. The government is now conducting PETS on a regular basis and other countries (Chad,

Ghana, Honduras, Mozambique, Papua New Guinea, Peru, Rwanda, Senegal, Tanzania, and Zambia) have also undertaken PETS (World Bank, 2003).

Civil society organizations (CSOs) are also playing an increasingly effective role in planning and review of the budget process, demonstrating the importance of tapping communities in continuous monitoring of public expenditures for proper service delivery. For example, the Uganda Debt Network has established a PAF monitoring committee in each of seventeen districts to monitor progress on activities funded to address poverty reduction. This has contributed to consistent adherence to budget allocations, so resources are used for their intended purposes and in a manner yielding value for public money. Public dialogue meetings have also helped improve community awareness on government decisions on budget resource allocation.

Several major conclusions have emerged from the various diagnostic studies commissioned by the government:

Increased M&E will not by itself lead to a results orientation; what Uganda needs is better, not more, M&E.

Information on the status of poverty and progress with Millennium Development Goals, although important, is not sufficient for understanding the government’s own performance, that is, knowing which of its development interventions do or do not work well and the reasons why.

Civil society can handle some M&E functions better and at lower cost than government, for example, inspection of school facilities by local parentteacher associations.

A key challenge for M&E in Uganda is to link what the country wants to achieve—its national development objectives—with what government actually does, that is, the operational environment of public functions and service delivery. Having recognized that the foundation of results-oriented M&E is clarity of goals, GOU and its civil society and external partners are currently reviewing and revising the PEAP with attention to refining intermediate-term goals and targets at the output and outcome levels that apply to Uganda’s poverty eradication efforts. This will, in turn, provide a basis for further strengthening the results orientation of budget allocations and ministries’ work planning.

To clarify responsibilities in the national M&E arena, Uganda’s Cabinet recently adopted a framework of arrangements for coordination and monitoring within government as a whole. The government has thus recognized the need for addressing M&E in a strategic manner, in particular in terms of the decision making environment that data collection and information management are meant to serve. The key responsibility for interministerial coordination of M&E has been assigned to the Office of the Prime Minister.

At the same time, the PRSC process has become the vehicle for a systems approach to M&E capacity development in Uganda, supporting M&E in three main ways:

Inducing greater reliance on the government’s own M&E systems and practices, rather than multiple donor M&E rules and requirements

Identifying M&E, through the agreed PRSC policy matrix, as a priority in cross-cutting reform, with related changes in civil service pay, budget management, accountability and anticorruption, and engagement with civil society

Allowing policy-level M&E seminars and debates among officials from Uganda’s central and sector ministries, CSOs, and donors to encourage convergence in plans and expectations for strengthening M&E systems. 

In terms of capacity enhancement, the entire PRSC process has emphasized facilitating debate on options and priorities for M&E, especially coordination and rationalization of existing M&E systems and activities within government, rather than provision of technical skills or promotion of additional data collection systems. The PRSC policy matrix has thus become an agenda for results orientation and reform—a “rolling plan” of issues and specific actions to improve national M&E. By April 2003 these actions included:

Establishing an integrated national M&E system with a structure of responsibilities for implementing cross-cutting public management reforms under the entire PRSC Continuing to refine PEAP sector goals and performance targets

Building a common platform for local-level information collection and data entry Consolidating inspection into multidisciplinary teams and have CSOs undertake some tasks Establishing a mechanism for setting an agenda for evaluations, reviews, and ad hoc research, include ing expenditure tracking and value-for-money studies

Adopting findings of the 2003 national service delivery survey to establish baseline performance measures, gauge targeting effectiveness of government service delivery, and enable benchmarking comparisons of performance at different locations and over time

Designing facility-level “client scorecards” managed by CSOs.

An M&E working group comprising representatives of the Office of the Prime Minister and ministries of finance and planning, public service, and local government, is coordinating implementation of PRSC M&E actions.

The Uganda experience dispels the notion that increased M&E activities by themselves will lead to improvements in results orientation. What matters is not so much the data collected or the facts available, but how they are used to inform choices. In Uganda, attention is slowly shifting from activity monitoring to service delivery targeting and outcomes. Cost-effectiveness questions are beginning to be raised and discussed, even if not yet resolved. Growing trans

* This CE Brief was based on OED’s evaluation capacity development work in Uganda.

parency and public participation bring a degree of assurance that government actions are congruent with citizens’ perceptions of need. Bringing managerial attention to poverty outcomes is an ongoing exercise achieving important progress. The government fully appreciates that much unfinished business remains and only sustained effort will strengthen its M&E systems. Its strong commitment to tackle this task, however, provides good grounds for optimism that it will succeed.

Peer Reviewers: Jit Bahadur S. Gill, Sector

Manager, AFTPR, and William Eckert, Senior

Evaluation Officer, WBIEG

Reading

Hauge, Arild. 2001. Strengthening Capacity for Monitoring and Evaluation in Uganda: A Results-Based Management Perspective. ECD Working Paper Series No.8. OED. Washington, D.C.: World Bank.

———. 2003. The Development of Monitoring and Evaluation

Capacities to Improve Government Performance in Uganda. ECD Working Paper Series No.10. OED. Washington, D.C.: World Bank.

Hauge, A., W. Kisembo, M. Makumbi, and O. Omongin. 2002. Effectiveness of Uganda’s PEAP: The Role of M&E in the Health, Education and Water Sectors. Kampala: MOFPED.

Mackay, K. 1999. Evaluation Capacity Development: A Diagnostic Guide and Action Framework. ECD Working Paper No. 6. OED. Washington, D.C.: World Bank.

Ministry of Finance, Planning, and Economic Development.

2002. Poverty Monitoring and Evaluation Strategy. Kampala.

World Bank. 2002. 2002  Annual Report on Evaluation Capacity Development. OED. Washington, D.C.

———. 2003. 2004  Making Services Work for Poor People. World Development Report. Washington, D.C.

Leave a Reply

Your email address will not be published. Required fields are marked *