The Framework
Part of Toolkit for the Economic Evaluation of World Bank Transport Projects
(Institute for Transport Studies, University of Leeds, 2003)
1 Introduction
Transport appraisal is the assessment of value for money of transport projects and policies. This statement implies many questions; assessment by whom, for whom, from what perspective, at what stage. One of the features of transport decisions is that they typically impact on many parties – transport operators, individual transport users, local residents and businesses, land and property owners, national and local taxpayers. Each of these stakeholders will seek to assess the impact of a project from the perspective of his/her own interest. But the overarching perspective of transport appraisal needs to be a social one, that is, one which takes account of significant impacts of the project or policy whoever is affected.
So the key question which appraisal seeks to address is:-
· Is a project or a policy intervention worthwhile from an overall social point of view?
Other important questions which appraisal should inform are
· What is the pattern of gains and losses? Do the benefits and costs accrue to the rich or the poor? What is the impact on identified social groups?
· Is the project financially sustainable? Is there a revenue stream to maintain the asset? How will the project actually be funded and how will the debt be repaid?
· Is the project practicable? Are there social or technical barriers to implementation? “Will it fly”?
This Toolkit on transport economic appraisal sits at an intermediate level within the hierarchy of World Bank advice. The Operational Manual forms the highest level of advice and contains multi-sectoral operational statements, including statements on economic appraisal [[1]]. Additionally, The Handbook on Economic Analysis of Investment Operations (World Bank, 1998) [[2]] contains more detailed multi-sectoral advice specifically focussing on economic appraisal. This transport sector appraisal guidance therefore sits below the Operational Manual and the ‘Handbook’ and as such needs to conform to the general principles set out in those documents.
This Toolkit does, however, sit above detailed advice regarding project specific problems, case study examples or software specific advice, such as those contained within the Infrastructure Notes [[3]] or the Highway Development and Management suite of software (HDM4) [[4]].
The Toolkit consists of:
- This Framework document;
- A checklist;
- Twenty Notes on specific topics; and
- A paper on the assessment of distributive and poverty impacts.
The notes refer to many sources and references on the treatment of particular issues at a more detailed level. Many of these references are World Bank staff papers. Whenever possible links have been provided, however, as the currency of these links may be limited the sources are referenced fully.
2 TRANSPORT APPRAISAL WITHIN THE POLICY PROCESS
Transport appraisal lies at the interface between technical work (engineering, planning, economics) and political decision-making. This is a pivotal position which requires not only that appraisal should be technically sound, but also that it must be capable of explanation and communication to the decision-makers and map on to their information needs. It is worth noting two features of transport decisions which are sometimes neglected in academic discussions. In practice there are likely to be several parties involved in the decision process for a major project. Different parties may view elements of the appraisal differently, but the appraisal should be a common reservoir of information on which the parties can draw. It is unrealistic and wrong to think of the appraisal as usurping the function of the decision-takers; rather it should provide them with the raw material.
Secondly, it is important to stress the process nature of transport appraisal. Several stages in the process of scheme/policy development may be identified.
- the initial definition of a project or policy for feasibility study
- sifting/screening from a large number of possible projects or project options to a manageable set of alternatives for full appraisal
- project selection including accept/reject, choices between alternatives and prioritisation.
Clearly, at the initial stage of project conception and also at the sifting/screening stage, it will not be cost-effective to develop the appraisal to the level needed for project selection. But the indicators used at the earlier stages of the project cycle should relate to the criteria by which the final decision will be taken. Even if the signpost is a very simple one, it should be pointing in the right direction.
Characteristics of Transport Appraisal
Some characteristics of transport appraisal may be noted:-
Appraisal is a comparative tool. It considers the difference between alternative states of the world (such as Do-Something against Do-Minimum) and the cost and benefits of a project or policy intervention. A Do-Something scenario is one in which the project/policy is included in the transport system. A separate Do-Something scenario is required for each alternative tested. The Do-minimum scenario needs to be a realistic base case against which the project/policy options are assessed therefore appropriate definition and selection of the alternatives for consideration is crucial.
Appraisal relies on data, modelling and forecasting. Without the straw of basic data on demand and supply, an economic model (which may be very simple) and a way of rolling forward from the base year into the future, the bricks of the appraisal cannot be made. The phrase ‘garbage in, garbage out’ applies, in that if the economic appraisal is based on weak data inputs then the appraisal itself will be weak.
Appraisal should reflect human behaviour and be evidence-based as far as is reasonably possible. So knowledge of the factors which drive behaviour, and the way in which transport improvements are likely to impact is important. In practice there is a trade-off between the cost and time to acquire data and the need for local evidence to support the appraisal. In any case, local data on key parameters such as the values of time will need to be benchmarked against wider evidence reviewed here.
Appraisal needs to be holistic in nature, that is to say, it needs to cover economic, social and environmental impacts of projects and policies in a coherent and consistent manner, and therefore
Appraisal needs to respect capital budgeting rules, dealing with costs and benefits over the life of the project/policy and representing risk and uncertainty.
Appraisal needs to be in scale with the size of the project and the risks involved. The economic appraisal treatment of a new bridge or tunnel across an estuary needs to be more sophisticated than the appraisal of the re-design of a junction because more public resources are at stake the risks are greater, and the effects are likely to be wider-reaching. The estuary crossing probably requires a purpose-built transportation and economic model, while the junction re-design problems might rely on standardised data and forecasting procedures.
Critiques of Transport Appraisal
Before looking in more depth at the technical aspects, let us consider briefly some of the main critiques of transport cost-benefit analysis.
Critique 1 “It doesn’t relate to country economic growth objectives in a clear manner”.
Response It is true that transport cost-benefit analysis does not provide a direct proxy for the impact of the scheme on GDP. This is partly because the appraisal includes impacts on leisure time, safety and environment which do not figure in GDP. But also, transport cost-benefit analysis relies on measuring the costs and benefits to transport users and providers. In an ideal world, we would like to conduct a multi-sector economic analysis of transport projects, measuring the benefits as changes in prices and wages across the economy. In practice this is too ambitious. A well conducted partial analysis of the project impacts in the transport market will in general provide a good indicator of the aggregate economic benefits of the project. In some cases, where the project is expected to stimulate structural economic changes, supplementary analysis may be required (see Notes Projects with Significant Expected Re-structuring Effects [Link] and Low Volume Rural Roads [Link]).
Critique 2 “It doesn’t relate to distributive concerns or to poverty alleviation”.
Response This is a valid criticism of approaches which focus exclusively on the economic rate of return. This Toolkit therefore proposes methods for assessing the distributive impacts of projects (see Note Distribution of Benefits and Impacts on Poor People [Link]). However it should be recognised that measuring who gains and who loses relies on many economic assumptions and is, in general, an order of magnitude more difficult than measuring the size of the aggregate benefits.
Critique 3 “To do it properly requires far too much data and analysis”.
Response Clearly there is a need to consider carefully the cost-effectiveness of appraisal resources, and to relate the level of analysis to the costs and risks of wrong decisions. As discussed above the appraisal needs to be in scale with the size of the project. In general, economic appraisal consumes only a small part of the total resources involved in project planning, and a very small fraction of the total costs of the projects. It is considered that there is a high payoff to good quality data and economic analysis provided the evidence is used in an open-minded and constructive manner within the decision-making process.
Critique 4 “It comes too late in the project cycle to have a real influence”.
Response If appraisal is used as a final check on whether a hurdle has been jumped than its value will be limited. Appraisal needs to be an integral part of the planning cycle, relevant to screening, prioritisation ranking, and design options. If appraisal is used narrowly – ‘does a scheme pass the hurdle rate’ – rather than broadly – ‘what is the best use of resource among alternatives’ – it loses much of its potential.
Transport Cost Benefit Analysis
While all the above commentaries have some validity, overall, there is a strong case for a good quality assessment of the impacts which projects are expected to have and of the economic value of those impacts.
So, appraisal needs to be fit for purpose:
- Timely and cost-effective in relation to the resources at stake;
- Provide evidence on project impacts on society as a whole and from the perspective of individual agents or social groups; and
- Support the decision against the key tests of social value for money, financial sustainability and practicability.
The form of appraisal which best meets these requirements is a transport cost-benefit analysis and this Toolkit reflects that. In most applications, the main focus of the appraisal should be on the impacts on transport users and operators and taxpayers.
The core aim of transport appraisal must be to get a good measure of the primary impacts of the project on travel patterns, journey times and costs, and hence on user benefits, revenues and costs. This needs to be complemented by an environmental assessment of transport infrastructure and operations. Consideration of wider impacts on the economy and society may be required in particular cases (e.g. project with significant ‘opening-up’ or generative effects). The appraisal process will include the following stages:
· The initial definition of a project or policy for feasibility study;
· Sifting/screening from a large number of possible projects or project options to a manageable set of alternatives for full appraisal; and
· Project selection including accept/reject, choices between alternatives and prioritisation.
The Transport Appraisal Process within the World Bank
Gwilliam (2000) [[5]] sets out the manner that economic appraisal integrates in to the policy decision process within the World Bank.
As stated within the Handbook on Economic Analysis of Investment Operations (World Bank, 1998) [2] the main purpose of project economic analysis is to help design and select projects that contribute to the welfare of a country. It is most useful when used early in the project cycle, and has very limited use when used solely as a single figure hoop through which projects must jump once prepared. The handbook lists ten questions that an economic analysis should answer, namely:
(i) What is the objective of the project?
(ii) What will happen if it is implemented, and what if it is not?
(iii) Is the project the best alternative?
(iv) Are there any separable components, and how good are they separately?
(v) Who are the winners and losers?
(vi) Is the project financially sustainable?
(vii) What is the project’s fiscal impact?
(viii) What is the project’s environmental impact?
(ix) Is the project worthwhile?
(x) Is this a risky project?
In addition, it is necessary to analyse, in economic terms, whether the project is better carried out in the public or private sector.
The first step in the appraisal process is the development of the Project Concept Note (PCN). Such a note has the following functions:
· To examine the strategic rationale for Bank involvement;
· To promote consideration of alternative project concepts;
· To seek a go/no-go decision from Country Director;
· To obtain early guidance/agreement on issues and approach;
· To flag risks and potential mitigation measures;
· To seek early guidance on potential safeguard issues, consultation, and disclosure; and
· To agree on a resource estimate, schedule, and team.
The key features of the PCN are:
· It is very short—a maximum of four pages;
· It focuses on project concept, not design; and
· It is prepared immediately after identification or before expenditures exceed $30,000
If the general principles on transport appraisal, as set out in this section, were fully observed it is at this stage that formal economic evaluation would play its fullest role. In practice the economic evaluation at this stage tends to be at a sketch model level. The full economic justification, including that of the selection between alternative actions, has to be more fully worked up for inclusion in the Project Appraisal Document (PAD). It is within this document that the ten questions that are identified within the Handbook are formally addressed. Thus a PAD indicates the way in which project objectives, project components and outcomes are related, identifies the main perceived risks and the elements of the project design to mitigate such risks. A summary of the full economic evaluation is included.
3 Transport Cost Benefit Analysis
Overview
A key aim of an economic appraisal of a transport project is to measure the magnitude of the economic impact resulting from the investment. Ideally this would measure the total benefits from increased output across all final product sectors and would include measurement of the level of employment and the wage rate in the labour market, the prices of goods and services in the product market and the value of property in the land market. In practice, however, the analytical models required to undertake such an analysis require a level of sophistication and refinement that is typically beyond that available from both technical and resource standpoints. As such the cost benefit analysis process is based around a partial equilibrium approach[*] that concentrates on the “primary” impacts incurred by transport users, operators and governments. The basic calculation is summarised below:
|
Overall Economic Impact |
= |
Change in transport user benefits (Consumer Surplus) |
+ |
Change in system operating costs and revenues (Producer Surplus and Government impacts) |
+ |
Change in costs of externalities (Environmental costs, accidents, etc.) |
- |
Investment costs (including mitigation measures) |
This apparently simple calculation can in fact become a quite complex exercise as it becomes necessary to consider:
· The scope of the appraisal in terms of mode, study area and range of impacts [Link];
· The calculation of transport user benefits (consumer surplus)[Link];
· The calculation of impacts on transport providers and the government (includes producer surplus and investment costs) [Link];
· Monetary valuation of time and safety [Link];
· The treatment of environmental impacts and other externalities [Link].
· The mechanics of the process including inputs, project life, discounting, aggregation of benefits and costs, unit of account [Link].
Box 1 summarises the steps involved in carrying out the cost-benefit analysis for transport infrastructure projects and illustrates that it comprises of a number of distinct stages and that a range of internal inputs are required. The rest of this Framework document is structured around the elements in Box 1, and many of the Notes clearly link to this structure.
Box 1: Transport Economic Appraisal
Scope of CBA
At the start of the CBA process, a view will need to be taken on the scope of the analysis. This is often made simultaneously with the decision regarding the type and scale of the demand forecasting approach, as the two processes are inter-related. Ideally, the CBA process should include all impacts of the investment, no matter how small that impact is. However, setting such a broad scope for a CBA will result in extensive data collection and analysis that may well be expensive in terms of cost but also in terms of time required to complete, both will affect the ability to deliver the project. Given that the purpose of the CBA is to firstly ensure that a project is economically beneficial and secondly to aid the choice between alternatives, the scope of the CBA is in practice often narrowed by excluding minor or insignificant impacts as long as the exclusion of these impacts will not bias the appraisal. Key issues that require addressing in defining the scope of a CBA include:
Impacts: the measurement of changes in producer and consumer surplus requires the measurement of benefits, revenues and costs to transport operators and users. At a minimum these should include the investment cost and changes in infrastructure and system maintenance and operating costs, vehicle operating costs, journey times, safety, user charges and operator revenues.
Mode of transport: typically the modes of transport that are considered should include both those that will use the new infrastructure (e.g. a road) and those from which demand may be abstracted (e.g. rail). In addition, slow modes (Non Motorised Transport (NMT) and pedestrians) should be considered where impacts on them will be significant as in the case of low volume rural roads. The Note on Treatment of Pedestrians and NMTs discusses slow mode issues in more detail [Link].
Study area: should be the smallest area that allows for the development of robust results. It should therefore be large enough to capture network effects that include firstly the abstraction of demand from other routes and modes and secondly the impact of competing and complementary schemes that in combination with the project in question may comprise the country’s development strategy. If cross-border impacts are expected (e.g. from transit traffic associated with land locked countries) then the study area should be defined so as to incorporate both domestic and international travel.
Transport User Benefits
The essential measure of benefits to users is consumer surplus, that is, the excess of consumer willingness to pay over the cost of a trip. Normally, we are interested in the change in consumer surplus resulting from some change in the cost of travel brought about by an improvement in transport conditions. Operationalising this in transport poses some practical problems. For most consumer goods the cost of the good (to the consumer) is its price. When it comes to transport, prices and money costs are only a proportion of the composite cost of travel, which in principle also incorporates the time spent by the individual, access times to public transport, discomfort, perceived safety risk and other elements. Therefore price alone is not an appropriate measure of either the cost of travel or the consumer’s WTP, instead generalised cost is used. Generalised cost is an amount of money representing the overall cost and inconvenience to the transport user of travelling between a particular origin and destination by a particular mode. In practice, generalised cost is usually limited to a number of impacts which when summed comprise the components of user benefit:
(i) Time costs (Time in minutes * Value of Time in $/minute);
(ii) User charges (e.g. fares/tolls); and
(iii) Operating costs for private vehicles (VOCs), Non-Motorised Traffic (NMT) and pedestrians.
It is important to note that the components of generalised cost tend to vary by mode. Public transport users (bus, coach, train, air and ferry) will pay a money fare and give up time in order to travel to their destination. Car users and own-account freight users give up time, may be asked to pay an infrastructure access charge or toll, and pay for their own fuel and VOCs. Therefore there is a fundamental difference in the reported user benefits for users of different modes. Additionally, it is important to recognise that Values of Time vary between individuals and even for the same individual, depending on for example trip purpose. There is no unique willingness-to-pay for travel time savings. This has consequences for modelling and appraisal, especially for toll roads or urban mass transit, where suitable market segmentation is needed.
Box 2 describes the concept of consumer surplus measure of user benefits. The light shaded area in Figure 1 is known as the Rule of a Half measure of user benefits, for reasons discussed in Box 2. The Note Treatment of Induced Traffic [Link] discusses in more detail the application of the Rule of a Half for the measurement of user benefits of Base and Generated traffic as well as its application to networks. The Note also discusses the conditions under which the Rule of a Half breaks down and gives advice in those circumstances. The Rule of a Half can also be applied separately to each of the user benefit impacts to provide a disaggregation by time, money costs and user operating cost savings. Such a disaggregation will most likely be required for presentation of the cost benefit analysis results, as discussed below.
Clearly the computation of generalised cost needs a calculation of the benefits of time savings, the valuation of which are discussed below. The computation of transport user operating costs is discussed in detail in the Note Sources of Operating Costs [Link], and for slow modes in particular in the Note Treatment of Pedestrians and NMTs [Link].
Box 2: Consumer Surplus AND THE RULE OF A HALF
|
In Figure 1 an improvement
in transport supply conditions, such as an investment in the road
infrastructure between locations i and j is shown. The fall in transport costs have effects on two groups of
users: (i)
Existing users – these gain the benefit of the cost change (C0
– C1) each, or area C0AEC1. (ii)
New users – these gain a benefit equal to the excess of their
willingness to pay over their cost of travel, or area ABE. User benefits are the sum
of (i) and (ii) and can be written: (C0 – C1) T0 + ½(C0 – C1)(T1
– T0) or ½ C0 – C1)
(T0 +T1) This is rule of a half measure of user
benefits. It can be extended to
treatments of networks with simultaneous changes in costs on links and across
modes.
The
rule of a half formula assumes the demand curve is linear between points A
and B. Therefore, it is only an
approximation to the true benefit – the more convex (or concave) the demand
curve, and the larger the cost charge, the less accurate the approximation
will be. Given the many sources of
error in practical appraisal work, the rule of a half is considered
acceptable except in cases such as estuary or mountain crossings, where cost
changes may be considered “large” relative to base cost levels. |
Impacts on Transport Providers and Government
Although the user benefit analysis will often be the most testing part of the cost-benefit analysis, it needs to be undertaken alongside an analysis of revenues and costs which impact on both the transport providers and the government.
PRODUCER SURPLUS
Cost-benefit analysis is concerned not only with consumer surplus, but with total social surplus. This includes producer surplus (PS) as well as consumer surplus. The greatest scope for changes in producer surplus arises from public transport projects or toll road projects, which can affect operators’ revenues without having an equal and offsetting effect on operating costs. Producer surplus is defined simply as total revenue (TR) minus total costs (TC):
PS = TR-TC and therefore ΔPS = ΔTR - ΔTC
It should, however, be noted that there is an implicit assumption here that if the additional demand for this service is associated with reduced consumption of some other goods or services elsewhere in the economy, those goods and services are being priced at marginal cost, so that there is no offsetting or additional change in producer surplus elsewhere. This assumption is a facet of the partial equilibrium approach adopted, as discussed earlier, and whilst usually made is worth making explicit in the interests of transparency.
Revenue forecasts depend on traffic forecasts, and both depend on pricing policy. Therefore it is essential in appraisal that the price policy assumptions on which the traffic and benefit estimates are based are consistent with those used for revenue forecasting. The size of the revenue and user benefit effects, as well as their distribution, depends upon the pricing policy. Although this seems obvious, in many practical situations, the appraisal may be undertaken before the details of the toll or price regime have been finalised, so that provisional assumptions made for the appraisal can turn out to be wide of the mark.
Revenue forecasts will be needed both for the Cost Benefit Analysis and for the assessment of financial sustainability of projects. There may be a trade off in tariff-setting between the desire to maximise social benefits from the project and the imperative to satisfy budgetary constraints. This is discussed further in the Note Evaluation Implications of Sub-Optimum Pricing [Link].
MAINTENANCE AND SYSTEM OPERATING COSTS
Proper estimates are also needed of the costs of operating both the infrastructure and the services, which are mode and country-specific. The main items will typically be:
· The costs of infrastructure operation (e.g. signalling/traffic control);
· Maintenance costs (cleaning, minor repairs, winter servicing);
· Costs of renewals (road/rail reconstruction); and
· Changes in the vehicle operating costs of public transport services.
Maintenance costs from an important component of the definition of the Do Minimum scenario (the without project scenario) and should always be included in the definition of that scenario.
Additionally, any disruption to transport users that occurs during periods of routine maintenance should be reflected in the appraisal as a user benefit impact. In situations of regulatory reform (such as the commercialisation of railways or port operations) it maybe expected that system operating costs may differ significantly in the do-something situation compared to the do-minimum, which should be reflected in the appraisal. These issues as well as general issues associated with the calculation of maintenance costs and system operating costs are discussed in detail in the Notes Treatment of Maintenance [Link], Sources of Operating Costs [Link] and Economic Appraisal of Regulatory Reform – Checklist of Issues [Link].
TAXATION AND GOVERNMENT REVENUE EFFECTS
When a project leads to a shift in demand between private and public transport, the implications for government tax revenue may be significant because private transport is often relatively heavily taxed and public transport is often relatively lightly taxed. These changes in indirect tax revenue to the government should be shown in the cost-benefit outputs.
Such a project is also likely to involve changes in operating costs for private transport and opposing changes in operating costs for public transport. These changes in costs and revenues need to be shown explicitly in the appraisal.
The Note Fiscal Impacts [Link] discusses in detail the range of impacts that will be felt by governments and presents an example of how these should be included in the project appraisal.
Typically the investment costs for transport infrastructure projects will be derived from engineering design studies and estimates. A number of adjustments, however, may have to be applied to these engineering cost estimates before they can be included in the appraisal. These adjustments are as follows:
· The cost of mitigation measures required by the Environmental Assessment and Resettlement Action Plan must be included (see the section on environmental impacts below);
· Conversion of the engineering costs to the correct price base and unit of account (discussed in section on Cost Benefit Analysis Parameters below). This will include adjustments for:
· Taxation (including import duties);
· Inflation (between year of the engineering cost estimate and price base of the appraisal);
· Shadow price of foreign exchange;
· Shadow price of labour (if there is significant underemployment of unskilled labour or severe shortages of skilled labour);
· Shadow price of public capital (if there is a constraint on public capital)
No adjustment is made for the manner in which the project is financed. That is the investment costs used in the appraisal are the same whether or not the project is financed directly by the government or financed through some form of private sector involvement (such as through public private partnerships (PPP) or ‘lease-back’ arrangements). This is because the costs of financing under PPP or ‘lease back arrangements represent a transfer payment from the public to the private sector. As a transfer payment, the profit element of the costs of financing the project will not affect the project’s overall economic value (Net Present Value). However, the method of financing will have financial and distributive impacts. It is therefore important that a financial appraisal, comparing the costs and revenues of procuring the project by different methods, is undertaken. This is discussed in the section on financial appraisal below.
Additionally, whilst not forming part of the investment costs, it is important that user benefits reflect any travel time and cost delays during construction. In the case of public transport projects, bitter experience suggests a need to allow for the effect on traffic and revenues of any disruptions to existing service quality while new schemes are constructed. The treatment of user delays during construction within the appraisal is similar to the treatment of user delays during routine maintenance, which is discussed in the maintenance section above.
For non-infrastructure projects, such as a bus fleet replacement programme or regulation reform programme the ‘set up’ costs of the programme maybe either included in the stream of system operating costs or separately identified as set up investment costs. If the initial investment is large as may occur with a fleet replacement programme then the set up costs are often identified as an investment cost rather than as part of the system operating costs. Again it is worth emphasising that it is the engineering costs that are of importance and not the financial implications of the manner that the project is financed.
Money Values of Time and Safety
Whilst some benefits and costs within the appraisal are naturally valued in money terms - vehicle operating costs, investment costs for example – others are not and values need to be inferred from relevant evidence. Market research studies are a major source of values based on ‘stated preference’. Values can also be obtained from revealed preference, that is, data on actual travel behaviour. In principle these values should cover all dimensions of consumers’ willingness-to-pay for transport services, including comfort and reliability. In practice most work has focussed on travel time savings and safety improvements.
VALUES OF TIME
Attaching a value to time spent travelling is a matter of considerable complexity, however, it is also extremely important for appraising transport infrastructure investments. When valuing time a basic distinction which is usually made between:
· Travel in working time (or “on employer’s business”);
· Travel in non-working time (usually defined to include all other travel purposes, such as shopping, commuting, education, personal business and leisure); and
· Freight travel time.
Such a distinction between travel time categories is clear within industrialised countries and in urban areas (both in the developing and industrialised world), but can become blurred in rural areas of developing countries. This is because often in rural areas of developing countries:
· Productive economic activities include tasks associated with subsistence as well as the cash economy
· A single trip may be made to fulfil several productive tasks; and
· Many trips are made for a mixture of productive and social purposes.
The Note Valuation of Time Savings [Link] discusses the above issues in detail and outlines the approach that should be adopted for valuing time.
CALCULATION AND VALUATION OF SAFETY BENEFITS
Accidents, particularly in developing countries, are a leading cause of death and bring about significant social and economic costs. Valuing safety issues is therefore important within the appraisal. A separate Note, Valuation of Accident Reduction [Link], describes in detail the manner that accidents should be valued, however, it is worth outlining here a few points of principle.
By convention, safety is treated differently from the other components of user benefit. Rather than being included as a component of generalised cost per trip, accidents and casualties are typically treated as random, occasional costs arising from the transport system, which can be evaluated by applying unit values per accident and per casualty, to forecast data on accident and casualty numbers by mode. The calculation is a simple multiplication of forecast accident numbers (by severity) with the costs of accidents (by severity). This treatment is akin to that of externalities (e.g. the environment (see below).
When placing a value on safety, it is important to include both the physical or direct costs of an accident (e.g. damage to property and insurance costs) and casualty related costs (e.g. lost output and human costs). Direct accident costs should be based on actual costs incurred or inferred from the analysis of markets, whilst casualty costs should ideally be based on the Willingness To Pay (WTP) approach, but may be derived from income data if WTP evidence is unavailable and cannot feasibly be gathered within the project. The methodology is explained in the accompanying Note Valuation of Accident Reduction [Link].
Impacts on the Environment
The environment of a transport project includes the surrounding objects and conditions (natural and man made) as well as the circumstances of human society in that area. This is a broad definition and includes amongst others:
· Health and safety impacts (i.e. the human costs of pollution and accidents, the latter of which has been discussed above);
· Involuntary re-settlement;
· Impacts on indigenous people;
· Impacts on the natural environment; and
· Impacts on the man made environment (e.g. cultural heritage).
The objective of transport investment projects is to improve the social and economic welfare of those that will be affected by the project, and as part of this projects are required to adhere to the principle of ‘no detriment’. As such an economic appraisal should only be undertaken for projects that satisfy the World Bank’s procedures on Environmental Assessment and Involuntary Resettlement. Mitigation and compensation costs must be included in the project costs. Furthermore, where transport projects act to improve the environment, this should be recognised, always through the Environmental Assessment or Resettlement Action Plan, and – where technically feasible – through monetary valuation of the benefits.
It must be recognised, however, that the current state of the art regarding the valuation of environmental impacts is such that even in the developed world, where data and resources are more readily available than in the developing world, these impacts are handled via a mixture of qualitative, physical and monetary measures. Therefore within the economic appraisal it is considered that the key objectives with respect to environmental impacts are to:
· Firstly, ensure that all environmental impacts are considered within the Environmental Assessment; and
· Secondly, to assign an economic cost to quantified impacts if that cost represents a reasonable proxy for the Total Economic Value.
The treatment of environmental impacts, including their valuation, and issues associated with involuntary resettlement are discussed in detail in the Notes Treatment of Environmental Impacts [Link] and Evaluation of Resettlement Compensation Payments [Link].
Cost Benefit Analysis Parameters
So far the first six boxes of the CBA process detailed in Box 1 have been dealt with. These together give us the investment costs and the operating costs, revenues and user benefits for the base year. This section of the Framework addresses issues associated with moving consistently from a single year to the valuation of the project over its whole life. This involves explicit treatment of forecasting growth over time, numbers of forecast years, accounting issues such as the treatment of inflation and the unit of account, and capital budgeting issues such as discounting. Finally, the summary indicators of value for money need to be briefly considered.
Typical values for the CBA parameters are contained in Box 3.
Box 3: Typical COST BENEFIT ANALYSIS PARAMETER VALUES
|
Parameter |
Value |
|
Project start year (when works begin) |
Project specific |
|
Project opening year (first full year of
operation) |
Project specific |
|
Design Year |
Project specific |
|
Number of forecast years from which CBA is
undertaken |
Minimum of 2 (e.g. Opening
Year and Design Year) |
|
25 years (from and
including opening year) |
|
|
Discount rate |
12% |
|
Real or Nominal |
Real |
|
Unit of Account |
Factor prices |
|
Use of Shadow Prices |
Shadow price of Labour Shadow price of Foreign
Exchange Shadow price of capital |
The key inputs to a CBA are trip, cost and time matrices (for origins i to destinations j by modes m) derived from from the modelling and forecasting process; estimates of the investment costs, changes in infrastructure and service operating costs; unit values; estimates of the numbers of accidents and casualties and environmental impacts (chiefly noise and air pollution changes). With the exception of the modelling and forecasting process, all these inputs have been discussed previously.
The Note Demand Forecasting Errors [Link] discusses the principal issues of the demand forecasting process that impact on the CBA. These include:
· The size of the study area;
· The modes of transport available to users within the model and behavioural responses users are permitted to make (e.g. change in destination and trip generation);
· Assumptions regarding background economic growth, population change, employment change and car ownership;
· The need for the model to be validated against observed data; and
· The need for realistic assumptions regarding the nature of the proposed transport project (e.g. anticipated journey times and fares on a new rail link, or system operating cost reductions brought about by regulatory reform).
Additionally, demand forecasts should be undertaken for a minimum of two years – opening year (defined as the first full year of operation) and the design year which should be chosen taking account of macro-economic forecasts and other data (typically around the 10th year of operation). The opening year is required to check that the project is worth undertaking now. The design year is required to check that the design is appropriate for the forecast volume of traffic. Both are required in order to establish the benefit and cost streams over the appraisal period.
PRICES AND THE UNIT OF ACCOUNT
It is important to undertake the appraisal as a whole on a consistent basis and fundamental to that is a consistent treatment of prices. In economies where distortions are few, market prices provide a reasonably good approximation to the opportunity costs of inputs and outputs. Unfortunately, such situations are rare as market distortions occur through for example taxation, fixed foreign exchange rates and rigidities in markets (e.g. the labour market). Ideally, before an economic appraisal all prices should be adjusted to compensate for price distortions, though in practice only a few adjustments are undertaken. The most important adjustments concern the prices of tradeable goods, the exchange rate and the wage rate. This issue is pertinent across all sectors, not just transport, and detailed appraisal advice is provided in the Handbook on Economic Analysis of Investment Operations (Chapters 3 and 6) (World Bank, 1998) [2]. It is, however, worth outlining a few key points here.
With respect to pricing and the unit of account the analyst faces a number of key choices:
(i) Whether to use the domestic currency or a foreign currency (e.g. US dollars);
(ii) Whether to use factor or market prices; and
(iii) Whether to use shadow prices for labour, foreign exchange and capital[†].
Additionally, all prices should be real prices rather than nominal prices (i.e. general price inflation should be stripped out) and if growth rates in real prices cannot be accurately forecast then constant prices should be used.
Typically, appraisals are undertaken in factor prices in real terms (as indicated in Box 3), implying the necessity for the following price adjustments:
· Market prices for domestically traded goods adjusted to be net of taxes or subsidies;
· Border prices for internationally traded goods adjusted by the shadow price of foreign exchange; and
· Wage rates for local labour adjusted by the shadow price of labour if there is significant underemployment of unskilled labour or severe shortages of skilled labour.
In appraisal, the aim is to capture the full economic benefits of the project and to do this one would appraise over the expected life of the longest lived asset. In practice, the choice of the appraisal period becomes a matter of judgement for two principle reasons: the fact that projects create a mixed set of assets, some of which will require replacement before the end of the appraisal period; and by the fact that the appraisal period may also be limited by the time period over which demand can be confidently foreseen. For most transport infrastructure related appraisals a 25 year appraisal period is regarded as appropriate.
In certain situations projects may be considered to have a longer life than 25 years, in which case it could be appropriate to either include a residual value for the asset (reflecting the value of the asset at the end of the appraisal period) or to use a longer appraisal period. The Note Projects with a Very Long Life [Link]. discusses these issues in more depth. It should, however, be borne in mind that in the CBA calculations, net benefits beyond the 25 year operating period will be heavily discounted (for example, for a base year of 2000, the discount factor on benefits in year 2030 will be 0.033 at a 12% discount rate).
User benefits are estimated using the rule-of-a-half (see Box 2), whilst a simple do-minimum versus do-something comparison is undertaken for other cost and benefit items. These calculations will often only be for several forecast years (a minimum of two) as demand forecasting data may only be available for a limited number of years.
INTERPOLATION AND EXTRAPOLATION
To obtain cost and benefit streams over the entire appraisal period, user benefits and costs have to be estimated for each year of the appraisal period. Often only two or three estimates are available (e.g. opening year and opening year plus fifteen years). Interpolating between these years and extrapolating beyond them therefore gives an estimate of annual benefits and costs.
To allow costs and benefits accruing at differing points in time to be aggregated a discounting process is used, for which the specification of an appropriate rate is necessary. The rate used is crucial and ideally should reflect social time preferences within the borrower country, that is the marginal rate at which savers are willing to save in the country (i.e., the rate at which the value of consumption falls over time, or “consumption rate of interest”). However, should the investment crowd out alternative investment projects the discount rate should also reflect the likely returns of funds in their best relevant alternative use (i.e., the opportunity cost of capital or “investment rate of interest”). Often these two requirements of the discount rate conflict as social time preferences can suggest a lower discount rate than the opportunity cost of capital does.
The Bank traditionally has not calculated a discount rate but has used 12 percent as a notional figure for evaluating Bank–financed projects. This notional figure is not necessarily the opportunity cost of capital in borrower countries, but is more properly viewed as a rationing device for World Bank funds. Analysts may use a different discount rate, as long as departures from the 12 percent rate have been justified in the Country Assistance Strategy. Analysts are referred to Handbook on Economic Analysis of Investment Operations (Technical Appendix) (World Bank, 1998) [2] for a more detailed discussion and guidance.
The principal output from the CBA process are a number of summary economic indicators giving an overall measure of the project’s performance in cost-benefit terms. These include the Internal Rate of Return (IRR) and the Net Present Value (NPV). These measures not only describe the economic performance of the project and indicate its economic acceptability, but can be used to choose between project alternatives. Their calculation and use are described in the Note Where and how to use NPV, IRR and adjusted IRR [Link].
Risk and Uncertainty
One statement that can confidently be made about almost any transport project is that the costs and benefits are uncertain. This is because many of the elements of the IRR or NPV estimation are subject to error. It is therefore important to analyse the sensitivity of the calculated net benefit indicators to ranges in individual parameters (capital cost, traffic growth rate, etc.). The treatment of risk and uncertainty within the economic appraisal is discussed extensively in the Note Risk and Uncertainty Analysis [Link], however, it is worth setting out a few points here.
It is important to undertake a risk analysis for all economic appraisals of transport infrastructure projects:
- At a very basic level the development of a risk register detailing the risk, its impact and its significance should be undertaken (e.g. background economic growth maybe uncertain);
- Sensitivity of the summary economic indicators to these risks can be examined through a “What If” scenario approach (e.g. “What if the background economic growth was half that assumed?) or a “switching values” approach (e.g. What would the economic growth be for the project to become economically marginal, i.e. NPV=0); and
- Whilst both approaches enable an understanding of the robustness of the appraisal to each element of risk to be understood, it becomes complex to understand the risks when several variables acting simultaneously (e.g. the impact of background economic growth and the investment costs of the project). In such situations Monte Carlo simulations can be used to provide a higher level of analysis.
Financial Appraisal and Cost Effectiveness Analysis
Financial appraisals are a requirement of the appraisal process within the World Bank [[6]]. This is because it is important to know whether a project is viable from a financial standpoint. As discussed in the Introduction to the Toolkit this is one of the key questions, after social desirability, that appraisal should inform. Financial appraisal differs from a cost benefit analysis because it focuses on the impacts to one organisation, typically a firm or an agency of government, whilst a cost benefit analysis takes a social perspective and considers all those who experience an impact.
A number of procurement options for transport projects are available to government including Public Private Partnerships (PPP) and Build Operate Transfer (BOT). Typically, these involve a stream of payments from the public agency to the private sector, which maybe more financially sustainable than a lump sum payment at the period of construction. A financial appraisal should identify the most financially sustainable option.
Financial appraisals are also important as often the operation and maintenance of the asset, whether it be a piece of infrastructure such as a road or a fleet of buses, requires a stream of expenditure from either the government or the public transport operator. Long term financial sustainability of the project may therefore require that incoming revenue, from taxation or fare revenue, meets the maintenance and system operating cost requirements.
Apart from the narrower perspective considered within a financial appraisal other differences may arise through:
· Use of a commercial rate of interest as a discount rate;
· Attitude to risk including that posed by competition from other firms (e.g. existing road haulage firms competing with a new freight railway service); and
· Use of market prices as the price base.
It should always be possible to rationalise a financial appraisal against a social CBA. The Note Relationship between Financial and Economic Evaluations for Different Types of Project discusses the relationship between the two forms of appraisal in more depth [Link]. The Note Evaluation of public sector contributions to public private partnership projects [Link] also discusses issues that are pertinent to the financial sustainability of a project.
Cost effectiveness techniques involve a comparison between the costs of a project and the achievement of stated objectives or outcomes. As such they have an intuitive appeal as they directly focus on delivering transport related improvements to meet certain goals (e.g. maximising the number of people within 1 day’s travel of a road). They are also particular strong in assessing the most effective measure for delivering a project whose benefits are not readily measurable in monetary terms, an area in which cost benefit analysis is traditionally weak.
The principal difficulties associated with cost effectiveness techniques are that if the project has multiple goals, weights associated with each of those goals have to be derived, often subjectively. There is therefore the potential that decisions based on cost effectiveness techniques maybe biased by the method in which the weightings are developed (e.g. consultation only occurs with sectors of the community who have a particular interest). Additionally, unlike cost benefit analysis, with cost effectiveness there is no threshold that would indicate whether the opportunity cost of the investment is greater than the benefit that will be received. Thus whilst cost effectiveness can inform the choice between alternatives, it is weak at informing the decision regarding whether to invest or not.
The Note Where to Use Cost Effectiveness techniques rather than Cost Benefit Analysis [Link] discusses these issues in more detail. It is however worth stating that cost effectiveness techniques are particularly useful for:
· Project sifting and screening (during the early stages of the planning process); and
· Project evaluation where a substantial component of the benefits are difficult to or cannot be monetised, where options are similar in nature (as when prioritising the re-habilitation of a number of roads) and/or there is a high cost of data collection, relative to the cost of the proposed intervention.
Typical projects would include low volume rural roads and other rural transport infrastructure, though if cost effectiveness criteria are to be used as the basis for the justification of the investment some thresholds of economic viability should also be demonstrated. The Note Low Volume Rural Roads [Link] also discusses the use of cost effectiveness techniques in the appraisal of these projects.
Wider Impacts on the Economy
When transport investments are made, changes are expected to occur to distribution and production patterns, market areas served and labour market catchment areas. These impacts are often termed final or wider economic impacts. However, as discussed earlier, the cost benefit analysis process does not start by attempting to estimate these wider impacts, instead the CBA measures the benefits in the transport market. This is appropriate provided that transport using sectors of the economy are competitive (so that surpluses are passed on to final users) and that there are no significant scale economies in production which a narrow transport sector analysis would fail to take into account.
From an appraisal point of view, the issue is not therefore whether transport projects produce wider economic impacts, as that is one of their purposes, instead the issue is:
(a) Whether and in what circumstances the absolute final economic impacts may exceed the initial transport impacts – so-called additionality; and
(b) Where the incidence of the final impacts is more (or less) socially beneficial than the incidence of the transport impacts. Even if there is zero additionality, are there distributive effects?
The latter issue, that of distributive effects, is addressed in the section following this – Impacts on Society, the Poor and Minorities – whilst the issue of additionality is set out here.
The presence of additionality depends on the presence of imperfect competition in the goods market, the land market or the labour market. The source of any additional benefits is to be found in divergences between prices and marginal social costs in relevant markets. In such cases, reductions in transport costs feed through to some extent into reductions in final goods prices and increases in output. Whereas under perfect competition, the price of this output reflects the marginal cost of production, in imperfect competition a correction is required to allow for the fact that price exceeds marginal cost. The size of this benefit depends on the size of the output effect, and the price: marginal cost margin.
Undoubtedly, conditions of imperfect competition exist within developing country economies, particularly developing country rural economies. For projects that bring about significant changes in levels of accessibility in such economies there will be a case for examining whether additional economic benefits exist to those measured within the transport market. Such projects would include projects that break new ground such as low volume rural roads and feeder roads as well as projects that address significant barriers to movement (e.g. estuary and mountain crossings). The following two Notes provide further guidance on the manner that the issue of additionality should be addressed within the appraisal:
· Low Volume Rural Roads [Link] discusses the manner that the CBA process can be enhanced to incorporate additionality effects in situations where the existing level of accessibility is poor; whilst
· Projects with Significant Expected Restructuring Effects [Link] discusses the issue of additionality in the context of large projects such as an urban rail project or major barrier crossings.
Impacts on Society, the Poor and Minorities
In addition to measures that indicate the overall economic impact of a project a key output from a social cost benefit analysis is the distribution of those impacts. That is which sectors of society are expected to gain and which sectors will experience a negative impact. Such distributive information is important for policy making, as investments are often targeted at improving socio-economic conditions within a particular area or for a particular group of people. It is therefore important from a policy perspective to know whether the targeted area or group is receiving the benefit.
It is possible to disaggregate the components of benefit by impact group; transport users (by mode), operators and providers (including the government), as discussed earlier. This disaggregation process is an important first step to understanding distributional impacts. Ideally, transport user benefits should also be segmented by socio-economic group if the data allows, however, impacts by mode (e.g. Non-Motorised Transport, pedestrians, bus, truck and private motorised transport) will give some information regarding those members of society who are set to gain by the investment. Such information should be presented in a cost benefit table as illustrated in Section 4 of this Framework document.
The impacts detailed in the cost benefit analysis table are transport impacts, and do not necessarily represent the distributive implications of the final economic impacts. To determine if the incidence of the final impacts is more (or less) socially beneficial than the incidence of the transport impacts two steps are required.
First, the impact on the distribution of economic activity needs to be forecast. It is essential to avoid jumping to conclusions. The fact that accessibility from a peripheral area to a regional centre is improved is no guarantee that economic activity will migrate to the periphery. Low transport costs are a centralising force, so if anything, improved transport facilities are more likely to cause migration to the regional centre. This is the ‘two way road’ effect. The demand and supply elasticity conditions required for a favourable result from the peripheral location’s perspective are set out in Dodgson (1973) [[7]]. In pure production terms perhaps the most important requirement is that the region should hold natural or cost advantages for a reasonable range of economic activities in which transport is a significant input. Primary products are a relevant example of where improved transport may permit exploitation of assets fixed in location. The analysis should not be constrained to pure economic production as in certain rural economies welfare benefits associated with better health care and education will also be important.
If market conditions are favourable for a displacement of economic activity, then the second question is whether there is a net social advantage from the redistribution of economic activity. Is the gaining region a target area for economic development? Is activity being displaced from more prosperous locations or from even more acute problem locations?
When developing investment projects that will primarily benefit the poor and/or other minority and disadvantaged groups cost benefit analysis techniques can be weak at determining the best project of a set of alternatives. Gannon and Liu (1997) [[8]] identify the following reasons for this:
· Measurement of benefits and costs based on monetary willingness-to-pay, as registered through the market system, tends to favour higher-income groups;
· Exclusive focus on the efficiency criterion tends to neglect the needs of the poor;
· Orientation toward efficiency leads to a higher dependence on motorised transport which tends to displace infrastructure for non-motorised transport, to the disadvantage of the poor;
· Rights-of-way are often imposed on poor communities for high mobility projects that may not benefit the poor of these communities directly; and
· Commercialisation in the transport sector may lead to higher prices for services that previously were affordable to the poor.
In such a situation alternative methods may be required. These methods and more detailed advice regarding the distribution of benefits and impacts on poor people are given in the Special Advice Note Distribution of Benefits and Impacts on Poor People [Link]. Additionally, the Note Low Volume Rural Roads [Link] and The Note Where to Use Cost Effectiveness techniques rather than Cost Benefit Analysis [Link] discuss the application of CBA to projects which focus on delivering welfare improvements to the poor.
4 Reporting the Cost Benefit Analysis
As discussed in Section 2 of this Framework, Transport Appraisal Within The Policy Process, the cost benefit analysis results have to be read alongside other components of the decision process including the environmental assessment and wider planning issues such as distributional and poverty impact analyses, financial sustainability, risk analyses and a consideration regarding whether or not there are social or technical barriers to implementation. The cost benefit analysis framework described within this document can be an important source of information regarding many of these wider issues.
To aid the decision process described above it is important to present the cost benefit analysis results in a clear and concise form. The key information that should be reported will be:
· Initial assumptions and scenario definitions;
· CBA parameters including:
- Start Year, Opening Year;
- Discount rate
- Price base (e.g. 2002 real factor prices, US dollars)
- Shadow pricing assumptions
· Summary Measures of social value;
· Disaggregated CBA results, highlighting the following distributional issues within the overall costs and benefits:
- Users’ benefits versus net impact on operators;
- Shares of user benefits by mode;
- Composition of user benefit by item of benefit (Time, VOCs, etc);
- Shares of time savings made up by personal travel (in non-working time) and business travel including freight and personal travel in working time;
- Shares of international traffic versus domestic traffic in user benefits;
- Shares of operator costs and revenue by mode;
- Investment costs by group (that is, private operators, national government, financial institutions).
This disaggregated information could be presented in a range of different formats, some of which are more suitable than others for particular uses of the appraisal outputs. A set of example reporting tables is given below; and relate to an example in which a road infrastructure investment is made affecting domestic (i.e. not international) road based transport modes only (e.g. motorised vehicles, buses and coaches, NMTs and potentially pedestrians). Such an example table will act as useful starting point for the development of project specific cost benefit analysis reporting table. Such reporting tables are often referred to as Transport Economic Efficiency (TEE) tables.