About this example
This is a hypothetical composite. The district, the data sources and the findings are invented to show how an evaluation might be set up and what it can and cannot tell us. It does not describe a real place, project, agency or result. No figures are given because none would be real; the aim is the method.
Call the fictional area the Millgate district: a former industrial and warehouse area at the edge of a downtown, with many empty buildings. The city adopts a renewal plan that includes rehabilitating several large buildings, upgrading streets and offering support to small firms. The city also commits to tracking whether the plan achieves its economic development aims.
Step 1: Stating the goals
The plan lists three goals: increase employment among residents of nearby neighborhoods, reduce vacant commercial floor space, and expand the local tax base without displacing existing small firms. Each goal suggests specific indicators. Having goals written down means that later claims of success can be tested against them, instead of picking whatever number looks best.
Step 2: Setting a baseline
Before work begins, the city collects starting data:
- A vacancy count of buildings and ground-floor units, using one definition and one method that can be repeated. This is the type of work done in vacancy surveys.
- A parcel-level record of ownership, use and assessed value, similar to a parcel inventory.
- Employment of residents in the surrounding neighborhoods, from available administrative or survey sources.
- A list of existing businesses, their sizes and how long they have operated.
- Rents and sale prices in the district and nearby areas.
The baseline has a limitation: some data are only available at coarse geographic levels or with a delay. The team notes these gaps instead of hiding them.
Step 3: Choosing a comparison area
The team picks a comparison area with a similar history, mix and location but no renewal plan. It tracks the same indicators there. The comparison is imperfect, because no two areas are identical, and the team records how the comparison differs. Still, if both areas change in the same way, the renewal plan cannot take all the credit.
Step 4: Tracking over time
Each year the team repeats its measures. In this imagined story, vacancy falls in Millgate, resident employment rises modestly, and assessed values grow. In the comparison area, vacancy falls slightly and values rise more slowly.
Several readings are possible, and the team has to consider them.
- The renewal plan may have helped.
- The region’s economy may have strengthened, lifting both areas, with Millgate gaining more because it had more to recover.
- Businesses may have moved from the comparison area into Millgate, which would make Millgate look better and the comparison look worse without a net citywide gain.
- Households may have changed, so resident employment may have risen because different people live there, not because original residents found work.
The team can reduce some of these doubts by tracking cohorts of the same households or firms where data allow, but not all.
Step 5: Looking beyond averages
The team splits the results. How did long-standing small firms fare compared with new ones? Did rents rise faster than incomes for residents of the district? Did jobs go to nearby residents or to commuters? The results are mixed in the story: new firms thrive, a number of older firms close, and the share of jobs held by nearby residents is lower than hoped.
This is typical of the way averages hide distribution. A district can show growth and still miss one of its stated goals.
Step 6: Public finance
The team compares the city’s costs with the revenue changes in the district. Property tax receipts increase, but the city also paid for streets, grants and administration. The team cannot be sure how much of the tax increase is due to the plan and how much to citywide trends, so it presents a range. Some revenue may be moved from elsewhere in the city. The point is the discipline of stating what is known and what is estimated.
Step 7: Reporting
A good final report in this story would:
- Say plainly which goals were met, partly met and not met.
- Present the comparison area results alongside Millgate’s.
- Describe the limits of the data and the assumptions.
- Separate short-run effects from those expected to last.
- Avoid claiming that the plan caused changes the data cannot attribute to it.
These practices match the logic of project evaluation after a project opens, and the use of planning indicators with clear definitions.
Where evidence is uncertain
Even in a careful exercise like this one, much stays unsettled. The comparison area may differ in ways the data do not capture. Effects may take longer than the tracking period to appear or disappear. Data on residents’ circumstances are often limited. Costs and benefits outside the district are hard to see. And the story is a single imagined place: other cities, with different markets, rules and funding, could see different patterns.
What to take from it
The value of the exercise lies less in whether Millgate succeeded than in how the question was asked: with goals, a baseline, a comparison and honesty about uncertainty. When you read a claim that a renewal area boosted its economy, a useful test is whether anything like these steps lies behind it.