Comments on
Estimating Recreation Trip Related Benefits for the Klamath River Basin
with TCM and Contingent Use Data - Draft
A.J. Douglas and A. Sleeper
By Richard J. McCann, Ph.D, Partner, M.Cubed
2655 Portage Bay, Suite 3, Davis, CA 95616
(530) 757-6363 rmccann@cal.net
This paper is the same paper that I reviewed at the Western Economics
Association meeting in Seattle, Washington on July 1, 2002. The paper did
not incorporate any responses to my comments, nor the comments of any of
the other two reviewers. My comments follow by section.
In summary, after examining the data used in the analysis, the paper’s
findings simply fail the common sense test. The results imply that groups
of 4 or more individuals are traveling together over 10 times per year to
recreate in a single location for at least 12 weeks per year, and that an
average 100,000 people are doing this every day throughout the warmer
months. Add to this the implication that almost 10% of Oregon’s population
averages at least 12 weeks of vacation per year, and that they spend all
of it in the Klamath River Basin.
Other general observations: The values shown include too much spurious
precision. Dollar amounts, average rates and percentages are shown with
too many significant digits. Average annual income cannot be calculated to
the tenth of a cent, especially since the IRS doesn’t even require income
to be reported in increments less than a dollar. There are many
unsupported allusions to data sources and unconventional methods that
require substantially more documentation and explanation. The authors mix
and match values between different numbers of cases, e.g., using the
average income for 648 cases with the average number of hours on site for
547 cases. The proper method is to use the values for the smallest number
of cases so as not to reduce potential bias in the results.
3. The Klamath River Survey
The user survey shows an unusually high survey response. However, the
response rate, and the usable data set ratios are calculated incorrectly.
The response rate to the mail survey should have been (382 + 234) / 1,100
mailouts, or 61%. The phone survey response rate is impossibly high—no
phone survey has only 2% of those called refusing to complete the survey.
In addition, the telephone response rate should include all attempted
phone numbers that were not disconnected.
The high response rates raise questions about the validity of the sample
data set. Why would the response rate be so extraordinarily high,
especially on the phone survey? Why are these respondents so motivated,
well beyond what is found in most other surveys, where response rates
around 40% or even less are much more common? The answer may lie in
looking at the response themselves, as discussed below.
Participation Rates
First, this section implies that the participation survey was somehow
linked with the user survey by organization of this and subsequent
sections. Close reading indicates that they are quite distinct surveys.
This needs to be clarified.
The statewide participation rates were estimated using a different survey.
The report states that 200 households were contacted, but the author
clarified at the WEA conference that the survey covered 200 households per
state, for a total of 800 respondents. The participation rates are
extremely low for 3 of the 4 states, and quite small even for Oregon. To
estimate small proportions in a large population with precision requires a
very large data set—one much larger than when making estimates where the
proportion is closer to 50%. For example, when trying to estimate the
number of unregistered vehicles in California, with an expected rate of
about 3%, the sample data set included 20,000 cars. The imprecision of the
participation rate survey can be seen by looking at the number of positive
responses from each state: Nevada – 1; California – 2; Washington – 0,
Oregon – 18. In other words, the participation rate for the entire state
of Nevada was extrapolated from a single positive response. This would be
akin to surveying Washington, D.C. about occupations and calling 1600
Pennsylvania Ave. That survey would project that 0.5% or at least 1,000
individuals were the President of the United States. Whether the survey
actually covered the entire state of California, or the other states, is
another key question not answered in the paper, and no reference is given
to any supporting documentation.
Another fact that raises questions about this participation rate is the
projection that 114,460 California households visited the Klamath River in
a given year. Later in the report, the visitors coming more than 400 miles
are considered outliers. A check of California county populations reveals
that about 458,000 households reside in the 19 counties within 300 miles
of the Klamath Falls. If we assume that the vast majority of California
visitors come from this region, the state level participation rate implies
that over 20% of this population visits the Klamath annually, and that
they spend an average of 12 weeks per year in that location. Apparently 9%
of the entire population of Oregon has averages 12 weeks of vacation per
year.
The authors never explain why the survey percentage must be divided by 3
to arrive at the participation rates. Perhaps this is an assumed number of
individuals per household?
4. The TCM Data
The paper does not make it clear which survey data is being used to
estimate the mean income of the respondents. I assume that it’s the user
survey.
The mean income of $64,880 is extraordinarily high. This about 60% above
the California median household income of $40,312, and Oregon income
averaged $37,827 and Nevada, $39,459 in 1997. If we focus on the Northern
California counties that are most likely to send tourists to the Klamath,
this is about 85% higher than the median income for those counties. While
one might argue that this region draws higher income visitors, this
presumption is inconsistent with other findings in the data discussed
below.
The authors argue that the average hourly income (for all hours of the
year—8,760) is the appropriate measure despite the fact that almost all
workers have institutional and contractual limits on their work time. The
authors appear to be arguing that these visitors are in fact dropping out
of the work force to visit the Klamath, and therefore they directly trade
off leisure versus labor time. This might be a valid argument if the
visitors were drawn from a transient work force that exhibited high rates
of unemployment. However, the extraordinarily high average incomes
indicate that these visitors are generally fully employed, and do face the
worktime constraints that imply that they are not foregoing any income to
visit the Klamath.
Transient Visitors
The average number of trips per year shown per visitor was 7.6 for annual
visitors, and 10.1 when including “transient visitors.” This is an
extraordinary number of average visits to a single site. This number
becomes even more extraordinary in the context of other data discussed
below. The authors use a form of a gravity model to impute the number of
visits by transient visitors. The model estimates that a visitor living
within 50 miles would visit 50 times a year. That implies weekly round
trips of about 4 hours each. Again, this seems to indicate extraordinary
usage rates well beyond that experienced by any other recreation site in
the world.
The mean distance traveled was 268 miles (or more precisely, 268 miles,
859 feet, 7 inches). Combined with the average number of trips of 10.1,
this implies that the average visitor drives 5,400 miles per year just to
recreate at the Klamath. The average driver covers about 21,000 miles per
year, so this implies that the average visitor may be doing upwards of 25%
of their driving just to go to the Klamath, and that doesn’t include any
driving once they arrive.
The authors then estimate the average period spent at the Klamath,
arriving at 2,045 hours per year (more precisely, 2.045 hours, 30 minutes,
0.6 seconds). This works out to 85.2 days or more than 12 weeks per year.
In other words, the average visitor spends almost a quarter of the year
recreating on the Klamath. Given that the average income is more than 60%
above the 3-state average, this is simply impossible. The authors’
assumptions about foregone income implies that this population could
increase its annual income to an average of $76,000 per year by not
visiting the Klamath.
If these visitors are spread evenly over the entire year, this implies
that at any one moment at least 54,500 tourists are recreating in the
Klamath River Basin at any one moment, based on the state-level
participation rates. In fact, it’s more likely that they are visiting
during the late spring through early fall, implying that the average
number would have to be in excess of 100,000 at any one moment. I am not
familiar with the tourism industry in the region, but I am familiar with
the Bay Area and Seattle, and I cannot imagine a daily average of 100,000
tourists being in either of those locations, which are much stronger
tourism attractions.
The authors try to explain this extraordinary fact by noting that there
are a “very large number of unrestricted free access sites on the Klamath
River Basin.” Yet it’s hard to believe that a launch fee of $5 per day
would inhibit in any way visitors who are willing to forego $12,000 per
year in income, and spend a quarter of the year, and probably all of their
vacation time, recreating in a single spot. Plus the authors never address
the real cost of vacationing, lodging and meals, that would overwhelm any
possible access fees.
The real answer can be deduced from the data. The respondents in fact earn
a substantial amount of their income from recreation and fisheries related
activities in the basin. The only way to maintain an annual income of
$65,000 and reside in the area for almost 3 months is to generate income
from activities in the area. In other words, being on the Klamath is their
business and livelihood and has little to do with recreation. The high
survey response rate reflects their strong direct economic interests in
maintaining the Klamath’s fishery and recreational values. (In fact, it is
likely that the responses also are strategically biased.) There is little
or no foregone income, and these individuals derive no more or less
economic value than the farmers upstream.
5. Consumer Surplus
The travel cost value is set at $0.31 (per mile?) with no citation. This
is the government reimbursement rate and the IRS deductible expense, so I
assume that this is the source of the data.
6. Klamath River Regression Models
The authors do not explain the difference, and its significance, between
household and aggregation data. The authors also state that they used
other models with “little success.” What is the definition of “success?”
The authors do not list what other explanatory variables were used, so it
is difficult to assess whether the t-statistic on the coefficients of
interest are truly significant given the low R-squareds. It’s also not
clear how the regression can use 665 or 649 observations when only 547
observations could be use to calculate the average foregone income.
7. Klamath River Benefits Estimate
The survey data shows an average of 4.17 recreationists per vehicle. This
is absolutely extraordinary given that the average number of passengers
per vehicle is typically less than 1.3. In fact, given that most vehicles
can hold only 5 (or less) adults comfortably, except vans (and this
doesn’t account for packing supplies for the average trip of 8.5 days),
this implies that almost every trip is made with a nearly full vehicle. In
other words, groups of 4 or more individuals are traveling together over
10 times per year to recreate in a single location for at least 12 weeks
per year, and that an average 100,000 people are doing this every day
throughout the warmer months. Add to this the implication that almost 10%
of Oregon’s population averages 12 weeks of vacation per year, and that
they spend all of it in the Klamath River Basin. All of this simply fails
the common sense test.
The remainder of the benefits calculations are not valid due to all of the
problems with the survey data.
8. Contingent Use Data and Benefits Estimates
The CU non-response rates seem to be taken from a completely different
study, and there is no explanation of how they are applied.
The survey presents 3 different scenarios for amenity improvements. The
authors then state that they have no means of determining whether the
identified restoration activities can achieve these improvements. Given
that at least one of the improvements, an increase in water quality, is
the maximum possible, this approach likely will underestimate the
restoration costs required to achieve the enhancements identified in the
survey. (How those enhancements were conveyed to survey respondents is not
discussed in this paper, but the companion CV study paper also presented
at the WEA meeting has serious problems here as well.)
The CU survey is used to estimate the increase in trips with each amenity.
However, it’s not clearer that the increased trips would be of the same
length as the average trip, as assumed by the authors. Given that the
initial trips are probably the longest, and each marginal trip diminishes
in both length and enjoyment, the survey should have estimate the
additional number of days recreating. The analysis also assumes that the
increase in water quality has an additive effect with increased harvest.
However, to increase harvest, it’s likely that water quality also would
have to be improved. Thus, adding the incremental effects from both
improvements is incorrect. Table 6 implies that the greatest improvement
would lead to an average increase in recreation of 30 days per year! In
other words, the average visitor would go from recreating one-quarter of
the year to one-third of the year in a single location, giving up an
additional month’s of income.
9. Habitat Restoration Costs
The authors estimated the value of acquiring Klamath Project farmland
using gross Oregon state data. The authors should have relied on one of
two other sources, either the 1997 Census of Agriculture, which breaks out
land values by county and by irrigated versus non-irrigated, or reports
from the relevant counties’ agricultural commissioners.
The value of lost electricity is too low. The appropriate measure is to
calculate the cost of replacing the lost power. Hydropower plants
typically are used to generate for higher value peak load periods. Natural
gas fired combined cycle plants are the most common type constructed
today. The California Public Utilities Commission recently adopted a
benchmark value of 4.3 cents per kWh based on these power plants for
comparing power purchase contracts. Bonneville Power Administration, the
federal power marketing agency for the Pacific Northwest, is using a
similar benchmark for generation planning.
Nevertheless, it’s not clear what the authors are proposing here. Are they
proposing the decommissioning of these dams to allow fish bypass? If so,
they also have failed to include the regulatory and demolition costs,
which will be substantial.
I don’t have sense of the value of forest land. The water value per
acre-foot is consistent with water purchase costs in California.
The authors then discuss a restoration scenario of a 12-year ban on sport,
commercial and tribal fish harvests. The authors fail to include the costs
of purchasing the boats and other equipment that the fisherman have
invested in when making this calculation.
But the authors make a more serious error. A 12 year ban on sports fishing
would reduce the tourism in the region, as evidenced by the CU survey
responses on increases in fish harvests. The ban would be eliminating one
of the primary reasons that recreational visits are expected to increase.
In other words, the authors were counting phantom benefits during the 12
year period to offset costs. In fact, the benefits would have to be
delayed at least 12 years, and given the long period of recovery that
would be likely for the local tourism industry, even longer. This delay
would reduce the net benefits by at least 58 percent. Given that the costs
of this scenario is at least $4 billion, and that the benefits have been
obviously overstated for all of the reasons discussed above, this strategy
would likely have a negative net benefit.
The discussion about statistical reliability is greatly misplaced given
the various data problems and the error in estimating participation rates
with a small sample.
In the conclusion, the authors state that the nonmarket benefits are
greater than the estimated costs of the amenity enhancements. Yet earlier,
the authors had said that they could not quantify the enhancements that
would occur from the proposed restoration measures. This earlier statement
indicates that the authors cannot draw the conclusion as stated.
Foot notes:
1. Robert Dulla and Yji Horie, Unregistered Vehicle Study -
Field Survey and Analysis, Contract No. A866-163, Draft (Sacramento,
California: Prepared for California Air Resources Board by Sierra Research
and Valley Research Corp., November, 1991).
2.U.S. Census Bureau, Statistical Abstract of the United States,
2000, Table 742.
3.Ibid, Table 1033.
4.I reviewed a recreation survey of the Sacramento-San Joaquin
Delta which calculated the average number of passengers per vehicle at 3.5
per trip. However, a closer examination of the survey found that the
authors had attributed a party of 300 to a single vehicle (I termed this
the "747 effect"). The corrected average after removing this one party was
2.1, which is consistent with both household size and average number of
passengers per vehicle data.
5.CPUC decision in proceeding R.01-10-024.
6.Richard James McCann, "California's Evolving Water Management
Institutions: Markets and Agricultural Water Districts" (Ph.D.
Dissertation, University of California, Berkeley, 1998).
|