With the Minnesota Bicycle Summit last week and the National Bike Summit this week, I have been noting a pernicious habit among cycling advocates and friends that I wish I could punch out of everyone: Quoting studies selectively or in ways that simply do not apply.
As cyclists and advocates argue for ongoing funding for alternative transit infrastructure in a difficult budget environment, data helps. With those who already support bicycles, data is a tool to offer them to help argue the position or defend their position to others. To doubters, data can be a means to shift opinion.
But using data badly does no one any favors. It’s easily assaulted by opponents. It makes cycling advocates look stupid.
Some recent examples:
- That damn cycletracks study. I have ranted about this seventy-eleven times at this point. There are major flaws in this study’s data methodology.
- Studies that say more people would ride if more facilities were built. Very often the data collection in these meets appropriate statistical standards. But these really do become a tyranny of the masses — what is popular is not always a good idea. National obesity trends are one example of how what is popular (being sedentary, high fat convenience foods) not necessarily being a good plan. It’s important not to let opinion polling override other forms of science.
- The Baltimore study that says that investing in bicycle infrastructure creates more jobs than highway projects. The data in this study is specific to one metro and one series of projects, and is thus difficult to credibly extend across all projects and metros. However, I am definitely seeing bicycle advocacy groups try to do so.
- I saw someone reference a study the other day from Bristol, England that says that “pedestrians, cycle and public transport users provide as much if not more spending power than car users in town centres.” It’s been re-tweeted a lot by people going to the National Bike Summit.
One issue: This is a UK study. The way UK cities and neighborhoods are built around “High Streets” is completely unlike how most of the United States is built. I’ve lived near a UK High Street, when I was attending school in London. Neighborhoods are built around a core intersection/broadway/circus in which most of the basic needs of life can be procured, and major transit transfers are possible.
There are some junctions within cities that act like high streets in the UK — an intersection like Cleveland and Ford Parkway in Saint Paul comes to mind, where you can get almost all the amenities of life within a short walk of the core intersection. But more often the setup is more like MN65 in Fridley/Blaine — a series of strip-malls along a high-speed state highway corridor. To invest on a High Street model would mean blowing up a lot of America to start from scratch.
I’m sure there are more out there that would just make me snarl to hear cited.
Not all studies based on small geography or populations are of no use. The health study in Madison and Milwaukee has broader applicability, because the controlled factors are such that you can credibly say: We don’t know what the total financial savings would be in THIS metro, but based on the savings in THOSE metros it’s pretty safe to bet it’d be a good chunk of change, eh.
I have seen a number of advocacy groups stick to citing well-controlled data studies and facts and figures that can easily be applied within a region without acrobatics. The Bicycle Alliance of Minnesota is one such organization.
As advocates, we don’t need to try to mutilate data to serve our needs. There are studies and data to support our goals that are credible as they stand, without trying to say “we could be more like Europe!” (which is not a good message with even some moderate Republicans, and is often not realistic based on existing build patterns). There are countless health and environmental benefits. There are social benefits. Infrastructure investment can reduce congestion and thus increase business productivity. Infrastructure can attract educated workforces to urban cores. Citing those factors, and data collected in the United States in broadly applicable studies, is going to have a greater impact on fiscal conservatives and the unconverted than using data dodgily.
March 9, 2011 at 11:01 am
I don’t think talking about these studies is inherently misleading or wrong, as long as folks acknowledge the limits of their claims. In fact, a lot of these studies conclusions seem to have a great deal of merit and make sense, in my opinion.
Biking is notoriously difficult to study when compared to auto infrastructure and habits. Its more expensive and time consuming to measure rates, accidents go unreported or reported in a hundred different ways, and (when it comes to things like surveys) these are the only way to get at the great variety of motives and desires of cyclists. Basically, there’s no “perfect study” and there probably will never be one.
The safety in numbers effect you mention is a great example of why its so difficult to compare different areas. There’s no doubt that this is a real phenomenon (e.g. the Dutch safety record). The only problem cycling advocates face, however, is how to get there, how to achieve a “phase transition” to a multi-mode society. Actually, there’s a great case to be made that cycle tracks at least make you “feel” safer, which is a very important outcome…
March 9, 2011 at 11:45 am
Bill: I don’t think there need to be “perfect” studies to be citable. I do think that I’ve seen a lot of over-reaching lately, possibly because everyone WANTS data in the current budget environment. The Montreal cycletracks study is a good example, because there have been numerous studies that deal with the safety in numbers hypothesis, which is a factor that complicates the cycletracks study significantly. But the single study is being waved like a flag without thought or comment to the prior body of work.
There are things that can be taken from foreign experience and study as well. But I have even seen and heard legislators dismiss that data because it’s “European” — either based on a bit of patriotic jingoism, or based on the very real underlying concern that many European cities have had an easier time adapting because of their longer history or greater compactness. Amsterdam has NEVER been a good car city, whereas the suburban boom in the US has car culture at its heart. Development patterns and planning here face that as a different sort of barrier.
And while “feeling safer” is a nice outcome, I can cite many examples where that feeling is pretty illusory versus engineering and practice — many intersections along Summit Avenue come to mind. I think encouraging people to feel safe and be willing to ride is important, but I also think advocates and planners have access to more and better information than Joe Public and need to make decisions based on that data, and not just the opinion of the masses.
March 9, 2011 at 2:22 pm
Basically agree w/ everything you said here. Only, here’s the feedback loop I want to see: design something where you feel safer –> more cycles and cyclists –> critical mass of folks on the street actually makes it safer…
In a strange way, perception can create its own reality, so that breaking past the fear of the street becomes a self fulfilling prophecy.
How to get there, though, is a challenge. Riding Boldly is a good start, but how do we entice people into making that leap?