Tasmanian Mountain Biking Visitor Statistics - In More Depth

Mountain biking is booming in Tasmania, you just have to head up to Derby to see that, however there are a number of claims being made about vistior numbers that just aren't really supported by the source they come from ... if you dig into the data a bit.

For example what if I were to tell you that if you examined statements such as "Tasmania Visitor Survey data shows the number of people taking part in mountain biking in the state increased by more than 120 per cent in the 12 months to September last year." (reported in the ABC online article 5 march 2017) in more detail, you'd find that while the statment is technically true, most of those 'mountain bikers" (86% in fact) didn't even go near the trails at Derby?  That can't be true ... right?

Also, just because I'm picky, if you wrote that same statement for the 2016 year (or any other year), the increase would be be much lower as the September to September data quoted happened to be an outlier as shown in the graph below:  

For the last few years the Tasmanian Visitor Survey data has been the best indicator we’ve had for the growing success of Mountain Biking in Tasmania.

And it’s been pretty good news, with, as the ABC article says, the number of visitors undertaking mountain biking more than doubling in the two years since the Blue Derby trails opened.

But, the statements and conclusions drawn in the media (and on my own site) have always bugged me because of the question asked on the Survey:

"During this trip did you go mountain biking".

Just think about that question for a moment.

That's a lot different from asking "Did you come to Tasmanian to go mountain biking" or even "Was mountain biking one of the reasons you decided to come to Tasmania". These last two questions clearly imply that one of the reasons you came to Tasmania was to mountain bike, and that it's not an afterthought or minor part of your holiday. I'd argue that it is this distinction between those that come here with an interest and intention to go mountain biking as against those that just go mountain biking while they are here that distinguishes between a mountain bike tourist and a visitor who comes to Tasmania and happens to go mountain biking.

Someone who comes to Tasmania to visit Mona and then hires a mountain bike to ride out there could tick the box "Did you go mountain biking", but I don't think that makes them part of the mountain biking tourism sector.

As anyone who reads my blog will know, I recently went to New Zealand to ride the Old Ghost Trail and only one mountain biker I met over there actually knew Tasmania had any mountain biking trails, all the rest opened with the question "do you have any trails over there".

This got me pondering again the limitations of the data we have in this space, and lamenting that no one had thought to do anything so obvious as analyse the Tasmanian Visitior Statistics (TVS) data to see what more we could learn about these people who went mountain biking.

So ... figuring if no one else would do it, I'd give it a try, I wrote to Tourism Tasmania and asked them if they could give me a breakdown of the data from the TVS for those people who had ticked the box saying they had gone mountain biking or cycling, and also show me where those people stayed, how much they spent and what else they did.

Much to my surprise, one week later, all the requested data turned up in my inbox.

So, the (sort of) good news first - I had previously guessed that if around 9,000 people did the TVS each year, and that around 1.6% of total visitors indicated they went mountain biking, then just over 100 people in the survey would have ticked the "During this trip did you go mountain biking" box.

In fact, the TVS data indicated that for calendar year 2014, 53 people ticked that box, 71 people in 2015 and 89 in 2016. So good news is that I was close, bad news is that it confirms my hypothesis that any assumptions drawn from this mountain biking data is based on a very small sample set.

The next, and most interesting question for me was this:

"How many of these people indicated that they had stayed, or at least stopped, at Derby?"

Now my thinking behind this question is pretty unsophisticated, but I think solid: If you came to Tasmania to go mountain biking, you would try and get to Derby. Sure there are some people who may come to Hobart or Launceston and try and squeeze in a days ride there, but if you're coming to Tassie to ride our trails, then Derby would be where you want to get to. I think that's a fair assumption.

Now the TVS asks a series of questions about where people stayed and visited, essentially asking people to nominate whether they stayed somewhere (and if so for how many nights), then if they didn't stay there did they at least stop (visit) that place, and then finally the survey ask if the respondents "just passed through".  As Derby is one of the 36 locations listed on the survey, this means we can get a good snapshot of how many people who said they went mountain biking also stayed in or, at least stopped at (visited), Derby.

So how many of these people came to Derby 80%? 70%? 40%?

Unfortunately, the answer is just 9% in 2015 and 14% in 2016.

Now before anyone gets too carried away, yes the trails only opened up in 2015 (April I think), and yes that is an increase (woo hoo) but my point is we need to stop and think what is behind those numbers ... which is next to nothing.

If 9% of people that filled in the survey indicated that they went mountain biking and either stayed in, or visited, Derby in 2015 - then that's less than 7 people out of a survey of 9,000. Similarly for 2016, that works out to 12 or 13 people out of a sample of 9000.  Hopefully you don't need to be a statistician to realise that that is a really small number that is prone to a lot of error.

If it gives it anymore perspective for you, here is the graph showing the number of visitors that ticked that box for a range of other places (unfortunately places like Kellevie and Penguin which are known mountain biking destinations aren't included in the survey).  What you can clearly see is that Derby is not a stand out (but again - don't read anything into this data).

With such low response numbers in the survey, imagine how an event like the recent Enduro World Series could potentially skew a data set like the TVS if just a handful of those riders were picked up in the 2017 survey.

So, when all is said and down, there's only two take home messages I can take from this data:

1. The TVS data does not provide sufficient support to the proposition that the Blue Derby trails are driving a mountain biking boom in Tasmania (or that it's not) and;
2. The data set is so small that to draw any conclusions from it about increases or decreases would be fraught.

So am I saying that the the mountain biking boom is a con? Of course not, just go up to Derby and see how may people are there, but what I am saying is that anyone who tries to quote the TVS data as proof of a mountain biking lead boom ... well point them this way and hopefully they'll correct their assumptions like I have.

I also want to be really clear that I am also not criticising the TVS or how it is collected.  The TVS was never designed to support anaylsis of niche visitor activities, although I do understand that the Mountain Bike question was added to the TVS following a recommendation in the Tasmanian Mountain Biking Masterplan to collect more data on visitors undertaking mountain biking in Tasmania.

What I am trying to point out is the limitations in how it is being used to support the case for a mountain biking boom - something which I hope this article shows is beyond what the data collected can support.  I think the TVS is a brilliant data source and Tourism Tasmania is very clear on their website to indicate the limitations of the data, and in particular that any visitor estimates of less than 5000 people are subject to high data variability.  

What I am saying is that if we want an unbiased estimate of how many people are coming to Tasmania to come mountain biking, then we need to think of another way of collecting that data.