Breaking down the Manitoba BSL Effectiveness Study

Last week, a new study was published out of Manitoba, Canada, about the effectiveness of breed-specific laws (BSL).  And contrary to all of the other studies on the subject (which they acknowledge), this study says that BSL MAY work. Here is their conclusion:

“BSL may have resulted in a reduction of Dog Bite Injury Hospitalizations in Winnipeg, and appeared more effective in protecting those aged <20 years.”

So with great interest, I read the article. You can read the article here, in its entirety (it’s only about 7 pages) for yourself.

The study is a fairly detailed study of 16 jurisdictions with pit bull bans in Manitoba that covers the Dog Bite Injury Hospitalization (DBIH) rates before and after they enacted their pit bull bans. The study did take into account population changes and rated the injuries on a per 100,000 person basis. The study focused on DBIH because their premise was that ‘pit bulls’ cause more injury when they bite and this would be a better measure of public safety than just bites.

They did also measure for Rabies exposure tests to see if bites that broke the skin increased or decreased (although this included all potential rabies exposures, and not just dog bites). This information proves to be pretty useless however as they later note that only about 30% of all potential rabies exposures come from dogs while skunks are the primary carrier.

The test included mostly rural communities, but did also include the Provence’s 2 largest cities: Winnipeg (which has a breed ban that it enacted in 1990) and Brandon, which has no ban. Brandon was one of 15 communities without breed-specific laws that were used as a control group. All of the other communities were labeled as rural communities. The study covered the years 1984-2006.

Unfortunately, the way the results were published, was a bit, ahem, misleading.

For the most part, they grouped all of the cities with BSL and those without together to show results – the lone exceptions were the two urban communities (Brandon and Winnepeg) and two small northern cities: The Pas (which enacted a ban in 2005, the last year of the study) and the town of Flin Flon (which has no ban). The rest were all grouped together.

During the years studied, there were 838 total DBIH in Manitoba – a total of 36 per year – and a total of 3.19 per 100,000 people.  Of those, 363 (43%) occurred in cities without breed bans, 417 (57%) occurred in cities that enacted breed bans during this time.

Now here is where the numbers start breaking down for the study.

If you look at the communities that enacted breed bans during this time, of the 417 total DBIH, 144 of them occurred prior to the city implementing their breed ban. Of those, 122 were in the cities of Winnipeg and The Pas (the only cities with bans listed individually).  So of the other 14 communities studied, there were only a total of 22 DBIH before they passed the bans – so about 1.5 per community over the course of 7-22 year time frame. So we’re talking, on average, about one incident every ten years here for all the communities that are not the Pas or Winnipeg. 

After the bans were enacted, there were a total of 331 DBIH – of those, 307 were in Winnipeg alone (the Pas has zero in its one year after the ban). So post bans, there were only 24 total DBIHs for the other 14 cities combined – again, a very low number over a range of 2-15 years.

So the sample size of the number of incidents in surrounding communities is VERY small. All of which of the communities were very rural (and this study confirmed what other studies have reported that major injuries are more likely to occur per 100,000 people in rural areas than in urban ones). Also, based on the types of dogs owned in many of these areas for working purposes, and based on Canadian dog bite fatality studies, most of these incidents likely involved Huskies or other sled-type dogs.

So given all of this, almost the only conclusions that can be drawn from the study itself is based on what happened in Winnipeg and The Pas (the two communities broken out separately).

But even The Pas is impossible to study, because it instituted its ban in 2005 – right before the last year of the study. In the 22 years prior to passing the ban, they had a total of 12 DBIH – so essentially one every two years. Then, in the one year after passing the ban, they had zero – which statistically was just as likely to happen prior to the ban as it was after the ban - -so not much can be gleaned from that.

So let’s look at Winnipeg.

In the 6 years prior to them passing their ban in 1990, they had 110 total DBIH – an average of 2.93 per 100,000 persons-years.  In the 16 years following the ban, they had 307 total incidents – an average of 2.81 incidents per 100,000 persons-year.

So, in other words, in the 16 years after the ban they lowered their DBIH numbers an average of .12 incidents per 100,000 people – or one incident per 800,000 people – or, based on Winnipeg’s population of 660,000 that is a drop of less than one incident per year during a 16 year stretch when dog bites throughout the US and Canada have been dropping. (Interestingly, there is no comparision in this study of whether or not bites dropped in cities without BSL during the same time period).

Meanwhile, in Brandon (the only other “urban” city studied), for the entire 24 year study, they had a total of 24 DBIH (1 per year) and a incident rate of 2.5 per 100,000 persons-years – so lower than either period for Winnipeg.

The researchers in this case seem to get that they were having a tough time proving their hypothesis. From the discussion:

“When jurisdictions were used as their own controls in a pre-BSL versus post-BSL comparison of incidence of DBIH, no significant reduction in the period after BSL implementation was observed.” (Emphasis mine)

The researchers also noted other reasons for changes in DBIH numbers that they didn’t not account for in their study – in part due to the long-amount of time (23 years) included in the study:

“Longer periods under observation have the potential to be influenced by period effects or confounding factors such as changes in the number of pet dog populations, changes in the popularity and ubiquity of breeds, changes in the number of dog-owning households, parallel and related ordinances, and public safety education campaigns.”

The study also notes that there was absolutely no inclusion of DBIH by breed of dog involved in the incident – either before, or after the bans. So while it measures the impact of DBIH before and after a ban, it does not indicate, at all, if ANY of the incidents involved pit bulls. Pit bulls have not been a major factor in Canadian fatality figures as from 1990-2007, only 1 of the 28 dog attack fatalities in Canada involved a pit bull-type dog. So it is likely that very few of the incidents either before or after the ban involved the type of dog that was banned.

The media, of course, has picked up on the story, bending whatever data they want to to prove whatever opinion they already have – like this author who compares the bites per 1000 people for two different data-sets of cities to 'prove' her point, the 'pre' number included all 31 cities studied and the "post" number just the cities that passed BSL.

So, in short, the study – which reports to show that BSL has been effective in Manitoba, doesn’t.

-          It doesn’t show a significant decrease in dog bite hospitalizations after a ban has taken place

    -   It doesn’t even indicate what number of incidents before or after a breed ban involved the banned breed

    -   Because of the extreme length of time for the study, it was not able to isolate out cultural differences in dog ownership that would influence the data in the study.

    -     And based on the low incidences of bites per 100,000 people (less than 4 in all cases), there appears to be no evidence to support that any of these jurisdictions had any type of dog bite problem in the first place.

UPDATED:

The National Canine Research Council has offered up their view of the study -- and again noted that about 90% of the total person-years covered in the study -- which essentially means that the study is a study about the effectiveness of the law in Winnipeg -- which, as noted above, showed no measurable decrease in hospitalizations before and after the enacting their breed ban. Thanks JM for the link.

Comments

[Joel]

Any analysis of the incremental cost to the taxpayer to get basically no change in public safety?

[Brent]

Joel, no word on the cost of the policies that showed no decrease in bite hospitalizations...But assuming there was one (and there always is), it was too much.

[J.M.]

NCRC has also taken a look at this.
http://bit.ly/PAIkr8

[Brent]

Thanks J.M. I've updated to include the link.

[Jen Brighton]

Your final bullet points, Brent, say it all. When will communities and governments get off the pit bull wagon and focus on reducing serious dog bites, period?

It's so ironic. All this focus on pit bulls infers that if you are bitten by a dog other than a pit bull, it's okay--let's just keep spending/wasting $$ on going after those dratted pit bulls and never mind other dogs that bite and/or kill.

[J.M.]

I`m wondering if it would be possible to find out who funded that Study?
Ontario Gov`t?

[liz]

Thanks KC Blog for the time and effort put in to breaking down this study. Noticeable that ALL other studies that show breed bans are NOT effective were ignored by the press, but this one flawed study received focus and attention. It appears the media hype on this issued still exists although there are many reporters out there now that have come out to the ban was just a "sound bite". http://www.torontosun.com/2012/03/12/worthington-time-to-repeal-pit-bull-ban
Further facts and information on Ontario's Breed Specific Ban can be obtained here www.supporthersheysbill.co,

[Selma]

@J.M.: The study was funded by CIHR, Canada's national research funding agency (formerly Medical Research Council, or MRC). It was also apparently peer-reviewed.

Thank you very much, Brent, for such a detailed report on this piece of nonsense. I don't have much of a problem with the actual research, since I think it was just trying to add something to what is basically a desert in terms of hard information. The title needs some tweaking though, even a question mark would have helped.

I DO have a problem with the twits in media who can barely interpret a press release, let alone read an actual journal article, but have no problem trumpeting their nonsense without any kind of accountability.

I wish these research types would stop sending out press releases, since the newsies aren't exactly combing PubMed looking for new information.

Thanks again this was great and extremely helpful to us here in Ontaristan.

[Gaynor Henry]

It's a ridiculous law and does nothing to account for the thousands of bites committed by other dogs, many of whom are far nastier than pitbulls. It also contributes to media hysteria and knee jerk reactions to any dog that even resembles a pitbull which of course could include boxer, lab, weimaraner, bulldog or just about any mix. It offensive and grossly prejudicial.

[Social Mange]

Thank you very much for your time in analyzing this study. As far as I'm concerned, the study has two major flaws. Its conclusions are all qualified, which makes them meaningless. Then, the researchers would have been as well off studying unicorn bite injuries, since there ain't no such thing as a "pit bull".