Earlier this week, several media outlets began covering some news of a new "research" paper that ran in the April issue of Annals of Surgery that highlighted 'pit bulls' as being more dangerous than other types of dogs (the link in that sentence goes to the San Antonio Express News -- who actually did a somewhat fair job of reporting on the research).
I debated on this for a bit -- because at first glance, the paper seemed to be a little shaky and I wasn't sure if I should even highlight it or not. But upon reading the paper in its entirety, I realized just how poorly researched the paper was.
And I think completely highlights how if you go into a research paper trying to find statistical information you can probably create it in a way that supports your point of view.
The "research paper" begins with telling story of three fatal dog attacks in the San Antonio area over the past 15 years. The stories all involve 'pit bull' type dogs -- but the stories seem to leave out important details -- including that one of the incidents involved a 7 month old child (not 11 months old as the paper indicates) that was left alone with a dog with a bite history and in another that involved a 10 year old girl that went out to rescue a chained up 'pit bull' that was being strangled by it's chain as it got hung up on a fence and during the girl's rescue attempt the girl was bitten by the scared/injured/choking dog. Some very important details in both of these incidents were left out of the "research" paper to make it seem more like specific "pit bull" related deaths than circumstantial (and tragic) attacks).
At this point, I want highlight an earlier post by Jim Crosby -- who highlights some of the details of those incidents that were ignored by the paper, as well as some glaring errors in the paper -- all of which are worth reading and I don't want to repeat most of his work here.
After one of these incidents, Dr. John Bini of the University Hospital began a 'research' project at his own hospital with the objective (taken straight from the paper): "We postulate that patients admitted to a level I trauma center with dog bites would have severe injuries and the gravest of injuries would be caused by pit bulls".
And if you go out looking to prove something, you might be able to create the statistics that support your point of view.
The 'study' then looks back 15 years worth of hospital data to support its claim.
Over the course of 15 years (Jan 1994 to April 2009), the hospital had 228 dog attacks that fit their criteria. That is roughly 15 dog bites per year that required medical treatment - -roughly one every 3 1/2 weeks. I think this important to put the size of this problem in perspective: That's one case every 3 1/2 weeks.
Of those 228 victims, breed of dog info was only available for 82 (35%) of the total cases. At this point any of the 'statistical' analysis breaks down - because the sample size is extremely low. Even the writers of the study note:
"We should state that our study is limited by its restrospective nature and the limited number of cases in which the breed of dog responsible for the attack could be determined. This lack of information may compromise the validity of our results implicating the pit bull as a major culprit in severe dog bites admitted to our trama center." (Emphasis mine).
This, of course, doesn't prevent them from trying to use the data to implicate 'pit bulls' -- in spite of their own warning of having a statistically invalid sample.
Among the statistically flawed findings:
- Of the 228 dog bite cases seen at the trauma center, 29 had 'pit bull' listed as the breed causing the attack (this is less than 2 cases per year), 53 to all other breeds combined where the breed was known, and 146 of unknown breeds.
- The study claims that 'pit bull' victims had fewer hospital-free days than other victims -- with the average 'pit bull' victim having 22.5 hospital-free days (median 28, range 0-29), where non-pit bull bite victims had 28 hospital-free-days (median 28, range 21-29). They make the same claim that for ICU-free days --with the average 'pit bull' victim having 25.5 ICU-free days (median 30, range 0-30) and non-pit bulls having 29.7 ICU-free days (with a median of 30, range 23-30). Statistically, it's important to note that the median is exactly the same for both groupings of dogs -- which means that more than half of all incidents, regardless of breed grouping, had zero ICU days and less than 2 hospital days. The main difference is the range - where one or two statistical outliers that led to 0 ICU-free days caussd a small swing in the averages because of the extremely small sample size.
- The "study" then uses data from Merrit Clifton (you can find out all you need to know about Clifton's 'studies' here, here, here and here.) and then tries to marry that data with the total number of AKC registered dogs to try to create a relative risk factor for all breeds. What the "study" doesn't mention is that AKC registered dogs only make up an estimated 1.3 million of the 78 million dogs in the US (about 1.5% of all dogs). So not only is that far from a random or representative sample, the 'study' also fails to mention that the ACK doesn't even recognize American Pit Bull Terriers -- the largest of the 'study's' grouping of dogs at 'pit bulls'. Thus, 'pit bulls' in this case are going to be clearly under-represented -- and yet, they use this data nonetheless to suppor their position.
- The study, to its credit, acknowledges scientific data that pinpoints that there is no such thing as a locking jaw in 'pit bull' type dogs, and acknowledges that jaw strength data indicates that 'pit bulls' bite with less bite force (235 psi) than Rottweiler (328 psi)s, German Shepherds (238 psi) - -and certainly less than wolves(400 psi) and Lions 600 psi).
The final conclusion states:
"Attacks by pit bulls are associated with higher morbidity rates, higher hospital charges and higher risk of death than are attacks by other breeds of dogs. Strict regulation of pit bulls may substantially reduce the US mortality rates related to dog."
It should specifically be noted here that at no point in the paper does it ever address the actual success or failures of laws that regulate breed of dog ownership -- and thus, they have no idea whether such laws would actually work. In fact, studies in this regard have repeatedly shown that because major attacks are so rare, diverting resources from dogs that have shown signs of aggression toward dogs that are a particular breed have actually DECREASED overall public safety. But this study never once explores that assumption in its conclusion.
And then, there is this little gem from the conclusion:
"These breeds (pit bulls) should be regulated in the same way in which other dangerous species, such as leopards, are regulated."
First off, it should be noted that 'pit bulls' are a type of dog, making up various breeds, within the species of canines. They are not a unique species at all - -and certainly do not in any way resemble Leopards (Leopards mre closely resemble lions, which we've already noted have nearly 3x the bite strength of 'pit bulls'.
Again, there are a lot of major errors in the study - -and I'll refer you to Jim Crosby again for those.
I think the study gives an almost laughable example of what happens when you let people with a set point of view set out to 'prove' their established point of view vs letting the data guide their point of view. In this case they relied heavily on faulty data from Merrit Clifton (and never seemed to question it) and dogsbite.org (which also has credibility issues with misleading claims) and then relies on their own data -- which they acknowledge isn't statistically valid and has a small sample size that allows a couple of statistical outliers to skew their data.
So in spite of the science pointing to something different, they still let skewed "statistical" data change their point of view -- and even drive themselves to the ridiculous conclusion that 'pit bulls' are like leopards and that a premise they never even research, breed restrictions, might help solve the problem.
Meanwhile, one has to wonder why the mounds of peer reviewed research on the limitations of breed-specific legislation, studies on the failure of breed specific laws, scientific studies on the differences in canine aggression among dog breeds, similar research (with a deeper data set and a different conclusion) in Colorado, an outstanding article on Dog Bite Prevention from the AVMA, were all ignored in favor of non-peer reviewed information. It's also amazing that there was never a single mention of any discussions with canine behaviorists, hands on experience with dogs, or even talking to the Canine Research Council.
But hey, why bother reading it if it doesn't support your point of view, right?