What kind of doggie is that in the window?

A couple of weeks ago, the University of Florida, along with Maddie's Fund, announced the results of a study that they have been working on all summer.

More that 5,000 dog experts -- including breeders, trainers, groomers, veterinarians, shelter staff, and rescuers completed a survey in which they were asked to identify the primary breed of dog of 20 of 119 dogs in the database.

The results were then matched with DNA tests. Here are the 100 dogs, with their DNA results, and the top responses from the experts.

I realize that there is a lot of skepticism about the accuracy of DNA testing. I will note that my experience has been pretty good with the Mars Wisdom Panel test -- at least within the past couple of years. I do think it has issues at times with uncommon breeds that are less likely to have deep DNA pools for which to pull genetic markers. But otherwise I think the results are fairly reliable.

I also think it's then interesting to see the variety of different breeds that were assigned (based solely on picture identification) to different types of dogs by the panel of experts -- with many of the responses varying pretty widely.

Specifically, I think a lot of "lab mixes" end up having no Lab in them at all and a lot of "American Staffordshire Terrier mixes" have 25% or less AST DNA in them. The testing group also did pretty well on identifying purebred dogs, or ones that were 75% of more of a certain breed, but often had some pretty far-off results for ones that were 50% or less of a certain breed. This means that even people with a fairly good understanding of dog breed charactersistics did fairly poorly at identifying mixed breed dogs of which were just 1 generation away from being purebred. They were awful at identifying ones that were 2 generations or more away from being purebred.

The study is yet another in the growing list of data that suggests that looks-based breed identification for mixed breed dogs is very unreliable -- even if "experts" are doing the identification. This means that any dog bite data based on visual identification, and newspaper reports indicating breed,  is going to be VERY unreliable with the exception of truly purebred dogs.

Thus, pretty much every dog bite study in existence is inaccurate, and pretty much any law targeting dogs based on how they look is completely unenforceable. And once DNA gets diluted into mixed breed dogs, a dog can easily look very different than what you'd expect based on its genetic makeup.

 

 

 

Comments

[H. Houlahan]

Absolute junk science.

It's as if we the researchers compared the efficacy of divination by astrology against, not actual events, but used as a control divination by examining the entrails of sheep.

"Errr, sorry, your astrological chart is incorrect. The sheep's guts say that we should go to war with Iran."

[Brent]

H - I know there are a small group of people who I respect (you included) who like to post that anything DNA test related is "junk science" - but I've never seen anyone post any actual logic, or fact, behind it. Please support your claim.

[Alana]

Brent do you know what happened to these dogs? I can't see anywhere on the site that explains if they were adopted or euthanized or...? I know it doesn't matter to the subject at hand, but I always wonder.

[Brent]

I have no idea Alana.

[Jess]

The way the Wisdom panel tests work is to compare a number of chromosomes to a pool of purebred markers. These markers are based on North American purebreds.

Here's the interesting thing: a dog from another population may not actually group with the North American population. So a Saluki, even though it is purebred, from Saudi Arabia, or from Finland, or from Russia, may not be recognized as a Saluki by the Wisdom panel test. I actually contacted them about testing my Kazakh and Iranian Salukis to compare them to my domestic American dogs and that is the answer I got.

In the vast majority of mixed dogs, you will not have enough matching markers to accurately assign it to any breed. Only a dog with a purebred parent, grandparent, or great-grandparent, is going to give you any accuracy (please see their study on the Chinook outcross project dogs for more information), and ONLY on that particular ancestor, and even then, it's going to be iffy. For a dog with a truly mixed background, you just have the computer going "oh, hey, that one there is a match." It's no better than random guessing, which is why you see matches with these extremely rare breeds which are highly unlikely to be components of a mix.

Their 'purebred' test will not detect 'impurity' farther back than the great grand-parents. That should tell you something about the accuracy of the mixed breed test in an animal which may have no 'purebred' ancestry for several generations.

[Brent]

Jess,

I think we can probably assume that the vast majority of dogs in North America are ancestors of North American dogs -- and in the rare cases that they're not, the dogs are probably so highly specialized that the owners know specifically the breed of dog they have. We can definitely assume most shelter dogs in North American Shelters have North American ancestors.

Meanwhile, I completely agree that the genetic markets become goofy after you dilute the gene pool more than a couple of generations -- which is why you have some really rare breeds that show up as "8%" even though the liklihood of that breed being in the mix is miniscule. But I think that's kind of the point, as a lot of mixed breeds that are getting called "shepherd mix' or "lab mix" or "pit bull mix" in shelters have little to no ancestory link to any of those breeds...

[Jess]

You missed my point entirely. Many people will take these DNA tests as gospel. They will really think Dog 94 is *really* 10.35% Glen of Imaal Terrier, or Dog 98 is *really* 14.38% Sealyham Terrier, or Dog 99 is *really* 21.41% Black Russian Terrier and 19.86% Norwegian Buhund. All of which has a statistical chance of almost nil considering that these are all *very* rare breeds and exist in the US in small numbers which are highly unlikely to be producing multi-generational mixed dogs.

IOW, this study was done to show that labeling of mixed breeds is inaccurate. If the DNA tests are also inaccurate, what was the point? That both methods are highly inaccurate? Based on trying to communicate what DNA studies actually mean to dog breeders, who *should* be able to understand the concepts, IMO, this study will be *extremely misleading* to the general public, especially in regards to how many rare breeds are out there randomly breeding. There is *nothing* on the page preceding the results about the accuracy of mixed breed DNA tests. Nothing. The entire thing strongly *implies* that the DNA results are entirely accurate, which is UNTRUE.

Is this study going to be submitted for peer review? If so, how will they explain that even Mars doesn't admit any kind of accuracy rate on the test? In fact, the preceding page states quite baldly, "their actual DNA breed results" which is, quite frankly, a lie, given that these tests are highly inaccurate.

If they wanted to prove that visual identification does not always reflect the genetic composition, they should have used *known mixes* with a provenance that proved the genetic background. In fact, Mars itself has not *published* any kind of study using their test with known mixes, as evidence of any kind of accuracy.

[Brent]

Yes, I should have noted that the test also has issues with dogs of very distant ancestory from purebreds because the test cannot pick up the markers (which is also related to the very small breed groups having limited pool sets, which then causes them to be best matched when there are no other matches).

All that said, we do know that genetically there are many different types of markers that are fairly consistent -- so scientificaly the test can be accurate with purebreds or clean mixes -- but maybe not at very distant ancestory.

And yes, agreed that the study should have noted that the DNA isn't 100% precise. It also should have given the percentages for each of the different breeds given by the field.

[Jen Brighton]

What I get from the conversation between Jess and Brent is that at this point we cannot label dogs of unknown heredity and make laws according to supposed type or breed. Dogs should be judged on actions and especially actions of their owners.

I had a good lesson in identification just the other day. I was at an outdoor restaurant and along came a wonderful dog that looked exactly like a border collie and had the same focus on its owners (it was off leash) as my next door neighbor's purebred border collie. I mean, how can you not recognize a border collie? He looked exactly like my neighbor's dog. But no--when I asked the owners it was Australian shepherd and golden retriever.

[H. Houlahan]

First of all, Brent, I do not argue that "anything DNA related" is junk science.

I maintain that commercially-peddled dog "breed identification" products are junk science.

It is not my responsibility to "prove" that the results are nonsense -- though the preponderance of vanishingly rare breeds allegedly detected in this sample, if compared to the statistical incidence of these breeds in the US dog population, would *in itself* falsify this "study."

Barring that, google will reveal many amusing and well-documented examples of dogs of known, often purebred, ancestry coming up as ridiculous mixes when their DNA is submitted for breed-typing to one of the commercial outfits.

It is the responsibility of the entities that are *selling* this product to publish peer-reviewed research that shows that the product provides results that are better than random chance. They are making an extraordinary claim, and they are doing so in pursuit of profit. Where is the concurrent evidence?

Of course, many applications of DNA typing and analysis -- including many commercial products -- are settled science in many respects. Parentage identification is utterly mainstream and uncontroversial. (This does not mean "100% accurate in all circumstances" or "foolproof," BTW.) Genetic markers for some heritable diseases have been identified, and there are commercial tests for many of them, in many species. My own dogs' samples were used in the development of the test for the MDR1 mutation, which is generally accurate enough to be worth the moderate cost, though not 100% consistent. (Dogs that tested normal/normal have later tested mutant/normal.)

This is not an episode of CSI. We are not under the obligation to swoon at the invocation of "What the DNA says."

[Brent]

Of course H, I've seen the youtube videos of the dogs of known heritage with ridiculous results that have come back. And nearly all of them are about 5 years old, or more, at this point -- and several involved tests that didn't have the known breed in the database -- so OF COURSE it's not going to come back with accurate results.

What I'm suggesting, and that this report suggest, is that at least the Wisdom Panel test has made a lot of progress over the past several years -- including getting more samples of breeds in their database to fine-tune the markers, and adding more breeds to the database.

It's still not perfect -- and the more mixed the dog is, the less precise the markers show (this makes sense at this point).

I'm merely suggesting that the results while definitely not 100% accurate, have a fair amount of scientific validity -- certainly more so than any report at all that has relied 100% on visual breed identification.

[H. Houlahan]

Where are the *published peer-reviewed studies* that show that the Wisdom Panel has made so much "progress" that it is now, compared to visual ID by an expert (itself problematic -- who is an expert? anyone who took the survey and said they were?) as accurate, nearly as accurate, half as accurate, or in any way better than a SWAG at breed identification.

Know why those YouTube videos are all several years old? Because nobody with any sense who had already seen them is going to pay $75 for the chance to make yet another YouTube video that exposes the fiction.

I appreciate that you are looking for ammunition to debunk bogus "breed statistics" as they pertain to dog bites and attacks.

This is not the place to look, unless you like the kind of ammunition that blows up your house when you store it in the basement.

Would it be better if we suddenly had a rash of towns and states banning Norwegian buhund and harriers because someone started "DNA testing" all the frank mutts that bit people?

[Brent]

H. It's pretty telling to me that the most popular of these videos includes someone with a dog of known heritage that sent off a DNA test to a company KNOWING the known breed wasn't even in the database and then used the results to 'prove' the tests were inaccurate.

Everyone has an agenda -- and theirs was to prove the tests false for fear that they would be used for bad.

Scientists almost universally agree that the SCIENCE behind breed markings is scientifically valid. Why then would it be completely unreasonable that improvements in the knowledge and the science would follow.

In this case I think you're putting FAR too much stock in old youtube videos and less in what is being written out there currently about the testing.

[is.gd]

Do you mind if I quote a couple of your posts as long as
I provide credit and sources back to your site? My blog is in the very same niche as yours and
my users would certainly benefit from a lot of
the information you provide here. Please let me know if this
okay with you. Many thanks!

[Edgeazoo]

I have been doing some looking for documents and information on dog genetics and DNA testing. I have found the patent for the test here. http://patft.uspto.gov/netacgi/nph-Parser?Sect2=PTO1&Sect2=HITOFF&p=1&u=/netahtml/PTO/search-bool.html&r=1&f=G&l=50&d=PALL&RefSrch=yes&Query=PN/7729863
Actually, google patents has the PDF, which is better.

From the patent, remembering that this is a much smaller group of breeds than they currently test for...
"Using a Bayesian classification model on in silico mixes of genotype information for 96 markers from 429 canids representing 88 breeds, the methods of the invention have been used to correctly assign more than 98% of F1 mixes and more than 94% of F2 mixes, as described in EXAMPLE 7. Using this model on genotype information for 72 markers from 160 known mixed-breed canids, the methods of the invention have been used to correctly assign more than 96% of F1 mixes and more than 91% of F2 mixes, as described in EXAMPLE 7. "

On the other hand...

Here is by far the funniest result from the test, from 2011. http://www.diaryofalatebloomer.com/2011/04/01/day-185-doggie-dna-results-are-in/

[Katie]

I doubt the accuracy of these tests simply based on the breeds that show up. They're not consistent with breed popularity- rare breeds show up quite often and in very odd combinations that seem unlikely to happen.