Considering your sources - redux - what is a reliable source?

A few weeks ago I got an email from a long-time reader in response to this post: Considering your source?

The email asked, I think, a good question: With so much information out there, often contradictory, how does one determine which sources are reliable, and which ones aren't? It's a good, legitimate question, and not always easy to determine. So, with some thought, I've put together a few thoughts on how people should think about sources in deteriming whether or not they are reliable.

#1) Anonymous sources are not reliable. If someone is not willing to put their real name behind their point of view then their opinion is not worth your time to consider. Knowledgable people put their names on their ideas. Trolls remain anonymous. Don't feed the trolls.

 

#2) Does the source have any kind of credibility in the topic they're writing about? If the topic is training dogs, or dog behavior, is this person experienced with the topic. Are they certified? Do they have credentials? Do they have a track record of success? The same goes for veterinarians and successful shelter directors. Are they successful at what they do, and is what they do relevant?  Most national organizations have these types of people on their staffs so generally their ideas are legit. Most dog trainers, shelter managers, veterinarians, etc would also have credibility. The more successful they are in their practice the better. Typically, bloggers (who don't have credentials from above) fall below that line (exceptions below). I think this is extremely important and why we should never take advice on how to prevent dog bites from someone who's only real experience with dogs involves being bitten by one.

#3 Do other credible people generally share the same point of view as the source?  ie, if one trainer has a POV that is counter to what the majority of other respected trainers have, then I am often skeptical. This doesn't mean they're always wrong, but I"m highly skeptical of people who htave POVs that are completely divergent from what most people with their expertise think. The same goes for those in the veterinary community. You can almost always find one person who is an outlyer in their viewpoint -- but if their opinion is completely different than everyone else in their field, then they are nothing more than an outlyer with little credibility.

#4 Does the source generally rely on research from other credible sources?  If a person is referencing material from recognized experts, or scientific journals, or other credible sources based on #1 above) they are generally more credible than people who's primary outside sources are those that lack credibility. This is why I think some bloggers get credibility (even without their own credentials) because they accumulate information from a variety of reliable sources. If someone relies almost solely on their own point of view, or relies almost solely on other non-credible sites to support their point of view, beware. If the person provides their own "research" but yet doesn't let anyone see the methodology or hard data that supports this "research", beware.

#5 Does the person consistently have a reliable viewpoint?  This is where some of the humane orgs get into trouble because they don't tend to be terribly consistent in their views (in part, because of their mass size). Just because someone is right once, doesn't mean they're always right. And by the same token, just because someone is wrong most of the time, doesn't mean they're always wrong. However, I'm more skeptical if they are wrong a lot in their point of view and information and more trusting from sources that have proven themselves correct often.

#6 What is someone's agenda?  Everyone has one to some degree, so figuring out their angle helps. Just because someone has an agenda doesn't make them wrong, but it's fair to note that they're likely over-playing one side of the information.

That's my list of how I look at sources. Anything else you'd add?

Comments

[Randy]

Excellent as always! Another thing I look at is what is the goal of the organization or poster. And I like to seperate the stated goal from their actual actions to see it they are the same. Unfortunatley, it seems some have a hidden agenda at times. I am also skeptical of blogs that delete counter opinions. Don't get me wrong I don't mind deletion of posts that use inappropriate language or are hatefull but to simply delete a post because it has a well thought out counter opinion does not seem right. Unless of course you just want all to believe that the "cause" is supported by all. Kind of an enforced "group think". Some pages I have seen even go so far as to imply their is some sort of grand effort against them and soon the the issue gets completely off track. You kind of touch on this in your item #6. As for anomyous posts. I would generally agree with the fact that those that post should be willing to be idnetified. However, I can understand why some that have valuable information to share may be in a position that they are not comfortable being identified. So if the post has reliable data I would still consider that. In the world of Animal Rights it seems like emotion trumps reason and it is sad to see some of the issues being debated disolve into "fight" reminecent of the third grade.

[anonymous]

#1 somewhat contradicts #4. A person can be anonymous/pseudonomous and still provide credible information from reputable sources.

I think your points are broadly, to look at the credibility of the author(#1,#2, #4, #5, #6) and whether the point of view is in line with professional consensus(#3).

One important thing when looking at logical arguments derived from factual claims:

A person can list multiple facts with sources and yet make invalid arguments or conclusions from them. On the other hand, a person may list multiple "facts" that are in fact false, but make sound logical arguments from those false premise.

The first type of errors tend to be(but is not always) easy to spot. You don't generally need to know anything about a topic to spot reasoning errors, you just need a critical mind.

For the second type of errors, if you don't have enough knowledge about the topic, you won't know that the premises are false. Which would mean that the conclusion, despite being based on solid reasoning, is invalid. So it's important to not just accept a premise at face value.

[Brent]

Touche' annoymous.

And yes, definitely agree on the types of errors -- and while I agree that #1 is not necessarily an absolute, I've found very few cases where someone remains provides information anonymously for long periods of time and that information is of much value. It can happen, but I've seldom seen it.

Randy, I generally agree that leaving out important dialogue, skepticism and counter points is often a sign of trouble as well. It does lead to group think. I also don't like when there is no commentary allowed at all -- although I do realize when some people reach "celebrity" type status that keeping up with comments can be overwhelming...

[Harold]

Kind of an enforced "group think".