Have you ever been confronted by someone who claims something outrageous like “75% of serious dog attacks are by ‘pit bulls’”? Then they’ll point to a “study” reporting this, as if that settles the issue.
That's a big NOPE. And we have a series of #BSLbytes lined up to help you understand why.
You probably know that not all “studies” are created equal. Anyone can publish a “study” using Google research, or their friends list as a survey audience. If you want to go a step further, there are even fake journals, some with official-sounding titles, that will “publish” your study for a fee.
(Bonus tip: Researchers have a methods of evaluating a journal’s “impact factor” and that is a great place to start when presented with a “journal article” you’re not sure about).
There are many things can look for to ensure the study is legit. Some relate to the study design. A serious researcher will do everything possible to eliminate bias in his/her study and obtain their data from trustworthy sources. Many of the more sensationalistic “studies” on dog bite stats rely on internet search data or media clippings. This is not a good source of data, because it’s unlikely that all incidents would be equally reported in the news. It’s not the media’s job to report on every dog attack – just the ones they hear about, or they think will be interesting to their readers.
Let’s use an example we can more easily wrap our heads around. If we wanted to study something related to car accidents, would we scan the news and assume that gives us an accurate picture of all mishaps on our roads? No way!
Some days, relatively minor accidents might get covered. Maybe it was a slow news day. Maybe the reporter happened to be there and got some great footage. Maybe a local celebrity was involved. Maybe it tied up traffic during rush hour.
But on the other hand, a serious car accident may not get reported at all. It could have happened in the middle of the night. Maybe reporters were tied up with other assignments. Maybe it got bumped because the US President sent out an especially dramatic tweet that day. Maybe the wreckage was cleared up quickly and the families weren’t willing to speak to reporters.
So by NO MEANS would media clippings be a reliable source to study the prevalence or seriousness of car accidents in our community. We’d be better off looking at towing company records, police records, or insurance claims. Of course, these sources also have their limitations, since it is humans collecting and recording data, but they would infinitely more useful than press clippings.
To recap! Step One in analysing a study’s merit is to look at the data collection methods. Is the researcher making an honest effort to get good, solid, unbiased data? Are there any mitigating factors that might be skewing the results? Does the researcher acknowledge them and factor them into the analysis or exploit them for more dramatic results?
#endBSL #notoBil128 #Canada150 #BSLbytes
#BSLbytes is a joint initiative of Hugabull Advocacy & Rescue Society and Justice for Bullies.
For background on the #BSLbytes campaign visit the HugABull blog: http://blog.hugabull.com/take-a-byte-out-of-bsl