The shady tactics that some food companies and organizations will use to market their foods are well known within agricultural circles. Propaganda-like videos from Stonyfield Organic and the tone-deaf Dirty Dozen list, which tries to scare people into eating fewer fruits and vegetables, are now regular parts of the landscape. And the fact that they are so commonplace means that urban consumers and others in the public may not see past the they’re-trying-to-sell-you-an-alternative-product tactics. The reality of our food system, which is the safest in history, is being masked.
For example, on the latest Dirty Dozen list, conventional strawberries were again a target, with claims that they and other common produce items are supposedly loaded with toxic pesticides. With all the press this list gets, you might assume the research behind it must be scientifically credible, right?
Well …
Developing our ‘Reliability Radar’
Some of our everyday news sources, like social media, flood us with information. And unfortunately, much of it isn’t credible. Many websites try to cloud our objective reasoning by intentionally misrepresenting data to “sell” a perspective, much like the sudden popularity of the celery juice diet that we at Dirt-to-Dinner previously wrote about.
So how can we build our analytical defenses back up? We’ve put together our own unique list — “The Discerning Dozen” — a compilation of tips to help you identify good science from pseudoscience. This way, you can be the judge when catchy news stories like The Dirty Dozen are released.
The Discerning Dozen explores four topics to help determine a site’s credentials: credibility, accuracy/transparency, bias, and quality.
In each, we’ll walk you through a few questions to ask about any article that will help you spot problems in the logic. We’ve even created an infographic to have handy for future readings!
Credibility
Though challenging to read, studies from .edu and .gov websites lay the foundation for good research. Try to stick with sources that use respected institutions to verify their practices and reporting.
1) Is it written by someone from a credible establishment? Reports and studies from recognizable institutions (academic, governmental, and/or medical) often have the most detail and are peer-reviewed, meaning other institutions have verified the research. Accredited medical journals with .org and .com sites, of course, can be good resources, too. Newbies to exhaustive reports can read the overviews typically found on the first page to understand the big picture. Still need an interpreter? Check out usefulscience.org; it’s a great resource for deciphering studies and has a simple, intuitive interface. Scholar.google.com is easy to navigate, too.
2) Does it include knowledgeable industry experts and authors? Trustworthy reports and articles come from professionals with credentials in their related industry. These experts usually provide insight garnered from data, rather than opinion and specious claims.
3) Which references does the report cite? Reliable research that’s not written by a credible establishment and doesn’t come from an industry expert should, at the very least, cite credible sources, like the U.S. Department of Agriculture or Massachusetts Institute of Technology, to substantiate any claims.
That’s a nice little circle of trust there, right? But as we know from life, few things are that easy … yes, there’s more to consider.
Accuracy/transparency:
So the site ends in .org or .com and it’s not a medical journal. What’s the next step to check its trustworthiness?
4) Is it cherry-picking data? This is when an organization only shows data that supports their agenda but fails to address conflicting info or cites data out of context, and/or relies on outdated data since nothing more recent aligns with their purpose.
5) Do other sites use the same facts? Hopefully, you can find the same information cited by other credible institutions.
6) How much do they spend on researching their cause — and how much for marketing? All non-profits must publicly disclose financials on their site; you just have to dig for it. For instance, on EWG’s Statement of Activities page, 13 percent of their expenses went toward marketing and fundraising — not horrible. However, Functional Expenses reveals a much higher figure: in addition to fundraising, each subcategory also has its own marketing expenses. Furthermore, only a paltry 2 percent of their expenses is going to research and data. That’s not much funding for finding solutions to a problem, is it?
Bias
Time to take a peek under the hood when the site’s validity is not easily determined.
7) Do a domain double-check: Sites ending in .com and .org aren’t as regulated as the .gov and .edu sites of the world, so you’ll need to dig into the “about us” page for some background. Sometimes it’s hard to tell reputable foundations from organizations peddling questionable products or ideas. So be sure to read the bios of the management team and authors to determine reliability.
8) Is there political pull? Any site can have an agenda, but not all of them explicitly state it. Advocacy websites, like PETA, are quite clear in their intentions. Reading the “About Us” page can tell you which policies, actions, campaigns, and lobbies they promote.
9) Is it clear who wrote the article? This is a simple one we often overlook. If the author isn’t stated and/or doesn’t cite sources used for its research, then you can quickly determine it was written in-house to promote the organization’s stance.
10) Is there only one answer? Does the author address alternative viewpoints on the topic? Good writers don’t omit or contest credible data that conflicts with their intent.
Quality
If you manufacture a product, you know about quality control – measures and precautions taken to ensure customers that everything is in good working order. This goes for websites, too…
11) How’s the quality of the writing? Typos? Wrong words used? Time to check the author and his/her data.
12) Don’t judge a site by its homepage. Does the site look rather simplistic? Or so polished and bursting with content that you feel like you found a goldmine of good material? No matter the design, the site is only as good as its underlying content, so vet it accordingly.
The bottom line
If you’ve read a recent report like the Dirty Dozen, now may be the time to re-evaluate it with these questions in mind. Does your brain hurt after reviewing the report again? Yes? Congrats! You exercised your critical-thinking brainpower!
Dirt-to-Dinner strives to help the public better understand how food is grown and processed, and why this is important. D2D also helps investigate the future of food and our global food system and explains how this system can be sustainable. A version of this article was originally published here.
