How I evaluate sources

I want to keep challenging my biases by reading posts I disagree with.
But I also don’t want to waste my time on nonsense or propaganda.

This week, one of my social-media friends posted a link from a blog I’d never heard of. This particular article claimed Russia is winning its war against Ukraine, and criticized a Western leader for claiming that Russia would lose a war against all of NATO. These observations seemed unlikely to me, but I try not to write blogs off just because I disagree with them. (That’s a good way to trap yourself in an ideological silo.) So I asked myself: What is this blog? Is it a reliable source?

These questions come up all the time, and by now I have a fairly standard technique for answering them. After I finished my assessment — I eventually decided it wasn’t a reliable source — I realized I’d never described the technique to Sift readers. Arguably, the technique is more valuable than the conclusions I draw with it.

The first step is obvious: Read the article in question. If, in addition to the parts I initially disagreed with, it references long-debunked claims and conspiracy theories without acknowledging the arguments that have been made against them, I feel comfortable trashing the article without wasting any more of my time. For example, if you say that voting machines stole the 2020 election from Trump, you need to explain all the states where hand recounts came to the same totals, within the usual error bands of recounts. If you have a believable explanation of that — I can’t imagine what it could be right now, but never mind — I might pay attention.

But suppose the article isn’t that obviously bad. This particular one wasn’t: Its assessment of the Ukraine War was attributed to Polish generals I didn’t recognize. So maybe the author is plugged in to sources I don’t know about, and maybe those sources know something.

So the next step is to look at the front page of the blog or news source. A Japanese proverb says: “When the character of a man is not clear to you, look at his friends.” The other articles the source is promoting are the “friends” of the article I’m evaluating. If a bunch of them are obviously nonsense, it’s not a big leap to assume the article I’m assessing is nonsense too.

The day I was looking at it, this blog was still just barely making the cut. (Today it might not. It’s full of glowing assessments of the Durham report, buying into the idea that the whole Trump/Russia thing was a hoax. More about that topic in today’s other featured post.) It had a bunch of other articles about Ukraine being in trouble, which could be legit if the article I was assessing was legit.

The final step is to look back in time. In general, well-constructed propaganda can look pretty good in the moment, but it usually doesn’t age well. The same is true of delusional points of view. In the moment, people can convince themselves of all kinds of things and be pretty persuasive about it.

The Iraq invasion is a good example. Back in 2002-2003, it was far from obvious what a stupid idea this was. Maybe Saddam did have weapons of mass destruction. Maybe the Bush administration really did know things we didn’t. Maybe Iraq was eager for democracy, and even if not, Saddam was such an awful ruler that getting rid of him would create a lot of room for improvement. When Saddam’s army collapsed so quickly, a lot of people wondered why we hadn’t invaded a long time ago. Sure, some contemporary observers saw the folly from the beginning, but a lot didn’t, and not all of them were stupid or crazy.

With twenty years of hindsight, though, hardly anybody defends the invasion any more. Time tends to clear the fog that blinds us to contemporary events.

A simpler and more recent example: A lot of pundits predicted last year (after the Dobbs decision) that voters would forget about abortion by the time the fall elections rolled around. At the time, that claim was hard to assess, but now we can clearly see that it was wrong.

So anyway, if today’s front page is hard to assess, look back six months or a year. That might be easier.

But when you do that, be careful. Because simply finding something the source got wrong isn’t discrediting in itself. We all get stuff wrong, so you will find an excuse to write the source off, if that’s what you’re looking for. If you’re trying to make an honest assessment, though, the process is a little more complicated. Finding a mistake is just the first step.

The point isn’t just to find things the source got wrong, but to see how they responded as events went some other way. What I hope to find is a reaction like Paul Krugman’s: In 2021, Krugman was wrong about the risks of inflation, and then he was slow to recognize how big a problem inflation was becoming. (If you’re looking for an excuse to write Paul off, there it is.) But that mistake bothered him as much as it bothered anyone else. He has written several columns since trying to figure out what led him astray.

In early 2021 there was an intense debate among economists about the likely consequences of the American Rescue Plan, the $1.9 trillion package enacted by a new Democratic president and a (barely) Democratic Congress. Some warned that the package would be dangerously inflationary; others were fairly relaxed. I was Team Relaxed. As it turned out, of course, that was a very bad call.

But what, exactly, did I get wrong?

The Ukraine War itself is a good topic to examine, because at the beginning, just about everybody expected Ukraine’s defenses to collapse in a few weeks. A credible military blog might have made that mistake, but then they should have spent the summer reevaluating. It’s possible that by now they might have come back around to the idea that Ukraine will lose (or not). But if they’ve been holding steady on the Ukraine-is-about-to-collapse narrative all year, they’re not credible.

So Krugman is the gold standard, but I’ll give a silver medal to anybody whose mistake made them realize they don’t understand the subject they got wrong, and who subsequently shifted their attention elsewhere. Or maybe they reevaluated and downgraded the sources they got their wrong opinion from.

So, for example, picture a Republican who took Trump’s claims of election fraud seriously at first, but then stopped repeating them when no supporting evidence emerged. They may not ever have acknowledged their mistake in so many words, but they’ve taken steps not to keep doing it, i.e., not just blindly repeating whatever Trump says any more. I’m not going to write that source off forever. On the other hand, if they’re still pushing that stolen-election nonsense today, they’re not worth my time.

So anyway, when I looked back on the past record of the blog in question, I found claims that Trump was framed in both his impeachments, the FBI framed Michael Flynn, the Russians didn’t interfere in the 2016 election, Covid was exaggerated by the Deep State, Dominion voting machines stole the 2020 election from Trump, it was Seth Rich (and not the Russians) who leaked the Clinton campaign emails, Russia has been winning the Ukraine War from the very beginning, and many others.

In short, it was down-the-line pro-Russia pro-Trump stuff, with no acknowledgment that any of those claims hadn’t panned out. So I’m not taking the new claims seriously either.

So that’s the technique: Read the article, then look at the front page, then look back until you find a mistake and see how they handled it.

Post a comment or leave a trackback: Trackback URL.


  • Prof Tom  On May 22, 2023 at 9:26 am

    Sadly Winston Churchill was correct saying history is always written by the victors – in essence we all have been lied to for 2,500 years, ever since we were told that human history was 6,000 years old, to justify bringing religion to the heathens to conquer the world and it’s indigenous population and their resources.

    The past 50 years are no exception and we still don’t know who killed JFK

    • pauljbradford  On May 22, 2023 at 10:36 am

      Winston Churchill never said history is written by the victors.
      The rest of us know who killed JFK.
      That’s such an ironic comment to add to a post about evaluating online sources.

      • Prof Tom  On May 22, 2023 at 11:06 am

        Well if you know and have proof by all means educate me who killed JFK and RFK as RFK Jr says he “knows” it was CIA? Plausible yes but a little boy Jr certainly no first hand knowledge. Like Bible written by people never at crucifixion.

        As to Churchill there are similar quotes both in Italian and French much before Churchill and it’s debated to this day if he did say or not – but I picked these as examples of that truth is not absolute unless proven in a court of law and even then on occasion can later be shown not to have been true.

        That’s why science is never settled as science is a journey not a destination.

        Schopenhauer said truth comes in three steps first ridiculed then violently attacked and finally self-evident.

    • George Washington, Jr.  On May 22, 2023 at 1:04 pm

      We do know who killed JFK. Lee Harvey Oswald killed him, and acted alone. There are over 50 separate pieces of evidence linking Oswald to the crime, and no credible evidence that anyone else was involved.

      But with 60 years of contradictory conspiracy theories, along with the outright lies of Oliver Stone’s film “JFK” (which Stone has admitted was a fabrication and a glorification of a theory even other conspiracy theorists find embarrassing), more people are aware of the grassy knoll than the Texas School Book Depository. And today, more people think there was a conspiracy than accept the truth.

  • paranoid  On May 22, 2023 at 10:04 am

    I used to do things similarly, but the volume of garbage that flows through social media has made it unsustainable to read that much. Mike Caulfield’s SIFT method, where slowing down and deciding if the source is even trying to be accurate before reading, is closer to my current method.

    • Prof Tom  On May 22, 2023 at 10:31 am

      The risk with this method is that DaVinci would never have been able to convince we live in a solar centric world as all “experts” were on payroll of Catholic Church.

      Instead start with common sense like in ideation process. Take discussion about climate change.

      Something caused Fleuve Manche floods 13,000 years ago and it was not humans then what caused such CO2 impact on Guif stream.

      Before 2023 most of us never heard about glaciers melting from underneath and that 20-70,000 seafloor volcanoes exist many of them near tectonic rifts.

      Why did we not hear? Because those with invested interest in human-only caused climate change said debate was over and others were deniers.

      Yes human causes is one big component today but was not 13,000 years ago, so why not quantify how big part is caused by seafloor volcanoes.

      After all we have no warning system for another major fleuve manche

      • Jim  On May 22, 2023 at 10:53 am

        Ah, there it is … the tell. You had to blame believers in climate change for our collective ignorance about the geologic history of our planet. Could it also be that we simply didn’t have reason to care about the Fleuve Manche floods or volcanoes on the ocean floor until our forests were aflame and affecting our ability to breathe clean air. The science of probability is still science, even if we don’t want to accept its conclusions. The oceans ARE acidifying and the rate of glacial retreat far exceeds anything in the geologic record so far. And these human-influenced changes are changing weather drivers. That has nothing to do with who does or doesn’t believe in human causation of climate changes. It’s entirely believable, based on scientific research, that human activity is amplifying the impact of naturally evolving geological processes. Politics has zero to add to this knowledge base.

  • Jim  On May 22, 2023 at 10:55 am

    Thanks for your enlightening advice on how to sift (pun intended) through the miasma of deception that pollutes our online information environment.

  • Rick  On May 22, 2023 at 11:12 am

    Interesting article, thanks! It reminds me very much of how I evaluate articles sent to me by (sometimes crazy) friends in online political discussions. The best test for me is the who-are-the-article’s-friends approach that you mention, i.e. checking to see what publication the article comes from. Even if the magazine/website/whatever isn’t mentioned, a quick search on representative text can frequently find the article in its originally published context. Often it’s from some hilariously (or tragically) insane right-wing conspiracy-obsessed site, at which point, of course, the wise thing is to be a bit more skeptical.

  • Lan Mosher  On May 22, 2023 at 11:19 am

    How can we punish these liars who keep spreading obvious Russian-Republican propaganda? Let’s start by disbarring and jailing the crooked lawyers that bring frivolous suits. Refusing to take cases and appeals that delay justice might also help. Let’s hope that other violent means, while well deserved, can be avoided. Does anyone else see the irony in the country that tracked down BinLadin but is paying to protect the man responsible for the deaths of many thousands of Americans and led a coup in plain view!

  • dhkinsey  On May 22, 2023 at 11:38 am

    how many pundits are so often wrong! why is that ?
    also – it was widely known at the time there were no WMD’s in iraq, they had been looking for a decade, those who voted for the war knew they were lying and that the war was illegal- very similar to what Putin is doing in Ukraine

    • Prof Tom  On May 22, 2023 at 12:18 pm

      Wasn’t it obvious W saw an opportunity to “finish Daddy’s job” to prove himself to daddy being better than his brother in Florida.

      Saudi terrorists funded by Afghanistan common sense not Iraq. Even Hillary voted for it.

      Power corrupts all of them.

      Obama drew line in Syrian sand against chemical weapons then backed off when had no support. Putin offered to stop these weapons and instead was allowed to take Crimea which is why he felt comfortable going in after Biden was elected but got surprised by influence of Defense and State working together.

      When approval of both houses around 30% and incumbents 80 isn’t it clear we are all fooled?

      • George Washington, Jr.  On May 22, 2023 at 1:08 pm

        Some of the Democrats who voted for the Iraq War, including Hillary, have stated that they did so in the belief that Bush wasn’t going to actually invade.

      • Prof T  On May 22, 2023 at 1:15 pm

        Isn’t that nice – we vote to invade when country is in patriotic fury then when it serves better to find excuse “we did not…”

        You can’t be half pregnant if you are in power – you are or you are not – this goes for both sides and independent too – it’s called spine or character

  • Wade Scholine  On May 22, 2023 at 12:01 pm

    I went to the link and sort of worked through the post there along with you as I read your post.

    The first thing *I* noticed about the venue was the page banner, with the Punisher skull/American Flag mashup as a sort of logo. Here we see the old saw, “a picture is worth 1,000 words,” in action. By choosing this image, the writer is telling us who he is, and we should not ignore that. The content of this image is glorification of deadly vigilante violence, wrapping itself in the flag and calling itself “patriotism.”

    Looking over to the right sidebar, I see that the writer has provided a capsule explanation of why I should care about his views: his purebred ancestry of Revolutionary-era immigrants is what he sees as his main qualification for having opinions that other people might want to know.

    Moving on to the content of the post itself, and just reading the text without bothering to check the video it’s supposedly talking about, I see a description of the Polish officer giving a completely uncontroversial description of the state of affairs: Russia is not close to collapse and has the wherewithal to continue the struggle almost indefinitely (with the right management, the Polish general does not add). Ukraine, on the other hand, is in rather dire straits and is dependent on outside help to continue the fight.

    That is the alleged blockbuster revelation. There is some kind of implied strawman being burnt here, I suppose, but that does not constitute a “claim that Russia is winning,” nor can it be taken as evidence of such a claim. It probably would be possible to dissect a number of classical fallacies out of the “reasoning” that follows the leap at the beginning of the 4th paragraph, which characterizes the conflict as “NATO’s proxy war against Russia.” It is safe to stop reading right there: the most charitable interpretation of the writer’s stance at that point is that they are a useful idiot for Russian propagandists. I will not rehearse all of the ways that it is incorrect to call the Ukraine War a “proxy fight” between NATO and Russia, but will insist that anyone who tries to make this a NATO/Russia fight is either too ignorant and/or delusional to have an opinion worth respecting, or an agent knowingly propagating disinformation.

    • weeklysift  On May 22, 2023 at 2:05 pm

      In this case, that was a more efficient way to get the same result.

    • Chris Feldman  On May 23, 2023 at 11:46 pm

      Wade, I’d really appreciate if you could explain how you can tell it’s incorrect to say the Ukraine War is a proxy fight between NATO and Russia. I mean that sincerely.

      I read many writers who take the mainstream stance that we’re engaged in a necessary response to Putin’s unprovoked aggression, but I also keep seeing people taking the stance that the US provoked this war intentionally, starting with supporting a coup in Ukraine in 2014 after the then-President of Ukraine, Yanukovych, pulled out of the NATO-friendly “EU Association” agreement. I genuinely don’t know how to determine if one of these narratives is somehow implausible. They both seem internally consistent, so which you believe seems to me depend entirely on which authors you’re predisposed to trust.

      Some, like the post Doug analyzed, seem to be amateur puffery, though that post was apparently written by one Mike Crupa, not the purebred guy described in the sidebar, but some seem harder to dismiss.

      There seem to be a whole group of relatively well known but no-longer-mainstream writers who subscribe to a very different narrative, in which our support for Ukraine against Russia is a way to drum up business for defense contractors and achieve various geopolitical goals, and the Biden administration sabotaged the NordStream pipeline to keep Germany aligned against Russia (as reported in February by former New Yorker reporter Seymour Hersh).

      One example is in which Dennis Kucinich (who admittedly seems to have taken some bizarre pro-Trump stances since he retired from Congress) compares the folly of our invasion of Iraq after 9/11 (which as Doug points out is much plainer to see now than it was at the time) to our support for Ukraine against Russia now (which has unanimous mainstream support now just like Iraq did in 2003).

      Do you believe that Kucinich, Hersh, Chris Hedges, and all the other writers taking the alternative stance must either be Russian agents or have been duped by Russian propaganda? If so, how did you know you’re right?

      Unfortunately it’s hard to apply Doug’s evaluation process to Kucinich’s post because his substack only goes back to February, so there’s not much history to refer back to.

  • Philocrites  On May 22, 2023 at 5:10 pm

    I happen to be reading a paper right now about the ways that think tanks and research centers establish and signal credibility. The authors describe the ways each reader arrives at their own “credibility judgment” using a combination of “systematic processing” (evaluating the message and the source, as you are describing, which takes more time and mental effort) and “heuristic processing” (the mental shortcuts we depend on to do most of our evaluative work). It’s “De-Constructing Credibility” by Andrea Baertl Helguero, On Think Tanks Working Paper 4,

  • Thomas Paine  On May 23, 2023 at 3:02 am

    There’s another reason those such as Krugman get a mile of consideration and deferment whereas some anonymous, breathless conspiracy nut who manages to figure out how to post somewhere on the Internet gets nothing: he’s not only won the Nobel in economics, but the even more prestigious John Bates Clark Medal as well.

    That doesn’t automatically make him right about any given opinion or forecast he might have, of course. But it does signal that his body of professional work is considered to have achieved the highest standards over time, so we best pay attention to what he has to say, even if down the road a specific correction is in order or, after careful review of accumulated FACTUAL evidence we arrive at a differing opinion.

    The problem with contemporary media is that the ability for anyone to instantaneously publish, including non-humans, has destroyed the gatekeeping function of expertise and opened the floodgates of lies, propaganda, and nonsense, making it far more challenging to extract signal from overwhelming, suffocating noise. I, for one, appreciate your efforts and contributions toward improving that ratio.

    • Prof Tom  On May 23, 2023 at 4:24 am

      Alfred Nobel’s father was called by the French press “merchant of death” because he had arms factories in St Petersburg Russia supporting the Dictator Tsar who had slavery laws until 1853 and sold slaves to Iran and Turkey. Alfred invented dynamite and continued in his fathers footsteps and after that article he white washed the Nobel name and assigned it to Norway to rule as his own country that had lost Finland to Russia did not like him continuing trading with the enemy.

      Like an old version of Putin and his supporters.

      I guess if your ancestor was 1727 colonized and then enslaved the “Nobel Blood Price” of Krugman does not make him as glorious as others seem to think.

      Common sense says if current law of federal debt limit is not increased 13-14% must be allocated of budget to service that debt obligation. Then entitlements like SS must be covered and there will be less money to spend for other things President wants to do. It does not mean automatically US fails on its debt obligations only if President refuses to pay.

      Ask Krugman and he will reluctantly agree as that’s how the numbers prove regardless of if the “expert” has Nobel blood price or not.

      Clearly this current or future presidents do not want a balanced budget limiting them politically but that’s what economically is sound in an economic sense.

      • Thomas Paine  On May 23, 2023 at 6:35 am

        Yes, let’s ask Paul Krugman.
        “The point is that in the early 2010s, the last time we faced a potential crisis over the debt ceiling, there was an elite consensus that budget deficits were a severe, even existential threat. This consensus was, in retrospect, completely wrong.”

        Between your appalling level of ignorance regarding fiscal policy and the fact that federal spending is nothing like a family’s personal budget and all the other blather you’ve swamped the replies to this blog with that call into question your attachment to reality, you’re exactly the noise drowning out the signal I’m talking about. And it needs to stop. This is not the place for whatever your agenda is.

      • Prof Tom  On May 23, 2023 at 6:55 am

        Deficits ARE indeed threats – no matter how much you try to deflect implying they are not by attacking me personally.

        France is a great example. Any country can’t default on paying its debt nor default on paying its retirement obligations. Moody and other services effectively set the price any government has on borrowing and the weaker the economy the higher the risk and cost of borrowing. This reality resulted in France in changing the rules of retirement and resulted in riots and some people dying.

        Canada some years ago showed how maximum point of taxing people was reached by increased % reducing net receipt of government income from taxes as more payers shifted over to being receivers of benefits.

        Effective use of limited resources for best results is law of economics – how then to share into the results is a political decision.

        Confusing these two is why inflation appears and central banks use interest rates to reduce them – always hurting most the poorest living from paycheck to paycheck.

        Politically maybe attractive to have more people depend on government but economically its better with more payers and less users.

      • Creigh Gordon  On May 23, 2023 at 11:30 am

        Deficits are a threat to entities that borrow and promise repayment in currencies they do not create. Households, corporations, state and local governments, Eurozone governments, and governments that want or need things that are not available in the local economy do that. The US Government does not. Absent sabotage by ignoramuses and ideologues in Congress, there is no threat, to us or to our children and grandchildren.

      • George Washington, Jr.  On May 23, 2023 at 12:03 pm

        The president cannot unilaterally decide what congressional budget appropriations to fund and which ones to ignore, except in extremely limited circumstances such as Trump diverting military housing construction funds to the wall. Biden can’t just say he will stop funding one department just to keep Social Security going. The debt ceiling sets up a constitutional crisis, where on one hand, the government is obligated to spend money that Congress appropriated, while not borrowing above the arbitrary debt limit. To comply with the 14th Amendment, Biden must violate either one of those laws, or the other. Or, he can mint a trillion dollar coin and deposit it in the Treasury, or issue consol bonds, or perform some other accounting trick.

        I hope he’s made this clear to McCarthy that he’s willing to do one of these to avoid default if McCarthy refuses to negotiate. In the end, I don’t think McCarthy wants a default on his watch, either. But it does benefit him to say “we tried to work with Biden, but he decided to break the law instead” as that will go over well with his low-information base. They can then have a performative impeachment and everyone’s happy.

      • Prof T  On May 23, 2023 at 12:16 pm

        Yes of course I said Biden referring to total administration .

        Every politician makes decisions not on altruism but on desire to be liked enough to be elected or cheered – another reason for term limits as running for posterity brings out more truth

        14th only triggers courts involvement and nobody benefits

        Small increase but balancing after ten years and give and take between them should do it and more important put on track towards reduction in spending and balanced budget

        I sincerely think getting to center point support by 65% of Americans through compromise better than this


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: