Jump to content

Wikipedia talk:Reliable sources/Archive 73

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia
Archive 70Archive 71Archive 72Archive 73Archive 74Archive 75

Lead doesn't say what reliable source means

Compare with WP:NPOV, WP:V and WP:OR. Rollinginhisgrave (talk) 00:58, 14 November 2024 (UTC)

What would you propose the lead to say? Nikkimaria (talk) 05:41, 14 November 2024 (UTC)
Perhaps "Reliable sources have a reputation for factchecking and accuracy. They are published, often independently from their subject." It's also troubling that the "page in a nutshell" doesn't reference reliability. Rollinginhisgrave (talk) 06:19, 14 November 2024 (UTC)
I've had a think to try to sum up the spirit of the guideline: "A reliable source is a source (four meanings in WP:SOURCE) that is just as willing to turn its critical attention inwardly as outwardly." I also think there is an implication that by doing this, they will necessarily be recognized for it which is the foundation for "reputation". This definition has a bias towards the source as a publisher/creator as described in WP:SOURCE. Rollinginhisgrave (talk) 07:45, 14 November 2024 (UTC)
If you want to begin with a definition, then AFAICT the actual definition is:
  • "A reliable source is a published document that experienced Wikipedia editors will accept for supporting a given bit of material in a Wikipedia article."
You have probably noticed the absence of words like reputation, fact-checking, accuracy, independence, etc. That's because those aren't actually required. We cite self-published, self-serving, inaccurate, unchecked, non-independent sources from known liars all the time. See also {{cite press release}} and {{cite tweet}}. WhatamIdoing (talk) 05:02, 27 November 2024 (UTC)
The first part makes some sense, although I would appreciate some clarification on the second part. If every source is reliable for something, the second sentence of this page "If no reliable sources can be found on a topic, Wikipedia should not have an article on it" is redundant. Rollinginhisgrave (talk) 05:50, 27 November 2024 (UTC)
The second sentence is incomplete. It probably ought to say "If no Wikipedia:Independent reliable sources can be found..." WhatamIdoing (talk) 07:15, 27 November 2024 (UTC)
Ah, I assumed the exclusion of independent was intentional. There does seem to be a distinction between sources reliable for verifying the content they express (wikivoice), and sources reliable for verifying that such a source expressed content (requiring attribution). I'll add in the "independent", although it does seem a bit like a non-sequitur. Rollinginhisgrave (talk) 07:22, 27 November 2024 (UTC)
I'm not sure that it should be in the lead at all, but if it's going to be there, it should match WP:NOT, which says "All article topics must be verifiable with independent, third-party sources" and WP:V, which says "If no reliable, independent sources can be found on a topic, Wikipedia should not have an article on it". WhatamIdoing (talk) 19:58, 27 November 2024 (UTC)
Yes, I'm not sure it should be in the lede either. I tried to express this by labelling it a non-sequitur.
If I were to read the lede to find out what constitutes a reliable source for a statement, I would learn it's what experienced editors thinks verifies it. This is really useless, I would have no idea how to apply this guideline, which is surely the first consideration in reading a guideline. Defer to experienced editors, who apparently deferred to experienced editors, who... Rollinginhisgrave (talk) 00:35, 28 November 2024 (UTC)
I don't think it's useless, but I agree that it isn't immediately actionable. It tells you what a RS actually is. What you need after that is some way to determine whether the source you're looking at is likely to be RS.
This could be addressed in a second sentence, perhaps along these lines:
"Editors generally prefer sources that have a professional publication structure with editorial independence or peer review, have a reputation for fact-checking, accuracy, or issuing corrections, are published by an established publishing house (e.g., businesses regularly publishing newspapers, magazines, academic journals, and books), and are independent of the subject. Reliable sources must directly support the content and be appropriate for the supported content."
It's that last bit ("appropriate") that throws over the preceding sentence. If a BLP is accused of a crime, and posts "I'm innocent! These charges are false!" on social media, then that's a source with no professional publication structure, no reputation for fact-checking or accuracy, no reputable publishing house, and no independence from the subject. But it's 100% appropriate for the article to contain this information, and 100% reliable for the statement "He denied the charges on social media". WhatamIdoing (talk) 18:01, 28 November 2024 (UTC)
The thing is, the rest of the page explains in detail what qualities are (and aren't) seen in sources that are usually accepted, so I'm not sure that duplicating that information in the lead is necessarily the best choice. Perhaps it would be better to say something like "This guideline describes the characteristics of sources that are generally preferred by editors." WhatamIdoing (talk) 18:11, 28 November 2024 (UTC)
This ridiculous situation is a product of us not distinguishing reliable for verifying the content of a source, and reliable for verifying the existence of a source. I understand that there is some complexity with DUE on this front (SME may fall into the latter category given what we can verify is that an expert is saying this but not that it is something we can put in wikivoice), but the current approach, where we try to bundle it into "appropriate" is insufficient.
Small note, we should try to keep comments such as "But it's 100% appropriate for the article to contain this information" out of the convo to keep the streams from getting crossed with WEIGHT. probably too aspirational, and my comments may be read as doing this. Rollinginhisgrave (talk) 21:52, 28 November 2024 (UTC)
The idea isn't "duplicating" information, but summarizing it. The lede of NPOV could say "An article's content can be said to conform to a NPOV if experienced editors accept it as appropriate to include" but this doesn't explain what it is as representing "fairly, proportionately, and, as far as possible, without editorial bias, all the significant views that have been published by reliable sources on a topic" does. Rollinginhisgrave (talk) 21:50, 28 November 2024 (UTC)
@Newimpartial, since I doubt that the sentence should be in this guideline at all, I don't really mind you removing the word 'independent', but I did want to make sure you understood the context. As a result of your edit, we have
  • WP:NOT saying that "All article topics must be verifiable with independent, third-party sources",
  • WP:V saying that "If no reliable, independent sources can be found on a topic, Wikipedia should not have an article on it", and
  • WP:RS implying that non-independent sources aren't all that important.
If we have to have this sentence at all, I'd rather have this sourcing guideline match the core content policies. If, on the other hand, you're thinking about the fact that NPROF thinks independent sources are unimportant, then I suggest that the place to fix that is in the policies rather than on this page. WhatamIdoing (talk) 20:09, 28 November 2024 (UTC)
It is my understanding that several SNGs can be satisfied without (what some editors consider to be) fully independent sources, not just NPROF.
Fundamentally, I think the actual policy problem is that the degree of independence that should allow a source to be used to establish the notability of a subject is poorly-defined and easily weaponized. So it seems to me that no academic is notable based on their own self-published statements, but a legitimate claim to notability can be based on reliable statements from universities and learned societies (to satisfy NPROF criteria).
An author or artist can't be notable based on their own self-published statements, either, but a legitimate claim to CREATIVE notability can be based on reliable statements from the committee granting a major award.
In a way, I think the older "third-party" language more clearly supported these claims to notability, whereas many editors will now argue that employers and award grantors are "not independent enough" for their own reliable claims to count for notability. Newimpartial (talk) 21:17, 28 November 2024 (UTC)
Indeed, an employer is not independent of the employee they tout, though an award granter (usually) is independent of the winner (though not of their award).
I agree that there might be a problem, but I think that if we're going to have this sentence in this guideline at all, it should match what the policies say.
How do you feel about removing this as unnecessary for the purposes of this guideline? WhatamIdoing (talk) 21:24, 28 November 2024 (UTC)
This may be a controversial opinion, but I think an institutional employer with a decent reputation is a reliable source for the employment history and job titles of an employee, which is what NPROF requires in some cases. No "tout"ing necessary. Under those circumstances, I don't think a more purely independent source is "better" than the employer for establishing that kind of SNG notability.
Similarly, I have most certainty seen enthusiastic arguments that an announcement by an award grantor of a grantee is not independent of the grantee (presumably for reasons of "touting") and therefore does that such sources do not contribute to notabiiity under CREATIVE (even though the latter does not actually require independent sourcing, only reliable sourcing).
I would also point out again that "third party", which is what NOT says now and what WP:V said until 2020, seems slightly less amenable to weaponization in this way than does "independent", for whatever reason. Newimpartial (talk) 22:20, 28 November 2024 (UTC)
I agree that employers are usually "reliable" for the kinds of things they publish about their employees, but they're never "independent".
Wikipedia:Third-party sources redirects to Wikipedia:Independent sources, and has for years. There is a distinction – see Wikipedia:Independent sources#Third-party versus independent – but the distinction is not observed in Wikipedia's rules. WhatamIdoing (talk) 03:37, 29 November 2024 (UTC)
If we do not remove this sentence, can we add independent, and then have a follow-up sentence or footnote saying "independent sources are not required to meet some SNGs" or the sort? Seems like that would clear up any confusion by not having "independent" in the sentence. Rollinginhisgrave (talk) 22:04, 28 November 2024 (UTC)
A second thought: I don't think using a narrowly "document" definition for source adequately accounts for the other meanings of source used (i.e. a publisher), and I'm not sure how you could do that. I imagine that's just a case of reliable source having multiple definitions depending on the way it's being used. This may be the source of conflict with the second sentence: a different meaning being invoked. Rollinginhisgrave (talk) 06:24, 27 November 2024 (UTC)
In discussions, we use that word in multiple different ways, but when you are talking about what to cite, nobody says "Oh, sure, Einstein is a reliable source for physics". They want a specific published document matched to a specific bit of material in a specific article.
"Document" might feel too narrow, as the "document" in question could be a tweet or a video clip or an album cover, none of which look very document-like, but I think it gets the general jist, which is that RS (and especially WP:RSCONTEXT) is focused on "the work itself". WhatamIdoing (talk) 07:22, 27 November 2024 (UTC)
Yes, I am thinking more Breitbart, although you describe well what the underlying dispute is, describing a publisher as unreliable is making a presumption for specific cites. Rollinginhisgrave (talk) 07:27, 27 November 2024 (UTC)
This is a known difficulty. There's the "Oh, everyone agrees that Einstein's reliable" sense and the "Yeah, but the article is Harry Potter, and Einstein's not reliable for anything in there" sense.
I have previously suggested encouraging different words. Perhaps Einstein is reputable, and an acceptable source+material pair is reliable. But we aren't there, and for most purposes, the distinction is unimportant. If someone says that "Bob Blogger isn't reliable", people glork from context that it's a statement about the blog being unreliable for most ordinary purposes. WhatamIdoing (talk) 20:10, 27 November 2024 (UTC)
Yes, the definition Google gives for reliable: "consistently good in quality or performance; able to be trusted" does not caveat that it only speaks to verifying single pieces of information. This implies the Wikipedia definition is unintuitive. I do think this isn't the biggest problem here although it does make it confusing. Rollinginhisgrave (talk) 00:35, 28 November 2024 (UTC)
We use the "able to be trusted" sense. And the question is: Able to be trusted for what? There are people you can trust to cause problems. There are source we can trust to "be wrong". That's not what we're talking about. We're talking about whether we can trust this source to help us write accurate, encyclopedic material.
A source can be "consistently bad in quality" and still be reliable. Many editors would say that anything Donald Trump posts on social media is bad in quality. But it's reliable for purposes like "Trump said ____ on social media", because it is "able to be trusted" for that type of sentence. WhatamIdoing (talk) 18:21, 28 November 2024 (UTC)
Context makes a difference. We should never ask: “Is this source reliable?” but rather, we should ask: “Does this source reliably verify what is written in our article?” Blueboar (talk) 22:23, 28 November 2024 (UTC)
  • I find the formulation added ("A reliable source is a published document that experienced Wikipedia editors will accept for supporting a given bit of material in a Wikipedia article.") quite bizarre. Firstly, it is, in effect, saying a reliable source is a source which "experienced" editors say is reliable. Isn't that utterly circular? It tells me nothing if I want to work out whether a source is reliable or not. Secondly, it fails WP:LEAD. I can't see anything about "experienced" editors' opinions being decisive in the rest of the guideline. It's completely out of the blue. Fundamentally, it's not what the guideline says. Thirdly, why do "experienced" editors views get priority? I've come across plenty of experienced editors with highly dubious views about RS and relatively new editors with compelling opinions. Fourthly, "a given bit of material". Seriously, are we going to have such slangy sloppy language in the opening of one of the most prominent guidelines in WP? Apart from that it's great. I don't disagree that a summarising opening sentence would be useful, but this is not it or anything like it. DeCausa (talk) 22:16, 28 November 2024 (UTC)
    @DeCausa, what would you write instead, if you were trying to write the kind of definition that MOS:FIRST would recommend if this were a mainspace article?
    Usually, editors start off trying to write something like this: "A reliable source is a work that was published by a commercial or scholarly publisher with peer review or editorial oversight, a good reputation, independent, with fact-checking and accuracy for all."
    And then we say: We cite Donald Trump's tweets about himself. They are ☒N self-published with ☒N no editorial oversight, ☒Nno peer review, a ☒N bad reputation, ☒N non-independent, with ☒N no fact-checking, and ☒N frequently inaccurate. And despite matching exactly 0% of the desirable qualities in a reliable source, they are still 100% checkY reliable for statements that sound like "Trump tweeted _____".
    An accurate definition needs to not completely contradict reality.
    As for your smaller questions:
    1. Is this circular? No. "A reliable source is whatever we say it is" tells you what you really need to know, especially in a POV pushing dispute, which is that there is no combination of qualities that can get your source deemed reliable despite a consensus against it. It doesn't matter if you say "But this is a peer-reviewed journal article endorsed by the heads of three major religions and the committee for the Nobel Peace Prize, published by a university press after every word was publicly fact-checked, written by utterly independent monks who have no relationship with anyone!": If we say no, the answer is no.
      Does that sentence tell you everything you need to know? No. I agree with you that it does not tell you everything you need to know about reliable sources. We have been discussing that in the thread above.
    2. See WP:NOTPART.
    3. Why "experienced" editors? Because, to be blunt, experienced editors control Wikipedia, and especially its dispute resolution processes. A source is not reliable just because three newbies say it is. However, it's unusual to have three experienced editors say that a source is reliable, and end up with a consensus the other way, and I don't ever remember seeing that happen with only newbies opposing that.
    4. If you dislike that particular phrase, then I invite you to suggest something that you prefer. As long as it prioritizes text–source integrity – by which I mean that we're talking about whether this source is reliable for this word/phrase/sentence/paragraph rather than vaguely about "the subject in general" or "some unknown part of the thousands of words on this page" – I'm personally likely to think it's an improvement.
    WhatamIdoing (talk) 04:03, 29 November 2024 (UTC)
    I don't think putting forward a strawman and then demolishing it is particularly useful. Your strawman clearly fails for its narrowness. The current text (and the premise of the above thread) fails because it's circular (yes it is - I'll come back to that) and doesn't tell us anything about what an RS is.
    To do that, one has to reach back to the fundamentals that, absent the detail of WP:RS, gives editors a guiding principle by which to judge whether a source is RS or not. For me, the fundamental concept is that RS are the means by which WP:V is delivered in practice. If it delivers it, it's RS. If it doesn't, it's not. I'm sure there are a number of ways this can be formulated. Here's one - I don't say it is the only one or the best one or it can't be improved. A reliable source is a previously published source of information in any medium which has all the attributes necessary to enable a reader to check the veracity of a statement in an article and to be assured that it is not derived from editors' own beliefs or experiences. That's pretty much all you need to know to understand why this is RS for "Donald Trump has claimed on Twitter that China created the concept of global warming" but not for "global warming was created by China" (to take your example).
    Turning to your numbered points:
    1. How can it be anything other than circular? I have 37k edits over 12 years so I would think by most standards I am "experienced". So if I am in doubt on whether something is RS or not and I look at that I'm told that whatever I think is RS is RS. There are no inputs given to me other than what I already thought and what I already thought is validated. Circular. Obviously WP:CONSENSUS is how all disputes are ultimately resolved. But what you say is clearly not true - otherwise !votes contrary to policy in RFCs wouldn't be disregarded. Of course 'might is right' works from time to time here, but not always or inevitably and to embed and codify it is really inappropriate in a guideline.
    2. The great thing about WP policies and guidelines is that (for the most part) they reflect commonsense. Taking a bureaucratic approach to ignore LEAD, a very commonsense guideline, doesn't make sense to me. Of course, the opening should be relatable to what is actually said in the policy. There is absolutely nothing in the current opening that foreshadows the rest of the guideline.
    3. Just need to look at RFC's to see that's not how we work. And ultimately an RS dispute will end up there. And what's an "experienced" editor anyway? I can see it feeding arguments along the lines of "I've got 40k edits and you've got 2k so my opinion counts more than yours".
    4. For the reasons I've given above, the current text isn't salvageable. It needs a different concept. But a "bit of material" is particularly cringworthy.
    I really think the addition should be reverted. Also, given its prominence whatever the final proposal is it needs an RFC rather than 3 editors deciding it. DeCausa (talk) 11:23, 29 November 2024 (UTC)
    I like the direction your definition takes, except that it's not really reflective of actual practice. Here is an example of a source that has all "the attributes necessary to enable a reader to check the veracity of a statement in an article and to be assured that it is not derived from editors' own beliefs or experiences", but which is not a reliable source:
    • Source: Soviet propaganda article in a government-funded partisan newspaper saying HIV was intentionally created as part of a American biological warfare program.
    • Article content: "HIV was created as part of a US biological warfare program."
    The reader can check source and "be assured that it is not derived from editors' own beliefs or experiences". That (dis)information definitely came from the cited newspaper and not from Wikipedia editors.
    But it's still not a reliable source, and at the first opportunity, editors will remove it and, if necessary, have a discussion to demonstrate that we have a consensus against it. WhatamIdoing (talk) 00:44, 30 November 2024 (UTC)
    It's not a reliable source for exactly the same reason (in my example) as the Trump tweet is not a reliable source for the statement "global warming was created by China". It not being derived from editors' own beliefs is only an additional qualifier. The main one is that "it has all the attributes to check the veracity of the statement". When the tweet is used to say Trump said X, it has all the attributes to check the veracity of the statement "Trump said X". However, it has none of the attributes to check the veracity of the statement that "global warming was created by China" (expertise, independent fact-checking etc etc.) it's exactly the same as your Soviet propaganda article.
    I'm reverting the addition to the guideline - it consensus needs to be more than a couple of editors for something as prominent as that. DeCausa (talk) 16:09, 1 December 2024 (UTC)
    WP:V begins this way:
    "In the English Wikipedia, verifiability means that people are able to check that information comes from a reliable source."
    Older versions opened with statements like "Wikipedia should only publish material that is verifiable and is not original research" (2005) and "The threshold for inclusion in Wikipedia is verifiability, not truth. This means that we only publish material that is verifiable with reference to reliable, published sources" (2006) and "The threshold for inclusion in Wikipedia is verifiability, not truth. "Verifiable" in this context means that any reader should be able to check that material added to Wikipedia has already been published by a reliable source" (2007).
    There's nothing about veracity in the policy, nor any similar word, except to tell people that The Truth™ explicitly isn't our goal.
    Additionally, if we demand veracity, rather than verifiability, we can realistically expect POV pushers to exploit that. "Sure, you cited a scientific paper about HIV causing AIDS, but that paper doesn't provide enough information to 'check the veracity of the statement'." Or "You cited lamestream media to say that Trump lost the 2020 election. You can't actually 'check the veracity of the statement' unless you go count the ballots yourself." And so forth.
    I think we have intentionally avoided any such claims for good reasons, and I don't think we should introduce them now. The purpose of a source is to let others (primarily other editors, since readers rarely click on sources) know that this wasn't made up by a Wikipedia editor but was instead put forward by the kind of source that a person of ordinary skill in the subject would be willing to rely on. WhatamIdoing (talk) 19:08, 1 December 2024 (UTC)
    I think you are missing the point. "Verifiability not truth" is about something else entirely. That's about the encyclopedia reflecting what's published in reliable sources rather what an editor believes to be the truth. It's not saying that it has no bearing in determining what an RS is. Fundamentally, we need to use sources that have all the attributes that support the objective of veracity. Whether or not they do convey the "truth" is a different question - and we can't know that. All we can do is check as best we can that they have the attributes that can potentially do that. If we do anything else then we just repeat flat earth nonsense. This is basic. DeCausa (talk) 21:59, 1 December 2024 (UTC)
    You want us to define a reliable sources as having all "the attributes necessary to enable a reader to check the veracity [correctness or accuracy; conformity to truth; truthfulness; accordance with the truth; the quality of being true, honest, or accurate] of a statement".
    But you don't think veracity has much to do with the truth.
    I wonder if a word like trustworthy would serve your purposes better. That is, we can't promise you it's true, and almost none of the sources will give you the material necessary to check the actual veracity, but we can give you a source that we trusted. WhatamIdoing (talk) 23:33, 1 December 2024 (UTC)

Circularity

This is about the comments on the definition ("A reliable source is a published document that experienced Wikipedia editors will accept") being circular.

Circular reasoning is this case: "A is true because B is true; B is true because A is true." For example: "This drug was proven to work because 100% of the people taking it got better afterwards. I know they get better because of the drug (and not due to random chance, placebo effect, natural end of the disease, etc.) because the drug has been proven to work."

Circular reasoning is not: "A reliable source is whatever editors say it is".

This is the old joke about reality, perception, and definition: Three baseball umpires are talking about their profession and the difficulty of making accurate calls in borderline cases. One says "Some are strikes, and some are balls, and I call them as they are." The next feels a little professional humility is in order and says "Some are strikes, and some are balls, and I call them as I see them." The third thinks for a moment and says "Some are strikes, and some are balls, but they ain't nothing until I call them."

The last umpire speaks of definition: What makes a source be "reliable" is that editors accept it. If it is not reliable, it is not reliable because they do not accept it. They might (and should!) give reasons why they don't accept it, but it is not unreliable because of the reasons (which may vary significantly between editors, or even be completely incorrect); it is unreliable because they don't accept it.

Imagine that you have applied for a job somewhere. They do not choose to hire you. You ask why. They say "We felt like you had too little education and not enough experience". You reply: "That's wrong! I've got five advanced degrees, and I've been working in this field for a hundred years!" Even if you're 100% right, you're still not hired. So it is with sources: No matter what the rational arguments are, a source is reliable if we say it is, and it's unreliable if we say it isn't.

When you write above that if "I am in doubt on whether something is RS or not and I look at that I'm told that whatever I think is RS is RS", you are making the mistake of assuming the plural is accidental. It is not what "I" think; it is what "we" think. Another way to say it might be "A reliable source is a published document that The Community™ will accept", though that will draw objections for its unsuitable level of informality. WhatamIdoing (talk) 01:10, 30 November 2024 (UTC)


Have a more simple "Ovrview section" .....then let page explain... KISS....somthing like

A reliable source is one that presents a well-reasoned theory or argument supported by strong evidence. Reliable sources include scholarly, peer-reviewed articles or books written by researchers for students and researchers, which can be found in academic databases like JSTOR and Google Scholar. Magazine and newspaper articles from reputable sources are generally reliable as they are written by journalists who consult trustworthy sources and are edited for accuracy. However, it's important to differentiate between researched news stories and opinion pieces. Websites and blogs can vary in reliability, as they may contain misinformation or be genuine but biased; thus, it's essential to evaluate the information critically. Online news sources are often known for sharing false information.

Wrote a fast essay Wikipedia:What is a reliable source?Moxy🍁 16:30, 1 December 2024 (UTC)

@Moxy, you said:
  • A reliable source is one that presents a well-reasoned theory or argument supported by strong evidence
but if Trump posts on Twitter that China caused global warming, and we write in an article "Trump claimed China caused global warming on Twitter", where is the "well-reasoned theory" or "strong evidence"? There is no theory at all, and there is no evidence that this wasn't the one time when someone picked up his device and tweeted a joke post for him.
Or think about something perfectly ordinary, like "Big Business, Inc. has 39,000 employees". We'd normally cite that to the corporate website. There's no "reasoning", no "theory", no "argument", and no "evidence" involved.
This sort of strong source might be true for major sources on substantial topics (e.g., as the main source for Health effects of tobacco), but it's inapplicable to most of our ordinary everyday content. WhatamIdoing (talk) 19:14, 1 December 2024 (UTC)
Should deal with the vast majority of articles. Trump posts on Twitter is not reliable for statements of fact just the opinion. The vast majority of articles dont have to deal with junk of this nature...let these cases be dealt with edit by edit. As for Big Business not sure it's even worthy of inclusion. .but if no other source contradicts the statement who really cares. Moxy🍁 19:26, 1 December 2024 (UTC)
Sources have to be measured against individual statements, not whole articles.
The vast majority of articles deal with quite a lot of very ordinary content: The company said they have n employees. The singer said she got married last week. The author wrote this book. The definition of reliable source has to fit for all of these circumstances, not just the contentious ones. WhatamIdoing (talk) 19:31, 1 December 2024 (UTC)
Then what is needed is a definition of reliable source (used academically) as we have now vs a source for a statement that in no way would ever come under a peer review process or be historically relevant in the future. What is needed is more and separate information about how we can use non-academic sources. Should have a page dealing with modern media junk, company or government data and social networking sites that promote oneself. There's a whole generation coming up consisting of 50% of English-speaking editors that will never go on to formal education to understand the differences. Moxy🍁 22:35, 1 December 2024 (UTC)
I think that it would be desirable to have a definition for reliable sources, but I suggest to you that this definition says much more about us (i.e., the Wikipedia editors who are making the decisions about which sources to "rely on") than about the objective, inherent qualities of the sources.
As I said above, a source can have none of the qualities we value and still be reliable for certain narrow statements. It can also have all of our favorite qualities and still be unreliable for other (e.g., off-topic or misrepresented) statements.
IMO the unifying theme between "This comprehensive meta-analysis of cancer rates, published in the best journal and praised by all experts, is a reliable source for saying that alcohol causes 8.7924% of cancer deaths in developed countries" and "Yeah, his tweet's a reliable source for the fact that he said it" is what the community accepts for each of those statements. That's our baseline: It's reliable if we say it is, and it's not reliable if we say it isn't. WhatamIdoing (talk) 23:41, 1 December 2024 (UTC)

I wrote something up after this, if only to clarify my understanding. Could someone here have a look and see if I am accurately reflecting what's conveyed here? Particularly the part on 'intangible preferences' I'm a little shaky on.

Strictly speaking, a source cannot by itself be described as reliable. A source can only be reliable for verifying a piece of information.
There are two types of statements a source can verify: those that are attributed and those that are not. With the former, editors look for attributes such as independence, peer-review and a reputation for fact-checking. This can indicate it is reliable for such a statement. They also look for counter-considerations, such as contradicting other sources that also have such attributes and a lack of expertise to make such a statement.
How considerations and counter-considerations are weighted, and the determination of reliability for a statement is made, comes down to any consensus editors can form. The community has some preferences for which considerations are more relevant; experienced editors are more able to apply such intangible preferences. If a source meets this, the material can be put in wikivoice.
When a source falls short of this, we can move from using the source to verify the content of what they said, to verifying that they said something. If the source has a credible claim to representing what it purports to be, it is considered a reliable source to verify the attributed claim. An example of a "credible claim": Donald Trump's Twitter may post something, but whether the tweet is a reliable source that Trump said it or merely that his Twitter account posted it is evaluated (considering the potential that his social media team tweeted it).

Rollinginhisgrave (talk) 02:50, 2 December 2024 (UTC)

I was with you until the last sentence. The difference between "Alice said" and "The people Alice hired for the purpose of saying things for her" is immaterial.
The difference that matters is whether Trump's tweet can be presented as something that is true ("Trump likes McDonald's food", with a tweet saying "At McDonald's. Best french fries in the world. Love all their stuff. Should make the Navy serve this in the White House.") or only as something that he said ("Trump once tweeted that Ruritanians 'should be deported from their own country'", with a tweet saying "Those Ruritanians are strange! They should be deported from their own country!!"). WhatamIdoing (talk) 06:31, 5 December 2024 (UTC)
Both matter. We can see the former matter when we write The Art of the Deal as ghostwritten rather than authored by Trump even when "The people [Donald] hired for the purpose of saying things for [him]" wrote it. Rollinginhisgrave (talk) 06:40, 5 December 2024 (UTC)
But we can't use a ghostwritten book to say that it was ghostwritten. We need a different source for that (i.e., one that actually says it was ghostwritten). WhatamIdoing (talk) 07:28, 5 December 2024 (UTC)
Oh I understand your point. Yes, this is contingent on external sources making comments to this effect rather than simple editor speculation, I should have made that clearer. Rollinginhisgrave (talk) 07:48, 5 December 2024 (UTC)
On the second half, I've had a think. I don't think it is a distinct issue from non-primary sources. I think the key consideration from the first is independence. For the second, due to ambiguity in tone, to put it in wikivoice would be an extraordinary claim for which the tweet is insufficiently reliable. Interested to hear your thoughts. Rollinginhisgrave (talk) 08:42, 5 December 2024 (UTC)
Were you using non-primary and independence as interchangeable words in this comment? They're not.
Assuming 'the second' is the made-up tweet about the purely fictional Ruritania, the tweet would be:
  • primary for its contents (WP:ALLPRIMARY)
  • non-independent of himself/his view
  • self-published because the author and the person who made it available to the public are the same.
But it would be reliable. All sources are reliable for statements that say "The source contains the following words: <exact words in the source>" or "The person posted <exact words the person posted> on social media".
With this reliable source in hand, one still has to decide whether the content belongs in the article. Verifiability does not guarantee inclusion. Just because you have a reliable source doesn't mean the inclusion would be WP:DUE or comply with rules against WP:INDISCRIMINATE inclusion of random factoids. But even if you conclude that it's not sufficient to justify putting it in the article, the source is still reliable for the statement. WhatamIdoing (talk) 20:01, 5 December 2024 (UTC)
No I wasn't, in fact I was trying to make the more bold claim that the principles of independence could be applied to a primary source. The idea being that if a claim is self-serving, the publisher is less reliable for its contents, necessitating attribution. There's no principle of "self-serving", but there is of bias and independence; I think the latter fits better here as a biographical subject can have more or less of a vested interest in a topic, which is what an independent source is: "a source that has no vested interest in a given Wikipedia topic".
"But it would be reliable." There's some imprecision here between reliable with or without attribution. You are saying all are reliable for the latter, which is true. How that attribution is given depends on whether the source "has a credible claim to representing what it purports to be". For the former, the reason there is an affirmative assumption of reliability for claims made by individuals is because they are regarded as a SME, on matters such as whether Donald Trump does indeed love McDonalds. Counter-considerations then apply for reliability: self-serving, contestable etc.
Agree on DUE. I am trying to keep that discussion out of this one, with mixed results. Rollinginhisgrave (talk) 21:53, 5 December 2024 (UTC)
  • Source1 says <something>.
  • Is Source1 reliable for the claim that Source1 says "<something>"? Yes.
  • Source2 says <something self-promotional>.
  • Is Source2 reliable for the claim that Source2 says "<something self-promotional>"? Yes.
There is no difference here. The Source2 is not less reliable for its own contents. Just because it says something self-promotional does not make you any more likely to read Source2 and think "Huh, Source2 didn't really say <something self-promotional>". WhatamIdoing (talk) 23:36, 5 December 2024 (UTC)
The claim made by source one can be put in wikivoice, the claim made by source two should be attributed. It seems obvious to me that source two is less reliable for content, we can't trust them as easily as a source that isn't self-promotional because they have strong, relevant motives contrary to accurately reflecting the truth. They are both just as reliable as each other for "source said something". The question here is the distinction you drew: "The difference that matters is whether Trump's tweet can be presented as something that is true or only as something that he said." For the former we can present it as true, for the latter we can only present it as something he said. Rollinginhisgrave (talk) 23:50, 5 December 2024 (UTC) Rollinginhisgrave (talk) 00:02, 6 December 2024 (UTC)
I agree with you that They are both just as reliable as each other for "source said something".
WP:INTEXT attribution is required whenever editors choose (i.e., by consensus) to require it. That is not always for primary sources, not always for non-independent sources, and not always for self-promotional sources.
Consider a self-published, self-promotional, non-independent primary source:
  • Social media post: "Congratulations to all the staff on our 20th anniversary! Thank you to all the customers who have supported us since 2004. We're going to give away treats to the first 100 customers today, and we'll have hot gas station grub all day long for the low price of $5."
  • Wikipedia article: "WhatamIdoing's Gas Station opened in 2004."
Nobody would expect us to add INTEXT attribution, e.g., "According to a social media post by WhatamIdoing's Gas Station, the business opened in 2004." The source is self-promotional, but our use of it is for non-self-promotional, basic facts. We can trust the source even though the source is promotional. WhatamIdoing (talk) 02:16, 6 December 2024 (UTC)
While it may be true that INTEXT is technically only applied whenever editors choose, INTEXT also seems to make clear — "For certain frequently discussed sources, in-text attribution is always recommended" — that when material is DUE but is insufficiently reliable to be put in wikivoice, it is generally attributed.
If I may, a source can only be reliable for a given piece of material. It does not speak to the whole source as we've noted above extensively. The material the text is supporting, "WhatamIdoing's Gas Station opened in 2004" is not particularly promotional. If the material being supported was "WhatamIdoing's gas station gave away treats to the first 100 customers on this day", that the material was self-serving would be a consideration in whether it could be considered reliable or not. Rollinginhisgrave (talk) 02:39, 6 December 2024 (UTC)
If you see an advertisement from a business that says it will give away treats to the first 100 customers, what makes you think you can't rely on that claim? Imagine that a store near you ran an ad making exactly that claim. Would you show up wondering whether it was true? Or would you be confident that (barring extenuating circumstances, assuming the whole store hadn't burned down over night, etc.) that they actually would do this?
A promotional source doesn't give us a reason to include promotional material, but it is reliable for the facts of the promotion. A Wikipedia editor would likely omit any mention of giving away treats, but if they included it, they would not say "The gas station posted on social media that they would give away treats to the first 100 customers", or anything remotely close to that. In fact, INTEXT would discourage that because "in-text attribution can mislead". Adding in-text attribution in that case might make it sound like we think the advertisement was lying.
There are self-serving cases when in-text attribution is necessary. Consider "Richard Nixon said I am not a crook" vs "Richard Nixon was not a crook", because the self-serving source is not reliable for a statement of fact. But a self-serving source can be reliable for a plain statement of fact, and when it is reliable for that plain fact, it should be presented in wikivoice – or omitted entirely for reasons unrelated to the source being reliable for the statement. WhatamIdoing (talk) 04:16, 6 December 2024 (UTC)
I agree with all of this, and I did when making my comment (hence why I said "would be a consideration", not the consideration). I think we're in agreement and I don't think any of this is at odds with my initial laying out of reliable sourcing, although some more clarifications may be needed.
Coming back now to the case you initially raised of Donald Trump liking McDonalds being in or out of wikivoice, I am saying the primary consideration is whether he is getting something out of it. More important considerations don't apply as they do with your gas station promo. If he tweeted endorsing MyPillow instead of McDonalds a different assessment of how self-promotional the material was would be made. I've lightly edited the original comment into User:Rollinginhisgrave/How I understand reliable sources. Do you think it needs further revision? Would you put it differently? Rollinginhisgrave (talk) 04:46, 6 December 2024 (UTC)
Consider this sentence above: If the material being supported was "WhatamIdoing's gas station gave away treats to the first 100 customers on this day", that the material was self-serving would be a consideration in whether it could be considered reliable or not.
This is wrong. Whether the material was self-serving is a consideration for "how we should handle it" or "what we should stick in the article", but it's not a consideration for "whether it could be considered reliable". That source is 100% reliable for that statement. It's just not something we'd usually want to stick in an article for non-reliability reasons.
The tendency in discussions on wiki is to fall for the Law of the instrument: I'm familiar and comfortable with using the WikiHammer of Reliability, so when the actual issue is anything else, I still pull out my hammer. I ought to use the whole toolbox and say that this is undue, unencyclopedic, poorly written, off topic for this article, etc., but instead I'm going to say: It's self-serving, so it's not reliable. It's trivia, so it's not reliable. It's a tiny minority POV, so it's not reliable.
The sentence that you wrote above should say something like this: If the material being supported was "WhatamIdoing's gas station gave away treats to the first 100 customers on this day", that the material was self-serving would be a consideration for multiple policies and guidelines, not to mention common sense, that are not about whether this source is reliable for this statement. WhatamIdoing (talk) 07:07, 6 December 2024 (UTC)
That source is 100% reliable for that statement. This doesn't contradict my statement. Whether it is self-serving is a consideration for whether it is reliable: this is why we hold "independence" as indicative of reliability. As I said above, determining whether it is reliable is a product of weighting "considerations and counter-considerations": here the counter-consideration is more impactful. It can still be a consideration even if it is ultimately overruled in a final assessment by counter-considerations.
And "whether the material was self-serving" is also a consideration for other multiple policies and guidelines. In this case, those are more relevant here; I am not discussing them however since this is a conversation about defining "reliable sources", not NPOV. As you say, it can be a reliable source while still being UNDUE, and I don't think I've mentioned anything on the material being verifiable necessitating inclusion. Rollinginhisgrave (talk) 07:57, 6 December 2024 (UTC)
For:
  • a statement in the Wikipedia article that "WhatamIdoing's Gas Station is giving away treats to the first 100 customers", and
  • a source that is an actual advertisement saying the same thing,
then: whether that advertisement is self-serving is not a consideration for whether the advertisement is reliable for a description of the promotional activity.
There are no worlds in which we would say "Oh, this advertisement would be reliable for "WhatamIdoing's Gas Station is giving away treats to the first 100 customers" except that the advertisement is just too self-serving". The advertisement is always reliable for that particular statement. WhatamIdoing (talk) 00:22, 7 December 2024 (UTC)
Again, none of this contradicts what I'm saying. There's a confusion of process and outcome. While determining if this source can verify this piece of material, we necessarily have to make a judgement on whether the material being self-promotional (which can indicate a source is unreliable in verifying material) would impact such a determination. Here, you make it clear that you think it is irrelevant. Which I agree on. But to do so, you have necessarily considered its relevancy; to disregard first necessarily requires consideration. This is the consideration I am speaking of. It is an application of determining if a source is reliable as the evaluation of considerations for why it may be reliable and counter-considerations for why it may not be. "The advertisement is always reliable for that particular statement" and "a consideration in determining if such a source includes it being self-promotional possibly indicating unreliability" are simultaneously true. I think I've said my peace here and am repeating myself so if my point doesn't come across I'll leave this here, although I obviously hope it does. Rollinginhisgrave (talk) 03:54, 7 December 2024 (UTC)
I don't think we're going to reach an agreement. I can't think of an example of an advertisement that we would consider reliable if we judged it non-self-promotional but unreliable if we judged it self-promotional. WhatamIdoing (talk) 18:15, 9 December 2024 (UTC)
It's not the advertisement as a whole being judged as more or less self-promotional, but the material therein. An advertisement could make two claims: Coca-Cola was founded in 1886. Pepsi puts poison in their cola. There is obviously a distinction to be made to the extent of self-promotion between the claims, even though they both appear in an advertisement. Rollinginhisgrave (talk) 13:09, 15 December 2024 (UTC)

Coming back to this, I can see the GNG makes an attempt to define reliable at odds with the above discussion: "Reliable" means that sources need editorial integrity to allow verifiable evaluation of notability, per the reliable source guideline." In addition to this, it requires sources be independent, so it's not that it's just talking about reliable as it relates to notability, but making a claim about reliability in general. Rollinginhisgrave (talk) 13:09, 15 December 2024 (UTC)

Everything in the GNG is written "as it relates to notability". Among its awkward statements are that "Sources should be secondary", which is true(ish) for notability but has nothing to do with the definition of either source or reliable. WhatamIdoing (talk) 19:16, 16 December 2024 (UTC)
  • Part of the confusion here stems from omitting context from the discussion. The goal is for sources to “reliably” verify what we write in our articles. However, the question of whether a specific source does this (or not) depends on what we write. Are we attempting to verify a statement of fact written in wikivoice (where we state “X” as fact, verified by citing source Y) or are we verifying an attributed statement of opinion (where we note that Y said “X”, verified by citing where Y said it). The same source can be unreliable in the first context, but reliable in the second context.

This, of course, does not mean we should write either statement (other policies impact what we write, as well as how and where we write it)… it only means that the specifics of reliability can shift depending on context. Blueboar (talk) 15:05, 15 December 2024 (UTC)

Preprints bullet, general audience writing about scholarship

Right now, the Preprints bullet says in part

Preprints, such as those available on repositories like arXiv, medRxiv, bioRxiv, or Zenodo are not reliable sources. Research that has not been peer-reviewed is akin to a blog, as anybody can post it online. Their use is generally discouraged, unless they meet the criteria for acceptable use of self-published sources, and will always fail higher sourcing requirements like WP:MEDRS.

Would it be clearer to replace that with something like

Preprints, such as those available on repositories like arXiv, medRxiv, bioRxiv, or Zenodo, have not undergone peer-review and therefore are not reliable sources of scholarship. They are self-published sources, as anyone can post a preprint online. Their use is generally discouraged, and they will always fail higher sourcing requirements like WP:MEDRS.

The similarity to blogs is that they're self-published, and there's no need to compare them to blogs to say that, especially since they're unlike blogs in other ways (e.g., in citing literature). I also removed the phrase about the criteria for acceptable use of self-published sources, as preprints generally come from "expert" sources, which is an exception for using SPS. Notwithstanding that preprints generally come from expert sources, their use is discouraged because we don't want readers to confuse them with peer-reviewed research and because editors should use reliable non-self-published sources when available, which often exist in the peer-reviewed literature.

Also, does it make sense to add something about popular discussions of scholarship (e.g., in a magazine)? FactOrOpinion (talk) 22:44, 16 December 2024 (UTC)

Whatever the outcome is, it is silly to say "they are not reliable sources. [A sentence later] they can be reliable sources." Rollinginhisgrave (talk) 23:33, 16 December 2024 (UTC)
The passage is fine as is, IMO. See also WP:SPSWHEN. Headbomb {t · c · p · b} 00:03, 17 December 2024 (UTC)
I was prompted by an exchange on the Autism talk page where another editor seemed to interpret "Research that has not been peer-reviewed is akin to a blog, as anybody can post it online. Their use is generally discouraged" along the lines of "the use of sources that have not been peer-reviewed is akin to a blog and their use is generally discouraged" (i.e., interpreting it as text about other sorts of sources, not limited to preprints or non-peer-reviewed scholarship more generally, as is the case with some conference proceedings). Not an accurate reading of those sentences, but it made a couple of us wonder whether the wording of the preprints paragraph could be improved. FactOrOpinion (talk) 17:03, 17 December 2024 (UTC)
I encouraged FOO to start this because of that discussion. We don't need a sentence that can be quoted out of context to say that "Research that has not been peer-reviewed is akin to a blog", because that isn't true. Outside the hard sciences, research is routinely published in non-peer-reviewed books, which are definitely not "akin to a blog". Research gets published in magazines and newspapers.
I like the proposed re-write. WhatamIdoing (talk) 23:05, 17 December 2024 (UTC)

Autocracy corrupting reliable sources, censoring what is published

As media are increasingly careful to avoid lawsuits like ABC recently settled, self-censorship will limit the neutral information available to publish. It is said that RFK will even censor releases by the FDA. In this type of media environment, truths must be published underground or at least in less well-resourced publications. How can reliable sources definitions deal with this new state of affairs? Jdietsch (talk) 20:20, 17 December 2024 (UTC)

We should be lending greater weight to academic and NGO sources and less on newsmedia. Simonm223 (talk) 21:35, 17 December 2024 (UTC)
When writing about politics, it's a good idea to look for sources from other countries, too.
For drug information, look for WP:MEDRS and other scholarly sources. WhatamIdoing (talk) 23:19, 17 December 2024 (UTC)

Reliability versus notability of an author of a source

Should sources be used or quoted in an article if the author of the quoted piece is not themselves a notable individual, with their own Wikipedia article? Is there any policy in Wikipedia that could be interpreted as requiring the author of a source to have their own Wikipedia page, or to be Wikipedia-notable? Conversely, if there is no such requirement, where is this specified? BD2412 T 03:02, 24 December 2024 (UTC)

Reliability is not notability, notability is not reliability. Nikkimaria (talk) 03:09, 24 December 2024 (UTC)
Where is this written? Asking for a friend. BD2412 T 03:22, 24 December 2024 (UTC)
Although this has been asked before, I'm not sure that we ever wrote it down. However, it obviously follows from the answer to "Are reliable sources required to name the author?" in the Wikipedia talk:Verifiability/FAQ: If you can cite a news article that doesn't have a byline, then sources can be cited even if the authors are not known to be notable. Obviously any such rule would be a nightmare, though perhaps we'd be a little amused by the chicken-and-egg aspect (nobody can be notable first, because only sources written by already-notable authors would count towards notability) while Wikipedia burned to the ground.
I suspect the other editor is using notable in its real-world sense, e.g., to prefer sources written by known experts or other reputable authors. WhatamIdoing (talk) 06:22, 24 December 2024 (UTC)
What about the specific context of quoting the author? For example, in Howard the Duck (film), we have: In The Psychotronic Video Guide, Michael Weldon described the reactions to Howard as being inconsistent, and, "It was obviously made in LA and suffered from long, boring chase scenes", with the "Michael Weldon" there being neither of the ones with Wikipedia articles, the Australian politician and the South African cricketer. BD2412 T 20:52, 24 December 2024 (UTC)
It's fine. You're supposed to provide WP:INTEXT attribution for most opinions/reviews.
Imagine a world in which we couldn't quote a scholar or an expert unless they qualified for a Wikipedia article. Or if we couldn't say something like "He denied the charges" about a non-notable person. Most editors would agree that such a result would violate NPOV. WhatamIdoing (talk) 18:59, 27 December 2024 (UTC)

The New York Times should not be considered a reliable source

Not the venue Andre🚐 08:30, 4 January 2025 (UTC)

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Plenty of evidence presented here, with information about many more sources: https://www.youtube.com/watch?v=mhYS59egWQc --Scottandrewhutchins (talk) 06:28, 2 January 2025 (UTC)

I'm not going to click on a video of unknown provenance. If you can't make your point in writing, I'm not going to take it seriously. —David Eppstein (talk) 06:42, 2 January 2025 (UTC)
It would be a lot of copying and quoting from articles shown in the video. Scottandrewhutchins (talk) 08:28, 4 January 2025 (UTC)
This is the wrong place. If you want to question the reliability of a source, start a section at WP:RSN. However, you would be wasting your time because there is no chance NYT would be judged as generally unreliable. Zerotalk 10:02, 2 January 2025 (UTC)
Because people are unable to think. It is an extremely propagandistic source as the October 7 rape story proved, and this video shows plenty more examples that have nothing to do with the Israel-Palestine conflict. --Scottandrewhutchins (talk) 08:29, 4 January 2025 (UTC)
The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Machine learning

Under § Sources produced by machine learning, I removed the statement ML generation in itself does not necessarily disqualify a source that is properly checked by the person using it (diff). What does "properly checked" mean? Does "the person using it" refer to the person submitting prompts to a chatbot or the Wikipedia editor using it as a source? Since it appears that most GenAI systems are trained using text scraped from the internet (including Wikipedia), I don't see any reason to treat large language models any differently to other § User-generated content. In other words, LLMs and other chatbots should be presumptively disqualified as sources until specifically verified by a human author with relevant expertise. —Sangdeboeuf (talk) 02:33, 12 January 2025 (UTC)

I assume "properly checked" referred to published sources that are checked by a human author, but I do not think the sentence you removed is necessary or helpful to include in the guideline, and I support the removal. I would also support bolstering the language of this section to explicitly state that sources composed of LLM-generated content are generally unreliable/unacceptable. I do not see a problem with authors using LLMs to assist with research, but any source that directly publishes LLM-generated content does not meet the "reputation for fact-checking and accuracy" required by this guideline. — Newslinger talk 02:59, 12 January 2025 (UTC)
OK, I added LLM-generated content from tools such as ChatGPT and other chatbots is not generally reliable etc. (diff). —Sangdeboeuf (talk) 04:41, 12 January 2025 (UTC)
It's not reliable at all! At best, and this is as permissive as people have proposed under the current tech, it is equivalent to our writing, that is, WP:OR. CMD (talk) 04:50, 12 January 2025 (UTC)
I believe the idea here was something like:
  • Rae Reporter interviews a dozen people plus gets hundreds of pages of information from a government agency. The interview transcripts and all the information gets dumped into a magical AI tool, with instructions to summarize it all in the style of a 600-word-long newspaper article. After several iterations, the journalist then decides that it sounds basically okay, re-writes part of it, and individually hand-checks each and every name, claim, and quote in the original documents, because journalists don't actually like misquoting people. This gets handed off to the editor for normal processing.
and in particular, I think we want to avoid:
  • A whistleblower leaks a massive amount of information to a journalist, who uses AI to summarize what's in the document trove. The journalist hand-writes a news article about the information in the documents, and it is published in a reputable newspaper. A POV pusher claims that the news article is unreliable because the journalist used AI as one tool among many.
What we don't want is:
  • Wikipedia editors to say "Dear LLM, here is a long list of people who sound like notable BLPs, so please write Wikipedia articles about each of them. They all need to have about 1,000 words and two inline citations to reliable sources per paragraph. The second sentence should say what they are best known for. Thank you."
WhatamIdoing (talk) 05:33, 12 January 2025 (UTC)
The first "Rae Reporter" example case sounds controversial. In their current state, I do not believe LLMs are able to process that volume of information into a 600-word article without significant inaccuracies or omissions that would compromise the quality of the output text. Additionally, LLMs are not yet sufficiently advanced to perform fact-checking on the original documents, which would result in incorrect and misleading claims being presented in the published article without appropriate context.
As the section text currently states, "It may not be known or detectable that ML was used to produce a given piece of text", so LLM-generated content that undergoes extensive rewriting and an adequate editorial process should theoretically be indistinguishable from human-written content that passes the same editorial process – a situation that might be comparable to the Ship of Theseus paradox. However, in practice, published articles that directly incorporate LLM-generated content tend to be less accurate to the point of being considered questionable, regardless of what the website claims to do editorially, because the direct use of LLM-generated content is a cost-cutting measure. This aligns with the consensus view expressed in the 2024 Red Ventures RfC and a 2023 discussion on G/O Media websites.
An example of LLM usage in published media that would be appropriate for citation on Wikipedia is the Pew Research Center's 2024 report "America’s News Influencers", which discloses in its methodology that GPT-4 was used for data processing during the research and analysis process, although the finished report was written by named humans. This type of report is similar to your second "whistleblower" example case. — Newslinger talk 07:12, 12 January 2025 (UTC)
I assumed the part I removed was referring to fully AI-generated content farms as potentially reliable sources in themselves, rather than LLMs as just another tool used by human authors of published, independent sources. I think it would be fine to add a caveat for things like the Pew report, making it clear that sources using LLMs for research need to separately have a reputation for fact-checking and accuracy. —Sangdeboeuf (talk) 09:35, 12 January 2025 (UTC)
An explicit caveat in the guideline would help clarify this, but I am not sure if it is necessary. Authors regularly use unreliable sources that are not LLM-generated as sources of data, and the author's writing can still be considered reliable as long as the author uses the data in an appropriate way that satisfies the "fact-checking and accuracy" requirement. The same would apply to authors using unreliable LLM-generated material as sources of data. — Newslinger talk 09:11, 13 January 2025 (UTC)

Independent or alternative media

In Wikipedia, for a long time we have considered legacy media (and their corporate offshoots) as usually "reliable", whereas we've considered independent journalists as self-published. With the massive shifts in the media landscape in recent years, as well as the politicizing of particular media outlets causing experienced journalists to "go independent", has this changed for us Wikipedia editors when evaluating whether a source is a reliable source or not? If so, have we updated any of our policies or guidelines to reflect changes?   ▶ I am Grorp ◀ 23:20, 2 February 2025 (UTC)

Has this changed? No. Will this change? Probably. Eventually.
Have we updated any policies or guidelines? Not yet. AFAIK we don't even have any essays explaining it. We could use a good pair of Wikipedia:You don't need to cite that the sky is blue vs Wikipedia:You do need to cite that the sky is blue to describe the challenges (e.g., figuring out which ones are good) and opportunities (e.g., greater voice for the previously voiceless). WhatamIdoing (talk) 03:01, 3 February 2025 (UTC)
Although I've become increasingly critical of traditional media I don'tthink that means we should shift to substack. While I have accepted that, at least for now, Wikipedia is stuck using news media in some circumstances I think the better response to the hollowing out of legacy news media is to pivot toward greater emphasis on academic journals and monographs rather than independent journos. WP:EXPERTSPS is a good policy for dealing with those who have relevant expertise and works correctly. Simonm223 (talk) 03:11, 3 February 2025 (UTC)
When it comes to commentary and analysis I suspect that the independent media may already be better than the legacy media if you can sift through the trash to find the good ones. The problem is how to do that and how do we decide which independents are the good ones. As an example, I suspect when it comes to analysis of an aviation incident, some of the YouTube channels run by current/former pilots provide much better analysis vs traditional media. However, how can we agree (and test) which of these alternative commentators really are the good ones? If they were publishing in academic journals we could use those articles but these topics often aren't of academic interest. I do think "alternative sources" is a struggle point for Wikipedia as the internet continues to allow independent voices to be heard (kind of like how Wikipedia allowed an alternative to mainstream encyclopedias). However, absent some clear way to filter the good from the bad I don't know how one would decide which sources are the good ones. Springee (talk) 04:19, 3 February 2025 (UTC)
Comment: Yeah, academic and monographs don't cover "events" in the same way that "news" does (both legacy and independent), so we cannot rely on academics to fill in the gap from the loss of news coverage by legacy media.   ▶ I am Grorp ◀ 04:46, 3 February 2025 (UTC)
In practice, confirmation bias rules, and that means that people decide which sources are the good ones by determining which sources reinforce their (i.e., the humans') prior/existing beliefs. If you believe the world is round, you will reject a source that says it is flat – and vice versa. If most editors disagree with you, then they will accuse you of "POV pushing" even if you are correct, because "POV pushing" is a label we give to people who want Wikipedia to represent more of the view they believe in and less of the view(s) that other editors believe in.
Research shows that people find sources credible when they match other sources, and discard outliers as incorrect. See also the famous Oil drop experiment#Millikan's experiment as an example of psychological effects in scientific methodology, in which the correct answer was repeatedly rejected because it didn't match the other sources. WhatamIdoing (talk) 04:47, 3 February 2025 (UTC)
My questions on this subject aren't because of conflicts or edit wars, but are more about selecting an independent media source that covers something that isn't being covered by legacy (because, you know, doesn't get them clicks anymore, or they're walking on eggshells or have limited resources and your esoteric topic just isn't in their "mainstream" coverage anymore).   ▶ I am Grorp ◀ 04:50, 3 February 2025 (UTC)
If:
  • it's uncontroversial content on an uncontroversial subject,
  • it's a fairly niche subject (e.g., construction techniques for train stations in Victorian England), and
  • it is used as a way of adding (e.g.,) colorful details to the article – not something you're trying to base the whole article upon or prove notability with,
then I'd try to find something that passes WP:EXPERTSPS and not worry too much beyond that. WhatamIdoing (talk) 05:02, 3 February 2025 (UTC)

journals supplements -clarification needed

Subject: scientific conferences, a.k.a. symposia

TL;DR; they are unreliable primary sources, even when "peer-reviewed"

The current version says:

"Symposia and supplements to academic journals are often (but far from always¹) unacceptable sources. They are commonly sponsored by industry groups with a financial interest in the outcome of the research reported. They may lack independent editorial oversight and peer review, with no supervision of content by the parent journal. Such articles do not share the reliability of their parent journal, being essentially paid ads disguised as academic articles. Such supplements, and those that² do not clearly declare their editorial policy and conflicts of interest, should not be cited."

1: This "far from always" lost me. I need clarifications. Are supplements that clearly declare their editorial policy and COI acceptable? It think they often are not:

a)They are still primary sources, so not ideal.

b) They often include early stage results (not reliable; please see the paragraph about symposia on the Medicine page Wikipedia:Identifying reliable sources (medicine)).

c) And if they don't include early stage results, we should then cite the original paper rather than the conference. Note: conferences are NOT an acceptable type of secondary sources, because they don't follow any scientific protocol; unlike secondary studies (a.k.a. reviews).

Anyway, I understand that some flexibility is needed. So how about simply deleting that "(but far from always)" parenthesis?

2: "and those that": it doesn't make grammatical sense. If we are to keep the ambiguity on whether such supplements are valid sources, let's insert "and especially those that". Okay? Galeop (talk) 04:04, 1 February 2025 (UTC)

Supplements that are paid ads are a COI problem. Conference papers typically are not; they may lack peer review, but in many cases the authors would qualify as subject-matter experts. I think it would be helpful to more clearly differentiate between these cases, and more clearly point to SPS for the evaluation of the latter (and potentially introduce the MEDRS issue). Nikkimaria (talk) 06:07, 1 February 2025 (UTC)
It depends on the field. Conference papers in computer science are good; conference papers in pharmaceutical drug development are not so good. WhatamIdoing (talk) 02:59, 3 February 2025 (UTC)
Just because of the COI issue (case 1), or are you thinking of another case-2 problem specific to that field? Nikkimaria (talk) 04:53, 3 February 2025 (UTC)
No, it's not just COIs. Conference papers don't get peer reviewed, so it's easier to overstate your results, use the wrong statistical test, or whatever other problems might get flagged and corrected in the peer review process.
See also Wikipedia talk:Reliable sources/Archive 72#Conference proceedings from two months ago. WhatamIdoing (talk) 04:57, 3 February 2025 (UTC)
Ah, okay. I'd think "could have problems that would be caught by peer review" would be the case for any field, which is why it makes sense to treat these as SPS. Nikkimaria (talk) 05:17, 3 February 2025 (UTC)
I certainly can't speak to all research areas but every conference I presented at was peer reviewed. However, the review process typically wasn't as stringent as a conference paper review. At the same time we discouraged citing a conference paper if the same authors had a more in-depth journal article on the same subject. In my field a conference paper was typically a smaller chunk of research. For example, a conference paper might present the results of a new test method or control algorithm. The combination of that new method with others to show a new capability might result in a journal article. Often if you found a journal article it would contain work that was previously presented at a conference. This is why, in my area, it was fine to cite a conference paper but it typically had less depth, or substance vs the journal paper. Springee (talk) 05:32, 3 February 2025 (UTC)
https://ncu.libanswers.com/faq/364411 says that IEEE requires all conference papers to undergo peer review before publication, but that appears to be an outlier. It may be more/less frequent in some fields, and of course individual publishers will set their own standards. WhatamIdoing (talk) 05:37, 3 February 2025 (UTC)

Updating RSSELF

We are talking about changing the wording in WP:SPS to be clearer (specifically, to remove the idea "third-party source" language). The current draft is in this comment at WT:V. Please join us if you're interested.

Please also see Wikipedia talk:Verifiability/SPS RfC for the larger conversation. WhatamIdoing (talk) 22:00, 3 February 2025 (UTC)

Proposal: Let we the audience vote for what we consider left-leaning and right-leaning sources

WP:1AM - nothing more to discuss here
The following discussion has been closed. Please do not modify it.

The discussion about Wikipedia's left-leaning bias 1 never goes anywhere in this page because there is a self-referencing loop involving Wikipedia Consensus -> aleggedly far-left, or very left-of-center and not-that-reliable sources -> someone brings up the perception of a left-wing bias -> Wikipedia editors point to a supposed "reliability" of a source without actually providing evidence for such reliability, except perhaps for academic articles on humanities, that don't prove objective facts either. What if both academic sources and media sources validate each other's "reliability" while not actually being reliable in the perception of the society? That's why democracy and suffrage exist.

Are you guys scientifically minded? Rationally minded? Are you against absolutism? Allow me to present a point.

Is it possible to reach an absolute truth about a government or a candidate? Can an administration or a candidacy be objectively qualified as "100% positive" or "100% negative"? Or course not. In any democratic system, an administration may reach an approval rate of, say, 70-90%, but there will be always people that perceive that administration as negative. The outcome of an election legitimates a consensus, not an objective truth.

The same holds true for thoughts, for philosophy, and for subjective classification of things based in consensual taxonomy frameworks. So we come down to left-right and reliable-unreliable classification:

Where did the reliable sources consensus come from? As far as I know, the bulk of it came arbitrarily from MrX's point of view in 07/28/2018. Who is he/she/they? Is this legitimate? Does the consensus of Wikipedia reflect the consensus of the general public? Who said so?

Let's suppose a consensus exists among the general audience, that there is a leftist bias in Wikipedia. Not only we are failing to properly address this, by not measuring or acknowledging it, but also Wikipedia would be contributing negatively for a biased media environment.

Let's suppose, for contrast, that there isn't a consensus among the general public that the leftist bias of Wikipedia is real. In this scenario Wikipedia would be luckier, but still negligent because it lacks a legitimate evidence for the perceived reliability and bias of its sources. What legitimates a president? There is a reason why he/she can't be elected by a special chaste of "specialists". The only legitimate means to claim power is through direct vote. Similarly, I propose that the only legitimate means to claim that a certain source is "reliable" and "has a certain political bias" is through vote.

I noted that Fox News isn't considered reliable specifically for transgender topics. What if the consensus among the general public is that several sources aren't reliable specifically for politically-charged topics? And... If the perceived consensus of left-right in the US is different from the rest of the world, we can address politics of each country separately. To be honest, I don't actually agree that the left-right division in the US is that much different from the rest of the world. What I see are left-friendly editors using very questionable and fragile statements ("the Democratic Party would be center-right in Europe"/"it doesn't matter if practically every self-identified leftist votes blue"/"source X follows the broader capitalist economic agenda, therefore it can't be called leftist") to pass far-left and verifiably flawed sources as flawless and reliable. And, by verifiably, I mean that it's verifiable through factual confrontation with other sources, suffrage, and intense civil scrutiny of what common citizens perceive, verify, think and say. Does anyone here value common citizens? There is a thing named afer this, it's "Communism" you know. Some people confuse it with free healthcare, but the historical consensus is that we were never capable of implementing it.

I know I may sound harsh and pretentious, but the political bias debacle is really annoying and tiresome. In my perception, Wikipedia's credibility for politically-charged topics has deteriorated since its foundation.

To wrap things up, in my point of view the current sources guidelines are a false consensus. They weren't built bottom-up from a consensus to begin with. They are illegitimate in the present moment, and have to be replaced by a proper consensus built from scratch. I propose that you all Wikipedia editors gather valid evidence - in the form of popular votes from the general audience - so you can legitimately claim that some source is "reliable" or "non-reliable", and "right-leaning" or "left-leaning" as well. Otherwise, the existing "reliable sources" and "center-right/center/left" labels are nothing but arbitrary and personal. And Wikipedia, once envisioned as a tool by the community, for the community, is just another voice of arbitrary "truths" as told by media oligarchs. JC Beltrano (talk) 07:49, 21 January 2025 (UTC)

@JC Beltrano, you are in the top 20% of contributors to Wikipedia. Congratulations. But this means that you stopped being part of "we the audience" several years ago.
Wikipedia:Reliable sources/Perennial sources ("WP:RSP") came from hundreds of prior discussions, mostly at Wikipedia:Reliable sources/Noticeboard. Most of those discussions are linked on the page. If you want to influence RSP's contents in the future, then keep an eye on that noticeboard. At the moment, I notice a discussion in which an editor proposes slightly loosening the RSP classification on a left-wing source, and the proposal is being soundly rejected. Another left-wing source looks like it might get downgraded.
Something that you might want to think about is that what's "left" in US politics is "center" in European politics, so if we open it to a popular vote, you might get outvoted. Personally, I think it'd be best if we evaluated sources based on whether they get the facts right, rather than whether they're left or right or center. Good Wikipedia editors can build a neutral article even if the sources have the "wrong" political slant. WhatamIdoing (talk) 09:14, 21 January 2025 (UTC)
Wikipedia is not a forum. So let's stop this silliness. Simonm223 (talk) 12:36, 21 January 2025 (UTC)
"But this means that you stopped being part of "we the audience" several years ago." The categories aren't mutually exclusive. Also, this is irrelevant. "If you want to influence RSP's contents in the future, then keep an eye on that noticeboard." From that noticeboard, I gather that Wikipedia is now a purple place: several reconsiderations and revisions denied. But it's pointless anyways, because something happened in this site in the last years so that discussions end with 10-20 votes favorable to non-consensual left-wing views that don't reflect society, thus should have no place in an encyclopedia that claims to be neutral. I actually found zero evidence to support the claim that HuffPost deserves to be treated better than Breitbart in a neutral encyclopedia. This isn't too different from a solo work of a random MrX - that also happens quite frequently. I mean... Drawing an analogy between Wikipedia and a printed encyclopedia, a chief editor is free to editorialize it to reflect only left-wing POV, but the problem is to claim neutrality. All the people involved in the typical RfC and editing of project pages won't consider for a second that they maybe, just maybe, are throwing the neutrality of their own project on the mud for not acknowledging that they are basing an entire encyclopedia on anecdotal samples of a dozen left-sympathetic people. Now, how this anecdotal leftist sample took over and is granted to win all critical decisions in the wiki, is beyond my research. You will keep editorializing this with a left bias until the worse happens (some right-leaning tech oligarch takes this project over) and it's not like my complaints will make any difference. I'm doing this mostly for the sake of historical record. The irony is that if a right-leaning "fascist" technocrat takes over this (very politically strategic) website, he/she probably will appoint an admin team more balanced and diverse than the current one. "Something that you might want to think about is that what's "left" in US politics is "center" in European politics, so if we open it to a popular vote, you might get outvoted. If this is true at all, let's open to popular community vote then. JC Beltrano (talk) 01:30, 22 January 2025 (UTC)
Re: consensus of the general public.
The general public is an idiot. What matters is the consensus of expert reliable sources. The general public may believe that earth is 6000 years old. Their view is irrelevant. What matters is the view of planetary scientists, geologists, and other qualified experts, who all agree that Earth's about 4.5 billion years old.
If the general public has a problem with this, too bad. Headbomb {t · c · p · b} 12:51, 21 January 2025 (UTC)
"The general public is an idiot." Great! Wow, this is seriously some of the most reactionary statements I've read in a while. Let's raise the minimum voting age to 25 then. Let's restrict voting to only real estate-owner men. Why not change the main page from "Welcome" to "Welcome, idiot. We hope Wikipedia makes you less idiot"? "What matters is the consensus of expert reliable sources." What experts? Galileo was dismissed by the most credible board of experts of his time. Jordan Peterson is an expert, Alexandr Dugin is an expert, and taking the word literally, even Steve Bannon is an expert. But some experts are, somehow, being called by other experts "fascist" of "far-right". Joseph Goebbels was an expert, too. The Wiki is treating some experts as more expert than the others. Every "expert" association have their own set of oligarchs and trillionaire assets management funds behind them enough to raise conflict-of-interest questions. The best a free and neutral encyclopedia can do is to treat all oligarchs and billionaires equally, with the same degree of untrustworthiness. "What matters is the view of planetary scientists, geologists, and other qualified experts, who all agree that Earth's about 4.5 billion years old." This is a false equivalence. You are comparing apples to oranges. The approximate age of planet Earth is an objective datum that has been estimated using best evidence. So, what is the best evidence to estimate the political alignment of a media vehicle or a party? Well, reach to the civil society itself. The best political scientist gives more importance to the self-identification of voters of that party, and the perception of the audience of the media, than what his academic colleagues have to say individually, or to what the party declares officially. If you do a poll and the evidence shows that practically all the self-identified left-wing voters choose the Democratic Party, you, as a good political scientist, don't blindly conform to the self-designation of that party as "centrist", "big-tent" or "center-right". The best current evidence points to the Democratic Party being a center-left party, the same way it showed a center-right party until 1996. The people's perception and political action has greater relevance than whatever bullshit the party says about itself. Adolf Hitler put the -socialist suffix in his party, for instance. And the Republican Party doesn't self-identify officially as fascist. JC Beltrano (talk) 01:30, 22 January 2025 (UTC)
Wikipedia is not a democracy and our original research cannot be used in Wikipedia articles. Wikipedia editors are indeed "scientifically minded"; see User:Guy Macon/Yes. We are biased. — Newslinger talk 14:22, 21 January 2025 (UTC)
So the policy of the site is to just ignore problems. Fine, I guess? Just apathetically point to links. No arguments, no evidence for decisions. This deadpan attitude is making Wiki go downhill. No comment on Larry Singer's statements? He is an expert, you see. No shock for the current impossibility of referencing right-leaning sources on political topics while left-leaning sources aren't banned? No one here provided valid evidence that MSNBC is more reliable for politics (thus, presumably less biased) than Fox News and New York Post. Anyways, this section is ready for purple. JC Beltrano (talk) 01:30, 22 January 2025 (UTC)
The policy of the site is that failing to kowtow to popular opinion, including but not limited to the popular opinion in the US, is not actually a problem.
An argumentum ad Larryem sounds pretty irrelevant. I can agree that he's an expert on his own opinions, though. WhatamIdoing (talk) 02:17, 22 January 2025 (UTC)
Your very long comment has not actually identified a problem, especially because your premises are wrong. While MrX started the perennial sources list, it is incorrect to claim that "the bulk of it came arbitrarily from MrX's point of view in 07/28/2018". That list is a summary of discussions on the reliable sources noticeboard, a venue that has received contributions from thousands of editors representing a wide variety of views. There are plenty of right-leaning sources that are cited on Wikipedia, of which the ones listed here are a very small selection. If you have new evidence concerning MSNBC, Fox News, the New York Post, or any other source that you would like to start a discussion about, you are free to present that evidence on the noticeboard.
Reading, understanding, and applying the policies and guidelines is a fundamental part of being a Wikipedia editor. I recommend reading the policy on consensus to learn about how editors make decisions on Wikipedia. Soliciting votes from readers to determine article content is not how Wikipedia is written, but you are welcome to create your own online encyclopedia on your personal website if you would like to engage in that experiment.
And no, Larry Sanger is not an expert in politics, because experts in politics do not make statements like this. — Newslinger talk 03:01, 22 January 2025 (UTC)
  • The key to political labeling (far-right, right, center, left, far-left) is INTEXT attribution. Wikipedia should not label things ourselves, we should inform our readers of the labels applied by our sources. Blueboar (talk) 14:39, 21 January 2025 (UTC)
    In an article, I completely agree with you. But the OP's concern appears to be that Wikipedia:Reliable sources/Perennial sources#New York Post does not endorse the newspaper's (lack of) fact checking and corrections. He'd probably be upset about the facts reported (and cited) in New York Post#Content, coverage and criticism, too, but the complaint itself is about our internal notes at WP:RSP. WhatamIdoing (talk) 18:15, 21 January 2025 (UTC)
  • What we consider a WP:RS depends on what the highest-quality sources available say (see WP:ACADEMICBIAS for some details about this, although WP:GLOBAL also explains why we can't just put our thumb on the scale and "balance" things for what an American audience considers balanced.) How we handle things like global warming, or COVID-19 treatments and its origins, or the efficacy of vaccines, or who won the 2020 US election, or the effectiveness of faith-healing, or how we cover the history of the Civil War, or how we describe fascism as an ideology - all of these things have clear-cut answers within the highest-quality academic sources. Individual editors might have strong opinions on them, and the public as a whole might not agree with those academic conclusions, but as an encyclopedia our job is to summarize those high-quality sources, not the random opinions of the public. So a vote is inappropriate; it's why we decide things based on a consensus-building process that places the most weight on policy-based arguments and high-quality sources, rather than just editors' gut feelings. In short, the leaning of a source is best determined the same way we determine everything else - by looking at high-quality (ideally academic) coverage of it. This may sometimes differ wildly from the perception of the public, but we're an encyclopedia, not a polling firm, so that means nothing to us. --Aquillion (talk) 19:06, 5 February 2025 (UTC)

New FAQ suggestion

Can we add to the FAQ

Q: "Does this (whichever) election change source reliability guidelines?" A: "No." Simonm223 (talk) 19:09, 5 February 2025 (UTC)

Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Reliable_sources/Archive_73&oldid=1276748134"