Talk:Reproducibility

This  level-5 vital article is rated C-class on Wikipedia's content assessment scale. It is of interest to the following WikiProjects:

Science High‑importance This article is within the scope of WikiProject Science, a collaborative effort to improve the coverage of Science on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.ScienceWikipedia:WikiProject ScienceTemplate:WikiProject Sciencescience articles High This article has been rated as High-importance on the project's importance scale. Skepticism Mid‑importance This article is within the scope of WikiProject Skepticism, a collaborative effort to improve the coverage of science, pseudoscience, pseudohistory and skepticism related articles on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.SkepticismWikipedia:WikiProject SkepticismTemplate:WikiProject SkepticismSkepticism articles Mid This article has been rated as Mid-importance on the project's importance scale. Statistics Mid‑importance This article is within the scope of WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.StatisticsWikipedia:WikiProject StatisticsTemplate:WikiProject StatisticsStatistics articles Mid This article has been rated as Mid-importance on the importance scale. Psychology Mid‑importance This article is within the scope of WikiProject Psychology, a collaborative effort to improve the coverage of Psychology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.PsychologyWikipedia:WikiProject PsychologyTemplate:WikiProject Psychologypsychology articles Mid This article has been rated as Mid-importance on the project's importance scale. Philosophy Low‑importance This article is within the scope of WikiProject Philosophy, a collaborative effort to improve the coverage of content related to philosophy on Wikipedia. If you would like to support the project, please visit the project page, where you can get more details on how you can help, and where you can join the general discussion about philosophy content on Wikipedia.PhilosophyWikipedia:WikiProject PhilosophyTemplate:WikiProject PhilosophyPhilosophy articles Low This article has been rated as Low-importance on the project's importance scale.

Previous versions of this article were, to say the least, rather strange. I've cut away a lot of non-sequiturs and stuff that was just plain wrong, and marked what remains as a stub. -- Anon.

Carrionluggage is unhappy with the statement in the notes: "See the U.S. DoE 2004 Cold Fusion Review which states that half the reviewers found the evidences of excess heat convincing, and 1/3 found the evidences of nuclear reactions convincing.".

This issue has been discussed at large at the cold fusion article (in particular here), which represents the consensus that emerged. This is not the place to repeat the debate, but here are some quick pointers:

• page 3 paragraph 3 of the report discusses the views of the reviewers on "excess heat" (starting with "The excess power..."). Please read carefully as this is the central issue of the debate.
• page 4 paragraph 2 of the report discusses the views of the reviewers on "evidences of low energy nuclear reations". Again please read it carefully.

While the report raises many issues with the current state of the field, one cannot say it is "preponderently negative" about the evidences supporting the phenomenon, as the 2 paragraphs show. Its successful independent reproduction convinced some, while the lack of repeatability bothered others.

If you still disagree, you are likely to disagree on the content of the cold fusion article too. Please raise your points in the cold fusion talk page, so that it can be addressed in the proper audience. Pcarbonn 19:14, 21 December 2006 (UTC)[reply]

I removed the reference to orgone... I do not think this is a "Famous problem" of reproducibility. It is more a famous problem of a scientist gone haywire (if he (Wilhelm Reich) should be considered a scientist at all). Calling Paulo Correa & Alexandra Correa researchers is a bit far stretched as well. Looking at their CV their claims are not those that a lot scientists would even consider trying to reproduce. —Preceding unsigned comment added by 86.87.108.95 (talk) 17:16, 2 August 2008 (UTC)[reply]

It's always difficult making such judgements of people. They thought the same about Wegener's Continental Drift, didn't they -- and the idea that you could reduce hospital infections by handwashing? --Brian Josephson (talk) 08:03, 12 August 2013 (UTC)[reply]

I very much doubt that the term "reproducible research" was first proposed or defined in 2009. The concept must go back at least one hundred years as people were beginning to get a firm grasp of statistics: Standard Deviation, 1894; Student's t-test, 1908; etc.. I think the header and the paragraph has to be refined and restricted to Claerbout's particular usage and intention. AdderUser (talk) 20:39, 2 July 2012 (UTC)[reply]

It should be mentioned in articl that there are cases of qualitative/statistical reproducibility like the distribution of nuclear fission products.--5.15.205.173 (talk) 15:41, 4 August 2013 (UTC)[reply]

This seems to be based on a rather novel definition of the term "qualitative reproducibility" created by a lone cold fusion researcher: [1]. Since it doesn't seem to have been picked up by the scientific community at large – or even used widely within the cold fusion community – it is probably not appropriate WP:WEIGHT to introduce this terminology here. (See also my comment at Talk:Cold fusion.) TenOfAllTrades(talk) 13:52, 5 August 2013 (UTC)[reply]

It is definitely not a rather novel definition of the term, is about stating obvious facts. The example given will be restored and rephrased.--5.15.210.25 (talk) 20:35, 10 August 2013 (UTC)[reply]

But is your statement correct? Under what conditions is the isotope distribution different from the usual one? Reference please! --Brian Josephson (talk) 21:11, 10 August 2013 (UTC)[reply]

Surely a source which can give further details about the conditions of different isotope distribution (like a standard texbook on fission reactors theory) would be useful, but not absolutely necessary.--5.15.210.25 (talk) 21:59, 10 August 2013 (UTC)[reply]

Agreed, a source may not be needed for the article, but I have no idea what you are referring to and so you'll need to produce a source to convince me and other editors that the statement that you have added to the article is correct.--Brian Josephson (talk) 22:12, 10 August 2013 (UTC)[reply]

Some off-line sources are not very reachable to readers of this page, thus on-line sources about statistical variability of nuclear fission products could be used. Nevertheless I'll mention some off-line sources when I'll reach them.--5.15.179.181 (talk) 17:44, 12 August 2013 (UTC)[reply]

If it's such an obvious fact, it should be easy to find in physics textbooks. --Enric Naval (talk) 20:45, 11 August 2013 (UTC)[reply]

I can't say I've ever heard of it. Of course, if you change the conditions by changing the details of the process that generates fission that could alter the isotopic distribution of the products, but that hardly counts as lack of reproducibility, and one would need more than an isolated claim as something may have gone wrong in that particular instance. I think 5.15.210.25 needs to back up his statement or withdraw it.--Brian Josephson (talk) 20:57, 11 August 2013 (UTC)[reply]

I've checked out the ref. given by ToaT above, and see that it is in essence a reference to situations where there is statistical uncertainty. That is important in situations such as drug testing, or with some high energy physics expts., where for purely statistical reasons different experiments can give rise to different conclusions (in fact many judgements of the efficacy of drugs are now believed to be seriously in error, though physicists and parapsychologists treat statistical error more carefully). This is definitely worth including and within the scope of the article, but if so it needs to be explained properly, which is not the case with what 5.15.210.25 has added -- or there could simply be a reference to an appropriate article on the subject which no doubt there is somewhere in w'pedia.--Brian Josephson (talk) 21:22, 11 August 2013 (UTC)[reply]

Of course a better phrasing regarding statistical variability of fission products is needed. I'll appeal to memory for now and say that the equations of conservation for fission products nuclides are in number of about 100.-5.15.179.181 (talk) 17:53, 12 August 2013 (UTC)[reply]

Nuclear fission products article states that an individual fission event is not predictable giving random fission products.--5.15.179.181 (talk) 18:00, 12 August 2013 (UTC)[reply]

That's a rather selective quote. The full sentence is: However, while an individual fission is not predictable, the fission products are statistically predictable. You've quote mined to state the opposite of that article. If something is statistically predictable, the statistical distribution is reproducible in a further experiment, it is not qualitative. IRWolfie- (talk) 12:17, 13 August 2013 (UTC)[reply]

The quote is not selective. I expressed the second part of the sentence with equivalent words random fission products to statistically predictable fission products. The fission in two nuclear research reactors operating and beeing started simultaneously for instance would not give the same isotopic composition at the same instant due to statistic variability at the level of single nuclei. Perhaps you do not fully understand the concept statistical reproducibility.--5.15.200.163 (talk) 21:31, 18 August 2013 (UTC)[reply]

It means that the more events you observe, the closer the actual proportion is to the expected proportion (see Law of Large Numbers). I expect you would get a good approximation after only a few hundred events. By contrast, in 1g of uranium, there are 2.5 x 10²¹ atoms (2,500,000,000,000,000,000,000). At that level, the difference from the expected proportion, even though not zero, is negligible. I am sure that the actual probability distribution can be calculated using statistical laws. Arc de Ciel (talk) 22:16, 18 August 2013 (UTC)[reply]

The yield of an individual fission product is a statistical averaging of individual fission event.--5.15.215.54 (talk) 18:03, 12 September 2013 (UTC)[reply]

I've now made a change in the article in the light of the above discussion. It would be good if a reference were to be included, but I don't have time to find a good one.--Brian Josephson (talk) 09:16, 12 August 2013 (UTC)[reply]

The changes you introduced failed WP:OR, WP:RS and WP:NPOV. We do not give undue weight to novel and solipsistic positions and definitions, and WP articles are not the place to engage in special pleading. Dominus Vobisdu (talk) 11:26, 13 August 2013 (UTC)[reply]

Would you please stay out of things you don't understand. Your comment is ridiculous.--Brian Josephson (talk) 08:44, 19 August 2013 (UTC)[reply]

Out of curiosity, how do you determine if a comment is ridiculous when you haven't read the cited policies? IRWolfie- (talk) 00:47, 31 August 2013 (UTC)[reply]

I was referring there to the suggestion that the definitions etc. were 'novel and solipsistic'. The policies are irrelevant to the question of whether it is ridiculous to suggest that.--Brian Josephson (talk) 07:59, 31 August 2013 (UTC)[reply]

Comment: The situation from this section underlies the necessity of some knowledge and understanding of the topic from those who post comments here. Those who cannot display a minimal understanding of the topic as pointed out by their comments are not much in the position to make assertions regarding the reliability of some sources whose content is not fully understood, here and on other talk pages. In the absence of full understanding the comments of some editors are wp:tendentious editing.--5.15.178.52 (talk) 19:45, 20 August 2013 (UTC)[reply]

I do happen to think that this proposal has merit, but I am still waiting for a reliable source that verifies that this type of reproducibility is a established term in the field, and what name is used for it.

Mr. Josephson and 5.15.x.x should stop the off-hand comments about other editors' intelligence / other editor's understanding of the topic, and start investing their time into finding sources for verification. This is the way wikipedia works, and complaining about it is only wasting their time and our time. --Enric Naval (talk) 10:04, 25 August 2013 (UTC)[reply]

I'm afraid I have more important things to do with my time, but when I see that an article can be improved for whatever reason I do occasionally take a moment to make that improvement. I think it is worth doing this, though often the effort is regrettably wasted 'because of the way wikipedia works'.

And there can be situations where it is relevant to comment on other editors' understanding of a topic or otherwise, since it is perfectly conceivable that someone with limited understanding thinks they are applying a rule correctly when in fact they are not. I do not have time to document this in detail, however.--Brian Josephson (talk) 11:18, 25 August 2013 (UTC)[reply]

However, it could help if those concerned with this could say which of these items is sufficiently uncertain as to merit a reference being included:

• repeatability of scientific experiments is desirable
• repeatability is not considered necessary to establish the scientific validity of a claim.
• the cloning of animals is difficult to repeat
• cloning has been reproduced by various teams working independently
• cloning is a well established research domain.
• One failed cloning does not mean that the claim is wrong or unscientific.
• Repeatability is often low in protosciences.

Since there is a w'pedia article on cloning, that may well provide a useful reference -- and if it does not fall under NOR (ad it should not) then item 2 is confirmed by what follows.--Brian Josephson (talk) 11:41, 25 August 2013 (UTC)[reply]

The last point smells wrong to me. Low reproducibility does not seem to be a defining characteristic of a protoscience. The question is whether the results have been reproduced at all. (I am looking at Questions to help distinguish a pseudoscience from a protoscience. Reproducibility is not even mentioned in Truzzi's lecture). --Enric Naval (talk) 18:30, 30 August 2013 (UTC)[reply]

The first 2 points, and the last point, they are general assertions about science. Evolution is just used as an example. --Enric Naval (talk) 18:37, 30 August 2013 (UTC)[reply]

First I should mention that item you dispute ('the last point') wasn't anything I wrote. Besides this, the disputed text doesn't say what you attribute to it ('defining characteristic'). Hence your objection has no force. The actual text ('repeatability is often low in protosciences') is, I believe, correct and could no doubt be backed up by examples. In any case, the penultimate part ('One failed cloning does not mean that the claim is wrong or unscientific'), which is undoubtedly true, is an important part of the analysis, and should remain even if the last part is removed.--Brian Josephson (talk) 19:27, 30 August 2013 (UTC)[reply]

To cut a long story short, where are the sources, or why are we still discussing this? The phrase protoscience isn't well defined, and the example given, cloning, is one where reproducibility is done if with difficulty. I've changed the text as such, IRWolfie- (talk) 00:55, 31 August 2013 (UTC)[reply]

I agree re protoscience. Re cloning, now it may be reasonably easy for some labs to do this but there was a time when it was difficult and chances were low. Then again skill may be an important aspect. Someone involved with a semiconductor device told me that one of their staff turned out to be very good at running the machinery involved in making the device and there was a reasonable yield (quite often such processes are not 100% reliable and many of the products are thrown away as they do not come up to standard). When this person was away on holiday the yield dropped to near zero. Thus your statement about 'reproducibility with difficulty' does not entirely reflect the reality. And in biology there are many experiments where reproducibility is similarly problematic.

In an ideal world, all this would be covered in the article. In a small move in that direction, I have made a slight rewording to the last sentence to make it less restrictive.--Brian Josephson (talk) 08:19, 31 August 2013 (UTC)[reply]

Let's take the anecdote at face value for sake of argument. A skilled member of staff creating a physical good is not doing a scientific experiment. They are creating a commodity. One can not disagree that the good was created. That the other workers are not as skilled when they take the staff members position is hardly surprising (or perhaps they are also incompetent). IRWolfie- (talk) 11:12, 31 August 2013 (UTC)[reply]

There's a difference here between science as you read it in the text books or from lectures, and the real world of experimental science. Anyone familiar with the latter will tell you that real experimental science is very often similar to the process of creating a physical good.--Brian Josephson (talk) 15:00, 31 August 2013 (UTC)[reply]

A situation where qualitative aspects are more obvious is the deformation of identical vehicles in identical conditions (i.e. speed, etc) in crash tests which is not the same for each vehicle subjected to the crash test. The deformation geometry is different and requires a topological qualitative description.--5.15.194.246 (talk) 09:28, 9 September 2013 (UTC)[reply]

The exact relation between the two concepts should be underlined more clearly.--5.15.194.246 (talk) 18:12, 8 September 2013 (UTC)[reply]

I agree, this article is quite misleading. I would not recommend this as a source of information on repeatability and reproducibility. Repeability is when an experiment is done exactly the same way as the original and yields the same results. Hence the word repeat. Reproducbility is when a third-party with different materials, equipment, software etc, does a similar experiment and gets the same result. Hence the word reproduce (make a copy). Reproducibility is a much much stronger statement and is what is at the core of the scientific method. This article mixes the two and perpetuates the confusion. See the follow article and may others that support than same notion.

Nor is it Good Science Drummond, Dr. Chris (2009) Replicability is not Reproducibility: Nor is it Good Science

Rhodydog (talk) 18:07, 23 July 2016 (UTC)[reply]

The debate on repeatability vs replicability vs reproducibility is far from settled. The paper from Drummond that Rhodydog cited simply adopts the ACM's definitions and is more an opinion piece arguing against a mandate that code accompany ML papers. A reference that gives a nice historical perspective on this debate can be found below. I agree that the article is unclear, and I would argue that it shouldn't take a hard stance on the exact definitions, and should focus more on the intent behind the word as it is commonly used and its associated movement. Maybe the best solution is to add a section to the article about this debate.

Plesser, H. E. (2018). Reproducibility vs. replicability: a brief history of a confused terminology. Frontiers in neuroinformatics, 11, 76.

Neheller (talk) 15:51, 1 April 2020 (UTC)[reply]

The discussion continues in Section "Reproducibility" vs "replicability" below... — Preceding unsigned comment added by Werner A. Stahel en (talk • contribs) 08:41, 15 October 2020 (UTC)[reply]

"The Nature study was itself reproduced in the journal PLOS ONE, which confirmed that a majority of cancer researchers surveyed had been unable to reproduce a result" This is not true. The PLOS ONE paper referenced in 13 is a separate study that finds supporting results. It is not a replication of the Nature study.

It is also a stretch to say "Attempts to reproduce studies often strained relationships with the laboratories that were first to publish" The article only claims that 66.7% of conflicting results were not resolved through communication with the authors of the original publication. The categories they use for these types of responses are "no response" and "negative or indifferent." There are no qualitative comments included in the study concerning the interactions between researchers. Kotfic (talk) 13:39, 14 October 2013 (UTC)[reply]

Kotfic, I agree with your comment on "strained relationships"; I was looking in the article that is cited in support of that sentence, and I was baffled by the lack of any such statement. So while it may be true that strained relations resulted, it isn't supported by the cited reference, and I edited out that sentence (but left the citation, which I believe was in support of earlier statements in that para). If someone thinks the sentence I removed should remain, then I think they need to provide a better citation.Mcswell (talk) 20:44, 2 June 2017 (UTC)[reply]

• The ireproducibility of materials is discussed in:Bodyagin, N.V.; et al. "Structural and Behavioural Irreproducibility of Solid Materals". Russian microelectronics. Springer. Retrieved 2 October 2013. {{cite web}}: Explicit use of et al. in: |last= (help)--5.15.0.241 (talk) 10:00, 30 November 2013 (UTC)[reply]

There are these projects http://psychfiledrawer.org/ and https://osf.io/ezcuj/ that might be important and have been widely cited like in Nature twice: [2] [3], [4] or for the general public here in the Guardian: [5]. The German version has a whole article on the topic: de: Reproduzierbarkeit (Psychologie).
How is http://www.rrplanet.com/ related to the topic? Seems misplaced to me.Galant Khan (talk) 15:54, 8 October 2014 (UTC)[reply]

The two terms - "reproducibility" and "replicability" - are sometimes used interchangeably. However, according to some [6], their meaning should be used in the following way:

• Reproducibility - is being able to reproduce the analysis of the data in order to arrive to the same numerical results.
• Replicability - is being able to reach the same scientific finding, when the experiment is conducted under similar conditions, in a secondary research lab.

This confusion should be (at some point) clarified in the article (and maybe lead to some major changes).

I am not (yet) making any changed to the article, but would first like to read other peoples opinion on this matter.

Tal Galili (talk) 21:56, 9 November 2014 (UTC)[reply]

Here's a better link. --Dodi 8238 (talk) 16:19, 10 November 2014 (UTC)[reply]

Very nice discussion of these two terms on the Language Log site, with lots of examples of usage. Comtebenoit (talk) 12:39, 12 January 2016 (UTC)[reply]

I added the "self-contradictory" template. It seems to me that the article is incoherent in the definition of "reproducibility". One paragraph in introduction section referencing ASTM (I can't check because of paywall) is presumably using a different definition than the definition used in PNAS article by Leek and Peng referenced in the first paragraph. Maybe "reproducibility" is used in different ways in different fields? 119.35.103.193 (talk) 05:20, 24 September 2017 (UTC)[reply]

Reproducibility is defined in the same way, both by NIST and JCGM. This is a technical standard definition, so everybody on Earth can express the same concept. I don't understand why the definition have to be different here. I had changed the definition before, but someone reverted it. I'm sorry, but I am re-reverting it. Rudolf Hellmuth (talk) 12:47, 13 April 2018 (UTC)[reply]

Rudolf Hellmuth Since you said that you have re-reverted the topic, should the "self-contradictory" template be removed from the page? Tibbs runner (talk) 16:30, 7 May 2018 (UTC)[reply]

Tibbs runner, I wouldn't remove it before someone reviews this article with more attention. The ASTM standard would have to be checked too. I don't have it (the paywall, as commented above). I suppose the ASTM standard follows the GUM standard, as NIST does. Rudolf Hellmuth (talk) 10:56, 9 May 2018 (UTC)[reply]

Rudolf Hellmuth So if we were able to get a copy of the document, and then confirm whether or not it agrees, we would be able to remove the template, but until then we don't do anything? Tibbs runner (talk) 15:16, 9 May 2018 (UTC)[reply]

Tibbs runner -- Yeah, that's the how open projects, like Wikipedia, work. Wikipedia is dynamic. It will be fixed when someone, who has access to that document, and good-will and free time, decides to contribute to this article. I try to contribute every the time I come across to something in my work that is not properly explained on Wikipedia (see Wikipedia:WikiGnome). Feel free to track down the ASTM standard, and update the article. Just don't forget to include the reference. --Rudolf Hellmuth (talk) 16:05, 9 May 2018 (UTC)[reply]

A recent reference on terminology ("Terminologies for Reproducible Research", Lorena Barba) is available at https://arxiv.org/pdf/1802.03311.pdf — Preceding unsigned comment added by Nicolas P. Rougier (talk • contribs) 17:42, 26 June 2018 (UTC)[reply]

Thank goodness this was fixed, this was a glaring error in the definition of reproducibility and prevented me from recommending it to others. I made a comment on this page back in 2013 but couldn't change the meaning on the page myself. Rhodydog (talk)

This discussion seems to have reached a saturation, and the article still has remained with the multiple issues. I make an attempt to implement the result, see the revised article. Thanks everybody for the contributions to the discussion. Werner A. Stahel en (talk) 08:36, 15 October 2020 (UTC)[reply]

A new paper in Nature discusses a major problem uncovered regarding the reproducibility of tests that use commercial antibodies. This will require re-examination of a decade's worth of medical research. Something about it belongs in this article. LeadSongDog come howl! 15:36, 22 May 2015 (UTC)[reply]

Further papers doi:10.1371/journal.pbio.1002333 and doi:10.1371/journal.pbio.1002331 in PLoS Biology examine just how rarely articles include the necessary information to enable reproduction.LeadSongDog come howl! 21:39, 6 January 2016 (UTC)[reply]

In a deterministic world, no experiment is GLOBALLY reproducible (ie: the dropping of the atomic bombs on Hiroshima and Nagasaki could not be repeated/reproduced). LOCALLY (over a small region of spacetime) experiments APPEAR to be reproducible. Ultimately, I feel the article should comment upon how the lack of free-will, or superdeterministic nature of reality, would impact upon the notion of repeatability/Reproducibility. For an experiment to be TRULY repeatable, you have to Reproduce the EXACT circumstances in which in first occurred. However, this cannot be done BECAUSE there are always variables (including potentially hidden variables) which change the second time the experiment occurs in comparison to the first time that the experiment occurs. However, I would agree, that at the macroscopic level, reality does appear to be HIGHLY reproducible (in that the same local patterns in spacetime occur - eg: a fast car crash will usually lead to death for someone BUT each person can only die once - so even that pattern is not fully Repeatable/reproducible).

ASavantDude (talk) 22:01, 12 April 2016 (UTC)[reply]

Thanks for weighing in, User:ASavantDude. What we're doing here on Wikipedia is summarising what the best published reliable sources say about the topic, not our own opinions. The article has to explain reproducibility as the term it is used in science, not in a Wikipedian's own theory. Fortunately, a lot has been published about the topic. Another tip: overuse of capital letters in a forum like this is regarded as shouting, and people will react the same way they would to someone coming into the room and shouting at them. You can make the same point just as well without the excess capitalisation. Cheers, MartinPoulter (talk) 17:01, 27 May 2016 (UTC)[reply]

Can the reproducibility requirement be applied to stochastic processes?--79.119.213.36 (talk) 16:23, 20 August 2016 (UTC)[reply]

This article states in the opening paragraph that repeatability and replication are different, and then proceeded to conflate them throughout the entire text. We may need to clean this up. --Wikiman2718 (talk)

There are now individual sections on this talk page for every pair of these three so I think we should unify the discussion here. The apparent source of the conflict is that different scientific communities use these words in different ways, in many cases providing their own definitions and standards which conflict with the definitions and standards of others. Recent academic papers (listed below -- both open access) have recounted the history of this debate.

• Barba, L. A. (2018). Terminologies for reproducible research. arXiv preprint arXiv:1802.03311.
• Plesser, H. E. (2018). Reproducibility vs. replicability: a brief history of a confused terminology. Frontiers in neuroinformatics, 11, 76.

In light of this, the fact that the article attempts to present a single definition of these terms is problematic. That the article proceeds to conflate them is more problematic still. In my opinion, the lack of consensus should be addressed in the introduction and expanded on in its own section, and consistent terms should be used thereafter, potentially the explicit terms proposed by Goodman, Fanelli, and Ioannidis:

• Methods Reproducibility
• Results Reproducibility
• Inferential Reproducibility

in the following publication (also available open access)

Goodman, S. N., Fanelli, D., & Ioannidis, J. P. (2016). What does research reproducibility mean?. Science translational medicine, 8(341), 341ps12-341ps12.

Neheller (talk) 16:38, 1 April 2020 (UTC)[reply]

This article may be too technical for most readers to understand. Please help improve it to make it understandable to non-experts, without removing the technical details. (Learn how and when to remove this message)

This topic is in need of attention from an expert on the subject. The section or sections that need attention may be noted in a message below.

The lead paragraphs of this article are hard to understand, so I propose to re-write the lead, as follows:

In science, reproducibility is the level of agreement between the results of measurements of the same thing under different conditions of measurement,^[1][2][3][4] for example, to verify the results of a scientific experiment, or to check that the results are not an artefact of the measurement procedures.

A related concept is replication, which is the ability to independently achieve non-identical conclusions that are at least similar, when differences in sampling, research procedures and data analysis methods may exist.^[5] Reproducibility and replicability together are among the main tools of the scientific method^[6]; to facilitate this, scientific literature should describe “an experiment or analysis . . . in sufficient detail that other scientists with sufficient skills and means can follow the steps described in published work and obtain the same results within the margins of experimental error.”^[7]

The exact definitions of the terms “reproducibility” and “replicability,” however – or at least the difference between them – vary considerably across research disciplines and fields of study.^[7][4][8][9] The study of reproducibility is an important topic in metascience.^[10]

Or, in the alternative, here is a shorter version (where most of the text has been removed, leaving a chain of references):

In science, reproducibility and replicability are related concepts pertaining generally to the level of agreement between the results of similar experiments. An important function of scientific papers and journal articles is to describe “an experiment or analysis . . . in sufficient detail that other scientists with sufficient skills and means can follow the steps described in published work and obtain the same results within the margins of experimental error.”^[7] The exact definitions of the terms “reproducibility” and “replicability” vary considerably across research disciplines and fields of study^{[3][4][11][2][7][12][4][13][14][15]}; indeed, even though it is agreed that the two terms are similar but not identical, the differences between them are sometimes reversed. Reproducibility and replicability together are among the main tools of the scientific method.^[16]

This is a basic topic in science, and so the first sentence should not seem so technical as to repel the general reader away, but to avoid sacrificing correctness, I have self-reverted all my edits and pasted my revision above, instead. Bwrs (talk) 23:25, 5 June 2020 (EDT)

Replication is the foundation of reproducibility, not the same thing. Am I wrong? Houshya (talk) 18:48, 29 November 2022 (UTC)[reply]

I've added a link to Reproducibility Networks that are now quite widespread in Europe (see https://www.ukrn.org/global-networks/). I've also created the entry for Reproducibility Networks - Nicolas P. Rougier (talk) 19:06, 19 April 2024 (UTC)[reply]