{"id":324,"date":"2019-02-28T08:30:29","date_gmt":"2019-02-28T14:30:29","guid":{"rendered":"http:\/\/www.nano-blog.com\/?p=324"},"modified":"2022-09-02T13:01:02","modified_gmt":"2022-09-02T18:01:02","slug":"the-challenges-of-reproducible-research","status":"publish","type":"post","link":"http:\/\/www.nano-blog.com\/?p=324","title":{"rendered":"The Challenges of Reproducible Research"},"content":{"rendered":"<p>Twice last year, April and November, this blog addressed fake science.\u00a0 That challenge has not gone away based on recent news articles.\u00a0 But if one assumes the research is accurately reported and the independent research has differing results, the question is: \u201cHow can that be explained?\u201d<\/p>\n<p>One of the first considerations needs to be the equipment employed.\u00a0 Ideally the research report will include the type (make and model number) of equipment employed in the experiments.\u00a0 Equipment model numbers would provide a means of determining the capability of equipment identified to provide both accuracy and precision of results.\u00a0 One could assume that this should be sufficient to get good correlation between experimenters.<\/p>\n<p>One example of experimental correlation that I have close knowledge about was between researchers in Houston, Texas and Boston, Massachusetts.\u00a0 The experiments involved evaluating properties of graphene. A very detailed test was conducted in the Houston lab.\u00a0 The samples were carefully packaged and shipped to Boston.\u00a0 There was no correlation between the two researchers on the properties of the graphene. \u00a0The material was returned and reevaluated.\u00a0 The results were different from the original tests.\u00a0 Eventually, it was shown that the material had picked up contamination from the air during transit.\u00a0 This led to a new round of testing and evaluations.<\/p>\n<p>The surprise was that the results from testing by the different researchers were close but not identical.\u00a0 So, another search was started.\u00a0 Equipment model numbers were checked, and then serial numbers were checked and sent to the equipment supplier.\u00a0 It turned out that during the production run, which was lasted a number of years, a slight improvement was made in the way the equipment evaluated the samples.\u00a0 This was enough to change the resulting measurements at the nanoscale.\u00a0 Problem solved.<\/p>\n<p>This example points to the need not only for calibration but also working with other researchers to determine discrepancies in initial research results.\u00a0 Reports need to include test procedures and equipment employed.\u00a0 Ideally, it should include some information on calibration.<\/p>\n<p>In many cases, we take calibration for granted.\u00a0 How many people assume that their scales at home are accurate?\u00a0 How many check oven temperatures?\u00a0 The list goes on.\u00a0 I have been in that group.\u00a0 We replaced our old range\/oven this past December.\u00a0 The oven appeared to be working fine until Christmas cookies were baked.\u00a0 They came out a bit overdone.\u00a0 The manufacturer indicates in the instructions that it may take time to make slight modifications in how you bake.\u00a0 I decided to check the actual temperatures.\u00a0 I found that a setting of 350<sup>o<\/sup>F resulted in an oven temperature of 350<sup>o<\/sup>F.\u00a0 However, a setting of 400<sup>o<\/sup>F resulted in an oven temperature of 425<sup>o<\/sup>F, which is enough difference to over bake the cookies.\u00a0 Almost weekly visits over two months by the manufacturer\u2019s service technician has resulted in replacement of all replaceable part and a number of recalibrations with no apparent change in performance.\u00a0 After the first few weeks, I decided to do a more thorough test of the oven.\u00a0 (My measurements agree with the technician\u2019s better equipment to within a couple of degrees.)\u00a0 What I found is that at 350<sup>o<\/sup>F the oven is solid.\u00a0 At 360<sup>o<\/sup>F, the oven temperature is almost 395<sup>o<\/sup>F.\u00a0 At higher temperatures, the difference between the oven setting and actual reading decreases to 25 degrees.\u00a0 Between 450<sup>o<\/sup>F and 500<sup>o<\/sup>F the temperature difference decreases from a plus 25<sup>o<\/sup>F to a minus 5<sup>o<\/sup>F.\u00a0 The point of this example is that assuming calibration is accurate even for common appliances is not something to be taken for granted.\u00a0 Calibration must be done, and calibration for scientific equipment must be done regularly to ensure accuracy of results.<\/p>\n<p>Disagreement among researchers in test results does not indicate there are errors in the experiments, but that the equipment could be out of calibration.\u00a0 It is critical to verify calibration of equipment employed in research before claiming faulty research.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Twice last year, April and November, this blog addressed fake science.\u00a0 That challenge has not gone away based on recent news articles.\u00a0 But if one assumes the research [..]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5,12],"tags":[],"class_list":["post-324","post","type-post","status-publish","format-standard","hentry","category-nanotechnology-risk-management","category-science"],"_links":{"self":[{"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=\/wp\/v2\/posts\/324","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=324"}],"version-history":[{"count":1,"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=\/wp\/v2\/posts\/324\/revisions"}],"predecessor-version":[{"id":325,"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=\/wp\/v2\/posts\/324\/revisions\/325"}],"wp:attachment":[{"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=324"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=324"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.nano-blog.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=324"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}