Research In-teg-ri-ty: Find out what it means to me

The Office of Research Integrity (ORI) is within the U.S. Department of Health & Human Services and oversees and directs Public Health Service (PHS) research integrity. My first question: What is the PHS? The PHS consists of the National Institutes of Health (NIH), the Food and Drug Administration (FDA), and the Centers for Disease Control and Prevention (CDC), just to name a few. Basically if you are conducting biomedical or behavioral science research at a university or research institute there’s a good chance you are funded by the PHS, at least $30 billion was awarded for health research and development in 2004. Now my second question: What is research integrity? This question is a little harder to answer. The ORI has defined research integrity and here’s the part that stands out to me, “While science encourages (no, requires) vigorous defense of one’s ideas and work, ultimately research integrity means examining the data with objectivity and being guided by the results rather than by preconceived notions.” I especially like the “(no, requires)” part, gives it a real colonial proclamation feel. On a serious note, scientists battling confirmation bias is an issue. The scientific method requires experimenters to construct a hypothesis, or educated guess, before testing. This can make it very difficult to interpret the results without bias. Fortunately, we can use techniques like blinding and statistics to keep observations objective and to make inferences. However, research misconduct is a slippery slope. What could start as data manipulation for clarification can turn into intentional, knowing, or reckless falsification and/or fabrication of data. It’s hard to believe, but it happens. In fact, the ORI publishes case summaries for misconduct involving PHS research.

One of the most recent cases involves a researcher from the NIH by the name of Dr. Skau. Is it lying if you don’t tell the whole truth? Well for research, the whole truth is pretty important, and lying by omission to foster misconception is a serious offense. In this case, the researcher was “selectively reporting by inappropriate inclusion/omission or alteration of data points in 10 figures and falsely reporting the statistical significance based on the falsified data.” One thing I found particularly interesting in the case summary was that the researcher reported that the error bars in one of the figures represented standard deviation when they actually represented standard error of the mean. To understand why this is an issue, we have to go into a little statistics. The example plot below shows the mean (M), standard deviation (SD), 95% confidence interval (CI), and standard error (SE) for three different sample sizes (n = 3, 10, 30). You can see the SE is always smaller than the SD, not to mention the SE gets smaller as the sample size gets larger, SE = SD/root(sample size). Tying it back to the case summary, Dr. Skau made it appear as though the error in her figure was smaller than it actually was.

Credit: Error bars in experimental biology

For these serious allegations, the ORI conducts rigorous investigations and uses a variety of tools. From a 2013 article, the ORI uses forensic imaging tools including forensic droplets, Adobe Bridge, and ImageJ to detect fraud. These tools can do things like compare images, detect lighting adjustments, and organize images by date or file size to trace an image’s history. I expect that there are even more advanced tools today, 5 years later. Of course, that means there are more advanced manipulation tools available to researchers as well. It’s astonishing to read about some of the other ORI cases in 2018 with researchers falsely reusing and relabeling images and falsifying and/or fabricating data published in Nature (With an impact factor of 40.137, Nature is one of the most cited scientific journals).

In the end, it doesn’t end well for the researchers found guilty of research misconduct. Dr. Skau entered into a Voluntary Settlement Agreement with consisted of:

• Research supervision for 3 years
• Confirmation from her employer regarding the legitimacy of her data (if she were to work for a PHS-funded institution within the 3-year period)
• Exclusion for a PHS advisory position
• Retraction of her 2 fraudulent papers

And I’d imagine having research misconduct on your resume makes it awfully difficult to find future work…

My takeaway: Researchers have a responsibility to present their data honestly and clearly. To me, misconception can begin long before blatant falsification and/or fabrication of data. While many of the offenses listed by the ORI seem to be deliberate actions from the researchers to deceive the readers (i.e. mislabeling multiple images), some things, such as referring to the error bars as standard deviation instead of standard error, could foreseeably be unintentional. For Dr. Skau, given the long list of other offenses, along with her admission, the incorrect error bars don’t seem to be an honest mistake. This gets back to the ORI’s assessment of “intentional, knowing, or reckless” behavior. Even when unintentional, misconception is dangerous in scientific publishing. Whether you know or not that the SE error bars would be smaller than the SD error bars, you have a responsibility as a researcher to do your scientific due diligence and report your data honestly and clearly. Honesty should be a relatively easy first step. Don’t change the data or if there is data processing involved, be explicit about your process. The second step, clarity, is a bit harder and is especially important for visualizations. This involves more than double checking that your error bars are in fact standard deviation. There are plenty of other more subtle ways to lie with visualizations. I took an awesome class last semester called Information Visualization and learned all about good (and bad) visualizations and how to leverage human intuition when designing a visualization. You may have heard about how to lie with statistics, similar story for visualization. Be on the lookout for easy-to-spot red flags in visualizations. In my opinion, an individual’s research integrity is largely reflected in his/her visualizations. Beyond the basics of adhering to the ethical principles and professional standards of research, I challenge researchers to step up to challenge of visualizing their information clearly and give their visualizations a little more attention and R-E-S-P-E-C-T.