Critique of the DOI Scientific Integrity Policy
Critique of the DOI Scientific Integrity Policy
August 8, 2012
Introduction:I served as a member of the Department of Interior (DOI) and the Bureau of Reclamation (BOR) Scientific Integrity Policy writing team which assembled thisPolicy. After this Policy was adopted I served as the BOR Scientific Integrity Officer (BSIO). In that capacity, I was the official who processed a significant scientific integrity allegation (Judge Wanger’s September 2011 allegations on Delta-Smelt issues).
At the same time, I have personally experienced that Policy from another perspective. I was the subject of whistleblower retaliation related to scientific integrity. In this connection, I authored and filed a scientific integrity allegation under this Policy concerning Klamath Dam removal.
From these experiences on both sides of the divide, I am in a unique position to offer a critique of the DOI’s Scientific Integrity Policy.
I. Lack of Independence and Consistency in Allegation Inquiry Process
Section 3.8 crudely outlines the process for inquiries into allegations of scientific and scholarly misconduct and contains a number of flaws related to the formality of the inquiry process, due process, independence, and accountability that allows the Department to make up the procedures as it likes.
A. Too Much Discretion. The inquiry process called for in the Policy is largely controlled by the DSIO and the Bureau Scientific Integrity Officer (BSIO) who have the authority to summarily dismiss the allegation after reviewing the submitted information. If they determine that an investigation is warranted, then they can perform fact finding, and convene a panel of experts to advise them on the merits of the investigation. As written, these procedures give too much discretion to the DSIO and the BSIO to decide the fate of the allegation and the procedures by which it should be investigated. These procedures should be significantly improved with appropriate oversight, checks and balances.
B. Lack of Independence. The oversight independence of the DSIO and the BSIO’s are dubious since they report to the regular chain-of-command. Additionally, the inquiry process calls for involving the subject’s manager and Departmental leadership in the process with little regard for conflicts of interest. The policy needs to establish a separate oversight function that does not report to political appointees or is itself subject to Departmental politics.
The DSIO, BSIO’s and the Department leadership are naturally biased in favor of the Department, and against the allegation: they naturally want the Department to be found to uphold scientific integrity. However, this bias can also purturb the inquiry process. One example: pre-written questions asked of expert panels can naturally lead the panel to a pre-determined conclusion.
DOI often convenes these panels via sole-source contracts to companies (e.g. ATKINS) that want repeat business; if the panel hired by the company does not find in favor of the Department, it may risk future business. Therefore, it is imperative that the Policy directly address these biases and conflicts of interest, and establish the DSIO and BSIO’s with truly independent oversight.
C. Preeminence of Departmental Mission. The scientific integrity policy creates conflicts when science results do not support the mission or agenda of the Department. In these situations, scientific integrity should not be overridden or bypassed.
A special provision for political appointees should be included in the Policy that prevents them from managing or influencing the scientific integrity policy or process. The Policy should explicitly state that political agendas and initiatives must be guided by scientific integrity, and that scientific integrity trumps Departmental policies or political agendas.
Section 3.7A states:
“I will act in the interest of the advancement of science and scholarship for sound decision making, by using the most appropriate, best available, high quality scientific and scholarly data and information to support the mission of the Department.” (Emphasis added)
By including “…to support the mission of the department” in this statement, the Policy explicitly places Departmental initiatives and political agendas above scientific integrity. This statement must be dropped.
D.The Policy’s inquiry process does not establish an explicit due process. Even standard scientific peer-review procedures allow for a dialogue to develop better information and resolve issues. The Policy’s fact finding and expert panel process should explicitly involve the accuser and the accused in due process proceedings.
E. No Penalties. The policy does not establish penalties for scientific misconduct, but rather leaves them up to the manager. A formal establishment of penalties and accountability of anyone found guilty of scientific misconduct or retaliation/suppression of scientific freedom should be explicitly included in the Policy.
F. Policy Inconsistencies. The Policy offers a broad code of scientific and scholarly conduct (Section 3.7), and separately offers definitions of scientific and scholarly integrity (Section 3.5L), scientific and scholarly misconduct (Section 3.5M), and procedures for reporting and resolving allegations regarding a loss of scientific and scholarly integrity (Section 3.8). While there are some ties between these policy statements (for example Section 3.7A(6) and Section 3.7B(2)), there are many guidelines offered in the code of conduct, that when violated are not traceable to the procedures for resolving and reporting a loss of scientific and scholarly integrity (Section 3.8).
Further, the responsibilities sections (Section 3.6G-I) offers different guidance for the same groups of people that the code of conduct addresses (Section 3.7). These definition and Policy inconsistencies make the Policy confusing and less enforceable. These conflicts need to be resolved, with explicit procedures for reporting and resolving any intentional breach of the code of conduct and/or scientific and scholarly integrity (not just plagiarism, falsification and fabrication).
II. Debilitating Lack of Transparency
The Policy would greatly benefit from strong and explicit guarantees of transparency. This disturbing lack of openness can be found through the DOI scientific process encompassed by the Policy;
A. Misconduct Inquiries. The Policy’s inquiry process has no requirements for public transparency or reporting. To gain the public trust, the Policy should have explicit requirements for transparency and reporting about the way that the Policy is being implemented, the reason decisions were made, and scientific misconduct correction actions.
The Policy should commit to publicly reporting alleged and confirmed lapses in scientific integrity, and develop and incorporate additional mechanisms to enhance transparency in DOI’s adherence to its Scientific Integrity Policy.
B. Open Science. The Policy should explicitly grant all government scientists the right to freely communicate with the press and the public, without fear of retribution, censorship or consequence. Section 3.4E directs the Department to develop a communications policy along these lines, which was finally issued in March 2012.
The Policy should ensure that Federal science and decision making is communicated freely and transparently for public scrutiny; this is an important way to reveal and end political interference in science. Federal scientists should be performing and reporting on science that is in the public interest, and the American public (who pay for this science) should be able to trust that its science is not being performed in support of a political agenda.
Section 3.7A(2) states “I will communicate the results of scientific and scholarly activities clearly, honestly, objectively, thoroughly, accurately, and in a timely manner”. This statement should be modified to explicitly include public communication.
C. Remove FOIA Gag. Further, the March 2012 DOI Communications Policy forbids employees from disclosing anything covered by a FOIA exemption, such as “pre-decisional” information, and discourage specialists from revealing any information not previously published or otherwise publicly released by the Department. These rules effectively restrict scientists from saying anything new, and significantly impede the development of a culture of openness and transparency with the public.
D. Creating a Clear Scientific Record. Section 3.7A9(10) states: “I will be diligent in creating, using, preserving, documenting, and maintaining scientific and scholarly collections, records, methodologies, information, and data in accordance with federal and Departmental policy and procedures”. This should include providing easy public access to this information. Similar modifications are needed for Section 3.7B(3-4).
The policy should mandate the communication of scientific and technological findings by including a clear explication of underlying assumptions; accurate contextualization of uncertainties; and a description of the probabilities associated with both optimistic and pessimistic projections, including best-case and worst-case scenarios. This information, even if presented in very simple summaries, is critical to support good decision making.
III. No Whistleblower Protection
The DOI policy only provides a cursory mention of whistleblower protections, in directing the Department to provide its employees information (3.4F).
Current whistleblower laws are complex, are stacked in the governments favor, and generally do not protect employees who raise concerns about subjects that are part of their jobs. Until these policies are improved, DOI should develop and incorporate
additional whistleblower protections into the Scientific Integrity Policy and provide a more detailed explanation of procedural safeguards to be instituted, in order to adequately protect agency scientists and employees who report scientific misconduct or political interference with their research.
The policy should explicitly grant scientists who report political interference in their work protection from retaliation. Explicit whistleblower protections must be included in the Policy to ensure not only government accountability, but also protection for agency employees who exercise their free speech rights and who facilitate the free flow of scientific and technological information to challenge institutional illegality, abuse of power, or other betrayals of the public trust. Government scientists must have the assurance that their primary duty is to the American people, and that they have an obligation and full protection to uphold the public trust.
Finally, the policy should explicitly protect the accused against retaliation or censorship of all parties throughout the allegation and inquiry process.
IV. Public and Peer Review
The Policy refers to reviews in several sections, but never explicitly defines the review guidelines. The Policy should include explicit public- and peer-review definitions and set guidelines for review procedures. These guidelines should establish the kinds of work that require review, the processes to ensure independent and conflict-free reviews and procedures to include due process (reviewer-reviewee iterations) and public transparency in the review processes.
There also needs to be an explicit response to review comments, as many programs profess that their programs or science are peer-reviewed as a justification for their validity, without ever taking action or even responding to review findings or suggestions. Finally, the Policy should establish procedures for appropriately handling differing scientific opinions and ensuring that these opinions are included in the final versions of scientific documents.
V. Conflict of Interest
Section 3.5A offers a broad definition of conflict of interest, which gives great leeway in subjective interpretation, and does little to give practical examples or to enforce conflict of interest rules.
The Policy needs to explicitly define conflict of interest, and give practical guidelines and rules. The conflict of interest policy also needs to have time guidelines, because conflicts of interest do not necessarily disappear once a financial or professional relationship is concluded. For example, an individual should be barred from handling of scientific decision making (peer-reviews, panels, funding, policy, etc.) if they are conflicted in among the following ways:
Lifetime for academic advisee/advisor relationship.
- 5-Years for scientific collaboration on a project, report, or paper.
- 5-Years for having worked at the same institution.
- 5-Years for having had any financial or political interests, or potential to gain or lose.
- Any of the above concerning family members.
Moreover, intentional violations of conflict of interest rules should be considered scientific misconduct.
The Policy should go beyond a simple definition of conflict of interest by strengthening the disclosure of and reducing conflict of interest among employees and reviewers.
Section 3.7B(1) states:
“I will place quality and objectivity of scientific and scholarly activities and reporting of results ahead of personal gain or allegiance to individuals or organizations.”
This statement implies a subjective and personal managing of conflicts of interest that could be dangerous. This statement needs significant revision to report on and remove employees from real and perceived conflicts of interest situations.
Section 3.7B(6) states:
“I will provide constructive, objective, and professionally valid peer review of the work of others, free of any personal or professional jealousy, competition, non-scientific disagreement, or conflict of interest.”
This statement also encourages a scientist to internally manage their own conflicts of interest. This statement needs to be modified to direct the scientist to voluntarily declare any conflicts of interest and excuse themselves from the peer review.
###