Among the (less technical) items sent my way over the past few days are discussions of the so-called High Quality Research Act. I’d not heard of it, but it’s apparently an outgrowth of the recent hand-wringing over junk science, flawed statistics, non-replicable studies, and fraud (discussed at times on this blog). And it’s clearly a hot topic. Let me just run this by you and invite your comments (before giving my impression). Following the Bill, below, is a list of five NSF projects about which the HQRA’s sponsor has requested further information, and then part of an article from today’s New Yorker on this “divisive new bill”: “Not Safe for Funding: The N.S.F. and the Economics of Science”.

[DISCUSSION DRAFT]

A BILL

April 18, 2013

TO [BE SUPPLIED]

Be it enacted by the Senate and House of Representatives of the United States of America in Congress assembled,

SECTION 1. SHORT TITLE.

This act may be cited as the “High Quality Research Act”.

SECTION 2. HIGH QUALITY RESEARCH.

(a) CERTIFICATION.—prior to making an award of any contract or grant funding for a scientific research project, the Director of the NSF shall publish a statement on the public website of the Foundation that certifies that the research project—

(1) is in the interests of the U.S. to advance the national health, prosperity, or welfare, and to secure the national defense by promoting the progress of science;

(2) is the finest quality, is ground breaking, and answers questions or solves problems that are of utmost importance to society at large; and

(3) is not duplicative of other research projects being funded by the Foundation or other Federal Science agencies.

(b) TRANSFER OF FUNDS.—Any unobligated funds for projects ot meeting the requirements of subjection (a) may be awarded to other scientific research projects that do meet such requirements.

(e) INITIAL IMPLEMENTATION REPORT.—Not later than 60 days after the date of enactment of this Act, the Director shall report to the Committee on Commerce, Science, and Transportation of the Senate and the Committee on Science, Space, and Technology of the House of Representatives on how the requirements set for in subsection (a) are being implemented.

(d) NATIONAL SCIENCE BOARD IMPLEMENTATION REPORT. __ Not later than 1 year after the date of enactment of this act, the national science board shall report to the committee on commerce, science, and transportation of the senate and the committee on science, space and technology of the house of representatives its findings and recommendations on how the requirements of subsection (a) are being implemented.

etc. etc.

Link to the Bill: 

Rep. Lamar Smith,author of the Bill, listed five NSF projects about which he has requested further information. 

1. Award Abstract #1247824: “Picturing Animals in National Geographic, 1888-2008,” March 15, 2013, ($227,437); 

2. Award Abstract #1230911: “Comparative Histories of Scientific Conservation: Nature, Science, and Society in Patagonian and Amazonian South America,” September 1, 2012 ($195,761);

3. Award Abstract #1230365: “The International Criminal Court and the Pursuit of Justice,” August 15, 2012 ($260,001);

4. Award Abstract #1226483, “Comparative Network Analysis: Mapping Global Social Interactions,” August 15, 2012, ($435,000); and

5. Award Abstract #1157551: “Regulating Accountability and Transparency in China’s Dairy Industry,” June 1, 2012 ($152,464).

________________________

MAY 9, 2013

NOT SAFE FOR FUNDING: THE N.S.F. AND THE ECONOMICS OF SCIENCE

POSTED BY DYLAN WALSH (The New Yorker)

Last month, Representative Lamar Smith, chairman of the House Committee on Science, Space, and Technology, introduced a divisive new bill, the High Quality Research Act, that would change the criteria by which the National Science Foundation evaluates research projects and awards funding. (The N.S.F., with a budget of seven billion dollars, funds roughly twenty per cent of federally supported basic research in American universities.) Currently, proposals are evaluated through a traditional peer-review process, in which scientists and experts with knowledge of the relevant fields evaluate the projects’ “intellectual merits” and “broader impacts.” Peer review is a central tenet of modern academic science, and, according to critics, the new bill threatens to supersede it with politics.

John Holdren, the director of the White House Office of Science and Technology Policy, said last week that “adding Congress as reviewers is a mistake.” Representative Eddie Bernice Johnson warned more forcefully that Representative Smith was “sending a chilling message to the entire scientific community that peer review may always be trumped by political review.” But in a statement, Representative Smith said the draft bill “improves on [the peer-review process] by adding a layer of accountability.” The bill’s new three-point criteria for funding require that a project be “in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense”; solve “problems that are of the utmost importance to society at large”; and not be “duplicative of other research projects being funded by the Foundation or other Federal science agencies.”

Implicit in the proposal’s language is a desire for oversight built into the process of determining which areas of study are significant. (Representative Smith cited five N.S.F.-funded social science projects, with concerns as to whether they “adhere to NSF’s ‘intellectual merit’ guideline”(above). To Smith’s point, despite sizable public investment in research and development (nearly $150 billion this year), relatively scant attention is devoted to investigating whether the process of science, in its current form, is well-designed for generating knowledge.

As it turns out, the N.S.F. recently awarded a grant to Kevin Zollman, an assistant professor of philosophy at Carnegie Mellon University, to investigate what drives researchers to pursue particular projects, and how they secure funding for them. In an ideal world, all science would proceed from the simple and lofty social goal of expanding human knowledge, but researchers are of course subject to the constraints of economic reality. To disentangle these knotted incentives, Zollman is applying a branch of game theory known as “mechanism design,” which uses simple models to understand how individual players within a system balance competing motivations to arrive at different ends.

Zollman’s first object of study is the “priority rule,” which states that all of the credit earned by scientific discovery—the money, the praise, the promotion and adulation—is directed to the first researcher across the finish line, whether by weeks, days, or hours. The priority rule was first elaborated in the nineteen-fifties by the sociologist Robert K. Merton, who observed that “In the institution of science, originality is at a premium.” He described its role over centuries of scientific progress, with fierce and stubborn quarrels punctuating the time line of discovery: Newton and Leibniz fought over the invention of calculus, Galileo and Simon Mayr over the discovery of Jupiter’s moons.

As researchers scramble to carve out intellectual real estate, however, overly aggressive competition and singular focus on originality can elicit a host of negative behaviors: bias toward reporting positive or rushed results, withholding or fabricating data, and counterproductive levels of output. Of fifty million scholarly articles published since 1665, more than half of them appeared in the last twenty-five years. While a number of factors contribute to this glut, one of them is the pressure to publish results even when they’re not immediately relevant. “Much of the recent scientific literature is repetitive, unimportant, poorly conceived or executed, and oversold; perhaps deservingly, much of it is ignored,” wrote the microbiologist Ferric Fang in a 2012 editorial. The problem is becoming particularly acute as a growing pool of scientists face a shrinking pot of money, both from drying stimulus funds scheduled to disappear this September and 2.9-percent budget cuts forced on the N.S.F. through sequestration. The currently overcharged pull of priority, by spurring unnecessary questions and hairs to be split, can thus dilute or stifle the advancement of science.

In light of these problems, Representative Smith proposed his bill. Yet the economic straits in science already encourage research that targets specific, funder-driven priorities over riskier, more open-ended questions. As the Nobel laureate Roger Kornberg lamented in 2007, “If the work that you propose to do isn’t virtually certain of success, then it won’t be funded.” Elevated political scrutiny would likely only reduce the willingness of agencies like the N.S.F. to fund projects without clearly defined, or even expected, outcomes. In tension with this reality is the fact that revolution in science is often indebted to prolonged exploration of basic research projects, which may at their outset fail to meet Representative Smith’s criteria. Forecasting how well research will “advance the national health, prosperity, or welfare,” or predetermining the degree to which a project is or is not “groundbreaking,” demands an unlikely prescience. Further embedding specific requirements in grant allocation could make improvements at the margins by defunding egregiously conspicuous research. But it also threatens to close off a large landscape of research questions with unforeseen potential.

The article is here.