Equity & Access
AMS Ethical Guidelines
Prioritizing Scientific Initiatives
(Posted with the kind permission of John Bahcall and Science.)
What are the most important aspects of the universe to explore? What are the best ways to make discoveries in astronomy and astrophysics? These are tough questions because researchers have many different approaches and it is usually not clear, until the most interesting problems are solved, which method will yield the most important results. Individual astronomers present strong arguments for many potential approaches that require federal funding.
We are well into an era of limited research budgets, however, and choices have to be made. Astronomers have recognized that if they do not set their own priorities, then funding agencies and congressional officials will do it for them. Moreover, the process of convincing colleagues in different specialties improves the proposals and provides a broader outlook for the community of researchers.
Astronomers have recently provided some answers to the hard questions of what to fund and, by implication, what to cut. Working under the auspices of the National Research Council, the astronomers have recommended funding for a limited number of initiatives, ranked in order of priority. Only one out of every ten highly promising initiatives survived this rigorous selection.
In this article, I will describe, from my perspective as chairman of the committee, how we came to a consensus on these priorities. I hope that an understanding of our experience may provide further support for the results of our study, as well as offer a possible mechanism for others who must make difficult choices at a time when discretionary budgets are limited.
The group charged with setting priorities, the Astronomy and Astrophysics Survey Committee for the 1990's, was established by the National Research Council (NRC) in May 1989, following my appointment as chair in February 1989. The report of the committee, The Decade of Discovery in Astronomy and Astrophysics, was published in March 1991 by the National Academy Press.
The first step was to find an outstanding group of scientists who were willing to sacrifice a significant part of their research time in order to serve on the committee. I spent most of the months between February and May of 1989 talking to hundreds of astronomers about potential members who might serve on the advisory panels of the survey and on the executive committee (hereafter, the survey committee). I also wrote to the chair of every astronomy department in the United States, as well as to many other prominent astronomers, requesting nominations. I invited each person to suggest themes and questions for the study. In addition, I wrote to a number of distinguished astronomers abroad asking about astronomical programs in their countries and requesting advice about possible international collaborations.
The 15 members of the survey committee were nominated by the appropriate committees of the National Research Council and were appointed by Frank Press, the president of the National Research Council. The survey committee contained six members of the National Academy of Science, two Nobel Prize winners, and two directors of national observatories. The committee selected the chairs of 15 advisory panels for different subdisciplines, based on discussions with astronomers of different specialties at institutions throughout the country. The panel chairs and the survey committee selected 300 people for the advisory groups who had a high level of scientific achievement and who also represented different research approaches, different kinds of institutions, and different geographical areas.
Each panel met at different sites in the United States in order to help stimulate wide participation by the astronomical community I also wrote to each of the panel members asking them to solicit the views of colleagues at their home institutions. The survey committee itself considered projects that spanned more than one subfield or which tell between the assigned responsibilities of the panels.
Prior to the formation of the survey committee, Frank Press and I visited major agency heads and congressional and administration, leaders in order to obtain their advice on what issues the report should address and in what form the results should be presented. I did not ask for support of any projects, but I did hope to create favorable climate for future consideration of astronomy initiatives I also did not ask what answers would be politically most desirable Participants in the survey were encouraged to solicit facts from agency and administration authorities, but we evaluated ideas and initiatives independently and in confidence Agency leaders, congressional staffers, senior people at the Office of Management and] Budget, and the President's science advisor (who had gone through a similar experience as chair of a previous NRC decade survey for physics) all provided valuable advice.
The consultations in Washington resulted in several important sections of the final report: a chapter on the lunar initiative, a chapter on high speed computing, an emphasis on priorities for technology in this decade that will lead to science in the next decade, 3 recommendations of what astronomers should do pro bono to help with the crisis in education, a chapter on astronomy as a national asset, an examination of the technical heritage of proposed initiatives, realistic estimates of the costs for each of the new projects, an examination of thc role of American astronomy in the international context, some guidelines for assessing when international collaborations would be fruitful, and thumb-nail sketches of major projects that could be used conveniently by those drafting legislation.
We felt it was essential to involve the community as much as possible: Every astronomer who had something to say had an opportunity to be heard. Open discussions were held in conjunction with meetings of the American Astronomical Society (AAS) and at several other professional meetings. In January 1990, at the Washington D.C. meeting of the AAS, nearly 1000 astronomers participated in open sessions that involved all 15 of the panels. The names of the survey committee members and of the chairs of the panels were published in the newsletter, along with remarks encouraging individual astronomers to present their ideas directly to survey committee members, panel chairs, or panel members.
The most intense discussions in the first nine months of the survey occurred within the panels. In order to ensure good communication between the panels and the survey committee, each member of the survey committee served as the vice-chair of one of the panels. This arrangement worked well, keeping the survey committee apprised of ideas as they developed and enabling each panel to understand the goals and procedures of the full survey.
The survey committee avoided many potential problems by deciding that the panel reports would be advisory rather than part of the findings of the survey and that the reports would not be refereed by either the survey committee or by the NRC. The recommendations of the panels were not binding on the survey committees but 4 the panel reports contain important technical information, as well as detailed arguments advocating specific initiatives. The reports of the panels were published separately from, but simultaneously with, the full survey report by the National Academy Press under the title Working Papers: Astronomy and Astrophysics Panel Reports.
Establishing the recommendations of the survey took 14 months, about a year less than was projected. The survey committee had six meetings at astronomical centers throughout the country.
I was surprised by one thing. Veterans of similar activities assured me that there would be a difficult and tense period of bargaining before we agreed on the final recommendations. This never happened. I am not certain why. One possible reason is that the committee judged the initiatives on the basis of scientific potential, not political considerations.
The list of priorities was established by a gradual process that was much easier than any of the survey committee members anticipated. The committee voted on straw ballots on three occasions, using as background material the preliminary reports of the advisory panels. The straw ballots focused the discussion on projects that were most likely to be considered important in the final deliberations. As a preliminary to the final ballot, the committee heard advocacy presentations from the panel chairs. The chairs also participated in discussions of the relative merits of all the initiatives, although the final recommendations were formulated by the survey committee in executive session.
Two strategic decisions helped the committee reach a consensus quickly and smoothly. First, the committee decided that if we failed to reach agreement in July 1990 at the pleasant facilities of the National Academy, within reach of the cool breezes from the beach on Irvine, California, then we would mect a month later in the least desirable place in the middle of summer that we could think of, namely, Washington, DC.
Second, several committee members proposed that I draw up, on the evening before the final voting, a draft list of recommended initiatives in order of priority. Their suggested that the committee alter by consensus the draft set of recommendations in order to arrive at the final list of priorities. The proposers hoped that, by this process, the committee could avoid having "winners or losers." I was skeptical of the chances for success when the idea was proposed, but I agreed to try.
Having drawn up a handwritten list of priorities on the night before our formal voting, I was surprised the next day at how rapidly we reached a consensus. We began with those equipment categories concerning which we were most in agreement and then worked our way to the more difficult choices. We went around the table, everyone stating their views about what change, if any, needed to be made in the ordered list that we were considering. By the time we had all spoken, the consensus was obvious and we adopted unanimously our priorities in each category.
In preliminary discussions, most agency personnel opposed absolute rankings that combined ground and space initiatives, worrying that their top priorities might be adversely affected by ineffectiveness at some other agency. The survey committee provided both separate and combined rankings of ground and space initiatives, believing that good citizenship required us to use our expertise to provide the maximum possible guidance.
In times of budgetary crisis, good citizenship also requires fiscal restraint. The survey committee studied approximately ten times as many initiatives as were endorsed, recommending that funding agencies invest in astronomical initiatives according to the scientific priorities established in the survey report.
The committee assigned its highest priority for ground-based astronomy to the revitalization of the infrastructure for research, both equipment and people. Continuing to develop a space program with an improved balance between large and small projects, with emphasis on quicker and more efficient missions, was the committee's highest priority for space research.
The committee recommended that an increased emphasis be given in the astronomy research budget to small and moderate programs (see Table 1). The committee did not prioritize small programs, recognizing that the agencies could use peer review for small initiatives to respond quickly to new scientific or technological developments.
The 180 page book presenting the recommendations was written in about three months. National Research Council reports are reviewed carefully. They must meet high standards of logic, of evidence, and of objectivity. In our case, the National Research Council selected 18 formal references, in addition to a report review committee. The reviewers were anonymous National Academy members and other qualified scientists, in physics, in astronomy, and in other related disciplines. The formal review process was painful, but I answered each review comment, even rhetorical questions, with a specific written response in order that we could complete the review quickly. The 18 referees helped to sharpen our arguments and to clarify our logic, but did not suggest revisions of our priorities.
This is the fourth in a series of decade surveys by astronomers, led by A. Whitford, J. Greenstein, and G. Field, respectively. The highest priority initiatives in each survey were successfully undertaken, encouraging astronomers to submerge parochial interests and focus on the most important initiatives.
Would another committee of astronomical experts have recommended a similar set of priorities? I think so, provided that they had also spent a year learning about and comparing all the proposed initiatives in this country and abroad.
These are the things that worked for us: enlisting as committee members active research scientists eager to finish the job and get back to their own work; recruiting an effective executive secretary; insisting on adequate budgeting and staff support; having a logical plan and a specific timetable for completing the report; listening to everyone who wanted to be heard; concentrating on issues within the committees competence, in our case, scientific priorities; having a talented editor who could sharpen the final report; and working with a community that believes it is better for astronomers to make imperfect judgments about priorities for astronomy than it is to leave the decisions to Washington administrators.
Table 1. Recommended equipment initiatives (combined ground and space) and estimated costs.
(The author is at the Institute for Advanced Study, Princeton, NJ 08540).
From Science, Policy Forum, Vol. 251, March 22, 1991, pp. 1412-1413. Copyright 1991.
Home || The Math Library || Quick Reference || Search || Help