Prioritizing Scientific Initiatives
(Posted with the kind permission of John Bahcall and Science.)
What are the most important aspects of the universe to explore? What are
the best ways to make discoveries in astronomy and astrophysics? These
are tough questions because researchers have many different approaches
and it is usually not clear, until the most interesting problems are
solved, which method will yield the most important results. Individual
astronomers present strong arguments for many potential approaches that
require federal funding.
We are well into an era of limited research budgets, however, and
choices have to be made. Astronomers have recognized that if they do not
set their own priorities, then funding agencies and congressional
officials will do it for them. Moreover, the process of convincing
colleagues in different specialties improves the proposals and provides
a broader outlook for the community of researchers.
Astronomers have recently provided some answers to the hard questions
of what to fund and, by implication, what to cut. Working under the
auspices of the National Research Council, the astronomers have
recommended funding for a limited number of initiatives, ranked in order
of priority. Only one out of every ten highly promising initiatives
survived this rigorous selection.
In this article, I will describe, from my perspective as chairman of
the committee, how we came to a consensus on these priorities. I hope
that an understanding of our experience may provide further support for
the results of our study, as well as offer a possible mechanism for
others who must make difficult choices at a time when discretionary
budgets are limited.
The group charged with setting priorities, the Astronomy and
Astrophysics Survey Committee for the 1990's, was established by the
National Research Council (NRC) in May 1989, following my appointment as
chair in February 1989. The report of the committee, The Decade of
Discovery in Astronomy and Astrophysics, was published in March 1991 by
the National Academy Press.
The first step was to find an outstanding group of scientists who were
willing to sacrifice a significant part of their research time in order
to serve on the committee. I spent most of the months between February
and May of 1989 talking to hundreds of astronomers about potential
members who might serve on the advisory panels of the survey and on the
executive committee (hereafter, the survey committee). I also wrote to
the chair of every astronomy department in the United States, as well as
to many other prominent astronomers, requesting nominations. I invited
each person to suggest themes and questions for the study. In addition,
I wrote to a number of distinguished astronomers abroad asking about
astronomical programs in their countries and requesting advice about
possible international collaborations.
The 15 members of the survey committee were nominated by the
appropriate committees of the National Research Council and were
appointed by Frank Press, the president of the National Research
Council. The survey committee contained six members of the National
Academy of Science, two Nobel Prize winners, and two directors of
national observatories. The committee selected the chairs of 15 advisory
panels for different subdisciplines, based on discussions with
astronomers of different specialties at institutions throughout the
country. The panel chairs and the survey committee
selected 300 people for the advisory groups who had a high level of
scientific achievement and who also represented different research
approaches, different kinds of institutions, and different geographical
areas.
Each panel met at different sites in the United States in order to
help stimulate wide participation by the astronomical community I also
wrote to each of the panel members asking them to solicit the views of
colleagues at their home institutions. The survey committee itself
considered projects that spanned more than one subfield or which tell
between the assigned responsibilities of the panels.
Prior to the formation of the survey committee, Frank Press and I
visited major agency heads and congressional and administration,
leaders in order to obtain their advice on what issues the report should
address and in what form the results should be presented. I did not ask
for support of any projects, but I did hope to create favorable climate
for future consideration of astronomy initiatives I also did not ask
what answers would be politically most desirable Participants in the
survey were encouraged to solicit facts from agency and administration
authorities, but we evaluated ideas and initiatives independently and
in confidence Agency leaders, congressional staffers, senior people
at the Office of Management and] Budget, and the President's science
advisor (who had gone through a similar experience as chair of a
previous NRC decade survey for physics) all provided valuable advice.
The consultations in Washington resulted in several important
sections of the final report: a chapter on the lunar initiative, a
chapter on high speed computing, an emphasis on priorities for
technology in this decade that will lead to science in the next decade,
3 recommendations of what astronomers should do pro bono to help with
the crisis in education, a chapter on astronomy as a national asset, an
examination of the technical heritage of proposed initiatives, realistic
estimates of the costs for each of the new projects, an examination of
thc role of American astronomy in the international context, some
guidelines for assessing when international collaborations would be
fruitful, and thumb-nail sketches of major projects that could be used
conveniently by those drafting legislation.
We felt it was essential to involve the community as much as possible:
Every astronomer who had something to say had an opportunity to be
heard. Open discussions were held in conjunction with meetings of the
American Astronomical Society (AAS) and at several other professional
meetings. In January 1990, at the Washington D.C. meeting of the AAS,
nearly 1000 astronomers participated in open sessions that involved all
15 of the panels. The names of the survey committee members and of the
chairs of the panels were published in the newsletter, along with
remarks encouraging individual astronomers to present their ideas
directly to survey committee members, panel chairs, or panel members.
The most intense discussions in the first nine months of the survey
occurred within the panels. In order to ensure good communication
between the panels and the survey committee, each member of the survey
committee served as the vice-chair of one of the panels. This
arrangement worked well, keeping the survey committee apprised of
ideas as they developed and enabling each panel to understand the
goals and procedures of the full survey.
The survey committee avoided many potential problems by deciding that
the panel reports would be advisory rather than part of the findings
of the survey and that the reports would not be refereed by either the
survey committee or by the NRC. The recommendations of the panels
were not binding on the survey committees but 4 the panel reports
contain important technical information, as well as detailed arguments
advocating specific initiatives. The reports of the panels were
published separately from, but simultaneously with, the full survey
report by the National Academy Press under the title Working
Papers: Astronomy and Astrophysics Panel Reports.
Establishing the recommendations of the survey took 14 months, about a
year less than was projected. The survey committee had six meetings at
astronomical centers throughout the country.
I was surprised by one thing. Veterans of similar activities
assured me that there would be a difficult and tense period of
bargaining before we agreed on the final recommendations. This never
happened. I am not certain why. One possible reason is that the
committee judged the initiatives on the basis of scientific potential,
not political considerations.
The list of priorities was established by a gradual process that was
much easier than any of the survey committee members anticipated. The
committee voted on straw ballots on three occasions, using as background
material the preliminary reports of the advisory panels. The straw
ballots focused the discussion on projects that were most likely to be
considered important in the final deliberations. As a preliminary to the
final ballot, the committee heard advocacy presentations from the panel
chairs. The chairs also participated in discussions of the relative
merits of all the initiatives, although the final recommendations were
formulated by the survey committee in executive session.
Two strategic decisions helped the committee reach a consensus quickly
and smoothly. First, the committee decided that if we failed to reach
agreement in July 1990 at the pleasant facilities of the National
Academy, within reach of the cool breezes from the beach on Irvine,
California, then we would mect a month later in the least desirable
place in the middle of summer that we could think of, namely,
Washington, DC.
Second, several committee members proposed that I draw up, on the
evening before the final voting, a draft list of recommended initiatives
in order of priority. Their suggested that the committee alter by
consensus the draft set of recommendations in order to arrive at the
final list of priorities. The proposers hoped that, by this process, the
committee could avoid having "winners or losers." I was skeptical of the
chances for success when the idea was proposed, but I agreed to try.
Having drawn up a handwritten list of priorities on the night before our
formal voting, I was surprised the next day at how rapidly we reached a
consensus. We began with those equipment categories concerning which we
were most in agreement and then worked our way to the more difficult
choices. We went around the table, everyone stating their views about
what change, if any, needed to be made in the ordered list that we were
considering. By the time we had all spoken, the consensus was obvious
and we adopted unanimously our priorities in each category.
In preliminary discussions, most agency personnel opposed absolute
rankings that combined ground and space initiatives, worrying that their
top priorities might be adversely affected by ineffectiveness at some
other agency. The survey committee provided both separate and combined
rankings of ground and space initiatives, believing that good
citizenship required us to use our expertise to provide the maximum
possible guidance.
In times of budgetary crisis, good citizenship also requires fiscal
restraint. The survey committee studied approximately ten times as many
initiatives as were endorsed, recommending that funding agencies invest
in astronomical initiatives according to the scientific priorities
established in the survey report.
The committee assigned its highest priority for ground-based astronomy
to the revitalization of the infrastructure for research, both equipment
and people. Continuing to develop a space program with an improved
balance between large and small projects, with emphasis on quicker and
more efficient missions, was the committee's highest priority for space
research.
The committee recommended that an increased emphasis be given in the
astronomy research budget to small and moderate programs (see Table 1).
The committee did not prioritize small programs, recognizing that the
agencies could use peer review for small initiatives to respond quickly
to new scientific or technological developments.
The 180 page book presenting the recommendations was written in about
three months. National Research Council reports are reviewed carefully.
They must meet high standards of logic, of evidence, and of objectivity.
In our case, the National Research Council selected 18 formal
references, in addition to a report review committee. The reviewers were
anonymous National Academy members and other qualified scientists, in
physics, in astronomy, and in other related disciplines. The formal
review process was painful, but I answered each review comment, even
rhetorical questions, with a specific written response in order that we
could complete the review quickly. The 18 referees helped to sharpen our
arguments and to clarify our logic, but did not suggest revisions of our
priorities.
This is the fourth in a series of decade surveys by astronomers, led
by A. Whitford, J. Greenstein, and G. Field, respectively. The highest
priority initiatives in each survey were successfully undertaken,
encouraging astronomers to submerge parochial interests and focus on the
most important initiatives.
Would another committee of astronomical experts have recommended a
similar set of priorities? I think so, provided that they had also spent
a year learning about and comparing all the proposed initiatives in this
country and abroad.
These are the things that worked for us: enlisting as committee
members active research scientists eager to finish the job and get back
to their own work; recruiting an effective executive secretary;
insisting on adequate budgeting and staff support; having a logical plan
and a specific timetable for completing the report; listening to
everyone who wanted to be heard; concentrating on issues within the
committees competence, in our case, scientific priorities; having a
talented editor who could sharpen the final report; and working with a
community that believes it is better for astronomers to make imperfect
judgments about priorities for astronomy than it is to leave the
decisions to Washington administrators.
Table 1. Recommended equipment initiatives (combined ground and space) and
estimated costs.
| Initiative |
Decade
Cost
($M) |
| Large Programs |
| Space Infrared Telescope Facility (SIRTF) |
1,300 |
| Infrared-optimized 8-m telescope |
80 |
| Millimeter Array (MMA) |
115 |
| Southern 8-m telescope |
55 |
| Subtotal for large programs |
1.550 |
| Moderate Programs |
| Adaptive optics |
35 |
| Dedicated spacecraft for FUSE |
70 |
| Stratospheric Observators for Far-lnfrared Astronomy (SOFIA) |
230 |
| Delta-class Explorer acceleration |
400 |
| Optical and infrared interferometers |
45 |
| Several shared 4-m telescopes |
30 |
| Astrometric Interferomety Mission (AIM) |
250 |
| Cosmic ray telescope (Fly's Eve) |
15 |
| Large Earth-based Solar Telescope (LEST) |
15 |
| VLA extension |
32 |
| International collaborations on space instruments |
100 |
| Subtotal for moderate programs |
1,222 |
| Subtotal for small programs |
251 |
| Decade Total |
3,023 |
(The author is at the Institute for Advanced Study, Princeton, NJ 08540).
From Science, Policy Forum, Vol. 251,
March 22, 1991, pp. 1412-1413. Copyright 1991.
The Math Forum is a research and educational enterprise of the Goodwin College of Professional Studies.