Guidelines for Self-Assessment of Graduate Programs in the Biomedical Sciences
A Report of the Task Force on Benchmarks of Success in Graduate Programs AAMC GREAT Group. September 2000
Survey Instruments
Report of the Task Force on Benchmarks of
Success in Graduate Programs
AAMC GREAT Group
September 2000
ii Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
To order additional copies of this publication, please contact:
Association of American Medical Colleges
Section for Publication Orders
2450 N Street, NW
Washington, DC 20037-1134
Phone: 202-828-0416
Fax: 202-828-1123
Price: $35.00, plus shipping
(Item Code: GBSN)
AAMC Members: $25.00, plus shipping
(Item Code: GBSM)
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 iii
iiiGREAT Group Task Force on Benchmarks of
Success in Graduate Programs
Thomas Fox, Ph.D., Chair, Associate Dean for Graduate Education,
Harvard Medical School
Roger Chalkley, D.Phil., Director, Interdisciplinary Graduate Programs,
Vanderbilt University
Brenda Russell, Ph.D., Executive Associate Vice Chancellor for
Research, University of Illinois-Chicago, College of Medicine
Chair, Associate Dean for Graduate Education,Harvard Medical School
Roger Chalkley, D.Phil., Director, Interdisciplinary Graduate Programs,
Vanderbilt University
Brenda Russell, Ph.D., Executive Associate Vice Chancellor for
Research, University of Illinois-Chicago, College of Medicine
AAMC Staff
Allan C. Shipp, Assistant Vice President for Biomedical and Health
Sciences Research
Stephen J. Heinig, Senior Staff Associate, Division of Biomedical and
Health Sciences Research
This document was posted to the GREAT Group Web Site
<http://www.aamc.org/about/gre> in June 1999 as a means to solicit comments
as well as to provide electronic access to the survey instruments. For the
valuable input received, the Task Force would like to thank the GREAT
Group membership and leadership.
iv Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
Table of Contents
I. Preamble ..............................................................................................................1
..............................................................................................................1II. The History and Mission of Graduate Education in Medical Schools ................ 2
................ 2III. The Goals and Objectives of Successful Graduate Programs................................4
................................4IV. How Programs Can Work Toward Achieving Success..........................................6
..........................................6Step 1: Defining Program Goals and Objectives ..................................................6
Defining Program Goals and Objectives ..................................................6Step 2: Pursuing Activities That Promote Attainment of Goals and Objectives....6
Pursuing Activities That Promote Attainment of Goals and Objectives....6Step 3: Identifying Indicators of Success ..............................................................7
Identifying Indicators of Success ..............................................................7Step 4: Data Gathering ........................................................................................8
Data Gathering ........................................................................................8Step 5: Implementing Change ..............................................................................9
Implementing Change ..............................................................................9V. Self-Assessment Surveys......................................................................................10
......................................................................................10Surveys:
Qualitative Questionnaire for Self-Evaluation of Graduate Programs ................11
Assessment of Faculty and Ph.D. Program Characteristics ................................14
Assessment of Admissions and Graduate Student Populations by Program........18
Survey of Student Career Goals and Satisfaction................................................21
Aggregated Data on Student Career Goals and Satisfaction ..............................23
Faculty Assessment of Recent Graduates............................................................25
Tables:
Existing Sources of Data for Graduate Program Self-Assessment
Table 1. Assessment of Faculty and Program Characteristics ......................26
Table 2. Assessment of Graduate Student Populations ................................27
Table 3. Assessment of Recent Graduates ....................................................28
Other Resources ................................................................................................29
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 1
1Self-Assessment of Graduate Programs
in the Biomedical Sciences
NARRATIVE GUIDE AND COMPANION SURVEY INSTRUMENTS
I. Preamble
The Association of American Medical Colleges (AAMC) Group on Graduate Research, Education,
and Training (GREAT Group) is a professional body of approximately 300 individuals with a
responsibility for, or interest in, Ph.D. training that occurs in the medical school setting. At its
1996 and 1997 conferences, the membership of this group recognized the value of having graduate
programs periodically conduct a formal process of self-assessment to discern the extent to which
they are meeting their goals. Participants at these meetings also expressed the desire for resources
of which program directors could avail themselves when taking on the task of defining goals and
identifying the indicators that allow assessment of whether their objectives are being achieved. In
light of this, the GREAT Group membership called on its Steering Committee to appoint a task
force to develop just such a resource.
The Task Force on Benchmarks of Success in Graduate Programs was thus assembled and worked
over the following year on this narrative guide. The purpose of the guide is to outline for graduate
programs a process by which they can develop and measure their own benchmarks of “success,”
however they choose to define it. This guide is not prescriptive and thus does not impose on
programs any particular criteria or approach. Rather, it is suggestive, pointing to a wide array
of possibilities and outlining a model process that users of this guide should adapt to their own
particular circumstances and priorities.
A companion to this narrative guide is a set of survey instruments that are attached as appendices.
One is a suggested tool to allow programs to conduct qualitative assessments of their own programmatic
and educational characteristics. Other surveys, also useful as institutional assessment
tools, will have the added benefit of allowing individual programs to compare their own measures
against other programs that have used the same or comparable instruments.
It is important to note that this project is not a prelude to further accreditation of graduate
programs, most of which are already accredited by state, regional, or local accrediting bodies. Nonetheless, the AAMC and the Liaison Committee on Medical Education (LCME) do encourage self-evaluation for the purposes of maintaining the quality of Ph.D. education, and thus this document was conceived of and prepared by the GREAT Group as a resource to its own membership. Input on this guide and other aspects of this project are welcomed and may be sent to the GREAT Group Executive Secretary, AAMC, 2450 N Street, N.W., Washington, D.C. 20037.
programs, most of which are already accredited by state, regional, or local accrediting bodies.
Nonetheless, the AAMC and the Liaison Committee on Medical Education (LCME) do encourage
self-evaluation for the purposes of maintaining the quality of Ph.D. education, and thus this document
was conceived of and prepared by the GREAT Group as a resource to its own membership.
Input on this guide and other aspects of this project are welcomed and may be sent to the GREAT
Group Executive Secretary, AAMC, 2450 N Street, N.W., Washington, D.C. 20037.
, most of which are already accredited by state, regional, or local accrediting bodies.Nonetheless, the AAMC and the Liaison Committee on Medical Education (LCME) do encourage
self-evaluation for the purposes of maintaining the quality of Ph.D. education, and thus this document
was conceived of and prepared by the GREAT Group as a resource to its own membership.
Input on this guide and other aspects of this project are welcomed and may be sent to the GREAT
Group Executive Secretary, AAMC, 2450 N Street, N.W., Washington, D.C. 20037.
2 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
II. The History and Mission of Graduate
Education in Medical Schools
For almost a century, medical schools have played an important role in graduate education, a
role that was significantly expanded after World War II, when the federal government vested in
academic institutions the predominant share of responsibility for conducting the national research
effort and for training successive generations of researchers. This arrangement, largely the vision
of Vannevar Bush, Director of the Office of Scientific Research and Development under the
Roosevelt and Truman administrations, was successful because it integrated teaching and research.
This allowed the educational process to benefit from the knowledge emerging from cutting edge
scientific activity, while research benefitted from the exuberance, creativity, and effort of young
scholars eager to learn and become skilled investigators.
This paradigm has produced a highly successful system of training and has led to graduate programs
as diverse as the institutions that house them. Many programs, including some in biomedical
research, are based within schools of arts and sciences at colleges and universities, an arrangement
that integrates graduate training within the broad tradition of scholarship at these institutions and
allows undergraduates exposure to the research programs and graduate level activities of departments.
While learning from their own mentors, graduate students in turn can develop their own
didactic skills in serving as teaching assistants and instructors in baccalaureate-level courses.
Medical schools, on the other hand, are responsible for most of the graduate training that occurs
in the biomedically-related sciences, a system that has grown out of the relationship between basic
biomedical research and clinical activity. Research is an iterative process in which the findings of
fundamental scientific investigations yield knowledge leading to new, more effective clinical practices,
while clinical experience generates more questions that must be answered through basic
research. Thus, the presence of a graduate program within the medical school allows doctorate
students to appreciate the clinical relevancy of their basic science curriculum. Medical students, in
turn, benefit from the presence of an active program of research since the latest research findings
can be readily incorporated into their curricula, and they can be exposed to, and in some cases
inspired to pursue, research careers.
As a consequence of these benefits, medical school-based graduate programs have been highly
successful. Ninety-eight percent of U.S. medical schools conduct Ph.D. training. In a 1997 survey
of these institutions, 81 schools responded and identified 629 individual departments and programs
responsible for graduate education (by extrapolation, the total number may be as high as
800-900). The same survey revealed that a majority—some 60 percent—of Ph.D.s in the biomedically-
related sciences, in fact, emerge from medical school-based programs. Graduates of these
programs go on to become faculty, industrial researchers, and work in other capacities that serve
the biomedical sciences.
biomedically-related sciences, a system that has grown out of the relationship between basicbiomedical research and clinical activity. Research is an iterative process in which the findings of
fundamental scientific investigations yield knowledge leading to new, more effective clinical practices,
while clinical experience generates more questions that must be answered through basic
research. Thus, the presence of a graduate program within the medical school allows doctorate
students to appreciate the clinical relevancy of their basic science curriculum. Medical students, in
turn, benefit from the presence of an active program of research since the latest research findings
can be readily incorporated into their curricula, and they can be exposed to, and in some cases
inspired to pursue, research careers.
As a consequence of these benefits, medical school-based graduate programs have been highly
successful. Ninety-eight percent of U.S. medical schools conduct Ph.D. training. In a 1997 survey
of these institutions, 81 schools responded and identified 629 individual departments and programs
responsible for graduate education (by extrapolation, the total number may be as high as
800-900). The same survey revealed that a majority—some 60 percent—of Ph.D.s in the biomedically-
related sciences, in fact, emerge from medical school-based programs. Graduates of these
programs go on to become faculty, industrial researchers, and work in other capacities that serve
the biomedical sciences.
1 Ammons, S.W. and D.E. Kelly. “Profile of the Graduate Student Population in U.S. Medical Schools.” Academic Medicine, Vol 72, No. 9,
September 1997: pp 454-459.
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 3
3One way graduate programs ensure their success is to conduct self-assessments. This is a process
by which programs regularly revisit, and if necessary redefine, their goals while looking toward
indicators that allow them to determine the extent to which those goals are being achieved. The
concepts of programmatic and institutional diversity, as alluded to previously, are key. Each program
fills a different niche, and programs should accordingly use varying criteria for self-assessment.
Some programs may emphasize special features of the didactic experience, whereas others
may focus on exposure to specialized research approaches. Some programs excel in producing
clinical researchers, others basic researchers. Some are successful at training academics, others
industry scientists. Thus, programs must discern their primary goals with regard to these and
other objectives, and assess themselves accordingly.
This guide is intended to aid programs in that process of self-assessment. The following section
identifies goals and objectives commonly valued by graduate programs. Section IV outlines a
process by which programs can achieve success that involves goal setting, self-evaluation, and
efforts toward improvement. The process of self-assessment is discussed in Section V, and a series
of survey instruments and tables designed to facilitate the collection of benchmarking data are
provided after the narrative section of this document. The final section points readers to other
resources (e.g., articles, books, World Wide Web sites) that can shed further light on ways to
conduct self-assessments.
4 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
III. The Goals and Objectives of Successful Graduate Programs
As a consequence of the diversity discussed above, graduate programs have different missions,
goals, and objectives, and serve various student populations. This diversity lends in part to the
success of these programs collectively. Nonetheless, there are some educational goals that virtually
all programs aim to achieve. Most program directors agree that they intend to produce graduates
who are:
• Competent, skilled experimentalists,
• Problem solvers,
• Critical and independent thinkers, and
• Expert in their field, with both depth and breadth of knowledge.
In addition, programs aim to instill in individuals personal characteristics that are key to
professional success in any career, leading to scientists who are also:
• Leaders,
• Excellent communicators,
• Good mentors, nurturers, and teachers,
• Organized administrators,
• Exemplars of high ethical standards, and
• Collaborators and team players.
Finally, there are special characteristics that some programs may wish to see in their graduates,
given the program’s special mission or setting. Specific programs may wish to produce graduates
who excel in techniques particularly relevant to a given discipline, seek to conduct patient-oriented
research, or have other specific qualities, training, or goals. Moreover, various programs will seek
to serve students who are primarily from underrepresented minorities, are state or community residents,
are internationally competitive, or have other unique characteristics. Definitions of program
success should encompass these objectives as well.
Graduate deans and faculty judge their success not only by the quality and skills of the graduates
they produce, but by the attributes of the program itself. Indeed, some characteristics in particular
seem to be common to programs that are successful. In a sense, these are the prerequisites of creating
a faculty, a student body, and an environment consistent with excellence in education and
the program’s overall objectives. Thus, a successful program is generally one that:
• Has a faculty who are talented educators and capable researchers — Faculty must be
skilled in pedagogic techniques, be excellent communicators, and through research, be
current on the latest science in their discipline.
• Provides adequate resources for teaching and research — Even the most talented and
skilled faculty cannot educate effectively without the necessary instructional tools, library
facilities, and research infrastructure, to which students also need access as part of the
didactic experience.
Has a faculty who are talented educators and capable researchers — Faculty must beskilled in pedagogic techniques, be excellent communicators, and through research, be
current on the latest science in their discipline.
• Provides adequate resources for teaching and research — Even the most talented and
skilled faculty cannot educate effectively without the necessary instructional tools, library
facilities, and research infrastructure, to which students also need access as part of the
didactic experience.
Provides adequate resources for teaching and research — Even the most talented andskilled faculty cannot educate effectively without the necessary instructional tools, library
facilities, and research infrastructure, to which students also need access as part of the
didactic experience.
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 5
5• Conducts quality research programs — Research and education go hand in hand, not only
to assure currency and credibility on the part of faculty, but also to expose students to the
research process, to conduct their own investigations, and to learn from their mentors.
• Creates and fosters an intellectual environment and scientific probity — Students develop
a sense of scholarship, integrity, and professionalism in large part through the examples set
by those around them; thus fostering an environment that strives for the highest standards
in all these areas is key.
• Stimulates and promotes a quest for learning and a drive for creative inquiry — Good
scientists are both creative thinkers and life-long learners who thirst for answers to
unsolved problems and gaps in their knowledge base.
• Retains good students and minimizes unnecessary attrition — Another indicator of a program’s
success is its ability to retain the best students. Conversely, a large attrition of good
students or a high occurrence of academic performance problems is undesirable. These
characteristics may be signs that the program is not meeting in some way students’ expectations
or not properly identifying applicants with adequate prerequisite training.
• Provides effective and efficient support of students’ needs — Program administration has a
responsibility to facilitate graduate education and career development. Services must be
offered to students that permit identification of scholarships, access to constructive criticism
and feedback, contacts with potential employers and fellowships, career counseling,
and the provision of other services that benefit individuals while students or once launched
into their careers.
• Is valued by the faculty and central administration — For a program to be successful, it
must have a prominent place within the mission of the institution and enjoy political support
from the faculty and administration. This characteristic is key to accomplishment and
obtaining necessary resources.
Conducts quality research programs — Research and education go hand in hand, not onlyto assure currency and credibility on the part of faculty, but also to expose students to the
research process, to conduct their own investigations, and to learn from their mentors.
• Creates and fosters an intellectual environment and scientific probity — Students develop
a sense of scholarship, integrity, and professionalism in large part through the examples set
by those around them; thus fostering an environment that strives for the highest standards
in all these areas is key.
• Stimulates and promotes a quest for learning and a drive for creative inquiry — Good
scientists are both creative thinkers and life-long learners who thirst for answers to
unsolved problems and gaps in their knowledge base.
• Retains good students and minimizes unnecessary attrition — Another indicator of a program’s
success is its ability to retain the best students. Conversely, a large attrition of good
students or a high occurrence of academic performance problems is undesirable. These
characteristics may be signs that the program is not meeting in some way students’ expectations
or not properly identifying applicants with adequate prerequisite training.
• Provides effective and efficient support of students’ needs — Program administration has a
responsibility to facilitate graduate education and career development. Services must be
offered to students that permit identification of scholarships, access to constructive criticism
and feedback, contacts with potential employers and fellowships, career counseling,
and the provision of other services that benefit individuals while students or once launched
into their careers.
• Is valued by the faculty and central administration — For a program to be successful, it
must have a prominent place within the mission of the institution and enjoy political support
from the faculty and administration. This characteristic is key to accomplishment and
obtaining necessary resources.
Creates and fosters an intellectual environment and scientific probity — Students developa sense of scholarship, integrity, and professionalism in large part through the examples set
by those around them; thus fostering an environment that strives for the highest standards
in all these areas is key.
• Stimulates and promotes a quest for learning and a drive for creative inquiry — Good
scientists are both creative thinkers and life-long learners who thirst for answers to
unsolved problems and gaps in their knowledge base.
• Retains good students and minimizes unnecessary attrition — Another indicator of a program’s
success is its ability to retain the best students. Conversely, a large attrition of good
students or a high occurrence of academic performance problems is undesirable. These
characteristics may be signs that the program is not meeting in some way students’ expectations
or not properly identifying applicants with adequate prerequisite training.
• Provides effective and efficient support of students’ needs — Program administration has a
responsibility to facilitate graduate education and career development. Services must be
offered to students that permit identification of scholarships, access to constructive criticism
and feedback, contacts with potential employers and fellowships, career counseling,
and the provision of other services that benefit individuals while students or once launched
into their careers.
• Is valued by the faculty and central administration — For a program to be successful, it
must have a prominent place within the mission of the institution and enjoy political support
from the faculty and administration. This characteristic is key to accomplishment and
obtaining necessary resources.
Stimulates and promotes a quest for learning and a drive for creative inquiry — Goodscientists are both creative thinkers and life-long learners who thirst for answers to
unsolved problems and gaps in their knowledge base.
• Retains good students and minimizes unnecessary attrition — Another indicator of a program’s
success is its ability to retain the best students. Conversely, a large attrition of good
students or a high occurrence of academic performance problems is undesirable. These
characteristics may be signs that the program is not meeting in some way students’ expectations
or not properly identifying applicants with adequate prerequisite training.
• Provides effective and efficient support of students’ needs — Program administration has a
responsibility to facilitate graduate education and career development. Services must be
offered to students that permit identification of scholarships, access to constructive criticism
and feedback, contacts with potential employers and fellowships, career counseling,
and the provision of other services that benefit individuals while students or once launched
into their careers.
• Is valued by the faculty and central administration — For a program to be successful, it
must have a prominent place within the mission of the institution and enjoy political support
from the faculty and administration. This characteristic is key to accomplishment and
obtaining necessary resources.
Retains good students and minimizes unnecessary attrition — Another indicator of a program’ssuccess is its ability to retain the best students. Conversely, a large attrition of good
students or a high occurrence of academic performance problems is undesirable. These
characteristics may be signs that the program is not meeting in some way students’ expectations
or not properly identifying applicants with adequate prerequisite training.
• Provides effective and efficient support of students’ needs — Program administration has a
responsibility to facilitate graduate education and career development. Services must be
offered to students that permit identification of scholarships, access to constructive criticism
and feedback, contacts with potential employers and fellowships, career counseling,
and the provision of other services that benefit individuals while students or once launched
into their careers.
• Is valued by the faculty and central administration — For a program to be successful, it
must have a prominent place within the mission of the institution and enjoy political support
from the faculty and administration. This characteristic is key to accomplishment and
obtaining necessary resources.
Provides effective and efficient support of students’ needs — Program administration has aresponsibility to facilitate graduate education and career development. Services must be
offered to students that permit identification of scholarships, access to constructive criticism
and feedback, contacts with potential employers and fellowships, career counseling,
and the provision of other services that benefit individuals while students or once launched
into their careers.
• Is valued by the faculty and central administration — For a program to be successful, it
must have a prominent place within the mission of the institution and enjoy political support
from the faculty and administration. This characteristic is key to accomplishment and
obtaining necessary resources.
Is valued by the faculty and central administration — For a program to be successful, itmust have a prominent place within the mission of the institution and enjoy political support
from the faculty and administration. This characteristic is key to accomplishment and
obtaining necessary resources.
6 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
IV. How Programs Can Work Toward Achieving Success
This document is predicated on the value of self-assessment, and thus review is a key activity in
allowing programs to discern whether they are meeting their objectives and maintaining the standards
that they have set for themselves or that they must meet to remain accredited. Review is,
in effect, a circular process by which goals are defined, activities undertaken to achieve them,
outcomes observed, and measures taken to assess progress. Informed by the knowledge that
measurements allow, programs then revisit their goals and the process begins anew.
Of course, outside entities will also be eyeing the extent to which programs achieve certain goals,
including the parent institution, visiting consulting committees, NIH training grant reviewers,
accrediting bodies, and the authors of independent reputational surveys published in the popular
press to assist prospective students in the selection of programs. The presence of rigorous external
review enhances, rather than diminishes, the value of self-assessment. The process of self-assessment
will inevitably lead to improvements that will benefit the program when these external
evaluations are conducted.
What follows is a general outline of the process of self-assessment that individual programs may
choose to undertake.
Step 1: Defining program goals and objectives — Programs must first consider the goals and
objectives that they are trying to meet. Some of these may be universal in nature—those common
to the educational and research missions that all programs share—and some may be specific to the
particular niche that the program fills, as described earlier. The goals and objectives should relate
to the type of student that the program hopes to attract, the training environment the program
wishes to create, and the product the program intends to generate in terms of researcher competencies
and career paths. The desired characteristics of faculty and the training environment
should also be articulated.
Defining program goals and objectives — Programs must first consider the goals andobjectives that they are trying to meet. Some of these may be universal in nature—those common
to the educational and research missions that all programs share—and some may be specific to the
particular niche that the program fills, as described earlier. The goals and objectives should relate
to the type of student that the program hopes to attract, the training environment the program
wishes to create, and the product the program intends to generate in terms of researcher competencies
and career paths. The desired characteristics of faculty and the training environment
should also be articulated.
Step 2: Pursuing activities that promote attainment of goals and objectives — Successful programs—
those that achieve their goals and objectives—generally undertake a common set of
activities to enable their success. These include:
• Recruiting students who are a good match with its mission — This is achieved in part by
communicating effectively to prospective students about the program’s mission and characteristics
through literature and other recruitment tools. Providing data on career outcomes
of graduates can also enable prospective students to assess if their own aspirations are in
line with the achievements of the program’s graduates.
• Admitting applicants who will do well within the program — Programs use various criteria
to identify students that seem to have the preparation and personal attributes that are
required for success in graduate school. These may include GRE scores, GPAs, personal
essays, undergraduate faculty recommendations, prior research experience and so forth,
and the weight given to each criterion will vary depending on the program’s experience and
goals.
Pursuing activities that promote attainment of goals and objectives — Successful programs—those that achieve their goals and objectives—generally undertake a common set of
activities to enable their success. These include:
• Recruiting students who are a good match with its mission — This is achieved in part by
communicating effectively to prospective students about the program’s mission and characteristics
through literature and other recruitment tools. Providing data on career outcomes
of graduates can also enable prospective students to assess if their own aspirations are in
line with the achievements of the program’s graduates.
• Admitting applicants who will do well within the program — Programs use various criteria
to identify students that seem to have the preparation and personal attributes that are
required for success in graduate school. These may include GRE scores, GPAs, personal
essays, undergraduate faculty recommendations, prior research experience and so forth,
and the weight given to each criterion will vary depending on the program’s experience and
goals.
Recruiting students who are a good match with its mission — This is achieved in part bycommunicating effectively to prospective students about the program’s mission and characteristics
through literature and other recruitment tools. Providing data on career outcomes
of graduates can also enable prospective students to assess if their own aspirations are in
line with the achievements of the program’s graduates.
• Admitting applicants who will do well within the program — Programs use various criteria
to identify students that seem to have the preparation and personal attributes that are
required for success in graduate school. These may include GRE scores, GPAs, personal
essays, undergraduate faculty recommendations, prior research experience and so forth,
and the weight given to each criterion will vary depending on the program’s experience and
goals.
Admitting applicants who will do well within the program — Programs use various criteriato identify students that seem to have the preparation and personal attributes that are
required for success in graduate school. These may include GRE scores, GPAs, personal
essays, undergraduate faculty recommendations, prior research experience and so forth,
and the weight given to each criterion will vary depending on the program’s experience and
goals.
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 7
7• Providing the appropriate training — To ensure the success of enrolled students, a program
must provide a comprehensive set of training experiences that not only includes
course work and time at the laboratory bench, but that also permits opportunities to
attend seminars, to develop communication skills, to observe the professional conduct of
mentors and other professionals they may wish to emulate, and to receive appropriate levels
of supervision and oversight.
• Establishing quality control of student progress and performance — Vigilant monitoring
of student performance within the framework of programmatic expectations is necessary to
ensure that standards are maintained and to identify when intervention may be necessary
to correct deficiencies on the part of students.
• Providing guidance for careers — Once enrolled, students may change career directions as
they are exposed to and engaged by new areas of science and professional activity. Mentors
and advisors must be poised to work with students to understand the range of possibilities
within the career choices they may make and to provide practical advice on achieving their
goals.
• Recruiting effective faculty — The most effective faculty will naturally be those individuals
whose talents, research interests, educational skills, and personal values best accommodate
the program’s mission. Recruitment efforts should be structured to enhance the yield of
applicants with these characteristics. Prior success as a faculty member is key, indicators of
which may include history of research support, quality of publications, citation indices,
communication and teaching experience, positions previously held, letters of reference, and
other assessments.
• Training to improve the pedagogical process — Once having recruited the right faculty,
programs have a responsibility to assure their continued success as teachers. Assessment of
faculty pedagogical skills and offering opportunities for faculty to improve their abilities as
educators is a fundamental responsibility of any program.
Providing the appropriate training — To ensure the success of enrolled students, a programmust provide a comprehensive set of training experiences that not only includes
course work and time at the laboratory bench, but that also permits opportunities to
attend seminars, to develop communication skills, to observe the professional conduct of
mentors and other professionals they may wish to emulate, and to receive appropriate levels
of supervision and oversight.
• Establishing quality control of student progress and performance — Vigilant monitoring
of student performance within the framework of programmatic expectations is necessary to
ensure that standards are maintained and to identify when intervention may be necessary
to correct deficiencies on the part of students.
• Providing guidance for careers — Once enrolled, students may change career directions as
they are exposed to and engaged by new areas of science and professional activity. Mentors
and advisors must be poised to work with students to understand the range of possibilities
within the career choices they may make and to provide practical advice on achieving their
goals.
• Recruiting effective faculty — The most effective faculty will naturally be those individuals
whose talents, research interests, educational skills, and personal values best accommodate
the program’s mission. Recruitment efforts should be structured to enhance the yield of
applicants with these characteristics. Prior success as a faculty member is key, indicators of
which may include history of research support, quality of publications, citation indices,
communication and teaching experience, positions previously held, letters of reference, and
other assessments.
• Training to improve the pedagogical process — Once having recruited the right faculty,
programs have a responsibility to assure their continued success as teachers. Assessment of
faculty pedagogical skills and offering opportunities for faculty to improve their abilities as
educators is a fundamental responsibility of any program.
Establishing quality control of student progress and performance — Vigilant monitoringof student performance within the framework of programmatic expectations is necessary to
ensure that standards are maintained and to identify when intervention may be necessary
to correct deficiencies on the part of students.
• Providing guidance for careers — Once enrolled, students may change career directions as
they are exposed to and engaged by new areas of science and professional activity. Mentors
and advisors must be poised to work with students to understand the range of possibilities
within the career choices they may make and to provide practical advice on achieving their
goals.
• Recruiting effective faculty — The most effective faculty will naturally be those individuals
whose talents, research interests, educational skills, and personal values best accommodate
the program’s mission. Recruitment efforts should be structured to enhance the yield of
applicants with these characteristics. Prior success as a faculty member is key, indicators of
which may include history of research support, quality of publications, citation indices,
communication and teaching experience, positions previously held, letters of reference, and
other assessments.
• Training to improve the pedagogical process — Once having recruited the right faculty,
programs have a responsibility to assure their continued success as teachers. Assessment of
faculty pedagogical skills and offering opportunities for faculty to improve their abilities as
educators is a fundamental responsibility of any program.
Providing guidance for careers — Once enrolled, students may change career directions asthey are exposed to and engaged by new areas of science and professional activity. Mentors
and advisors must be poised to work with students to understand the range of possibilities
within the career choices they may make and to provide practical advice on achieving their
goals.
• Recruiting effective faculty — The most effective faculty will naturally be those individuals
whose talents, research interests, educational skills, and personal values best accommodate
the program’s mission. Recruitment efforts should be structured to enhance the yield of
applicants with these characteristics. Prior success as a faculty member is key, indicators of
which may include history of research support, quality of publications, citation indices,
communication and teaching experience, positions previously held, letters of reference, and
other assessments.
• Training to improve the pedagogical process — Once having recruited the right faculty,
programs have a responsibility to assure their continued success as teachers. Assessment of
faculty pedagogical skills and offering opportunities for faculty to improve their abilities as
educators is a fundamental responsibility of any program.
Recruiting effective faculty — The most effective faculty will naturally be those individualswhose talents, research interests, educational skills, and personal values best accommodate
the program’s mission. Recruitment efforts should be structured to enhance the yield of
applicants with these characteristics. Prior success as a faculty member is key, indicators of
which may include history of research support, quality of publications, citation indices,
communication and teaching experience, positions previously held, letters of reference, and
other assessments.
• Training to improve the pedagogical process — Once having recruited the right faculty,
programs have a responsibility to assure their continued success as teachers. Assessment of
faculty pedagogical skills and offering opportunities for faculty to improve their abilities as
educators is a fundamental responsibility of any program.
Training to improve the pedagogical process — Once having recruited the right faculty,programs have a responsibility to assure their continued success as teachers. Assessment of
faculty pedagogical skills and offering opportunities for faculty to improve their abilities as
educators is a fundamental responsibility of any program.
Step 3: Identifying indicators of success — Keeping in mind what the program is trying to achieve,
program leaders should consider what factors may serve as reasonable indicators of program success.
Many of them have been identified earlier in this guide and include quantitative measures
that reflect both the background and preparation of individuals prior to becoming affiliated with
the program (e.g., GRE scores, GPAs, etc.), as well as their achievements during and after. Other
indicators may correspond to qualitative considerations, such as student satisfaction, producing
scientists with a potential for leadership, creating a nurturing environment, and so forth. The
process of identifying both types of indicators may involve faculty and student representatives, and
as well as someone with the appropriate expertise to validate the correlation between the factors
being measured or studied and the program characteristic that is being assessed.
Identifying indicators of success — Keeping in mind what the program is trying to achieve,program leaders should consider what factors may serve as reasonable indicators of program success.
Many of them have been identified earlier in this guide and include quantitative measures
that reflect both the background and preparation of individuals prior to becoming affiliated with
the program (e.g., GRE scores, GPAs, etc.), as well as their achievements during and after. Other
indicators may correspond to qualitative considerations, such as student satisfaction, producing
scientists with a potential for leadership, creating a nurturing environment, and so forth. The
process of identifying both types of indicators may involve faculty and student representatives, and
as well as someone with the appropriate expertise to validate the correlation between the factors
being measured or studied and the program characteristic that is being assessed.
8 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
Step 4: Data gathering — Once a program has identified appropriate indicators of success, it will
then have to obtain data on and assess the relevant characteristics of students, faculty, and other
program elements as appropriate. Some indicators that programs may examine are identified in
Tables 1 through 3 at the end of this document, grouped according to the aspect of the program
being assessed. These tables focus specifically on assessment of faculty and program characteristics,
graduate student populations, and recent graduates and their career outcomes. Shown in these
tables are sources of these data within the institution as well as external sources that allow comparison
to national statistics, where available. It should be noted that these tables focus largely on
programs’ characteristics that are inherently quantifiable.
Some characteristics of successful programs are highly qualitative in nature, aspects of which can
nonetheless be assessed through quantitative means, such as the use of psychometric tools. Such
qualitative characteristics include student satisfaction, environmental characteristics, integrity
among student and faculty, and so forth. Some possible institutional mechanisms for assessing
these characteristics include:
• Surveys of students — with regard to the quality of faculty, the quality of their education,
career goals, and the program environment. These results are potentially valuable on their
own, or can be compared with similar surveys in the popular press or at other institutions.
• Surveys of faculty — with regard to their teaching objectives, the quality of the student
body, and the availability of needed resources for teaching and research.
• Self-evaluation questionnaires — these are targeted at the program directorship
and serve to make leaders conduct critical evaluations of the state of their own
programs.
• Focus groups — conducted with either students or faculty to allow for identification and
articulation of program strengths, weaknesses, and goals.
As noted earlier, programs may wish to compare data on their own characteristics against those of
other programs. In many cases, however, comparison data may not be current or readily available,
may require significant manipulation to be useable, or may not provide the desired type of information
about programs and their components. Therefore, it is hoped that the survey instruments
found later in this document will help standardize approaches that programs use in collecting this
data, facilitating the efforts of individual programs to compare their own measures against those
of other programs that are similar in mission or scope. The survey instruments developed for this
purpose are discussed in greater length under Section V of this document. Programs are also
encouraged to establish their own benchmarks or thresholds based on their prior experience and
current goals.
Data gathering — Once a program has identified appropriate indicators of success, it willthen have to obtain data on and assess the relevant characteristics of students, faculty, and other
program elements as appropriate. Some indicators that programs may examine are identified in
Tables 1 through 3 at the end of this document, grouped according to the aspect of the program
being assessed. These tables focus specifically on assessment of faculty and program characteristics,
graduate student populations, and recent graduates and their career outcomes. Shown in these
tables are sources of these data within the institution as well as external sources that allow comparison
to national statistics, where available. It should be noted that these tables focus largely on
programs’ characteristics that are inherently quantifiable.
Some characteristics of successful programs are highly qualitative in nature, aspects of which can
nonetheless be assessed through quantitative means, such as the use of psychometric tools. Such
qualitative characteristics include student satisfaction, environmental characteristics, integrity
among student and faculty, and so forth. Some possible institutional mechanisms for assessing
these characteristics include:
• Surveys of students — with regard to the quality of faculty, the quality of their education,
career goals, and the program environment. These results are potentially valuable on their
own, or can be compared with similar surveys in the popular press or at other institutions.
• Surveys of faculty — with regard to their teaching objectives, the quality of the student
body, and the availability of needed resources for teaching and research.
• Self-evaluation questionnaires — these are targeted at the program directorship
and serve to make leaders conduct critical evaluations of the state of their own
programs.
• Focus groups — conducted with either students or faculty to allow for identification and
articulation of program strengths, weaknesses, and goals.
As noted earlier, programs may wish to compare data on their own characteristics against those of
other programs. In many cases, however, comparison data may not be current or readily available,
may require significant manipulation to be useable, or may not provide the desired type of information
about programs and their components. Therefore, it is hoped that the survey instruments
found later in this document will help standardize approaches that programs use in collecting this
data, facilitating the efforts of individual programs to compare their own measures against those
of other programs that are similar in mission or scope. The survey instruments developed for this
purpose are discussed in greater length under Section V of this document. Programs are also
encouraged to establish their own benchmarks or thresholds based on their prior experience and
current goals.
Surveys of students — with regard to the quality of faculty, the quality of their education,career goals, and the program environment. These results are potentially valuable on their
own, or can be compared with similar surveys in the popular press or at other institutions.
• Surveys of faculty — with regard to their teaching objectives, the quality of the student
body, and the availability of needed resources for teaching and research.
• Self-evaluation questionnaires — these are targeted at the program directorship
and serve to make leaders conduct critical evaluations of the state of their own
programs.
• Focus groups — conducted with either students or faculty to allow for identification and
articulation of program strengths, weaknesses, and goals.
As noted earlier, programs may wish to compare data on their own characteristics against those of
other programs. In many cases, however, comparison data may not be current or readily available,
may require significant manipulation to be useable, or may not provide the desired type of information
about programs and their components. Therefore, it is hoped that the survey instruments
found later in this document will help standardize approaches that programs use in collecting this
data, facilitating the efforts of individual programs to compare their own measures against those
of other programs that are similar in mission or scope. The survey instruments developed for this
purpose are discussed in greater length under Section V of this document. Programs are also
encouraged to establish their own benchmarks or thresholds based on their prior experience and
current goals.
Surveys of faculty — with regard to their teaching objectives, the quality of the studentbody, and the availability of needed resources for teaching and research.
• Self-evaluation questionnaires — these are targeted at the program directorship
and serve to make leaders conduct critical evaluations of the state of their own
programs.
• Focus groups — conducted with either students or faculty to allow for identification and
articulation of program strengths, weaknesses, and goals.
As noted earlier, programs may wish to compare data on their own characteristics against those of
other programs. In many cases, however, comparison data may not be current or readily available,
may require significant manipulation to be useable, or may not provide the desired type of information
about programs and their components. Therefore, it is hoped that the survey instruments
found later in this document will help standardize approaches that programs use in collecting this
data, facilitating the efforts of individual programs to compare their own measures against those
of other programs that are similar in mission or scope. The survey instruments developed for this
purpose are discussed in greater length under Section V of this document. Programs are also
encouraged to establish their own benchmarks or thresholds based on their prior experience and
current goals.
Self-evaluation questionnaires — these are targeted at the program directorshipand serve to make leaders conduct critical evaluations of the state of their own
programs.
• Focus groups — conducted with either students or faculty to allow for identification and
articulation of program strengths, weaknesses, and goals.
As noted earlier, programs may wish to compare data on their own characteristics against those of
other programs. In many cases, however, comparison data may not be current or readily available,
may require significant manipulation to be useable, or may not provide the desired type of information
about programs and their components. Therefore, it is hoped that the survey instruments
found later in this document will help standardize approaches that programs use in collecting this
data, facilitating the efforts of individual programs to compare their own measures against those
of other programs that are similar in mission or scope. The survey instruments developed for this
purpose are discussed in greater length under Section V of this document. Programs are also
encouraged to establish their own benchmarks or thresholds based on their prior experience and
current goals.
Focus groups — conducted with either students or faculty to allow for identification andarticulation of program strengths, weaknesses, and goals.
As noted earlier, programs may wish to compare data on their own characteristics against those of
other programs. In many cases, however, comparison data may not be current or readily available,
may require significant manipulation to be useable, or may not provide the desired type of information
about programs and their components. Therefore, it is hoped that the survey instruments
found later in this document will help standardize approaches that programs use in collecting this
data, facilitating the efforts of individual programs to compare their own measures against those
of other programs that are similar in mission or scope. The survey instruments developed for this
purpose are discussed in greater length under Section V of this document. Programs are also
encouraged to establish their own benchmarks or thresholds based on their prior experience and
current goals.
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 9
9Step 5: Implementing change — One important objective of assessing a program’s success is to
identify those areas where success, however defined, is not being achieved and to correct the deficiencies
or to overcome the obstacles contributing to this situation. Difficulty in achieving success
may stem from any number of causes, including: inadequate financial resources, inadequate physical
resources, failure to define realistic goals, poor evaluation of applicants, inability to recruit
appropriate faculty, undesirable climate or ethics within the program, weak leadership, or many
other factors that could be cited. Thus, for assessment to yield a productive outcome, programs
must be poised to implement change to allow program success. Implementation of change is not a
simple matter and there are a number of issues that programs will have to face in the process:
• Administrative and financial resources — Implementation of curricular changes, recruitment
of new faculty, development of new pedagogical tools, and other improvements will
inevitably take resources which may not be readily at hand. Some resources will be financial
and others will involve finding the information, skills, and tools necessary for the task.
There may also be limitations to change based on what the administration of the parent
institution is willing to accept, as well as restrictions placed on programs by state and
regional accrediting boards.
• Faculty involvement — It is also a human characteristic to resist change, and faculty may
resist the implementation of new approaches unless there is a careful process of discourse,
education, participation, and acceptance. The faculty must be in a position to recognize
needed improvement and to accept the means for change, particularly since these measures
may involve significant effort on their part.
• Cross-departmental communications — Many programs are interdisciplinary, involving
several departments or even multiple institutions. Challenges related to resources, faculty
involvement, and procedural matters may be complicated by these arrangements and by
the additional administrative structures involved.
Implementing change — One important objective of assessing a program’s success is toidentify those areas where success, however defined, is not being achieved and to correct the deficiencies
or to overcome the obstacles contributing to this situation. Difficulty in achieving success
may stem from any number of causes, including: inadequate financial resources, inadequate physical
resources, failure to define realistic goals, poor evaluation of applicants, inability to recruit
appropriate faculty, undesirable climate or ethics within the program, weak leadership, or many
other factors that could be cited. Thus, for assessment to yield a productive outcome, programs
must be poised to implement change to allow program success. Implementation of change is not a
simple matter and there are a number of issues that programs will have to face in the process:
• Administrative and financial resources — Implementation of curricular changes, recruitment
of new faculty, development of new pedagogical tools, and other improvements will
inevitably take resources which may not be readily at hand. Some resources will be financial
and others will involve finding the information, skills, and tools necessary for the task.
There may also be limitations to change based on what the administration of the parent
institution is willing to accept, as well as restrictions placed on programs by state and
regional accrediting boards.
• Faculty involvement — It is also a human characteristic to resist change, and faculty may
resist the implementation of new approaches unless there is a careful process of discourse,
education, participation, and acceptance. The faculty must be in a position to recognize
needed improvement and to accept the means for change, particularly since these measures
may involve significant effort on their part.
• Cross-departmental communications — Many programs are interdisciplinary, involving
several departments or even multiple institutions. Challenges related to resources, faculty
involvement, and procedural matters may be complicated by these arrangements and by
the additional administrative structures involved.
Administrative and financial resources — Implementation of curricular changes, recruitmentof new faculty, development of new pedagogical tools, and other improvements will
inevitably take resources which may not be readily at hand. Some resources will be financial
and others will involve finding the information, skills, and tools necessary for the task.
There may also be limitations to change based on what the administration of the parent
institution is willing to accept, as well as restrictions placed on programs by state and
regional accrediting boards.
• Faculty involvement — It is also a human characteristic to resist change, and faculty may
resist the implementation of new approaches unless there is a careful process of discourse,
education, participation, and acceptance. The faculty must be in a position to recognize
needed improvement and to accept the means for change, particularly since these measures
may involve significant effort on their part.
• Cross-departmental communications — Many programs are interdisciplinary, involving
several departments or even multiple institutions. Challenges related to resources, faculty
involvement, and procedural matters may be complicated by these arrangements and by
the additional administrative structures involved.
Faculty involvement — It is also a human characteristic to resist change, and faculty mayresist the implementation of new approaches unless there is a careful process of discourse,
education, participation, and acceptance. The faculty must be in a position to recognize
needed improvement and to accept the means for change, particularly since these measures
may involve significant effort on their part.
• Cross-departmental communications — Many programs are interdisciplinary, involving
several departments or even multiple institutions. Challenges related to resources, faculty
involvement, and procedural matters may be complicated by these arrangements and by
the additional administrative structures involved.
Cross-departmental communications — Many programs are interdisciplinary, involvingseveral departments or even multiple institutions. Challenges related to resources, faculty
involvement, and procedural matters may be complicated by these arrangements and by
the additional administrative structures involved.
10 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
V. Self-Assessment Surveys
As previously noted, a companion to this narrative guide is a set of surveys to allow individual
programs to conduct their own self-evaluations and to see where they stand relative to other programs
that may have conducted the same data collection efforts. There are four elements to the
survey project. The first two surveys collect data at the institutional and program levels. The second
set of surveys collect data on the qualities, objectives and satisfaction of individual students
within the program. The results of these surveys can be compiled by individual programs and,
once aggregated, may yield generalizable information on student populations. The specific survey
topics are:
• Faculty and Ph.D. program characteristics — The faculty are the core of any program and
thus pivotal to its success. Consequently, it is worth looking at the numbers, demographics,
credentials, and research experience of faculty, in addition to the resources and environment
within which the program takes place.
• Admissions and graduate student populations — Programs should look toward the qualifications,
training, research experience, and demographics of incoming students, in addition
to their progress as students, and their retention and attrition rates.
• Student career goals and satisfaction — The student perspective is equally important, particularly
as regards their view of faculty, coursework, research environment, and how the
program enabled them to meet their career objectives. This document includes a survey for
dissemination to individual students in specific graduate programs, as well as another
instrument designed to allow reporting of aggregate data for comparison purposes.
• Recent graduates — Faculty assessments of the intellect, skills, creativity, independence,
and leadership qualities of the students they produce are important. These student characteristics
are a measure of the faculty’s success in producing excellent researchers, and are
addressed in the final survey instrument in this report.
If programs using these surveys choose to compare data, the goal should not be to assess relative
“quality” based on any single set of data. In fact, there is no reason to believe that, on any single
criterion, one value is inherently better than another. Having a high faculty-to-student ratio, for
example, is not inherently a “good” characteristic if faculty devote little time to students and
teaching. So each variable will have to be assessed in the context of a particular program’s
setting and goals.
The surveys are found in the following section of this document. Individual programs should
modify them in whatever way best meets their internal assessment needs.
Faculty and Ph.D. program characteristics — The faculty are the core of any program andthus pivotal to its success. Consequently, it is worth looking at the numbers, demographics,
credentials, and research experience of faculty, in addition to the resources and environment
within which the program takes place.
• Admissions and graduate student populations — Programs should look toward the qualifications,
training, research experience, and demographics of incoming students, in addition
to their progress as students, and their retention and attrition rates.
• Student career goals and satisfaction — The student perspective is equally important, particularly
as regards their view of faculty, coursework, research environment, and how the
program enabled them to meet their career objectives. This document includes a survey for
dissemination to individual students in specific graduate programs, as well as another
instrument designed to allow reporting of aggregate data for comparison purposes.
• Recent graduates — Faculty assessments of the intellect, skills, creativity, independence,
and leadership qualities of the students they produce are important. These student characteristics
are a measure of the faculty’s success in producing excellent researchers, and are
addressed in the final survey instrument in this report.
If programs using these surveys choose to compare data, the goal should not be to assess relative
“quality” based on any single set of data. In fact, there is no reason to believe that, on any single
criterion, one value is inherently better than another. Having a high faculty-to-student ratio, for
example, is not inherently a “good” characteristic if faculty devote little time to students and
teaching. So each variable will have to be assessed in the context of a particular program’s
setting and goals.
The surveys are found in the following section of this document. Individual programs should
modify them in whatever way best meets their internal assessment needs.
Admissions and graduate student populations — Programs should look toward the qualifications,training, research experience, and demographics of incoming students, in addition
to their progress as students, and their retention and attrition rates.
• Student career goals and satisfaction — The student perspective is equally important, particularly
as regards their view of faculty, coursework, research environment, and how the
program enabled them to meet their career objectives. This document includes a survey for
dissemination to individual students in specific graduate programs, as well as another
instrument designed to allow reporting of aggregate data for comparison purposes.
• Recent graduates — Faculty assessments of the intellect, skills, creativity, independence,
and leadership qualities of the students they produce are important. These student characteristics
are a measure of the faculty’s success in producing excellent researchers, and are
addressed in the final survey instrument in this report.
If programs using these surveys choose to compare data, the goal should not be to assess relative
“quality” based on any single set of data. In fact, there is no reason to believe that, on any single
criterion, one value is inherently better than another. Having a high faculty-to-student ratio, for
example, is not inherently a “good” characteristic if faculty devote little time to students and
teaching. So each variable will have to be assessed in the context of a particular program’s
setting and goals.
The surveys are found in the following section of this document. Individual programs should
modify them in whatever way best meets their internal assessment needs.
Student career goals and satisfaction — The student perspective is equally important, particularlyas regards their view of faculty, coursework, research environment, and how the
program enabled them to meet their career objectives. This document includes a survey for
dissemination to individual students in specific graduate programs, as well as another
instrument designed to allow reporting of aggregate data for comparison purposes.
• Recent graduates — Faculty assessments of the intellect, skills, creativity, independence,
and leadership qualities of the students they produce are important. These student characteristics
are a measure of the faculty’s success in producing excellent researchers, and are
addressed in the final survey instrument in this report.
If programs using these surveys choose to compare data, the goal should not be to assess relative
“quality” based on any single set of data. In fact, there is no reason to believe that, on any single
criterion, one value is inherently better than another. Having a high faculty-to-student ratio, for
example, is not inherently a “good” characteristic if faculty devote little time to students and
teaching. So each variable will have to be assessed in the context of a particular program’s
setting and goals.
The surveys are found in the following section of this document. Individual programs should
modify them in whatever way best meets their internal assessment needs.
Recent graduates — Faculty assessments of the intellect, skills, creativity, independence,and leadership qualities of the students they produce are important. These student characteristics
are a measure of the faculty’s success in producing excellent researchers, and are
addressed in the final survey instrument in this report.
If programs using these surveys choose to compare data, the goal should not be to assess relative
“quality” based on any single set of data. In fact, there is no reason to believe that, on any single
criterion, one value is inherently better than another. Having a high faculty-to-student ratio, for
example, is not inherently a “good” characteristic if faculty devote little time to students and
teaching. So each variable will have to be assessed in the context of a particular program’s
setting and goals.
The surveys are found in the following section of this document. Individual programs should
modify them in whatever way best meets their internal assessment needs.
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 11
11Survey QUALITATIVE QUESTIONNAIRE FOR SELF-EVALUATION OF GRADUATE PROGRAMS
QUALITATIVE QUESTIONNAIRE FOR SELF-EVALUATION OF GRADUATE PROGRAMSSurvey
Qualitative Questionnaire for Self-Evaluation
of Graduate Programs
External program review should generally be preceded by a process of self-evaluation in which the
directorship examines the goals, strengths, weaknesses, and other features of the program. This
kind of qualitative examination can be helpful in preparation for externally conducted evaluations
as well as in providing points of focus for more detailed, quantitative self-assessments that might
follow. Thus, this questionnaire is a tool for program leaders to use for their own internal purposes
in reflecting on the mission, organization, administration, and educational processes that are
characteristic of the program.
Mission and Organization
1. What are your program’s goals and objectives and how were these determined?
2. In terms of teaching and research, what are the program’s greatest strengths
and greatest current needs? Are there important areas of the field not adequately
represented in your department?
3. How do the program’s strengths and weaknesses affect the graduate curriculum
and laboratory training?
4. What is the nature of research funding in your program (e.g., what share is hard
money or soft?)?
5. How does your graduate program substantially differ from other programs in
major universities?
12 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Survey Association of American Medical Colleges
Association of American Medical CollegesEducational Process
6. What is the process by which you revise your curriculum? By whom? Do students have input?
7. When was the last time your curriculum offering and requirements at the graduate
level were reviewed? How were curriculum changes implemented?
8. Describe your admission process. How do you determine the target number of
students to recruit?
9. What faculty advising and mentoring is provided to graduate students in the first
year and dissertation stages?
10. Is there any mechanism for helping faculty become better mentors?
11. What are the teaching obligations of faculty?
12. How do your students acquire skills for professionalism, ethical standards, and career
development? Do you have a formal course on these subjects?
13. If you could change one thing about your graduate program, what would it be if (a) no more
resources were available, and (b) if you had new funds to achieve significant change?
14. What is your student disciplinary mechanism? What are the mechanisms for dealing with
violations of academic and research integrity? How is the student represented in the process?
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 13
1315. Do your students have formal channels of communication with the faculty? Is there a
student organization?
16. What opportunities do students have to teach or to conduct lectures? To what degree
are students supported on teaching assistantships?
17. Do you have a student exit interview? If so, how does it influence your program?
18. Do you track your graduates in their careers? How do you track them?
Program administration
19. What kinds of resources and assistance does the program provide to faculty in
carrying out their teaching and research responsibilities?
20. How does the program, department, or institution reward faculty for effort applied
to teaching?
21. How does the program assist faculty in obtaining external support for their research?
22. In what other ways does the program administration contribute to the success of its
faculty and students?
Survey
14 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
Assessment of Faculty and Ph.D. Programs
The faculty are the core of any program and thus pivotal to its success. Consequently, it is worth
looking at the numbers, demographics, credentials, and research experience of faculty, in addition
to the resources and environment within which the program takes place. Graduate deans and
others may thus wish to use this questionnaire as a tool to develop data on and assess those
characteristics.
Respondent Information:
__________________________________________________________________________________
Name of individual providing this data
__________________________________________________________________________________
Title of the person named above
__________________________________________________________________________________
Name of institution to which data pertain
__________________________________________________________________________________
Address
__________________________________________________________________________________
Telephone Fax
__________________________________________________________________________________
E-mail address
Survey
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 15
15Survey ASSESSMENT OF FACULTY AND PH.D. PROGRAMS
ASSESSMENT OF FACULTY AND PH.D. PROGRAMSSurvey
I. Characteristics of Ph.D.-awarding programs
at your medical school
A. How many programs do you have: _______
B. How many programs are interdisciplinary:
___________
C. Do you have an M.D./Ph.D. program?
_____Yes _____No
II. Faculty characteristics
A. Faculty to student ratios
1. a. How many graduate students were enrolled
in your medical school’s Ph.D. programs as
of October of last year:_______
b. How many first-year graduate students
participated in your Ph.D. programs as of
October of last year:_______
2. a. How many faculty participated in your
graduate programs through teaching, advising,
and committees as of October of last
year:_______
b. How many basic science faculty participated
in your graduate programs through teaching,
advising, and committees as of October of
last year:_______
3. Estimate the number of the faculty cited in
question II.A.2.a. that were eligible to be thesis
advisors to students in your graduate programs:
_______
4. What percentage of the faculty in question
II.A.3. have the following number of students in
their labs:
1 _____%
2 _____%
3 _____%
4 _____%
5 _____%
>5 _____%
_______
100%
B. Graduate program faculty characteristics
1. Percent of graduate program faculty who have
a primary appointment in a
Basic science department _____%
Clinical science department _____%
Other _____%
______
100%
2. Number and percentage of underrepresented
minorities among all graduate program faculty:
_____ individuals _____%
3. Number and percentage of women among
all graduate program faculty:
_____ individuals _____%
4. Indicators of national and international
leadership in the scientific community:
Number of graduate program faculty in
October of last year who were:
_____ Officers of scientific societies
_____ Member of honorary societies
(e.g., NAS) and/or recipients of
competitive fellowships (e.g.,
AAAS fellow)
_____ Scientific review body
(e.g., NIH study sections)
_____ Editorial boards
_____ International honors
5. Over the last five years, has the number of
your graduate faculty:
______ risen
______ declined
______ stayed the same
(continued on next page)
16 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Survey Association of American Medical Colleges
Association of American Medical CollegesC. Faculty research
Number of faculty involved in graduate education
(from question II.A.2.a.) who had a grant in the
previous 12 months from:
1. National Institutes of Health ________
2. National Science Foundation ________
3. Other federal sources ________
4. Industrial sources (pharmaceutical, ________
biotechnology, etc.)
5. HHMI ________
6. Other private foundations, ________
voluntary health organizations,
(e.g., American Heart Association)
7. Other sources of external ________
research support
How many peer-reviewed articles were published
in the previous 12 months that were authored
by faculty affiliated with your graduate
programs? _________
III. Characteristics of your graduate programs
A. Training support
1. National Research Service Award (NRSA)
a. How many NRSA training grants do you
have in all your graduate programs combined?
_____
b. How many NRSA-supported full time training
positions (FTTPs) do you have in all
graduate programs combined? _____
c. Do you have an MSTP grant?
_____ Yes _____ No
d. How many FTTPs do your graduate programs
support with MSTP funds?_____
e. How many FTTPs are supported by NRSA
fellowships? _____
2. All sources of support
a. Pooling all funds for stipend support at your
entire medical school (including NRSA),
what percent comes from:
The NRSA program _____%
Research grant assistantships _____%
NSF fellowships _____%
Other federal sources _____%
Teaching assistantships _____%
Institutional sources (non-TA) _____%
Self (trainee) support _____%
Industrial support _____%
Foreign support _____%
Other non-profit (i.e., foundations _____%
and voluntary health organizations
such as HHMI, AHA, etc.)
B. Other characteristics
1. a. Stipends: Do the graduate programs at your
medical school offer a minimum stipend for
all students?
Yes _____ No _____
b. If yes, what is the amount? ______
2. Do you offer tuition remission?
Yes _____ No _____
(continued on next page)
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 17
17Survey
3. Benefits
Health Maternity Dental Other
a. Do graduate students at your medical school have access
to the following benefits (check all that apply):
b. For graduate students, does the institution contribute to
premiums for the following benefits (check all that apply):
c. Do dependents of graduate students at your medical
school have access to the following benefits (check all that apply):
d. For dependents, does the school contribute to premiums
for the following benefits (check all that apply):
4. Are graduate programs at your medical school accredited?
Yes_____ No _____
If so, by what body (bodies)
_______________________________________
_______________________________________
_______________________________________
18 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Survey Association of American Medical Colleges
Association of American Medical CollegesAssessment of Admissions and Graduate Student
Populations by Program
This questionnaire should be disseminated to each of the graduate doctorate-granting programs
within your medical school to assess the qualifications, training, research experience, and
demographics of incoming students, in addition to their retention and attrition rates.
each of the graduate doctorate-granting programswithin your medical school to assess the qualifications, training, research experience, and
demographics of incoming students, in addition to their retention and attrition rates.
Respondent Information:
__________________________________________________________________________________
Name of department or program to which this survey applies
__________________________________________________________________________________
Name of person responsible for this survey on behalf of the medical school
__________________________________________________________________________________
Address
__________________________________________________________________________________
Address continued
__________________________________________________________________________________
Telephone Fax
__________________________________________________________________________________
E-mail address
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 19
19Survey ASSESSMENT OF ADMISSIONS AND GRADUATE STUDENT POPULATIONS BY PROGRAM
ASSESSMENT OF ADMISSIONS AND GRADUATE STUDENT POPULATIONS BY PROGRAMSurvey
I. Incoming Student Populations
For students who matriculated into the program named above in the fall of the most recent year
for which data are available, please provide the following data:
A. GRE scores (for the fall of the most recent year)
Are GRE scores required as a criterion for determining admissions into this program?
______Yes ______ No
If yes, please provide the following data: Ph.D. only M.D./Ph.D.
1. Verbal: Median1 score for matriculating class: ______ ______
2. Verbal: Minimum score for matriculating class: ______ ______
3. Quantitative: Median score for matriculating class: ______ ______
4. Quantitative: Minimum score for matriculating class: ______ ______
5. Analytical: Median score for matriculating class: ______ ______
6. Analytical: Minimum score for matriculating class: ______ ______
B. Undergraduate GPAs (for the fall of the most recent year)
1. Mean GPA in science-related courses for matriculating class: ______ ______
C. Diversity (as of the fall of the most recent year)
1. Total number of matriculating students ______ ______
2. Percentage of women in matriculating class: ______ % ______ %
3. Percentage of underrepresented minorities: ______ % ______ %
4. Percentage of students who are US citizens or permanent residents: ______ % ______ %
D. Acceptance of offers
1. Number of completed applications to the program: ____________
2. Approximate percent of applicants who received offers of admission in
the fall of the most recent year (not including M.D./Ph.D. students): ______ % ______ %
3. Approximate percent of those offered admission who actually
matriculated in the fall of the most recent year (not including
M.D./Ph.D. students): ______ % ______ %
1 score for matriculating class: ______ ______2. Verbal: Minimum score for matriculating class: ______ ______
3. Quantitative: Median score for matriculating class: ______ ______
4. Quantitative: Minimum score for matriculating class: ______ ______
5. Analytical: Median score for matriculating class: ______ ______
6. Analytical: Minimum score for matriculating class: ______ ______
B. Undergraduate GPAs (for the fall of the most recent year)
1. Mean GPA in science-related courses for matriculating class: ______ ______
C. Diversity (as of the fall of the most recent year)
1. Total number of matriculating students ______ ______
2. Percentage of women in matriculating class: ______ % ______ %
3. Percentage of underrepresented minorities: ______ % ______ %
4. Percentage of students who are US citizens or permanent residents: ______ % ______ %
D. Acceptance of offers
1. Number of completed applications to the program: ____________
2. Approximate percent of applicants who received offers of admission in
the fall of the most recent year (not including M.D./Ph.D. students): ______ % ______ %
3. Approximate percent of those offered admission who actually
matriculated in the fall of the most recent year (not including
M.D./Ph.D. students): ______ % ______ %
II. Development as a Scientist
For students who graduated from your program in the spring of the most recent year for which
data are available, please provide the following data:
A. Time to degree (defined as years [express with one decimal point] registered full-time
in graduate school until completion of thesis, subtracting months on leave or in medical school):
1. Mean time to degree: ____.__ years
B. Admission to candidacy2
21. Do all programs at your institution admit to candidacy in the same way? _____ Yes ____ No
1 Median may best reflect the distribution of scores; other calculations may be used as well, such as mean or ranges. The intent is not to place excessive
emphasis on standardized testing scores, which can have limited predictive value.
2 Admission to candidacy means successful completion of all academic requirements preceding thesis and defense.
20 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Survey Association of American Medical Colleges
Association of American Medical Colleges2. In your program, is admission to candidacy predicated on (check all that apply):
_____ satisfactory performance in required courses?
_____ demonstration of cumulative knowledge?
_____ preparation of an original research proposal?
3. a. Does your institution impose time limits on reaching candidacy status: _____ Yes ____ No
b. If yes, within what number of years must candidates reach this status? _____ Years
C. Degrees conferred
1. Number of Ph.D. degrees conferred in the previous academic year
(summed over 12 months): ______
2. Median age of these students (in question C.1) at receipt of Ph.D.: ______
III. Attrition Ph.D. only M.D./Ph.D.
A. 1. Percent of students who left the program in the past year without completing
all degree requirements:3
Ph.D. only M.D./Ph.D.A. 1. Percent of students who left the program in the past year without completing
all degree requirements:3
3______% ______%
2. Percent of students who left in the past five (5) years without completing all degree requirements:4
4______% ______%
B. Reasons given for leaving the program (per III.A.2.):
Male Female Male Female
career changes:5 ____% ____% ____% ____%
relocation: ____% ____% ____% ____%
poor performance: ____% ____% ____% ____%
other: ____% ____% ____% ____%
_________________________________________________________________
total 100% 100% 100% 100%
5 ____% ____% ____% ____%relocation: ____% ____% ____% ____%
poor performance: ____% ____% ____% ____%
other: ____% ____% ____% ____%
_________________________________________________________________
total 100% 100% 100% 100%
IV. Other Aspects of the Graduate Student Experience
A. Do students receive counseling from an advisor in first year? ____Yes ____No
B. How often at a minimum must graduate students meet with
the thesis committee? _____times per year
C. What percent of the students take an ethics curriculum: ____%
D. Do you offer a course on career/survival skills? ____Yes ____No
3 Calculate as number of students who left in the last 12 months divided by all students matriculated in the program.
Also include students on leave for greater than two years as of this date.
4 Calculate as number of students who left in past five (5) years divided by the number matriculating in the same five-year period.
5 Including decisions to switch to a different degree program or moving from a dual to a single degree program.
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 21
21Survey of Student Career Goals and Satisfaction
This survey should be completed by individual students within each graduate program at the
medical school to assess their view of the faculty, coursework, research environment, and how
the program enabled them to meet their career objectives.
Respondent Information:
Today’s date: ____________________________________
Name of student completing questionnaire (optional): _____________________________________________________________________________
Name of program in which you are/were enrolled: ________________________________________________________________________________
Number of years you have been in the program: ______________ Years
Have you graduated? _______ Yes _______ No
1. Do you plan on pursuing post-doctoral training: ____Yes ____ No
2. I ultimately plan on a career in (check all that apply):
___ academia (teaching) ___ academia (research) ___ industry
___ other (please indicate: __________________) ___ don’t know
3. A. In selecting my next professional position the most important considerations
will be (rank order from 1 to 4):
1. ____ professional development and satisfaction
2. ____ quality of life and family issues
3. ____ geographic location
4. ____ salary level
B. With regard to my long-term career goals, the most important considerations
are (rank order from 1 to 4):
1. ____ professional development and satisfaction
2. ____ quality of life and family issues
3. ____ geographic location
4. ____ salary level
Survey SURVEY OF STUDENT CAREER GOALS AND SATISFACTION
SURVEY OF STUDENT CAREER GOALS AND SATISFACTIONSurvey
22 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
4. How would you rate the following characteristics of your graduate experience
(1=excellent, 5=very poor):
Scale (circle one)
Excellent Very Poor
a. Level, breadth, and content of courses 1 2 3 4 5
b. Quality of instruction 1 2 3 4 5
c. Mentoring/career guidance 1 2 3 4 5
d. Research environment 1 2 3 4 5
e. Program activities (e.g., seminars) 1 2 3 4 5
f. Interactions with fellow students 1 2 3 4 5
g. Thesis advisory committee 1 2 3 4 5
h. Program administration 1 2 3 4 5
I. Mechanism for addressing grievances 1 2 3 4 5
j. Overall level of satisfaction 1 2 3 4 5
5. Comments:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
Survey
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 23
23Aggregated Data on Student Career
Goals and Satisfaction
This instrument can be used to aggregate the data from the Survey of Student Career Goals and
Satisfaction, which will allow for analyses of program-wide data and comparisons with other
programs that have used this instrument.
Respondent Information:
__________________________________________________________________________________
Name of person providing these aggregated, program-level data
__________________________________________________________________________________
Title of the person named above
__________________________________________________________________________________
Name of program to which data pertain
__________________________________________________________________________________
Address
__________________________________________________________________________________
Address continued
__________________________________________________________________________________
Telephone Fax
__________________________________________________________________________________
E-mail address
Survey AGGREGATED DATA ON STUDENT CAREER GOALS AND SATISFACTION
AGGREGATED DATA ON STUDENT CAREER GOALS AND SATISFACTIONSurvey
24 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
Aggregated Responses to Survey on Student Career Goals and Satisfaction
1st and 2nd 3rd and 4th Students in the 5th
Year Students Year Students Year and Beyond
1. Percent of students planning on
postdoctoral training _____% _____% _____%
2. Percent of students planning a career in:
A. academia (teaching) _____% _____% _____%
B. academia (research) _____% _____% _____%
C. industry _____% _____% _____%
D. other _____% _____% _____%
E. don’t know _____% _____% _____%
3. A. Average ranking (to tenth decimal;
e.g., 2.6) of considerations for
respondent’s next professional position.
1. professional development and satisfaction _______ _______ _______
2. quality of life and family issues _______ _______ _______
3. geographic location _______ _______ _______
4. salary level _______ _______ _______
B. Average ranking (to tenth decimal;
e.g., 2.6) of considerations for
respondent’s long-term career goals
1. professional development and satisfaction _______ _______ _______
2. quality of life and family issues _______ _______ _______
3. geographic location _______ _______ _______
4. salary level _______ _______ _______
Survey
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 25
25Survey FACULTY ASSESSMENT OF RECENT GRADUATES
FACULTY ASSESSMENT OF RECENT GRADUATESSurvey
Faculty Assessment of Recent Graduates
Faculty assessments of the intellect, skills, creativity, independence, and leadership qualities of the
students they produce are important. These student characteristics are a measure of the faculty’s
success in producing excellent researchers, and are addressed in this survey instrument.
Respondent Information:
Name of faculty member completing questionnaire: _______________________________________________________________________________
Name of student being evaluated: ____________________________________________________________________________________________
Name of program with which student affiliated: __________________________________________________________________________________
______________________________________________________________________________________________________________________
Please use a separate form for each student who graduated and rate him or her on a
scale of 1 = excellent, 3 = satisfactory, to 5 = very poor.
How would you rate this student with regard to the following characteristics:
Scale (circle one)
Excellent Very Poor
1. Graduate coursework relative to other students? 1 2 3 4 5
2. Commitment, drive, determination, and perseverance? 1 2 3 4 5
3. Creativity and imagination, in terms of experimental 1 2 3 4 5
interpretation as well as design?
4. Technical abilities? 1 2 3 4 5
5. Keeping up with and understanding the literature? 1 2 3 4 5
6. Output and effectively completing tasks (e.g., 1 2 3 4 5
translating observations into a presentable paper)?
7. Ability to write clearly and persuasively? 1 2 3 4 5
8. Ability to speak clearly and persuasively? 1 2 3 4 5
9. Leadership qualities (in the lab and/or program)? 1 2 3 4 5
10. Being well organized (good record keeping and 1 2 3 4 5
well prepared lab notebooks) and effective time
management?
11. Potential for independence (i.e., are they on a 1 2 3 4 5
successful career trajectory)?
12. Overall assessment as a productive scientist? 1 2 3 4 5
26 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Survey Association of American Medical Colleges
Association of American Medical CollegesExisting Sources of Data for Graduate
Program Self-Assessment
Pending conduct of the surveys, programs may wish to turn to existing sources of data, both internal
and external, to benchmark their success in achieving educational and programmatic goals.
The following tables present various characteristics that programs may wish to examine relative to
their faculty, graduate student populations, and career outcomes of recent graduates.
Table 1
ASSESSMENT OF FACULTY AND PROGRAM CHARACTERISTICS
SSESSMENT OF FACULTY AND PROGRAM CHARACTERISTICSData Internal Sources of Data Sources of Comparison Data
Publication rates of faculty Faculty evaluations PubMed/MEDLINE
Citation statistics regarding Faculty evaluations ISI Citation Index,
faculty research NRC Research Doctorate
Programs in the United States: Continuity and Change
Programs in the United
States: Continuity and Change
Grant portfolio of faculty Sponsored projects office NIH Web Site, WebCASPAR
Cooperative research with Sponsored projects office
industry
Awards and honors to Department office Honorific societies and
program faculty academies
Faculty/student ratios Program office Peterson’s Graduate Programs in
the Biological Sciences
the Biological Sciences
Demographic characteristics Program office
of faculty
Publication rates of faculty Faculty evaluations PubMed/MEDLINE
Types and amount of training Program office NIH, NSF
support (NRSA, TAs,
RAs, etc.)
Accreditation Program office Peterson’s Graduate Programs in
the Biological Sciences
the Biological Sciences
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 27
27Table 2
ASSESSMENT OF GRADUATE STUDENT POPULATIONS
SSESSMENT OF GRADUATE STUDENT POPULATIONSData Internal Sources of Data Sources of Comparison Data
GRE scores Dean of Graduate School Educational Testing Service
Department Office
Undergraduate GPAs Student applications Peterson’s Graduate Program in
the Biological Sciences
the Biological Sciences
Honorary scholarships Fiscal officer, department
Proportion of underrepresented Student applications NRC Research-Doctorate
Research-Doctorateminorities and women Programs in the United
States: Continuity and Change
States: Continuity and Change
Length of prior research Student surveys Not available
experience
Time to degree Registrar’s office NRC Research-Doctorate
Programs in the United States: Continuity and Change
Programs in the United
States: Continuity and Change
Retention/Attrition Department Peterson’s Graduate Programs in
the Biological Sciences
the Biological Sciences
First year GPAs Registrar’s office
Graduate GPAs Registrar’s office
“Admission to Candidacy” Program office
standing
Awards, fellowships, and Program office NRC Research-Doctorate
Research-Doctoratehonors received by Programs in the United
Programs in the Unitedstudents States: Continuity and Change
States: Continuity and ChangeShare of training grants from Registrar’s office NRC Research-Doctorate
Research-Doctorateexternal scholarships Programs in the United
States: Continuity and Change
States: Continuity and Change
Student publication and Program office NRC Research-Doctorate
Research-Doctoratecitation rates Programs in the United
States: Continuity and Change
States: Continuity and Change
Number of degrees conferred Program officer NRC Research-Doctorate
Programs in the United States: Continuity and Change
Programs in the United
States: Continuity and Change
28 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
Table 3
ASSESSMENT OF RECENT GRADUATES
SSESSMENT OF RECENT GRADUATESData Internal Sources of Data Sources of Comparison Data
Post-degree employment Development office Commission on Professionals
and placement statistics Program office in Science and Engineering
Survey of Doctorate
Recipients
Amount of external research Federal agencies
support
Involvement of graduates in Federal agencies,
academies, panels national committees, panels,
policy bodies
Receipt of special honors by Not available
graduates
Publication and citation ISI Citation Index
statistics of graduates
Career status well beyond Development office, Survey of Doctorate
graduation (e.g., 5 or 10 graduate survey Recipients
years out)
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 29
29Other Resources
The literature and the Internet contain a number of resources that graduate programs may find
useful as they embark on the task of assessing their own success. Some of the more notable works
are cited below:
ARTICLES
RTICLESHolzemer, W.L. “Doctoral Education in Nursing: An Assessment of Quality, 1979-1984.”
Nursing Research. Vol. 36, No. 2. March/April 1987.
Vol. 36, No. 2. March/April 1987.This article demonstrates how changes in the quality of graduate education in a particular
discipline (in this case, nursing) can be assessed through questionnaires that capture data on
quality-related program characteristics. Eighteen doctoral programs in nursing participated in
1979 and again in 1984, providing data on faculty training and accomplishments, student ability
and performance, resources, academic and social environments, program processes, and alumni
achievements. Readers may cull from this paper ideas about quantifiable characteristics of their
own programs to assess.
Kassebaum, D.G. “The Measurement of Outcomes in the Assessment of Educational
Program Effectiveness.” Academic Medicine. Vol. 65, No. 5. May 1990.
Academic Medicine. Vol. 65, No. 5. May 1990.Although the context of this paper is medical education, rather than Ph.D. training, a number
of parallels can be drawn with regard to assessment activities. The author explains why outcome
measures have become an area of increasing emphasis in educational assessment, illustrates how they
have been used, and provides a model linking various educational goals to quantifiable indicators.
BOOKS AND REPORTS
OOKS AND REPORTSAssociation of American Universities Graduate Education Committee. Graduate Education
Report. Association of American Universities: Washington, D.C. 1998.
Report. Association of American Universities: Washington, D.C. 1998.
. Association of American Universities: Washington, D.C. 1998.This report, which describes the state of graduate education at AAU member institutions, includes
a chapter on “best practices,” which is relevant to the task of self-assessment and program
improvement. The chapter includes recommendations relevant to recruitment, admissions, graduate
curriculum, mentoring, program evaluation, and other topics.
Commission on Professionals in Science and Technology. Road Map for Conducting
Employment Surveys of Doctoral Graduates in S&E. CPST: Washington, D.C. 1998.
Employment Surveys of Doctoral Graduates in S&E. CPST: Washington, D.C. 1998.
. CPST: Washington, D.C. 1998.This publication is intended to assist science and engineering professional societies and others in
conducting surveys concerning the employment of recent doctoral graduates. It discusses survey
design, data collection and management, and reporting. It may be of use to graduate programs
that wish to conduct examinations of the career outcomes of their own graduates.
30 Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000Association of American Medical Colleges
Council of Graduate Schools Task Force on Academic Review of Graduate Programs.
Academic Review of Graduate Programs. Council of Graduate Schools: Washington, D.C. 1990.
Academic Review of Graduate Programs.Academic Review of Graduate Programs. Council of Graduate Schools: Washington, D.C. 1990.
This guide takes the reader through the process of program review. It is applicable to graduate
programs in all disciplines and offers much useful and practical advice.
National Research Council. Research Doctorate Programs in the United States: Continuity
and Change. National Academy Press: Washington, D.C. 1995.
and Change. National Academy Press: Washington, D.C. 1995.
. National Academy Press: Washington, D.C. 1995.This very thick volume provides a wealth of data on virtually all research doctorate programs in
the country. Representative data elements include demographic characteristic of students, average
time to degree, number of degrees conferred, citation statistics of program faculty, and so on. The
1995 report includes data collected in 1993.
National Research Council. Summary Report 1996: Doctorate Recipients from United States
Universities. National Academy Press: Washington, D.C. 1996.
Universities. National Academy Press: Washington, D.C. 1996.
. National Academy Press: Washington, D.C. 1996.Data on recipients of research doctorates awarded by U.S. universities from July 1, 1995 through
June 30, 1996 may be found in this report, which derives from the 1996 Survey of Earned
Doctorates. It contains dozens of tables with statistics on the demographics, disciplines of study,
sources of support and other characteristics of new doctorates.
Peterson’s Graduate Programs in the Biological Sciences. Peterson’s: Princeton, New Jersey. 1998.
Graduate Programs in the Biological Sciences. Peterson’s: Princeton, New Jersey. 1998.This publication, largely oriented to students exploring graduate school possibilities, provides a
narrative description of each program that includes a few useful statistics as well, such as the numbers
and demographic characteristics of faculty and students, average age of students, numbers of
applicants and matriculants, minimum entrance exam requirements, and so forth.
University of California Council of Graduate Deans. Excellence at Risk: The Future of
Graduate Academic Education in the University of California. University of California: Oakland, California. 1997.
Graduate Academic Education in the University of California. University of California: Oakland,
California. 1997.
. University of California: Oakland,California. 1997.
The subject of this document is how the University of California system should build and sustain
the highest quality graduate programs in the face of the many challenges confronting them. The
recommendations may be applicable to other systems and include increasing academic specialization;
rigorously measuring quality, productivity, and student outcomes; collecting and using data
to inform decision-making; and attaining student body diversity.
Association of American Medical Colleges
Self-Assessment of Graduate Programs in the Biomedical Sciences — 2000 31
31INTERNET SITES
NTERNET SITESAssociation of Graduate Schools (AGS)
The AGS is a body of the Association of American Universities (AAU) that brings together its
graduate school deans. Its Web site includes data from a recent survey conducted to assess the
steps that its membership has taken to modify graduate programs, including such issues as timeto-
degree, program size, interdisciplinary programs, and career advising and placement. Access:
http://www-ags.ucsd.edu/ags.html
NIH Grants Page
Data on individual grantees and principal investigators can be obtained at this site, and basic
reports produced. Access: http://grants.nih.gov/grants/award/award.htm
NSF Division of Science Resources
NSF Division of Science Resources
This site allows access to tremendous amounts of data, though prepared data briefs and Web
CASPAR, from the Survey of Earned Doctorates, the Survey of Doctorate Recipients, and the
Survey of Graduate Students and Postdoctorate Fellows in Science and Engineering. Also, federal
grant data by institutions can be obtained and multiple cross-tabulations performed. Access:
http://www.nsf.gov/sbe/srs/stats.htm
Peterson’s Education Center, Graduate and Professional Study
This page offers information on graduate programs across the U.S. as well as background
information on graduate education issues. Access: http://www.petersons.com/graduate
PhDs.Org
PhDs.Org
This site, which has a tremendous amount of information and links related to employment and
careers for Ph.D.s, has a page specifically dedicated to ranking programs. The unique feature of
this page is that it allows users to rank various characteristics of programs according to their
importance, and then, drawing from data contained in the NRC publication Research-Doctorate
Programs in the United States: Continuity and Change, the page orders programs according to the degree to which they fit the user’s stated priorities. Access: http://www.phds.org/ratings
Programs in the United States: Continuity and Change, the page orders programs according to
the degree to which they fit the user’s stated priorities. Access: http://www.phds.org/ratings
, the page orders programs according tothe degree to which they fit the user’s stated priorities. Access: http://www.phds.org/ratings
http://www.phds.org/ratings