- Explanation of Material Transmitted: This revised chapter contains an updated description of the policies and procedures associated with the National Institutes of Health Evaluation Set-Aside Program.
Note: These policies and procedures only apply when funding is available for a given fiscal year.
- Filing Instructions:
Remove: NIH Manual Chapter 1965 dated 03/01/2006
Insert: NIH Manual Chapter 1965 dated 02/27/2017
PLEASE NOTE: For information on:
- Content of this chapter, contact the issuing office listed above.
- NIH Policy Manual, contact the Division of Management Support, OMA, on 301-496-4606 or enter this URL: https://oma.od.nih.gov/DMS/Pages/Manual-Chapters.aspx
This chapter describes the Evaluation Set-Aside (ESA) Program that may be used to fund program evaluations and evaluation-related activities at the National Institutes of Health (NIH). The following sections discuss the U.S. Public Health Service (PHS) authority for this program and delegation of authority to the NIH. They also describe NIH delegated authority for policies and procedures governing requests for funds, record keeping, and reporting requirements.
The policy and procedures in this chapter apply to all personnel within NIH Institutes and Centers (ICs) or Office of the Director (OD) Offices who seek to be considered for funding to implement eligible evaluation and evaluation-related projects.
Section 241(a) of the Public Health Service Act authorizes the Department of Health and Human Services (HHS) to allocate a percentage of annual appropriations for PHS programs to evaluate their effectiveness. Historically this amount was set at 1 percent of annual appropriations until Fiscal Year (FY) 2002 when the HHS began to increase the percent set-aside for this purpose. In FY 2014, Congress set the amount of the “tap” on PHS appropriations at 2.5 percent. This amounted to a total of $1.08 billion available for PHS evaluation funding in FY 2014, of which the NIH contributed $702 million. Of this amount, the NIH Evaluation Set-Aside Program received $13.9 million. In short, the PHS Act provides a funding mechanism by which PHS agencies can carry out their responsibilities to evaluate and report on program performance.
 See page 378 (General Provisions, Sec. 205) of the Consolidated Appropriations Act, 2014 (http://www.gpo.gov/fdsys/pkg/BILLS-113hr3547enr/pdf/BILLS-113hr3547enr.pdf).
In FY 2013, the Office of Program Evaluation and Performance (OPEP) implemented a new process for submitting, reviewing, and funding ESA requests. Three types of activities are eligible for funding consideration: (1) program evaluation (needs assessment, process evaluation, outcome evaluation, or feasibility study to design a process or outcome evaluation); (2) development of program performance measures; and (3) evaluation training.
In addition, at the discretion of the Evaluation Policy and Oversight Committee (EPOC) Chairperson, ESA funds may be used to support special projects that have trans-NIH significance; past examples include the NIH Peer Review Surveys, NSF-NIH Survey of Earned Doctorates, NSF-NIH Survey of Doctorate Recipients, NSF-NIH Survey of Graduate Students and Postgraduates in Science and Engineering, and the Association of American Medical Colleges Faculty Roster System.
The Secretary, HHS, identifies the amount of ESA funds available to each PHS agency annually. At the NIH, the Office of Budget (OB) within the OD, NIH, allocates the IC assessment for the ESA Program. After examining the NIH review procedures in 1991, the Assistant Secretary for Health delegated the authority to the NIH for approval of evaluation proposals requesting Evaluation Set-Aside funds.
Three separate but interdependent NIH entities are responsible for different aspects of the Evaluation Set-Aside Program: the Office of Program Evaluation and Performance (OPEP), the Evaluation Policy and Oversight Committee (EPOC), and the Technical Merit Review Committee (TMRC). Working together, they comprise a system of checks and balances that covers the entire range of tasks and responsibilities necessary to meet NIH program evaluation needs.
- OPEP, located in the Division of Program Coordination, Planning, and Strategic Initiatives (DPCPSI), OD, manages the day-to-day activities of the ESA Program. OPEP, which consists of Federal employees and contractors, has several responsibilities:
- Soliciting, receiving, reviewing, and funding requests for program evaluations and evaluation-related activities (development of program performance measures and evaluation training) from ICs as well as offices within the OD.
- Staffing and coordinating the activities of the EPOC and the TMRC.
- Maintaining the OPEP website as an online resource for information and guidance about the ESA Program.
- Maintaining official files for the ESA Program.
- Disseminating timely information on evaluation projects funded by the ESA Program. This includes posting final reports on the OPEP website and reporting findings of completed projects to the HHS and to Congress.
- Proposing policies and/or procedures to make the ESA Program more efficient, more effective, and more useful NIH-wide. Disseminating these across the NIH community via revisions to this chapter, the OPEP website, and in meetings of IC and OD Planning and Evaluation officers and staff.
- The EPOC consists of NIH senior-level personnel, such as IC Deputy Directors, IC Associate Directors, NIH Associate Directors, or OD Office Directors. The EPOC is chaired by the Director of DPCPSI, who has authority to approve requests for ESA funds. The EPOC Chairperson also appoints members to the committee in three-year terms. The EPOC is primarily responsible for:
- Assessing the potential impact or significance of proposed projects outlined in letters of intent (LOIs).
- Reviewing and setting policy governing the ESA Program, including appropriate uses of funds.
- The TMRC consists of NIH employees who have evaluation expertise and are knowledgeable about NIH programs. They are often active members of the NIH Planning and Evaluation community. The TMRC is chaired by the Director of OPEP, who has the delegated authority to approve certain funding requests. The TMRC Chairperson also appoints members to the committee in three-year terms. The TMRC is primarily responsible for:
- Assessing the technical approach of proposed projects described in applications, based on established review criteria.
- Reviewing policy and procedures and recommending policy changes to the EPOC.
Funding requests to implement a program evaluation or to develop program performance measures undergo a multi-stage review process. First, applicants submit a letter of intent (LOI) for each proposed project. LOIs for eligible projects are reviewed by the EPOC, which rate the projects’ potential impact on the IC sponsors and/or the NIH in general. Second, applicants submit an application detailing the technical approach for each proposed project. The TMRC reviews and scores applications for program evaluation, and OPEP does the same for applications for the development of program performance measures. Lastly, review scores assigned to the LOIs and applications are used to prioritize the proposed projects and to assist the EPOC Chairperson in making funding decisions.
Funding requests for evaluation training are reviewed by OPEP, and funding decisions are made by the Director of OPEP.
Detailed information about these review procedures, including review criteria, can be found in the Guidance for Requesting NIH Evaluation Set-Aside Funds, which is available on the OPEP website.
Each fiscal year, HHS identifies the amount of PHS evaluation funding NIH may use for internal evaluation activities not specified in appropriations. When a final amount has been determined, the NIH Office of Budget notifies the ICs to transfer funds to the ESA Program. Once the OD Budget Office confirms that funds are available, it authorizes OPEP to begin issuing awards for approved projects and trainings.
ESA funds are one-year funds and must be obligated within the same fiscal year the awards are issued. In the final quarter of the fiscal year, OPEP works with award recipients and the OD Budget Office to ensure that all ESA funds are obligated and that obligations have not exceeded the approved amounts.
All records pertaining to this chapter must be retained and disposed of under the authority of NIH Manual 1743,"Keeping and Destroying Records," Appendix 1, "NIH Records Control Schedules" (as amended). These records must be maintained in accordance with current NIH Records Management and Federal guidelines. Contact your IC Records Liaison or the NIH Records Officer for additional information.