BACKGROUND
The Transportation Pooled Fund (TPF) Program is a collaborative program involving state departments of transportation (DOTs), AASHTO, and the Federal Highway Administration (FHWA) that has existed for more than 45 years. The program was first defined in 1977 per Title 23 Code of Federal Regulations (CFR) § 560.3 as a FHWA administered program in coordination with state DOTs. The TPF program creates an opportunity for partners to pool their funds, subject matter expertise, and other resources to conduct high-priority research to meet a wide variety of shared transportation problems. Being able to pool funds allows participants to achieve more from a study than if they conducted a study on their own. By leveraging funds and expertise, participants develop innovative solutions with a smaller investment while extending the reach and impact of their research. Projects that previously could have been cost prohibitive become more achievable through participation in the TPF Program. By offering hundreds of active pooled fund studies, the TPF Program assists partners in finding and funding projects applicable to their agency’s needs.
The FHWA Research & Technology (R&T) evaluation program was initiated in 2013 to assess and communicate the value and effectiveness of FHWA R&T investment. The objective of the R&T evaluation program is to document the impact of the projects, demonstrate accountability to funders and policymakers, and identify lessons learned and best practices that can be applied to future projects/programs, thus completing the innovation lifecycle. The Transportation Research Board (TRB), in collaboration with the FHWA, is overseeing the TRB/FHWA Program Evaluation (TFPE) to evaluate such programs. There is a need to evaluate the TPF to establish the degree of efficiency, implementation, effectiveness, cost-effectiveness, and attribution to assess and communicate its value.
OBJECTIVE
The objective of this research is to develop and execute an evaluation plan for the FHWA’s Transportation Pooled Fund Program to answer these fundamental questions:
- Efficiency: Are the TPF Program’s activities conducted with an appropriate use of resources, such as budget and staff time (e.g., research and implementation approach, funding level)? How well is the administration of the program functioning? What is the appropriate level of involvement from the FHWA division office, particularly as it relates to time? Are there best practices from FHWA division offices that can be shared with other division offices?
- Implementation: Are the results of TPF Program research being applied? Are project areas being adopted by the users (e.g., change in culture, state of the practice)? Is the FHWA TPF providing the tools, support, and outputs that DOTs need to implement? How does the FHWA TPF consider implementation for projects? How does the collaboration between FHWA and state transportation agencies help the state agencies implement the program? How does participation on a TPF affect future implementation of a project’s results?
- Effectiveness: Is the TPF Program achieving the goals and objectives it was intended to accomplish (e.g., impact)? How satisfied are states with the TPF Program and results?
- Cost-effectiveness: Does the value or benefit of achieving the TPF Program’s goals and objectives exceed the cost of producing them (e.g., return on investment and cost-benefit ratio)? Should there be a sliding scale for all projects based on the amount of projects states are allocated? How much staff time is needed for administration, especially for larger projects?
- Attribution: How is the TPF Program helping states address their strategic goals (e.g., outcome)?
Accomplishment of the project objective will require at least the following tasks.
TASKS
Task descriptions are intended to provide a framework for conducting the research. TRB is seeking the insights of proposers on how best to achieve the research objective. Proposers are expected to describe research plans that can realistically be accomplished within the constraints of available funds and contract time. Proposals must present the proposers' current thinking in sufficient detail to demonstrate their understanding of the issues and the soundness of their approach to meeting the research objective.
Task 1. Prepare an Evaluation Scoping Report presenting the TPF Program Evaluation team’s understanding of the objective, likely measures of effectiveness, analysis activities, and budget for evaluation.
Note: The research team shall plan to submit the draft Evaluation Scoping Report for TRB and FHWA review and approval. After meeting with the TRB panel, the research team shall address review comments and submit the final Evaluation Scoping Report that will become the basis for the Evaluation Plan.
Task 2. Prepare an Evaluation Plan, describing the evaluation strategy and specific tasks to be performed, planned calendar schedule, intermediate evaluation products, and the role and activities that each member of the evaluation team will have in the evaluation plan. Include a logic model of the TPF Program. Discuss the sample size of TPF projects and case studies to be included.
Note: The research team shall plan to submit the draft Evaluation Plan for TRB and FHWA review and approval. After virtually meeting with the TRB panel, the research team shall document key points of the discussion, address review comments, and submit the final Evaluation Plan. Work on Task 3 of the project will not begin until the final Evaluation Plan has been approved and authorized by TRB.
Task 3. Execute the Evaluation Plan and prepare an Interim Report on the TPF Program Evaluation, prepared approximately midway through the Evaluation Plan, describing the evaluation team’s progress on the Evaluation Plan, difficulties encountered in conducting the work, and any preliminary assessment supported by work so far accomplished.
Note: Following a 1-month review of Interim Report by TRB and FHWA, the research team will be required to meet in person with the project panel to discuss the Interim Report and document in a memorandum of key points of the discussion and other comments received and modifications of the Evaluation Plan as may be appropriate. For budgeting purposes, the meeting venue and travel for the TRB panel members to attend the meeting will be paid separately by TRB. Work on Task 4 of the project will not begin until authorized by TRB.
Task 4. Prepare a TPF Program Evaluation Report documenting the evaluation and presenting the results. Solicit comments on the draft from TRB and FHWA, prepare a memorandum responding to comments, and revise the draft as appropriate to prepare the final Project Evaluation Report. Recommendations shall be prepared and compiled separately for presentation to the Research and Technology Coordinating Committee (RTCC) and shall not be part of the final Project Evaluation Report.
Note: Following receipt of the draft final TPF Program Evaluation Report, 3 months shall be for TRB review and comment and for research agency preparation of the final deliverables. The Final TPF Program Evaluation Report shall be completed 12 months after the project begins.
Task 5. Prepare briefing materials as appropriate and participate in an in-person meeting of the RTCC to present the final TPF Program Evaluation Report and recommendations. Discuss the evaluation and its findings. Prior to the RTCC meeting, present the materials to and participate in a virtual meeting with FHWA. The RTCC will review the final report and issue any recommendations to the FHWA the RTCC deems appropriate in the form of a letter report.
SPECIAL NOTES
A. Travel, accommodations, and meals and incidentals for travel to the panel and RTCC meetings for the contractor will be a responsibility of the selected contractor and must be budgeted within the project’s total budget of $300,000. For budgeting purposes, the meeting venues and travel for the panel members to attend meetings will be paid separately by the TRB.
B. The interim panel meeting will be held in person at the Keck Center in Washington, DC.
C. The RTCC meets twice per year in May and December. The December meeting is usually in Washington, DC, but the May meeting can be in Washington, DC, or at the Beckman Center in Irvine, California. Additional information about the RTCC is available at https://www.nationalacademies.org/our-work/research-and-technology-coordinating-committee-federal-highway-administration.
D. TRB recognizes the value of a multidisciplinary approach to research and encourages participation in the team by experts with project management professional (PMP) certification, program management experience, programmatic evaluation experience, knowledge of the FHWA TPF, and knowledge of federal programs and funding.
E. The Evaluation Scoping Report will replace the Amplified Research Plan described in the Procedural Manual for Contractors Conducting Research in the Transportation Research Board’s Cooperative Research Program.
F. FHWA Research and Technology Evaluations Program publications through 2022 are available at https://highways.dot.gov/research/publications/rtevaluations-list.
TRB has published the final reports of three TFPE evaluations so far, which are available at the links below:
G. The Information and Instructions for Preparing Proposals for the Transportation Research Board’s Cooperative Research Programs were revised in May 2023. Please take note of the new and revised text which is highlighted in yellow.
H. Proposals must be submitted as a single PDF file with a maximum file size of 10 MB. The PDF must be formatted for standard 8 ½” X 11” paper, and the entire proposal must not exceed 60 pages (according to the page count displayed in the PDF). Proposals that do not meet these requirements will be rejected. For other requirements, refer to chapter V of the instructions.
I. The Information and Instructions for Preparing Proposals for the Transportation Research Board’s Cooperative Research Programs have been modified to include a revised policy and instructions for disclosing Investigator Conflict of Interest. For more information, refer to chapter IV of the instructions. A detailed definition and examples can be found in the CRP Conflict of Interest Policy for Contractors. The proposer recommended by the project panel will be required to submit an Investigator Conflict of Interest and Disclosure Form as a prerequisite for contract negotiations.
J. Proposals will be rejected if any of the proposed research team members work for organizations represented on the project panel. The panel roster for this project can be found at https://www.mytrb.org/OnlineDirectory/Committee/Details/7038. Proposers may not contact panel members directly; this roster is provided solely for the purpose of avoiding potential conflicts of interest.
K. Proprietary Products - If any proprietary products are to be used or tested in the project, please refer to Item 6 in the Information and Instructions for Preparing Proposals.
L. Proposals are evaluated by the TRB staff and project panels consisting of individuals collectively knowledgeable in the problem area. The project panel will recommend their first choice proposal considering the following factors: (1) the proposer's demonstrated understanding of the problem; (2) the merit of the proposed research approach and experiment design; (3) the experience, qualifications, and objectivity of the research team in the same or closely related problem area; (4) the plan for ensuring application of results; (5) how the proposer approaches inclusion and diversity in the composition of their team and research approach, including participation by certified Disadvantaged Business Enterprises; and, if relevant, (6) the adequacy of the facilities. A recommendation by the project panel is not a guarantee of a contract. The National Academy of Sciences (NAS - the contracting authority for the National Academies of Sciences, Engineering, and Medicine) will conduct an internal due diligence review and risk assessment of the panel’s recommended proposal before contract negotiations continue.
Note: The proposer's approach to inclusion and diversity as well as participation by Disadvantaged Business Enterprises should be incorporated in Item 11 of the proposal.
M. Copyrights - All data, written materials, computer software, graphic and photographic images, and other information prepared under the contract and the copyrights therein shall be owned by the National Academy of Sciences. The contractor and subcontractors will be able to publish this material for non-commercial purposes, for internal use, or to further academic research or studies with permission from TRB Cooperative Research Programs. The contractor and subcontractors will not be allowed to sell the project material without prior approval by the National Academy of Sciences. By signing a contract with the National Academy of Sciences, contractors accept legal responsibility for any copyright infringement that may exist in work done for TRB. Contractors are therefore responsible for obtaining all necessary permissions for use of copyrighted material in TRB's Cooperative Research Programs publications. For guidance on TRB's policies on using copyrighted material please consult Section 5.4, "Use of Copyrighted Material," in the Procedural Manual for Contractors.
N. The text of the final deliverable is expected to be publication ready when it is submitted. It is strongly recommended that the research team include the expertise of a technical editor as early in the project timeline as possible. See Appendix F of the Procedural Manual for Contractors Conducting Research in the Transportation Research Board’s Cooperative Research Program for technical editing standards expected in final deliverables.
O. Proposals should include a task-by-task breakdown of labor hours for each staff member as shown in Figure 4 in the Information and Instructions for Preparing Proposals. Proposals also should include a breakdown of all costs (e.g., wages, indirect costs, travel, materials, and total) for each task using Figures 5 and 6 in the brochure. Please note that TRB Cooperative Research Program subawards (selected proposers are considered subawards to the National Academy of Sciences, the parent organization of TRB) must comply with 2 CFR 200 – Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards. These requirements include a provision that proposers without a "federally" Negotiated Indirect Costs Rate Agreement (NICRA) shall be subject to a maximum allowable indirect rate of 10% of Modified Total Direct Costs. Modified Total Direct Costs include all salaries and wages, applicable fringe benefits, materials and supplies, services, travel, and up to the first $25,000 of each lower tier subaward and subcontract. Modified Total Direct Costs exclude equipment, capital expenditures, charges for patient care, rental costs, tuition remission, scholarships and fellowships, participant support costs and the portion of each lower tier subaward and subcontract in excess of $25,000.
P. If the team proposes a Principal Investigator who is not an employee of the Prime Contractor, or if the Prime Contractor is proposed to conduct less than 50% of the total effort (by time or budget), then section five of the proposal should include: (1) a justification of why this approach is appropriate, and (2) a description of how the Prime Contractor will ensure adequate communication and coordination with their Subcontractors throughout the project.
Q. All budget information should be suitable for printing on 8½″ x 11″ paper. If a budget page cannot fit on a single 8½″ x 11″ page, it should be split over multiple pages. Proposers must use the Excel templates provided in the Information and Instructions for Preparing Proposals for the Transportation Research Board’s Cooperative Research Programs.
