Estimating the value of NCHRP research products is challenging. A research product can have multiple outcomes, which in turn can lead to multiple impacts. Significant time can pass between when the research product is developed, when it is put into practice, and when the impacts of that practice are realized; this affects the timing of any evaluation activities. The nature of research products is wide-ranging; some research products are geared towards improving existing practices, while others are useful for informing policy decisions. A variety of techniques may be used to measure the impacts themselves, influenced by what measures would be most useful, how easy performance data are to collect, monitor, and replicate. Finally, attributing impacts to a research product, when the research product is used as part of a portfolio of other research products, needs to be considered. Estimating the value of NCHRP research will likely require approaches that are sensitive to the context of the research and the perspective of the stakeholders trying to understand its benefits.
NCHRP has contracted a consultant to lead this research effort, that will result in the development of quantitative and qualitative methods for capturing the impacts and value of NCHRP research.
The effort will take 24 months, and will help fulfill the following objectives:
- 1. Enabling NCHRP to provide program sponsors/state DOTs with useful and actionable information about the impacts and value of NCHRP research products.
- 2. Developing guidance (including definitions, assumptions, assessment tools, and methods) to consistently measure the value and impacts of its research products over time. The guidance may also be applicable to state DOTs.
- 3. Demonstrating how the guidance can be applied to a variety of NCHRP research products—and research portfolios—to estimate, measure or assess the value of NCHRP research products.
- 4. Recommending changes to the NCHRP program and processes that will enable future assessment of value. Recommendations must consider the cost of implementation, including training on how to implement the guidance, and likelihood of success by NCHRP and state DOTs.
The guidance will be implementable by NCHRP, and state DOTs where appropriate, to periodically determine the effectiveness of NCHRP research products in (1) changing practices and/or standards, (2) informing policy decisions, (3) educating the transportation workforce, and (4) ultimately generating benefits to society. It will also be used to encourage wider implementation of NCHRP research products, and for the continuous improvement of the NCHRP research process. The guidance will complement previous NCHRP research on measuring and communicating the value of transportation research, and will also draw on other relevant research and practices, including from other sectors and disciplines.
The research will be delivered through the fulfillment of the following tasks:
- Develop an Amplified work plan detailing how the research will be structured and undertaken
- Conduct literature reviews on previous efforts on capturing impact of research, and produce a technical memo on the same
- Conduct stakeholder interviews with key personnel at state DOTs and other organizations, and produce a technical memo on the same.
- Develop guidance methods based on the literature review, stakeholder interviews, and expert panel consultation.
- Implement the guidance methods and produce an implementation plan based on the same.
- Develop training and communications materials as required to support implementation of the guidance methods
- Produce a project report that captures the process used in developing this research, and containing all of the tools and methods developed as a result of the research.