BACKGROUND

Transportation research programs conducted by state Departments of Transportation (DOTs) consist mostly of applied research projects. At the conclusion of each research project, DOT management expects answers to some important questions, among which are the following: What has changed? How will our Department and customers benefit? Have the results been implemented and to what effect? Consistent with these questions, NCHRP Synthesis 280: Seven Keys to Building a Robust Research Program, points out the importance of "rooting research in economics" and "insisting on accountability."

Research program managers need to analyze their respective programs and individual projects in order to improve these programs and justify decisions. However, there are no clear, commonly accepted methods for evaluating the performance of applied research programs and projects. In addition, research programs vary significantly in size, type, focus, direction, and nature of operations. Nevertheless, research managers share a conviction that a quality research program is defined by the ability to meet short- and long-term organizational research needs in a timely, efficient, and effective manner. Managers must maintain customer satisfaction while balancing research objectives, anticipated returns on investment, and effective application of scientific inquiry.

NCHRP Synthesis 300: Performance Measures for Research, Development, and Technology Programs, provides an excellent synopsis of performance measures for research, development, and technology programs as they exist in state DOTs. The report provides much useful information, and it highlights some opportunities for improvement. For example, "Approximately, half of the respondents indicated that they have no formal performance measures in place....a few programs have performance measures in place for virtually all functions. However, even these programs indicated needs for improvement for some or all of their performance assessments." The synthesis further states: "Overall, what many state DOTs appear to be seeking is an ability to capture, and perhaps measure against benchmarks, the value of their program." The synthesis demonstrates that many states would like to find a viable and valid approach for measuring the benefits of their individual programs.

Methods of measuring performance have not been adequately developed for the unique needs of research programs or projects. Furthermore, a systematic approach to demonstrating the value of state DOT research projects, individually or collectively on a national basis, does not exist and would be beneficial to the continued support of state DOT research programs. This research project will address these issues.

OBJECTIVES

The objectives of this research project are to develop (1) a tool box for measuring the performance of applied transportation research programs and projects and (2) a web-based system for reporting the accomplishments of individual state research activities and their aggregate value on a national level.

TASKS

The objective of the follow-on project is to provide refinements (Version 1.1) to the deliverables of NCHRP Project 20-63, “Performance Measurement Tool Box and Reporting System for Research Programs and Projects,” through beta testing of software products and efficiency reviews of computer coding and functionality.   The objective will be accomplished by conducting the following tasks.

Task 1.  Host web site.

Task 2.  Develop and administer an electronically distributed software beta test to up to 20 participants.  Beta testing will cover RPM-Web, RPM-Tools, and interaction between them.

Task 3.  Obtain independent acceptance testing of RPM-Web.

Task 4.  Obtain independent acceptance testing of RPM-Tools and the RPM-Tools interface with RPM-Web.

Task 5.  Modify RPM-Web and RPM-Tools screens, help information, instructions, and functionality programming based on suggestions and information obtained from Tasks 2, 3, and 4. Submit for review and comment before finalizing deliverables.

Product Availability: Please see https://www.trb.org/news/blurb_detail.asp?id=9122 for an electronic version of the final project report.  A follow-on project to simplify the website is shown under 20-63B.