OBJECTIVE

The objective of this project was to develop guidance for effective experimental design and data analysis for transportation research.

STATUS

NCHRP Report 727: Effective Experiment Design and Data Analysis in Transportation Research has been published.

BACKGROUND

Transportation agencies spend millions of dollars conducting research to improve their ability to plan, design, construct, maintain, and operate the transportation system. These research projects cover a broad range of topics and use approaches ranging from fully controlled laboratory experiments to field observational studies. Unfortunately, some research projects use inappropriate experimental designs or data analysis techniques thereby increasing costs and decreasing the likelihood of success.

There are many excellent university-level texts on experimental design and data analysis, but these are often not well suited to the needs of those involved in state DOT research. Principal investigators and DOT research program and project managers need practical information that focuses on common problems that DOTs face so that they can make better decisions when planning and conducting research.

NCHRP Project 20-45, "Scientific Approaches for Transportation Research," produced NCHRP CD-22 that presents valuable material for transportation researchers. It does not, however, cover experimental design.

TASKS 

Task 1.  Review available literature to identify experimental design methods and data analysis techniques usable in transportation research. Determine how state DOTs handle experimental design for contract and in-house research and to what extent the results from NCHRP Project 20-45 are in use.

Task 2. Describe basic principles that should guide experimental design and data analysis. Describe generic approaches that are well suited to transportation research projects. Develop a framework for organizing the guide.

Task 3.  Construct example cases (e.g., evaluation of anti-icing treatments, determining the variability of different methods of measuring asphalt density, evaluation of pedestrian crosswalk effectiveness) across various functional areas to illustrate the principles identified in Task 2. The range of example cases should be sufficiently broad that most people involved in state DOT research would readily understand at least one. For each of the example cases, identify experimental design and data analysis elements that are commonly used, abused, or overlooked. Describe the likely consequences of using each.

Task 4.  Within 6 months of the project's initiation, submit an interim report documenting the results of Tasks 1 through 3 and presenting a detailed outline of the guide clearly showing the underlying organizing principles. The report should describe the issues that have facilitated or hindered the implementation of the results of NCHRP Project 20-45 and describe how those issues have influenced the outline of the guide. Meet with the project panel approximately 1 month following NCHRP's receipt of the interim report to discuss the report and to obtain approval to proceed. 

Task 5.  Prepare the guide. The guide should present the principles and approaches identified in Task 2 and describe issues that project managers should consider in evaluating research proposals and interpreting research results. The example cases from Task 3 should be fully developed and include: (1) the criteria that will be used to judge the success of the research, (2) experimental designs and data analysis techniques that may be considered to satisfy those criteria, (3) the consequences of using inappropriate experimental designs and data analysis techniques, and (4) situations where expert statistical advice should be sought. The guide should include an index and an annotated bibliography at an intermediate level, concentrating on texts that are accessible to a state DOT research project manager.

Task 6.  Develop a detailed implementation plan to facilitate incorporation of the guide developed in Task 5 into practice. Present the guide and the implementation plan at a national AASHTO Research Advisory Committee meeting. 

Task 7.  Submit a final report documenting the entire research effort and including the Task 6 guide as a stand-alone appendix.