Panel Peer Review of PHMSA Pipeline Safety Research Projects: 2015
R&D Menu
The Pipeline and Hazardous Materials Safety Administration’s (PHMSA) Pipeline Safety Research and Development (R&D) Program has held annual structured peer reviews of active research projects since 2006 in accordance with mandates by the Office of Management and Budget (OMB) and the Office of the Secretary of Transportation (OST) to maintain research data quality. PHMSA holds these reviews virtually via teleconference and the Internet, saving time and resources. This execution is also working well with panelists, researchers, Agreement Officers’ Technical Representatives, and project co-sponsors. Most impressively, the PHMSA approach facilitates attendance from all U.S. time zones, Canada, and Europe.
The annual peer review continues to build on an already strong and systematic evaluation process developed by PHMSA’s Pipeline Safety R&D Program and certified by the Government Accountability Office. The 2015 peer review panel consisted of primarily retired government employees, some of which are now independent contractors. The panel also had an active government employee and one academic representative.
Fifteen research projects were peer reviewed by expert panelists using 11 evaluation criteria. These criteria were grouped within the following four evaluation categories:
- Project management.
- Approach taken for transferring results to end users.
- Project coordination with other closely related programs.
- Quality of project results.
The rating scale possibilities were “Ineffective,” “Effective,” “More than Effective” or “Very Effective.” During the May 2015 review, the average program rating between all the evaluation categories was “More than Effective.” For this year, 10 projects were rated “Very Effective” with 5 projects ranked as “More than Effective.” The average sub-criteria scoring were also rated very high and underpin these findings. The majority of peered projects and the overall program rating remained the same from the 2014 rating of “More than Effective.” Weakness in project management contributed to the no-change in the program average.
| Very Effective | 4.5 to 5.0 (10 Projects) |
| More than Effective | 3.0 to 4.4 (5 Projects) |
| Effective | 1.9 to 2.9 (0 Projects) |
| Ineffective | 0.0 to 1.8 (0 Projects) |
| Average Program Score | 4.5 |
| Review Categories and Sub-Criteria | Score | Rating |
|---|---|---|
| 1. Project Management. | 4.4 | More than Effective |
| 1.1. How well is the project being managed (on budget and schedule)? | 4.1 | More than Effective |
| 1.2. How well is the project making progress toward the work scope objectives? | 4.7 | Very Effective |
| 2. Approach taken for transferring results to end users. | 4.5 | Very Effective |
| 2.1. Is there a plan for dissemination of results, including publications, and reporting? | 4.6 | Very Effective |
| 2.2. How much end user involvement is incorporated into the work scope? | 4.6 | Very Effective |
| 2.3. For results that may include marketable products and technologies, are commercialization or U.S. Patent plans established? | 4.4 | More than Effective |
| 3. Project coordination with other related programs. | 4.5 | Very Effective |
| 3.1. Does the project build on, or make use of, related or prior work? | 4.7 | Very Effective |
| 3.2. Is the work of the project being communicated to other related research efforts? | 4.4 | More than Effective |
| 3.3. Has consideration been given to possible future work? | 4.3 | More than Effective |
| 4. Quality of project results. | 4.6 | Very Effective |
| 4.1. Are the intended results supported by the work performed during the project? | 4.5 | Very Effective |
| 4.2. Are the intended results consistent with scientific knowledge and/or engineering principles? | 4.6 | Very Effective |
| 4.3. Are the intended results presented in such a manner as to be useful for identified end users? | 4.6 | Very Effective |
| Average Category Score and Rating: | 4.4 | More than Effective |
| Project | Rank | Contract | Project Title | Score | Rating |
|---|---|---|---|---|---|
| 558 | 1 | DTPH56-14-H-00005 | Threat/Anomaly Mitigation Decision-Making Process | 4.9 | Very Effective |
| 555 | 2 | DTPH56-14-H-00002L | Consolidated Project Full Scale Testing of Interactive Features for Improved Models | 4.8 | Very Effective |
| 561 | 2 | DTPH56-14-H-00008L | Definition of Geotechnical and Operational Load Effects on Pipeline Anomalies | 4.8 | Very Effective |
| 494 | 3 | DTPH56-13-T-000004 | Advanced Leak Detection LiDAR | 4.7 | Very Effective |
| 501 | 4 | DTPH56-13-T-000012 | Evaluation of Structural Liners for the Rehabilitation of Liquid and Natural Gas Piping Systems | 4.6 | Very Effective |
| 503 | 4 | DTPH56-13-T-000008L | In-Ditch Validation Methodology for Determination of Defect Sizing | 4.6 | Very Effective |
| 498 | 5 | DTPH56-13-T-000009L | Improve and Develop ILI Tools to Locate, Size, and Quantify Complex/Interacting Metal Loss Features | 4.5 | Very Effective |
| 554 | 5 | DTPH56-14-H-00001 | Effects of Hydrocarbon Permeation on Plastic Pipe Strength and Fusion Performance | 4.5 | Very Effective |
| 556 | 5 | DTPH56-14-H-00003L | Strain-Based Design and Assessment of Segments of Pipelines with and without Fittings | 4.5 | Very Effective |
| 560 | 5 | DTPH56-14-H-00007 | Improving Leak Detection System Design Redundancy & Accuracy | 4.5 | Very Effective |
| 499 | 6 | DTPH56-13-T-000010 | Development of an Industry Test Facility and Qualification Processes for Inline Inspection (ILI) Technology Evaluation and Enhancements | 4.4 | More than Effective |
| 492 | 7 | DTPH56-13-T-000003 | INO Technologies Assessment of Leak Detection Systems for Hazardous Liquid Pipelines | 4.2 | More than Effective |
| 493 | 8 | DTPH56-13-T-000002 | Real-Time Multiple Utility Detection During Pipe Installation Using Horizontal Directional Drilling (HDD) System | 4.1 | More than Effective |
| 500 | 8 | DTPH56-13-T-000011L | Above-ground Detection Tools Including Disbondment and Metal Loss for all Metals Including Cast-Iron Graphitization | 4.1 | More than Effective |
| 557 | 8 | DTPH56-14-H-00004 | Improving Models to Consider Complex Loadings, Operational Considerations, and Interactive Threats | 4.1 | More than Effective |