Panel Peer Review of PHMSA Pipeline Safety Research Projects: 2016

R&D Menu


The Pipeline and Hazardous Materials Safety Administration’s (PHMSA) Pipeline Safety Research and Development (R&D) Program has held annual structured peer reviews of active research projects since 2006 in accordance with mandates from the Office of Management and Budget (OMB) and the Office of the Secretary of Transportation (OST) to maintain research data quality. PHMSA holds these reviews virtually via teleconference and the Internet, saving time and resources. This execution works well for panelists, researchers, Agreement Officers’ Representatives, and project co-sponsors, and facilitates attendance from all U.S. time zones, Canada, and Europe.

The annual peer review continues to build on an already strong and systematic evaluation process developed by PHMSA’s Pipeline Safety R&D Program and certified by the Government Accountability Office. The 2016 peer review panel consisted of one Federal employee and five academic representatives.

Fifteen projects were peer reviewed by expert panelists using 11 evaluation criteria grouped within the following four evaluation categories:

  1. Project management;
  2. Approach taken for transferring results to end users;
  3. Project coordination with other closely related programs;
  4. Quality of project results.

The rating scale possibilities were “Ineffective,” “Effective,” “More than Effective,” or “Very Effective.” During the May 2016 review, the average program rating throughout all the evaluation categories was “More than Effective.” For this year, eight projects were rated “Very Effective,” with 16 projects ranked as “More than Effective.” The average sub-criteria were also rated highly, underpinning these findings. The majority of peered projects and the overall program rating remained the same from the 2015 rating of “More than Effective.” Weakness in project management and communication with other related efforts contributed to the lack of change in the program average.

Rating Scale
Very Effective4.5 to 5.0 (8 Projects)
More than Effective3.0 to 4.4 (16 Projects)
Effective1.9 to 2.9 (0 Projects)
Ineffective0.0 to 1.8 (0 Projects)
Average Program Score4.2

Program Averages - Review Categories and Sub-Criteria
Review Categories and Sub-Criteria Score Rating
1. Project Management. 4.3 More than Effective
  1.1. How well is the project being managed (on budget and schedule)? 4.2 More than Effective
  1.2. How well is the project making progress toward the work scope objectives? 4.5 Very Effective
2. Approach taken for transferring results to end users. 4.2 More than Effective
  2.1. Is there a plan for dissemination of results, including publications, and reporting? 4.1 More than Effective
  2.2. How much end user involvement is incorporated into the work scope? 4.3 More than Effective
  2.3. For results that may include marketable products and technologies, are commercialization or U.S. Patent plans established? 4.3 More than Effective
3. Project coordination with other related programs. 4.1 More than Effective
  3.1. Does the project build on, or make use of, related or prior work? 4.4 More than Effective
  3.2. Is the work of the project being communicated to other related research efforts? 4.1 More than Effective
  3.3. Has consideration been given to possible future work? 3.9 More than Effective
4. Quality of project results. 4.3 More than Effective
  4.1. Are the intended results supported by the work performed during the project? 4.4 More than Effective
  4.2. Are the intended results consistent with scientific knowledge and/or engineering principles? 4.3 More than Effective
  4.3. Are the intended results presented in such a manner as to be useful for identified end users? 4.2 More than Effective
Average Category Score and Rating: 4.2 More than Effective

Project Rankings
Project Rank Contract Project Title Score Rating
642 1 DTPH5615T00008L Statistical Review and Gap Analysis of LNG Failure Rate Table 4.7 Very Effective
651 1 DTPH5615T00016L Rapid Aerial Small Methane Leak Survey 4.7 Very Effective
654 1 DTPH5615T00019L Intrinsically Locatable Technology for Plastic Piping Systems 4.7 Very Effective
640 2 DTPH5615T00005L Comparison of Exclusion Zone Calculations and Vapor Dispersion Modeling Tools 4.6 Very Effective
641 2 DTPH5615T00006L Characterization and Fitness for Service of Corroded Cast Iron Pipe 4.6 Very Effective
648 2 DTPH5615T00013L Development of an AMR Eddy Current-Based Crack Detection Sensor for the Live Inspection of UnPiggable Natural Gas Transmission Pipelines 4.6 Very Effective
643 3 DTPH5615T00007 Slow Crack Growth Evaluation of Vintage Polyethylene Pipes 4.5 Very Effective
649 3 DTPH5615T00014 Use of Electromagnetic Sensors to Quantify Strength and Toughness in Steel Pipelines In and Out Of Service 4.5 Very Effective
652 4 DTPH5615T00017L Pipeline Damage Prevention Radar 4.4 More than Effective
653 4 DTPH5615T00018L EMAT Sensor for Small Diameter and Unpiggable Pipes; Prototype and Testing 4.4 More than Effective
555 5 DTPH56-14-H-00002L Consolidated Project Full Scale Testing of Interactive Features for Improved Models 4.3 More than Effective
650 5 DTPH5615T00015L Natural Gas Pipeline Leak Rate Measurement System 4.3 More than Effective
645 6 DTPH5615T00009 Development of Comprehensive Pressure Test Design Guidelines 4.2 More than Effective
644 7 DTPH5615T00010L Human Centric Approach to Improve Pipeline Non-Destructive Evaluation (NDE) Performance and Reliability 4.1 More than Effective
646 7 DTPH5615T00011 Development of a Hardness Tester for Quantification of Material Properties in Live Natural Gas Transmission Pipelines 4.1 More than Effective
647 7 DTPH5615T00012L Emissions Quantification Validation Process 4.1 More than Effective
560 8 DTPH56-14-H-00007 Improving Leak Detection System Design Redundancy & Accuracy 4.0 More than Effective
655 8 DTPH5615T00020L Combined Vibration, Ground Movement, and Pipe Current Detector 4.0 More than Effective
656 8 DTPH5615T00003L Critical Review of Candidate Pipeline Risk Models 4.0 More than Effective
499 9 DTPH56-13-T-000010 Development of an Industry Test Facility and Qualification Processes for Inline Inspection (ILI) Technology Evaluation and Enhancements 3.8 More than Effective
638 9 DTPH5615T00001L Approaches for Preventing Catastrophic Events 3.8 More than Effective
639 9 DTPH5615T00002L White Paper on Risk Tolerance 3.8 More than Effective
657 9 DTPH5615T00004L Framework for Verifying and Validating the Performance and Viability of External Leak Detection Systems for Liquid and Natural Gas Pipelines 3.8 More than Effective
557 10 DTPH56-14-H-00004 Improving Models to Consider Complex Loadings, Operational Considerations, and Interactive Threats 3.4 More than Effective