Panel Peer Review of PHMSA Pipeline Safety Research Projects: 2007

R&D Menu


 The Pipeline and Hazardous Materials Safety Administration’s (PHMSA) Pipeline Safety Research and Development (R&D) Program held its first structured peer review of active research projects in February 2006 and the most recent peer review on March 27-29, 2007. Mandates by the Office of Management and Budget (OMB) and the Office of the Secretary of Transportation (OST) govern these reviews and are keeping PHMSA “Green” with research data quality. Conducting peer reviews via teleconference and the Internet is working well with panelists and researchers and facilitated attendance from all U.S. time zones.

The peer review continues to build on an already strong and systematic evaluation process developed by PHMSA’s Pipeline Safety R&D Program and certified by the Government Accountability Office. The peer review panel consisted of nine government and industry experts. Four of the nine panelists are active government representatives from the National Institute of Standards and Technology and the Minerals Management Service. The remaining five panelists are retired government and industry personnel who have active roles as peers for the American Society of Mechanical Engineers (ASME), National Association of Corrosion Engineers (NACE), and other standards developing organizations. Twenty seven active research projects were peer reviewed by expert panelists using 23 evaluation criteria. These criteria were grouped within the following six evaluation categories:

  1. Is the project still relevant to the PHMSA mission?
  2. Is the project well designed?
  3. Is the project still well managed?
  4. What is the approach taken for transferring results to end users?
  5. Is the project well coordinated with other closely related programs?
  6. Is the project producing high quality results?

The rating scale possibilities were "Ineffective," "Moderately Effective," "Effective," or "Very Effective." During the March 2007 review, the average program rating was “Very Effective” for each of the above six evaluation categories. Twenty-six projects were rated “Very Effective,” with only one project rating “Effective.” Sub-criteria scoring ranged between “Effective” and “Very Effective.” Additional details are available in Section 7, Tables 3 and 4 of the report.

Rating Scale
Very Effective3.9 to 5.0 (26 Projects)
Effective2.6 to 3.8 (1 Project)
Moderately Effective1.3 to 2.5 (0 Projects)
Ineffective0.0 to 1.2 (0 Projects)
Average Program Score4.270

Program Averages - Review Categories and Sub-Criteria
Review Categories and Sub-Criteria Score Rating
1. Is the project still relevant to the PHMSA mission? 4.5 Very Effective
  1.1. Is the project still relevant for enhancing pipeline safety or protecting the environment? 4.7 Very Effective
  1.2. Does the project support rulemaking, statutory requirements, inspection activities, or stakeholder recommendations? 4.4 Very Effective
  1.3. Does the project address a technology gap or consensus standard or general knowledge? 4.5 Very Effective
2. Is the project well designed? 4.4 Very Effective
  2.1. Does the project have appropriate objectives and milestones? 4.4 Very Effective
  2.2. Are the deliverables well defined? 4.4 Very Effective
  2.3. Is the scope of work clear, limited, and well defined? 4.4 Very Effective
  2.4. Are the capabilities of the project team appropriate to the work? 4.7 Very Effective
  2.5. Has the project a well designed plan for transferring results to end users? 4.3 Very Effective
3. Is the project well managed? 4.2 Very Effective
  3.1. Does the project have an up-to-date work plan? 4.2 Very Effective
  3.2. Is the project making progress toward the scope and the PHMSA goals? 4.2 Very Effective
  3.3. Is the project being managed on budget and schedule? 4.1 Very Effective
4. What is the approach taken for transferring results to end users? 4.2 Very Effective
  4.1. Is there a plan for dissemination of results, including publications, reporting, and patents? 4.2 Very Effective
  4.2. How much end user involvement is incorporated into the work scope? 4.4 Very Effective
  4.3. Have efforts been made to protect the intellectual property in a manner that allows for the greatest public impact? 4.0 Very Effective
  4.4. For results that may include marketable products and technologies are commercialization plans established? 4.0 Very Effective
5. Is the project well coordinated with other closely related programs? 4.0 Very Effective
  5.1. Does the project build on, or make use of, related or prior work? 4.4 Very Effective
  5.2. Is the project work being communicated to other related research efforts? 3.8 Effective
  5.3. Has consideration been given to possible future work? 3.9 Very Effective
  5.4. Is the project coordinated with related projects or programs in PHMSA, industry, or other government agencies? 3.9 Very Effective
6. Is the project producing high quality results? 4.3 Very Effective
  6.1. Are the intended results supported by the work performed during the project? 4.3 Very Effective
  6.2. Are the intended results consistent with scientific knowledge and/or engineering principles? 4.3 Very Effective
  6.3. Are the intended results appropriate for the resources expended? 4.3 Very Effective
  6.4. Are the intended results presented in such a manner as to be useful for identified end users? 4.3 Very Effective
Average Category Score and Rating: 4.3 Very Effective

Project Rankings
Project Rank Contract Project Title Score Rating
145 1 DTRS56-04-T-0003 Human Factors Analysis of Pipeline Monitoring and Control Operations 4.8 Very Effective
159 2 DTRS56-05-T-0002 Design, Construction and Demonstration of a Robotic Platform for the Inspection of Unpiggable Pipelines Under Live Conditions 4.7 Very Effective
207 2 DTPH56-06-T-000019 Augmenting MFL Tools with Sensors That Assess Coating Condition 4.7 Very Effective
160 3 DTRS56-05-T-0002 Design, Construction and testing of a segmented MFL sensor for use in the inspection of unpiggable pipelines 4.6 Very Effective
192 3 DTPH56-06-T-000010 Internal Corrosion Direct Assessment Detection of Water 4.6 Very Effective
199 3 DTPH56-06-T-000013 Guidelines for the Identification of SCC Sites and the Estimation of Re-Inspection Intervals for SCCDA 4.6 Very Effective
195 4 DTPH56-06-T-000001 Demonstration of ECDA Applicability and Reliability for Demanding Situations 4.5 Very Effective
209 4 DTPH56-06-T-000018 Dissecting Coating Disbondments 4.5 Very Effective
210 4 DTPH56-06-T-000021 Method for Qualification of Coatings Applied to Wet Surfaces 4.5 Very Effective
211 5 DTPH56-06-T-000020 Phase Sensitive Methods to Detect Cathodic Disbondment 4.4 More than Effective
200 6 DTPH56-06-T-000014 Validation and Documentation of Tensile Strain Limit Design Models for Pipelines 4.3 More than Effective
208 6 DTPH56-06-T-000017 Improved In-field Welding and Coating Protocols 4.3 More than Effective
176 7 DTPH56-05-T-0001 Understanding Magnetic Flux Leakage (MFL) Signals from Mechanical Damage in Pipelines 4.2 More than Effective
196 7 DTPH56-06-T-000012 ECDA for Unique Threats to Underground Pipelines 4.2 More than Effective
198 7 DTPH56-06-T-000007 Ultra-Low Frequency Pipe and Joint Imaging System 4.2 More than Effective
212 7 DTPH56-06-T-000023 Effect of Surface Preparation on Residual Stress in Multi-layer and Other Pipeline Coatings 4.2 More than Effective
171 8 DTPH56-05-T-0003 Corrosion Assessment Guidance for Higher Strength Pipelines 4.1 More than Effective
193 8 DTPH56-06-T-000011 Guidelines for Interpretation of Close Interval Surveys for ECDA 4.1 More than Effective
202 8 DTPH56-06-T-000015 Pipeline Integrity Management for Ground Movement Hazards 4.1 More than Effective
206 8 DTPH56-06-T-000022 External Pipeline Coating Integrity 4.1 More than Effective
187 9 DTPH56-06-T-000002 Define, Optimize and Validate Detection and Sizing Capabilities of Phased-Array Ultrasonics to Inspect Electrofusion Joints in Polyethylene Pipes 4.0 More than Effective
188 9 DTPH56-06-T-000003 Characterization of Stress Corrosion Cracking Using Laser Ultrasonics 4.0 More than Effective
197 9 DTPH56-06-T-000006 Long Term Monitoring of Cased Pipelines Using Long-Range Guided-Wave Technique 4.0 More than Effective
204 9 DTPH56-06-T-000016 Investigate Fundamentals and Performance Improvements of Current In-Line Inspection Technologies for Mechanical Damage Detection 4.0 More than Effective
190 10 DTPH56-06-T-000005 Differential Impedance Obstacle Detection Sensor (DIOD) - Phase 2 3.9 More than Effective
203 10 DTPH56-06-T-000016 Development of Dual Field MFL Inspection Technology to Detect Mechanical Damage 3.9 More than Effective
194 11 DTPH56-06-T-000004 Plastic Pipe Failure, Risk, and Threat Analysis 3.8 More than Effective