Panel Peer Review of PHMSA Pipeline Safety Research Projects: 2018
R&D Menu
Since 2006, the Pipeline and Hazardous Materials Safety Administration's (PHMSA) Pipeline Safety Research and Development (R&D) Program has held annual structured peer reviews of active research projects to maintain research data quality, in accordance with mandates from the Office of Management and Budget (OMB) and the Office of the Secretary of Transportation (OST). PHMSA holds these reviews virtually via teleconference and the internet in order to save time and resources. Virtual teleconferences facilitate attendance from all U.S. time zones, Canada, and Europe, making it easier for panelists, researchers, project co-sponsors, and representatives of Agreement Officers to participate.
The annual peer review continues to build on an already strong, systematic evaluation process developed by PHMSA's Pipeline Safety R&D Program and certified by the Government Accountability Office. The 2018 peer-review panel, which consisted of three academic representatives, reviewed five projects using 11 evaluation criteria grouped into the following four categories:
- Project management;
- The method used to transfer results to end users;
- Project coordination with other closely related programs; and
- The quality of project results.
The potential ratings assigned by the peer-review panel are: Ineffective, Effective, More than Effective, and Very Effective. The average score for the five projects assessed during the May 2018 review was More than Effective; the average sub-criteria were also rated highly, underpinning these findings. All peered projects and the overall program retained the rating of More than Effective-the same rating received in 2017. At the time of the reviews, the majority of the projects were approximately 60 to 90 percent complete. The panelists made several recommendations associated with each project during the course of the review that were categorized into Strong and Weak points. However, none of these comments identified the critical actions required to salvage a project from failure, but instead recommended actions to further improve good performance.
| Very Effective | 4.5 to 5.0 (8 Projects) |
| More than Effective | 3.0 to 4.4 (16 Projects) |
| Effective | 1.9 to 2.9 (0 Projects) |
| Ineffective | 0.0 to 1.8 (0 Projects) |
| Average Program Score | 4.2 |
| Review Categories and Sub-Criteria | Score | Rating |
|---|---|---|
| 1. Project Management. | 4.3 | More than Effective |
| 1.1. How well is the project being managed (on budget and schedule)? | 4.2 | More than Effective |
| 1.2. How well is the project making progress toward the work scope objectives? | 4.5 | Very Effective |
| 2. Approach taken for transferring results to end users. | 4.2 | More than Effective |
| 2.1. Is there a plan for dissemination of results, including publications, and reporting? | 4.1 | More than Effective |
| 2.2. How much end user involvement is incorporated into the work scope? | 4.3 | More than Effective |
| 2.3. For results that may include marketable products and technologies, are commercialization or U.S. Patent plans established? | 4.3 | More than Effective |
| 3. Project coordination with other related programs. | 4.1 | More than Effective |
| 3.1. Does the project build on, or make use of, related or prior work? | 4.4 | More than Effective |
| 3.2. Is the work of the project being communicated to other related research efforts? | 4.1 | More than Effective |
| 3.3. Has consideration been given to possible future work? | 3.9 | More than Effective |
| 4. Quality of project results. | 4.3 | More than Effective |
| 4.1. Are the intended results supported by the work performed during the project? | 4.4 | More than Effective |
| 4.2. Are the intended results consistent with scientific knowledge and/or engineering principles? | 4.3 | More than Effective |
| 4.3. Are the intended results presented in such a manner as to be useful for identified end users? | 4.2 | More than Effective |
| Average Category Score and Rating: | 4.2 | More than Effective |
| Project | Rank | Contract | Project Title | Score | Rating |
|---|---|---|---|---|---|
| 724 | 1 | DTPH5616T00003 | Development of High Performance Gas-Coupled Ultrasonic Transducers for Inspection of Unpiggable Natural Gas Pipelines | 4.3 | More than Effective |
| 725 | 1 | DTPH5616T00002 | Development of EMAT Sensors for Corrosion Mapping of Unpiggable Natural Gas Pipelines Using ILI Tools | 4.3 | More than Effective |
| 653 | 2 | DTPH5615T00018L | EMAT Sensor for Small Diameter and Unpiggable Pipes; Prototype and Testing | 4.1 | More than Effective |
| 723 | 3 | DTPH5616T00004 | EMAT Guided wave technology for inline inspections of unpiggable natural gas pipelines | 4.0 | More than Effective |
| 644 | 4 | DTPH5615T00010L | Human Centric Approach to Improve Pipeline Non-Destructive Evaluation (NDE) Performance and Reliability | 3.8 | More than Effective |