Control Room Management: FAQs
Here you will find a listing of the most frequently asked questions (FAQs) related to the implementation of the control room management final rule. You may:
A.01 When does PHMSA plan to initiate inspections on CRM plans and procedures?
PHMSA will begin inspecting operators’ control room management plans starting August 1, 2011, which is the deadline for certain portions of the regulations.
Original: 6/17/2011A.02 Will PHMSA make its CRM inspection checklists available to the public?
PHMSA is planning to make inspection checklists publically available on the CRM website [http://primis.phmsa.dot.gov/crm/index.htm] when they become available, which is anticipated at the end of second quarter of CY 2011.
Original: 6/17/2011A.03 If the owner of a pipeline contracts for the operation of the pipeline by another party, who is the responsible party for compliance with the CRM rule?
The CRM regulations apply to all “operators” of the pipeline. The term operator is defined in 49 CFR 192.3 and 195.2.
Original: 6/17/2011A.04 If controllers are located in a control room that monitors and controls an intrastate pipeline, but the control room is located in a different state than the actual pipeline, do the CRM regulations apply?
Yes. The state or location of the control room operating regulated pipeline facilities does not determine the applicability of the CRM regulation.
Original: 6/17/2011A.05 How does the term “pipeline facility,” as used in the definitions of Control Room and Controller, relate to other terms such as “pipeline system” that were not used in those definitions?
Since both 49 CFR 192.3 and 195.2 define “pipeline facility,” PHMSA found it was better to use the same terminology in both regulations. “Pipeline facility” is defined broadly and includes line pipe, pipelines, pipeline systems, valves, rights-of-way, buildings, and any other equipment used in the transportation of gas and hazardous liquids. Part 192 does not define “pipeline system.”
Original: 6/17/2011A.06 Does the CRM rule apply to non-line pipe facilities such as breakout tanks, pumps or compressors?
Pipeline facility is defined in 49 CFR 192.3 and 195.2 and means any equipment used in the transportation of gas or hazardous liquids. The CRM regulations apply to control rooms and controllers that remotely monitor and control pipeline facilities, including but not limited to, breakout tanks, pumps, compressors or other equipment along the pipeline.
Original: 6/17/2011A.07 If a person in a control room monitors a Supervisory Control and Data Acquisition (SCADA) system and directs a technician in the field to manipulate a valve, is that person in the control room considered to be a controller?
Yes, a person that has responsibility to monitor a SCADA system and contacts others to initiate corrective actions is considered a controller. Also, a person that has responsibility to monitor a SCADA system and personally initiates corrective action via the SCADA system is also a controller.
Original: 6/17/2011A.08 If a controller directs a technician in the field to manipulate a valve, or take other action that does not involve use of, or access to, the SCADA system, is the technician in the field considered to be a controller?
No, in this scenario the technician is not a controller.
Original: 6/17/2011A.09 If an individual does not use a computer and display screen, but only monitors several discrete alarm indicator lights from a remote location and initiates action when an alarm (light) occurs, is that person a controller?
No, an individual who does not use a computer type interface with a keyboard/mouse, and display screen (or touch-controlled screen) is not considered to be a controller.
Original: 6/17/2011A.10 If a person monitors a pipeline status indication for non-operational purposes, and does not have assigned responsibility to initiate corrective action, is this person a controller?
No. Persons that monitor a pipeline status indication for non-operational purposes, such as business or maintenance personnel, would not normally be considered controllers.
There is no “minimum time of operation” criterion or a “minimum compressor size” criterion associated with the exception in 49 CFR 192.631(a)(1)(ii). Therefore, the full CRM rule would apply to this LDC since the pipeline is controlled by a controller from a control room that meets the requirements of the CRM rule.
Revised: 7/11/2011A.12 Does the CRM rule apply to a pipeline that has no SCADA system or control room?
Original: 6/17/2011A.13 How does the CRM rule apply to control rooms for gathering lines?
For gathering lines monitored and controlled by a controller in a control room with a SCADA system, the CRM rule applies to the regulated gathering lines as provided in the scope of Parts 192 and Part 195. For example, the CRM rule applies to regulated liquid gathering lines in non-rural areas (see § 195.1), but not certain other regulated rural liquid gathering lines (see § 195.11). As another example, the CRM rule applies to regulated “Type A” gas gathering lines (see § 192.9(c)), which may be treated the same as transmission lines for purposes of § 192.631(a)(1)(ii), but the rule does not apply to regulated “Type B” gas gathering lines (see § 192.9(d)). The CRM rule does not apply to unregulated gas or hazardous liquid gathering lines.
Original: 6/17/2011A.14 If an operator has more than one control room that independently controls separate pipeline systems, must all control rooms use the same procedures, SCADA displays, shift rotations, alarm management practices, etc?
Separate control rooms may have their own specific CRM programs. Each control room management program can be tailored to the unique aspects of the control room and its related pipeline system. PHMSA would expect any differences between the CRM programs to be accounted for in the operator’s controller training and qualifications. If, however, one control room serves as a back-up control room for another control room, then consistency and controller cross-training should be considered, and training and qualification material adjusted as necessary.
Original: 6/17/2011A.15 Does the CRM rule apply to special applications (e.g., leak detection, batch tracking)?
Special applications can be subject to aspects of the CRM regulations. Many of these applications are extensions of the SCADA system that provide operational information through computer displays and/or alarms the controller monitors. Information to/from special applications should be consistent with SCADA displays for the same reasons consistency and accuracy of traditional SCADA displays is critical for controllers.
Original: 6/17/2011A.16 What are “safety-related” operations and parameters in the CRM rule?
For purposes of Control Room Management, PHMSA considers safety-related to mean any operational factor that is necessary to maintain pipeline integrity or that could lead to the recognition of a condition that could impact the integrity of the pipeline, or a developing abnormal or emergency situation.
Original: 6/17/2011A.17 Are controllers subject to the CRM rule if the SCADA system automatically recognizes abnormal conditions and automatically places the pipeline in a “safe” condition without human controller intervention?
Yes, controllers are subject to the CRM rule, independent of the particular automated capabilities of the SCADA System.
Original: 6/17/2011A.18 If a distribution operator has its own control room with less than 20,000 services, but shares SCADA servers with an operator that has greater than 250,000 services managed by their own control room, does it meet the exception in 49 CFR 192.631(a)(1)?
The exception in 49 CFR 192.631(a)(1) is applicable to the control room, not the location of the SCADA server. In this example, the control room with less than 250,000 services being served from that location would meet the exemption for number of services.
Revised: 9/30/2011A.19 A gas distribution holding company operates multiple distribution systems in several cities. Each of the operating entities has its own SCADA system and control room. None has any compressor stations. None of the individual entities has over 250,000 services. However, collectively, the holding company has over 250,000 services. Do any of these operating entities meet the exceptions in 49 CFR 192.631(a)(1) if they are owned by the same company?
The exceptions in 49 CFR 192.631(a)(1)(i)and (ii) are for the control room. There is no language in the regulation regarding exemptions concerning holding companies or operating entities. Each independent control room in this scenario will meet the exception in 192.631(a)(1)(i) and (ii) and therefore will need to comply with only the requirements for fatigue management, validation, and compliance and deviations. However, if any of these control rooms serve as a back-up for other control rooms, then the combined number of services during back-up conditions may exceed the criteria for the exemption and would be required to comply with the entire CRM rule.
Original: 6/17/2011A.20 Does the CRM rule apply to a local control room and station personnel that monitor and control a local operation that is completely within the fenced boundary of the local facility?
Field personnel who exclusively operate station equipment within the defined station boundaries (fence lines or property/map boundaries) and who are not responsible for connected pipelines beyond the boundaries are not considered to be remotely monitoring and controlling a pipeline. Therefore, such personnel are not considered to be controllers. However, field personnel who operate station equipment within the station boundaries and also have either full-time or part-time control room operational responsibility for connected regulated pipelines beyond the station boundaries are considered controllers.
Original: 6/17/2011A.21 Do control rooms located in Canada need to comply with the CRM rules if they control pipelines operating in the United States?
If the operational activities in a control room impact pipeline facilities located in the United States, PHMSA will expect those activities to comply with the CRM rules. A coordinated effort between PHMSA and the National Energy Board (NEB) of Canada regarding cross-border pipeline facilities is addressed in the agencies’ written arrangement dated November 2005, which is available on the PHMSA website.
Original: 6/17/2011A.22 What does “services” mean in 192.631(a)(1)(i)?
“Services” means the number of services as reported on the operators annual report submitted to PHMSA in accordance with 49 CFR 191.11.
Original: 6/17/2011A.23 For off-shore applications, are individuals located on a platform with outbound PHMSA-regulated pipelines considered to be controllers if they only control platform operations (i.e., the transportation pipeline is controlled exclusively by a control room located onshore or on another platform)?
No. As long as persons on the platform exclusively operate equipment on the platform, and do not control the pipeline, they are not considered to be controllers subject to the CRM rule.
Revised: 9/30/2011A.24 With regard to the exceptions for gas operators in 192.631(a)(1), please clarify the phrase “are limited to either or both of.” Does this mean that if an operator meets either criterion (i) or (ii), only paragraphs (d), (i), and (j) need to be implemented?
No. The language in the rule addresses distribution operators, transmission operators, and distribution operators with transmission pipelines as part of their system. To clarify, this phrase means “either” (if only one of the criteria is applicable), or “both” (if both criteria are applicable). If an operator operates only transmission pipeline (no distribution), then criterion (i) is not applicable and the operator must meet criterion (ii) in order to only implement paragraphs (d), (i), and (j). If an operator operates only distribution pipeline (no transmission), then criterion (ii) is not applicable and the operator must meet criterion (i) in order to only implement paragraphs (d), (i), and (j). If an operator has both types of pipelines in its system, it must meet both criteria (i) and (ii) in order to only implement paragraphs (d), (i), and (j).
B.01 Do roles and responsibilities have to be formally defined and documented?
Yes. The rule requires that the operator develop and follow written procedures that implement all applicable requirements of the CRM rule. This includes formal definition and documentation of controller roles and responsibilities in its CRM program.
Original: 6/17/2011B.02 Do I need shift hand-over procedures if I do not have 24 hour shift coverage?
Yes. Anytime a controller completes his/her shift and/or control of the pipeline is transferred from one person to another person, shift hand-over requirements apply, even if there is a portion of time when the control room is planned to be unattended.
Original: 6/17/2011B.03 Can a control room supervisor direct or advise a controller on actions to take to complete a safety-related task?
A control room supervisor may direct or advise a controller on specific actions to take to complete a safety-related task, if and only if, the supervisor is also a qualified controller. If the supervisor is not a qualified controller (administrative supervisor), then the supervisor may only advise the controller on what general tasks to accomplish, but not the precise actions that would otherwise come from a controller-qualified supervisor.
Original: 6/17/2011B.04 What, if any, shift change requirements apply when a controller is temporarily relieved for a brief period of time, such as during a coffee break?
The operator’s procedures, training, and practices must address the appropriate level of hand-over of responsibility for short breaks, or other times, when the controller leaves the console. Operator guidelines should consider the controller’s proximity to the console, duration of absence, and the type of alarm interface (e.g., audible vs. visual alarms) in use. An operator’s program for cross-training controllers on multiple consoles can enhance flexibility to backup controllers for short breaks. Based on factors like these, an operator can have more than one strategy in place to ensure adequate console coverage is maintained.
Original: 6/17/2011B.05 Must controllers be qualified in all aspects of an operator’s control room?
Controllers only need to be qualified on the tasks for which they are assigned roles and responsibilities. In control rooms with multiple desks/consoles, controllers may be qualified only on one desk, or qualified on multiple desks to create more operational flexibility. Some operators may establish special limited roles and responsibilities for reaction to emergency conditions, where an individual is only qualified to return a line segment to normal or shutdown status.
C.01 Is point-to-point verification required for all SCADA points or only safety-related SCADA points?
The requirement is to verify all safety-related points in the SCADA system. This would also include calculated (software generated) points that are safety-related. Safety related points often, but do not necessarily, have alarms associated with them. Examples of points that may be considered safety related (and therefore would need to be verified when changes are made to field equipment or SCADA displays) include, but may not be limited to:
Original: 6/17/2011C.02 What constitutes an adequate point-to-point verification?
Principally, the process should verify the actual physical location and sequence among other devices and equipment at the location; and verify the data, information and any control or alarm functions to/from the point are being accurately represented on all SCADA displays on which it resides.
Original: 6/17/2011C.03 If changes are made to a SCADA display only, with no change to field equipment, do 49 CFR 192.631(c)(2) and 195.446(c)(2) require a point-to-point verification between the SCADA display and related field equipment?
Yes. The rule requires that point-to-point verifications between SCADA displays and related field equipment be conducted when changes are made to the field equipment or SCADA displays associated with safety-related points. In this case, such verification ensures any unintended errors that may have occurred during changes in SCADA displays are identified and corrected.
Original: 6/17/2011C.04 What is required in a “point-to-point verification between SCADA displays and related field equipment,” and what type of documentation should be generated and maintained?
Point-to-point verification means confirming that the input or output of each field instrument is accurately and reliably reflected in the SCADA information presented to the controller. Operators should document the actual field parameters, as measured in the field, and the corresponding SCADA display information, to record that the SCADA information displays accurately reflect field measurements. The date and names of individuals involved in the verification should also be recorded as a means to help demonstrate thoroughness and authenticity. Alarm set-point values should also be checked at the same time. Operators should remember that this may also apply to changes that are the result of pressure restrictions.
Original: 6/17/2011C.05 Must field devices be operated during a point-to-point verification between SCADA displays and related field equipment?
Operation of field devices is the most rigorous method of performing point-to-point verifications, but may not be practical in some situations. For example, closing main line valves to verify SCADA position indication might be too disruptive to system operation. In such cases, applying a simulated signal at the field instrumentation might be justified. Operators should develop point-to-point verification procedures with a view toward the most rigorous and all encompassing verification approach that is practical. The application of simulated signals should be used sparingly, and be configured as close to the actual field devices as possible.
Original: 6/17/2011C.06 Can point-to-point verifications be performed by a sampling process?
No. All those points specified in the rule that have changed must be verified.
Original: 6/17/2011C.07 What is the intent and general expectation for complying with the requirements for testing and verifying an internal communication plan for manual pipeline operation?
The intent of this requirement is that operators perform exercises or drills to assure that communication plans will be effective during an actual emergency involving loss of all SCADA system functions or other systems relying on SCADA data such as leak detection. Functions that must be verified during testing include, but are not limited to, (1) communication between and among operational and maintenance personnel using voice, fax, messaging, radio, etc., and (2) communication of pipeline operational data such as dial-in polling of field equipment, manually reading gauges and field instrumentation, etc. Note that equipment and modes of communication that are likely to be inoperable during a manual operation scenario should not be used during the test. Problems identified during exercises/drills should be corrected promptly and the effectiveness of corrective actions should be explicitly verified at the next exercise/drill.
Operators should also document and review related tasks required of controllers and field personnel during these type events. Operators should also verify that training and operator qualification programs include these tasks.
Original: 6/17/2011C.08 What types of systems are considered “backup SCADA systems,” e.g., computers, software, telecommunication systems, others?
Backup SCADA systems are independent or redundant systems that provide similar functionality to the primary SCADA system. Backup systems can be as simple as a redundant server and as complex as an entire backup control room with duplicate SCADA and communication systems. These systems are often located in a geographically diverse location not susceptible to a single natural disaster such as a hurricane or earthquake that might impact the primary system. Backup SCADA systems are unique to each pipeline system, and may not necessarily duplicate all of the performance and functionality of the primary system. Regardless of the nature, extent or location of any back-up SCADA system, all of its specified functional capabilities need to be verified annually.
Original: 6/17/2011C.09 In the event of a SCADA failure, what is meant by an adequate means for manual operation of the pipeline safely?
If an operator does not intend to continue operating the pipeline in the event of a catastrophic SCADA failure, then only procedures to safely perform a controlled shutdown and maintain and monitor pipeline integrity need to be in place. If an operator chooses to continue all, or partial, pipeline operations in the event of a catastrophic SCADA failure, the rule requires that operators have some reliable means to monitor and operate the pipeline system manually.
The nature and extent of the means used to monitor and operate the pipeline under such circumstances must be commensurate with (i) the level of operational performance being maintained during the SCADA outage (e.g., reduced operational capabilities, continued full pressure operation, etc.), (ii) the functional capability of the command-and-control infrastructure that would be available during a SCADA outage (e.g., disaster recovery center, local station manning, emergency communications systems, etc.), (iii) the availability and location of field personnel to monitor and operate the pipeline, and (iv) the logistics for manual overrides of equipment.
Original: 6/17/2011C.10 If no unusual events occurred during an entire shift, would a shift hand-over procedure still have to be performed?
Yes. The CRM regulations require the operator to define the information that will be transferred during shift turnover and the process by which this information is exchanged. The fact that no unusual events occurred is in itself information that an incoming controller is expected to know.
Original: 6/17/2011C.11 If an operator has a controller on regular day shifts only (e.g., 8-5 M-F) and uses callouts to handle off-shift needs, is a shift hand-over process still needed, since the same person would be returning the next day?
Yes. Even if the same person is scheduled to return, the controller may unexpectedly have to be replaced as the result of illness or other circumstance that prevents the controller from returning to duty the next day as planned. Even if the same individual returns the next morning, the shift hand-over process will help ensure no critical information has been forgotten.
Original: 6/17/2011C.12 If an operator upgrades or modifies a portion of its SCADA system that results in some changes to the SCADA display, and the display symbols unaffected by the upgrade/modification are different than those recommended in API RP 1165, would the operator need to update all SCADA screens (even those that were not modified or upgraded) so that all would use the same symbols?
APIRP 1165, Section 8 must be implemented in accordance with 49 CFR 192.631(c)(1) and 195.446(c)(1), which state in part "[c]reating a standard, consistent set of symbols is essential to efficient and understandable display design.” If such changes impacted only a certain group of consoles in the control room, and assigned controllers and qualified control room supervisors do not cross-train on or move from these consoles to unchanged consoles, then API RP 1165 may be appropriately limited to the affected group of consoles.
Original: 6/17/2011C.13 When testing the “backup SCADA system,” must the test include restoration of, and transfer of control back to, the primary SCADA system?
To ensure that the backup SCADA system will function as designed to support the safe operation of the pipeline in the event the primary SCADA system being unavailable, PHMSA would expect that returning the pipeline operation to the primary SCADA control be a part of the process and procedures.
Original: 6/17/2011C.14 With respect to testing and verification of backup manual operations, is the intent to test every manual operation or to test the capability to execute the backup plans, procedures, and processes?
Operators must test and verify that its internal communications plan can effectually implement backup manual operations in the event of a SCADA system failure. The test and verification process must be designed to confirm that the operator has adequate personnel, procedures, processes, communications infrastructure, and manual command-and-control capabilities to assure safe, reliable operations and pipeline integrity when operating manually. Such testing should (i) verify the sufficient and timely deployment of qualified personnel to field locations necessary to adequately operate equipment and monitor pipeline integrity, (ii) establish, supplement, and/or verify performance of its communications or command center, and (iii) exercise critical decision-making processes. Testing and verification should address all types of actions necessary to mobilize manual operations. Testing and verification should be performed on at least a representative sampling of the processes and equipment intended to be used during backup operations.
Revised: 6/20/2012C.15 If an operator upgrades or modifies a portion of an existing SCADA system (e.g., upgrades to later version of SCADA software or upgrades to larger/faster hard drives), must the operator implement API RP 1165 with respect to the upgrade/modification?
Operators must delineate what does and does not constitute the need to implement API RP 1165 within their plans and procedures. The CRM rule requires that the operator implement API RP 1165 whenever a SCADA system is added, expanded, or replaced. Routine upgrades or modifications of existing SCADA systems that do not impact display parameters, such as operating system, application software or hard drive upgrades do not necessarily require implementation of API RP 1165. However, changes that impact display parameters (such as display symbols, color palettes, or anything that affects the controller-machine interface) would require implementation of API RP 1165.
Original: 6/17/2011C.16 Is a Master-Slave SCADA configuration considered a backup system requiring an annual test as stated in 49 CFR 192.631(c)(4) and 195.446(c)(4)?
Yes. The slave unit should be tested annually to verify it is capable of performing its designed capabilities.
Original: 6/17/2011C.17 Are operators required to incorporate built-in automatic safety actions into their SCADA systems?
Operators may incorporate automatic SCADA safety actions into their SCADA systems, but this practice is not required in the CRM rule. However, automatic safety actions should be considered when testing and verification requirements are reviewed.
Original: 6/17/2011C.18 If an operator experiences an actual SCADA failure that results in the back-up SCADA system being pressed into service, can the operator claim that event as testing and verifying their back-up SCADA system?
Yes. As long as an adequate representative sampling of functions are performed, verified and documented during back-up operations. Operators may be able to use alarm and event logs generated during back up operations to help demonstrate that an adequate representative sampling of functions were tested.
Original: 6/17/2011C.19 If an operator expands or replaces a SCADA system, when must the SCADA system be in compliance with the API RP 1165 and alarm management requirements?
In such cases, if it is not practical for the SCADA system to be in immediate compliance with CRM requirements, operators must document the deviation in accordance with paragraph (j)(2) of the CRM rule. The documentation must demonstrate why immediate compliance with all CRM requirements is not practical, how the deviation is necessary for safe operation, and include a justified project timeline that includes an indication when full compliance is to be attained.
Original: 6/17/2011C.20 When must point-to-point verification be completed following field changes or SCADA display changes?
PHMSA expects operators to diligently and promptly complete actions required by the rule. PHMSA inspectors will assess an operator's plans, procedures and associated records to evaluate the operator’s process for completing point-to-point verification in a timely manner. Operators may include multiple timing criteria within their procedures for completing point-to-point verifications. Although there may be others, two examples of timing criteria are: data points already being used in the control room; and data points being added or checked out as a part of a system enhancement or replacement. Those data points already being used by controllers should be verified the same day a verification process became necessary. Those data points being added or checked out as a part of a major system enhancement or replacement should be verified before those data points are turned over to controllers for use.
D.01 What activities are considered to be off-duty time for a controller?
Off-duty is defined as time in which the controller is not performing any work, duties, meetings, training, or other assignments for the operator. The controller's commute time to and from work, and any time which the controller is not working for the operator is considered off-duty time.
Original: 6/17/2011D.02 What on-duty time must be included in the tabulation of duty hours for fatigue mitigation consideration?
Hours of service include time while an individual is performing controller activities, including shift-change and overlap, on-call duties, events, emergency or spill drills, meetings, training, receiving or providing performance reviews and all other time the individual performs activities for the operator. Any and all non-controller type duties a controller performs for the operator are considered on-duty time for fatigue mitigation purposes. Note that on-duty time must also be tracked and tabulated for individuals that are not normally performing controller duties, but that might be called on to perform controller duties on short notice if needed (such as qualified supervisors or others who are maintaining their qualified controller status).
Original: 6/17/2011D.03 What minimum time should be scheduled between shifts to provide controllers off-duty time sufficient to achieve eight hours of continuous sleep?
Controllers must have an opportunity for eight hours of continuous sleep between shifts. PHMSA encourages at least ten continuous hours of off-duty time to allow for commutes and other personal activities prior to going to sleep or after waking up. Shorter/longer commute times or the availability of nearby sleep facilities may influence the appropriate amount of off-duty time.
Original: 6/17/2011D.04 What are some specific elements that should be included as part of a fatigue mitigation training and education program?
The following are examples of elements that should be considered as part of fatigue mitigation training and education:
Original: 6/17/2011D.05 What are some examples of fatigue mitigation tactics (countermeasures)?
The operator is responsible for determining the fatigue risks that exist in its program, and appropriate mitigation tactics to implement given the operating environment (schedule, control room set-up, etc.) to reduce those risks. In general, fatigue mitigation tactics may include provisions for on-the-job napping, provisions for tactical caffeine use, standing (e.g., use of sit/stand workstations) procedures for double-checking checklist completions, task rotations to reduce the effects of task-specific fatigue, exercise areas, activities intentionally injected at specific times in the shifts when the risk of fatigue is high, and mechanisms in place to help deal with controllers who are self-identified or identified by supervisors as being fatigued. The operator should be aware that certain mitigation tactics may or may not work for certain individuals. There should be some flexibility to allow for countermeasures based on individual differences, and communication amongst the appropriate stakeholders within the organization to know what typically does or does not work best for certain individuals.
[§§ 192.631(d)(4) and 195.446(d)(4)]
Revised: 9/30/2011D.06 What hours-of-service limits should an operator use if its controllers never work nights/weekends, i.e. have a traditional 8-5 type, 5 day schedule?
Given that good quality sleep is capable of being obtained every night for a traditional "day" job, 5 days a week, controllers should be able to work to up 12 hours (plus 1 hour for shift hand-over) per day across a week with a modest risk of fatigue. To keep risk minimal, and reduce the need for a more elaborate fatigue mitigation program, the following parameters should be followed:
Original: 6/17/2011D.07 What hours-of-service limits should an operator use if its controllers work beyond or outside of a traditional 8-5 type, 5 day schedule?
Working beyond or outside of a traditional 8-5 schedule has inherent fatigue risks. In these cases reasonable maximum normal limits on controller hours of service should be:
Original: 6/17/2011D.08 Why should operators be concerned about controller fatigue?
All controllers are limited by the bounds of human physiology with respect to fatigue. Although there is variability between people in their responses to longer and nighttime work hours, fatigue affects everybody, affects everything we do, and is potentially dangerous: we are often unaware of how tired we are and how much fatigue is affecting our thinking. One problem with being fatigued is that we may tend to accept more risk than usual, just to get a task finished.
Original: 6/17/2011D.09 Does the CRM rule require a fatigue risk management system (FRMS) for control room operations?
No. PHMSA promotes the use of a fatigue risk management system (FRMS) as a tool for implementing fatigue mitigation. A FRMS is an operator-defined process by which a company intends to manage fatigue risk with involvement by all stakeholders including senior management. As part of any FRMS, operators should factor in any unique aspects of their operations, be able to deal with extraordinary cases of individual fatigue and individual differences between controllers that can increase the risk of fatigue. The involvement of top management is essential, as it is difficult to implement fatigue management without top-down buy-in.
Revised: 9/30/2011D.10 Should operators consider task-specific fatigue in their mitigation strategies?
Yes. Repeated and/or demanding work causes task-specific fatigue and the need for recovery. An extreme control room specific example might be when a controller is on the last work day of a cycle, working near maximum hours and/or days permissible, and is also working on the desk that is experiencing a high rate of interaction with people and radical hydraulic variations, at the fatigue peak of 4 - 6 am. Operators should consider the impact of various interactions and plan ahead because the combination of multiple factors may not be obvious. An operator should be aware that a combination of factors causing elevated fatigue risk also occurring at 4 - 6 am might not be entirely obvious if control rooms operate pipelines in multiple time zones.
Sometimes habituation and/or boredom occur in these situations. Generally, task-specific fatigue of this nature can be dealt with by rotating the worker from one task to another several times within the shift.
Another task-specific phenomenon that operators should be aware of is "Technostress”, which can be one effect of automation that may be seen in a control room. Work that is assisted by automation often requires specific, fine-motor and visual functions, vigilance, and repetitive operations. This kind of work produces task-specific fine-motor fatigue, visual fatigue, vigilance failures, monotony, and potentially repetitive-stress injuries.
Original: 6/17/2011D.11 How can an operator determine how many controllers are needed to adequately staff 24/7 operations to minimize the potential for fatigue risk?
There are various methods that can be used. The number of controllers an individual operator needs depends on the specific operation and requirements in each operator’s control room. A useful (but not definitive) indicator of adequate staffing is an employment ratio. Additional guidance on how to calculate and use an employment ratio to help determine adequacy of staffing levels is provided in the “Staffing of Regular, Cyclic 24/7 Operations” white paper on PHMSA’s Control Room Management Fatigue Mitigation website.
Original: 6/17/2011D.12 How may I determine whether control room fatigue contributed to an accident/incident?
To help identify the potential effects of fatigue on pipeline operations, operators are encouraged to review and apply the concepts and procedures described in “Investigating the Possible Contribution of Fatigue to Pipeline Mishaps” white paper on PHMSA’s Control Room Management Fatigue Mitigation website.
Original: 6/17/2011D.13 What are PHMSA’s expectations for emergency deviations from the maximum limit on controller hours of service?
The regulation requires operators to maintain documentation that demonstrates any deviation from the maximum limit on controller hours of service was necessary for the safe operation of the pipeline facility. Operators should plan for anticipated emergency deviations in advance and evaluate their potential for additional risks of controller fatigue. If additional risks exist as a result of any deviation, the operator would be expected to have or develop a corresponding plan to employ appropriate countermeasures, and demonstrate how those measures offset the additional risks. Frequent occurrence of the same type of deviation should prompt the operator to review policies and procedures to minimize their occurrence.
The operator has the flexibility to determine how best to demonstrate adequacy of deviation management through the structure and content of its processes and procedures. Many operators are implementing an exception review/approval process and form. Such a process should include analysis of events leading to the deviation, actions taken, as well as review following the deviation.
PHMSA encourages a process to take place with provisions for written approval in advance of anticipated deviations (PHMSA recognizes some deviations cannot be forecasted). For such a process and form, PHMSA would expect to see items such as:
PHMSA understands that unforeseen circumstances do occur which may make written approval in advance impractical. In such instances written documentation should be completed at the first practical moment after the event.
Revised: 9/30/2011D.14 If an operator doesn’t have shift lengths, schedule rotations, and maximum limit on hours of service explicitly noted in their fatigue mitigation related procedures, is an operator’s actual shift schedule and related records adequate to demonstrate compliance with (d)(1) and (d)(4)?
No. The rule requires operators to have written procedures that implement the fatigue mitigation requirements. PHMSA would expect an operator’s fatigue mitigation related procedures to describe the bounding parameters in shift lengths and schedule rotations, and maximum hours of service limits they have established as the general framework for their program. PHMSA would then expect to see examples of actual schedules, timesheets and other records to show how those procedures are implemented, including if/how any changes to those schedules are managed in the context of the procedures. Schedules and timesheets alone are likely not adequate, as they are subject to change in the case of call-outs, vacation, etc. and would only give a certain snapshot in time. Written procedures would provide the framework in which those schedules can be changed and managed to reduce the risks associated with fatigue.
Original: 6/20/2012D.15 Does the information presented in the other CRM FAQs and inspection criteria account for schedules of 7 consecutive day or night shifts followed by 7 consecutive days off, generally referred to as 7 on/7 off type shift schedules?
Control Room Management regulations do not exclude the use of 7 on/7 off type schedules. The regulations do, however, require the implementation of methods, including establishing shift lengths and schedule rotations, to reduce the risk associated with controller fatigue for any and all schedules. As part of fatigue mitigation strategies, PHMSA expects operators to have a scientific basis for the schedules and limits they select, and consider circadian effects, different types of shifts, the need for rest, and other factors highlighted by relevant research. As part of an overall fatigue mitigation program, operators need to take into account the relative fatigue risks of whatever schedule and limits they select, and ultimately be able to demonstrate how either the limits they select reduce the risk of fatigue, or how fatigue mitigation tactics (countermeasures) and other aspects of their overall program are sufficient to reduce the risk for fatigue.
There is an increased risk for fatigue as the number of successive shifts increase, particularly successive shifts involving night work. PHMSA’s CRM website is a resource for information about fatigue management and related mitigation strategies. FAQs provide some reasonable limits to consider, along with shifts/times where fatigue risks are elevated and where fatigue mitigation tactics should be implemented. Still other FAQs provide some examples of fatigue mitigation tactics (countermeasures.) The level of risk appears to increase in shift plans with periods that approach 7 successive shifts in a row, particularly if the 7 successive shifts all involve night work.
If an operator chooses to use limits past those recommended in FAQs and Inspection Guidance material, including 7-on/7-off schedules, an operator would need to provide sufficient justification on how their overall program reduces the risk for fatigue. Such justification may require different or additional countermeasures or a more comprehensive fatigue risk management approach.
There are a number of trade-offs in considering any schedule rotation, including 7-on/7-off type schedules. In addition to other guidance already provided, a white paper entitled “Shift Plans with Seven Consecutive Shifts (Miller, April 2012)” discusses the pros and cons behind such schedules, including some countermeasures that should be considered above and beyond those already included in other FAQs.
Operators should expect that inspectors would be more inquisitive about how they protect against fatigue risks on the 6th and 7th successive days of work (if day only), and on the 4th through 7th successive nights of work.
E.01 If an operator acquires a new SCADA system after all of the CRM compliance deadlines have passed, what is expected of the alarm management system?
Many SCADA systems have integral alarm management capabilities. However, an operator may choose to acquire an independent alarm management system which processes data from the SCADA system. If an operator acquires a new SCADA system with integral alarm management capabilities after all the compliance deadlines have passed, the alarm management capabilities must be fully functional whenever the new SCADA system becomes operational. The required periodic review of alarms would likely reveal opportunities to improve the initial alarm management capabilities over time.
Original: 6/17/2011E.02 What types of alarms does PHMSA consider “false alarms”?
For the purpose of the CRM regulations, any alarm that is presented to the controller that did not accurately reflect the actual operational parameter or condition, or an alarm that can mislead a controller to believe a condition exists, but that does not exist, is considered a false alarm.
Original: 6/17/2011E.03 Are alarms generated during testing considered “false alarms”?
Testing and maintenance activities should be planned in advance except during emergency situations. A controller should be aware in advance to avoid confusion. If a controller is aware of ongoing work, then such alarms would not be considered false alarms as they would accurately reflect activities in progress. If, however, the controller is not aware of testing activities or that the alarms are a result of the testing, then such testing could produce false alarms and should be considered as such.
Original: 6/17/2011E.04 What types of alarms are considered to be “safety-related alarms”?
PHMSA expects operators to designate safety-related alarms, and to train its controllers to understand which alarms are safety-related along with their individual implications to safety. In general, “safety-related” alarms include operating parameters and do not include items such as equipment efficiency alarms and related measurements. Refer to FAQ’s in section A for more information about the term “safety-related.”
Original: 6/17/2011E.05 What are “safety-related alarm setpoint values”?
Safety-related alarm setpoint values are thresholds, which if achieved or exceeded, will present an indication to the controller that equipment or processes are outside of the operator-defined normal parameters.
Original: 6/17/2011E.06 Does the requirement to annually monitor alarm activity require an operator to conduct a formal work / time-study?
This section of the CRM rule is not limited to alarms. While this requirement is located in the alarm section, this element of the rule requires that operators review all required activities of a controller. For example, this should include manual calculations, alarms, training, setpoints or control manual entries, phone calls, etc. Operators must monitor the overall content and volume of activity for a controller. The process by which this monitoring is to be done is not specified, but is expected to have a sufficient degree of formality and documentation. Operators must monitor the content and volume of activity being directed to a controller to substantiate any conclusions about maintaining or changing assigned duties.
Original: 6/17/2011E.07 If the controller’s workload has not changed for several years, can an operator assume the current workload represents an acceptable benchmark or basis for future comparisons?
No. Current operating practices alone may not be used as a sole basis or justification to maintain the status quo.
Original: 6/17/2011E.08 Is there an objective measure of controller workload that PHMSA is using as a benchmark?
The rule does not establish a uniform benchmark for controller workload. PHMSA expects operators to establish, annually evaluate and document the substantive adequacy of controller workload criteria. Job task analysis or related evaluations are encouraged. Periodic analysis, annually and whenever significant changes are being made, will help operators recognize and react to changes.
Original: 6/17/2011E.09 In regard to monitoring the content and volume of general activity directed to and required of each controller, what does PHMSA consider to be “sufficient time” for controllers to analyze and react to incoming alarms?
The CRM rule does not establish a uniform limit on controller time for responding to alarms. No one answer fits all, as the conditions specific to a pipeline facility and specific console may vary.
Original: 6/17/2011E.10 In regard to monitoring the content and volume of general activity directed to and required of each controller, what does PHMSA consider to be “general activity”?
“General activity” means any activity that is required of the controller. This includes, but is not limited to, pipeline operations, handling SCADA alarms, conducting shift change, greeting and responding to visitors, administrative tasks, impromptu requests, telephone calls, faxes, or other activities such as monitoring weather and news reports, checking security and video surveillance systems, using the internet, and interacting with colleagues, supervisors, and managers.
Original: 6/17/2011E.11 How many alarms per shift does PHMSA consider acceptable?
Since every pipeline system is unique, PHMSA has not established a set criterion. Each operator must have an alarm management plan and monitor/evaluate its alarms periodically in accordance with the rule. Operators are encouraged to review and apply similar concepts as those presented in the International Society of Automation (ISA) “Management of Alarm Systems for the Process Industries” (ANSI/ISA 18.2), The Engineering Equipment and Materials Users’ Association (EEMUA) “191 Alarm Systems - A Guide to Design, Management and Procurement” (EEMUA 191), or other applicable guidance when establishing alarm handling practices and considering a maximum number of alarms.
Original: 6/17/2011E.12 How many alarm priorities (e.g., low, medium, high, urgent, etc.) are considered acceptable?
The number of alarm priority categories should be sufficient for the controller to easily distinguish between operational, control limit restrictions and higher critical priorities, but should not exceed those recommendations found in the National Transportation Safety Board (NTSB) presentation regarding SCADA Safety Study of 2005. Operators are expected to document the basis for selected alarm priorities and to periodically monitor the alarm priority scheme to achieve and maintain effectiveness. The NTSB presentation is available here.
Revised: 9/30/2011E.13 When analyzing and reacting to alarms, are controllers expected to acknowledge alarms first and then analyze/react to them? Or does PHMSA expect that controllers will analyze and react to alarms first, and then acknowledge the alarm?
Each operator must determine and establish alarm response strategies in their procedures. Since there are numerous variations of conditions that may create any individual alarm, regimented alarm handling procedures may be inappropriate. However, an operator’s controller training and guidance material is expected to help controllers quickly analyze alarm situations in order to promptly determine a course of action.
Original: 6/17/2011E.14 49 CFR 192.631(e)(2) and 195.446(e)(2) require that operators identify points that have been taken off scan in the SCADA host, have had alarms inhibited, generated false alarms, or that have had forced or manual values for periods of time exceeding that required for associated maintenance or operating activities, on a monthly basis. What is PHMSA’s expectation for dealing with non-functional alarms after they have been identified?
Operators should promptly return alarm points to service. Operators should troubleshoot the cause of non-functional alarms, take appropriate corrective actions in a timely manner, and endeavor to return alarm points to service in an expedited manner.
Original: 6/17/2011E.15 With respect to the verification of the correct safety-related alarm set-point values and alarm descriptions when associated field instruments are calibrated or changed, can verification be done at a later time, such as during OM&ER book reviews, station checks, etc.?
No. Verification of safety-related alarm set-points and descriptions is an integral part of the work package whenever changes are made to field equipment or whenever field equipment is calibrated. Such calibration or field change tasks are not complete until the verification is successfully accomplished.
Original: 6/17/2011E.16 How should operators address alarm management deficiencies?
Operators are expected to promptly correct specific issues commensurate with their importance to safety. Operators should maintain an itemized list of deficiencies and their date of discovery, the corrective action to be taken, and the completion date (or schedule) for corrective actions. The operator’s documentation should also record the basis for the selection and scheduling of corrective action. In addition, the operator’s alarm management plan should include provisions to analyze its specific deficiencies to identify root cause, common cause, trends, etc., that are indicative of systemic deficiencies that need to be identified and corrected.
Original: 6/17/2011E.17 Should controllers be able to change alarm set-point values?
Many alarms points have been established that represent the absolute critical maximum or minimum for process variables, for which controllers should not be able to change set-points. However, many operators have incorporated operational alarms. These type alarms typically warn controllers of approaching conditions before they become critical and are considered operational tools. Controllers can adjust these alarm set-points. An operator’s procedures and training must reflect their specific alarm management practices.
F.01 Are an operator’s emergency procedures, which require operations personnel to contact the control room in the case of an emergency, adequate for compliance with the change management requirement that field personnel contact the control room when emergency conditions exist?
No, emergency procedures are not sufficient as means of compliance with this section of the CRM regulation. This requirement is broader than emergency conditions and also requires that field personnel contact the control room when making changes that effect control room operations or supportive systems in non-emergency situations (such as routine transmitter calibrations). Operators should develop documentation that identifies when and under what conditions the control room has been contacted by field personnel. In addition, operators should review data available internally (such as near miss or incident/accident data) to confirm that emergency contact to the control has been made as required.
Original: 6/17/2011F.02 If piping changes are being considered at a field location that do not impact any SCADA data, must the control room still be involved in change management discussions?
Yes. Even though SCADA data may not be impacted, the hydraulics performance of the pipeline system could be affected in a way that impacts control room operations. SCADA data is not the only thing that could impact control room operations. For example, replacing a mainline valve may not change any SCADA data, but such a change may impact valve cycle time which can be a very critical factor for controllers. The operator must define, in its CRM program, what impending field changes will be included in the management of change process.
Original: 6/17/2011F.03 Why does Part 192 have an extra paragraph in subsection (f) compared with Part 195?
The requirement for natural gas pipeline control rooms to participate in planning (49 CFR 192.631(f)(3)) also applies to hazardous liquid pipelines by virtue of the incorporation by reference of API RP 1168 at 49 CFR 195.446(f)(1). Since the API standard is not incorporated in its entirety in 49 CFR 192.631(f), it is necessary to explicitly include this requirement in the code for natural gas pipelines.
G.01 In regard to incorporating lessons learned into an operator’s control room management plan after reviewing reportable incidents/accidents to determine any deficiencies related to field equipment, what are some examples of field equipment that an operator must consider in relation to control room management?
Some examples of deficiencies (which are not meant to be all inclusive) in field equipment that could affect control room operations include: (i) instrumentation that is out of calibration that results in a false alarm or inaccurate display of operational parameters such as pressure or flow, (ii) valve limit switches that provide incorrect valve status, (iii) inappropriate setting for relief equipment compared to alarm set-points, and (iv) the discovery of a manual mainline valve previously unknown to the controllers.
Original: 6/17/2011G.02 Does “[i]nclude lessons learned from the operator's experience in the training program required by this section” apply to reportable accidents/incidents only?
No. This requirement applies to reportable accidents/incidents as well as other experience such as near misses, non-reportable events such as small leaks, audit findings, and any other source of operating experience that could better inform and better train controllers to safely control the pipeline and recognize and correctly respond to abnormal, unusual, or emergency conditions.
Original: 6/17/2011G.03 Does “[i]nclude lessons learned from the operator's experience in the training program required by this section” apply only to accidents/incidents/events in which the controller caused or contributed to the event?
No. This requirement applies to all accidents, incidents, events, and circumstances that could better inform and better train controllers to safely control the pipeline and recognize and correctly respond to abnormal, unusual, or emergency conditions. Certainly events in which controllers caused or contributed to the event are important to preclude recurrence of controller mistakes. However, proper controller reaction is an important aspect in precluding recurrence of other types of incidents as well.
H.01 What would PHMSA consider to be operating setups that periodically but infrequently occur?
Such circumstances would be unique to each operator. The operator must address, in its training program, all reasonably foreseeable operational configurations (i.e., setups), not just the routine setups. Examples, among many others, could be seasonal operating parameters, start-up and shutdown, line reversals, combining pipelines through valving to run in common versus split, bleed valve operations, power loss failure modes, slack line conditions, purging, and running ILI tools. Note that this requirement applies to all controllers subject to paragraph (h) of the CRM rule, even if their SCADA system only provides monitoring functionality and control functions are provided by controller interaction with field personnel.
Original: 6/17/2011H.02 If a controller passed his/her qualifications just prior to the implementation date of the new CRM rule, does the controller have to re-qualify after the implementation date?
CRM establishes the need for certain procedures and operating practices that would need to be incorporated into an operator’s qualification program. If the prior qualification includes and meets all applicable requirements of the CRM plan and associated activities, the controller does not need to re-qualify.
Original: 6/17/2011H.03 What if any additional covered tasks, as required by the Operator Qualification (OQ) regulations, will PHMSA want to see in an operator’s OQ Program?
An operator should continue to implement the OQ regulations through the application of the four part test for covered tasks, and determine whether any new tasks will be added to their program as a result of their actions under the CRM rule. The identification of any additional covered tasks will be operator-specific. Operators are expected to define both generic and task-specific covered tasks for controllers.
Original: 6/17/2011H.04 Does PHMSA endorse any commercial training sessions or products that are marketed for compliance with the CRM rule?
No. PHMSA cannot guarantee that information distributed at commercial training sessions accurately reflects CRM requirements, or that an operator’s adoption of such methods or purchase of any commercial products will result in compliance with the CRM rule.
I.01 During an inspection, does PHMSA expect an operator to be able to produce documents that validate compliance?
PHMSA expects operators to have and maintain records to demonstrate that required activities were satisfactorily accomplished. Upon request, the operator must provide CRM procedures and associated documentation and records to PHMSA or the appropriate state agency for review to validate compliance.
Original: 6/17/2011I.02 What level of detail should records contain?
Any records intended to demonstrate compliance should include sufficient details as a means to help demonstrate thoroughness and authenticity. Only annotating work performed/completed on a certain date would usually be deemed as inadequate.
Original: 6/17/2011I.03 How much time do I have to submit documents for compliance validation, upon receipt of a request?
The rule does not specify a mandatory deadline for submitting documents for compliance validation. PHMSA (or the State Agency) will endeavor to include in its request a specific deadline on a case-by-case basis that reflects the need date. For example, in preparation for an inspection, PHMSA (or the State Agency) may request the operator to submit documents by a specified date, or time frame, in advance of the inspection. Operators must submit documents by any reasonable deadline so requested. If PHMSA (or the State Agency) does not include a specific need date in the request, operators are expected to submit the information no later than 45 days from the date of the request.
J.01 In regard to maintaining records to demonstrate compliance, how long must an operator maintain records demonstrating compliance with each of the code requirements, for example, routine shift turnovers, and exceptions to controller HOS limits?
The rule does not establish specific record retention periods, but operators should address record retention requirements in their program to ensure that sufficient documentation is maintained to demonstrate compliance with the CRM rule. Generally, such records should include at least one year, or the last two periodic tests or validations, whichever is longer.
Original: 6/17/2011J.02 Must I document deviations from the FAQs?
The CRM rule requires that operators document deviations from the operator’s procedures, not the FAQs. These FAQs are intended to clarify, explain, and promote better understanding of issues concerning implementation of the Control Room Management Rule, but are not substantive rules themselves. They are provided to help the public understand how to comply with the regulations. If the operator chooses to address a consideration differently than recommended in the FAQs, the operator needs to develop and document a technical justification for its course of action and demonstrate that the level of safety is consistent with the regulatory requirements.
Original: 6/17/2011J.03 In cases where pipeline assets are acquired, is the new owner required to obtain records demonstrating compliance with the CRM rule from the former owner?
An operator acquiring pipeline assets is responsible to demonstrate compliance with the CRM rule. Records from the former operator may assist the new operator in demonstrating compliance. If an operator’s records do not demonstrate full compliance, the operator would be expected to promptly initiate actions to achieve full compliance.
Original: 6/17/2011J.04 Are electronic records acceptable to demonstrate compliance?
Yes. In general, records kept electronically are acceptable if they are maintained in a format that ensures the integrity, authenticity, and date of the records, and that they are readily retrievable and accessible for inspection. Operators must assure that changes or upgrades in technology do not make the media used to store prior electronic records unreadable.