Questions about the science and techniques!

We are here to help you learn and understand everything available on FIT Online!

Re: definition for a critical step…do you have any specific examples (e.g., opening a drain valve, introduction of flammable solvent into pressure vessel, etc.)? I can think of certain activities that are "recoverable" and could lead to significant harm, which would not meet the definition per se (e.g., wrong equipment locked out during LTCT, incorrect interlock limit in the DCS, etc.). For those steps which are recoverable but still could lead to significant harm, are these considered / classified differently than those which do not lead to significant harm?

Great question and this is where it gets a bit tricky… there are steps that are "risk important" but recoverable and steps that are "CRITICAL" and unrecoverable. So it is almost a phased approach. There are usually hundreds of risk important steps for each critical step and organizations can get overwhelmed is they call and act like they are all critical. I like to think of the positioning of the wrong valve in a LOTO (if that valve is the primary reduction in energy) as unrecoverable because there are typically no barriers between the positioning of that valve and the individual impacted by the correct valve not being in the right position. For instance, if I make an error and CLOSE and LOTO valve 2 instead of valve 1 then there are no real recoveries between that action and the incident where the worker gets sprayed by acid out of the system that was supposed to be closed - making the closing of Valve 1 a critical step. By contrast… if I am doing a LOTO on a pump and I throw the wrong breaker, when I check for 0-energy I will see that the pump still has power (risk important is the breaker throw and the check). I hope this helps - thanks for the question - Rob

Helpful! So do you recommend that for every task that the critical steps are identified and designated (formatted) differently in the procedures than the "risk important" or other steps that would not result in significance harm? The concept of "recoverability" for a critical step seems to introduce a dependence or reliance on other layers of protection, tasks, or conditions. For example, we had a mistake where an operator closed the wrong process block valve during start-up from the DCS control room which caused an overpressure of process piping. At that time, it was critical because the system was operating at full rate. However, typically during start-up the system is depressurized and closing the valve would not result in loss of containment. It seems that one would need to perform a scenario analysis or HAZOP-type review of the procedure to determine whether the actions prescribed in the operating steps could result in a bad outcome to determine what is or is not "potentially" a critical step.

Elliot - I am a big fan of "identifiers" for critical and risk-important steps - as long as they don't overwhelm or provide a distraction. They need to be standardized as part of the process for developing an effective procedure process is the standardization of both the iconology and the definitions for critical steps, notes, cautions, warnings, etc.. It is worth NOT letting this be up to the plant or area, but part of the writing standards for the organization.
As for your specific example - a HAZOP type review, along with the anomalies review that you get form operators, can provide the needed IF-THENS for less probable but higher consequence actions or incidents.
I hope this helps - Rob

Another question for you, no rush…is there a designation for a step or action that is "potentially" critical dependent upon prior Risk-Important Actions? For instance, I would concur that pulling the trigger on a firearm (loaded or unloaded) is a Critical Step, as well as entering the line of fire. However, activating the safety mechanism on a firearm ordinarily would be a RIA since recoverable…but I have seen failures / defects associated with safety mechanisms where gun discharges unexpectedly upon flipping safety OFF (no trigger pull). Rather specific example but was thinking about how front-line personnel may need to anticipate differently those RIAs which could proceed directly to unrecoverable significant outcome due to failure of a protection layer versus those that could not. Perhaps this is were it gets fuzzy and depends upon knowledge of the known failure mechanisms (PHA / risk assessment).

I never (I should capitalize that) use the "gun" example for a few reasons, not the least of which is it is very confusing on the differences between critical steps and actions and risk important actions (not to mention it can be a bit polarizing when talking about it to others). I have always acknowledged that if a step COULD be unrecoverable based on a previous action or condition, list it as critical. Sometimes organizations flag the previous steps as risk important because of the impact of an error or condition at that step, on the one that could hurt, maim, or kill someone, or destroy equipment or the environment.
An easy way to think about it is that we don't want every step to be "critical" because they aren't, but we also don't want to limit those that MAY meet the definition just because we are not sure. I believe that erring on the cautious side works best, but when you start to identify too many steps as critical there may two things that are happening (1) you may be leaving out the steps that arent critical but may still be needed, and/or (2) the process may be more 'dangerous' than it has been given credit for in the past. Either way, think about the "perception of risk" part of deviation potential. That always helps me.
To complicate things, think about parachuting. Pulling the ripcord is certainly a critical step, but hte ony real way to mess it up (an error) would be to not pull it, and that iasnt really impacted by any previous steps. However, packing a parachute would seem to have some critical or at least risk important steps, since once you finish packing it there is no way to tell whether a previous error was made until you pull the cord, which seems a bit late in the process…
Rob

Better example with the parachute in illustrating the point…I can see how the gun is confusing and could be polarizing. Moreover, this idea that a prior step could be critical because verification that it was performed successfully is coupled with the subsequent critical step (pulling ripcord) and no indication of reliability in between. Thanks!

Connecting the dots here between the AERO and Deviation Analysis modules. In the Deviation Analysis (Block 1), Performance Modes should be used to determine whether the individual was in SB, KB, or RB mode. Absent from this module though but present in the AERO module is a discussion on the Top 10 Error Traps, 4 of which put you into KB performance mode. Thus, I have a three-part question each successively building on the other:
1) If either Poor Communication, Vague Guidance, Overconfidence, or Infrequent/First Time Task are applicable, then would this be processed as an error directly to Block 3?
2) More generally, if any of the Top 10 Error Traps are present, then would this be processed as an error directly to Block 3?
3) Maybe a dumb question, does the presence of an error trap (e.g., from the Top 10) constitute an error (or system-induced violation)? In other words, given that 1 or more error traps influenced the action-outcome pair, then it would seem unlikely the event could be individual (isolated) in nature. If so, then should we be seeking the absence of error traps in the final determination of whether we are inside or outside the "little red box"?
One of the more difficult situations to deal with in Lock/Tag/Clear/Try is when someone claims to have "forgotten" to do it or forgot to complete a step (system de-inventoried) as part of the work permit process. In these situations and others, I am looking to provide a more formulaic approach so that analysts can check all the relevant boxes (i.e., Top 10 Error Traps or Top 25 Written Error Traps) before coming to the conclusion of individual accountability. Otherwise, without a process and guidance, the discussion easily leads to a dead end…"you should have known better", "they were complacent", "everybody knows to do it the right way", etc. with disciplinary measures to follow.

This is an awesome "level of detail" question that I should probably address in a video. In an effort to make Deviation Analysis a workable tool, I chose for all of the "traps" to come under the definition of error, deviation, violation instead of trying to evaluate each one independently. The traps, while called "error traps", they are actually traps that drive actions that could be an error (90% probability) or a violation (10% probability). So if we stick to the definitions, once we see a trap, then we run it thru the definitions in Block 1 for error and violation. Would you be interested in coming on my Essential Leadership Cycle Podcast and having this discussion? I hope this helps - Rob

Absolutely, I would be delighted to participate in the discussion. Thanks! - Elliot

A more theoretical question that I have been pondering…are the 90% / 10% probability splits derived from a ratio with the denominator equal to "actual" deviations or only “revealed” deviations which would be a subset of the actual total (i.e., under-reporting effect)? I am guessing it is the former where studies were performed under reasonably controlled conditions (Deming, Reason, PII) but I haven't seen the data myself to draw any conclusions. Yet, in order to calibrate internal company data then one would need to understand that the apparent ratio to an analyst / leader could be higher than the industry benchmarks simply due to a denominator problem, not a numerator problem. Hypothetically, if there were 100 deviations which included only 1 isolated violation with no systemic-drivers then the actual ratio would be 1%, but if only 50 were revealed/observed, then the apparent ratio would be 2% (1/50). Likewise, if only 10 were revealed assuming this included the 1 isolated violation, then the apparent ratio to an observer would be 10%. So I am thinking that organizations who perceive an apparent ratio significantly greater than 1% for isolated violations, then perhaps they are not uncovering all deviations. This creates a bit of a dilemma in that one would either be calling Vs right and under-reporting or calling to many Vs and reporting accurate. Which sort of leads to another issue…how to incentivize workers to recognize and report errors (e.g., an Error Log).

[conflict alert…] We have said for years, much to the chagrin of HP purists, that you CANNOT count errors and violations - you will always be wrong. What is preferred is to get the workforce to start identifying the precursors to error (and violation) in the form of traps and systemic drivers. Then the organization is taking a proactive approach to prevention, reduction, and mitigation, instead of a reactive approach to the errors or violations, regardless of the outcome. I encourage organizations to use the probabilities as guides (does a violation rate of 1% or 2% really represent a difference?) instead of as places to watch for data. now, if every other thing they see winds up being an isolated violation, you probably have a different problem with reporting (go back to the Essential Leadership Cycle and wonder why the organization doesn't trust leadership…). In the Deviation Analysis model when we say that if more than about 10% of the deviations you are assessing wind up being violations, then you probably have a problem with using the process, not an actual mismatch in the probabilities of violations. I have just never seen it manifest itself in an organization that they need to slice and dice the data on errors and violations so thin that they can effectively predict in a manner to change their systems because by the time they do that - their leaders have altered their knowledge and behaviors enough to get the big gains instead of the little ones.
Another interesting question, probably worth a long chat! Thanks for asking - Rob

Yes, I agree and in the words of the renowned statistician George E.P. Box, "all models are wrong, but some are useful". My question was more conceptual though about can you actually validate the 90/10 split with real world data, whether this is a robust organizational decision is another issue I suppose. To your point, the "every other violation being isolated" in a qualitative sense is what I would be trying to address/correct based on anecdotal evidence at select sites. Thanks for the insight!

Great discussion - thanks!

Referencing and Branching - Is there an "optimal" for the limit of procedure pages at which point you would branch and/or reference to a separate procedure? For a single unit / distillation column, we could have 7-10 separate tasks associated with the unit that need to occur…some may include all in one procedure, others may break them apart. I have read the Writer's Manual but specific numerical values not provided, as with Performance Modes. Is there a relation to error rates and how these are broken apart? Thanks.

Any relation of the 7 step upper limit on skill-based behavior and Miller's Law???
Miller's Law - "The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information"[1] is one of the most highly cited papers in psychology.[2][3][4] It was published in 1956 in Psychological Review by the cognitive psychologist George A. Miller of Harvard University's Department of Psychology. It is often interpreted to argue that the number of objects an average human can hold in short-term memory is 7 ± 2.

Miller et.al.! The 7 action / step / item / memory "limit" has been so well studied and documented. It amazes me how often the people in a workshop have never heard of it. It also amazes and impresses me how quickly they buy into it and understand it once they are taught it - at ALL levels of the organization. It is why when I am in front of a group of SR leaders I tell them that when they are deploying HP they should set a 5 step limit on performing without written guidance.

SRK model probabilities…I'm sure this is a softball question for you since you're writing the book. Help me with the references to the studies that demonstrated 1/1000 (Skill), 1/100 (Rule), and 1/10 to 1/2 (Knowledge) which differ from the INPO HP Reference Manual for S and R. I realize you have stressed these are "average" probabilities, not the best or lowest achievable under ideal conditions as indicated below. The sources would help me so I know where they came from. Thanks!

The INPO HP Reference Manual references the follow:
Skill-based: under "ideal" conditions, the chance for error is less than 1 in 10,000 (Health and Safety Commission. “Advisory Committee on the Safety of Nuclear Installations Study Group on Human Factors, Second Report: Human Reliability Assessment - A Critical Overview.” ) Her Majesty's Stationery Office. 1991, p.9.
Rule-based: with less familiarity with the activity, the chance for error increases to roughly 1 in 1,000 (Swain & Guttman. Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications (NUREG/CR-1278). 1983.)
Knowledge-based: the chance for error is particularly high, approximately 1 in 2 to 1 in 10 (According to a conversation with Dr. James Reason, former professor of psychology at the University of Manchester in the United Kingdom, the chances for error in a knowledge-based situation are roughly a toss-up, “If you're good.” Otherwise, the chances for success get worse.
February 1997. )

Elliot - one of the ways we try to say this is that there is a difference between the normal or natural error rates for SB-RB-KB, and the published "ideal" error rates; and that REALLY matters to both the individual whom you are asking to reduce their probability, and to the organizations and the way it is taught. The original research is VERY hard to find (I won't go into why but it is an interesting story) so many people and places that use the error rates, use the ideal rates instead of the normalized rates. The adverse impact of this has been enormous in the misuse of performance modes at the individual and organizational levels. Here are a few references that may help (and yep - they are in the book!)…
• Jensen, A., Rasmussen, J., & Timmerman, P. (1963). Analysis of failure data for electronic equipment at Risø (Risø Report No. 38). Roskilde, Denmark: Danish Atomic Energy Commission, Research Establishment Risø
• Rasmussen, J., & Jensen, A. (1973). A study of mental procedures in electronic troubleshooting (Risø-M-1582). Roskilde, Denmark: Danish Atomic Energy Commission, Research Establishment Risø.
• Jens Rasmussen, Skills, Rules, and Knowledge; Signals, Signs, and Symbols, and Other Distinctions in, Human Performance Models (pp. 257), IEEE Transactions On Systems, Man, And Cybernetics, Vol. smc- 1 3, No. 3, May, 1983
• Swain, A.D. and H.E. Guttmann. NUREG IcR-1278, Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications" SAND 80-0200. Washington, DC: NRC. 1983.