2012 Articles, What's New?

Pipeline Risk Assessment—Controlling the Bias

Written by WKM Staff on .

In the first installment of this column, we introduced the concept of pipeline risk assessment Essential Elements.  This is a list of ingredients that arguably must be included in any pipeline risk assessment.  Last time, we covered one of these essential elements—the need for measurements.  This time, we address another essential element, closely related to the use of measurements—controlling the bias.

Bias

The meaning behind the phrase—controlling the bias—can be less succinctly described as:  identify, understand, and manage the uncertainty, conservatism, and subjectivity of the assessment. Much has been written about uncertainty, quantifications of uncertainty using statistical theory, and philosophical implications of knowledge types and lack of knowledge.  We will focus here on the more practical aspects of handling uncertainty in our pipeline risk assessments.

‘Controlling’ is a deliberate word choice.  We recognize that some bias is intentional and useful so we are not trying to avoid all bias.  The right amount of bias applied at the right points in the process, is key.  Therefore, the intent is bias control.

Uncertainty is always present in our risk assessments because we have incomplete knowledge of true values—we don’t know the exact material properties at every point along every pipeline; we don’t know how many times an excavator will actually be digging near this segment next year; we don’t know where coating deterioration may have occurred, etc.

Even where we have values, we know that no measurement is perfect and all measurements are actually estimates.  Measurements sometimes do not even involve the use of a measuring tool.  For example, an estimate of 4 excavations per km-year is a measurement of the future activity level near the pipeline.

We estimate or measure things in full recognition that there is an inaccuracy associated with either.  We often measure samples to infer values of all members of the population.  We understand that the real world involves distributions of possible values, not point values.  The nominal wall thickness we record is 0.250” but we know that, at various points along the pipeline, the wall thickness may actually range from 0.231” to 0.267”.  We estimate the average risk (expected loss) for a segment of pipeline to be $220/year but we understand that there can be a multi-million dollar incident here next year, and again the year after!

We cannot eliminate uncertainty, but we can manage it.  This exactly mirrors risk—we cannot eliminate that either, but can manage it.

PXX

It is important that a risk assessment identify the role of uncertainty in its calculations.  Each assessment should be performed with a pre-determined target level of conservatism (which includes the handling of ‘uncertainty’, for our purposes here).  Depending on the intended use of the risk assessment results, various levels of conservatism might be appropriate.  As an aid to communication of conservatism level, a PXX designation can be used to show a level of confidence that actual experience will be no worse than estimated.  For instance, P90 is the point where 90% of future performance is expected to be ‘better’ than this value—one would be negatively surprised 10% of the time or once out of every ten episodes.  P99.9 is very conservative—a negative surprise occurs only once out of every 1,000 episodes.

The risk modeler should determine the level of conservatism appropriate to his audience (normally the users are decision-makers) needs.    The PXX designation communicates this to the user of the risk assessment.  PXX can refer to various aspects such as the conservatism in each input value or the conservatism in the final estimate.

P90+

A P90+ assessment (P99, P99.9, etc) intentionally contains layers of conservatism. A P90+ risk model assumes that things are ‘bad’ until proven otherwise. This is often done to encourage future data collection as a means of risk reduction and, more importantly, to ensure that risks are not underestimated    An underlying theme in a P90+ assessment is that ‘uncertainty shows as increased risk’. This is a conservative approach requiring that, in the absence of meaningful data or the opportunity to assimilate all available data, risk should be over-estimated rather than underestimated. Riskier values are generally assigned, reflecting the assumption of poor conditions, in order to accommodate the uncertainty. This results in a more conservative overall risk assessment.

A P90+ level of conservatism encourages and quantifies the value of data collection via inspections and testing.  It also avoids a discrediting of the model that would occur if, through the discovery of non-conservative estimates, it becomes apparent that the model is awarding the ‘benefit of the doubt’, thereby concealing possible risks.

So, a P90+ assessment is done to encourage data collection as a means of risk reduction as well as to protect the model’s credibility.  Some pipeline-specific examples of high conservatism include:

  • Assigning high rates to various potential exposures, eg using very aggressive corrosion rates, even when very rare
  • Assuming poor performance of older coatings and coatings of a certain type, even though, in the vast majority of cases, most coatings continue to perform very well.
  • Use of worst-case potential consequences, even when potential for larger consequence events is extremely small.
  • Assuming weaknesses in pipe strength, even if no direct evidence suggests their presence
  • Underestimating the likely benefit of mitigations

 

Note that when a number of P90+ inputs are used, they lead to final estimates that are much more conservative—perhaps P99.99 or higher. The P90+ assessment produces a point estimate for an extreme portion of the assumed distribution of actual values.  It suggests a very unlikely level of risk, but plausible.  Therefore, the P90+ assessment is more appropriate for use in risk management of individual pipeline segments.  A P90 level—negative surprises only 10% of the time—or higher is often warranted for risk management of specific pipeline segments.  More conservative assessments may also be appropriate when supporting new projects or for presentations in public forums.

P50

A P50 level of assessment represents the best estimates—the most likely values that will occur.    A P50 assessment best describes the anticipated behavior of the entire population of pipeline segments.  Such estimates are often used to calibrate the risk model.  However, P50 values will misrepresent the true risks for individual segments.  This is because the P50, as a point estimate for the mode or mean of the assumed distribution for the population, ignores the extreme values in that distribution.

In addition to its use in calibration, a P50 to P70 level of analysis might be appropriate for budget setting or long range planning.  The future behavior of whole pipeline systems is better understood via P50 assessments.  However, P50 estimates must be used very cautiously since they are designed to better measure the performance of populations rather than individual segments.  They are often inappropriate for use in risk management of specific portions of a pipeline.

Essential

This idea of bias-control might at first appear as a rather obscure, highly technical issue only.  However, it is actually an essential element and critical to proper risk assessment. It is essential to an understanding of the risk assessment and the subsequent use of the risk estimates.  If not already defined, one of the first questions to ask when viewing a risk assessment is:  ‘what is the level of conservatism in this assessment?’.

Published September 2012

Read the pdf version of the article