OMG Process Metrics Roundtable

• Framework for Classifying Metrics

Performance Metrics (How well does the process work?) vs. Management Metrics (How well do we manage the process?)   

Customer Metrics (things that interest our customers) vs. Internal Metrics (things that only interest us)  

Business Metrics (e.g. value/process instance) vs. Technical Metrics (e.g. server load)  

Goals (Goal attainment measured by metrics)

Actions (What actions are provoked if a metric has a particular value?)

• Basic measures identified included: time, quality, effort/cost and the 4th devils quadrant which was sometimes value, risk or other measure
• Metrics need to be goal / action driven
• An important predictor of success seen to be “referral to others” though felt there was not likely to be any 1 metric but rather a combination of metrics developed from common understanding
• Discussion around the use of leading and lagging measures
• The criticality of non-performance will help to drive the determination of correct metrics
• Understanding the relationships between the metric types and the consequence of different actions on these was important
• It was felt that most current measures tend to be lagging although perhaps there is more need for leading and enabling measures
• Terminology issues were identified such as my ‘late’ and the customers ‘late’ being too different things or the inclusion of week-ends etc
• Correct metrics came from understanding cause and effect

This summary was derived from the following notes:

General Metrics

• 4 basic measures – time quality, effort/cost, stuff that you get out of it i.e. value add / risk / devil’s quadrant
• Process metrics – timeliness – cycle time, wait time, elapsed time, put your process in place, time to manage project, time to market etc.
• Improvement – who did it, why, where are they from, analysis of new insights

Performance - Management

• Most value for least cost
  • Value creation, capture and maintenance
• Goal that comes from strategy is what drives the types of metrics are used
• All measures aimed at some primary value-add
• BPM provides more opportunities to collect data
  • Go from a more backward view to more forward looking view
  • How does this change the way reporting is done e.g. plane crash
• When you want the measure you don’t have what you need
• Platforms give you so much – use to figure out things like the time taken for invoicing – thus enable decisions to be made e.g. business activity monitoring v business intelligence
• All we have is “data in context” from systems – just one step away from “enlightenment”
• First order things – only have context of process that leads to other levels
• Mainstream production process (e.g. delivery of products)
  • Classes of metrics – how many products go through system etc
  • Management of the process – how many errors did we make
• Idea of process running and stable (operations) and managing change in process (how to change)
• Metrics come and go – concept called productivity
• Everything rolls up to $$$ (generated lots of discussion) – want to know %
• Making a distinction between process metrics and process management metrics
• Effects seem remote in process – lead to sub-optimisation – project actions forward in time
• This is the trend until it bends when doing a predictive thing

Internal - External

• Look at all stakeholders
  • What are the metrics that drive all of these – sometimes in conflict –  and resolve up front
  • Do internal metrics predict external metrics
• Adjustment of internal definitely affect external e.g. sales
• Internal are a validation of external but not a predictor
• If you are late internally may be late externally – our late different to his late
• Got to have leading indicators, again terminology issues, 5 day calendar vs 7 day
• Need metrics on sales things
• Non-awareness of establishing metrics e.g. promise things but don’t think of it as a metric as such
• Onion peel – layers of interaction with customer on the outside and CXO at centre
  • Clear articulation / recursive definition of the same metrics go all the way through
• Goal driven – critical and situation measures (TDB mining example with stockpiles)
• Criticality of non-performance – leads to inclusion of enablers for final required outcome rather than the outcome itself
• Identification of failure points
• Audience for the data

Business – Technical

• Definite relationship – understanding and availability
  • Difficult to establish direct relationship due to interfaces – gets back to modelling components•    Technical measures depend on getting the job done – get kept from business due to lack of understanding
• How the business metrics are influenced by technical measures
• Leading predictor – survive to make money – therefore recommendation important
• If metric customer experience – would be directly affected by technical measures
• Understanding the effect of one on the other