Thursday, January 19, 2017

Revamping SAFe's Program Level PI Metrics Part 2/6: Business Impact

Managers shape networks’ behavior by emphasizing indicators that they believe will ultimately lead to long term profitability” – Philip Anderson, Seven Levers for Guiding the Evolving Enterprise



Introduction

In Part 1 of this series, we introduced the Agile Release Train (ART) PI metrics dashboard and gave an overview of the 4 quadrants of measurement. This post explores the first and arguably most important quadrant – Business Impact.

As you may have guessed from the rather short list, there can be no useful generic set of metrics. They must be context driven for each ART based on both the mission of the ART and the organisational strategic themes that gave birth to it. As Mark Schwartz put it so elegantly in The Art of Business Value, “Business value is a hypothesis held by the organization’s leadership as to what will best accomplish the organization’s ultimate goals or desired outcomes”.

Definitions


Rationale

Fitness Function

When reading The Everything Store: Jeff Bezos and the Age of Amazon, I was particularly taken by the concept of the fitness function. Each team was required to propose ".. A linear equation that it could use to measure its own impact without ambiguity. … A group writing software code for the fulfillment centers might home in on decreasing the cost of shipping each type of product and reducing the time that elapsed between a customer's making a purchase and the item leaving the FC in a truck". Amazon has since moved to more discrete measures rather than equations (I suspect in large part due to the bottlenecks caused by Bezos' insistence on personally signing off each team's fitness function equation), but I believe the “fitness function mindset” has great merit in identifying the set of business impact metrics which best measure the performance of an ART.

To illustrate based on three ART's I work with:
  • An ART at an organisation which ships parcels uses "First time delivery %". They implement numerous digital features enabling pre-communication with customers to avoid delivery vans arriving at empty houses. Moving this a percentage point has easily quantifiable bottom-line ROI impacts.
  • An ART focused on Payment Assurance at an organisation which leverages "Delivery Partners" to execute field installation and service work. Claims for payment submitted by these partners are complex and require payment within tight SLA's. A fitness function based on Payment lead time and cost savings based on successful claim disputes would again easily be mapped to quantifiable ROI.
  • A telco ART focused on self-service diagnostics for customers. The fitness function in this case would reference “reduced quantity of fault-related calls to call centers” (due to the customer having self-diagnosed and used the tool to make their own service booking if required), “reduced quantity of no-fault-found truck rolls” (due to the tool having aided the customer in identifying ‘user error’), “increased first call resolution rates for truck rolls” (due to the detailed diagnostic information available to service technicians).
Considerations when selecting fitness function components
Obviously, the foremost consideration is identifying a number of components from which one can model a monetary impact. However, I believe two other factors should be considered in the identification process:
  • Impact on the Customer
  • Ensuring a mix of both Leading and Lagging Measures
Net Promoter Score (NPS) is rapidly becoming the default customer loyalty metric, and whilst Reichheld argues in The Ultimate Question 2.0 that mature NPS implementations gain the ability to quantify the value of a movement in a specific NPS demographic I have yet to actually work with an organization that has reached this maturity. However, most have access to reasonably granular NPS metrics. The trick is identifying the NPS segments impacted by the customer interactions associated with the ART’s mission and incorporating those measures.

When it comes to identifying useful leading metrics, there can be no better inspiration than the Innovation Accounting concepts explained by Eric Ries in The Lean Startup. In some cases (particularly Digital), it can also be as simple as taking the popular Pirate Metrics for inspiration. For many trains with digital products, I also believe abandonment rate is an extremely valuable metric given that an abandoned transaction tends to equate to either a lost sale or added load on a call center.

Program Predictability

This is the standard proxy result measure defined in SAFe. It is a great way of ensuring focus on objectives whilst leaving space for Product Owners and Managers to make trade-off calls during PI execution. In short, I paraphrase it as "a measure of how closely the outcomes from PI execution live up to the expectations established with Business Owners at PI planning and how clear those expectations were".

But wait, there's more!

A good train will use far more granular results metrics than those listed above. Each feature should come with specific success measures that teams, product owners and managers should be using to steer their strategy and tactics (fuel for another post), but I am seeking here a PI level snapshot that can be utilized consistently at portfolio levels to understand the success or otherwise of investment strategy.

A closing note on the Fitness Function

I believe the fitness function definition should be identified and agreed at the ART Launch Workshop. Well-launched ARTs will have all the key Business Owners present at this workshop, and I strongly believe that agreement on how the business impact of the ART will be measured is a critical component of mission alignment.

Series Context

Part 1 – Introduction and Overview
Part 2 – Business Impact Metrics (You are here)
Part 3 – Culture Metrics
Part 4 – Quality Metrics
Part 5 – Speed Metrics 
Part 6 – Conclusion and Implementation


The gold standard of metrics: Actionable, Accessible and Auditable ... For a report to be considered actionable it must demonstrate clear cause and effect … Make the reports as simple as possible, so everyone understands them … We must ensure that the data is credible” – Eric Ries, The Lean Startup

16 comments:

  1. Pretty! It was really a wonderful blog. Thanks for the provided information.
    Asigo System Review

    ReplyDelete
  2. I wish to show thanks to you just for bailing me out of this particular trouble.As a result of checking through the net and meeting techniques that were not productive, I thought my life was done
    Business Management Software

    ReplyDelete
  3. I guess this blog is perfectly incomparable.
    UX designers

    ReplyDelete
  4. Your contents are too straightforward to browse and easy to understand.
    webpage

    ReplyDelete
  5. I am seeking here a PI level snapshot that can be utilized consistently at portfolio levels to understand the success or otherwise of investment strateg WPS PDF to Word Converter serial key

    ReplyDelete
  6. Most service based forays are labor intensive which the business entity revolves around. Dollar General Hours

    ReplyDelete
  7. Interesting post. which i wondered about this issue so thanks for posting and very good article which is a really very nice and useful article. Thank you
    Data Science Course in Noida

    ReplyDelete
  8. I really enjoyed reading this post and keep up the good work and let me know when you can post more articles or where I can find out more on the topic.
    Data Science Online Course

    ReplyDelete
  9. Excellent work done by you once again here and this is just the reason why I’ve always liked your work with amazing writing skills and you display them in every article. Keep it going!
    Data Analytics Courses in Hyderabad

    ReplyDelete
  10. I like viewing this web page which comprehend the price of delivering the excellent useful resource free of charge and truly adored reading your posting. Thank you!
    Data Science Certification Course

    ReplyDelete
  11. Just a shine from you here and have never expected anything less from you and have not disappointed me at all which i guess you will continue the quality work. Great post.
    Data Science Training in Gurgaon

    ReplyDelete
  12. Excellent post to make this blog more wonderful, attractive and cool stuff you have. Thank You.
    Data Science Course in India with Placements

    ReplyDelete
  13. I just got to this amazing site not long ago was actually captured with the piece of resources you have got here and big thumbs up for making such wonderful blog page!
    Data Scientist Course

    ReplyDelete
  14. Informative Post. The information you have posted is very useful and sites you have referred was good. Thanks for sharing.
    Data Science Course with Placement

    ReplyDelete
  15. You have done a great job and will definitely dig it and personally recommend to my friends. Thank You.
    Data Science Online Training

    ReplyDelete