Sunday, February 5, 2017

Revamping SAFe's Program Level PI Metrics Part 5/6 - Speed

Changing the system starts with changing your vantage point so you can ‘see’ the system differently.  Development speed is often attributed to quick decisions.  Early definition of the requirements and freezing specification quickly are often highlighted as keys to shortening the product development cycle.  Yet the key steps required to bring a new product to market remain the creation and application of knowledge, regardless of how quickly the requirements are set.  The challenge in creating an effective and efficient development system lies in shortening the entire process.” – Dantar Oosterwal, The Lean Machine.

Series Context

Part 1 – Introduction and Overview
Part 2 – Business Impact Metrics
Part 3 – Culture Metrics
Part 4 – Quality Metrics
Part 5 – Speed Metrics (You are here)
Part 6 – Conclusion and Implementation


Introduction

As mentioned in my last post, the categorization of metrics went through some significant reshaping in the review process.  The “Speed” (or “Flow”) quadrant didn’t exist, with its all-important metrics divided between “Business Impact” and “Deployment Health”.  

Lead Time is arguably the most important metric in Lean, as evidenced by Taiicho Ohno’s famous statement that “All we are doing is looking at the customer time line, from the moment the customer gives us the order to the point when we collect the cash”.  Not only does it measure our (hopefully) increasing ability to respond rapidly to opportunity, but is a critical ingredient in enabling a focus on global rather than local optimization.

In this quadrant, the core focus is to employ two perspective views on Lead Time.  The first (Feature Lead Time) relates to the delivery of feature outcomes, and the second (MTTR from Incident) our ability to rapidly recover from production incidents.

The other proposed metrics highlight the cycle time of key phases in the idea-to-value life-cycle as an aid to understanding “where we are slow, and where we are making progress”.  In particular, they will highlight failure to gain traction in XP and DevOps practices.

There is, however, a caveat.  Many (if not most) Agile Release Trains do not begin life in control of the entire idea-to-value life-cycle.  On the one hand, its very common for features to be handed off to an enterprise release management organisation for production release.  On the other, whilst Lean Principles are at the heart of SAFe the framework centers on hardware/software development.  The (traditionally business) skill-sets in areas such as operational readiness, marketing and sales required to move from “Deployed product” to “Value generating product” are nowhere on the big picture. 

ARTs focused on bringing to life the SAFe principles will address these gaps as they inspect and adapt, but in the meantime  there is a temptation to “not measure what we are not in control of”.  As a coach, I argue that ARTs should “never let go until you’ve validated the outcome”.  You may not be in control, but you should be involved – if for nothing else than in pursuit of global optimization.  

Basic Definitions


Basic Metrics Rationale

Average Feature Lead Time (days)

This is the flagship metric.   However, the trick is determining "when the timer starts ticking".   For an ART maintaining the recommended 3-PI roadmap, feature lead time would rarely be shorter than a depressing 9 months.  
To measure it, one needs 2 things: A solid Feature Kanban, and agreement on which stage triggers the timer.  A good feature kanban will of necessity be more granular the sample illustrated in the framework (fuel for a future post), but the trigger point I most commonly look for is "selection for next PI".  In classic kanban parlance, this is the moment when a ticket moves from "backlog" to "To Do", and in most ARTs triggers the deeper preparation activities necessary to prepare a feature for PI planning.  The end-point for the measure is the moment at which the feature starts realizing value and is dependent on solution context, often triggered by deployment for digital solutions but after business change management activities for internal solutions.

Average Deployment Cycle Time (days)

This metric was inspired by the recently released Devops Handbook by Gene Kim and friends.  In essence, we want to measure “time spent in the tail”.  I have known ART after ART that accelerated their development cycle whilst never making inroads on their path to production.  If everything you build has to be injected in a 3-month enterprise release cycle, its almost pointless accelerating your ability to build!  
Whilst our goal is to measure this in minutes, I have selected days as the initial measure as for most large enterprises the starting point will be weeks if not months.

Average Mean Time to Restore (MTTR) from Incident (mins)

When a high severity incident occurs in production, how long does it take us to recover?  In severe cases, these incidents can cause losses of millions of dollars per hour.  Gaining trust in our ability to safely deploy regularly can only occur with demonstrated ability to recover fast from issues.  Further, since these incidents are typically easy to quantify in bottom-line impact, we gain the ability to start to measure the ROI of investment in DevOps enablers.

Prod Deploys Per PI (#)

Probably the simplest measure of all listed on the dashboard - how frequently are we deploying and realizing value?

Advanced Definitions


Advanced Metrics Rationale

Average Feature Execution Cycle Time (days)

This is one of the sub-phases of the lead time which are worth measuring in isolation, and is once again dependent on the presence of an appropriately granular feature kanban.  
The commencement trigger is "first story played", and the finalization trigger is "feature ready for deployment packaging" (satisfies Feature Definition of Done).  The resultant measure will be an excellent indicator of train behaviors when it comes to Feature WIP during the PI.  Are they working on all features simultaneously throughout the PI or effectively collaborating across teams to shorten the execution cycles at the feature level?

One (obvious) use of the metric is determination of PI length.  Long PI’s place an obvious overhead on Feature Lead Time, but if average Feature Execution time is 10 weeks its pointless considering an 8 week PI.  

Average Deploy to Value Cycle Time (days)

This sub-phase of feature lead time measures "how long a deployed feature sits on the shelf before realizing value".  
The commencement trigger is "feature deployed", and the finalization trigger is "feature used in anger".  It will signal the extent to which true system level optimization is being achieved, as opposed to local optimization for software build.  In a digital solution context it is often irrelevant (unless features are being shipped toggled-off in anticipation of marketing activities), but for internal solution contexts it can be invaluable in signalling missed opportunities when it comes to organizational change management and business readiness activities.

Average Deployment Outage (mins)

How long an outage will our users and customers experience in relation to a production deployment?  Lengthy outages will severely limit our aspirations to deliver value frequently.  

Conclusion

We’ve now covered all 4 quadrants and their accompanying metrics.  The next post will conclude the series with a look at dashboard representation, implementation and utilisation. 

High performers [in Devops practices] were twice as likely to exceed profitability, market share, and productivity goals.  And, for those organizations that provided a stock ticker symbol, we found that high performers had 50% higher market capitalization growth over three years.” – Gene Kim, Jez Humble, Patrick Debois, John Willis .. The Devops Handbook


72 comments:

  1. This is a nice and informative, containing all information and also has a great impact on the new technology. Thanks for sharing it
    website designing Service

    ReplyDelete
  2. Very detailed content on data safety, a pleasure to read while looking for enterprise backup solutions that work with aws

    ReplyDelete
  3. Not exclusively will this assist you with staying away from punishments, it will likewise assist you with boosting the productivity of your site improvement methodology. Obviously,blog comments service in 1$

    ReplyDelete
  4. It was a very good post indeed. I thoroughly enjoyed reading it in my lunch time. Will surely come and visit this blog more often. Thanks for sharing. dewapoker

    ReplyDelete
  5. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...

    AWS Online Training
    AWS Certification Training
    AWS Certification Course Online
    AWS Training
    AWS Online Course

    ReplyDelete
  6. The post is written in very a good manner and it contains many useful information for me.


    gexton security network it system

    ReplyDelete
  7. putlockerfree.pictures Great post, thank you for sharing this usefull information.It is very easy to understand.Keep regular update with your blogs.

    ReplyDelete
  8. My rather long internet look up has at the end of the day been compensated with pleasant insight to talk about with my family and friends.
    Cloud Business Management Software Suite

    ReplyDelete
  9. Thanks for the informative and helpful post, obviously in your blog everything is good..

    Data Science Training

    ReplyDelete
  10. This is an excellent article. Thanks for sharing this information. I will be visiting your blog regularly for the latest articles. I will be visiting your blog regularly to see some of the latest posts.

    Data Science Certification

    ReplyDelete
  11. I don’t suppose many of websites give this kind of information. Calgary SEO Agency

    ReplyDelete
  12. It would also motivate almost everyone to save this webpage for their favorite helper to help get the look published.

    Business Analytics Course in Bangalore

    ReplyDelete
  13. Just a shine from you here. I have never expected anything less from you and you have not disappointed me at all. I guess you will continue the quality work.

    Data Analytics Course in Bangalore

    ReplyDelete
  14. Honestly speaking this blog is absolutely amazing in learning the subject that is building up the knowledge of every individual and enlarging to develop the skills which can be applied in to practical one. Finally, thanking the blogger to launch more further too.

    Data Science training in Bhilai

    ReplyDelete
  15. Really wonderful blog completely enjoyed reading and learning to gain the vast knowledge. Eventually, this blog helps in developing certain skills which in turn helpful in implementing those skills. Thanking the blogger for delivering such a beautiful content and keep posting the contents in upcoming days.

    Data Science training

    ReplyDelete
  16. Truly incredible blog found to be very impressive due to which the learners who ever go through it will try to explore themselves with the content to develop the skills to an extreme level. Eventually, thanking the blogger to come up with such an phenomenal content. Hope you aarrive with the similar content in future as well.

    Digital Marketing training

    ReplyDelete
  17. Tremendous blog quite easy to grasp the subject since the content is very simple to understand. Obviously, this helps the participants to engage themselves in to the subject without much difficulty. Hope you further educate the readers in the same manner and keep sharing the content as always you do.

    Data Science certification in Raipur

    ReplyDelete
  18. stunning, incredible, I was thinking about how to fix skin inflammation normally.I've bookmark your site and furthermore include rss. keep us refreshed.
    data scientist malaysia

    ReplyDelete
  19. I truly like your style of blogging. I added it to my preferred's blog webpage list and will return soon…
    https://360digitmg.com/course/certification-program-in-data-science

    ReplyDelete
  20. Your work is generally excellent and I value you and jumping for some more educational posts
    https://360digitmg.com/course/certification-on-industrial-revolution-4-0

    ReplyDelete
  21. Succeed! It could be one of the most useful blogs we have ever come across on the subject. Excellent info! I’m also an expert in this topic so I can understand your effort very well. Thanks for the huge help. auto glass near me

    ReplyDelete
  22. I have read your blog it is very helpful for me. I want to say thanks to you. I have bookmark your site for future updates. used motorcycles for sale

    ReplyDelete
  23. A debt of gratitude is in order for the blog entry amigo! Keep them coming. polaris dealer

    ReplyDelete
  24. Hi, I log on to your new stuff like every week. Your humoristic style is witty, keep it up hebergement web Gatineau Ottawa

    ReplyDelete
  25. Hi, I log on to your new stuff like every week. Your humoristic style is witty, keep it up hebergement web Gatineau Ottawa

    ReplyDelete
  26. I am constantly surprised by the amount of information accessible on this subject. What you presented was well researched and well written to get your stand on this over to all your readers. Thanks a lot my dear schwank patio heaters

    ReplyDelete
  27. You…are…my…hero!!! I cant believe something like this exists on the internet! Its so true, so honest, and more than that you dont sound like an idiot! Finally, someone who knows how to talk about a subject without sounding like a kid who didnt get that bike he wanted for Christmas. real estate broker chelsea

    ReplyDelete
  28. It is quite beneficial, although we think about the facts when it reaches this target yoga ville de gatineau

    ReplyDelete
  29. I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well. Thanks. Terrasverwarmer

    ReplyDelete
  30. Very rapidly this site will be famous among all blogging and site-building users, due to it's good articles Water Softener System

    ReplyDelete
  31. Nice post. I learn some thing more difficult on various blogs everyday. It will always be stimulating you just read content using their company writers and use a little something from their site. I'd prefer to use some with all the content in my small weblog no matter whether you don't mind. Natually I'll provide you with a link on the internet weblog. Many thanks sharing Power Washing

    ReplyDelete
  32. Nice post. I learn some thing more difficult on various blogs everyday. It will always be stimulating you just read content using their company writers and use a little something from their site. I'd prefer to use some with all the content in my small weblog no matter whether you don't mind. Natually I'll provide you with a link on the internet weblog. Many thanks sharing Sanitation Services

    ReplyDelete
  33. Thanks for taking the time to discuss this, I feel strongly that love and read more on this topic. If possible, such as gain knowledge, would you mind updating your blog with additional information? It is very useful for me baltimore video production

    ReplyDelete
  34. This is such a great resource that you are providing and you give it away for free shipping from china to usa

    ReplyDelete
  35. I havent any word to appreciate this post.....Really i am impressed from this post....the person who create this post it was a great human..thanks for shared this with us buy website traffic

    ReplyDelete
  36. 나는 지금 당신의 웹 사이트에서 몇 가지 기사를 표현했고 당신의 블로그 스타일이 정말 마음에 듭니다. 즐겨 찾는 블로그 사이트 목록에 추가했으며 곧 다시 확인하겠습니다. 토토사이트

    ReplyDelete
  37. I can set up my new idea from this post. It gives in depth information. Thanks for this valuable information for all 토토사이트추천

    ReplyDelete
  38. I have read your article couple of times because your views are on my own for the most part. It is great content for every reader. 메이저놀이터

    ReplyDelete
  39. It is a good site post without fail. Not too many people would actually, the way you just did. I am impressed that there is so much information about this subject that has been uncovered and you’ve defeated yourself this time, with so much quality. Good Works! 먹튀안전사이트

    ReplyDelete
  40. I was just browsing through the internet looking for some information and came across your blog. I am impressed by the information that you have on this blog. It shows how well you understand this subject. Bookmarked this page, will come back for more. freelancer malta

    ReplyDelete
  41. This helps them in learning new things about Python and how to solve various issues that may arise while coding. data science course in india

    ReplyDelete
  42. 체중 감량에 성공하려면 외모 이상에 집중해야합니다. 기분, 전반적인 건강 및 정신 건강을 활용하는 접근 방식이 가장 효율적입니다. 두 가지 체중 감량 여정이 똑같지 않기 때문에 대대적 인 체중 감량을 달성 한 많은 여성에게 정확히 어떻게했는지 물었습니다. 출장샵

    ReplyDelete
  43. Cool stuff you have got and you keep update all of us 토토사이트

    ReplyDelete
  44. I truly like you're composing style, incredible data, thankyou for posting. 먹튀검증업체

    ReplyDelete
  45. I like this post and there is obviously a lot to know about this. I think you made some good points in Features also i figure that they having a great time to peruse this post. They might take a decent site to make an information, thanks for sharing it to me Keep working, great job!
    Braces in Bangalore

    ReplyDelete
  46. The Data Scientist Certification Programme is one of the most comprehensive Data Science Course in Bangalore. It is specially designed to suit both data professionals and beginners who want to make a career in this fast-growing profession. Over 6 months, students will learn key techniques such as Statistical Analysis, Regression Analysis, Data Mining, Machine Learning, Forecasting and Text Mining, and tools such as Python and R Programming.

    ReplyDelete
  47. Terrific post thoroughly enjoyed reading the blog and more over found to be the tremendous one. In fact, educating the participants with it's amazing content. Hope you share the similar content consecutively.

    Data Science Course in Raipur

    ReplyDelete
  48. web design, I must admit that your post is really interesting. I have spent a lot of my spare time reading your content. Thank you a lot! and if you need then contact us!

    ReplyDelete
  49. Hi would you mind letting me know which webhost you’re utilizing?

    I’ve loaded your blog in 3 different browsers and
    I must say this blog loads a lot quicker then most.
    Can you suggest a good internet hosting provider at a reasonable price?
    Many thanks, I appreciate it!

    Also visit my web page ::대구오피


    ReplyDelete
  50. Really Liked the information you have provided. I have an article relaed to it. I was searching about it
    Download Teen Patti Gold Mod

    ReplyDelete
  51. Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon. Big thanks for the useful info. matka result

    ReplyDelete
  52. Greetings! Very useful advice within this article! It is the little changes that will make the most important changes. Many thanks for sharing 코인카지노

    ReplyDelete
  53. This is a good post. This post gives truly quality information. I'm definitely going to look into it. Really very useful tips are provided here. Thank you so much. Keep up the good works. 룰렛

    ReplyDelete
  54. Nice post. Thanks for sharing! I want people to know just how good this information is in your blog. It’s interesting content and Great work…

    Data Science Training in Hyderabad

    ReplyDelete
  55. This post is very simple to read and appreciate without leaving any details out. Great work!
    data science course in malaysia

    ReplyDelete
  56. Very good message. I came across your blog and wanted to tell you that I really enjoyed reading your articles.
    Business Analytics Course

    ReplyDelete
  57. it is Very interesting posts, it is really informative to read these posts Keep posting. whatsapp mod

    ReplyDelete
  58. It's like you've got the point right, but forgot to include your readers. Maybe you should think about it from different angles.

    Best Cyber Security Training Institute in Bangalore

    ReplyDelete
  59. You have completed certain reliable points there. I did some research on the subject and found that almost everyone will agree with your blog.


    Best Ethical Hacking Institute in Bangalore

    ReplyDelete
  60. Thanks for sharing this blog with us. Really awesome blog, information and knowledgeable content. Keep sharing more.
    Data Science Training in Hyderabad

    ReplyDelete
  61. SEO service companies vancouver wa provides most trusted SEO service. Loved by hundreds of clients.

    ReplyDelete
  62. Simple Linear Regression is a logistic method used to find out the relation between a single input variable and an output variable when both variables are continuous. To learn more about Simple Linear Regression start your Data Science course today with 360DigiTMG.


    Business Analytics Course in Jodhpur

    ReplyDelete
  63. With decision making becoming more and more data-driven, learn the skills necessary to unveil patterns useful to make valuable decisions from the data collected. Also, get a chance to work with various datasets that are collected from various sources and discover the relationships between them. Ace all the skills and tools of Data Science and step into the world of opportunities with the Best Data Science training institutes in Bangalore.


    Data Science Training in Delhi

    ReplyDelete
  64. This blog is informative for us. We provide services in town.travel blogs

    ReplyDelete
  65. This blog is informative for us. We provide services in town.[url=https://www.wonderroadtravel.com/gallery]single mom travel blog [/url]

    ReplyDelete
  66. Data science is a process of analysing raw data collected from different sources to make useful data; join Data Science Training in Chennai at FITA Academy to learn more about data science.
    Data Science Training in Chennai

    ReplyDelete
  67. I really liked the way you are delivering the content. I am just waiting for more updates from your site. Enhance your understanding and excel in Class 12 Physics with Ziyyara Edutech's comprehensive online tuition program.
    Book A Free Demo Today visit Class 12 physics online tuition

    ReplyDelete