Deloitte DevOps transformation delivered at a global logistics company has been saved
Deloitte DevOps transformation delivered at a global logistics company
Best practices and lessons learned
This blog post will elaborate on the proven Deloitte DevOps transformation approach discussed in the previous blog post and give an insight into the best practices and lessons learned of a DevOps transformation project delivered at a global logistics company. As part of the transition delivery phase in the Deloitte DevOps transformation approach, a key objective is to have a DevOps Centre of Excellence (CoE) in place to drive the transformation. Today, Donald Pondman (Deloitte lead CoE) and a DevOps Engineer at the client share their key best practices and lessons learned from the DevOps transformation journey at client X.
By Marlies Quekel
Go directly to
The client started their DevOps transformation to help achieve its mission of delivering innovative, secure, standardized and centrally managed sort solutions to Operations. This DevOps transformation has been executed using the Deloitte DevOps transformation approach visualized in below picture and explained in previous blogpost.
As part of the DevOps transformation and to continuously improve delivery and way-of-working, each increment lessons learned are discussed in a retrospective by determining what to start, stop and continue doing. These lessons learned are captured in a DevOps playbook and acted upon. There are around 140 lessons learned from the transformation, most are already successfully addressed and improved by the teams. Let’s ask Donald and the DevOps Engineer, what their key best practices, challenges and lessons learned per phase of the transformation are.
At the start of the project, two project tracks are being set-up. One project track will be designing the DevOps target operating model and transformation roadmap for the organization. The other track will be implementing, scaling, or improving the CI/CD technology in the organization. The CI/CD stream, also technology track, is the enabling team in the transformation to quickly showcase value by accelerating and de-risking deployment. At the start of the project, the technology track will be defining, describing, and designing the MVP CI/CD pipeline. Requirements, technology stack, architecture and tooling should be considered in designing and defining an MVPipeline, a roadmap towards MVP and a comprehensive CI/CD pipeline. The client’s DevOps Engineer indicates that the interplay between organization and technology worked very well in the DevOps transformation at client X. He says: “Engineers have a tendency to prepare technical solutions but sometimes need to be reminded to take the time to explain the benefits, address cross functional dependencies and share their solutions with a wider group of colleagues.”
As key lessons learned in the start of the project, the DevOps Engineer says: “The DevOps and CI/CD principles should be thoroughly explained to all stakeholders to create a solid base for the DevOps transformation. Engage all stakeholders that will be required to contribute to the DevOps transformation early in the process. Keep them updated with progress and findings to create a solid support base for the DevOps transformation and implementation in the organization. Also, encourage a joint ownership (client & Deloitte) for the creation of deliverables to ensure acceptance by the client team(s).” Donald additionally emphasizes: “During the start of the project the project tracks should not solely focus on designing the DevOps transformation or implementing technology in the organization, but also explain to all stakeholders what’s in it for them. A traditional way of thinking is so embedded in the culture of the organization that more time should be allocated to explain what DevOps is and get people onboard in the DevOps transformation.”
Furthermore, it is important during the start of the project to create a detailed schedule of all meetings and expected timelines for deliverables, so a clear overview can be presented to stakeholders and work can be divided over the team. Also, consider that unforeseen activities may occur that could result in unplanned work. Demos on the progress are much appreciated and strongly enhance understanding of the project as well as internal support.
First objective during the transition delivery phase is to transition the DevOps TOM project stream into a DevOps Center of Excellence, the driver of the transformation. The second objective and best practice of the Transition Delivery phase is to transition the technology stream into the CI/CD team; the first DevOps team.
Centre of Excellence
The client’s DevOps Engineer elaborates: “The Centre of Excellence aims to land the DevOps way-of-working in an organization and supports communications around the DevOps transition for DevOps teams and the CI/CD project.” Donald adds to this by elaborating: “The Center of Excellence is the facilitator and coach of the transformation. The Center of Excellence replaces traditional governance with DevOps ceremonies and designs the Target Operating Model that lands the DevOps way-of-working in the client organization.”
As a best practice, the Center of Excellence will fulfill multiple roles going forward:
- Single point of contact for any DevOps related inquiries
- Partner for the teams to help with the design and implementation of the DevOps way-of-working
- Owner and single source of standardized DevOps way-of-working materials. The CoE will describe CI/CD capabilities and processes to enable the client to operate the uDS pipeline
- Facilitator of the DevOps transition for teams and project. The CoE will setup DevOps teams and enable them with CI/CD tooling to speed up the delivery OT software applications
- Liaison for the broader adoption of the DevOps transformation
To help new DevOps teams, the Centre of Excellence provides various services depending on the phase in the DevOps transition. These different services are visualized in below picture.
In the DevOps Target Operating Model advice is given on how to setup new capabilities and processes using DevOps concepts. The DevOps Playbook captures all material delivered as part of the DevOps journey. The DevOps Engineer says: “The DevOps playbook is good for reporting and keeping track of what has been done in the transformation already. It consists of main deliverables and lessons learned per increment. It was however a challenge to maintain the playbook since it is quite an administrative task.”
With issue & opportunity management, the CoE provides an overview of all issues & opportunities and proactively ensures the issue or opportunity is followed up. Donald indicates: “It’s nearly impossible to cover all topics in the Operating Model design, gaps will arise. These gaps need to be closed, and this is a responsibility often within the scope of the CoE.”
With training & coaching, the CoE onboards new DevOps teams to the DevOps way-of-working, it facilitates training & knowledge sharing on DevOps capabilities and way-of-working and provides on the job DevOps way-of-working support. Providing these services is experienced as most fun by Donald and the DevOps Engineer. The DevOps Engineer says: “By understanding DevOps, training team members and sharing knowledge about the principles, miss alignment that usually occurs in IT projects can often be prevented.”
With regards to communication, the CoE develops the communication strategy for the DevOps transition project and provides communication in the form of a State of DevOps report on how the DevOps transition is progressing. Donald and the DevOps Engineer indicate that the communication within the project and department was good. Donald says: “Communication outside of the DevOps teams could be improved.” DevOps Engineer: “Could have created more awareness about the DevOps transition and get more teams involved in the DevOps way-of-working.”
The CI/CD team is key to make the DevOps implementation successful since it is the team responsible to get the technology in place for service delivery in the software delivery lifecycle. It is also the first DevOps team in the transformation since the technology stream transitions to the CI/CD team. The CI/CD team gets onboarded in the DevOps way-of-working and learns from hands-on labs for creating pipeline capabilities. Donald says: “The CI/CD team is an enabling team that can really showcase the value of DevOps. It is the team that enables accelerated and lower-risk software delivery; typically showcased by deployment automation.” The CI/CD teams leverages a phased pipeline transition approach towards a first service in the software delivery process. In the end state, multidisciplinary DevOps teams share end-to-end responsibility for the development, delivery, and operations of service in the software delivery.
There is a wide variety of tools available to explore CI/CD concepts. The CI/CD team should invest time in the tool selection (best-of-breed vs. best-of-suite). Also, ensure parity between cloud and on-premises implementations (e.g., Azure DevOps). For example, a strict on-premises policy requires dedicated hardware infrastructure to be set-up.
Coach & scale
During the coach & scale phase, new virtual teams are onboarded, coached and handed over to scale the bi-model organization. In the coach & scale phase, Deloitte takes on the role as Coach. The client is in the driving seat of the transformation, where the Product Owner as well as the team members are all client employees. The role of Product Owner in a DevOps team comes on top of the existing work. DevOps Engineer indicates: “coaching by Deloitte creates some headspace in the minds of Product Owners and facilitates the DevOps transformation. Besides, the Scrum Master activities are initialized and set-up by Deloitte before these are handed over to a client Scrum Master. There is already a structure and handhold for the DevOps ceremonies in place to facilitate a smooth transition of Scrum Master activities towards a client Scrum Master.”
Result of this approach is that the organization has various DevOps teams, each at different maturity levels and stages in the DevOps transformation journey. Donald indicates as a key challenge here: “DevOps teams are onboarded incrementally. With these changes in teams and in the organization, responsibilities are shifting. Also, with this shift it can happen that responsibilities fall between two stools, and it is unclear who is responsible for certain activities. It was a challenge to continuously identify who is responsible for what.” As a best practice, Donald says: “It is important to have ongoing discussions about the roles and responsibilities of different teams. During the DevOps transformation at client X several roles & responsibility workshops have been organized”.
Showing value using DevOps metrics and to provide insight into how DevOps has been adopted in the organization was a key challenge, since there was a lot of resistance when introducing this. Why there was such a big resistance was inconclusive. DevOps Engineer does mention: “If a team is setting performance metrics/KPI’s, and this team does not achieve a target, they can have the feeling that they might be held accountable for not achieving. While the delivered work and trend might have been extremely positive a target might have been set with too much ambition and that should not reflect negatively on the team. The fact KPIs are not always interpreted as aspirational targets but sometime as minimal standards could explain the resistance.” It could be that the introduction of these DevOps metrics came too early in the DevOps transformation and adoption journey of client X. Being more DevOps mature now, could shed a completely different light on the willingness to show value using DevOps metrics.
The roll-off phase takes place once an organization is mature enough to continue working in the DevOps way of working and to onboard new DevOps teams and initiatives themselves. During the roll-off phase, coaching efforts are stopped once the organization is mature enough to continue themselves and transformation responsibilities are handed over.
Donald indicates as a key challenge during the roll-off phase at client X: “People could have a different vision on when the transformation is finalized. It could be the case that the transformation is not yet done, but people are done with it.” The DevOps Engineer confirms this by saying: “Certain teams are not yet onboarded in the DevOps way of working, while preferably they should be. There will probably come another wave of onboarding new DevOps teams, and this is for now in hands of client X itself. As our organizations Operating Model is constantly changing, it helps to have an external party as Deloitte in the future again as advisor and soundboard.”
As a best practice and lessons learned, DevOps Engineer and Donald emphasize that everything stands of falls with management support. DevOps Engineer: “Client X has a lot of excellent technical people that execute their work flawlessly, but if everyone works on their own island ultimately efficiency will stagnate. Therefore, constantly adapting the operating model and identifying which profiles fit well in which teams to fulfill gaps is crucial in becoming the most efficient organization. The Center of Excellence advising and supporting management with these tasks is the key to success.”
Want to learn more?
Want to learn more about the Deloitte DevOps transformation approach and curious how this can be implemented in your organization, please feel free to reach out to the Deloitte DevOps practitioners mentioned at the bottom of this blog post.