Introduction of Scrum with obstacles

26.09.2019 – Lev Stejngardt

Our internal and independently working team switched to Scrum in order to do justice to the spontaneity of their area of responsibility. For implementation they wanted help with organization and project management, which should be done by a Product Owner. They got a Product Owner with development background and invasive tendencies — me. Instead of playing the well-behaved secretary, I started to establish agile thinking and took over the backlog. This had to cause irritation!

The initial situation

In the following I would like to describe the assumptions I have encountered, how I approached them and what the current status is. But not so fast. Let's take a look at what the tasks of the team are. The core area includes:

  • Maintenance of existing infrastructure
  • Implementation & maintenance of CI pipelines for other teams
  • Support of other teams during deployment
  • Research in the DevOps area
  • Development of team-internal tools
  • As well as small administrative tasks, maintenance and rights management of the infrastructure

As you can see, the team covers a wide range of tasks. On the whole, all tasks can be divided into two disjoint groups:

  1. Plannable tasks, about which one is informed in advance, like updates, deployment, research, development
  2. Unpredictable tasks that force us to react, like system crashes or hardware problems

Tasks in both areas were completed on demand. Even scheduled tasks were not properly timed by the product owners of other teams. Occasionally, tickets were created, but the whole process was simply too cumbersome. Because there were no sprints to get those tickets done in an orderly fashion, the board became increasingly confusing and did not reflect what each team member was working on. The overview of the developments within the team was lost. It was simply impossible to set a clear focus or even read the current development status.

The unplannable tasks are part of the team's work, and the nature of the events causes the most problems: if the hard disk of a critical system is running out, we have to react immediately. Tasks already started must be interrupted. Concentrated work is rarely possible. Literally a disruptive work environment.

As if that wasn't enough, they also had to organize themselves. The team was well aware of some problems and wanted support in organizing their work. The expectations of the product owner were correspondingly simple:

  • he has to protect the team — which is the job of the Scrum Master
  • he should not question the way the team works — thus the PO would have no say in the processes
  • the team continues to determine what it is working on — the PO has no control over the backlog
  • the team continues to determine how it should work — this is the only demand that meets the requirements of Scrum

As can already be seen, they wanted a "toothless" Product Owner. Protect us from the evil others, we dictate the tickets to you, don't interfere in our affairs, period! This was not my idea of the role of a Product Owner.
I started using agile methods and poking around thoroughly in the usual workflow. Naturally, I encountered initial skepticism and resistance.

Here you can see the sticking points of the implementation based on some assumptions, and how I met them.

Assumption: "Backlog? We don't need it - we can remember it"

Whoever tries to transcribe spoken language will find that the cognitive abilities of humans are amazing. Transforming grammar-exhausted gibberish into meaningful statements is probably one of the greatest achievements of the human brain.
Nevertheless, very few people can immediately memorize a backlog of 200 items including all acceptance criteria, cross-references, deadlines, project contexts, business needs, priorities and requirements. This was also the case here, so there was a memory aid: a text file with notes.

When a task was completed, it was necessary to discuss what to do next. The team then looked at the text file together and decided which task the member should do next based on perceived urgency.

This often led to discussions about what was considered more useful or more urgent, and also to disagreement about what the individual key points actually meant. To make matters worse, these conversations were conducted several times, because each time you had to figure out anew what a key point actually means. As one can surely imagine, conversations of this kind are not always conducted with the necessary objectivity.

In one of these situations I pointed out to my colleagues that gruelling conversations of this kind are avoided with a clean backlog. You save the time to reconstruct the items, have all the information in one place, make fewer mistakes due to misinterpretations and have a clearly defined prioritization for the next tasks.

The team followed my suggestion. I transformed the item list into tickets and in cooperation with the team we jointly defined acceptance criteria, deadlines, business value and dependencies. I prioritized the items in the backlog by business value. Which brings us to the next problem.

Assumption: The Product Owner does not know what it is all about

With an inward-looking view, this assumption is inevitable. Here comes a person who has no professional skills and begins to decide which tasks are more important than others.

Why should an outsider, who has no technical understanding of the subject matter, be able to judge better what to do next? And on what basis does he make his decisions?

Items are prioritized based on business value and the associated implementation effort. As product owner, I have access to the business value of the individual items from the departments. This information is collected by me and it is my task to obtain this information. What I lack for adequate prioritization are the implementation efforts. The team can provide me with this information. Therefore the Product Owner is the only person who has all the necessary information to decide on an appropriate prioritization.

As an outsider, I am subject to a certain lack of emotion regarding the backlog, which makes it easier for me to make rational decisions. Don't get me wrong: Every team member always does what he or she thinks is right at all times and always gives 100 percent, but the intentions of individuals are not always transparent.

So it happens of course that exciting tickets are processed first. Of course it's great that we find interesting what we do - fascination is our most important source of motivation. Nevertheless, it is also part of our bread and butter to get unloved tasks behind us. Since I don't have to do the work myself, I am free of these emotions and can concentrate on business value.

The Product Owner is also just a person you can and should talk to. Discrepancies in prioritization can be clarified in conversation. At the same time, the product owner sharpens his understanding of the subject matter in discussions with the developers. The developers, in turn, understand the goals of the stakeholders better. In other words, the more frequently the discussion is held, the more clearly the team understands the needs of the stakeholders and the Product Owner familiarizes himself with the topic.

Finally, the Product Owner represents the interests of the product. In our case, the team itself is also a stakeholder and makes suggestions for further development, but they are only one of the stakeholders and represent only one of the perspectives. Through discussions with different stakeholders - and the team is of course included in this - I got the necessary know-how to be able to assess the business value appropriately.

Assumption: We do not need focusing

It can quickly become very monotonous to work on a topic over several weeks. Unwelcome tasks enrage me personally after only a few hours. People often approach the problem with distraction: What else is there to do? Unfortunately, the constant change of focus brings some unpleasant effects with it.

For us, the change in focus was reflected in the area of internal test coverage. There were tests for setting up our CI clusters, but the test coverage was nowhere near as comprehensive as we would have liked.

Since the topic was not worked on continuously, some components were tested, others only partially and some not at all. This was a very unpleasant starting point, since one could never be sure that one would not accidentally destroy the whole cluster when updating one component.

Apart from the drawbacks of lacking test coverage, there is also a psychological component: you never finish anything completely. What remains is the feeling of not having completed something, which is tiring in the long run.

The team has acknowledged this circumstance. I decided to make a wish list. The items on the list were clustered. Together we were able to agree on an item that would deliver the highest business value from an internal perspective and also add significant value to the work of the other teams. Of course, we still have to keep working and keep the business running, but when we have time, we focus on our common goal.

Assumption: We do not need a ticket for every task

Sometimes you see a small problem that you would like to fix quickly and you are sure you can fix it quickly. Why write a ticket for something where creating it takes longer than actually fixing it? It is only a small change, after all...
The attitude is understandable, but it raises three problems.

First, what is a small problem? When does a "small" problem become one that requires a ticket? And what exactly is a "small change"?
These questions cannot be answered with the help of measurable variables. Consequently, it would be at the discretion of an individual team member to decide whether or not to fix a problem. However, this decision-making authority is not the responsibility of the team members, but solely that of the product owner.

Second, if I make changes without a ticket, how will the rest of the team and the product owner be informed that I have made these changes? Of course, you can hope that the person remembers all the small changes the next day and will report it to the Daily. However, I don't consider hope to be an adequate means of controlling projects.

Third, even the 15 minutes spent on an insignificant problem is missing elsewhere. So why work on a small, insignificant problem when there are still many items with higher business value in the backlog? We have stuck with the idea that every activity requires a ticket. This has noticeably increased transparency within the team.

Assumption: Our work cannot be planned

As already mentioned at the beginning, the work of the team can be divided into plannable work and spontaneously emerging work. If there is a failure on an existing system, the team has to react - yes, this part of the work cannot be planned.

But the other and much larger part of the work can be planned: What further developments do we want to make to our tools? Which teams will soon need a CI pipeline or deployment? What updates do we need to do soon? What project phases are the other teams in and where will our support be needed? Which infrastructure do we want to renew next? This part of our work is predictable and can be scheduled in sprints.

The accusation of unplannability came not only from my own team, but also from other teams: "What happens if we need something immediately - do we have to wait a whole sprint until you help us?" or "We can't give our customers response times of one week!"

Many people felt deprived of the right to storm into our room and give us commands. With the help of some simple conversations the waves could be smoothed out: Of course, we won't abandon anyone, and if there's a fire, we'll put it out - but not every knocked over candle requires the fire brigade to be dispatched.

So we introduced week-long sprints. After a few sprints, the team accepted that a certain aspect of their work - or as I like to say, the much larger half - can be planned.

The tension of the teams we were working with subsided quickly. They realised that we were still responding swiftly to important inquiries.
At the same time, our demand for predictability has also led to better organization in the other teams. Other POs asked themselves more often whether the work had to be done immediately or whether it would be enough for the next sprint. Thus, our process fidelity has also led to discipline in other teams and thus had a positive effect throughout the company.

Lirum larum, Summa summarum

The team considers the backlog to be an important tool and shows great interest in the ticket quality and in prioritizing the backlog. They often try to extend acceptance criteria or inform me if I miss something. Tickets that are not clearly formulated are either criticized or rejected by them. This also applies to tickets from other teams, which in turn improves the ticket quality in the other teams.

Focusing on one topic has strengthened the team spirit, as we feel that we are working towards a common goal. The steady progress on our current project makes their work visible and creates a good atmosphere. Full of anticipation we are waiting to roll out the feature.
Support requests from other teams will be rejected if no ticket has been created for it. We regularly remind ourselves if someone tries to work without a ticket, and even the most stubborn have realized that a certain satisfaction is felt when you can move a ticket to the done column after two days of focused work. Every member has internalized that tickets lead to transparency within the team and promote knowledge sharing.

Shortly after starting my work, I conducted an anonymous survey in the team to record the situation. I repeated the same survey 3 months later.

The result was clear: the team feels that the work has become more predictable. Whether this is solely due to the changes I brought about cannot be clearly determined, but it is obvious that they have at least contributed to improving the situation.

The team has familiarized itself with the process and accepted it. Now they independently propose changes to the process and try to make their work more pleasant. This can definitely be interpreted as a good signal. The team has been working in sprints for four months. All in all one can say that the team, although they had many prejudices, is very satisfied with the process and the tools.

Epilogue

Whether Scrum as a process model was the right decision can certainly be questioned. However, Scrum offers enough freedom to function even under difficult circumstances. Presumably similar success would have been achieved with other agile approaches. Most assumptions were not directed against Scrum, but against any form of management. But the quintessence remains the same:

Yes, you can make tasks that seem unplannable manageable to a certain extent. After all, little planning is better than no planning. Yes, with very little order and a bit of discipline you can create a lot of transparency, which benefits everyone involved.