Lean Software Development

A book review

I've just finished reading Lean Software Development, a book written by Mary and Tom Poppendieck that introduced the lean practices used in the manufacturing world to software development as an evolution of Agile methodologies.

Lean software development in general is still a new process, but the theories are solidly anchored by the manufacturers that have made it work for them. Toyota has introduced the kanban system and the "decide as late as possible" technique. 3M and the way they empower their teams. The Theory of Constraints which is used, incidentally, at the company I work for.

What Mary and Tom did was to translate every good practice to the software world. But because of the novelty of the approach, there are very few examples in the book of the lean techniques as applied to software development. Some time has passed, however, since the book has been written and today the reader can find a plethora of examples dispersed on the Internet, via developer blogs everywhere. Alt.Net in particular is a group whose members adhere to agile and lean practices and would be a good place to start searching.

I consider this book an excellent introduction to lean software development and highly recommend it to anyone looking into this methodology wanting to know more about it, or wanting to implement it in their workplace. The language of the book provides an easy read and most real life examples are taken from Mary and Tom's personal experiences at the various companies they have worked for and are always very insightful.

What is lean?

Lean development is the elimination of waste.

That's it... Really! You can stop reading now.

Seriously, however, lean development is about using tools and techniques in accordance to seven basic principles in order to eliminate waste in the value creation process.

The value creation process is simply how your company (or department) creates value. In our case, how we develop software is the process we are talking about here. Software is the value we create for our customers.

One way we can visualize this process is by using a value-stream map, which depicts all the steps taken when designing, developing and deploying software, along with the time taken to go through each step and the time that has been lost just waiting for something (feedback, testers, DBAs, etc.)

Then we apply lean methodologies to try to shorten the value-stream map as much as possible without sacrificing on quality. All and all, it means having shorter software creation cycles.

Let's talk a bit about each of the seven lean principles, as discussed in the book.

Eliminate waste

Well I've rambled a little about this already. The main thing is to learn to see waste, however. One way to do this is to search for anything that isn't coding and analysis and see if it adds value to the product. If it doesn't, then it is waste.

The following is considered waste:

• Partially done work
• Extra processes
• Extra features
• Task switching
• Waiting
• Motion (how far do you need to walk to for an answer to a question, for example?)
• Defects

Another way is to map a value stream and try to reduce as much as possible the time spend on steps that don't provide a lot of value.

Amplify learning

Knowledge is key: knowledge about what the customer wants, knowledge about the system, knowledge about the constraints. We need to generate that kind of knowledge as fast as possible in order to make the best decisions we can.

Having short release iterations will generate customer feedback faster. Also, constraints might become more obvious sooner. Your code needs to speak volumes about intent and design without the need to spend time trying to decode it (a major waste!)

Try to do some set-based development also to generate as many ideas at the same time as you can.

Decide as late as possible

This is one of the things that Toyota does that makes them stand apart from the competition (and why they still make a profit and don't need their government to bail them out!)

Delay the critical, hard to undo, decisions until the last possible moment when delaying any further would reduce the value of the product or would eliminate important alternatives.

Decouple your modules, anticipate, code multiple options knowing that one might be chosen and the others discarded later. The price of losing the options in this case would be lower than not doing them at all. Use parameters, interfaces, abstractions. Encapsulate variation and do not repeat yourself.

Deliver as fast as possible

Use a Kanban system and code in short iterations. Enough said!

The key element in this principle is throughput: the rate at which value is created. The book mentions the theory of constraints, which deals with this in a manufacturing context where the entire shop cannot go faster than the slowest machine. The same can be said about software, which can't be written faster than the slowest process in the value-stream map. Everything must then be subordinated to the constraint since any extra effort by a non-constraint is waste. I suggest you read The Goal if you haven't already. It is an excellent book.

Empower the team

Teams are more efficient when managers give them free reins over their processes. Most successful companies have seen change happen within their walls from shop employees, not managers.

At 3M, for example, new inventions come from such employees, who are then expected to lead the effort to manufacture them!

Allow leaders within teams to emerge and influence others. Allow master developers to take charge. And make sure your team members keep improving on their skills. Set them free!

Build integrity in

Integrity is how the software behaves. Is is consistent? Are all the parts acting in synergy with one another? How is the user experience? This principle helps you answer those questions.

Use refactoring and testing to maintain a solid architecture and make sure you spend time designing a good user experience. The happier the user is, the less effort you'll spend down the road.

See the whole

Finally, focus on the whole. Do not spend time doing local optimizations like making the testing department faster when it is not the constraint of the system. Be weary of local measurements like lines of code per coder because you'll be tempted to improve on them (local optimization) in isolation of the big picture.

Instead, focus on you value-stream map! Work on reducing the amount of time it takes for an equal amount of value to be created. When you change a process, go back to this map and make sure you didn't make it worse, even though a local area might have improved.

Has throughput increased? Have operating expenses been reduced? Has the amount of work in process decreased? All three must be true.

Conclusion

Of course, there is much, much more being discussed in the book. But what I have written in this post should get anyone started with lean or should be a good reminder of the key points of the book if you have already read it. I know I will refer to this often in the future in my own quest to implement lean techniques with the rest of my team!

Activity Modeling and the Bag

This is the third and final part in this series. You can read the first two posts here:

• The Bag and the Kanban, part 1
• The Bag and the Kanban, part 2

On our last day at Kaizenconf, I participated in an action type of session where our aim was to figure out what we could do about Activity Modeling, a design method. After reviewing it extensively, we decided that we would all try to fit it into a kanban pull system and blog about it.

Activity Modeling

In essence, Activity Modeling is a technique that allows a designer to create a set of actions and operations that different users, or personas, can do using a certain software application. It involves creating files on different personas to keep track of their goals, motivations and values using, among other methods, non-directed interviews where the main question is never asked directly.

Then, using a very specific activity statement, actions are defined. From the example that was given to us at Kaizenconf, the activity statement could be: "Providing patients acute pain management services." Now an action could be: "Admit patients into the acute pain management system." Finally, operations are determined for each action: "Specify patient demographics," for example.

Our group decided that our action plan would be to incorporate Activity Modeling to Jef Newsom's Kanban technique, which I depicted in the first post of this series.

In retrospect, this seems easy enough, at least on paper! The first thing to do would be to make sure we have enough information about our main users, or our personas. This is a very important step, as it has a direct link to the "People" column in our Kanban board.

Filling up the BAG

So we have our Goal, and our personas, which we can translate to our Kanban board. Using what we know about them, we can write down their personal goals. A doctor wants to be able to edit information and wants to know what is going on at all times. A nurse might just need the history of the patient, or just what the current health care procedures that need to be performed are. Activity modeling helps us know all of this.

Our bag used to be full at this point, but let's go ahead and add a new column at the end: the Actions column. This is a further refinement on each personal goal, normally a step we were reserving for the S.A.F.E. column in the Kanban. However, since we are still in the bag, actions remain at fairly a high level of abstraction, which is good since we might have the user helping us defining them.

S.A.F.E. scenarios benefit from Activity Modeling

Now that we have refined the bag, the designer and developers can work together to fill up the S.A.F.E. column at the beginning of an iteration. In Activity Modeling, operations are defined for each action and it just seems to me that this is exactly what we are doing by creating our S.A.F.E. scenarios. Our success, advance, fail and error scenarios are operations that have been classified!

This is what really struck me in our session. Activity Modeling benefits from the S.A.F.E. scenarios idea by having its operations classified and our Kanban benefits from Activity Modeling for filling up the Bag. There is an almost perfect synergy between the two methods and it has enormous usefulness potential for developers and designers alike.

From there, developers can simply pull operations into their iteration and can keep going with the rest of the Kanban.

The Bag and the Kanban, part 2

You can view part 3 of this series here.

In my last post, I depicted the results of a workshop I attended at Kaizenconf called Driving Towards the Goal. Jef Newsom, our facilitator, had set up a little experiment  which we all voluntarily and happily participated in.

As a whole, the experiment simulated the creation of software from Goal to Code, each step being a column in a kanban board.

Even though it was a very enlightening experience, we quickly discovered that the further along you went, the longer it took to specify work items. For one, each work item became much more specialized the closer it got to being coded. Second, as they became more detailed, we needed a lot more work items to cover the previous ones.

In terms of workflow, you can think of it like a sandglass. The thinnest part, where only a few grains of sand can flow through at a time, is the coding part. The sides of the sandglass are time and resources constraining your team and the distances between the sides is how fast the task takes to perform.

Of course, specifying the goal can take years of efforts, but in terms of software development, since it is usually defined in advanced (I hope!), it is the fastest step.

The Bag

So how does one manage the flow in such a system? Well the idea came to me that the kanban in the previous post could be separated in two. The first part focuses more on the creative aspect of defining the problem, the solution, the people involved and their goals. Here, chaos can rule. People can jump in anywhere and add items to the board, prioritizing and moving stuff around. This is what I call the BAG, which happens to stand for BAck loG. It also makes me think of a bagpipe, where the player (the stakeholder) fills up a bag with air, which is then pressed into the flutes to produce a sound.

The Kanban

The second part of the kanban is where the more technical stuff happens. Here, work items actually have to be pulled from previous columns before any work can happen. This discipline and organization is important in order to maintain a good flow. Remember, since coding is the constraint, it must be protected at all costs. Any disruption in this part (called "kanban" from now on) will cause a drop in throughput.

The reason that the S.A.F.E scenarios are outside the cloud in the picture above is that I believe the entire team should be working on them at the beginning of every new iteration. This will give every member of the team a general understanding of what is going on in the project.

Playing the bagpipe

To play the bagpipe, you start by filling up your bag with air. Then, using your arm, you compress the bag so that it pushes just enough air through the pipes to make the sound you want at the volume you want.

Filling up our bag is fairly easy. You get your stakeholders together and use your favorite method for liberating the creative juices of every people in the room. When things have stabilized a bit, the developers can come in and pull a couple "features" from the personal goal column in the bag in order to create their first iteration.

It is then absolutely important that the stakeholders leave what the developers are currently working on alone during an iteration. They can come back to the bag and play with work items, but what the software team is working on is sacred, at least until the end of the iteration.

At the end of every iteration, when the stakeholders have a chance to try out their newest release. Then they will have a chance to introduce changes in the personal goals. Each change can then be pulled from the bag by the developers, perhaps with a higher priority than other regular features.

As this process advances in time, stakeholders will probably find themselves asking for change during iterations. This is fine, as long as the change doesn't affect what the developers are currently working on. Changes then go in the bag and await the next pull.

Now, I've never played the bagpipes but I'm pretty confident that that what I've described further up is approximately how they work. In the same way, I've never implemented this process, but is sure looks nice on paper! I've also been called lots of names, but as a programmer, being compared to compressed air flowing through a pipe is a first, I must admit.

This process has a lot of potential and I'd like to keep defining it. Unfortunately, my current work conditions prevent me from experimenting on it in the short term, but rest assured that as soon as I can claim ownership of a project, I will definitely use the "Bag and the Kanban" model and see how far is takes me. In the meantime, I must realize that it is just an idea.

In my next post, I will use what I've learned in a session given at Kaizenconf on activity modeling to go into more details about how to fill the bag.

The bag and the kanban, part 1

You can view part 2 of this series here.

I have been exposed to a high level or radiations last Friday at Kaizenconf, where I was busy wrapping my head around the latest and greatest thoughts of our community.

In a particular session that morning, I was listening to Jef Newsom, who was trying to set up a small experiment about a new twist on the kanban pull system he had thought out. We were supposed to form a line and one by one, moving towards the right, fill up up the kanban board.

The idea behind this process was to focus on the personal goals of the users, as described by user stories.

Let me start by showing you what the kanban board looked like towards the end of the workshop. This is what had been setup on the wall:

 

The goal was given to us by Jef and written in the "Goal" square:

Elect the President

Not too bad, huh?

Our line was formed from a couple daring volunteers and we stood on the left side of the board. Every minute, we were supposed to move one square to the right and add a post-it note in that square, effectively pulling from one item in the previous square.

 

The Kanban

Here is a quick explanation of the board.

People

People in the system are protagonists (your typical users,) antagonists, benefactors and victims. Antagonists are people who might want to abuse the system. Benefactors are the users who will benefit from your application, and victims are those that might be hindered by it.

Going with our previous goal of electing the president, two post-it notes were placed rapidly in this square: Valery the voter as a protagonist, and a Presidential Candidate as a benefactor.

Personal goals

The personal goals of our people are basically user stories, written in the form of "To <why>, a <who> <what>."

For example: "To enter a sales order (why), a clerk (who) brings up the new sales order panel (what)." Boring, yes, but it shows who needs to do what in order to accomplish their personal goal.

As you might imagine, multiple user stories can be thought up per person and pretty soon, we had:

• To vote, a voter must submit a ballot
• To show a user that he/she has voted, a message will be displayed
• To make sure a voter can't vote more than once, something unique about them will be stored with their ballot
• To verify the result, an official will be able to create a paper trail

S.A.F.E. Scenarios

S.A.F.E. stands for Success, Advance, Fail and Error. Each of these types of scenarios describe what could happen to each personal goal in our kanban.

Let's take our first personal goal: "To vote, a voter must submit a ballot." We can determine the following scenarios:

Advance: The voter has successfully submitted a ballot. This type of scenario is what is required to happen in order to reach one or all of your success scenarios. They are consequently very important.

Success: All voters have successfully submitted ballots. These scenarios must happen in order to accomplish the goal "elect a president." Yes, they are also very important.

Error: It is not clear what a voter had in mind when he submitted his/her ballot. Data is missing. Errors slow you down. They are obstacles standing in the way to your success scenarios. You will have to find a way to deal with error scenarios to maximize your advance.

Fail: Ballots are lost. Ouch... the election must start over. You have failed. Game over!

Interactions

For each scenario described above, we then have to define one or more interactions in our kanban. This is where we finally get to be technical.

What objects are going to be used, created or destroyed? What kind of data are we dealing with? When does it get created? Where do we save it? Are there any relationships that we will created or modify? Any properties we need to keep track of?

Jef was pretty vague here, mentioning the methodology that HP uses to define interactions. I guess this section is up to you. I don't quite know yet how I would deal with it.

Test/Specs or UI

These two are in parallel. From the interactions you can write tests, spec out a design or sketch out your UI. Here again, you can insert your favorite methodology.

Code

And finally, you code! Take a test, a piece of UI or a spec and code it!

In retrospect

Something became clear very fast: the amount of information that is put into the kanban is exponential as you move to the right. Towards the end, we had one goal, two people, four or five personal goals, two or three SAFE scenarios for each one of those personal goals, etc.

What's more, when one of us got to the coding square and actually had to write some code, the whole line screeched to a halt because coding takes longer than coming up with the rest. Coders are the constraint of this system.

Take a look a what the board looked like at the end:

This is an unmanageable process without creating more rules to better run the kanban. Also, what happens if you re-prioritize personal goals? Well you would then have to restructure your S.A.F.E. scenarios, your interactions, your tests, your UI, clearly a complicated task.

It got me thinking... There's got to be a way to make this more efficient, at least in my IT environment. Could we separate this kanban in two? Can we have a backlog from which the developers can pull at their own pace while the users, managers and project leaders refine the backlog?

In the end, what I want to achieve is to protect the coder, our constraint, without restraining the creative juices that fill up the first half of the kanban.

I'll go into more details in my next post: The Bag and the Kanban, part 2.