Saturday, February 26, 2011

Bad Smells

It had all the bad smells. This new project. The business was frustrated. They were convinced IT was incompetent. The project team warned me, "the business does not know what it wants". They were both right.

The prototype was built with offshore resources. They had completed two eight week drops. It worked well enough to demo as long as you stuck to the script.

The offshore team said "it is nearly done". The business had this endless list of things that "must be done". Were they working on the same project?

The current project team was making the rounds. Showing the demo. Creating the impression that the app was nearly ready. Everyone liked what they saw. They could hardly wait to see it deploy.

Meanwhile, the development team had a long list of defects. The more they fixed, the more there was to fix. Sixteen weeks of development followed by eight weeks of bug fixes, with no end in sight. Performance testing had not yet begun. Great, "almost done", yet endless defects, incomplete scope and no one knew if it would scale. This was going to be fun.

Release Retrospective

It deployed, then scaled and the feedback was positive. The product owner was delighted and the users loved it. This was the first major application to deploy nationally and enterprise wide in several decades. Our business partners held celebrations. The team was proud of what they have accomplished.

Many things were done right. Most importantly, we had the product owner in the room and the team was seeded with embedded coaches. Although many of the team members were employees, the iteration manager, analyst lead, development lead and test lead were skilled experienced agilist from a consulting firm known for its agile practices. The team used story cards, big visible charts, two week iterations, show and tells and iteration planning meetings, Pairing and continuous integration were used along with primitive automation of unit and functional tests. We could run a full regression test every three days.

However, the project had a near death experience. The agile team, living in a waterfall world was disconnected. Although it delivered functionality, it did not scale. It would barely run in the production environment for longer than 5 minutes.  How would it ever support the score of thousand of users planning to use it.

Mind you, the production environment was new. It was not understood and much had to be learned to make it scale. However, that did not matter to our business partners. If we could not figure it out, our users would never benefit from the team's efforts. We were on the verge of being yet another failed project for scaling reasons.

The team learned that for large applications, another kind of testing was required. Performance testing (PT). PT should not happen at the end as planned. By then it was too late.  This uncovered a dysfunctional attitude inside the team room. The team believed they delivered functionality and the infrastructure team delivered scale. Of course the infrastructure team believe it was the application that would not scale as their infrastructure was fine. We had the classic corporate finger pointing stand off.

The embedded coaches, the agile experts, did not prepare the team for this blocker. The team was very internally focused. It had no connection to the enterprise. It was clear that the team succeeded using agile ... but also nearly failed.

Nonetheless, we did succeed. We experienced bliss in our own ignorance. Yes, we practiced agile, but just barely. Yes we were able to build and fully deploy an enterprise release, but only through heroics and a swarm of engineers busting through a little understood infrastructure and round the clock performance testing and tuning staying just ahead of the deployment schedule.

We had much to learn, yet we knew we were on the right path. These lessons did not go unheeded. Our product owners had big plans. There was so much more they wanted to deploy. It was round two for the team.

Meanwhile, a second new enterprise application was in the works. It was having problems. We were asked to start a second agile team to help. We were pumped. We were ready. So we thought.

Tuesday, February 15, 2011

Success Gains a Toehold

Your team is a island. Off to the side. Away from the process police. After all, the process police are busy enforcing compliance on the big important projects. Never mind that the important projects are late. Over budget. Their business partners frustrated.

Your business partner is in the room along with everyone else. The room is abuzz. Noisy. People look busy, engaged. What is going on? How can people work like this? How can they concentrate?

But it is working.

The business partners knows what has been delivered. Likes what she sees. Has plans for more features. Is excited.

The team's reputation spreads. It is big news when a project is released on time. When the business is delighted.

How is this possible? After all, they are not using a real process. They don't have a project plan. There are no artifact that have been signed off. They don't even have detailed design documents. And what are all those things on the wall? "They just sit in a room and get stuff done" as an executive put it.

"This is fine for simple projects", says one defenders of the enterprise process. "It will never work on a real project", says another. It is the corporate immune system kicking in.

It is hard to deny success. You can't argue with excited users and business partners. Success stands out on the rough sea of failed projects.

The team has delivered. The business partner is happy, They have more work lined up for the team. There is even talk about starting another team. No one wants to stop what is working ... except the process police.

The challenge has just begin. The team is practicing agile. Doing so in a waterfall world. They're so few in number, just about 20 people.  What chance do they have against the system? How will they ever survive?

We shall see.

Monday, February 14, 2011

Enterprise Eco System

Complexity! Lots of complexity. Not all of it necessary. Much of it because of partially deployed systems that didn't scale or cut project funding. We see business processes built around IT assets instead of assets supporting business function. Duplication everywhere.

Mind you, the enterprise eco system is a result of both IT and business misfiring. Business does not understand its workflow. IT insist that workflow comply with its architecture. It is a symbiotic relationship. Not a healthy relationship.

Then there is the IT methodology. A methodology designed by IT for IT. It does not serve the business. But, we lie to ourselves. We say the process helps us deliver. Where is the evidence?

Does our business understand our methodology? Are they advocates of it? Do they insist we use it? Hardly.

Look at our relationship. The business says IT cost to much. Takes to long to deliver. Our systems don't work. Our business partners are not happy. They continual look for cheaper ways to ... fail.

Can the business exist without us? No. Are we thought of as strategic partners? No. Are we able to leverage technology to provide competitive advantage? How often?

We don' t realize our potential. We have excuses. "The business does not know what it wants". "Our systems and processes are complex". "We don't have the tools we need". "We can't find good people or are not willing to pay for them". In frustration our business attempts to control cost. They no longer think of us as adding value. We become a cost center. Something that must be controlled.

Then there are the silos. Funding is allocated to each business unit. They each want their own IT. Each defending their own funding source. Each with the number one priority. Each wanting to control their own eco system. Seldom reaching across business units to define enterprise business processes. All focusing on their business. Is it any wonder we have duplication?

Sound familiar? So how do we get to "what is" to "what is possible"? Who can untie this Gordian Knot? The good news is it is solvable. That success is the engine that powers the transformation.

It starts small. It builds. It iterates.

Sunday, February 13, 2011

Happy Birthday Agile

The agile manifesto was created ten years ago today. Thank you
Kent Beck
Mike Beedle
Arie van Bennekum
Alistair Cockburn
Ward Cunningham
Martin Fowler
James Grenning
Jim Highsmith
Andrew Hunt
Ron Jeffries
Jon Kern
Brian Marick
Robert C. Martin
Steve Mellor
Ken Schwaber
Jeff Sutherland
Dave Thomas

Why is it so hard?

Why do so few enterprise IT organizations delight their users and business partners? Why is it so hard to build good software? Even for enterprise IT organizations with mature defined processes.

To answer the question, we need to look at the classic way we build software. We start by setting up an adversarial relationship with our business partners without realizing it. Conventional wisdom says we need to define what we will build, up front and then control scope. We make our users and business partners commit by signing a requirements contract. Do you know of many marriage that were saved once the lawyers were hired? Bad contracts don’t bring collaboration, bad contracts bring conflict. We turn our users and business partners, our strongest assets into adversaries.

Look at the challenge we put in front of our users and business partners. We ask them to define a new system at great detail, up front when we all know the least about the system. It is clear that out business processes and systems are complex. Regulation and competition make it so. What chance do we have of getting the requirements right? Is it any wonder our  partners are hesitant to sign a requirements contract? Is it any wonder the requirements often are vague and imprecise?

Once scope is signed off, we finalize our estimates. We use the estimate to set a schedule and budget. This is our plan. All when we all know the least about the system. Our plan is a really a big guess. But is looks so precise. All those tasks and dependencies. Such folly.

Is it any wonder the users want to change the requirements and scope? To argue about what a requirement means? That the schedule runs long. That the budget is overrun. That defects are too numerous to count? Finally, we descend to arguing about what is and is not a defect.

Then, we do a post-mortem. Inevitable someone says, "we need to take more time to plan better"! Never mind that it already took too long to define the scope. That at some point an executive said, "enough, start coding". Yes, the obligatory post-mortem exclamation, "we need to plan better!". What a belly laugh!!!

Where did this idea come from about planning up front and locking scope? Is it because IT was tired of being beat up? Users change their minds. They do so because they learn as we build software. However, we don't know how to incorporate what was learned without impacting schedule and budget. So, we insist on freezing scope. Think about it. We don't allow ourselves a way to incorporate what we learned as we build. Is it any wonder the users says, "that is not what I asked for"? What they are saying is "never mind what I said, that is not what I need".

We need a serious change in mind set. We need to accept that conventional project management wisdom got it wrong. How many times do we need to fail before we can conclude our processes and systems are broken?

Lets accept the fact many details are not  knowable at a project start. That our attempt to control scope  results in unhappy users and unmeet business need.

Let's accept that fact that phasing our activity to plan, design, code, test and release in a single cycle over the life of the build doesn't work. We have proven that to ourselves, our users and our business partners, again and again. Yet, how many of us have this formula for failure codified in our enterprise IT methodology? Experienced developers can feel the churn in their stomachs every time they travel down this failed path.

Say it out-loud, "our processes are broken". Let's stop whispering "under commit and over deliver".  Let's stop covering our hides and start focusing on value. Let stops using our broken processes and systems and expect different results. Let’s stop acting insanely.

What is possible?

Do you believe it is possible to release software with zero defects? How about delivering again and again with zero defects?

Do you believe it is possible to delivery software on time and on budget? Not once but every time. And with zero defects.

Do you believe it is possible to cut the schedule and cost in half? Cost and schedule cut in half when compared to thousands of IT projects over the past 20 plus years. Again, with zero defects.

Do you believe it is possible to delight your users and business partners? Delight users because you built the right software, and you build it well. You built it with zero defects.

What do I mean by zero defects? I mean before the software is released to production
    • all unit test pass - with high code coverage
    • all acceptance test pass - with high scenario coverage
    • all user acceptance test pass - as defined by the users and business partner
Then, when the software is released to production and tested, no defects are found. But, most importantly, the users and business partners are delighted. Furthermore, should any defects be found during the warranty period, they are minor and very few in number, less then the fingers on one hand. A pretty tall order.

Do you know of any enterprise IT organization that has accomplished such a feat? Even once? How about over and over again? How about over 90% of the time? I do.

This blog will detail how it is possible to deliver the right software, built well, consistently at a sustainable pace. How it is possible to have the users and business partners rave over what was delivered? To witness the user and business community ask that YOU build their software, and ask you again and again, release after release.

Saturday, February 12, 2011

Enterprise IT Sucks ... Enterprise IT Rocks

Enterprise IT sucks. Not because the people are bad. It's our system. Deming said, systems and processes cause failure, not people.
This blog logs my experience fixing the systems and processes in enterprise IT. It is about agile management and engineering practices brought to scale with lean principles. When you fix the processes and systems, when leadership supports a healthy culture, enterprise IT rocks.