Work Life

AOL and Software Development and Work Life16 Oct 2007 07:09 pm

This sounds much stranger than it actually is, and every day I am less wary of saying it in public ;-) I am a Master of Scrums.1

The software development group I now work in did agile development in a kinda-sorta, half-assed way for about a year or two (during which time I happened to be their customer). Then about six months ago, a while after I joined the team, we decided to get serious about Agile — in particular, Scrum.

There were plenty of obstacles. The biggest was the fact that AOL has in recent years taken a very process-heavy, waterfall-like approach to product development. It was antithetical to developing software in an agile way.

Earlier this year our execs brought in an outside firm to do some Scrum training. Their two-day course — which I highly recommend — was a great foundation for us to build on. We found it most beneficial when all members of a team — product management, development, QA, project management — attended training together. This gave us all a common reference point over the coming months. On the whole, this has been a very useful and productive effort, and I hope Agile is here to stay at AOL.

Within the Scrum framework, there is some room to tweak things to suit your own organization. Half a year in, we’re still correcting mistakes, making adjustments, and learning what works best for us. One surprise: we find that best practices (which I’ll talk about in a future post) vary from one team to another.

So what are the benefits we’ve seen using Scrum?

One of the biggest benefits is that you get very rapid feedback on how things are going. Four days into a sprint,2 and you will know straightaway if you are having problems. We often find ourselves making adjustments after a week of work. In contrast, when using our older waterfall approach, we often wouldn’t become aware of a problem until we’d missed a major milestone, typically weeks or even months after a project had begun. Now these checkpoints and moments of awareness happen regularly: for small issues, on a daily basis, and on a larger scale every few weeks when we complete a sprint.

It is also much easier to make rapid adjustments. Even if you strictly follow the rules of Scrum, you can often get sudden requests into your next sprint, which means out to production within two to four weeks. In some cases we cheat a bit and incorporate smaller changes into the ongoing sprint.

If you are good about using a tool3 to keep track of tasks and progress, you gather invaluable data about your team: your development velocity, your hidden inefficiencies (like overhead for build management), how good you are about identifying tasks ahead of time, how well you estimate task effort, and more. Each team is unique, and the sprint histories provide valuable characterization of the team as well as a record that informs you going forward.

After you get a bit of history under your belt, you become a pretty good judge of the scope of a project — “this thing feels like three, maybe four two-week sprints…”

Finally, here’s a really cool benefit of Scrum: if you are diligent about knocking out the most important and highest-risk stuff first, you sometimes finish a project earlier than expected. On a couple of projects, we’ve suddenly realized that, while we still had a backlog of items (“stories”) to complete, none of them really needed to get done now, and hey — we’ve got more important things to do somewhere else. “So, umm, folks, I think we’re actually done here…” Man, imagine that!

Where have we had issues?

It definitely hurts when you try to do too much at one time (although this isn’t unique to Scrum!). We’ve had most success when a small team (1 to 3 developers) can be dedicated to a project until it is complete. If you find yourself spreading individual developers or QA people across multiple projects, you may have productivity problems. On the other hand, some of the ancillary team members, like the scrum master, product manager, or operational folks, seem to do ok handling multiple projects. As scrum master, I find that I do okay with up to 4 or 5 simultaneous projects (although that means I’m not getting a whole lot else done during the day!).

We’re still learning how to most efficiently manage projects where multiple remote teams are working on different components of a single, larger effort. This scenario — which is increasingly likely going forward — definitely presents challenges. For us a daily scrum of scrums improved communications a lot. And one definite win when teams span multiple locations: small- to medium-scale video conferencing; daily visual interaction can make a huge difference when working with remote teams.

It’s also worth noting that some of our groups that do heavy integration work, and others that work on projects with very infrequent but very large deliveries, have struggled with Scrum. I’m not directly involved with those efforts, though, and not particularly familiar with their situations, so I won’t comment further.

What else?

Agile methodologies were popularized in the software development world (though they really originated elsewhere), and are especially suited to the sorts of work in which the path to the desired solution is not clear at the onset.4 So where else could this approach be useful? I’ve worked in several fields that, in retrospect, could benefit from following a Scrum-like methodology (or maybe just using parts, like the daily scrum for keeping on top of things and getting frequent feedback):
* Physics. Namely, in the design of, and preparation for, medium- and large-scale physics experiments.
* Electrical engineering. For hardware and firmware design, especially where one is dealing with leading-edge technologies.

What do you think?


1 If you’re not familiar with Agile / Scrum, wikipedia has a good overview. The Google video of Ken Schwaber’s talk is also very useful.
2 We’ve been happiest with short sprints: typically two weeks, and sometimes less.
3 We’ve experimented with a few tools, with varying degrees of success. In a future post I’d like to discuss this in some detail.
4 Contrast this, for example, with building one of a dozen houses, or building a widget in a factory…

AOL and Work Life16 Oct 2007 12:56 pm

It’s been a sad and strange day here at AOL, with some good folks let go :-( We’ll miss you guys.

Nuff said… I think it’s time for a beer.