The Myth of the Interchangeable Programmer

Over my years of watching software projects, I have identified two mathematical formulas that guide much of the industry. These formulas are:

Software Produced = Number of Workers x Hours Worked.

And:

Cost of Software = Number of Workers x Hours Worked x Hourly Cost Per Worker.

I call these the “Software Management Formulas” or SMFs. Understanding these formulas makes much of the absurdity in the industry understandable.

Whether consciously or unconsciously, the SMFs are widely used in the industry. It seems the higher one goes up in the managerial chain, the more likely one is to find the SMFs being employed.

The fact that the SMFs have no relationship to reality has proved to be no obstacle to their widespread adoption. However, understanding the SMFs is the key to understanding the misguided industry clamor over H-1B visas and offshoring. The value of offshoring and H-1B visas flows from flawless logic when the SMFs are treated as axiomatic.

Here is real life example of the SMFs at work. A large company originally estimated that it would take about eight developers to complete a project within their schedule of six months. This project was relatively uncomplicated but they did not have programmers available internally. So they used a “bodyshop” to staff the project.

The project was behind schedule almost as soon as it began. The team failed to meet deadlines right from the start. As the project progressed, the gap between schedule and completion increased steadily.

Like many in the industry when faced with a project running behind schedule, this company’s management applied the SMFs to the problem. The SMFs predict that increasing the number of programmers will increase the amount of software produced. So the company went back to the “bodyshop” for more people. The bodyshop naturally was all too happy to add more developers billing by the hour.

With the old deadlines now past, the company revised the project schedule. Unfortunately, even with the larger project team they were unable meet the pushed back deadlines.

As deadlines continued to pass by unmet, the company’s management kept returning to the SMFs and the SMF’s kept telling them to add more people to the project. This process of schedule delay followed by adding more developers continued until the project had over 30 programmers on it.

The company persevered and, eventually, they finished the project. It took about 4 times longer than planned to complete and used about 4 times as many developers as planned. The project was over budget correspondingly.

Now look at the very same project from the programmer viewpoint.

The company placed the entire development team in a large conference room. Each time a telephone rang, it disturbed the entire project team. With eight developers in the room, each one was interrupted eight times as frequently as he would be in a separate office. Each visit from an outsider to discuss project requirements with one person disrupted the entire project team. Whenever two people casually mentioned NFL scores, the whole team became involved in a discussion on football. The physical environment was so chaotic that nothing could get done.

As the management added more developers, the chaos increased. Twice as many programmers meant twice as many telephone calls, with each call disrupting twice as many people. The number of conversations, both business and non-business related, similarly increased. Doing software development under these conditions was simply impossible.

The company kept creating more confusion by adding programmers to the room, and increasing the disruptions geometrically. They stopped adding people to the project only after a Vice President came down to the conference room with a yardstick and confirmed through actual measurement that there was no room to fit any more people.

For reasons completely unrelated to productivity, the project team was eventually moved out of the conference room. With the work environment problem largely solved, the project the project started to move forward, though at a slow pace. Now saddled with over 30 developers, weekly status meetings alone took up an entire workday.

The project should have taken one-third the time it did and could have been done in that time with one-quarter the number of developers.

The problem here is that the SMFs incorporate a number of flawed assumptions. The first of these assumptions is that programmers are fungible. The SMFs assume all programmers will contribute roughly the same amount to a project and that all programmers are interchangeable. If you have a development team of eight, the SMFs assume the individual contributions will look something like this:

The reality is that there are huge variations in skills among programmers. In a typical software development project, the individual developer contributions usually look something like this (note that Arnold is the most productive by far):

The wide productivity variations among programmers have been extensively written about but few organizations take notice (For more on Howard, see Grady Booche’s Managing The Object Orient Project, p. 189. See also PeopleWare by Tom DeMarco and Timothy Lister, p. 48). In the real world, one can add 30 programmers to the project and see no positive improvement whatsoever.

In the project described previously, data collected showed that six developers were doing over 90% of the work on the project. Even more striking was that just one of those six was doing 40% of the project’s work on her own. The other 80% of the developers were doing nothing other than to slow down the 20% who were doing the work.

Another flawed assumption in the SMFs is that they assume there is no overhead associated with increasing the number of people on the project. This overhead cost due to increased project size is a central theme of Frederick Brook’s classic “The Mythical Man Month.”

Each person added to the team has a cost in overhead time that gets applied to all of the other team members. Status meetings are a good example. A project team of 6 people can have a weekly status meeting over lunch. With 30 people, it becomes an all day affair as in the previous example. In observing this project, the ultimate irony was that at the status meetings each person who contributed real work could describe what he did in a couple of minutes, while those who did nothing typically spent ten minutes describing what they did not do.

This brings us to the 8-Man Rule:

The maximum number of people that can be on a discrete project without incurring significant overhead is about 8.

The 8-person figure is a general rule of thumb figure derived from experience. Once discrete projects start to get larger, then the value of additional people one the project drops precipitately.

The 8-Man Rule does not preclude out larger projects but rather dictates how they have to be approached. Obviously there are projects that will require more than 8 programmers in order to be completed within a reasonable period of time. Large projects need to be subdivided into smaller discrete units in order to conform the 8-Man Rule. Larger projects require much more up front work and are far less tolerant of “winging it” than smaller projects. In any event, the 8-Man Rule is in direct conflict with the SMFs.

There are a number of factors the SMFs do not take into account at all. In the project above, the physical work environment was one of the most significant factors in delaying the project. Putting this particular project team together in one room was a disaster. While the “team room” concept works extremely in certain situations, it is highly sensitive to the composition of the team (in regard to both personalities and individual contribution) and the type of project.

In this case the “team rooms” was an overcrowded tenement created by accident rather than through a plan (simply the result of lack of office space); the project was not of a nature suited for a team room; and the people mix was not a good one for working in an intimate environment.

This project was an unusual case where the company was willing to spend whatever it took to get the project finished. A new concept comes into play when using the SMFs as the basis for project management and cost is a consideration.

The Principle of Software Cost Reduction (PSCR) flows directly from the SMFs. The PSCR states that the key to reducing Cost of Software is to reduce Hourly Cost Per Worker. The PSCR can easily be derived from the SMFs above by mathematical substitution, giving:

Cost of Software = Software Produced x Hourly Cost Per Worker.

The PSCR “proves” that for any give software project, the cost is linear based upon the Hourly Cost Per Worker. Thus the key to reducing software development costs is to reduce the Hourly Cost Per Worker. To reduce the hourly cost per worker, one needs cheaper programmers.

Industry’s current focus on cheap programmers flows naturally then from the PSCR. Due to differences in living standard among countries, there is a disparity in wages between those of North American/Australian/European workers and other parts of the world. This creates pools of cheap labor just waiting to be tapped by industry countries. The problem is how to get at it.

For that there is offshoring and H-1B visas.

The slavish devotion to the SMFs is why we never see the industry lobbying for measures to improve software development productivity. We never see industry call for government funding for studies on best software development practices. We never see industry call for research funding for software development tools designed to produce software more reliably and efficiently.

Instead, we simply see repeated calls for access to more cheap foreign labor, often with mindless platitudes, such as calls to staple green cards to every diploma, and the tired canard about the state of K-12 education. Such is industry’s bankrupt state of leadership. The result is an industry whose symbol is “the Blue Screen of Death”; an industry whose probability of delivering a project on schedule and on budget is lower than the cure rate for most cancers.

The SMFs and PSCR provide management easy-to-understand solutions for developing software. Without the SMFs and PSCR, the key to reducing software development costs is to increase individual developer productivity. To increase developer productivity one has to know the process of software development; one has to be able to take on the corporate facilities bureaucracy to create work environments suitable for software development; one has to take on the HR bureaucracy to get the right people and reward them; one has to negotiate with top management the project tradeoffs among cost, time and features.

This takes highly skilled managers who either know the intricacies of software development or who can delegate to good advisors who do. Under the SMFs, and PSCR, the key to reducing software development costs is to find cheaper programmers. Any anyone with a high school education can understand the cop out of the SMFs.

After reading about the project described previously, you are probably wondering why the company did not apply the PSCR and send the project offshore? The answer is simple: They already had – and it was a disaster. The project was part of the recovery from an offshoring nightmare, the aspect of offshoring that rarely gets written about. Still, even after the disaster of offshoring, the company, like so many others, retained its devotion to the SMFs.

Plus ça change, plus c’est la même chose.

John Miano is an author and expert on the software industry. He may be contacted via Colosseum Builders.