I am often asked how I evaluate success in my projects. Since much of my work in in dealing with new technologies for which there are no benchmarks to measure against, I often have to resort to less tangible measurement frameworks. The old school ‘on time, to budget, to specification’ just doesn’t apply.

Measurements can be:

• Political volume – how well have you tamed the tigers in the company and gained support for the new directions you are proposing
• Thought leadership – what new ideas have you been able to promote, along with the benefits and risks each may have attached
• Impact analysis – this relies on just pure gut instinct as to determining how the organisation capability may have to change to accomodate new technologies, and the new skills that must be learned
• Maket Impact analysis – the success of many new technologies rely largely on how well educated the general market is. How ready are they to adopt and comprehensively use new features. A common mistake many technology vendors make is to overstuff their devices with too many new features at once – users can only evolve and adopt at reasonable rates, and sometimes it is better to delay features for future version roll outs.

The reason I stay away from the perils associated with time, cost and spec measures is that in much of the work we do – nobody knows or can know. So there is no point wasting the clients time in putting together an outdated form of measurement. Its often a ‘lets go out there and take a look’ approach.

I prefer to work closely with the client to determine what they are trying to achieve, and agree more appropriate measures. It is more important that you have close and trusted communcation with the business and keep them updated on findings and progress. This is very much an agile approach akin to that used by software developers. You just take one step at a time and re-evaluate at the end of each step.

Many project managers are not confident in this approach – and that’s fine. There are many projects that can be managed using known defined frameworks. For me – I love the challenge of new frontiers and am not intimidated by admitting I just don’t know. To me, its a more honest approach that pulling numbers out of the air that are not appropriate and most likely will not fit the end result.

I have read project simulation guidelines that suggest that the executive level for the accuracy of estimates should be within +300% and -50% of the actual project cost. What a great waste of time that estimate is from a project management perspective. But it does seem to satisfy the C-level as to a broad range of expectation. And it sure beats quipping over whether a component in a multi million dollar project is going to cost $50 or $55.

Overall, it can be very hard to pinpoint that intersection between the fear of project managers in not hitting their projections and the intellectual acceptance by executives based upon a broad range of potential outcomes. The great dividing point is political – so my advice is, deal with the politics and get on with the job. Leave the numbers off the table until you have a much better idea of what you will be dealing with. And this may be 6-12 months down the track.