Earned Value EVM and Level of Effort - Yes You Can

Oscar Wilde would have made a great program manager.

"Nowadays, people know the price of everything and the value of nothing" - Oscar Wilde.

We saw in an earlier post that you can do EVM with Fixed Price after all. In a true fixed price contract, the number of resources and working arrangements are not under the buyer's control; the vendor just gets the job done somehow.  You can still work EVM into it (see the prior post at the link above).

It is ironic that so much of the contract work for the government, which advocates very strongly for using EVM has migrated very rapidly from the much-maligned cost basis to LOE. Why is this a problem?  Because it appears to be unaccountable.  The team comes aboard to ... well, to do what?  Whatever needs to be done, meaning whatever the contract monitor wants done, which of course is a personal services contract and would therefore be illegal (in government), which no doubt explains why so many of them are in place in Washington.  But there they are, and we have to deal with it.

People think you cannot use EVM with LOE because there are no real deliverables and the planned and actual costs are always the same.  The "official" solution is that level-of-effort activities don't contribute to specific work packages and should be held in a separate line of accounting from the work packages.  Then they just accrue costs as the calendar passes; but they do not earn value. They are part of the project cost baseline but not the performance baseline. This is a good way to account for project-wide overhead costs, which contribute more or less equally to all ongoing work packages (or impede them, depending on your point of view!)

What happens if the level-of-effort people are working on the actual deliverables?  You can't exempt 95% of the project labor from the EVM metrics.  For true LOE contracts, whether on their face or disguised as fixed-price contracts, the people just show up to work ... and do what?.  Since it is illegal to award an actual personal services contract, you don't often see one that actually says "just give us xx bodies and we'll tell them what to do". What you see instead is a "fixed price" contract that is really level-of-effort: the vendor is to deliver a defined number of hours (conveniently 1980) of various labor categories, with a statement of work that is either very vague or quite specific but will be completely ignored.

At a very basic level, if the vendor fails to provide the agreed resources (a common occurrence), then you can show a delta between planned cost and actual cost.  The philosophical problem here is that no backlog of work builds up: customers may be poorly served at the time, but there is no way to go back and serve them later when the resource does show up.  That, of course, raises the uncomfortable question of what value was lost by the work not done, and if you use that math then you have to extend it to question whether any much more value is gained from the work that was done.

As to the so-called "fixed price" contracts to provide labor-hours, that's no different at all from other cost-based contracts; there's just less paperwork needed to establish the actual cost.  Many contract monitors prefer not to crack the whip, or don't know how, but since it is nothing but a typical cost-based contract you can give guidance on the deliverables to be produced and the hours (costs) needed to do it.  If it gets done sooner or cheaper, you are ahead of the game; if it takes longer or more work, then you are behind the plan.  Voila: we have EVM!

Let's suppose that the contract does provide for a real LOE and real (if vague) deliverables.  The very latest thing in this arena is agile-type work (especially Scrum, the most popular brand) in which a team assembles on a near-full-time basis to do whatever work they can get done to deliver for each sprint.  Despite the popular impression that Agile is just a license to throw all oversight away, there is actually a rich body of knowledge on Agile metrics.  Just as with a true fixed price effort, it is entirely possible to establish intermediate milestones for longer-term efforts (such as a release) in which the earned value might be as simple as a completed sprint and may be more complex to address the planned and actual velocity.  an even more sophistical scoring method can address the velocity at which delivered features are meeting value priorities.

Just as with the true fixed price scenario, the important thing is to separate the concepts of "value" and "cost". 

Earned Value and Fixed Price - Yes You Can.

Earned Value Management is at the core of any project management framework as the method of integrating cost, schedule and scope to assess the progress of a project.  The framework has been remarkably stable over at least the past 40 years, from its earlier incarnation as the Cost/Schedule Status Report (C/SSR), and since it pretty much conforms to common sense [we can talk about Schedule Variance in another post], it is quite likely that the Pharaohs had something similar. Tens of thousands of project managers have been trained in EVM, and it is quite impossible to pass any certification test without knowing at least the basics.  And it is Federal law that any contract over $1 million must be managed using EVM.  Yet at seminar after seminar, certified PMs report that they are not using EVM.  In the public sector it is resisted with a passion that most people had assumed bureaucrats do not possess.

Sure, most people, even trained PMs, and all organizations vastly prefer to evade being held accountable. That's why we need objective reporting systems in the first place.  Let's just deal with the argument that EVM does not apply to fixed-price work (which most Federal contract work is supposed to be) or to level-of-effort work (which almost all Federal contracts are).  Once you take those two approaches off the table, particularly if you buy into the nonsense that a labor-hours contract is "fixed price" because the unit price per hour is fixed, there's not much left. But what "everybody knows" is quite untrue.  Saying that something cannot be done does not really mean that it is impossible, just that you want me to think that it is impossible so I stop asking you to do it..

The argument against using EVM for fixed-price work is that the cost is accrued only at the point that the delivery is completed, at which point the cost is always equal to the negotiated price (unless you are selling airplanes to the Defense Department).  Likewise with level-of-effort work, where people just show up and do whatever the client wants, there is no contractual relationship between the completion of deliverables and the cost of the service (which is the reason that at the Federal level such contracts are illegal bwah-hah-hah).  At, at a certain level of disingenuity, that's true.  But you don't have to set up this work in an unaccountable way, unless of course you are just trying to avoid accountability.

Assume for the moment that both the vendor and customer are interested in a quality product (or service) at a reasonable cost.  Do we really think that the vendors let their people just do whatever until a few weeks before the contract delivery period expires?     Of course not.  There are interim milestones and estimates of the work needed to achieve them; in fact, those had to be developed to come up with the proposed cost of the work in the first place.  Except for the very smallest increment of delivery, there is no reason at all that we cannot establish intervening milestones and the value associated with achieving those milestones.  That value does not have to be the costs incurred to date; doing so merely converts the work back to cost-based.  And please do not allow the vendor (or a lazy contracting officer) to set up straight-line progress payments; that, too, converts the work to cost-based.  What are you going to do when after the 11th month it finally becomes clear that the deliverable will not arrive on time?  Now instead of 100% of the payment (highly motivational), then vendor only has 9% of the total payment at risk.  Good luck pressuring them into timely delivery at that point.  You should determine the value of the milestones and pay for those deliveries only when they are complete and acceptable - and if they are late or do not perform as required, you don't pay for them at all.  Now you have deliverables that have schedules and a planned value (which, notice, does not have to be a planned cost-to-date).

You can get a simple template that allows you to produce an EVM graph and metrics in this manner (see the Solutions tab).

Tune in next time for a discussion of EVM in a Level-of-Effort situation.

Trust but verify - heavy on the verify

If a decision happens but nobody writes it down, does anybody hear it?
If a decision is published but nobody checks to see that it is being followed, will anybody follow it?

Why can't you see the glass as half-full once in a while?
We've got good people here. We told them what we wanted to do. They don't need to be baby-sat.

If everyone was in agreement we would just go ahead and do it.
The reason we have highly-paid executives to decisions is that there are conflicting points of view as to the best course of action.
Therefore, whatever they decide is going to differ from what a good proportion of the next layer or two in the organization would have done if left to themselves.

At the same time, most executives didn't get where they are by having no political instincts, which means that quite often they say things that they don't really believe because that seems like the proper thing to say at the time.
So it's entirely possible that a decision isn't really a decision, more like a pronouncement to kick the real decision down the road a while.
All very Byzantine.

So what do people do while trying to get on with life in their smaller part of the organization?
If the executives don't have the gumption or the stamina to make sure that their decisions are carried out, it sends a pretty strong signal that they don't care too much either.
Then the people who didn't agree with the decision in the first place won't be very deterred from giving it at best lip-service until they find out whether this initiative is anything more than the fad of the month.
The first sign that the decisions aren't really decisions is that the deciders don't really want anyone to know what they decided or why.
It removes their latitude later for changing their minds.
Of course, in the short run, it also means that it is more likely that the decision will be ignored.

If our governance decision authorities (whether boards or individuals) are serious about having their decisions respected and followed, then:

Write down what was decided in clear terms: Who will do what by when with what resources

Include enough information to understand why the decision was reached and any issues that were raised<\li>

Publish the decisions to those who are affected - at all levels. That way intermediate managers can't slow-roll or pocket-veto.

Follow up on the progress of the actions that were decided on

When it makes sense to do so, change a decision and publish the fact that it has been changed

Assuming that the PMs and/or staff have done due diligence before offering up solutions for the board to approve, then the approved course of action should be achievable.
If the intermediate managers can't or won't get things done, find someone who will.
Preferably someone who was rooting for the original decision. It will greatly improve the quality of analysis in the future.

SAFe - acronym or solution?

The Agile approach to project delivery has gone mainstream over the past few years.  Now the big question is whether Agile, with its focus on self-directing small teams, really can be used effectively in to produce solutions that are necessarily very large-scale and highly complex, with serious consequences for getting it wrong.  "People might get killed" would be a serious consequence.

Such work is often found in the public sector, and you can tell that a concept has already been mainstream for a while when the government starts pushing it as the latest thing.  As usual, the latest thing is also being pushed as the silver bullet that will make government programs super-efficient.  The truth is that government projects don't fail because of process - they'll get enough money thrown at them to overcome a whole lot of process inefficiency.  They fail because they are - well, complex and very large scale to the point that nobody really understands the whole solution to the point of being able to assess the consequences of a butterfly wing flapping on one side of the effort.

Most of the heavy-duty engineering processes are intended to mitigate the risks posed by those conditions by at least trying to document what is going on and making sure that everyone involved has a chance to put in their two cents as to potential impacts when an change is contemplated. But it takes time (and money) to write all this down, time to read and understand it, and time to comprehend the implications of proposed changes to it.

Discussions of "Agile at Scale" have been going on for several years, but one leading candidate approach to achieving that goal is the SAFe method (http://www.scaledagileframework.com/).  I have not studied it enough yet to say that it provides all the meat that is really needed (for instance, a comparison with DSMD would be useful), but it has some solid case studies and it is certainly attracting the right sort of attention. A lot of that may be due to the brilliant acronym for dealing with government offices because it addresses precisely the number one concern: can a system developed via an Agile process be considered "safe"?  "Safe" having many meanings:

• Will it grow to support tens of thousands or millions of transactions or users?
• Can we trust it in life-or-death situations (military, air traffic, fire or police dispatching, etc.)?
• Will it be proof against determined cyber-attack?
• Will we be able to use it and repair it many years from now when the current leading-edge technology is long obsolete?

And of course the most important question: "Will this go well enough to delivery that I won't be embarrassed by having championed doing something different?"

There is no way that a development project can deliver quickly if every aspect of the surrounding infrastructure (both physical and process) has to be debated at length is not re-invented from scratch for each project.  My own interest in the topic was stirred through being unable in several different agencies to get any pragmatic or pro-active guidance from enterprise architecture or from IT security; even waterfall projects would complete half their development work in the time it would take to get a review of the initial concept.

That simply won't work for an Agile approach. The enterprise needs to make key decisions at a high enough level that they can be defined and decided (even if they have to be changed later) so that the development teams can get on with providing those useful deliveries. And the enterprise needs to be protected from short-cuts taken by fast-moving but tunnel-visioned project teams.

My favorite quote in the SAFe materials is the idea that Enterprise Architecture must be treated as a first-class citizen.  To that I would add, "and enterprise architecture needs to earn its citizenship", by providing the patterns and guidance that project teams need - or agreeing to work with the front-running project to tweak its requirements to permit the adoption of whatever the team comes up with as the enterprise standard.

I don't know that SAFe really does address all the legitimate concerns of an enterprise, or whether it really only does so using the same processes that would also be employed by any competent organization even if it were using the waterfall life-cycle model.

Anti-maturity; When It Just Isn't Worth the Investment

How often, really, have organizations started down the roads of project management, governance, enterprise architecture, internal controls .. you name it, they've tried it; but after the dust settles, the initiative has failed and faded like so many before it.  And now, you, lucky you, have been named to lead the next new fad.

Let's face it, a lot of organizations manage in spite of themselves to ride market forces to a place where corporate resources can absorb a lot of mistakes.  For optical purposes they may have to satisfy oversight groups, regulators, shareholders etc. by trying out some new approach, but they don't see any terminal consequences of paying nothing more than lip-service to the idea.  You, however, will join the long list of casualties.

Is there a way to tell whether an organization simply isn't interested in the types of benefits that these initiatives might create?  Kudos to Kik Piney for developing the Anti-Maturity Model.  You can read his approach (which he says isn't fully developed but you will find it pretty informative just the same). The gist of it is that you are wasting time trying to climb off Level 1 of any maturity model if the overall organizational culture is going to actively undermine most of the key elements of any process or discipline.  Immaturity suggests that you want to get better but don't quite know how; anti-maturity means you don't really want to improve.

This model might be also be helpful for many PMs in career considerations.  As you work your way up the PM ladder, the sign that you have actually arrived is that great interview where the panel emphasizes how the organization has come to really value the benefits that some aspect of PPFM can bring, and how they need someone with your experience to really bring it to life.  On the strength of an hour or two, how will you find out whether this is really the job you've been working towards for years, or the beginning of a 2 or 3 year nightmare featuring broken promises, carer threats, and endless wrangling?  You'll live through it, and be the stronger for it in the end, but how much nicer it would be for all concerned if you could determine in advance of taking the gig. Maybe a working knowledge of the symptoms of anti-maturity would help ask some penetrating questions during this same interview.

Kik doesn't get into why an organization would choose to behave that way.  It's often useful to observe the behavior and respond based on a knowledge of how an actor will behave, rather than speculating (often incorrectly) about why.  Having said that, recall that people rise to the top of an organization because their particular skills and preferences fir the way that organization works; once they rise to the top, they pass those same predispositions onward. For instance, IT organizations should in theory be run by managers with experience running IT operations; but all too often IT operations are run by people with a history of seeking to be responsive to customers  and other stakeholders by making undocumented on-the-fly changes that bring the whole system down -- and then build their "team player" credits by coming in overnight to fix the problem that they would never admit to having created.  Then, of course, they are promoted because they always got the extra mile.  Then they read a book about how great ITIL is, or maybe an external audit forces it on them. Do you really think there is any chance of success? Do you think it is a coincidence that something as inherently obvious as the precepts of ITIL has had rather limited adoption? The whole scenario is anti-maturity at work.

Enjoy the read!  

Minute-o-phobia

In a separate post we discussed several different views on the appropriate level of accuracy to include in meeting minutes.  I also noted that even more prevalent than different ways of taking minutes is for a meeting to have no record at all.

We all know that spies and politicians do this, but surely the entire PMBoK is about transparency, consistency and making a record. Yet so many PMs are among those who seldom document a meeting (at least for public consumption; what goes into your little black book for self-protection does not count!)

Why so little practice of transparency?

• Some PMs perhaps think that taking minutes (and more importantly, typing them up) is a secretarial function and they're much too important for that. Bad thinking. If you put the meeting record together you can make sure that the important things are included and that the needed actions are assigned. I certainly wouldn't want to say that you can remember the meeting sort of the way you want to ... but it happens that way sometimes ... don't overdo it.
• I know I often forget, or maybe a day or two of back-to-back meetings overcome your good resolutions, and the next thing you know you are twelve meetings behind and what the heck, nobody will notice missing a week. Which may well be true. And the next thing you know it's a month.
• The committee holding the meeting doesn't bother to read the minutes, offers no corrections or clarifications, and never votes to approve them. Why bother? {Ahem. See the first bullet).
• The committee members don't want to be held accountable for their decisions.  Now we're getting somewhere. Is it that they really don't want anyone to know what they decided? Ask them how anyone is supposed to do what they decided if nobody knows what that is. If you're still not getting any sensible answers after going down this road, consider whether your organization is so process-phobic that trying to implement PPFM is pretty much a waste of time (see the Anti-Maturity posting)
• Actually it's more likely that the members are OK with having votes etc recorded but they don't want a record of who said what in the deliberations leading up to the vote. Well, all right. If that's the only compromise you have to make in implementing your governance process, you are well on your way.

Here's another old favorite: our team is small and everyone knows what's going on. Why waste time on administrative overhead?
Hmm. Ever heard of turnover? Ironically, the logic is almost the reverse: the tighter your group, the harder it is for a new person to come up to operating efficiency. It's amazing how many processes that "everyone knows" turn out to be known in a different way, or not at all, by almost every member of the team. When a new person comes on board, it is of enormous help to be able to take a few hours reading through an abbreviated life history of how we got to where we are. Of course the minutes won't tell you everything you need to know (nor, unfortunately will your new team-mates) but within a few hours hours the newbie is at least familiarized with the issues and the stakeholders.

I'm not insisting that senior managers be the actual scribes or document typists. Go ahead and have someone else do it. As long as it gets done.

Which PPFM Skills to Implement First: The Spaghetti Axiom

People get all wrapped around the axle about how to implement PPFM.  Most of the methodologies have very structured prescriptions for doing this, not only at different "levels of maturity" but also detailed steps within a level.

For the most part, reality doesn't work that way.

Implementing PPFM in an organization for the first time (or worse, for the nth time) requires culture change, and it is quite a substantial one.  That's true even if the organization has at least the required degree of maturity to begin contemplating such an effort in the first place.  See Kik Piney's not-so-comedic article on the Anti-Maturity Model.

Think of PPFM as a nice steaming bowl of yummy spaghetti with equally yummy sauce that will, if mishandled, make some spectacularly messy and probably permanent stains on your best business suit. Obviously it must be handled with care.

Given enough time yo could probably come up with an optimal way to eat this delicious mess.  The third strand from the middle is the one hanging everything up, if you can just get to that one ... your lunch companions are not going to wait for this sort of nonsense.  So you stick your fork in there, twiddle it, and get what you can.  It may not be ideal, but it works well enough to keep everyone satisfied and the bowl gets eaten in a reasonably timely and tidy manner.

PPFM is like that.  Because of the interconnectedness of the iron triangle, any positive traction on managing scope, schedule or cost is going to have an immediate effect on the other two legs, and likely will impact the other PM areas also.

Are we having difficulty getting people to define the scope of their projects?  Move on, move on!  Get them to agree to making a list of the projects and committing to some due dates.  As the due dates get loser, the project teams will suddenly develop intense interest in the meaning of "done", and in order to avoid such misunderstandings in future the PMO will be required to write down that X is all that was agreed to.

"Stupid PMO, why didn't they think of that before?  Making us look bad."  Stupid like a fox.  Now the PMO is "forced" to implement proactive status reporting, scope management, schedule management, and quality management.   All the things people said there was no way they were going to do that.  Keep applying the Spaghetti Axiom - don't try to force people to eat the whole bowl, just pick a few things that will be productive and accepted, and will have a lot of knock-on effects. Sooner or later, and much sooner than anyone would have thought possible, you'll be at maturity level 2 or 3.

The Spaghetti Axiom: eat it or wear it.