feoNotes

Better IT Results

Gov tech - response to CFA question

CFA founder and executive director Jennifer Pahlka posed the following questions of medium recently.

• what can/should procurement officers, program staff and leadership in government do [tp change the rules of the procurement game to consistently get better outcomes for taxpayers and the users]?
• What’s worked so far that others in government should be borrowing? What bold ideas need championing and pioneering?
• What can/should government contractors do (both leadership and employees)? Given that they don’t tend to set the rules, what responsibility do they have to be part of changing them? Who’s been showing leadership here and how might others?
• What can/should the general public do? What does [changing procurement practices at the local level] look like and how do concerned citizens make a difference?

This post is a response to those questions.

In full transparency,I am currently a federal employee working in technology/policy, have a long career in government technology/policy, and have no specific ties to a vendor or Code for America. I know Jen reasonably well and worked with her a few years back.

Projects (any kind of project) are primarily driven by managing interrelated risks: risk that we won’t finish the project on time, the risk of exceeding the budget and the risk that we won’t get the return/results we want.

The procurement game in federal IT space is governed by the federal acquisition regulations (FAR). Used correctly the FAR can absolutely allow a department to purchase quality products, replace contractors that don’t meet needs, and make modifications without cost overruns. And, YES, we all have lots of complaints about the FAR. Much as I would like to see us rethink it, as a practitioner of IT in federal government, I know it would require an inordinately large shift, likely an Administrative Procedures Act rulemaking on perhaps the largest scale possible, or perhaps even legislation, both of which are very unlikely scenarios.

So what should stakeholders do to change the rules of the procurement game to consistently get better results? We don’t change the rules of the procurement game, but we incentivize timeliness over scope creep, change how we plan for implementation, and restructure our vendor relationships to leverage the skills and experience of our dedicated colleagues in public service. Finally, we must articulate why we are prioritizing one project over another and what the boundaries of each project really are.

The Deadline - Scope Creep Relationship Time is often the key function. If the date is important, get it codified. I have seen projects (medium size, ~$10s of millions) catalyze around the project outcomes because there is a mandated due date. By mandate here, I mean specifically something codified in statute not just something that says the organization wants it done by a due date or the project manager sketched a gantt chart with a milestone. Having a statutory deadline that is immovable forms a motivation for the due date. These motivations create a strong incentive to keep the scope aligned and not offer any creep outside what is really required. Too many times I see (particularly in agile blue sky kinds of experiments) massive scope creep. While it would be nice to include all of these functions, the real principles in agile offer the opportunity for continuous delivery and continuous improvement. It is far more important to provide a product on the due date and then improve it, than to think we can get all this done and bite off way more than we can chew, and end up failing to deliver something at all.

All Bow Before the SOW When putting out a procurement, the individual(s) writing the scope of work, the deliverables, and the expectations and all the rest that is required, regardless of the style of the procurement vehicle, have the pen. Because contractor selection and management stems from the the SOW (Scope of Work - the piece the government writes to tell vendors what they want delivered), this means that the opportunity for failure increases exponentially when a SOW is poorly written. It also means that failure can often rest on the employees scoping the work and writing the expectations rather than the vendor. The department needs to know and define its expectations of the vendor and the exact skillsets it needs. This is the case in agile or waterfall procurements (while we’re on this topic… agile is not a panacea, lots of success happen with waterfall, but if there is an expectation and it is not articulated in the SOW, then there is no vendor on earth who can read minds or really deliver. Moreover, in agile religion is not the answer either - lots more to say here in a different post). Sure vendors can show up with poor skills, but the FAR gives you all kinds of opportunity to manage the agreement based on performance.

Vendor Relationships, Skilled Staff and a Killer COR Before you sign a contract… Before you write an RFP… Before you do anything…You need to understand the skillset of your staff.

Relying on your current assets and knowing what skills the vendor market can augment your team with is a far better approach than to go out and attempt to procure a turnkey solution, even when the project is a greenfield. I have had the the fortunate opportunity of implementing two medium sized federal IT projects (+$10M). In both cases, we had existing federal employee staff and computing resources and we augmented them with very specific contractor skillsets. Had we taken the other (perhaps more common approach) of writing a single procurement to implement that project the risk of failure is significantly higher. Projects where one general contractor comes in and bids on the scope end up being a single point of failure. Augmenting programs with vendor resources matrixed in offer the opportunity for diversity and spread risk much more effectively.

One of the reasons we were able to do this effectively is because we knew what capacity we had and we had the KILLER COR. What is the KILLER COR? It is perhaps the most critical skillset in federal IT vendor management -what we call the ‘contracting officer representative’ or cor (sometimes contracting officer technical representative - cotr). I have had the great pleasure of working with a handful of (and one in particular) truly tremendous cors. It is not an understatement when we say that this component is by far the biggest key to success in any federal IT project. A KILLER COR cor has the technical depth to understand the project and the contracting expertise to help you write, evaluate, award and manage a contract.. They are not pointy-headed bureaucrats who only care about managing risk down to the lowest common denominator. KILLER CORs are your partners in successfully managing a project.

But here’s where it gets exciting. If you have a great COR and you’ve really thought about what you need and you have defined it well, don’t procure the vendor equivalent of a general contractor who is then going to sub out all the work. This actually increases the risk of failure and decreases control of the project. When you contract out the general and the subs, then risk is spread far and wide. I have found the most success when the COR handles a number of personal services contracts that explicitly say we need this skill, then the team (cor, pm, project sponsor, other technical staff etc) direct the work to say I need exactly this (and no mission creep or other things). This puts more burden on the COR and existing team for sure, but spreads risk and you end up with a better product. if one vendor’s resource doesn’t work out, replace them, fire them etc. imagine, on the other hand, that the vendor supplies the entire solution (planning, development, software, infrastructure and resources), well then, that means that the risk factor has a single point of failure. Every place I have worked in my nearly 30 years of government service has had well equipped deep technical staff who should augment vendor supplied teams. USE THEM. This kind of collaborative effort always results in a better approach.

Why are we really doing this again? Managing the scope of a government IT project is paramount to success, especially because you - or your boss - or your boss’s boss - will think of something new to add multiple times throughout the project. In all of the projects I work on I tend to have a key functional metric. The implementation of this project needs to perform x units per y cost. This metric is behind everything we do: the architecture, the selection of the technology, the composition of the team, and their skillsets. In fact I would say you can’t even begin to write a SOW unless you have this key metric in mind.

Let me be super clear here, it’s not enough to build a new website “because we don’t like the current design” or a new case management function “b/c this one has been in place for 10 years”. There has to be a compelling cost and/or return on investment calculation of these capital investments in order to ensure success. Only after projects are implemented can we measure the investment and actually determine success. (And yes, ROI can be different in government, but we have to be overcoming a real obstacle - a partner agency that can’t share information with us, a consumer that can’t ever find the online form, etc.) I am struck by how many projects that haven’t worked well also lack a clear goal and measurement outcomes (i.e. ‘better design’ as an end all, be all or building a new website because we think our website just “looks” too much like other government sites). Government provides service and we have every capacity to provide that service for a measurable increment of unit to maximize taxpayer investment.

When we are building a new IT project, we’ve got to manage risk. Sure, failing fast is better than failing after a $200M investment, but public servants know that any public failure often results in a very public flogging, and that they’re often the ones on the receiving end (pointy-headed bureaucrats and all). Many CIOs manage risk by choosing and staying with a single technology. I am adamantly opposed to this approach. I find that the explosion of cloud technology market over the last decade, the advancement and adoption of open source technologies and the principles of diversity in ecology lend itself to far more robust and successful outcomes than staying with the technologies we have always used. Great procurement strategies help us manage risk, increasing our chance of being on-time and under budget. Long-term, yes, let’s talk about the FAR, but short-term, let’s talk about our strategy and our human capital. Have time requirements. Know the exact technical assets you want/need. Develop and support your CORs, not just your technical staff - and then select a KILLER COR for your projectork on a matrix of vendor and government supplied resources. Finally, go into the procurement with a measurable outcome.

These aren’t revolutionary ideas. What would be revolutionary is if we could sustain attention on them.

As a final point, as with all things in policy, there is never a silver bullet. There is never one lever to pull which fixes all the problems. The four points in this response have worked for me, in the situations I have implemented. Others might have success with other levers. The thing we should be really wary of is someone hocking a solution to solve all the problems.