Tracking Policy Outcomes: An Exercise in Deception

But things work differently at Veterans Affairs!

My recent post on the expensive train wreck of federal government departmental self-assessments used annual reports from Canadian Heritage as a case in point. AI analysis found that high program success rates revealed nothing useful about real-world outcomes, since they only measured activity. At street level, nothing changed.

Everything I’d seen relating to other departments suggested a similar pattern. Official goals are typically process-oriented rather than tied to actual outcomes. Millions of dollars are regularly poured into generating detailed annual reports that, in fact, measure nothing of substance.

Just for completeness, I thought I’d look through the actual program performance indicators, targets, and results as they’re reported in the GC InfoBase. The detailed and focused performance data you’ll find there would be hard to duplicate at scale through individual department reporting. But it’s probably the closest we’ll get to useful accountability with the system.

Let me share some examples of actual results from 2024-25.

• Health Canada’s departmental results include “Canada has modern and sustainable health care systems”, whose target was per-capita spending between $4,386.00 and $5,361.00. No results were provided. But if they were, what difference would it make? How does heavy spending in any way guarantee a “modern and sustainable health care system?”

• Health Canada also aimed for drug spending reaching between one and two percent of Gross Domestic Product. The actual result was 1.7 percent. But, again, calculating spending amounts is a terrible proxy for the only thing that counts: medical outcomes.

• Canadian Heritage targeted at least 1,190 Canadian TV productions delivered across “multiple platforms.” Results were not provided, but even successfully producing 1,190 productions will have zero impact on Canada’s heritage if virtually no one watches them. “If you build it, they will come” was a great line in a fantasy movie, not a how-to manual for “social cohesion”.

• Canadian Heritage also targeted the publishing of at least 6,000 Canadian-authored books. In the end, they claim 6,094 were published. But as I’ve written, considering how the overwhelming majority of Canadian books sell less than a few hundred copies each, does a number like 6,094 really demonstrate any reasonable return on public investment?

• One more example. Canadian Heritage established a goal requiring that at least 86 percent of Canadians feel that human rights are a shared value. They claim that 2024-25’s result was 95 percent. Really? One of every 20 Canadians they managed to find didn’t believe in human rights? Where were they polling: Iran’s IRGC headquarters? And how was Canadian Heritage comfortable taking the credit for this?

There’s plenty more publicly-funded slop where all that came from. As far as I can tell, we’re looking at thousands of federal employees across the public service working 9-5 to game their own system. Unless I’m missing something here, it’s embarrassing.

But when I checked out Veterans Affairs Canada, I got something very different. First of all, you can see just how extraordinarily detailed their programs are from this document. But the real difference is in how practical, precise, and measurable their goals are.

The department’s GC InfoBase page offers plenty of examples:

• Percentage of veterans who are satisfied with their financial situation. The target is “to be determined”, but the actual percentage was 66 in 2024-25.

• Percentage of veterans who have high level of mastery of their life skills. Target: 40 percent, actual score: 27 percent (up from 24 percent in 2023-24).

• Percentage of veteran clients of Veterans Affairs Canada who report an easy adjustment to post-service life. Target: 70, actual score: 49.

• Percentage of veterans who report that their health is very good or excellent. Target: 50, actual score: 41.

• Percentage of recommendations made by the Office of the Veterans Ombudsperson related to systemic issues addressed by the Veterans Affairs Portfolio. Target: 74, actual score: 74.

• Percentage of individual complaints reviewed and escalated by the Office of the Veterans Ombudsperson to the Veterans Affairs Portfolio that are addressed. Target: 80, actual score: 92.

I can’t tell you whether Veterans Affairs’ numbers are accurate, nor whether or not there are veterans whose serious needs are being ignored. But those targets tell us that at least someone in the department is serious about addressing real problems and tracking how well those problems are being met over time. Even if they’re failing more often than succeeding, at least they’re working hard at understanding where they stand.

If you’d ask me, this is what program tracking should look like.

Perhaps, however, you’ll argue that the work Veterans Affairs Canada does lends itself to this kind of easily quantifiable metric more than its peer departments. I would respond by saying that governments have no business legislating policies whose outcomes can’t be measured: they’re wasteful by design.

---

There’s more:

---

Photo by Isaac Smith on Unsplash