Sep 4 • 21M

#184 The Persistence Of Failure

Long shadow of failed policies. US-China decoupling. Public policy as education.

Open in playerListen on);
Frameworks, mental models, and fresh perspectives on Indian public policy and politics. This feed is an audio narration by Ad Auris based on the 'Anticipating the Unintended' newsletter, a free weekly publication with 7000+ subscribers.
Episode details

India Policy Watch: Why Do We Not Get It Right, Ever?

Insights on topical policy issues in India


Policies fail more often than they succeed. This is a universal truth. We love to pontificate in this newsletter on why policies fail. It is the easiest of things to do. Crafting a policy that factors in the many variables that could impact its outcome and thinking through its implementation is one of the toughest skills to master. Things change, assumptions get invalidated, implementation gets botched up, politicians lose nerve. Before you know it, you have a failed policy on the debit side of your ledger. And on the credit side? Maybe only an Akshay Kumar biopic anthropomorphising your policy.

It is a tough life being a policymaker.

Even if you manage a smooth implementation, the bar for calling a policy successful in the long term is quite high. As Allan McConnell once argued, for a policy to be deemed a success, it will have to check the boxes in all the three realms of ‘program, process and political’ outcomes. You will have to be skilled and rigorous in your policy-making discipline, remarkably prescient on how a policy will be received by its stakeholders and, importantly, plain lucky to have a policy succeed in all these three realms. Policy failures, on the other hand, are understandably more common. All it takes is to fail in one realm to be considered one.

We laze on the cushion of hindsight here every week and easily pick holes in policies around us because there are so many things that can go wrong.

And they do.

Occasionally though, we also predict the failure of a policy. This is somewhat more difficult. We have some standard tools to do this. More often than not, we anticipate the unintended consequences of a policy which often derails the expected outcomes. Policymakers often miss this because they get so taken in by their own solution that they develop a blind spot to its risks. Or, there are political or ideological compulsions that determine the choice of a policy option. As somewhat objective analysts, we don’t have these blinkers. So, we have a fair share of successful predictions about what could go wrong with a policy prescription. 

Being Wrong For Long

But here’s a point. All of the above talk of policy failures makes sense when you take a shorter time horizon. Say, a decade. You could try out a policy and get it wrong because assumptions change or you didn’t think through its consequences. At the end of ten years or thereabouts, you can go back to the drawing board and rework the solution. If we go in with the assumption that most people involved in policy making - politicians, bureaucrats, experts, enlightened citizens - are sincere in trying to solve a problem and that they are equipped with the tools and knowledge to find the right answer, then they should be able to correct the course after one round of failure. In their second iteration, they should be able to arrive at a policy that has a significantly higher probability of success than the past attempt. At least, that’s what the theory says. But do they?

It brings me to the question that triggered this post. If your post-mortem of the policy helps you appreciate why it went wrong, what explains the persistence of bad or failed policies? Look, people get policies wrong because it is a difficult job. But why do they get it wrong for so many decades? Someone should have find the right solution in some iteration. Surely, no one group can sustain such spectacular incompetence.

Take India’s health policy, for instance. We have been trying to get it right since 1946 with many missions, yojanas and plans. You would think that we might have got it right over eight decades of trial and error. We might have had a few successes in running nationwide campaigns like Pulse Polio or Covid vaccination drive. We might have improved our life expectancy because of advances in medicine worldwide. But it is hard to argue that we have succeeded in the core objective of providing access to affordable healthcare to all our citizens and improving their well-being in these years. A similar story repeats itself in education. We have been trying for over six decades now. We even made education a fundamental right. Has it worked? Ask the daily wage workers who save up their meagre income to have their kids study in private schools because the state has failed to provide for a fundamental right. Or, take import substitution as a policy. After our spectacular failure following it in the 60s and 70s, what explains the fascination for it again? It doesn’t matter whatever nationalistic name we may come up with to defend it. Why do policy failures or bad policies persist? Why are we doomed to live in the Zombieland (to borrow a term from Krugman) of policies?

Maybe It’s The Norm

I will attempt to provide some answers to this question here.

Firstly, there is the usual explanation that every policy has an interest group or beneficiaries that have a significant incentive to continue with the policy. Once a policy is established, these groups entrench themselves in positions of power and come to impact political decision-making through a network of beneficiaries. A change in the status quo will benefit a larger group that’s diffused but whose costs will be concentrated in this network. This ‘small’ network will then have a much higher motivation to organise themselves and resist the change than a larger but less diffused set of beneficiaries.

The farm law in India and the inability to reform it is a good example of this at play. While this appears logical, the question to ask is, if the beneficiaries of the farm laws could organise themselves against the status quo, could they not have done the same to enforce similar farm laws in the past? If they could, then the bad policy was inevitable. The fact that a bad policy existed in the past therefore cannot be the reason it persists. Maybe the nature of the political process in certain sectors is such that there is always an interest group that can tilt the decision to a bad policy every time. The way to solve this is to reduce the political salience of the specific policy issue by propping up other issues and then correcting for the ‘natural tilt’ to bad decision by stronger communication of the alternative to the ‘diffused’ beneficiaries. These require heavy lifting by the political parties with a limited probability of success. It is one of the reasons why certain bad policies persist, and any attempt by a political formation to improve an evident terrible status quo is seen as ‘playing with fire’.

Secondly, there is a behavioural explanation for the endurance of policy failures. Incumbent governments are unwilling to accept policy failures because vote-seeking politicians don’t like to project weaknesses during election cycle. The usual strategy in the face of a policy failure for a politician is to deny the failure, deflect the blame to someone else or divert the attention to another issue where they might be on stronger ground. If this ploy succeeds, then the opportunity cost of walking back on a failed policy and correcting it becomes higher. Who would want to acknowledge something that you have already denied?

This means a policy failure continues. And the worse it gets, the greater the denial because the politician or the policy maker now cannot get out of the cycle. The usual point that’s made now is wouldn’t such political parties be punished for sustained failure in functioning democracies? The answer is yes. If the policy failure is so monumental that it becomes a salient election issue, then it becomes the reason for a change in government. However, the question to ask is, will a new government change the policy that failed? Charting a new policy path has two risks. The new policy could be seen to worsen the situation. This is usually the case because unwinding an entrenched bad policy comes with short-term costs. That apart, there is, of course, the blowback from entrenched beneficiaries that we have already spoken about. This makes governments risk-averse to changing the failed policy of their predecessors. There is also a structural reason here. Many political systems the world over are consensus-driven, where the difference between major parties isn’t substantial. The entrenched elites like the bureaucracy who are permanent or the corporates who fund these parties favour continuity. A new government is more likely to introduce policies in new areas altogether or rename old policies as a way of signalling change than actually reversing an old policy. 

Thirdly, there is the sunk cost involved in running with the old policies. Any policy that’s introduced will see a corresponding action by agents to take advantage of it. These agents will make investments for the future based on the current policy. Even if the policy is a failure, it is difficult for the agents to abandon their investments as sunk costs and move on. In fact, on the contrary, they will be willing to ‘pay’ more for the failed policies to continue in the hope that it may succeed in future. Who would want to abandon the investment made?

This is true for the policy of import substitution in India. Once you had the state and the private sector invest in building capacity for import substitution instead of selling their products worldwide, it became difficult to write off those costs even in the face of mounting evidence that the policy has failed. In fact, the moment you go down the path of import substitution and the private sector starts making investments based on it, your goose is cooked.  The switching costs are too high. Reversing this requires a bad economic shock of the kind we had just before 1991. 

Fourthly, bad policies often persist because there is just no good alternative to them when you factor in the constraints of the state finances and its capacity. There are certain policies that can only work if the state capacity is adequate to back its implementation. This might need the state to pull back from areas where it has spread itself too thin and focus on areas where it needs to go deep. Healthcare and education are sectors in India that have had many lucid policy recommendations on paper, but they have failed because those recommendations aren’t often in sync with the available state capacity. And any attempt at concentrating state capacity in these areas will mean a trade-off in retreating from some other areas. That will disturb the status quo there with consequent political price to pay. This act of political balancing is difficult to manage. 

Finally, there’s the anchoring effect of policies on the citizens and the media. The policies and their underlying logic, however flawed, get deeply embedded in the psyche over the years. In India, the desire among people for the state to intervene in the most trivial of issues is an example of this. We have written on many of them - service charges in restaurants, surge prices in Ola/Uber, price caps on sundry items etc. Despite evidence that such interventions have only led to worse outcomes and where the market has proven solutions, people in India reflexively look to the state to find solutions. The people of India and the state - it’s the greatest abusive love story ever told.

The reverse of this thinking is prevalent in the US where any whiff of socialism in your message is considered political hara-kiri. So, even if there are clear market failures that can only be solved through targeted state interventions, there is no consensus on reversing bad policies that have failed for long. US healthcare is a good example of this kind of anchoring effect.

This anchoring bias has two consequences. Failed old policies perpetuate and new bad policies continue to get introduced. Also, politicians find populist planks to perpetuate failed policies and it soon turns into a race to the bottom. 

As you will notice, the continued thriving of failed policies have many things going for them. There are no easy ways to break out of this cycle. The only sustainable solution is what we often bat for in this newsletter. You will have to work on the demand end of the political process rather than the supply side. The supply side will shift once the pattern of demand changes. So, the process of making people aware of policy failures, the reasons why they remain persistent and how they need to change their ways of thinking about public policy is the only solution here.

It is a long grind, but it is the only way. 

For more:

  1. A Framework a Week: What’s a Policy Success? from edition #147

  2. A Framework a Week: Wilson’s Interest Group (IG) Matrix from edition #12

Leave a comment

Matsyanyaaya: Decoupling Dynamics

Big fish eating small fish = Foreign Policy in action

— Pranay Kotasthane

One of the most significant questions in geopolitics today is about the future of the US-China economic relationship. Decoupling could have been a contender for the The Word of the Year, given how frequently it has appeared in various analyses over the last two years. And yet, the central question “How would the US-China decoupling might pan out?” hasn’t found a definitive answer.

There are quite a few takes though on this. The latest one from WSJ looks at share of global exports and concludes:

For all the talk in Western capitals of reducing reliance on Chinese factories, China has in the past two years consolidated its position as the world’s dominant supplier of manufactured goods.

How does one explain this? Here are a few possibilities.

  1. General equilibrium vs partial equilibrium: People focus on the latter. They pull out a number that shows that decoupling isn’t happening. But over time, the move is definitely towards a China + 1 model.

  2. Decoupling of any kind is a myth. It’s not possible. It’s just too costly. Look at the economics. Are you crazy?

  3. The rise reflects increasing prices, not trade volumes. The same article also says:

    The value of Chinese exports in June was 22% higher than a year earlier, according to an index of goods and services published by China’s General Administration of Customs. A similar index of trade volumes shows only a 5.5% gain.

  4. Decoupling is happening, but only in a few strategic sectors, such as scientific research, semiconductors, strategic materials, etc.

What do you folks think? Which of these four scenarios is more likely, and why? We will discuss more on this issue in the next edition.

Leave a comment

Global Policy Watch: The Educational Role of Policy

Global policy issues and their implication for India


Speaking of public policy and how it anchors the mindset of people brings me to this piece - “What Is the Purpose of Public Policy?” by Tyler Cowen in Bloomberg.

He argues:

“The purpose of any policy is not only to effect change but also to educate the public. This educational function is important yet rarely discussed explicitly, perhaps for fear of looking like a propagandist. So be it: I wholeheartedly embrace the notion of policy as a way of educating voters.”

For Cowen, a policy might fail or partly succeed in its objectives for various reasons. However, it would still serve the interest of society if it educates the people about the right kind of values. I find it a bit dubious because values differ among people and parties. And no one wants a scenario where an egregious policy failure is covered up by the argument that it taught people something good. That’s the road to justifying demonetisation.

But Cowen is approaching it from a place where a policy based on sound reasoning deserves to be judged on an additional parameter of success. And that is - did it convey the right message to the people? Did it move the Overton window for a future policy to succeed spectacularly? If yes, then we can say such policies are good because they have educated people about the right long-term things.

So, what are the kind of policy messages should a government be sending out to its people? 

Cowen would like:

“…increased respect for cosmopolitanism, tolerance, science, just laws, dynamic markets, free speech and the importance of ongoing productivity gains. Obviously any person’s list will depend on his or her values, but for me the educational purposes are more than just a secondary factor. When it comes to prioritizing reforms, the focus should be on those that will “give people the right idea,” so to speak.”

Cowen concludes with a familiar argument we have made here on umpteen occasions:

“Sadly, the American public doesn’t have many ways of receiving economic or policy analysis. Most people do not and never will know the finer points of various policies. But they do absorb the main messages of their political leaders. They also have a general sense of what their country is up to and, for better or worse, a certain loyalty to the status quo.

All of which is to say: For better or worse, our political choices often succeed in “educating” us. Just about everything we humans do, one way or another, is a form of communication. So it is with making policy.”

It is another way of saying that a policy can anchor people's attitudes and behavioural patterns. I think the converse of Cowen’s argument is even more important. Policies predicated on flawed reasoning are terrible for people. It is important to hold policymakers responsible for any policy that doesn’t make rational sense but is useful, politically or otherwise, in the short-term. It will teach people wrong things.

And bad education is hard to shake off.


Reading and listening recommendations on public policy matters
  1. [Paper] China’s emerging data protection framework by Rogier Creemers. This is an excellent paper explaining the current status of data security laws in China. TL; DR. The legal architecture for data protection has two pillars:

    1. Personal Information Protection Law (PIPL) focuses on data movements between the data principal (individuals) and the data controllers (governments and companies). But the multiple exceptions for government entities effectively mean that these restrictions apply only to companies. PIPL also has stricter data localisation requirements, which might dissuade foreign companies at the margin.

    2. The second pillar is the under-analysed Data Security Law (DSL). This law is primarily focused on national security. It also has a “core national data” classification.

    Both these pillars suggest the movement of all kinds of data outside China will become even more difficult. Data compliance requirements for foreign companies are likely to rise significantly. Another opportunity for India and other countries.

  2. [Database] A precious database of English periodicals from colonial India.

  3. [Article] Amit Cowshish criticises the move towards mindless indigenisation of easily importable, non-critical components for defence equipment.

  4. [Report] One, Two, or Two Hundred Internets? The Politics of Future Internet Architectures by Kevin Kohler is a must-read for anyone interested in technology policy.