When Is a Market Failure Not a Market Failure?

Coase would go on to win the Nobel Prize in 1991 for his work developed in “The Nature of the Firm” and his later paper “The Problem of Social Cost” (1960), which further explored the importance of transaction costs in understanding markets. In particular, he showed there that transaction costs were important to understanding what economists call “externalities.” Externalities are costs and benefits that accrue to parties outside of a given exchange. The classic example of a negative externality is pollution: the factory that pollutes the air imposes costs on its neighbors, many of whom have no dealings with the owners of the factory. Such externalities are one example of what are generally known as “market failures.”

Market failures have long been discussed as a necessary condition for government intervention in the market. When costs (or benefits) fall on those not directly involved in the transaction, government regulation (properly designed) can set things right.

While the presence of externalities is a necessary condition for legislation, is it a sufficient condition? Are there other mechanisms in place that work toward solving the issues caused by the externalities that could be rendered ineffective by legislation? In other words, could legislation, poorly considered, end up making things worse? This article seeks to answer this question. But first, we need a working theory of externalities in the market.

The perfect competition model is perhaps one of the best-​known models in economics. The model shows that, with many buyers and sellers, there will be a price which emerges where all resources are allocated most efficiently within the market system. There is no waste or under-​utilization as people can bid away resources from less valued uses toward more valued uses. Sellers make no more profit than they could make doing other things, buyers pay what they are willing, and output (and, thus, consumption) is maximized. Everything is hunky-​dory.

However, this happy state of affairs is not guaranteed. There are countless reasons that the market may be pushed away from the state of perfect competition. One such reason is externalities. Externalities can make the market price too high/​low by foisting costs onto some third party not involved in the transaction, resulting in inefficiencies. Consider the following example: A house next to a public beach overlooks the ocean. However, when people come to the beach to play, they may listen to very loud music. That music annoys the homeowner by disturbing her peace. The costs of listening to music are not borne entirely by the person listening to the music; he might be playing the music “too loud” and reducing the benefits to the homeowner. In short, movements away from the perfect competition model lead to what economists call a “deadweight loss” (wasted resources) and mean the market system is not operating at peak efficiency. This is termed a “market failure.” Regulation can, in theory, step in to move the market back to its perfectly-​competitive point by, for example, taxing polluters (or those playing loud music on the beach) to reduce the amount they generate and cover the costs they impose on others.

The problem with these regulatory theories is they tend to operate assuming zero transaction costs. In other words, they assume the market system, and subsequently the political system, are costless to use. They also tend to assume, politically, there aren’t any lobbying, informational asymmetries, or mistakes. In other words, they ignore the political equivalent to market failure: political failure. Figuring in these transaction costs (to which I will group in the costs of using the political system) gives us a better picture of when a market failure is truly a market failure worthy of government attention.

From a strictly theoretical point of view, market failures are any outcome that is not at the equilibrium point of the perfectly competitive model. What externalities do is move markets away from that optimum. However, the mere existence of these conditions is necessary but not sufficient to claim there is a market failure that can be remedied by political action.

Ronald Coase showed in “The Problem of Social Cost” that, absent transaction costs and with perfectly defined property rights, the affected individuals could come together, negotiate a deal, and the negative effects of the externality would be internalized. To go back to our beach example, the homeowner could offer money to the music players to turn down the volume on their stereo. Or, in a similar fashion, the music players could offer money to the homeowner to compensate her for their noise (funds which she may use to install more noise-​resistant windows). However, if transaction costs existed (e.g., if there are many, many music players), then the government or some other authority might step in and impose regulation to fix the externality.

But this analysis is incomplete. It assumes both parties have complete knowledge and that the only obstacle is the transaction costs involved. But this poses a problem: if the transaction costs are sufficiently high that the negotiation between the external polluter and the person affected does not take place, then that implies that the benefits of reducing the pollution are less than the costs of securing that reduction. Thus, the market is already at an optimal solution. To keep with the beach example, if the music players are far down the beach away from the home so that their music is just a dull roar, the homeowner may not feel it necessary to walk all the way down the beach just to lower the volume a little bit. People generally ignore minor annoyances (no one would negotiate with a smoker walking by them in the park to get them to stop smoking for those 10 seconds of interaction even if the smoke is an annoyance). It is only the major annoyances that are dealt with, where the benefits exceed the costs.

As Carl Dahlman explained, even this point is incomplete. What matters is not so much the ex post knowledge of the transaction costs, but the ex ante analysis by the potential negotiators. Essentially, we can imagine four possible states of the world between the polluter and the harmed person: 1) where both parties expected to benefit from the reduced pollution and actually did benefit (successful Coasian bargain), 2) where one (or both) parties expected to benefit from the reduced pollution but did not actually benefit from the reduction, 3) where both parties correctly anticipated that the costs of bargaining would be too high and did not reduce the pollution, and 4) where the parties incorrectly anticipated the benefits were too low and failed to negotiate. Of these four possible states, in only the fourth (where the parties incorrectly anticipated the benefits and costs of negotiation) can there be truly a market failure and thus a role for government. In the other three states, the costs of negotiating away the externality were higher than the benefits obtained, and thus the market was already at its optimum and government action would have reduced, not increased, market efficiency.

A key point worth elaborating on: in the above example, we implicitly assume that the analyst knows both the actors’ preferences and the transaction costs involved. Thus, the analyst can properly anticipate which of the four states of the world we are in. In reality, though, such knowledge is not particularly commonplace. Individual preferences are hard to obtain, and costs and benefits are subjective. Ultimately, it may come down to the analyst projecting their own preferences onto the parties involved, which may lead to an incorrect evaluation of the states of the world as well as the sort of regulation needed to fix the externality in State 4.

As with all economic questions, the answer is “compared to what?” Externalities, compared on an idealized hypothetical world where all information is known and transactions are costless, appear easy to solve via government regulation. However, when we compare a world of externalities to the real world, where costs and benefits are subjective and ultimately judgement calls must be made by analysts, we see that externalities are necessary but not sufficient justification for government intervention. But we need to say more on the political process.

Up to this point, I have implicitly assumed that there are no costs to using the political system to address market outcomes, choosing to take a market transaction costs perspective. Many economists acknowledge transaction costs play a role in determining market regulations and what is appropriate and not. They will argue that, if the transaction costs in the market are lower than the benefits of regulation, that constitutes a necessary and sufficient condition for regulation. However, political costs must be taken into account as well. The presence of externalities, and relatively low transaction costs (compared to benefits), are necessary but hardly sufficient for regulation.

Political costs are analogous to market transaction costs in that they are costs of using the political system. Politicians (or governmental agencies) do not inherently know market conditions. They must be informed of them: informational costs are ever-​present for politicians. Lobbying can overcome some of these costs, but lobbying involves costs as well, not the least of which is the danger that the lobbyists are aiming for something other than market efficiency. Furthermore, the government incurs costs: politicians and agents need to be paid, buildings run, studies undertaken, rules enforced, etc., all of which needs tax funding (which creates its own distortions in the market).

On top of the costs of running the operations of government, we have the costs of running analysis to determine the appropriate tax or regulation. Data need to be collected, analyzed, checked, double-​checked, and policy recommendations need to be implemented. All this is a costly procedure. The typical case for regulation assumes no costs in determining the proper level of regulation, but the models take the data as given, a situation which never exists in the real world.

An inherent assumption in the economic case for regulation is that the regulating agency is interested in principle as opposed to self-​interest. However, this is probably not the case for some types of regulations. Arguments by politicians and even some economists for taxes on pollution persist even though there is evidence suggesting there may already be more than the optimal tax in place already. In his Lectures on Jurisprudence, Adam Smith warns us: “They whom we call politicians are not the most remarkable men in the world for probity and punctuality.” If politicians are imprudent, if they have interests beyond strictly economic efficiency, the political costs become even higher.

In short: even if it can be shown in theory that regulation could improve a market outcome, political costs may be sufficiently high as to make regulation a net loss rather than a net gain. Externalities and net benefits from potential regulation are necessary but not sufficient to justify regulation in markets. The “inefficient” market (as compared to the unrealizable state of perfect competition) may be the best possible realizable state of the world in the current conditions once the transaction costs of the market and the costs of using the political process are fully accounted for.

As Harold Demsetz warned us, the mere fact that the real world differs from some theoretical outcome does not mean that alternatives are necessarily better, especially when one situation is viewed through the lens of reality and the other through the lens of theory. In theory, any deviation from perfect competition is sufficient to justify regulation if transaction costs are low enough. In reality, when we add in political costs, we see that while those conditions are necessary, they are hardly sufficient. The information and knowledge necessary to properly regulate are extremely hard to obtain and there are likely political considerations beyond economic efficiency. What’s more, given that political decisions necessarily affect everyone, political mistakes can have far reaching consequences. With a slow-​moving political system (such as a democracy), it can be extremely difficult to adjust to these mistakes or impute new data and recalculate optimal regulation even if we assume no self-​interested lobbying or other barriers preventing the regulation from being changed.

This adds up to a strong presumption of liberty. That is, we should presume that current outcomes, derived from the free and voluntary actions of individuals, likely represent those individuals’ best concerted efforts to improve their situations. Yes, they may make mistakes, but that is primarily only known after the fact and markets have a strong capacity to correct those errors. Regulation may lead to the mistakes becoming multiplied, or worse, entrenched.

What does a presumption of liberty mean for government regulation? For one, advocates of regulation have an extremely high burden of proof. Not only does the burden of proof fall upon those who wish to restrict liberty, but rather it is more than simply justifying their case theoretically. They must account for transaction and political costs as well. Furthermore, government regulation should be restricted primarily to negative rules. Negative rules are “thou shalt not” style rules, like “thou shalt not steal.” Rather than direct actions (positive rules), negative rules are merely designed to allow a wide range of actions within a very broad framework. Government only intervenes when the rules are violated rather than try to achieve a certain goal. This reduces the guesswork necessary on the part of government and encourages cooperative efforts to solve problems.

Externalities like pollution are a necessary but not sufficient cause for government intervention in the market. Indeed, government intervention could accidently make the externality problem worse, either by not correcting it enough or by reducing our ability to handle the effects of the externality by reducing it too much and harming our economic growth. There are many questions that need to be considered before government intervention can be sanctioned, not the least of which is “why aren’t the affected parties already solving the problem?” Careful considerations of these questions, as well as a realistic understanding of how government operates, are key to creating effective legislation. Theory is vitally important to our understanding of the world, but applying theory to the real world requires that we consider the complexity and costs of both the market process and the political process.