Is OPEC Dead? And Will Shale Oil Long Survive It?

With the price of crude oil dropping precipitously over the last two years — falling by over 65 percent between June 2014 and December 2015 — oil producers around the globe have looked to Saudi Arabia and the Organization of Petroleum Exporting Countries (OPEC) to boost prices by cutting output. And so, earlier this month, representatives of Russia, Mexico, and all the OPEC countries except Iran met in Doha, the capital of Qatar, to discuss doing just that.

They failed to reach an accord.

Absent action by the OPEC cartel, many experts foresee an extended period of low oil prices. For example, the U.S. Energy Information Administration projects that in 2016 the price of imported crude oil (in constant dollars) will be only 35 percent of that paid in 1980; 33 percent of the 2008 average price; and only 31 percent of the price in 2011, the year in which the annual average oil price peaked.

U.S. independent oilman Robert Mosbacher, Jr. attributes the fall in oil prices to what he calls Saudi Arabia’s “current strategy of flooding the market.” Much media coverage echoes his claim. But while Saudi production did increase by 1.7 million barrels per day (MBD) between 2010 and 2015, Iraq’s production rose faster. And an even more important factor is American oil production: during that same period, U.S. output increased by 3.8 MBD — an amount that exceeded the growth of the Saudis and the Iraqis combined. (See Figure 1.)

There is even less support for the claim that OPEC as a whole is the source of the current oversupply: total OPEC output grew by only about 0.9 MBD from 2010 to 2015. This is more like a streamlet than a flood when compared to the 2015 average world crude oil production of about 96 MBD. What’s more, the data show no coherent OPEC strategy to oversupply the market. If we exclude Saudi Arabia and Iraq, OPEC’s oil output during that same period decreased by almost 2.2 MBD. (And such losses likely resulted from external forces, such as civil war in Libya and economic sanctions on Iran, rather than internal economic policies.)

Some pundits have begun to conjure up theories to explain how the current low oil prices are actually somehow part of a Saudi grand geostrategic design. If so, Riyadh is paying a high price for it. An International Monetary Fund analysis shows that Saudi Arabia derives about 40 percent of its GDP and about 90 percent of its central government’s revenue from crude oil. The fiscal bite from low oil prices is compelling the Saudis to adopt spending cuts, and Riyadh is still running large fiscal deficits. As the country’s economy further contracts, more belt-tightening measures are inevitable.

The failure of OPEC and the other Doha meeting participants to reach an agreement provides an opportunity to ask some basic questions about the oil market: What power does OPEC now have to influence prices? What is the likelihood that oil exporters will coordinate in 2016 to reduce production and thereby raise prices? How has the U.S. oil boom of the past few years been affected by the low prices? And what might the 2016 U.S. presidential race imply for the future world oil market?

From 1973 to 1986, at the dawn of the modern world oil market, OPEC countries sought rapidly to increase their wealth through a system of high posted prices for crude oil. Even though exogenous events such as the Iranian Revolution may have had greater effects on oil prices than OPEC’s strategy, the cartel was still a major factor in the high oil prices during that period. But such high prices had a downside: OPEC was forced to defend its posted prices by restricting production, hemorrhaging OPEC members’ market share. OPEC data show that in 1973 its members supplied roughly 54 percent of the world crude oil market (with Saudi Arabia accounting for about 14 percent of the world total). By 1985, OPEC’s share had fallen to roughly 27 percent (with the Saudi total at about 6 percent).

In 1986 the Saudis concluded that defending high prices was unsustainable, and Riyadh jettisoned the posted-price system. Instead, OPEC came increasingly to rely on production quotas, which can indirectly influence the market price of crude oil. OPEC still produces a significant share of the world’s crude oil — about 41 percent in 2014, according to BP statistics. And, since OPEC’s overall production costs are below the world average, by increasing output it can displace from the market some of the oil that would cost more to produce and deliver. In theory, therefore, so long as it can successfully forecast the demand for oil and the marginal costs of non-OPEC suppliers, OPEC can influence the price of crude oil, at least indirectly.

But this only works if OPEC can reach and enforce a consensus among its members on how much oil to produce — no small task. In fact, it has grown only more nettlesome in recent years. The discord within OPEC is currently so great that the cartel has failed to agree on any production quotas. The problem is that choosing a target for the current price of oil requires weighing short-term against long-term gains and losses. In the short term, both supply and demand of crude oil respond very weakly to price signals, so oil exporters like OPEC reap quick gains from price hikes. Over the long term, though, high oil prices elicit more non-OPEC supply, induce greater reliance on oil-saving techniques, and depress world economic growth. And these effects lessen the future demand for OPEC oil. Still worse from the viewpoint of the exporter nations, high prices can stimulate investments in long-lived capital assets and new technologies, causing OPEC’s losses to be protracted or even irreversible.

The disagreement has usually played out in this way: Saudi Arabia and the other states in the rich Gulf Coordination Council (GCC) stress the importance of maintaining the long-term flow of oil revenues. (Since 1986, the Saudis have for the most part aimed to keep oil prices within a range consonant with the informal Saudi–U.S. bargain of stable oil prices in exchange for security assistance.) By contrast, more exigent pressure for a wider distribution of oil wealth has compelled the leaders of Venezuela, Iran, and some other non-GCC states to chance some loss of future oil revenue in order to obtain more resources with which to hold power today.

This enduring intra-OPEC discord has, over the years, caused some to liken OPEC to the medieval Church riven between two Popes, one in Rome and one in Avignon. In like manner, OPEC is split between Riyadh and Tehran. As analyst Bassam Fattouh put it in a 2007 study:

In their attempt to defend a target price, OPEC members would call for production cuts. However, because of the different features, needs and bargaining power and the divergent interests of member countries, OPEC cannot usually reach agreements on allocation of production cuts. Even when agreements are reached, each member has the incentive to go against these decisions. Because of the absence of a monitoring mechanism, these violations are not usually detected and even if they are, the organization does not have the power to punish and force member countries to abide by the agreed production cuts.

This is likely the reason that efforts to curtail supply so often fail. OPEC members cumulatively exceeded their production quotas in 2014 and 2015. And a 2009 analysis found that “since the quota system was adopted … total OPEC production has exceeded the agreed ceiling by 4 percent on average, but on numerous occasions the excess has run to 15 percent or more.” An urgent need for cash often impels governments and state-owned oil companies to overproduce, even those that most vociferously demand production cuts to boost short-run prices.

On occasion, OPEC has succeeded in raising oil prices by limiting output. Those successes share common features that reveal the underlying complexities that OPEC faces.

Perhaps surprisingly, OPEC’s power to raise prices is greatest when oil markets are slack and prices are falling. Low prices weaken two of the cartel’s most intractable impediments to collective action. First, low prices ease Saudi fears that restricting supply will drive prices high enough to decrease long-run demand. Second, low prices lessen the temptation to invest in capacity in excess of production quotas. Still, Saudi Arabia must contribute disproportionately to production cuts to make them large enough to matter. And in nearly every instance where OPEC’s quotas contributed to higher prices, OPEC’s output snapped back to normal levels within a year.

In 1998, for instance, the average price of crude oil fell by almost $10 per barrel. In response, OPEC agreed to two production cuts, one in spring 1998 and a larger one in spring 1999. As a consequence, OPEC output fell by about 1.44 MBD during the first and second quarters of 1999, and average annual crude oil prices in 1999 were up by more than $5 per barrel over 1998. (See Figure 2 below, and the band marked in yellow.) But the degree to which OPEC caused the price recovery is disputed. The end of the 1997–1998 Asian financial crisis raised world demand for oil, contributing to price increases. Moreover, some analysts argue that when prices are low, producers will sometimes restrict output in order to catch up on deferred maintenance of oil wells. In any case, the supply restraint of the late Nineties was short-lived: by the second quarter of 2000, OPEC output was back to its pre-1998 levels. In fact, the recession of late 2001 caused the demand for oil to weaken so much that OPEC decided to embark on a new round of supply cuts (see the green band on Figure 2).

It is worth noting that Saudi Arabia’s contribution to the 1998–1999 output cuts was disproportionate: the country made about 35 percent of the total 1999 OPEC production cutbacks (and did the same from 2001-2002), even though OPEC statistics indicate that in both episodes it was producing only 29 percent of OPEC’s oil when the cuts began. Because the Saudis can punish over production only by flooding the market, which is costly to them as well as for the cheaters, they are in a weak bargaining position vis-à-vis the more impatient exporters and for any deal to hold, the kingdom must bear a disproportionate share of the sacrifice.

This pattern was reprised a few years later. In 2006, oil prices fell from $75 per barrel to slightly above $50. The decline prompted OPEC to meet first in October and again in December that year. The meetings hashed out an accord on production cuts. Average daily output declined by 1.2 MBD from the third quarter of 2006 to the first quarter of 2007. (See the pink band on Figure 2.) In 2007, prices recovered and stabilized, and by the fourth quarter of that year OPEC output had returned to its prior levels. Once again, Saudi Arabia contributed disproportionately, accounting for 45 percent of the total OPEC supply cutback, while producing only 29 percent of OPEC oil.

The Great Recession triggered another OPEC supply cutback, following a similar pattern of events — except that this time supply recovered more slowly. Between 2008 and 2009, the average annual price for crude oil fell by more than $35 per barrel. In response, OPEC members slashed production by almost 2.8 MBD from the third quarter of 2008 to the first quarter of 2009 (see the gray band in Figure 2). This time, Saudi Arabia contributed almost 50 percent of the decrease. Then, in 2010, oil prices rose by nearly $18 per barrel, recovering about half of the previous year’s losses. Unlike in the previous cases, though, OPEC production remained below its 2008 peak as late as 2012.

This unusually slow rebound was likely due more to U.S. foreign policy than to OPEC policy. The 2011 U.S.-supported NATO bombing campaign of Libya reduced that country to anarchy and depressed its oil output by about 1 MBD. True to form, Saudi Arabia raised its average daily output by about 1 MBD. Then, in the fourth quarter of 2012, tougher sanctions pushed Iranian crude oil output to roughly 0.8 MBD below 2011 levels. The Saudis held output fairly steady from the third quarter of 2011 through 2014. That choice may have reflected the Saudis’ desire for public spending to head off the internal unrest that was breaking out in the Arab world at that time.

Today, with the price of crude oil again in the doldrums, distressed exporters like Venezuela and Nigeria are once more urging OPEC to cut production. Rather than rebuffing their demands outright, Saudi Arabia has responded in a way familiar since 1986: offering to restrict its output, but only if other OPEC countries do the same. As of this writing, talks are continuing.

It will be much harder for OPEC to make production cuts this time than it has been in the recent past. In the cases above, the economic environment tended to ease the cartel’s task. At least until about 2011, non-OPEC production growth was mainly confined to Russia. Moreover, political events often bolster OPEC’s efforts to limit its members’ output (such as the NATO attack on Libya and the sanctions on Iran, mentioned above). But even earlier, domestic strife surrounding “Chavismo” weakened Venezuela’s productive capacity, and Iraqi production was battered by sanctions, then invasion, then civil war.

Then, in just the past few years, American oil firms began widely implementing hydraulic fracturing (“fracking”) and other techniques for extracting energy resources from geological formations that were previously too difficult to exploit. American production of “tight oil” — also called shale oil — has made the United States one of the world’s leading oil producers. (I discussed the geostrategic implications of this boom in a recent New Atlantis essay called “Oil and World Power.”)

At the moment, many U.S. oil producers are disinvesting in tight oil. But if OPEC restrictions were to cause oil prices to rebound, many U.S. tight oil producers will begin expanding output once again. And compared to conventional wells and overseas megaprojects, tight oil wells can be completed very quickly. What’s more, U.S. tight oil producers have a backlog of almost-ready wells left over from the 2014 drilling binge that may have a productive capacity of up to 0.5 MBD. Although estimates differ about the size of this backlog and how costly it would be to bring these so-called “drilled-but-uncompleted” wells online, it seems likely that U.S. oil firms could use the backlog to respond to any OPEC supply cutback.

But U.S. tight oil is not Riyadh’s only concern — or even its most pressing. Conflict in the Persian Gulf is also impeding the quest for an accord on oil production. Iran is currently fomenting unrest among the Shi’ite minority in Saudi Arabia’s oil-rich Eastern Province as well as in some other Gulf monarchies. In response to this and other regional conflicts, Saudi Arabia is backing its own allies in proxy wars against Iran and its Shi’ite clients in other countries in the region, including Iraq, Syria, and Yemen, and the Saudis have broken off diplomatic and commercial ties with Iran. The two hostile blocs of Mideast oil exporters are now at daggers drawn.

Since the main antagonists on both sides depend heavily on oil exports to finance their war efforts, an oil-production quota that conferred relative gains on either of the two sides would shift the balance of power. Concerns about how an agreement would affect each country and its rival will likely outweigh other considerations, making a mutually beneficial bargain vastly more unlikely.

Mideast analyst Gregory Gause argues that conflicts need not preclude agreements on oil production quotas; the disputants’ shared economic interests can prevail. As evidence, Gause points to the 1986–1987 OPEC production cuts, which were implemented in the midst of the Iran-Iraq war (during which Saudi Arabia was openly backing Iraq). But this argument is somewhat tendentious, since it omits several critical details. In 1986, the two belligerent nations were unable to agree on mutually acceptable quotas, and Iraq temporarily seceded from OPEC. And while the new quotas reduced OPEC’s total output in 1987, by 0.82 MBD, neither Iraq nor Iran restrained production. On the contrary, OPEC data show that Iraq’s annual output rose by 26 percent during this time, while Iran’s rose by 13 percent. Annual OPEC production only fell because the Saudis and others made production cuts large enough to offset the increases.

Moreover, the diplomatic constellation in 2016 is less favorable to a deal than in 1986. Iran insists that it must fully regain its pre-sanction production levels before considering any limits on growth — and no one can be sure how long that will take. The Energy Information Administration projects that in 2016 average annual Iranian crude oil output will rise by about 0.3 MBD. EIA also foresees Iran’s production reaching about 3.3 MBD by the end of 2016 and 3.7 MBD by the end of 2017. To do so, Iran must increasingly rely on foreign capital and technology. But this will be challenging as some sanctions remain in place, and the Islamic Revolutionary Guard Corps is battling with President Rouhani and his supporters for control of the rents from new oil and gas investments. So long as the internal balance of power remains murky, foreign investors will remain skittish. Thus the Saudis might well discount Iran’s capacity to expand oil output or else adopt a wait-and-see attitude before offering any concessions.

And beyond OPEC looms the Russia problem. While some Russian officials have expressed support for an agreement to freeze current production levels (contingent on other exporters doing the same), Russian production cuts could be problematic. Russia’s Siberian wells were drilled through permafrost, and many of them produce large amounts of water. For this reason, some Russian oilmen argue that curbing production could cause freezing in these wells and even burst well casings (though others dispute such claims).

Also, Russia has in the past reneged on promises to reduce production, as it did in 1998. Although now under tighter Kremlin control, the Russian oil and gas sector has reasons to resist or even evade production limits. Oil tax rates in Russia rise steeply at prices above $25 per barrel, and some of the biggest Russian producers, including Rosneft, are already deeply indebted to foreign firms and troubled by cash-flow problems. Thus, Russian oil firms and the foreign investors on which they depend could have more to lose from decreasing output than they stand to gain from increasing prices.

The prospects for an agreement among exporters to effectively cut production in 2016 are very slim. This month’s Doha meeting sought such an accord; it led to a diplomatic debacle. The Iranians declined to attend, while urging others to accept production cuts. The Saudis, meanwhile, refused to accept a freeze in the face of Iranian intransigence. OPEC’s schism remains deep and wide.

The chairman of Saudi Aramco, the national Saudi oil company, recently opined that his country can withstand low crude prices for a “long, long time.” It will probably have to. The International Energy Agency estimates that in 2016 OPEC surplus capacity will reach 1.5 MBD, adding still more downward pressure on crude oil prices. If the Saudis were to cut back on output now, they would cede long-term market share without bringing worthwhile relief on oil prices. Instead, the Saudis and their GCC allies are planning to expand capacity to replace the expected coming decline in U.S. tight oil production.

Although a core of U.S. tight oil producers will likely survive an extended bout of low prices, it is not clear how many. In the past, bottlenecks in the crude oil transport system along with the ban on exports of U.S. crude oil forced tight oil producers to discount their output relative to that of other producers — a disadvantage in the battle to survive low prices. But there is good news on both fronts: new pipelines and other transportation improvements are easing some of the bottlenecks, and Congress lifted the decades-old export ban in December 2015. The U.S. tight oil price discount is fast disappearing. Moreover, the oilfield-services firms that oil producers hire have been heavily discounting their rates to provide the cash needed to keep their doors open. For example, in mid-2014, the average daily rate for a horizontal drilling rig was $26,000–28,000; by early 2016, it was less than $13,000.

New production techniques may also help oil producers to weather the low prices. But the geological realities of tight oil formations continue to pose stubborn challenges. A recent study found that in the first year of production, the rate of output for tight oil wells typically falls by 60–79 percent, and within three years production declines by more than 85 percent. However, oil producers have found ways to increase the recovery rate of tight oil wells or otherwise extend their productive lives. For example, simply placing a steel plate across part of the flow path of the oil can preserve some pressure within the well. And with oil prices so low, later gains in production are likely to more than offset initial losses. Instead of spending $3 million to $6 million to drill and use a new well, artificial lift can return most wells to 50–75 percent of their initial production rates for $500,000 or less.

These developments are helping U.S. tight oil producers to retain assets that will support a revival of U.S. crude oil output, if prices recover. In other words, American oil production offers a source of spare productive capacity that will exert downward pressure on long-term global crude oil prices. Of course, it remains to be seen how much of the U.S. tight oil asset base will survive the current low prices and to what extent U.S. investors will be willing to risk their money given the Saudis’ ability to ramp up production. But while Riyadh no doubt wishes that the U.S. tight oil technologies had never been developed, Saudi policymakers are astute enough to know there’s no going back.

Saudi Arabia has long maintained significant spare oil production capacity — not for altruistic reasons but to prevent oil price spikes from weakening either long-term demand for Saudi oil or U.S. willingness to help defend Saudi Arabia from its foes. Still, on at least two notable occasions, Saudi spare capacity has been a boon to the United States. When the First Gulf War reduced Iraq’s and Kuwait’s combined average daily crude oil production by about 2.5 MBD, Riyadh raised its own 1991 average daily output by about 1.7 MBD the following year, rapidly reducing the effects of that conflict on the price of oil and the global economy. And, as we saw above, Saudi Arabia repeated this pattern on a smaller scale after the Libyan crisis in 2011. This congruence of interests, which is built into the U.S.–Saudi trade relationship, has produced enough benefits for both parties that the relationship is likely to survive the conflicts that continue to roil it.

At present, the world crude oil market is clearly serving U.S. interests admirably, with various factors, including the core of U.S. tight oil producers, coming together to restrain oil prices.

It’s strange, then, that President Obama has lately adopted an adversarial tone toward domestic oil producers. As recently as early 2015, the president was boasting about increased U.S. oil production and falling gasoline prices. But by late 2015, his administration was threatening to veto the legislation then under discussion to lift the ban on crude oil exports, arguing that the country should instead move to a “low-carbon economy.” (He was eventually forced to sign the legislation, once it was bundled with a major budget bill.) And then his 2016 State of the Union address promised to hasten the transition to renewable energy sources.

Meanwhile, the two contenders for the Democratic Party presidential nomination want to go even further. Senator Bernie Sanders of Vermont has pledged to ban fracking and to end all new leases allowing the extraction of fossil fuels from federal lands. Former Secretary of State Hillary Clinton for her part vows to ban fracking in all but a few places. These promises, if implemented, would profoundly affect the world oil market.

A recent Energy Information Administration study found that in 2015, fracked wells produced about 51 percent of all U.S. crude oil and that fracked wells in tight oil formations produced more than 4.3 MBD of crude oil. Were fracking to be banned, the industry would halt efforts to drill or complete new tight oil wells. Hence, Senator Sanders’s proposed ban would cause 4.3 MBD of U.S. tight oil production to disappear. And given tight oil wells’ steep decline rates, the loss in output would be fast. At a stroke, a ban on fracking would render useless the entire physical, human, and financial infrastructure needed to produce U.S. tight oil. Secretary Clinton’s proposal would have similar (though slightly less extreme) effects.

Pledges like these may help in the battle for the Democratic nomination — after all, recent opinion polls suggest that a majority of U.S. voters now oppose fracking, and among Democrats the majority is strong. Moreover, the environmental movement is a key constituency of the Democratic Party. Oil producers are not, to say the least. Thus, when these two interests clash, all else being equal, Democratic politicians (save those representing major oil producing states) tend not to side with the oil producers. (Among the current Republican candidates, Senator Ted Cruz and Governor John Kasich are both from states in which fracking is used extensively, and Donald Trump appears to support the concept.)

Yet, despite the candidates’ bold statements, the president does not have the power to ban fracking on non-federal land — where most tight oil and gas production occurs — unless Congress first enacts new laws. And, without Democratic control of both houses, Congress would be unlikely to ban fracking, as a majority of Republican voters still support fracking. The oil industry is important to the economies of a number of states. Even in those states without much oil production, a ban on fracking would harm companies and workers in the larger oil industry supply chain. The legislative process offers many veto points at which opponents could delay or derail a bill that would cause visible harm to so many.

In reality, the normal regulatory process poses a bigger risk to the future of U.S. oil production than an outright ban on fracking. Since June 2015, President Obama has changed his mind about opening the south Atlantic to offshore drilling. And his administration is imposing more stringent standards on ozone emissions from oil and gas wells and new curbs on flaring, and further tightening methane standards for oil and gas operations on private, state, and federal lands. If the next president is inclined to limit fracking, he or she might choose to follow President Obama’s lead by tightening regulations.

In the early years of the tight oil boom, public concerns about jobs and gasoline prices were running very high and may have constrained President Obama’s policies toward tight oil producers. At that time, the surge in tight oil production was a bright spot in a bleak employment picture as well as a downward force on high fuel prices. Those concerns are increasingly in abeyance today. And now the Democratic nominees for president have pledged to end tight oil production or at least to curtail it severely. Washington, then, might be poised to do something beyond the power of the GCC states: it could largely negate U.S. tight oil producers’ capacity to limit future oil price spikes.

Perhaps U.S. oil producers and oil consumers should focus less on the supposed threat posed by Riyadh and more on that posed by Washington.