Prices, Commands, and the Flexible Grid, Part 1

What Boiteux, Schweppe, and two decades of experimentation add to the debate over real-time electricity pricing

Source: After ChatGPT surveyed my archives for RTP posts and read the Yglesias and Schein essays I asked it to generate this image to communicate my work.

Matt Yglesias recently defended “a great idea that everyone hates“: charging residential customers prices that track conditions in wholesale power markets. The argument is straightforward and well established in electricity economics. Flat retail rates conceal the cost of serving demand during scarce, congested hours, weakening incentives to shift flexible consumption and forcing utilities to build generation, transmission, and distribution capacity that sits idle most of the year. Real-time prices would reveal those costs, letting consumers decide which uses of electricity are worth maintaining when the system strains.

Andrew Schein of Octopus Energy Group’s Centre for Net Zero responded on LinkedIn that Yglesias is “nearly right”. A virtual power plant, or VPP, is a software-coordinated aggregation of distributed energy resources—smart thermostats, batteries, EVs, rooftop solar, flexible building loads—managed as a single portfolio to provide energy and grid services. Schein agrees about the value of real-time pricing but argues that Yglesias draws too sharp a contrast between prices and VPPs, which he sees as different, often complementary, ways to make electricity demand more flexible alongside time-of-use tariffs, critical-peak pricing, and automation. VPPs can automate household responses, pool the uncertainty across individual customers, participate in wholesale markets, and provide services to transmission and distribution systems.

Both essays make important points. Yglesias is right that prices do something administrative appeals and uniform commands cannot: they communicate scarcity while letting people with different circumstances and preferences respond differently. Schein is right that price volatility, consumer inattention, automation, aggregation, and local network constraints complicate any simple prescription to place every household on an unhedged wholesale spot price.

But they both enter a long-running conversation somewhere near its end. Real-time pricing is not an idea invented for smart thermostats, electric vehicles, or renewable-heavy grids; its intellectual pedigree reaches back at least to Marcel Boiteux’s work on marginal-cost and peak-load pricing in the 1940s. Fred Schweppe and his collaborators later developed a much richer spot-price-based vision of electricity markets, work that helped shape modern wholesale market design and anticipated many of today’s questions about automation, customer choice, risk, network constraints, and distributed resources.

Recovering that history clarifies the institutional question at the center of the current debate. A virtual power plant may respond to a price and aggregate devices to sell their flexibility into a market, but the command sent to an individual thermostat, water heater, battery, or vehicle is not necessarily an economic price, much less one discovered through a market process.

A market-generated price and an engineering control signal can both change electricity consumption. They do not perform the same function (unless a market-clearing price is used as an engineering control signal, a distinction I return to below).

Before real-time pricing, there was marginal-cost pricing

Marcel Boiteux’s foundational paper on peak-load pricing appeared in French in 1949 and in English in the Journal of Business in 1960, becoming one of the central contributions to the economics of pricing services whose demand varies over time and whose capacity must be built before it is used. Electricity is the canonical case. Demand shifts hour to hour and season to season, and the cost of serving another kilowatt-hour depends on which generators are running, how close the system sits to its capacity limits, and whether transmission or distribution equipment is congested. A system built to serve the hottest afternoon or coldest morning of the year carries substantial excess capacity during ordinary hours.

Boiteux’s question went beyond whether peak customers should pay more: how should prices relate short-run operating decisions to long-run capacity investment? An additional unit of electricity during off-peak periods may require little more than fuel or operating expense; during peak periods, additional demand may also drive the need for more generating and network capacity.

The vocabulary has evolved from marginal-cost pricing and peak-load pricing to time-of-use rates, critical-peak pricing, dynamic pricing, and real-time pricing. These concepts differ in mechanics—time-of-use prices are set in advance for broad blocks of time, while real-time prices vary with actual or anticipated system conditions—but they share an underlying proposition: the price of electricity should reflect meaningful variation in the opportunity cost of producing and delivering it.

That proposition long predates the technologies that now make granular pricing technically feasible.

Schweppe’s spot-price-based marketplace

Fred Schweppe and his collaborators—Michael Caramanis, Richard Tabors, and Roger Bohn—carried this logic much further in their 1988 book Spot Pricing of Electricity, arguing that electricity should be understood as a commodity whose value and cost vary over time and space. Their proposed marketplace used spot prices to coordinate generation, transmission, distribution, consumption, and investment.

Schweppe’s framework paired spot prices with longer-term contracts: spot prices provided the economic foundation, while buyers and sellers used contracts to allocate risk, stabilize payments, and accommodate differing preferences. That distinction has been lost in some modern discussions, which frame the choice as a regulated flat rate versus full exposure to every five-minute fluctuation. A market grounded in spot prices can also support fixed-price contracts, caps, hedges, subscriptions, critical-peak provisions, bill protection, and combinations of fixed and variable charges.

Schweppe’s work is also an important intellectual precursor of security-constrained economic dispatch (SCED) and locational marginal pricing (LMP) in organized wholesale markets, which use optimization models to select a feasible, low-cost dispatch while accounting for generation offers, transmission constraints, reserve requirements, and other operating conditions. The marginal values associated with that dispatch become prices varying by time and location.

Schweppe’s framework established many of the essential ideas behind today’s wholesale-market architecture: electricity carries time- and location-specific marginal values, network constraints belong in price formation, and prices can coordinate decentralized decisions within a tightly constrained physical system.

Wholesale markets have incorporated much of this logic. Retail electricity has not.

The missing retail market

Translating marginal wholesale prices into retail prices has proved difficult, especially for residential customers.

Some obstacles were once technological. Conventional meters recorded cumulative monthly consumption but not when it occurred; communications were expensive; appliances could not respond automatically; and settlement at short intervals would have been cumbersome.

Those constraints have weakened, dramatically. Smart meters, broadband connections, cloud computing, smart thermostats, electric vehicles, batteries, heat pumps, and connected appliances now make it possible to measure and automate electricity use at increasingly fine intervals.

The remaining barriers are largely institutional and contractual. Who bears the risk of price volatility? Who recovers distribution-system fixed costs? Who controls the customer relationship? May competing retailers offer differentiated contracts? Who owns and may access meter and device data? Can a customer delegate decisions to software while retaining meaningful control? How do regulators respond when a small number of customers receive conspicuously high bills even if most save money over time?

Yglesias uses Griddy’s collapse during Winter Storm Uri to illustrate the political problem. Griddy offered Texas customers essentially uncapped exposure to the real-time wholesale price, and when prices remained at the market’s legal maximum during the crisis, some customers received bills of thousands of dollars. Yglesias acknowledges that broader price-responsive demand might have softened the spike, but he also recognizes that few consumers will accept a contract capable of producing such an outcome.

That episode revealed the hazards of one particular retail contract. It did not settle the case against real-time pricing as a family of institutions.

Yglesias eventually imagines consumers facing spot prices buying “price insurance”—paying a premium for predictable bills, exactly the contractual variety such a market should produce. I’ve long been arguing for pairing retail spot pricing with a contract similar to the kind that we have access to for travel insurance, to buy as a hedge on top of the primary transaction. Indeed, Griddy had announced exactly such a product for its customers, and it was due to launch on March 1, 2021. They didn’t make it.

The economically relevant question is whether marginal prices should inform retail contracts and automated decisions, and whether consumers should be free to choose how much price risk, automation, and delegated control they want.

Twenty-four years of thinking in public

I began working on electricity technology and regulation in 2000 and started writing Knowledge Problem in 2002. Real-time and dynamic pricing became recurring subjects because they sit at the intersection of questions that have animated my work ever since: How do prices communicate knowledge? How does technology change the feasible set of institutions? How can electricity demand become an active part of system coordination? Why does regulation so often block technically possible forms of experimentation?

The original Knowledge Problem archive contains 4,690 posts published between 2002 and 2020, followed by my continuing work at the Knowledge Problem Substack.

A post-by-post audit of the combined archive identified 44 posts substantively discussing retail real-time pricing or closely related dynamic electricity pricing; I wrote 39 of them and Mike Giberson wrote 5. Twenty-one placed RTP or dynamic pricing at the center of the article, while 23 treated it as part of a broader analysis of smart grids, demand response, retail competition, rate design, distributed resources, reliability, or regulation. Nearly 60 percent appeared between 2008 and 2014, during the most active period of early smart-grid and transactive-energy experimentation.

Several themes recur across those posts.

The first is that prices convey knowledge. A dynamic price is a compact signal of the relative scarcity of electricity at a particular time and place, not just an inducement to use less of it (Marginal Revolution University fans will recognize “a price is a signal wrapped in an incentive“). No system operator, utility regulator, aggregator, or household possesses all the information needed to determine electricity’s best use, and prices let that dispersed knowledge enter decisions without first being assembled in one mind or organization. As I wrote in 2012 and many more times since, prices act as knowledge surrogates, communicating diffuse information about scarcity.

Markets Are Knowledge Ecosystems

The second theme is that demand can contribute to reliability. Electricity policy usually treats supply as active and demand as a quantity to forecast and serve, but both can be flexible. When demand responds to prices, the system can reduce peaks, relieve congestion, use existing infrastructure more intensively, and avoid or defer generation and network investment. A market process reveals whether the lowest-cost adjustment at a given moment is another unit of production, a battery discharge, delayed vehicle charging, or a small change in temperature.

The third is that technology and prices are complements. A smart meter attached to a flat tariff mainly makes the old regulated service easier to measure and administer, whereas a smart thermostat, battery, or EV becomes far more valuable when it can respond to changing economic conditions. I argued during the early smart-grid rollout that, forced to choose, I would prefer intelligent prices paired with relatively simple technology over intelligent technology with no meaningful price signal.

Fourth, consumers differ in comfort preferences, schedules, technologies, incomes, abilities to shift consumption, and tolerances for risk. Efficient retail design should therefore offer a menu of contracts rather than impose one uniform arrangement: flat rates, time-of-use rates, critical-peak rates, real-time rates, caps, and hedges can coexist.

Fifth, retail competition matters. Competing retailers and service providers can bundle electricity with automation, risk management, batteries, EV charging, home-energy management, renewable preferences, and other services. Regulation often blocks this experimentation by continuing to assign the incumbent utility both the customer relationship and the authority to define the standard product.

Finally, automation should preserve rather than eliminate consumer agency. People should not have to stare at an app, adjusting every appliance manually; they should specify preferences—desired temperature, charging deadline, minimum battery level, maximum acceptable price to pay, minimum acceptable price to sell—and let devices or software agents act within those constraints.

What the history clarifies

This history puts the Yglesias-Schein disagreement in a different light. Yglesias is right that flat rates conceal scarcity and suppress demand response that prices could elicit more flexibly, and that prices accommodate differences in circumstances and preferences that a uniform conservation appeal cannot. But the relevant alternative to flat rates is not universal, unhedged exposure to wholesale fluctuations. The Boiteux-Schweppe tradition points toward a market grounded in marginal prices but rich in contractual variety—direct exposure, time-of-use rates, price caps, fixed-price insurance, automated response, or combinations of these—where the spot price provides the economic foundation without dictating a single retail product.

Schein is right that pricing, automation, aggregation, and risk management complement one another. VPPs can help households participate in markets, pool uncertainty, and turn dispersed flexibility into a usable resource. But calling a system a VPP does not explain how it coordinates that flexibility. An aggregator may respond to a market price upstream while sending a schedule, set point, or direct command downstream, leaving the device price-responsive only in the indirect sense that someone else translated a price into an instruction.

That distinction raises the next question: must intelligent devices be centrally dispatched by utilities or aggregators, or can they express their owners’ preferences through bids and offers, participate in a market-clearing process, and respond to prices reflecting both scarcity and physical network constraints? That capability is the promise of transactive energy—joining real-time pricing to automated individual agency, local price discovery, and consumer choice. My next article examines that architecture through the GridWise Olympic Peninsula Demonstration and the more recent TESS project, comparing it directly with the VPP model.

The future flexible grid will contain algorithms, aggregators, and automated devices regardless. The question is whether they follow commands or participate in markets on our behalf.