Pyth White Paper

My notes and reading on Pyth Whitepaper.

Financial market data is often only accessible to a limited set of institutions and users.
Traditional markets typically maintain strict control over live and historical price feeds.
Cryptocurrency markets currently have fewer barriers, although there is no guarantee this arrangement will continue.
Consequently, only a selected group of users has access to the most timely, accurate, and valuable information.

The Pyth network aims to bring this valuable financial market data to Defi applications and the general public.
The network does so by incentivizing market participants - trading firms, market makers, and exchanges - to share the price data collected as part of their existing operations.
The network aggregates this first-party price data and publishes it on-chain for use by either on or off-chain applications.

The original white paper proposed an initial protocol design focused on the Solana blockchain.
This second version updates the design to reflect the current status of the Pyth ecosystem.
In particular, this version describes a novel cross-chain model for price delivery to target chains.

The Pyth Data Association, in collaboration with the community of Pyth network participants, has published this white paper to describe the state of the Pyth network.

Overview

The Pyth protocol is designed to incentivize participants to continually publish price updates for various products (such as BTC/USD).
Each product has a price feed that continually updates with its current price and a confidence interval representing the estimated uncertainty of the price.
For example, the current BTC/USD feed may say the price $65000 +- $50.
The feed for each product is published on-chain, where it is read by consumers, who may be either blockchain enabled programs (on various blockchains) or off-chain applications.
An on-chain program produces the price feed for each product by aggregating the price feeds of individual publishers.
The protocol is designed to attract publishers who are first party data providers with the ability to source high quality timely pricing information.
Thus the protocol has two different sets of participants:
- Publishers: Publish price feeds and earn fees in exchange. Publishers are typically market participants with access to accurate and timely price information.
- Consumers: Read price feeds and incorporate data into smart contracts or dApps. In the process, the consumers pay fees to the protocol to access the data. Fees are charged at a fixed rate per Pyth price update on a target chain 0.0001 ETH per update. Consumers are users of on-chain protocols that integrate Pyth for market data. These protocols may be running on various blockchains.

To make Pyth prices available to a wide range of consumers, the protocol must operate over multiple blockchains.
The Pyth protocol uses an appchain called Pythnet to store and update the state of each price feed.
Pythnet is a proof-of-authority blockchain, where every publisher runs a validator.
Pyth prices are broadcast from this appchain to other target chains by way of a cross-chain architecture that uses decentralized cross-chain messaging protocols, such as the Wormhole network.
Broadcasting data from Pythnet makes it easy for the protocol to support large number of target chains, thereby enabling consumers to use Pyth data no matter what blockchain they are on.

With this architecture, the aforementioned participants will interact via two mechanisms.
Both of these mechanisms will be implemented on one of the various chains detailed above:
- Price aggregation combines the price feeds of individual publishers into a single price feed for the product.
  - This mechanism is designed to produce robust price feeds, that is, feeds whose prices cannot be significantly influenced by small group of publishers.
  - This mechanism runs on Pythnet as an on-chain program.
- Pyth Governance determines high level parameters of the other mechanisms.

A critical challenge is designing these mechanics to be robust to various forms of price manipulation. Two specific attacks to consider are
- Participants could onboard as publishers and attempt to manipulate the oracle price.
  - The price aggregation mechanism is designed to guard against this attack by limiting the influence of publishers on the aggregate price.
  - Furthermore, the community will determine how publishers are permitted to use the protocol and can choose the limit set to highly reputable and honest participants
- Bad actors could try to move prices on exchanges to influence the aggregate Pyth price.
  - Robustness in the price aggregation and a diversity of publishers sourcing from different exchanges can decrease the risk of price manipulation efforts.

Cross Chain Architecture

Pyth features publishers who provide price data on Pythnet, where the aggregation mechanism, produces aggregate prices.
The aggregated prices are subsequently broadcast and transferred to other block chains where consumers can use them.
Pythnet is an application specific blockchain on which the only programs and activity are related to price publishing, aggregation, and initiation of the transfer to other blockchains.
Pythnet's consensus model and runtime are based off those of Solana.

A successful cross-chain architecture must achieve several important properties:
- Minimize transaction costs: Maintaining price feeds on multiple blockchains requires paying transaction costs on every single chain.
  - The resulting cost could be too expensive for the participants to bear.
- Support high frequency and low latency price updates.
  - These two properties are essential for applications that require timely price information.
- Scale to many blockchains
  - There are use cases for prices on different blockchains that feature a diversity of runtimes, consensus models, and gas costs.
  - If cross-chain transfer of price data is too expensive, it will not be possible to reach all these chains.
- Scale to many price feeds
  - The Pyth protocol should scale to support price feeds for a wide range of crypto and non-crypto assets.
  - As with scaling to many blockchains, cross-chain transfer of price data being too expensive will inhibit reaching this scale.

The cross-chain architecture is a "pull" oracle that addresses these challenges using a three part workflow.
The first part of the workflow is publishing prices on chain efficiently and cheaply.
In contrast to generalized blockchains that can feature high gas costs, on Pythnet, permissioned publishers are able to publish their updates for free.
These permissioned publishers can submit price updates with a valid price and confidence to the on-chain program, which stores their values in their field of the corresponding on-chain price account.
When invoked, the oracle program carries out aggregation on the prices in the price account.

The second part is a price broadcast operation performed by the Pythnet validators as part of Pythnet consensus.
After the oracle program aggregates publishers' prices into an aggregate price for each of the price feeds updated in a slot, it calls into the message buffer program with the aggregate prices of the updated price feeds.
The message buffer program allows for the creation of messages to be sent to other chains and can be called by authorized programs.
Here, the message buffer stores the aggregate prices of the updated price feeds.
Next, the validator reads the message buffer's accounts and constructs the merkle tree with the aforementioned prices as the leaves of the tree.
It then broadcasts the root of the tree to other chains using a cross-chain messaging protocol, such as Wormhole network.
The wormhole network is decentralized cross-chain messaging system that enables data on one blockchain to be read on other chains.
The broadcast operation is intended to be invoked at high frequency, essentially at the frequency of the Pythnet block time.
The output of this broadcast is a stream of attested merkle tree roots that are available via the cross-chain messaging infrastructure and the data of the trees themselves publicly visible on Pythnet for a rolling window of historical slots.

The third part is relaying, which occurs on a target chain where Pyth prices are available.
Each target chain has a Pyth receiver contract which stores a price for every Pyth price feed.
These contracts implement a permissionless operation to update the stored price.
The input to this operation is an attested merkle tree root, a price update for a particular symbol that represents a leaf in the corresponding merkle tree, and the path in the tree that leads from the leaf to the root.
The latter two can be accessed publicly on Pythnet.
The operation performs various checks on the message; if the update is valid, the new prices are stored in the Pyth receiver contract.

Consumers of Pyth prices are responsible for invoking permissionless price update, i.e. pulling the price update on chain, before using the price.
The price stored in the Pyth receiver contract grows stale over time unless it is updated.
Protocols integrating with Pyth price feeds can impose a limit on the maximum staleness of the price.
Anyone interacting with one of these protocols will therefore be required to invoke the permissionless price update before their interaction if the current on-chain price is stale.

In this design, consumers pay the transaction cost for updating Pyth price feeds only when the price is needed.
This property is how the cross chain architecture meets the goals set forth above.
The broadcast component can stream high frequency price updates for many different price feeds without incurring transaction costs.
Many of these price updates will never land on a target chain (and therefore never incur transaction cost).
Target chain transaction costs are themselves minimized because consumers only incur them when they need to use a price for an operation.

Update Fees

Consumers are required to pay an update fee to the protocol in order to invoke the price update operation on a Pyth receiver contract.
Fees are charged in the native token of the target chain.

Price Aggregation

The price aggregation mechanism combines each individual publisher's price and confidence feed into a single aggregate price and confidence feed.
For example, one publisher may say that the price of BTC/USD is $52000 +- 10 and another that it is $53000 +- 20, and price aggregation may combine these two prices into an aggregate price of $52500 +- 500.
This mechanism is part of the on-chain program and triggers when publishers submit price updates: the first price update on a given slot automatically aggregates the price from the previous slot.

The price aggregation algorithm is designed to have three properties:
- It is resistant to manipulation - both accidental and intentional - by publishers. For example, if most publishers submit a price of $100 and one publisher submits a price of $80, the aggregate price remains accurate even if small percentage of publishers submit price far from the market.
- The aggregate price appropriately weights data sources with different levels of accuracy.
  - The pyth protocol allows publishers to submit a confidence interval because they have varying levels of accuracy in observing the price of a product.
  - For example, some publishers are expected to be exchanges. Exchanges have different levels of liquidity, and less liquid exchanges tend to have wider bid/offer spreads than more liquid ones.
  - This property can result in situations where one exchange reports $101 +- 1, and another reports $110 +- 10.
  - Furthermore, the product's spread is a fundamental limit on the precision of the product's price.
- The aggregate confidence interval reflects the variation between publishers' prices.
  - In real world markets there is no single price for any given product.
  - Products trade at slightly different prices at various venues given at any time.
  - Furthermore, the product's spread is a fundamental limit on the precision of the product's price.
  - The aggregate confidence interval reflects both the variation across venues and these limitations.

The price aggregation algorithm uses a variant of weighted median.
An input to the algorithm is a stake weight for each publisher.
These weights will initially be set uniformly, but the future may be set using additional mechanisms as determined by the community.
The first step of the algorithm computes the aggregate price by giving each publisher three votes - one vote at their price and one vote each of their price plus and minus their confidence interval - then taking the stake weighted median of the votes.
The second step computes the distance from the aggregate price to the stake weighted 25th and 75th percentiles of the votes, then selects the larger as the aggregate confidence interval.

This simple algorithm is a generalization of the ordinary median.
Most people understand the median as the middle value in the data set, that is the 50th percentile.
However, the median is also the value $R$ that minimizes the objective function $\sum_i|{R-p_i}|$ where $p_i$ is the price of the $i$ th publisher.
This function penalizes $R$ based on its distance from the publisher's price $p_i$ .
The proposed algorithm computes the aggregate price $R$ that minimizes $\frac{1}{3}\sum_i S_i |R - p_i| + \frac{2}{3} \sum_i S_i \max(|R-p_i| - c_i, 0)$ where $S_i$ is the publisher's stake weight and $c_i$ is the publisher's confidence interval.
This objective does two things.
First it weights the publishers according to their stake, such that low-stake publishers have minimal influence on the price.
Second it combines the ordinary median objective with a second term that only assigns a penalty to $R$ if it lies outside the publisher's confidence interval.

Price Aggregation Algorithm Example

The price aggregation algorithm is a variant of the weighted median, designed to aggregate prices from multiple publishers. Each publisher provides:

A price $p_i$ .
A confidence interval $c_i$ , defining the range $[p_i - c_i, p_i + c_i]$ .
A stake weight $S_i$ , reflecting their influence.

Algorithm Overview

The algorithm has two main steps:

Aggregate Price Calculation:
- Each publisher gets three votes:
  - One at $p_i$ .
  - One at $p_i - c_i$ .
  - One at $p_i + c_i$ .
- Compute the stake-weighted median of all votes to determine the aggregate price $R$ .
Aggregate Confidence Interval:
- Compute the stake-weighted 25th and 75th percentiles of the votes.
- Calculate the distances from $R$ to these percentiles.
- The larger distance is the aggregate confidence interval.

The algorithm minimizes the objective function: $\frac{1}{3} \sum_i S_i |R - p_i| + \frac{2}{3} \sum_i S_i \max(|R - p_i| - c_i, 0)$

The first term penalizes $R$ based on its distance from each publisher's price, weighted by $S_i$ .
The second term penalizes $R$ if it lies outside a publisher's confidence interval ( $|R - p_i| > c_i$ ).

This generalizes the standard median, which minimizes $\sum_i |R - p_i|$ .

Example

Consider 3 publishers with the following data:

Publisher 1: $p_1 = 100$ , $c_1 = 10$ , $S_1 = 1$
- Votes: $100 - 10 = 90$ , $100$ , $100 + 10 = 110$
Publisher 2: $p_2 = 105$ , $c_2 = 5$ , $S_2 = 1$
- Votes: $105 - 5 = 100$ , $105$ , $105 + 5 = 110$
Publisher 3: $p_3 = 120$ , $c_3 = 15$ , $S_3 = 1$
- Votes: $120 - 15 = 105$ , $120$ , $120 + 15 = 135$

Stakes are uniform ( $S_i = 1$ ) for simplicity, but the algorithm supports non-uniform stakes.

Step 1: Compute Aggregate Price

Collect all votes:

Publisher 1: $90, 100, 110$
Publisher 2: $100, 105, 110$
Publisher 3: $105, 120, 135$

Total votes (9 votes, each with stake $S_i = 1$ ): $\{90, 100, 100, 105, 105, 110, 110, 120, 135\}$

Sort the votes: $[90, 100, 100, 105, 105, 110, 110, 120, 135]$

To find the stake-weighted median:

With uniform stakes, the median is the middle value.
For 9 votes, the median is the 5th value ( $\lfloor (9+1)/2 \rfloor = 5$ ).
Sorted: $[90, 100, 100, 105, \mathbf{105}, 110, 110, 120, 135]$

Thus, the aggregate price is: $R = 105$

Step 2: Compute Aggregate Confidence Interval

Compute the stake-weighted 25th and 75th percentiles:

Total stake: $9 \times 1 = 9$ .
25th percentile: Position = $0.25 \times (9 + 1) = 2.5$ , interpolate between the 2nd and 3rd values.
75th percentile: Position = $0.75 \times (9 + 1) = 7.5$ , interpolate between the 7th and 8th values.

Sorted votes: $[90, 100, 100, 105, 105, 110, 110, 120, 135]$

25th percentile:
- 2nd value = 100, 3rd value = 100.
- At position 2.5: Since values are identical, 25th percentile $\approx 100$ .
75th percentile:
- 7th value = 110, 8th value = 120.
- At position 7.5: $110 + 0.5 \times (120 - 110) = 110 + 5 = 115$ .

Distances from $R = 105$ :

To 25th percentile: $|105 - 100| = 5$ .
To 75th percentile: $|105 - 115| = 10$ .

The aggregate confidence interval is the larger distance: $\text{Confidence Interval} = 10$

Result

Aggregate Price: $R = 105$
Aggregate Confidence Interval: $\pm 10$ (range $[95, 115]$ )

Objective Function Evaluation

The algorithm minimizes: $\frac{1}{3} \sum_i S_i |R - p_i| + \frac{2}{3} \sum_i S_i \max(|R - p_i| - c_i, 0)$

Let's evaluate this for $R = 105$ (the aggregate price) and compare with $R = 100$ .

For $R = 105$

First term: $\frac{1}{3} \sum_i S_i |R - p_i|$
- Publisher 1: $|105 - 100| = 5$ , $S_1 = 1$ , contribution = $5$ .
- Publisher 2: $|105 - 105| = 0$ , $S_2 = 1$ , contribution = $0$ .
- Publisher 3: $|105 - 120| = 15$ , $S_3 = 1$ , contribution = $15$ .
- Sum: $5 + 0 + 15 = 20$ .
- Weighted: $\frac{1}{3} \times 20 = \frac{20}{3} \approx 6.67$ .
Second term: $\frac{2}{3} \sum_i S_i \max(|R - p_i| - c_i, 0)$
- Publisher 1: $|105 - 100| = 5$ , $5 - 10 = -5$ , $\max(-5, 0) = 0$ .
- Publisher 2: $|105 - 105| = 0$ , $0 - 5 = -5$ , $\max(-5, 0) = 0$ .
- Publisher 3: $|105 - 120| = 15$ , $15 - 15 = 0$ , $\max(0, 0) = 0$ .
- Sum: $0 + 0 + 0 = 0$ .
- Weighted: $\frac{2}{3} \times 0 = 0$ .
Total: $6.67 + 0 = 6.67$ .

For $R = 100$

First term:
- Publisher 1: $|100 - 100| = 0$ , contribution = $0$ .
- Publisher 2: $|100 - 105| = 5$ , contribution = $5$ .
- Publisher 3: $|100 - 120| = 20$ , contribution = $20$ .
- Sum: $0 + 5 + 20 = 25$ .
- Weighted: $\frac{1}{3} \times 25 = \frac{25}{3} \approx 8.33$ .
Second term:
- Publisher 1: $|100 - 100| = 0$ , $0 - 10 = -10$ , $\max(-10, 0) = 0$ .
- Publisher 2: $|100 - 105| = 5$ , $5 - 5 = 0$ , $\max(0, 0) = 0$ .
- Publisher 3: $|100 - 120| = 20$ , $20 - 15 = 5$ , $\max(5, 0) = 5$ .
- Sum: $0 + 0 + 5 = 5$ .
- Weighted: $\frac{2}{3} \times 5 = \frac{10}{3} \approx 3.33$ .
Total: $8.33 + 3.33 = 11.66$ .

Since $6.67 < 11.66$ , $R = 105$ is preferred, confirming the algorithm's choice.

Why This Works

Generalization of Median: The standard median minimizes $\sum_i |R - p_i|$ . This algorithm extends it by:
- Stake Weighting: Higher $S_i$ increases a publisher's influence.
- Confidence Intervals: The second term penalizes $R$ only if $|R - p_i| > c_i$ , encouraging $R$ to stay within trusted ranges.
Three Votes: Votes at $p_i$ , $p_i - c_i$ , and $p_i + c_i$ balance the central price with confidence bounds, making the median robust to outliers.
Confidence Interval Output: The 25th and 75th percentiles capture the spread of votes, providing uncertainty around $R$ .

Key Takeaways

The aggregate price ( $R = 105$ ) aligns closely with Publishers 1 and 2, with Publisher 3's higher price having less influence due to its distance.
The confidence interval ( $\pm 10$ ) reflects the vote spread, with the larger distance to the 75th percentile indicating data asymmetry.
The objective function confirms $R = 105$ minimizes both deviations and penalties for being outside confidence intervals.