Add support for new beacon chain /blobs endpoint by Tristan-Wilson · Pull Request #3826 · OffchainLabs/nitro

Tristan-Wilson · 2025-10-14T16:13:30Z

This change adds support for the new beacon chain endpoint /eth/v1/beacon/blobs/{block_id} introduced in Fusaka while maintaining backward compatibility with the legacy endpoint
/eth/v1/beacon/blob_sidecars/{slot}. block_id can be a slot so Nitro just uses slot.

The new endpoint supports server-side filtering by versioned hash via query parameters. Since the Arbitrum sequencer inbox message contains the versioned hashes of the blobs that were posted, we can include those in the query.

Key changes:

Added UseLegacyEndpoint flag to BlobClientConfig to control which endpoint to use
Created new GetBlobsBySlot() public method for direct slot-based blob fetching
Implemented getBlobs() method for the new endpoint with versioned hash verification
Updated beaconRequest() to support query parameters for filtering
Added KZG commitment verification when versioned hashes are provided

Created blobtool CLI utility for testing both endpoints:

 # Fetch specific blob using new endpoint (default)
 blobtool fetch --beacon-url=<url> --slot=<slot> --versioned-hashes=<hash>

 # Fetch using legacy endpoint (requires versioned hashes)
 blobtool fetch --beacon-url=<url> --slot=<slot> --versioned-hashes=<hash> --use-legacy-endpoint

 # Compare both endpoints side-by-side
 blobtool fetch --beacon-url=<url> --slot=<slot> --versioned-hashes=<hash> --compare-endpoints

The new endpoint is used by default, with automatic fallback behavior maintained through the existing secondary beacon URL mechanism.

Spec reference: https://github.com/ethereum/beacon-APIs/blob/master/apis/beacon/blobs/blobs.yaml

Testing

To test the new fetching method I added a new tool called blobtool. It's intended to be used for testing beacon endpoints, and it can use either the new or old method for fetching blobs. This is even more important after Fusaka where it is even more difficult to configure beacon nodes to have and keep all blobs. In future we may want to add some features like looking up blobs by block number, which is something operators often want to do.

$ target/bin/blobtool fetch --beacon-url=https://sepolia --slot 8726798                              

Fetching all blobs for slot 8726798 using new blobs endpoint...
Successfully fetched 4 blobs in 550.156533ms
Blob 0: versioned_hash=0x01ac7e2c7fff656a02bb7d8e9934798ab3d1df8fbf20acff5aec87b183fd99cf, size=131072 bytes
Blob 1: versioned_hash=0x01808fa4fb38311c3a2a7c8a37d95af5a18eb3ccfe279d7e02f5df7d839b4a3a, size=131072 bytes
Blob 2: versioned_hash=0x01e0b7530de207b244ce87448a633d7bb077339236d4a4e82985b944553913dc, size=131072 bytes
Blob 3: versioned_hash=0x012873697f6a3baa3ed5b0e53784a2b389fee4c341253a5ef69a85bd720d2aad, size=131072 bytes

$ target/bin/blobtool fetch --beacon-url=https://sepolia --slot 8726798 --versioned-hashes=0x01ac7e2c7fff656a02bb7d8e9934798ab3d1df8fbf20acff5aec87b183fd99cf,0x01808fa4fb38311c3a2a7c8a37d95af5a18eb3ccfe279d7e02f5df7d839b4a3a,0x01e0b7530de207b244ce87448a633d7bb077339236d4a4e82985b944553913dc                               
Fetching 3 blobs for slot 8726798 using new blobs endpoint...
Successfully fetched 3 blobs in 3.075861523s
Blob 0: versioned_hash=0x01ac7e2c7fff656a02bb7d8e9934798ab3d1df8fbf20acff5aec87b183fd99cf (computed=0x01ac7e2c7fff656a02bb7d8e9934798ab3d1df8fbf20acff5aec87b183fd99cf), size=131072 bytes
Blob 1: versioned_hash=0x01808fa4fb38311c3a2a7c8a37d95af5a18eb3ccfe279d7e02f5df7d839b4a3a (computed=0x01808fa4fb38311c3a2a7c8a37d95af5a18eb3ccfe279d7e02f5df7d839b4a3a), size=131072 bytes
Blob 2: versioned_hash=0x01e0b7530de207b244ce87448a633d7bb077339236d4a4e82985b944553913dc (computed=0x01e0b7530de207b244ce87448a633d7bb077339236d4a4e82985b944553913dc), size=131072 bytes

$ target/bin/blobtool fetch --beacon-url=https://sepolia --slot 8726798 --versioned-hashes=0x01ac7e2c7fff656a02bb7d8e9934798ab3d1df8fbf20acff5aec87b183fd99cf,0x01808fa4fb38311c3a2a7c8a37d95af5a18eb3ccfe279d7e02f5df7d839b4a3a,0x01e0b7530de207b244ce87448a633d7bb077339236d4a4e82985b944553913dc --use-legacy-endpoint                             
Fetching 3 blobs for slot 8726798 using legacy blob_sidecars endpoint...
Successfully fetched 3 blobs in 3.254657814s
Blob 0: versioned_hash=0x01ac7e2c7fff656a02bb7d8e9934798ab3d1df8fbf20acff5aec87b183fd99cf (computed=0x01ac7e2c7fff656a02bb7d8e9934798ab3d1df8fbf20acff5aec87b183fd99cf), size=131072 bytes
Blob 1: versioned_hash=0x01808fa4fb38311c3a2a7c8a37d95af5a18eb3ccfe279d7e02f5df7d839b4a3a (computed=0x01808fa4fb38311c3a2a7c8a37d95af5a18eb3ccfe279d7e02f5df7d839b4a3a), size=131072 bytes
Blob 2: versioned_hash=0x01e0b7530de207b244ce87448a633d7bb077339236d4a4e82985b944553913dc (computed=0x01e0b7530de207b244ce87448a633d7bb077339236d4a4e82985b944553913dc), size=131072 bytes

$ blobtool fetch --beacon-url=https://sepolia --versioned-hashes=0x01ac7e2c7fff656a02bb7d8e9934798ab3d1df8fbf20acff5aec87b183fd99cf,0x01808fa4fb38311c3a2a7c8a37d95af5a18eb3ccfe279d7e02f5df7d839b4a3a,0x01e0b7530de207b244ce87448a633d7bb077339236d4a4e82985b944553913dc --compare-endpoints
Comparing legacy blob_sidecars and new blobs endpoints...

Fetching with legacy blob_sidecars endpoint...
✓ Legacy endpoint: fetched 3 blobs in 2.242901587s

Fetching with new blobs endpoint...
✓ New endpoint: fetched 3 blobs in 498.302806ms

Comparing blob data...
  Blob 0: ✓ identical (0x01ac7e2c7fff656a02bb7d8e9934798ab3d1df8fbf20acff5aec87b183fd99cf)
  Blob 1: ✓ identical (0x01808fa4fb38311c3a2a7c8a37d95af5a18eb3ccfe279d7e02f5df7d839b4a3a)
  Blob 2: ✓ identical (0x01e0b7530de207b244ce87448a633d7bb077339236d4a4e82985b944553913dc)

Performance comparison:
  Legacy endpoint: 2.242901587s
  New endpoint:    498.302806ms
  New endpoint is 77.8% faster

This change adds support for the new beacon chain endpoint `/eth/v1/beacon/blobs/{block_id}` introduced in Fusaka while maintaining backward compatibility with the legacy endpoint `/eth/v1/beacon/blob_sidecars/{slot}`. block_id can be a slot so Nitro just uses slot. The new endpoint supports server-side filtering by versioned hash via query parameters. Since the Arbitrum sequencer inbox message contains the versioned hashes of the blobs that were posted, we can include those in the query. Key changes: - Added `UseLegacyEndpoint` flag to BlobClientConfig to control which endpoint to use - Created new `GetBlobsBySlot()` public method for direct slot-based blob fetching - Implemented `getBlobs()` method for the new endpoint with versioned hash verification - Updated `beaconRequest()` to support query parameters for filtering - Added KZG commitment verification when versioned hashes are provided Created `blobtool` CLI utility for testing both endpoints: ``` # Fetch specific blob using new endpoint (default) blobtool fetch --beacon-url=<url> --slot=<slot> --versioned-hashes=<hash> # Fetch using legacy endpoint (requires versioned hashes) blobtool fetch --beacon-url=<url> --slot=<slot> --versioned-hashes=<hash> --use-legacy-endpoint # Compare both endpoints side-by-side blobtool fetch --beacon-url=<url> --slot=<slot> --versioned-hashes=<hash> --compare-endpoints ``` The new endpoint is used by default, with automatic fallback behavior maintained through the existing secondary beacon URL mechanism. Spec reference: https://github.com/ethereum/beacon-APIs/blob/master/apis/beacon/blobs/blobs.yaml

james-prysm · 2025-10-14T16:34:34Z

+		queryParams.Add("versioned_hashes", hash.Hex())
+	}
+
+	response, err := beaconRequest[[]hexutil.Bytes](b, ctx, fmt.Sprintf("/eth/v1/beacon/blobs/%d", slot), queryParams)


for others looking at this pr,
confirmed this was intentional, if versioned hashes are not provided all will be returned, this was intentional to add the query params

yep, when posting 4844 enabled batches the sequencer inbox message contains the versioned hashes as calldata. So the new API is actually really nice because we can only fetch what we need, in the order we want it, and also we can verify the blobs retrieved against the versioned hash.

james-prysm · 2025-10-14T16:41:07Z

-func beaconRequest[T interface{}](b *BlobClient, ctx context.Context, beaconPath string) (T, error) {
-	// Unfortunately, methods on a struct can't be generic.
-
+func beaconRequest[T interface{}](b *BlobClient, ctx context.Context, beaconPath string, queryParams url.Values) (T, error) {


Prysm should learn from this request setup ( we could also use some generics here)

james-prysm · 2025-10-14T16:43:33Z

+	var blobs []kzg4844.Blob
+	var err error
+	if b.useLegacyEndpoint {
+		blobs, err = b.blobSidecars(ctx, slot, versionedHashes)


We don't use versionedHashes on the legacy endpoint, it's blob indicies

This is a quirk of how our blob fetching works for Nitro for the legacy endpoint since does client side filtering to filter out non related blobs.

james-prysm · 2025-10-14T16:45:08Z


 func (b *BlobClient) blobSidecars(ctx context.Context, slot uint64, versionedHashes []common.Hash) ([]kzg4844.Blob, error) {
-	rawData, err := beaconRequest[json.RawMessage](b, ctx, fmt.Sprintf("/eth/v1/beacon/blob_sidecars/%d", slot))
+	rawData, err := beaconRequest[json.RawMessage](b, ctx, fmt.Sprintf("/eth/v1/beacon/blob_sidecars/%d", slot), nil)


oh good, it's nil here, right we don't use versioned hashes here do we need to provide that?

Exactly, the versioned hashes are just used for client side filtering for the old endpoint, since we don't know what index a particular blob is at we had to do it entirely client side.

I agree. Why even pass in versionedHashes if they aren't going to be used in the body of the function?

please see my reponse to James

james-prysm · 2025-10-14T16:50:08Z

+		if queryParams != nil {
+			beaconUrl.RawQuery = queryParams.Encode()
+		}
+		req, err := http.NewRequestWithContext(ctx, "GET", beaconUrl.String(), http.NoBody)


response might be faster with ssz type instead of json

It works now so we can leave this for later.

james-prysm · 2025-10-14T16:53:08Z

+	}
+
+	if config.UseLegacyEndpoint && len(versionedHashes) == 0 {
+		return fmt.Errorf("--versioned-hashes is required when using --use-legacy-endpoint")


I don't think this is correct? ( or are you using versioned hashes for something else outside of the endpoint query)

should be required when not using the new endpoint (based on implementation?)

This is a quirk of how our blob fetching works for Nitro for the legacy endpoint since does client side filtering to filter out non related blobs.

eljobe · 2025-10-14T16:59:59Z

+
+func parseFetchConfig(args []string) (*FetchConfig, error) {
+	f := flag.NewFlagSet("blobtool fetch", flag.ContinueOnError)
+	f.String("beacon-url", "", "Beacon Chain RPC URL")


Can you document how much of the URL is expected? Or, maybe provide a default, or an example?
I take it that we don't include the "/blobs" part of the URL, but do we stop at v1 or just before eth?

It's just the base url, so none of the /eth/v1/beacon stuff. I don't know what to include as a default.

eljobe · 2025-10-14T17:03:58Z


 func (b *BlobClient) blobSidecars(ctx context.Context, slot uint64, versionedHashes []common.Hash) ([]kzg4844.Blob, error) {
-	rawData, err := beaconRequest[json.RawMessage](b, ctx, fmt.Sprintf("/eth/v1/beacon/blob_sidecars/%d", slot))
+	rawData, err := beaconRequest[json.RawMessage](b, ctx, fmt.Sprintf("/eth/v1/beacon/blob_sidecars/%d", slot), nil)


I agree. Why even pass in versionedHashes if they aren't going to be used in the body of the function?

joshuacolvin0

LGTM

codecov · 2025-10-14T17:57:20Z

Codecov Report

❌ Patch coverage is 0% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 22.73%. Comparing base (d58fe46) to head (4537ec9).
⚠️ Report is 6 commits behind head on v3.7.0-backports.

Additional details and impacted files

@@                 Coverage Diff                  @@
##           v3.7.0-backports    #3826      +/-   ##
====================================================
+ Coverage             22.68%   22.73%   +0.04%     
====================================================
  Files                   383      383              
  Lines                 58209    58253      +44     
====================================================
+ Hits                  13206    13241      +35     
- Misses                42963    42981      +18     
+ Partials               2040     2031       -9

Tristan-Wilson requested review from eljobe and james-prysm October 14, 2025 16:13

Tristan-Wilson assigned eljobe Oct 14, 2025

james-prysm reviewed Oct 14, 2025

View reviewed changes

gofmt to fix lint error

f6465c6

james-prysm reviewed Oct 14, 2025

View reviewed changes

eljobe requested changes Oct 14, 2025

View reviewed changes

eljobe assigned Tristan-Wilson and unassigned eljobe Oct 14, 2025

More descriptive help text for beacon-url

4537ec9

eljobe approved these changes Oct 14, 2025

View reviewed changes

eljobe added the design-approved label Oct 14, 2025

joshuacolvin0 approved these changes Oct 14, 2025

View reviewed changes

Tristan-Wilson mentioned this pull request Oct 14, 2025

Add support for new beacon chain /blobs endpoint #3830

Merged

amsanghi approved these changes Oct 14, 2025

View reviewed changes

Tristan-Wilson mentioned this pull request Oct 14, 2025

Add support for new beacon chain /blobs endpoint #3831

Merged

joshuacolvin0 merged commit 91ebb83 into v3.7.0-backports Oct 14, 2025
15 checks passed

joshuacolvin0 deleted the v3.7.0-backports-support-beacon-blob-api branch October 14, 2025 18:27

Conversation

Tristan-Wilson commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Tristan-Wilson Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshuacolvin0 left a comment

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Tristan-Wilson commented Oct 14, 2025 •

edited

Loading

Tristan-Wilson Oct 14, 2025 •

edited

Loading

codecov Bot commented Oct 14, 2025 •

edited

Loading