Skip to content

[perfnet] chaindata growth problem in EL backward downloader #15879

Description

@taratorio

the EL backward downloader does not take into account --sync.loop.block.limit (defaults to 5,000 blocks) when doing long backward syncs

we need to modify the logic:

  • if a backward download header chain is 1024 in NewPayload - skip it and return syncing
  • the CL will eventually have to send a ForkChoiceUpdated request for that large backward chain if it is a legit one - within a FCU we allow long backward downloads
  • when doing these long backward downloads with FCU - we add logic to take into account --sync.loop.block.limit - so we insert blocks to chain data (in batches of 500 blocks as currently done) then when we reach 5,000 blocks inserted we call UpdateForkChoice to trigger execution + prunning (to keep chaindata size low)

I attempted this on branch: https://github.com/erigontech/erigon/commits/performance-fix-large-chaindata
There is 1 small error I need to fix but the above logic should work

Even with this fix however there was still unexplained chaindata growth - maybe due to issues in our prunning logic or something else to do with long held roTx for the in-mem memory mutation batch - stopped here need to continue debugging efforts but the above change to the EL backward downloader can be shipped irrespectively of this (the work can be split in 2 parts)

Activity

  1. self-assigned this
    on Jul 1, 2025
  2. VBulikov commented on Jul 1, 2025

    @VBulikov
    Member

    taratorio added Imp2 as Importance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions