Skip to content

Let resource_set callbacks opt into the inputs' total size - #31833

Draft
rdesgroppes wants to merge 1 commit into
bazelbuild:masterfrom
rdesgroppes:resource-set-inputs-bytes
Draft

rdesgroppes wants to merge 1 commit into
bazelbuild:masterfrom
rdesgroppes:resource-set-inputs-bytes

Conversation

@rdesgroppes

@rdesgroppes rdesgroppes commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Description

Starlark resource_set callbacks declaring an inputs_bytes parameter (or **kwargs) now also receive the total size in bytes of the action's expanded inputs, as that keyword argument. Other callbacks keep being called with exactly (os, inputs_size).

Local runners pass the size lazily to the new Spawn.getLocalResources(InputsBytes), as computed by SpawnExecutionContext.getInputsBytes() from the input mapping and metadata they already hold. It's therefore only summed for declarations asking for it, and without downloading inputs. ResourceSetOrBuilder gains needsInputsBytes() and a 3-argument buildResourceSet, both defaulting to the existing behavior, so native estimators are unaffected. Callers without input metadata at hand, such as tests, pass InputsBytes.UNKNOWN explicitly.

Motivation

Link steps' memory scales with the bytes they link, which the input count misses for the links most at risk of OOM kills, as measured in #31827.

Fixes #31827.

Build API Changes

Does this PR affect the Build API? (e.g. Starlark API, providers, command-line flags, native rules)
=> Yes
If yes, please answer the following:

  1. Has this been discussed in a design doc or issue? (Please link it)
    => Pass the total size in bytes of an action's inputs to Starlark resource_set callbacks #31827.
  2. Is the change backward compatible?
    => Backward compatible: callbacks only receive inputs_bytes when declaring it (or **kwargs).
  3. If it's a breaking change, what is the migration plan?
    => Not a breaking change.

Checklist

  • I have added tests for the new use cases (if any).
  • I have updated the documentation (if applicable).

Release Notes

RELNOTES[NEW]: resource_set callbacks declaring an inputs_bytes parameter receive the total size in bytes of the action's inputs.

@rdesgroppes
rdesgroppes force-pushed the resource-set-inputs-bytes branch 2 times, most recently from e96a5fc to 91aea18 Compare October 17, 2026 21:14
### Description
Starlark `resource_set` callbacks declaring an `inputs_bytes` parameter (or `**kwargs`) now also receive the total size in bytes of the action's expanded inputs, as that keyword argument.
Other callbacks keep being called with exactly `(os, inputs_size)`.

Local runners pass the size lazily to the new `Spawn.getLocalResources(InputsBytes)`, as computed by `SpawnExecutionContext.getInputsBytes()` from the input mapping and metadata they already hold.
It's therefore only summed for declarations asking for it, and without downloading inputs.
`ResourceSetOrBuilder` gains `needsInputsBytes()` and a 3-argument `buildResourceSet`, both defaulting to the existing behavior, so native estimators are unaffected.
Callers without input metadata at hand, such as tests, pass `InputsBytes.UNKNOWN` explicitly.

### Motivation
Link steps' memory scales with the bytes they link, which the input count misses for the links most at risk of OOM kills, as measured in bazelbuild#31827.

Fixes bazelbuild#31827

### Build API Changes
Yes
1. Discussed in bazelbuild#31827.
2. Backward compatible: callbacks only receive `inputs_bytes` when declaring it (or `**kwargs`).
3. Not a breaking change.

### Checklist

- [x] I have added tests for the new use cases (if any).
- [x] I have updated the documentation (if applicable).

### Release Notes

RELNOTES[NEW]: `resource_set` callbacks declaring an `inputs_bytes` parameter receive the total size in bytes of the action's inputs.
@rdesgroppes
rdesgroppes force-pushed the resource-set-inputs-bytes branch from 91aea18 to 22791fd Compare October 18, 2026 15:47

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pass the total size in bytes of an action's inputs to Starlark resource_set callbacks

1 participant