-Zthreads=N support implemented as an additional dimension 'Parallel' by azhogin · Pull Request #2421 · rust-lang/rustc-perf

azhogin · 2026-03-23T06:57:18Z

-Zthreads=N support implemented as an additional dimension 'Parallel'.
Default values for parallels: [1,4].

Aggregations sample:

Primary benchmarks sample:

Secondary benchmarks sample:

Graphs sample (graphs have "scenario_parallel" names like "full_par1 / full_par4"):

Filters with par1/par4 added:

nnethercote · 2026-03-23T08:22:57Z

The problem with this is that parallelism is an entirely separate dimension: parallel Full, parallel IncrFull, parallel IncrUnchanged, parallel IncrPatched are all possible. And it's not even obvious from the name ParallelFrontend which of those scenarios is the one being adapted.

azhogin · 2026-03-23T08:54:55Z

The problem with this is that parallelism is an entirely separate dimension: parallel Full, parallel IncrFull, parallel IncrUnchanged, parallel IncrPatched are all possible. And it's not even obvious from the name ParallelFrontend which of those scenarios is the one being adapted.

Yes, the idea was to add only "Full/check" at first to estimate the increased execution costs and maybe implement additional modes later. Should I rename ParallelFrontend into ParallelFull?

nnethercote · 2026-03-23T09:03:09Z

ParallelFull (or ParFull) would be clearer. But again, given that it's really a separate dimension, shoehorning it in as a scenario may cause problems down the road.

@Kobzol, do you have any thoughts/plans about this?

Kobzol · 2026-03-23T09:24:55Z

I definitely have thoughts, but I'm sick now and don't have the energy to comment. I'll try to write something by the end of this week.

petrochenkov · 2026-03-23T12:24:47Z

I wish @azhogin wrote more about the motivation and goals when posting this.
The goals here are

Get @azhogin familiar with rustc-perf codebase so he could add more scenarios like this later, or potentially integrate use of tools like https://github.com/Zoxc/rcb to rustc-perf
Have something working to start talking with the infra team about resources available for benchmarking parallel frontend.

Having some of the check benchmarks running with parallel frontend will have the lowest cost/benefit ratio

One one hand we don't spend too much resource because the check runs are the cheapest ones
One another hand the parallel frontend doesn't affect LLVM, so measuring only the check part will show largest changes if something related to parallel frontend changes
Once some of the benchmarks is running we can look at the results and see how much the time variates, how the other metrics behave, and think whether we need something like rcb integrated instead of cloning the regular single-threaded setup.

lqd · 2026-03-23T13:30:09Z

Just as a note: we have I think at least one benchmark running with the parallel frontend (but 4 threads).

This setup doesn’t look like it should be a scenario to me either, but let’s wait for jakub’s thoughts once he feels better.

Kobzol · 2026-03-24T13:27:54Z

So, first, let me say that I agree that we should be preparing for benchmarking the parallel frontend in a better way, and I'm happy to help with that, both on the rustc-perf and the infra side!

Regarding the implementation, we should not be adding a new scenario. As Nick said, -Zthreads is an entirely new dimension, that works essentially with all existing scenarios and profiles. I'm sure that in the future we will want to benchmark most of them with the parallel frontend, and we will also want to benchmark different thread counts, which wouldn't be possible with this being a scenario.

That being said, we could do something temporary to benchmark only a few situations with the parallel frontend. And we actually already did that last year 😅 We have the serde benchmark running with 4 threads, across all scenarios and profiles. In terms of variance, it does not seem egregiously higher than the base variant (parallel vs serial), at least for the serde benchmark.

So I guess that if we want to extend this, we have the several options:

Add more hardcoded parallel benchmarks for other crates, or more thread counts for serde. This is easy to do and doesn't require almost any code changes, but it's not very "scalable", of course.
Add the thread count as another axis for compilation benchmarks, and either select a few benchmarks for which we will use higher values than 1, or enable e.g. 4 threads for all check profiles, or something like that. I don't think that we can afford enabling 2/4/8 for everything though, that would be too much work to benchmark, even with the two benchmarking machines that we have today.
Go all-in and just enable -Zthreads=<value that will be used on stable> for all benchmarks, and treat the parallel frontend as the default mode. We might want to go this way eventually, but I'm not sure if the parallel frontend is ready for this today.

So I think that 2) is the way forward. It's not completely trivial to add a new axis to the compilation benchmarks, but it's also not super hard, it "just" needs some boilerplate code that propagates the new config option throughout the collector, the database, the website backend, and then also the frontend. We have a bunch of PRs from last year that added the target compilation benchmark axis, so it would be mostly a copy-paste of that.

One additional thing to note is that we haven't yet figured out the best metric to check here, I think. I'm not sure if cycles really work, as they will grow with the number of threads. That being said, we mostly use rustc-perf to compare historical data for the same configuration, rather than to compare across configurations, so it doesn't matter that much. But still it would be good go have a metric that can show small changes in parallel code.

Regarding

Have something working to start talking with the infra team about resources available for benchmarking parallel frontend.

Happy to talk about that in the #t-infra stream on Zulip. I think that we have enough capacity now to add more benchmarks, so that shouldn't be a big issue.

petrochenkov · 2026-03-25T13:48:13Z

or enable e.g. 4 threads for all check profiles

enabling 2/4/8 for everything though, that would be too much work to benchmark

That's what this PR attempts to achieve, covering all the benchmarks, but only for check profiles, because using all the profiles would be too much work.

The number of threads depends on the number of cores on the benchmarking machines, if they have 8+ cores then I'd prefer to use -Zthreads=8 because that's considered the best-performing setup at the moment (even for machines with larger numbers of cores), otherwise we can use -Zthreads=4.

Go all-in and just enable -Zthreads= for all benchmarks, and treat the parallel frontend as the default mode. We might want to go this way eventually

I don't think we should go this way eventually, it will ruin icount-based benchmarking which is used as a source of truth for any changes not involving redistributing work between threads.

One additional thing to note is that we haven't yet figured out the best metric to check here, I think. I'm not sure if cycles really work, as they will grow with the number of threads.

Only wall time I guess.
Perhaps icounts/cycles too, as a secondary metric but in the opposite direction - core utilization improves (some work may need to be duplicated to achieve that) -> icounts grow. So if icounts don't change in single-threaded setup, but increase in multi-threaded setup, it may be a good sign.

petrochenkov · 2026-03-25T13:48:31Z

So I think that 2) is the way forward. It's not completely trivial to add a new axis to the compilation benchmarks, but it's also not super hard, it "just" needs some boilerplate code that propagates the new config option throughout the collector, the database, the website backend, and then also the frontend. We have a bunch of PRs from last year that added the target compilation benchmark axis, so it would be mostly a copy-paste of that.

@azhogin Could you implement this ^^^ suggestion?

azhogin · 2026-05-03T16:50:20Z

So I think that 2) is the way forward. It's not completely trivial to add a new axis to the compilation benchmarks, but it's also not super hard, it "just" needs some boilerplate code that propagates the new config option throughout the collector, the database, the website backend, and then also the frontend. We have a bunch of PRs from last year that added the target compilation benchmark axis, so it would be mostly a copy-paste of that.

'Parallel' implemented as an additional axis with default values [1, 4]. There are screenshots submitted with site pages how it looks with this change. @Kobzol, any comments about it?

Kobzol · 2026-06-11T14:37:13Z

Sorry, I didn't have time for most reviews in the past few months. Starting from next week, it should be better, I promise to take a look then.

Kobzol

Left an initial review, sorry that this took so long, I didn't have much time for reviews, or Rust for that matter, in the past few months.

I'd like for the website changes to be split into a separate PR, and land just the collector + DB side first, as the PR is quite big already.

Kobzol · 2026-06-15T12:31:04Z

    self_profile_storage: Option<Box<dyn SelfProfileStorage>>,
    bench_rustc: bool,
    targets: Vec<Target>,
+    parallels: Vec<Parallel>,


Suggested change

parallels: Vec<Parallel>,

frontend_threads: Vec<FrontendParallelism>,

That should describe more accurately what we are talking about.

Kobzol · 2026-06-15T12:31:54Z

+    /// Parallel frontend options in comma-separated list
+    #[arg(long, value_delimiter = ',', default_value = "1,4")]
+    parallels: Vec<u32>,


Suggested change

/// Parallel frontend options in comma-separated list

#[arg(long, value_delimiter = ',', default_value = "1,4")]

parallels: Vec<u32>,

/// Parallel frontend thread values in comma-separated list

#[arg(long, value_delimiter = ',', default_value = "1")]

frontend_threads: Vec<u32>,

Kobzol · 2026-06-15T12:32:30Z

    } else {
        Scenario::all_non_incr()
    };
+    let parallels = if collector::version_supports_parallel_frontend(&toolchain.id) {


I think that for now it would be enough to just use 1 for the stable benchmarks.

Kobzol · 2026-06-15T12:33:01Z

-                    if remaining_scenarios.is_empty() {
-                        continue;
-                    }
+                    for parallel in parallels {


Suggested change

for parallel in parallels {

for thread_count in frontend_threads {

Kobzol · 2026-06-15T12:34:59Z

+use std::fmt;
+
+#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
+pub struct Parallel(pub u32);


Suggested change

pub struct Parallel(pub u32);

pub struct FrontendThreads(pub u32);

Or FrontendParallelism, I think that both works. We should clarify that it is related to the frontend though, Parallel is ambigous.

Kobzol · 2026-06-15T12:36:00Z

+    }
+}
+
+impl fmt::Display for Parallel {


I don't think that the display implementation should be showing the compiler flag. I'd rather remove the Display impl completely or just let it print the inner value directly.

Kobzol · 2026-06-15T12:37:10Z

            let mut cmd = self.base_command(self.cwd, cargo_subcommand);
            cmd.arg("-p").arg(self.get_pkgid(self.cwd)?);
+            if self.parallel.0 != 1 {
+                cmd.env("RUSTC_THREAD_COUNT", self.parallel.0.to_string());


What is this environment variable? Shouldn't we be using -Zthreads?

Kobzol · 2026-06-15T12:38:22Z

+pub struct Parallel(pub u32);
+
+impl Parallel {
+    pub fn par_n(self) -> String {


What is this function for?

Kobzol · 2026-06-15T12:38:37Z

+        format!("par{}", self.0)
+    }
+    // Default parallel frontend options
+    pub fn default_opts() -> Vec<Parallel> {


The default for now should just be 1 thread.

azhogin force-pushed the azhogin/par-front-scenario branch from 00b1a2c to 8b6b683 Compare March 23, 2026 07:07

azhogin force-pushed the azhogin/par-front-scenario branch 6 times, most recently from 40c2dcc to 177f175 Compare April 16, 2026 09:47

azhogin changed the title ~~ParallelFrontend scenario added for -Zthreads=8 run, check only~~ -Zthreads=N support implemented as an additional dimension 'Parallel' Apr 19, 2026

azhogin force-pushed the azhogin/par-front-scenario branch 6 times, most recently from 882188d to ef56c53 Compare April 27, 2026 06:57

azhogin force-pushed the azhogin/par-front-scenario branch from ef56c53 to 75e9c33 Compare April 29, 2026 11:40

-Zthreads=N support implemented as additional dimension 'Parallel'

3ffe80f

azhogin force-pushed the azhogin/par-front-scenario branch from 75e9c33 to 3ffe80f Compare April 29, 2026 13:10

petrochenkov mentioned this pull request Jun 11, 2026

Support parallel front end test in rustc-perf #2068

Open

Kobzol reviewed Jun 15, 2026

View reviewed changes

petrochenkov mentioned this pull request Jun 23, 2026

MCP: Stabilization strategy for rustc parallel frontend rust-lang/compiler-team#1005

Open

3 tasks

	parallels: Vec<Parallel>,
	frontend_threads: Vec<FrontendParallelism>,

	for parallel in parallels {
	for thread_count in frontend_threads {

	pub struct Parallel(pub u32);
	pub struct FrontendThreads(pub u32);

Uh oh!

Conversation

azhogin commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nnethercote commented Mar 23, 2026

Uh oh!

azhogin commented Mar 23, 2026

Uh oh!

nnethercote commented Mar 23, 2026

Uh oh!

Kobzol commented Mar 23, 2026

Uh oh!

petrochenkov commented Mar 23, 2026

Uh oh!

lqd commented Mar 23, 2026

Uh oh!

Kobzol commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petrochenkov commented Mar 25, 2026

Uh oh!

petrochenkov commented Mar 25, 2026

Uh oh!

azhogin commented May 3, 2026

Uh oh!

Kobzol commented Jun 11, 2026

Uh oh!

Kobzol left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

azhogin commented Mar 23, 2026 •

edited

Loading

Kobzol commented Mar 24, 2026 •

edited

Loading