Tracking Issue for Explicit Tail Calls

[WaffleLapkin]

This is a tracking issue for the RFC "Explicit Tail Calls" (rust-lang/rfcs#3407) (note that RFC is not accepted yet, this feature is currently implemented as part of an experiment).
The feature gate for the issue is #![feature(explicit_tail_calls)].

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Implement the RFC
  • Parsing/HIR/MIR
  • Implement the necessary checks for tail calls
  • Add support in LLVM backend (draft commits: dee279d, 0722a1e)
    • Handle indirect ABI arguments
• Adjust documentation (see instructions on rustc-dev-guide)
• Stabilization PR (see instructions on rustc-dev-guide)

Unresolved Questions

• What exactly are the semantics of passing indirect arguments? What about by-move arguments? Also see Miri reports UB for explicit tail call with by-value non-ZST struct parameter #144820.
• We need to wait for good LLVM support of all supported targets before stabilization

(fill when the rfc is merged)

Related features

• Tracking Issue for extern "tail" calling convention #157427
• Tracking issue for way to express intraprocedural finite state machines #132306

Implementation history

• Syntactically accept become expressions (explicit tail calls experiment) #112790
• llvm ffi: Expose CallInst->setTailCallKind #112791
• hir: Add Become expression kind (explicit tail calls experiment) #112887
• thir: Add Become expression kind #113093
• Document become keyword #113095
• Support tail calls in mir via TerminatorKind::TailCall #113128
• implement checks for tail calls #133607
• fix tail call checks wrt #[track_caller] #135973
• Don't drop types with no drop glue when building drops for tailcalls #135976
• Implement support for become and explicit tail call codegen for the LLVM backend #144232
• detect infinite recursion with tail calls in ctfe #144756
• Properly reject tail calls to &FnPtr or &FnDef #144817
• Anonymize binders in tail call sig #144835
• Forbid tail calling intrinsics #144851
• Fix tail calls to #[track_caller] functions #144865
• Enforce tail call type is related to body return type in borrowck #144917
• Dont print arg span in MIR dump for tail call #144920
• Allocate arguments from topmost frame into temporary storage before popping stack frame in init_fn_tail_call #144933
• Defer tail call ret ty equality to check_tail_calls #144915
• tail calls: fix copying non-scalar arguments to callee #150473
• add assembly test for infinite recursion with become #148907
• error when ABI does not support guaranteed tail calls #148878
• rustc_codegen: fix musttail returns for cast/indirect ABIs #148240
• Tail call diagnostics to include lifetime info #145012
• Additional tce tests #144650
• Fix an ICE observed with an explicit tail-call in a default trait method #145270
• explicit tail calls: support indirect arguments #151143
• enable PassMode::Indirect { on_stack: true, .. } tail call arguments #153361
• Do proper signature equality checking for tail calls #156007

[scottmcm]

While looking at something else, I tried using tailcc, and got sad:

LLVM ERROR: Can't handle guaranteed tail call under win64 yet
error: could not compile `core` (lib)

[travis-leith]

@scottmcm can you clarify: is this issue (llvm/llvm-project#53271) holding up implemention of explicit tail calls in rust?

[WaffleLapkin]

@travis-leith this is not technically blocking the implementation, to my knowledge. But it is a big/noticeable problem for testing & stabilization.

[fitzgen]

Heads up: It seems like the existing explicit_tail_calls support has broken in recent-ish nightly rust: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=8f366ff56ee2e8d3d16862416becc99a

error: internal compiler error: /rustc/04a318787b39732e306faf5ef6dc584990f4f417/compiler/rustc_codegen_ssa/src/mir/block.rs:1372:17: `TailCall` terminator is not yet supported by `rustc_codegen_ssa`
 --> src/main.rs:4:5
  |
4 |     become bar()
  |     ^^^^^^^^^^^^

thread 'rustc' panicked at /rustc/04a318787b39732e306faf5ef6dc584990f4f417/compiler/rustc_codegen_ssa/src/mir/block.rs:1372:17:
Box<dyn Any>
stack backtrace:
   0:     0x7f53b794557a - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h130f426f1c581292
   1:     0x7f53b80e0297 - core::fmt::write::h0cc1f9d55d39ec26
   2:     0x7f53b8fee8f3 - std::io::Write::write_fmt::h950392c34828fc8f
   3:     0x7f53b79453d2 - std::sys::backtrace::BacktraceLock::print::hbe830e5cd1ea4e2a
   4:     0x7f53b7947b51 - std::panicking::default_hook::{{closure}}::h96684a7ce75dc743
   5:     0x7f53b7947984 - std::panicking::default_hook::he7b91f334cd6a44b
   6:     0x7f53b6a4bb1f - std[6975e77990629e34]::panicking::update_hook::<alloc[3015d2a6a2c334ca]::boxed::Box<rustc_driver_impl[179cb61893474dce]::install_ice_hook::{closure#0}>>::{closure#0}
   7:     0x7f53b7948278 - std::panicking::rust_panic_with_hook::h3e218010d763dc6f
   8:     0x7f53b6a85301 - std[6975e77990629e34]::panicking::begin_panic::<rustc_errors[c590d92528b1de35]::ExplicitBug>::{closure#0}
   9:     0x7f53b6a78976 - std[6975e77990629e34]::sys::backtrace::__rust_end_short_backtrace::<std[6975e77990629e34]::panicking::begin_panic<rustc_errors[c590d92528b1de35]::ExplicitBug>::{closure#0}, !>
  10:     0x7f53b6a740e9 - std[6975e77990629e34]::panicking::begin_panic::<rustc_errors[c590d92528b1de35]::ExplicitBug>
  11:     0x7f53b6a8e481 - <rustc_errors[c590d92528b1de35]::diagnostic::BugAbort as rustc_errors[c590d92528b1de35]::diagnostic::EmissionGuarantee>::emit_producing_guarantee
  12:     0x7f53b688650d - <rustc_errors[c590d92528b1de35]::DiagCtxtHandle>::span_bug::<rustc_span[f6fa8dc89880d37b]::span_encoding::Span, alloc[3015d2a6a2c334ca]::string::String>
  13:     0x7f53b68aab98 - rustc_middle[4c9d25e356f374ed]::util::bug::opt_span_bug_fmt::<rustc_span[f6fa8dc89880d37b]::span_encoding::Span>::{closure#0}
  14:     0x7f53b68aabca - rustc_middle[4c9d25e356f374ed]::ty::context::tls::with_opt::<rustc_middle[4c9d25e356f374ed]::util::bug::opt_span_bug_fmt<rustc_span[f6fa8dc89880d37b]::span_encoding::Span>::{closure#0}, !>::{closure#0}
  15:     0x7f53b689a2fb - rustc_middle[4c9d25e356f374ed]::ty::context::tls::with_context_opt::<rustc_middle[4c9d25e356f374ed]::ty::context::tls::with_opt<rustc_middle[4c9d25e356f374ed]::util::bug::opt_span_bug_fmt<rustc_span[f6fa8dc89880d37b]::span_encoding::Span>::{closure#0}, !>::{closure#0}, !>
  16:     0x7f53b689a077 - rustc_middle[4c9d25e356f374ed]::util::bug::span_bug_fmt::<rustc_span[f6fa8dc89880d37b]::span_encoding::Span>
  17:     0x7f53b8ee827b - rustc_codegen_ssa[434aa76437273bbd]::mir::codegen_mir::<rustc_codegen_llvm[639b1ab961efbf0a]::builder::Builder>
  18:     0x7f53b8eb7af5 - rustc_codegen_llvm[639b1ab961efbf0a]::base::compile_codegen_unit::module_codegen
  19:     0x7f53b8e91d13 - <rustc_codegen_llvm[639b1ab961efbf0a]::LlvmCodegenBackend as rustc_codegen_ssa[434aa76437273bbd]::traits::backend::ExtraBackendMethods>::compile_codegen_unit
  20:     0x7f53b8e8e80d - <rustc_codegen_llvm[639b1ab961efbf0a]::LlvmCodegenBackend as rustc_codegen_ssa[434aa76437273bbd]::traits::backend::CodegenBackend>::codegen_crate
  21:     0x7f53b91ae530 - <rustc_interface[5fcd8048dad8a70b]::queries::Linker>::codegen_and_build_linker
  22:     0x7f53b8dda869 - rustc_interface[5fcd8048dad8a70b]::interface::run_compiler::<core[7be738e5bd3064fd]::result::Result<(), rustc_span[f6fa8dc89880d37b]::ErrorGuaranteed>, rustc_driver_impl[179cb61893474dce]::run_compiler::{closure#0}>::{closure#1}
  23:     0x7f53b8e9b510 - std[6975e77990629e34]::sys::backtrace::__rust_begin_short_backtrace::<rustc_interface[5fcd8048dad8a70b]::util::run_in_thread_with_globals<rustc_interface[5fcd8048dad8a70b]::util::run_in_thread_pool_with_globals<rustc_interface[5fcd8048dad8a70b]::interface::run_compiler<core[7be738e5bd3064fd]::result::Result<(), rustc_span[f6fa8dc89880d37b]::ErrorGuaranteed>, rustc_driver_impl[179cb61893474dce]::run_compiler::{closure#0}>::{closure#1}, core[7be738e5bd3064fd]::result::Result<(), rustc_span[f6fa8dc89880d37b]::ErrorGuaranteed>>::{closure#0}, core[7be738e5bd3064fd]::result::Result<(), rustc_span[f6fa8dc89880d37b]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[7be738e5bd3064fd]::result::Result<(), rustc_span[f6fa8dc89880d37b]::ErrorGuaranteed>>
  24:     0x7f53b8e9bb7a - <<std[6975e77990629e34]::thread::Builder>::spawn_unchecked_<rustc_interface[5fcd8048dad8a70b]::util::run_in_thread_with_globals<rustc_interface[5fcd8048dad8a70b]::util::run_in_thread_pool_with_globals<rustc_interface[5fcd8048dad8a70b]::interface::run_compiler<core[7be738e5bd3064fd]::result::Result<(), rustc_span[f6fa8dc89880d37b]::ErrorGuaranteed>, rustc_driver_impl[179cb61893474dce]::run_compiler::{closure#0}>::{closure#1}, core[7be738e5bd3064fd]::result::Result<(), rustc_span[f6fa8dc89880d37b]::ErrorGuaranteed>>::{closure#0}, core[7be738e5bd3064fd]::result::Result<(), rustc_span[f6fa8dc89880d37b]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[7be738e5bd3064fd]::result::Result<(), rustc_span[f6fa8dc89880d37b]::ErrorGuaranteed>>::{closure#1} as core[7be738e5bd3064fd]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  25:     0x7f53b8e9bf6b - std::sys::pal::unix::thread::Thread::new::thread_start::hd156374755bce2a3
  26:     0x7f53b34ae609 - start_thread
  27:     0x7f53b33d3353 - clone
  28:                0x0 - <unknown>

This was reported as #127520 but that was WONTFIXed, so cross-posting here for posterity.