Ryzen AI 1.8 Whisper Medium encoder compile does not finish and cached graph times out on Ryzen AI 9 365`

[ScottLV-eng]

Description

I have been using version 1.7.0 and would like to migrate to version 1.8.0, but ran into these issues. When testing the Ryzen AI 1.8 Demos/ASR/Whisper example with AMD's pinned
amd/whisper-medium-onnx-npu artifacts. The Medium NPU path does not produce a
working encoder cache on an AMD Ryzen AI 9 365 laptop.

The established AMD whisper.cpp/FlexMLRT 1.7 path works correctly on this same
laptop with both pinned Whisper Medium and Large-v3 models. Both models have
completed repeated recorded-audio matrices with positive NPU encoder-execution
confirmation and no NPU fallback. The failure described below occurs while
evaluating the newer Ryzen AI 1.8 ONNX encoder/decoder path.

With the demo's supplied configuration, compilation created encoder and decoder
.rai files, but the first encoder inference failed with:

Subgraph vaiml_par_0 execution failed: Command failed to complete successfully
(ERT_CMD_STATE_TIMEOUT): XRT run index 0 failed!
Error State=ERT_CMD_STATE_TIMEOUT

Rebooting Windows and running AMD's unmodified run_whisper.py reproduced the
same first-encoder timeout.

I then applied the Ryzen AI 1.8 release-note requirement for
openai-whisper-medium-encoder:

FE_EXPERIMENT+=enable-binary-blob-inputs=0

using a clean cache key along with the demo's documented Medium options
optimize_level=3 and --system-stack-size=512. The clean encoder compilation
continued for more than 30 minutes without producing a final .rai file or
reaching inference.

Environment

• Processor: AMD Ryzen AI 9 365 with Radeon 880M
• OS: Windows 11 Home, version 10.0.26200, build 26200
• NPU driver: 32.0.203.314 dated 2025-10-10
• Ryzen AI SDK/source: 1.8.0
• Python: 3.12.11, conda-forge
• ONNX Runtime: 1.27.0 with VitisAI Execution Provider
• Model repository: amd/whisper-medium-onnx-npu
• Model revision: 8d3f2be23b32c4082edeca14322de112f1f0d709
• encoder_model.onnx SHA-256:
  78911A1CDCCD4FC48CCDD69EE12C715CA009061DA0B9B8CA635C297AA1BABC91
• decoder_model.onnx SHA-256:
  998ABB0AAE48AEE4A17F07672888FF45AFD9B01FD039E952ECE544784A01299C

Reproduction

From RyzenAI-SW-1.8.0/Demos/ASR/Whisper in the Ryzen AI environment:

python .\run_whisper.py `
  --model-type whisper-medium `
  --device npu `
  --input C:\path\sample.wav `
  --encoder C:\path\whisper-medium-onnx-npu\encoder_model.onnx `
  --decoder C:\path\whisper-medium-onnx-npu\decoder_model.onnx `
  --config-file .\config\model_config.json `
  --language es

The input is mono 16 kHz PCM WAV. The same failure occurs after reboot.

Controls

• The existing AMD whisper.cpp/FlexMLRT 1.7 backend works correctly with both
  pinned Medium and Large-v3 models on this laptop. A completed 70-evaluation
  baseline matrix had no scenario failure or NPU fallback, and all 60 NPU
  evaluations positively confirmed NPU encoder execution. Medium averaged
  0.3333 RTF and Large-v3 averaged 0.6127 RTF across that matrix.
• The working FlexMLRT 1.7 backend offloads the Whisper encoder and uses the CPU
  for most decoding. It is not the same full encoder/decoder ONNX execution
  path that fails in Ryzen AI 1.8, so this comparison confirms functioning NPU
  hardware and proven Medium/Large-v3 compatibility without claiming that the
  full ONNX path previously worked in 1.7.
• Large-v3 Turbo encoder and decoder ONNX sessions work in the same Ryzen AI
  1.8 environment and on the same NPU. A 56.2-second English recording ran at
  0.265 RTF with correct transcription.
• AMD whisper.cpp rebuilt against FlexMLRT 1.8 successfully runs the existing
  pinned Medium and Large-v3 .rai encoders and positively confirms NPU
  inference. This suggests the NPU, driver, and FlexMLRT installation are
  generally operational.
• FlexMLRT 1.8 did not improve the existing whisper.cpp encoder timing relative
  to 1.7, but both versions completed successfully.

Questions

1. What is the exact JSON syntax/location for applying
   FE_EXPERIMENT+=enable-binary-blob-inputs=0 to this demo's BF16 encoder
   configuration?
2. Is revision 8d3f2be23b32c4082edeca14322de112f1f0d709 compatible with Ryzen AI 1.8
   and NPU driver 32.0.203.314 on Ryzen AI 9 365?
3. Is a validated precompiled Medium encoder/decoder cache available for this
   hardware/runtime combination?
4. Are there additional compiler flags, a newer model revision, or a driver
   revision that resolves ERT_CMD_STATE_TIMEOUT?
5. Is this Medium ONNX behavior a known Ryzen AI 1.8 regression or limitation,
   given that the FlexMLRT 1.7 whisper.cpp Medium and Large-v3 encoder path works
   reliably on the same system?

Project decision

The existing verified FlexMLRT 1.7 whisper.cpp backend remains unchanged.
FlexMLRT 1.8 was compatible with the pinned encoder artifacts but showed no
speed benefit and slower initialization. The full ONNX Medium path remains an
offline experiment until AMD provides a validated correction.