Incremental Mutation Testing with CCACHE Corrupts Full Run Results

[ligogeorge]

When using incremental mutation testing (via gitDiffRef and gitProjectRoot in mull.yml) with ccache, subsequent full mutation testing runs produce incomplete or corrupted results (e.g., fewer mutants or incorrect kill statuses). This seems to happen because ccache caches bitcode from the incremental run’s subset of changed files, causing mismatches in full runs. Clearing ccache (ccache -C) fixes it, but this shouldn’t be required.

Environment

mull-runner-18 --version
Mull: Practical mutation testing for C and C++
Home: https://github.com/mull-project/mull
Docs: https://mull.readthedocs.io
Support: https://mull.readthedocs.io/en/latest/Support.html
Version: 0.26.1
Commit: 859af43
Date: 06 Apr 2025
LLVM: 18.1.3

[AlexDenisov]

This one might be tricky unless I'm missing something obvious. @ligogeorge do you have a clear suggestion what mull should be doing to avoid the cache-related issues? I think even changing the config file should invalidate any cache, but I'm not sure if ccache has such options.

[ligogeorge]

This is what I do now

• Add mull.yml to the CCACHE_EXTRAFILES environment variable so that any mutation testing configuration changes invalidate the cache.
• Update the workflow to add gitDiffRef to mull.yml only after building unit tests, ensuring diffs are used exclusively for mutation testing runs (not for builds).

[AlexDenisov]

Do you have a suggestion on what Mull could do to improve ccache compatibility?