Make dead-link check tolerant of transient failures#987
Open
ecomodeller wants to merge 1 commit into
Open
Conversation
The scheduled link check failed intermittently on slow/bot-hostile external servers (e.g. a timing-out citation host), treating a timeout the same as a 404. ScholliYT/Broken-Links-Crawler-Action has no way to distinguish transient errors from permanently-gone links. Switch to lycheeverse/lychee-action with --accept-timeouts and --accept 200..=299,403,429 so timeouts and rate-limiting no longer fail the run, while genuine 404/410 still do. lychee can't recursively crawl a live site, so build the docs and check the rendered HTML instead. Carry the previous URL exclusions over to .lycheeignore.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The scheduled broken-links check fails intermittently on slow or bot-hostile external servers — e.g. run 27932206514 failed only because a citation host (
archimer.ifremer.fr) timed out after retries. That is alert noise, not a broken link.The previous action,
ScholliYT/Broken-Links-Crawler-Action, treats a timeout identically to a 404 and exposes no option to accept transient errors. So the workflow cannot express "only fail on links that are permanently gone".This switches to
lycheeverse/lychee-action, which can:--accept-timeoutsmakes timeouts non-fatal, and--accept 200..=299,403,429tolerates rate-limiting/bot-blocking, while genuine 404/410 still fail the run.Because lychee does not recursively crawl a live site, the workflow now builds the docs (
just docs, same recipe as the docs deploy) and checks the rendered HTML underdocs/_site/. The two URL exclusions from the old config move to.lycheeignore(GitHub "edit" links and the crawler-hostile MIKE Powered by DHI doc/help portals).Verified locally against a fresh build: 3713 links OK, 0 errors (exit 0); an injected dead
dhi.github.ioURL still fails (exit 2).