Skip to content

lock the static token type and rule index caches in C++ Recognizer - #4996

Open
abdul7056 wants to merge 1 commit into
antlr:devfrom
abdul7056:cpp-recognizer-cache-lock
Open

abdul7056 wants to merge 1 commit into
antlr:devfrom
abdul7056:cpp-recognizer-cache-lock

Conversation

@abdul7056

Copy link
Copy Markdown
  1. Recognizer::getTokenTypeMap() and Recognizer::getRuleIndexMap() read and insert into the static _tokenTypeMapCache and _ruleIndexMapCache while holding _mutex, which is a per-instance member. Two lexers or parsers running on separate threads therefore modify the same std::map concurrently and corrupt it. Both are reached through getTokenType() and Parser::getRuleIndex(), which XPath and ParseTreePatternMatcher use.
  2. Both functions now lock one file-scope mutex shared by all recognizers, matching the Java runtime, which synchronizes on the cache object itself.

The new RecognizerTest calls both getters from 8 threads on distinct recognizers. On the current dev it hung or crashed in 7 of 20 runs; with the change it passed 30 of 30, and the full C++ test suite passes.

getTokenTypeMap() and getRuleIndexMap() update static caches while holding
only the per-instance _mutex, so recognizers on different threads race on
the same std::map. Use one mutex shared by all recognizers, like the Java
runtime which synchronizes on the cache itself.

Signed-off-by: obsidian_nomad <abdul@sproutxp.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant