fix: prevent crashes in jieba text_exists and rate_limit flush_cache#37548
Open
ifer47 wants to merge 1 commit into
Open
fix: prevent crashes in jieba text_exists and rate_limit flush_cache#37548ifer47 wants to merge 1 commit into
ifer47 wants to merge 1 commit into
Conversation
1. Jieba keyword table: `text_exists()` crashes with TypeError when
`_get_dataset_keyword_table()` returns an empty dict `{}` because
`set.union(*{}.values())` requires at least one argument. The method
only guarded against `None` but `_get_dataset_keyword_table` also
returns `{}` at line 184. Fixed by changing the guard from
`keyword_table is None` to `not keyword_table`.
2. Rate limiting: `flush_cache()` has a TOCTOU race condition between
`redis_client.exists()` and `redis_client.get()`. If the Redis key
expires between the two calls, `get()` returns `None` and calling
`.decode("utf-8")` on it raises `AttributeError`. Fixed by replacing
the `exists()` + `get()` pattern with a single `get()` call and a
`None` check on the result.
Co-Authored-By: zhipu/glm-5 <zai-org@claude-code-best.win>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
text_exists()crashes withTypeErrorwhen_get_dataset_keyword_table()returns an empty dict{}, becauseset.union(*{}.values())requires at least one argument. The method only guarded againstNonebut_get_dataset_keyword_tablereturns{}at line 184. Fixed by changing the guard fromkeyword_table is Nonetonot keyword_table.flush_cache()had a race condition betweenredis_client.exists()andredis_client.get(). If the Redis key expires between the two calls,get()returnsNoneand calling.decode("utf-8")on it raisesAttributeError. Fixed by replacing theexists()+get()pattern with a singleget()call and aNonecheck.Reproduction
Bug 1 —
set.unionon empty dict:This is reachable when
_get_dataset_keyword_table()returns{}, which happens at line 184 when no keyword table data exists.Bug 2 — Redis key expires between
exists()andget():Test plan
text_exists()with empty dict{}— verifies it returnsFalseinstead of crashingflush_cache()when Redis key disappears — verifies it falls back to local value instead of crashingtest_should_sync_max_requests_from_redis_on_subsequent_flushto remove staleexists.return_valuemock (no longer needed after fix)🤖 Generated with Claude Code Best