Repository navigation
Regex yes/no-pattern has undocumented implication #151819
Description
Documentation
Apparently, when using a subexpression with the (?(id/name)yes-pattern|no-pattern) syntax that
- references a previous capture group and
- omits the optional
no-pattern
and the "yes-pattern" is not matched, the construct will also cause the originally captured group to not match, even when the input would suggest that it should. Example:
re.search(r"(<)?\w+(?(1)>)", "<body>")
<re.Match object; span=(0, 6), match='<body>'>
The above matches the opening and closing brackets as expected. However:
re.search(r"(<)?\w+(?(1)>)", "<3")
<re.Match object; span=(1, 2), match='3'>
Only the 3 is matched; the < is not part of the matched substring, and even .groups()[0] of the result will be None.
This is generally helpful, like here, because I do only want the opening bracket matched if there is also a closing one.
But the description of the (?(id/name)yes-pattern|no-pattern) syntax does not mention this behavior at all. It only details the effect of the second group that includes the query for the first, not that the behavior of the first group will be affected also.
Linked PRs
- gh-151819: Clarify backtracking semantics of
(?(id/name)yes|no)inredocs (gh-151819) #151913 - gh-151819: Clarify the conditional-pattern email example in re docs #153072
- [3.15] gh-151819: Clarify the conditional-pattern email example in re docs (GH-153072) #153073
- [3.14] gh-151819: Clarify the conditional-pattern email example in re docs (GH-153072) #153074
- [3.13] gh-151819: Clarify the conditional-pattern email example in re docs (GH-153072) #153075
Activity
picnixz commentedon Jun 22, 2026
cc @serhiy-storchaka
serhiy-storchaka commentedon Jun 23, 2026
No-no-no, it has nothing to do with yes/no-pattern, has all to do with your use of
re.search()."
(<)?\w+(?(1)>)" is equivalent to "(<)\w+>|\w+". When you usere.search()for string"<3", it first try to match the pattern from position 0, fail, then advance and try to match it from position 1, success with the second alternative.You perhaps meant to use
re.match()orre.fullmatch()which only match from the beginning of the string.ffrank commentedon Jun 23, 2026
Humm my bad, I suppose the minimal example I came up with is too reductive of the issue I actually stumbled over. Here is a more "realistic" scenario that made me wonder:
First case does the thing we want: match exactly the open/close brackets.
Second case is odd, because there is no apparent reason why the opening
<should not be part of the match. Especially since in the third case, it will be.ffrank commentedon Jun 23, 2026
(As an aside, that PR was not mine, I guess that person just tried to be helpful. I do want to wait for y'all's feedback before suggesting any actual wording.)
serhiy-storchaka commentedon Jun 23, 2026
Because if "<" is the part of the match, group 1 is defined, and if group 1 is defined,
(?(1)>)requires matching ">".ffrank commentedon Jun 23, 2026
Yes it's clear thar
<should be part of the first match, but the documentation does not explain why it's not part of the second match.The description does make clear that
cannot match the closing
>(because group 1 is in fact not set) but the behavior change of(<)?sub-expression is not explained. (The change I mean is in the difference between case 2 and 3 in the previous comment.)ffrank commentedon Jul 3, 2026
Any more insights on this?
Ishwar2736 commentedon Jul 5, 2026
i am too working for the same , soon i will be there with better ideas
serhiy-storchaka commentedon Jul 5, 2026
To summarize: this is expected behavior and not specific to the conditional construct.
(<)?[A-Z]\w+(?(1)>)is equivalent to(<)[A-Z]\w+>|[A-Z]\w+. Written that way there is no puzzle left: one alternative captures<and requires a closing>, the other captures nothing and requires no>. A string without>can only match the second alternative, so<is left out. The capture is not cleared retroactively — the engine tentatively takes<, fails to find>, and backtracks the?to empty, exactly as any quantifier backtracks. Removing the conditional (your case 3) removes that constraint, so<is taken greedily.Because this is just ordinary alternation and backtracking, there is nothing to add to the description of
(?(id/name)yes-pattern|no-pattern)specifically.The only real inaccuracy is the wording of the existing example: it says the pattern does "not match"
'<user@host.com', whereasre.searchdoes finduser@host.comat span (1, 14) withgroup(1) is None— it just does not match the whole string. I have a small doc change for that.Closing as not a bug. Thank you for the report.
gh-151819: Clarify the conditional-pattern email example in re docs (G…
Ishwar2736 commentedon Jul 5, 2026
(?(id/name)yes-pattern|no-pattern)
Attempts to match
yes-patternif the specified group has participated in thecurrent successful matching path; otherwise attempts to match
no-pattern.Note that the participation of a capturing group is determined after any
backtracking. A group that matched during an earlier unsuccessful matching
attempt may become unmatched after backtracking, so the condition is evaluated
using the capture state of the final successful match.
[3.13] gh-151819: Clarify the conditional-pattern email example in re…
[3.14] gh-151819: Clarify the conditional-pattern email example in re…
[3.15] gh-151819: Clarify the conditional-pattern email example in re…