Skip to content

JIT: mark vpshufbitqmb as a three-operand AVX instruction - #134287

Open
jamesburton wants to merge 2 commits into
dotnet:mainfrom
jamesburton:fix/vpshufbitqmb-three-operand
Open

jamesburton wants to merge 2 commits into
dotnet:mainfrom
jamesburton:fix/vpshufbitqmb-three-operand

Conversation

@jamesburton

Copy link
Copy Markdown
Contributor

Fixes #134286.

VPSHUFBITQMB has the form k1 {k2}, xmm2, xmm3/m128 — a kmask destination plus two source operands — but its row in instrsxarch.h was missing INS_FLAGS_IsDstDstSrcAVXInstruction. Every other three-operand mask-producing EVEX instruction in the table has it: vptestmb/d/q/w, vptestnmb/d/q/w, vpcmpb, vpcmpub, vpcmpw, vpcmpuw, vpblendmb.

IsThreeOperandAVXInstruction is a straight lookup of INS_FLAGS_Is3OperandInstructionMask, so without the flag it returns false and emitIns_R_R_S / emitIns_R_R_R_I / emitIns_R_R_S_I assert:

Assertion failed 'IsThreeOperandAVXInstruction(ins) || IsApxExtendedEvexInstruction(ins)'
    File: src\coreclr\jit\emitxarch.cpp:8572

No shipping code is affected. Nothing maps to INS_vpshufbitqmb today — the row has been dormant since it was added, which is why this went unnoticed. This is a latent table defect, so there is no regression test to add here; the instruction has no reachable code path to test until a consumer exists.

Found while implementing the BITALG intrinsics for #96162, where Avx512BitAlg.ShuffleBits becomes the first consumer and trips the assert immediately. Sending it separately so the table is correct regardless of what happens to that work.

Validation

Verified locally on Zen 5 (AMD Strix Halo, which supports BITALG), Checked JIT:

  • with the flag added and the [API Proposal]: AVX-512 VPOPCNTDQ and BITALG Intrinsics #96162 intrinsics wired up, ShuffleBits emits and executes correctly across Vector128/256/512 for both byte and sbyte, at Tier0 and with DOTNET_TieredCompilation=0;
  • the full JIT/HardwareIntrinsics/X86_Avx512 suite passes 1775/1775;
  • this PR on its own (just the one-line table change) builds clean and is a no-op at runtime, since nothing reaches the instruction.

I also checked vpcmpd / vpcmpq / vpcmpud / vpcmpuq, which lack the flag while their b/w siblings have it and are reachable. Empirically they do not reach any asserting emit path (Vector512.LessThanOrEqual<int>, GreaterThan<uint>, LessThan<ulong>, register and memory operands, Checked JIT), so I have left them alone — noted in the issue in case they should be made consistent.

🤖 Generated with Claude Code

VPSHUFBITQMB has the form 'k1 {k2}, xmm2, xmm3/m128' -- a kmask
destination plus two source operands. Every other three-operand
mask-producing EVEX instruction in instrsxarch.h carries
INS_FLAGS_IsDstDstSrcAVXInstruction (vpcmpb, vpcmpub, vpcmpw, vpcmpuw,
vptestmb/d/q/w, vptestnmb/d/q/w, vpblendmb); vpshufbitqmb does not.

Without the flag IsThreeOperandAVXInstruction returns false, so any
attempt to emit the instruction asserts in emitIns_R_R_S
(emitxarch.cpp:8572) on a Checked JIT, and would emit a malformed
encoding on Release.

The row is currently unreachable -- no HARDWARE_INTRINSIC entry maps to
INS_vpshufbitqmb -- so no shipping code is affected today. It was found
while implementing the BITALG intrinsics for dotnet#96162, whose
Avx512BitAlg.ShuffleBits would be the first consumer. Fixing it
separately so the table is correct independently of that work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 19, 2026 20:47
@github-actions github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Sep 19, 2026
@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Sep 19, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 5 pipeline(s).
11 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The change is a low-risk metadata fix confined to the instruction table, with only a minor formatting nit noted in review comments.

Review effort: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

Updates the x86 instruction metadata table so vpshufbitqmb is classified as a three-operand AVX instruction (dst + two sources), aligning it with other EVEX mask-producing instructions and preventing emitter assertions when/if a future consumer starts using this instruction.

Changes:

  • Add INS_FLAGS_IsDstDstSrcAVXInstruction to the vpshufbitqmb row in instrsxarch.h.
File Description
src/​coreclr/​jit/​instrsxarch.h Marks vpshufbitqmb as a 3-operand AVX instruction in the instruction table metadata.

Comment thread src/coreclr/jit/instrsxarch.h Outdated
INST3(vpshrdvw, "vpshrdvw", IUM_WR, BAD_CODE, BAD_CODE, SSE38(0x72), ILLEGAL, ILLEGAL, INS_TT_FULL_MEM, Input_16Bit | KMask_Base8 | REX_W1 | Encoding_EVEX) // Concatenate and Variable Shift Packed Data Right Logical
INST3(vpshrdw, "vpshrdw", IUM_WR, BAD_CODE, BAD_CODE, SSE3A(0x72), ILLEGAL, ILLEGAL, INS_TT_FULL_MEM, Input_16Bit | KMask_Base8 | REX_W1 | Encoding_EVEX) // Concatenate and Shift Packed Data Right Logical
INST3(vpshufbitqmb, "vpshufbitqmb", IUM_WR, BAD_CODE, BAD_CODE, SSE38(0x8F), 6C, 1C, INS_TT_FULL_MEM, Input_8Bit | KMask_Base16 | REX_W0 | Encoding_EVEX) // Shuffle Bits From Quadword Elements Using Byte Indexes Into Mask
INST3(vpshufbitqmb, "vpshufbitqmb", IUM_WR, BAD_CODE, BAD_CODE, SSE38(0x8F), 6C, 1C, INS_TT_FULL_MEM, Input_8Bit | KMask_Base16 | REX_W0 | Encoding_EVEX | INS_FLAGS_IsDstDstSrcAVXInstruction) // Shuffle Bits From Quadword Elements Using Byte Indexes Into Mask

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feedback is correct and should be accounted for

Addresses the Copilot review nit: the added flag had compressed the
REX_W0 / Encoding_EVEX columns. Now matches the surrounding rows and
vptestmb's formatting exactly (Encoding_EVEX at column 255).

Whitespace only; no functional change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings September 19, 2026 21:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The change is a targeted, consistent metadata fix that correctly enables IsThreeOperandAVXInstruction for vpshufbitqmb with no broader behavioral risk.

Review effort: Lite
Findings: 1 Low severity

Open (1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JIT: vpshufbitqmb is missing INS_FLAGS_IsDstDstSrcAVXInstruction

3 participants