Skip to content

fix cv8 debug information generation for code sections with partial coverage in files - #292

Open
SEt-t wants to merge 1 commit into
netwide-assembler:masterfrom
SEt-t:master
Open

SEt-t wants to merge 1 commit into
netwide-assembler:masterfrom
SEt-t:master

Conversation

@SEt-t

@SEt-t SEt-t commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Fixes silent fail when building old versions of libvpx on Windows.

Also removed the same type name on cv8_state struct as it's unused elsewhere and confuses Visual Studio debugger.

rdoupe added a commit to rdoupe/obs-deps that referenced this pull request Oct 18, 2026
The Windows x64 FFmpeg job has failed in every run since MSYS2
replaced nasm 2.16.03 with nasm 3.02 (here since the first run on
master, https://github.com/obs-rust/obs-deps/actions/runs/37731278276,
latest nightly:
https://github.com/obs-rust/obs-deps/actions/runs/39002391991):

  x64\Debug\vpxrc\vp8_common_x86_dequantize_mmx_asm.obj :
    fatal error LNK1136: invalid or corrupt file

libvpx's Visual Studio project assembles its Debug configuration with
"nasm -Xvc -gcv8 -f win64". libvpx 1.14.1 puts its constant tables in
"section .rodata" on win64, which NASM treats as an executable code
section there. NASM 3.02's CodeView writer then dereferences a NULL
line table for that section while writing the debug info and crashes,
leaving a 0-byte object that the MSVC linker rejects. Assembling the
15 affected libvpx 1.14.1 files with -gcv8 -f win64 segfaults with
nasm 3.02 and works with nasm 2.16.03.

libvpx fixed this in 1.17.0 (commit 8b3ce4cfa488, "x86_abi_support.asm:
use .rdata for win32/win64"; libvpx bug 540261859): read-only data goes
to .rdata on win32/win64, which also stops it being mapped executable.
The NASM side is netwide-assembler/nasm#267, with an open fix in
netwide-assembler/nasm#292.

Apply that libvpx commit as a patch on Windows instead of pinning
NASM back to 2.16.03. With the patch, every libvpx 1.14.1 x86 asm source the build
assembles also assembles with nasm 3.02 -gcv8 -f win64. The patch can be dropped when libvpx is updated to 1.17.0 or later.

Co-Authored-By: James Zern <jzern@google.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rdoupe added a commit to rdoupe/obs-deps that referenced this pull request Oct 18, 2026
The Windows x64 FFmpeg job has failed in every run since MSYS2
replaced nasm 2.16.03 with nasm 3.02 (here since the first run on
master, https://github.com/obs-rust/obs-deps/actions/runs/37731278276,
latest nightly:
https://github.com/obs-rust/obs-deps/actions/runs/39002391991):

  x64\Debug\vpxrc\vp8_common_x86_dequantize_mmx_asm.obj :
    fatal error LNK1136: invalid or corrupt file

libvpx's Visual Studio project assembles its Debug configuration with
"nasm -Xvc -gcv8 -f win64". libvpx 1.14.1 puts its constant tables in
"section .rodata" on win64, which NASM treats as an executable code
section there. NASM 3.02's CodeView writer then dereferences a NULL
line table for that section while writing the debug info and crashes,
leaving a 0-byte object that the MSVC linker rejects. Assembling the
15 affected libvpx 1.14.1 files with -gcv8 -f win64 segfaults with
nasm 3.02 and works with nasm 2.16.03.

libvpx fixed this in 1.17.0 (commit 8b3ce4cfa488, "x86_abi_support.asm:
use .rdata for win32/win64"; libvpx bug 540261859): read-only data goes
to .rdata on win32/win64, which also stops it being mapped executable.
The NASM side is netwide-assembler/nasm#267, with an open fix in
netwide-assembler/nasm#292.

Apply that libvpx commit as a patch on Windows instead of pinning
NASM back to 2.16.03. With the patch, every libvpx 1.14.1 x86 asm
source the build assembles also assembles with nasm 3.02 -gcv8 -f
win64. The patch can be dropped when libvpx is updated to 1.17.0 or
later.

Co-Authored-By: James Zern <jzern@google.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zackees pushed a commit to obs-rust/obs-deps that referenced this pull request Oct 20, 2026
The Windows x64 FFmpeg job has failed in every run since MSYS2
replaced nasm 2.16.03 with nasm 3.02 (here since the first run on
master, https://github.com/obs-rust/obs-deps/actions/runs/37731278276,
latest nightly:
https://github.com/obs-rust/obs-deps/actions/runs/39002391991):

  x64\Debug\vpxrc\vp8_common_x86_dequantize_mmx_asm.obj :
    fatal error LNK1136: invalid or corrupt file

libvpx's Visual Studio project assembles its Debug configuration with
"nasm -Xvc -gcv8 -f win64". libvpx 1.14.1 puts its constant tables in
"section .rodata" on win64, which NASM treats as an executable code
section there. NASM 3.02's CodeView writer then dereferences a NULL
line table for that section while writing the debug info and crashes,
leaving a 0-byte object that the MSVC linker rejects. Assembling the
15 affected libvpx 1.14.1 files with -gcv8 -f win64 segfaults with
nasm 3.02 and works with nasm 2.16.03.

libvpx fixed this in 1.17.0 (commit 8b3ce4cfa488, "x86_abi_support.asm:
use .rdata for win32/win64"; libvpx bug 540261859): read-only data goes
to .rdata on win32/win64, which also stops it being mapped executable.
The NASM side is netwide-assembler/nasm#267, with an open fix in
netwide-assembler/nasm#292.

Apply that libvpx commit as a patch on Windows instead of pinning
NASM back to 2.16.03. With the patch, every libvpx 1.14.1 x86 asm
source the build assembles also assembles with nasm 3.02 -gcv8 -f
win64. The patch can be dropped when libvpx is updated to 1.17.0 or
later.

Co-authored-by: James Zern <jzern@google.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant