Add MKLMemory class to expose MKL allocated memory via Python buffer protocol

[ndgrigorian]

This PR proposes the introduction of _mkl_memory.pyx, which implements an MKLMemory class that exposes memory allocated via mkl_malloc and mkl_calloc to Python via the buffer protocol

The class uses an atomic counter incremented as __getbuffer__ and __releasebuffer__ are called to track the views on the buffer to permit use of mkl_realloc in the object (via realloc method). This concept was adapted from the PEP which revised the buffer protocol which proposed this kind of approach to tracking views on a buffer

Closes #18

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new MKLMemory Cython extension type backed by MKL’s allocator, exposes it from the top-level mkl package, and introduces tests/build changes to support C11 atomics and nogil MKL calls.

Changes:

• Introduce mkl._mkl_memory with MKLMemory (allocation, buffer protocol, pickling, realloc).
• Add pytest coverage for allocation, buffer protocol, and pickling behavior.
• Update MKL C-API declarations/build to support nogil calls and C11 atomics (plus MSVC flag).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
mkl/tests/test_mkl_memory.py	Adds tests for MKLMemory creation, buffer protocol, and pickling behavior.
mkl/_py_mkl_service.pyx	Releases the GIL around MKL buffer-free calls.
mkl/_mkl_service.pxd	Marks MKL externs as nogil and adds malloc/calloc/realloc/free declarations.
mkl/_mkl_memory.pyx	Adds the new MKLMemory Cython extension implementing allocation + buffer protocol + pickling.
mkl/init.py	Exposes MKLMemory at the package top level.
meson.build	Enables C11, adds MSVC atomics flag, and builds the new _mkl_memory extension.

[ndgrigorian]

@antonwolfy
Would be nice to get this reviewed and see if it can make it in the next release

[antonwolfy]

In overall LGTM with few minor nits below

[antonwolfy]

As a follow-up it might be good to add NumPy interop tests.
That seems never tested:

• np.asarray/np.frombuffer hold the export → realloc raises BufferError
• np.ndarray(buffer=) needs only refcheck

[antonwolfy]

It matches both the shared==1 and shared==2 messages, so it pins neither branch.

[antonwolfy]

It might be helpful to add 3.14+ specific test:

@pytest.mark.skipif(sys.version_info < (3, 14), reason="IsUniquelyReferenced path")
def test_uniquely_referenced_resizes_on_314():
    mem = mkl.MKLMemory(1024)      # bare local, sole owner
    mem.realloc(2048)             # must succeed via LOAD_FAST_BORROW/IsUniquelyReferenced
    assert mem.nbytes == 2048

Python 3.14 introduced LOAD_FAST_BORROW: when the interpreter calls a method on a local variable (mem.realloc(...)), it can push mem onto the evaluation stack as a borrowed reference (no incref), because the local slot provably keeps the object alive for the call's duration. So during the call the refcount stays 1, IsUniquelyReferenced is true, MayBeShared returns 0, and the resize is allowed.

[antonwolfy]

tobytes/_pointer/__repr__/__len__ are not export-gated.
On free-threaded builds they can tear/UAF under a concurrent realloc.

Another thread's realloc can mkl_realloc (free/move the block) while tobytes is mid-copy → use-after-free; or tobytes reads the old _memory_ptr together with the new, larger _nbytes → over-read of freed memory.

While the copy constructor bothers to claim for the identical kind of read, it might be treated as consistent with the caller-responsibility model (like documented in realloc).

It makes sense to add similar documentation note as in realloc here at least.

[antonwolfy]

A subclass that declares __slots__ (and no __dict__) stores its attributes in slot descriptors, and such instances have no __dict__ at all. So getattr(self, "__dict__", None) returns None, and those slot-held attributes are silently dropped on pickling — they come back unset after a round trip:

class Tagged(mkl.MKLMemory):
    __slots__ = ("tag",)

m = Tagged(1024)
m.tag = "hello"
m2 = pickle.loads(pickle.dumps(m))
m2.tag        # AttributeError — the slot value was lost

So we probably need to add a one-line note that __slots__-only subclasses don't round-trip. It's mostly a rare use case we mightn't want to cover.