Release: native-UIA depth pt.1 — virtualized items + rich properties + table addressing (v195–v197)

WHATS_NEW.md

@@ -1,5 +1,23 @@
 # What's New — AutoControl
 
+## What's new (2026-06-25) — Table Headers + Cell Addressing (UIA TablePattern)
+
+Assert "the Status column of row 5 says Shipped" — by header, not by guessing indices. Full reference: [`docs/source/Eng/doc/new_features/v197_features_doc.rst`](docs/source/Eng/doc/new_features/v197_features_doc.rst).
+
+- **`table_headers` / `table_cell` / `cell_by_header`** (`AC_table_headers`, `AC_table_cell`, `AC_cell_by_header`): `read_control_table` (GridPattern) dumps a flat 2-D list of cell names with no header labels and no way to address one cell by (header, row) — you can dump a grid but not test one. This adds the missing half: `table_headers` reads the row/column header labels (TablePattern), `table_cell` reads the cell at `(row, column)` with its span (GridItemPattern), and `cell_by_header` resolves the column index from the headers so you can read the cell at `(row, "Status")` directly. Dispatched through the injectable accessibility backend seam (headless-testable via a fake backend; real UIA in the Windows backend). No `PySide6`.
+
+## What's new (2026-06-25) — Rich UIA Element Properties
+
+Know if a control is enabled / off-screen / has a tooltip before you act. Full reference: [`docs/source/Eng/doc/new_features/v196_features_doc.rst`](docs/source/Eng/doc/new_features/v196_features_doc.rst).
+
+- **`get_element_properties` / `is_element_enabled`** (`AC_get_element_properties`): the flat element list carries only name/role/bounds/app/id, but automation needs more before it acts — **is the control enabled** (don't click a disabled button), **is it off-screen**, its **item_status** (field validation/error), **help_text** (tooltip), and **accelerator_key** (drive via hotkey). This reads those high-value UIA properties (`enabled`/`offscreen`/`help_text`/`item_status`/`accelerator_key`/`access_key`/`orientation`); `is_element_enabled` is the common pre-action guard. Dispatched through the injectable accessibility backend seam (headless-testable via a fake backend; real UIA reads in the Windows backend). No `PySide6`.
+
+## What's new (2026-06-25) — Realize Off-Screen Items in Virtualized Lists / Grids
+
+Reach a row that isn't scrolled into view yet — the "element not found in a long list" fix. Full reference: [`docs/source/Eng/doc/new_features/v195_features_doc.rst`](docs/source/Eng/doc/new_features/v195_features_doc.rst).
+
+- **`realize_item`** (`AC_realize_item`): long lists / data grids / trees only materialize visible rows, so an off-screen row has no accessibility element at all — `list_accessibility_elements` / `read_control_table` / `select_control_item` can't see it, and `scroll_control_into_view` can't help because the element doesn't exist yet. This locates the item by property (UIA `ItemContainerPattern.FindItemByProperty`) and realizes it (`VirtualizedItemPattern.Realize`) so it becomes a real, clickable element. Match `by` name (default) or `automation_id`; locate the container by name/role/app. Dispatched through the injectable accessibility backend seam (headless-testable via a fake backend; real UIA in the Windows backend). No `PySide6`.
+
 ## What's new (2026-06-25) — Per-Run Step Timeline (waterfall + bottleneck steps)
 
 Read why *this* run was slow — a step waterfall and its bottlenecks. Full reference: [`docs/source/Eng/doc/new_features/v194_features_doc.rst`](docs/source/Eng/doc/new_features/v194_features_doc.rst).

docs/source/Eng/doc/new_features/v195_features_doc.rst

@@ -0,0 +1,48 @@
+Realize Off-Screen Items in Virtualized Lists / Grids
+=====================================================
+
+Long lists, data grids and trees (WPF / WinUI / File Explorer / virtual
+treeviews) only materialize the rows that are scrolled into view — a row that is
+off-screen has **no** accessibility element at all. So
+``list_accessibility_elements`` / ``read_control_table`` / ``select_control_item``
+simply cannot see it, and ``scroll_control_into_view`` can't help because the
+target element does not exist yet. This is the classic "element not found in a
+long list" wall.
+
+``realize_item`` closes that gap: it locates the item inside its container by
+property (UI Automation ``ItemContainerPattern.FindItemByProperty``) and realizes
+it (``VirtualizedItemPattern.Realize``) so it materializes as a real element you
+can then click or read.
+
+It is a thin dispatch onto the injectable ``accessibility.backends.get_backend()``
+seam (the same seam the rest of the accessibility module uses) — headless-testable
+on any platform by injecting a fake backend; the real UIA calls live in the
+Windows backend. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import realize_item, click_accessibility_element
+
+    # Bring a far-down row into existence, then act on it:
+    row = realize_item("Order 5000", container_name="Orders")
+    if row is not None:
+        click_accessibility_element(name=row.name)   # now a real element
+
+    realize_item("row-42", by="automation_id", container_name="DataGrid")
+
+``item_name`` is matched against the item's Name (``by="name"``, default) or its
+AutomationId (``by="automation_id"``). The container is located by
+``container_name`` / ``container_role`` / ``app_name`` / ``automation_id`` (the
+same matchers as the other native-control actions). Returns the realized
+``AccessibilityElement``, or ``None`` if the container or item isn't found.
+
+Executor commands
+-----------------
+
+``AC_realize_item`` (``item_name`` / ``by`` / ``container_name`` /
+``container_role`` / ``app_name`` / ``automation_id``) returns
+``{found, element}``. It is exposed as the read-only ``ac_realize_item`` MCP tool
+and as a Script Builder command under **Native UI**.

docs/source/Eng/doc/new_features/v196_features_doc.rst

@@ -0,0 +1,44 @@
+Rich UIA Element Properties
+===========================
+
+``list_accessibility_elements`` / ``AccessibilityElement`` carry only name / role /
+bounds / app / pid / automation_id. Automation routinely needs more *before it
+acts*: **is the control enabled** (don't click a disabled button), **is it
+off-screen** (is it really visible?), its **item_status** (validation / error text
+on a field), **help_text** (tooltip), and **accelerator_key** (drive it via a
+hotkey instead of a click). ``ax_props`` exposes those high-value UIA properties.
+
+* :func:`get_element_properties` — the full property dict,
+* :func:`is_element_enabled` — the common pre-action guard.
+
+Each function is a thin dispatch onto the injectable
+``accessibility.backends.get_backend()`` seam — headless-testable on any platform
+by injecting a fake backend; the real UIA property reads live in the Windows
+backend. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import get_element_properties, is_element_enabled
+
+    get_element_properties(name="Save", role="button")
+    # {"enabled": False, "offscreen": False, "help_text": "Save the file",
+    #  "item_status": "", "accelerator_key": "Ctrl+S", "access_key": "S",
+    #  "orientation": 0}
+
+    if is_element_enabled(name="Submit"):
+        click_text("Submit")          # don't click a disabled button
+
+The control is located by ``name`` / ``role`` / ``app_name`` / ``automation_id``
+(same as the other native-control reads). ``get_element_properties`` returns the
+property dict or ``None`` when the control isn't found; ``is_element_enabled``
+returns the ``enabled`` flag (or ``None`` if not found).
+
+Executor commands
+-----------------
+
+``AC_get_element_properties`` returns ``{found, properties}``. It is exposed as the
+read-only ``ac_get_element_properties`` MCP tool and as a Script Builder command
+under **Native UI**.

docs/source/Eng/doc/new_features/v197_features_doc.rst

@@ -0,0 +1,46 @@
+Table Headers + Cell Addressing (UIA TablePattern)
+==================================================
+
+``read_control_table`` (GridPattern) dumps a flat 2-D list of cell names with **no
+header labels and no way to address a single cell by (header, row)** — so you can
+*dump* a grid but not actually *test* one. ``table_pattern`` adds the missing
+half:
+
+* :func:`table_headers` — the row / column header labels (TablePattern),
+* :func:`table_cell` — the cell at ``(row, column)`` with its span
+  (GridPattern.GetItem + GridItemPattern),
+* :func:`cell_by_header` — read the cell at ``(row, "Column Header")``, so you can
+  assert "the Status column of row 5 says Shipped" without guessing indices.
+
+Each is a thin dispatch onto the injectable ``accessibility.backends.get_backend()``
+seam — headless-testable by injecting a fake backend; the real UIA calls live in
+the Windows backend. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import table_headers, table_cell, cell_by_header
+
+    table_headers(name="Orders")
+    # {"columns": ["Order", "Status", "Total"], "rows": [...]}
+
+    table_cell(0, 1, name="Orders")
+    # {"value": "Shipped", "row": 0, "column": 1, "row_span": 1, "column_span": 1}
+
+    cell_by_header(0, "Status", name="Orders")          # "Shipped"
+
+The table is located by ``name`` / ``role`` / ``app_name`` / ``automation_id``
+(same as ``read_control_table``). ``table_headers`` returns
+``{columns, rows}`` (or ``None``); ``table_cell`` returns the cell record (or
+``None``); ``cell_by_header`` resolves the column index from the headers and
+returns the cell value (``None`` if the header or cell isn't found).
+
+Executor commands
+-----------------
+
+``AC_table_headers`` (``{found, headers}``), ``AC_table_cell`` (``row`` /
+``column`` → ``{found, cell}``) and ``AC_cell_by_header`` (``row`` /
+``column_header`` → ``{found, value}``). They are exposed as read-only ``ac_*``
+MCP tools and as Script Builder commands under **Native UI**.

docs/source/Zh/doc/new_features/v195_features_doc.rst

@@ -0,0 +1,41 @@
+實體化虛擬化清單 / 格線中的離畫面項目
+======================================
+
+長清單、資料格線與樹(WPF / WinUI / 檔案總管 / 虛擬化 treeview)只會實體化已捲入視野的列——
+離畫面的列**完全沒有**無障礙元素。因此 ``list_accessibility_elements`` /
+``read_control_table`` / ``select_control_item`` 根本看不到它,而 ``scroll_control_into_view``
+也幫不上忙,因為目標元素根本還不存在。這就是經典的「長清單裡找不到元素」的牆。
+
+``realize_item`` 補上這個缺口:它以屬性在容器內定位該項目(UI Automation
+``ItemContainerPattern.FindItemByProperty``)並將其實體化(``VirtualizedItemPattern.Realize``),
+使其成為一個真正、可點擊或可讀取的元素。
+
+它是對可注入的 ``accessibility.backends.get_backend()`` 接縫的薄分派(與無障礙模組其餘部分相同的
+接縫)——可在任何平台透過注入 fake backend 進行無頭測試;真正的 UIA 呼叫位於 Windows 後端。
+不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import realize_item, click_accessibility_element
+
+    # 讓一個很下方的列「存在」,然後對它操作:
+    row = realize_item("Order 5000", container_name="Orders")
+    if row is not None:
+        click_accessibility_element(name=row.name)   # 現在是真正的元素
+
+    realize_item("row-42", by="automation_id", container_name="DataGrid")
+
+``item_name`` 會比對項目的 Name(``by="name"``,預設)或其 AutomationId
+(``by="automation_id"``)。容器以 ``container_name`` / ``container_role`` / ``app_name`` /
+``automation_id`` 定位(與其他原生控制動作相同的比對方式)。回傳實體化後的
+``AccessibilityElement``,若找不到容器或項目則回傳 ``None``。
+
+執行器指令
+----------
+
+``AC_realize_item``(``item_name`` / ``by`` / ``container_name`` / ``container_role`` /
+``app_name`` / ``automation_id``)回傳 ``{found, element}``。以唯讀 ``ac_realize_item`` MCP
+工具及 Script Builder 指令(位於 **Native UI** 分類下)形式提供。

docs/source/Zh/doc/new_features/v196_features_doc.rst

@@ -0,0 +1,40 @@
+豐富的 UIA 元素屬性
+===================
+
+``list_accessibility_elements`` / ``AccessibilityElement``只帶有 name / role / bounds /
+app / pid / automation_id。自動化在*動作之前*常需要更多資訊:**控制項是否啟用**(別點停用的
+按鈕)、**是否在畫面外**(是否真的可見)、其 **item_status**(欄位的驗證 / 錯誤文字)、
+**help_text**(工具提示),以及 **accelerator_key**(以快捷鍵而非點擊來驅動它)。``ax_props``
+就提供這些高價值的 UIA 屬性。
+
+* :func:`get_element_properties` ——完整的屬性字典,
+* :func:`is_element_enabled` ——常見的動作前守衛。
+
+每個函式都是對可注入的 ``accessibility.backends.get_backend()`` 接縫的薄分派——可在任何平台透過
+注入 fake backend 進行無頭測試;真正的 UIA 屬性讀取位於 Windows 後端。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import get_element_properties, is_element_enabled
+
+    get_element_properties(name="Save", role="button")
+    # {"enabled": False, "offscreen": False, "help_text": "Save the file",
+    #  "item_status": "", "accelerator_key": "Ctrl+S", "access_key": "S",
+    #  "orientation": 0}
+
+    if is_element_enabled(name="Submit"):
+        click_text("Submit")          # 別點停用的按鈕
+
+控制項以 ``name`` / ``role`` / ``app_name`` / ``automation_id`` 定位(與其他原生控制讀取相同)。
+``get_element_properties`` 回傳屬性字典,找不到控制項時回傳 ``None``;``is_element_enabled``
+回傳 ``enabled`` 旗標(找不到則為 ``None``)。
+
+執行器指令
+----------
+
+``AC_get_element_properties`` 回傳 ``{found, properties}``。以唯讀
+``ac_get_element_properties`` MCP 工具及 Script Builder 指令(位於 **Native UI** 分類下)
+形式提供。

docs/source/Zh/doc/new_features/v197_features_doc.rst

@@ -0,0 +1,42 @@
+表頭 + 儲存格定址(UIA TablePattern)
+====================================
+
+``read_control_table``(GridPattern)輸出的是一份扁平的 2D 儲存格名稱清單,**沒有表頭標籤,也無法
+以(表頭, 列)定址單一儲存格**——所以你能*傾印*一個格線,卻無法真正*測試*它。``table_pattern``
+補上所缺的另一半:
+
+* :func:`table_headers` ——列 / 欄的表頭標籤(TablePattern),
+* :func:`table_cell` ——位於 ``(row, column)`` 的儲存格及其跨距
+  (GridPattern.GetItem + GridItemPattern),
+* :func:`cell_by_header` ——讀取位於 ``(row, "欄表頭")`` 的儲存格,於是你可以斷言「第 5 列的
+  Status 欄是 Shipped」,而不必猜索引。
+
+每個都是對可注入的 ``accessibility.backends.get_backend()`` 接縫的薄分派——可透過注入 fake backend
+進行無頭測試;真正的 UIA 呼叫位於 Windows 後端。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import table_headers, table_cell, cell_by_header
+
+    table_headers(name="Orders")
+    # {"columns": ["Order", "Status", "Total"], "rows": [...]}
+
+    table_cell(0, 1, name="Orders")
+    # {"value": "Shipped", "row": 0, "column": 1, "row_span": 1, "column_span": 1}
+
+    cell_by_header(0, "Status", name="Orders")          # "Shipped"
+
+表格以 ``name`` / ``role`` / ``app_name`` / ``automation_id`` 定位(與 ``read_control_table``
+相同)。``table_headers`` 回傳 ``{columns, rows}``(或 ``None``);``table_cell`` 回傳儲存格記錄
+(或 ``None``);``cell_by_header`` 由表頭解析欄索引並回傳儲存格值(找不到表頭或儲存格則為
+``None``)。
+
+執行器指令
+----------
+
+``AC_table_headers``(``{found, headers}``)、``AC_table_cell``(``row`` / ``column`` →
+``{found, cell}``)與 ``AC_cell_by_header``(``row`` / ``column_header`` → ``{found, value}``)。
+皆以唯讀 ``ac_*`` MCP 工具及 Script Builder 指令(位於 **Native UI** 分類下)形式提供。

je_auto_control/__init__.py

@@ -66,6 +66,16 @@
 from je_auto_control.utils.focus_order import (
     audit_focus_order, focus_control, is_interactive_role, tab_order,
 )
+# Realize off-screen items in virtualized lists / grids (UIA VirtualizedItem)
+from je_auto_control.utils.virtualized import realize_item
+# Rich UIA element properties (enabled / offscreen / help / status / keys)
+from je_auto_control.utils.ax_props import (
+    get_element_properties, is_element_enabled,
+)
+# Table headers + cell addressing for native grids (UIA TablePattern)
+from je_auto_control.utils.table_pattern import (
+    cell_by_header, table_cell, table_headers,
+)
 # Rich clipboard formats — RTF + CSV/TSV codecs and Windows get / set
 from je_auto_control.utils.clipboard_rich_formats import (
     build_rtf, csv_to_rows, get_clipboard_csv, get_clipboard_rtf, rows_to_csv,
@@ -1666,6 +1676,9 @@ def start_autocontrol_gui(*args, **kwargs):
     "control_type_name", "humanize_role", "humanize_tree",
     "assign_node_paths", "find_by_path",
     "is_interactive_role", "tab_order", "audit_focus_order", "focus_control",
+    "realize_item",
+    "get_element_properties", "is_element_enabled",
+    "table_headers", "table_cell", "cell_by_header",
     "build_rtf", "rtf_to_text", "rows_to_csv", "csv_to_rows",
     "set_clipboard_rtf", "get_clipboard_rtf",
     "set_clipboard_csv", "get_clipboard_csv",

je_auto_control/gui/script_builder/command_schema.py

@@ -1608,6 +1608,41 @@ def _add_native_control_specs(specs: List[CommandSpec]) -> None:
         fields=fields,
         description="Scroll a control into view (ScrollItemPattern).",
     ))
+    specs.append(CommandSpec(
+        "AC_realize_item", "Native UI", "Realize Virtualized Item",
+        fields=(
+            FieldSpec("item_name", FieldType.STRING),
+            FieldSpec("by", FieldType.ENUM, optional=True, default="name",
+                      choices=("name", "automation_id")),
+            FieldSpec("container_name", FieldType.STRING, optional=True),
+            FieldSpec("container_role", FieldType.STRING, optional=True),
+            FieldSpec("app_name", FieldType.STRING, optional=True),
+            FieldSpec("automation_id", FieldType.STRING, optional=True),
+        ),
+        description="Realize an off-screen item in a virtualized list/grid.",
+    ))
+    specs.append(CommandSpec(
+        "AC_get_element_properties", "Native UI", "Get Element Properties",
+        fields=fields,
+        description="Read rich UIA props (enabled/offscreen/help/status/keys).",
+    ))
+    specs.append(CommandSpec(
+        "AC_table_headers", "Native UI", "Get Table Headers",
+        fields=fields,
+        description="Read a table's row/column header labels (TablePattern).",
+    ))
+    specs.append(CommandSpec(
+        "AC_table_cell", "Native UI", "Get Table Cell (by index)",
+        fields=(FieldSpec("row", FieldType.INT),
+                FieldSpec("column", FieldType.INT)) + fields,
+        description="Read the cell at (row, column) with its span.",
+    ))
+    specs.append(CommandSpec(
+        "AC_cell_by_header", "Native UI", "Get Table Cell (by header)",
+        fields=(FieldSpec("row", FieldType.INT),
+                FieldSpec("column_header", FieldType.STRING)) + fields,
+        description="Read the cell at (row, named column) — assert by header.",
+    ))
     specs.append(CommandSpec(
         "AC_get_control_text", "Native UI", "Get Control Text",
         fields=fields,