Add table_pattern: table headers + cell addressing for native grids

WHATS_NEW.md

@@ -1,5 +1,11 @@
 # What's New — AutoControl
 
+## What's new (2026-06-25) — Table Headers + Cell Addressing (UIA TablePattern)
+
+Assert "the Status column of row 5 says Shipped" — by header, not by guessing indices. Full reference: [`docs/source/Eng/doc/new_features/v197_features_doc.rst`](docs/source/Eng/doc/new_features/v197_features_doc.rst).
+
+- **`table_headers` / `table_cell` / `cell_by_header`** (`AC_table_headers`, `AC_table_cell`, `AC_cell_by_header`): `read_control_table` (GridPattern) dumps a flat 2-D list of cell names with no header labels and no way to address one cell by (header, row) — you can dump a grid but not test one. This adds the missing half: `table_headers` reads the row/column header labels (TablePattern), `table_cell` reads the cell at `(row, column)` with its span (GridItemPattern), and `cell_by_header` resolves the column index from the headers so you can read the cell at `(row, "Status")` directly. Dispatched through the injectable accessibility backend seam (headless-testable via a fake backend; real UIA in the Windows backend). No `PySide6`.
+
 ## What's new (2026-06-25) — Rich UIA Element Properties
 
 Know if a control is enabled / off-screen / has a tooltip before you act. Full reference: [`docs/source/Eng/doc/new_features/v196_features_doc.rst`](docs/source/Eng/doc/new_features/v196_features_doc.rst).

docs/source/Eng/doc/new_features/v197_features_doc.rst

@@ -0,0 +1,46 @@
+Table Headers + Cell Addressing (UIA TablePattern)
+==================================================
+
+``read_control_table`` (GridPattern) dumps a flat 2-D list of cell names with **no
+header labels and no way to address a single cell by (header, row)** — so you can
+*dump* a grid but not actually *test* one. ``table_pattern`` adds the missing
+half:
+
+* :func:`table_headers` — the row / column header labels (TablePattern),
+* :func:`table_cell` — the cell at ``(row, column)`` with its span
+  (GridPattern.GetItem + GridItemPattern),
+* :func:`cell_by_header` — read the cell at ``(row, "Column Header")``, so you can
+  assert "the Status column of row 5 says Shipped" without guessing indices.
+
+Each is a thin dispatch onto the injectable ``accessibility.backends.get_backend()``
+seam — headless-testable by injecting a fake backend; the real UIA calls live in
+the Windows backend. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import table_headers, table_cell, cell_by_header
+
+    table_headers(name="Orders")
+    # {"columns": ["Order", "Status", "Total"], "rows": [...]}
+
+    table_cell(0, 1, name="Orders")
+    # {"value": "Shipped", "row": 0, "column": 1, "row_span": 1, "column_span": 1}
+
+    cell_by_header(0, "Status", name="Orders")          # "Shipped"
+
+The table is located by ``name`` / ``role`` / ``app_name`` / ``automation_id``
+(same as ``read_control_table``). ``table_headers`` returns
+``{columns, rows}`` (or ``None``); ``table_cell`` returns the cell record (or
+``None``); ``cell_by_header`` resolves the column index from the headers and
+returns the cell value (``None`` if the header or cell isn't found).
+
+Executor commands
+-----------------
+
+``AC_table_headers`` (``{found, headers}``), ``AC_table_cell`` (``row`` /
+``column`` → ``{found, cell}``) and ``AC_cell_by_header`` (``row`` /
+``column_header`` → ``{found, value}``). They are exposed as read-only ``ac_*``
+MCP tools and as Script Builder commands under **Native UI**.

docs/source/Zh/doc/new_features/v197_features_doc.rst

@@ -0,0 +1,42 @@
+表頭 + 儲存格定址(UIA TablePattern)
+====================================
+
+``read_control_table``(GridPattern)輸出的是一份扁平的 2D 儲存格名稱清單,**沒有表頭標籤,也無法
+以(表頭, 列)定址單一儲存格**——所以你能*傾印*一個格線,卻無法真正*測試*它。``table_pattern``
+補上所缺的另一半:
+
+* :func:`table_headers` ——列 / 欄的表頭標籤(TablePattern),
+* :func:`table_cell` ——位於 ``(row, column)`` 的儲存格及其跨距
+  (GridPattern.GetItem + GridItemPattern),
+* :func:`cell_by_header` ——讀取位於 ``(row, "欄表頭")`` 的儲存格,於是你可以斷言「第 5 列的
+  Status 欄是 Shipped」,而不必猜索引。
+
+每個都是對可注入的 ``accessibility.backends.get_backend()`` 接縫的薄分派——可透過注入 fake backend
+進行無頭測試;真正的 UIA 呼叫位於 Windows 後端。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import table_headers, table_cell, cell_by_header
+
+    table_headers(name="Orders")
+    # {"columns": ["Order", "Status", "Total"], "rows": [...]}
+
+    table_cell(0, 1, name="Orders")
+    # {"value": "Shipped", "row": 0, "column": 1, "row_span": 1, "column_span": 1}
+
+    cell_by_header(0, "Status", name="Orders")          # "Shipped"
+
+表格以 ``name`` / ``role`` / ``app_name`` / ``automation_id`` 定位(與 ``read_control_table``
+相同)。``table_headers`` 回傳 ``{columns, rows}``(或 ``None``);``table_cell`` 回傳儲存格記錄
+(或 ``None``);``cell_by_header`` 由表頭解析欄索引並回傳儲存格值(找不到表頭或儲存格則為
+``None``)。
+
+執行器指令
+----------
+
+``AC_table_headers``(``{found, headers}``)、``AC_table_cell``(``row`` / ``column`` →
+``{found, cell}``)與 ``AC_cell_by_header``(``row`` / ``column_header`` → ``{found, value}``)。
+皆以唯讀 ``ac_*`` MCP 工具及 Script Builder 指令(位於 **Native UI** 分類下)形式提供。

je_auto_control/__init__.py

@@ -72,6 +72,10 @@
 from je_auto_control.utils.ax_props import (
     get_element_properties, is_element_enabled,
 )
+# Table headers + cell addressing for native grids (UIA TablePattern)
+from je_auto_control.utils.table_pattern import (
+    cell_by_header, table_cell, table_headers,
+)
 # Rich clipboard formats — RTF + CSV/TSV codecs and Windows get / set
 from je_auto_control.utils.clipboard_rich_formats import (
     build_rtf, csv_to_rows, get_clipboard_csv, get_clipboard_rtf, rows_to_csv,
@@ -1674,6 +1678,7 @@ def start_autocontrol_gui(*args, **kwargs):
     "is_interactive_role", "tab_order", "audit_focus_order", "focus_control",
     "realize_item",
     "get_element_properties", "is_element_enabled",
+    "table_headers", "table_cell", "cell_by_header",
     "build_rtf", "rtf_to_text", "rows_to_csv", "csv_to_rows",
     "set_clipboard_rtf", "get_clipboard_rtf",
     "set_clipboard_csv", "get_clipboard_csv",

je_auto_control/gui/script_builder/command_schema.py

@@ -1626,6 +1626,23 @@ def _add_native_control_specs(specs: List[CommandSpec]) -> None:
         fields=fields,
         description="Read rich UIA props (enabled/offscreen/help/status/keys).",
     ))
+    specs.append(CommandSpec(
+        "AC_table_headers", "Native UI", "Get Table Headers",
+        fields=fields,
+        description="Read a table's row/column header labels (TablePattern).",
+    ))
+    specs.append(CommandSpec(
+        "AC_table_cell", "Native UI", "Get Table Cell (by index)",
+        fields=(FieldSpec("row", FieldType.INT),
+                FieldSpec("column", FieldType.INT)) + fields,
+        description="Read the cell at (row, column) with its span.",
+    ))
+    specs.append(CommandSpec(
+        "AC_cell_by_header", "Native UI", "Get Table Cell (by header)",
+        fields=(FieldSpec("row", FieldType.INT),
+                FieldSpec("column_header", FieldType.STRING)) + fields,
+        description="Read the cell at (row, named column) — assert by header.",
+    ))
     specs.append(CommandSpec(
         "AC_get_control_text", "Native UI", "Get Control Text",
         fields=fields,

je_auto_control/utils/accessibility/backends/base.py

@@ -162,6 +162,25 @@
         """
         self._unsupported("get_properties")
 
+    # --- table headers + cell addressing (TablePattern / GridItemPattern) ---
+
+    def get_table_headers(self, name: Optional[str] = None,
+                          role: Optional[str] = None,
+                          app_name: Optional[str] = None,
+                          automation_id: Optional[str] = None,
+                          ) -> Optional[Dict[str, Any]]:
+        """Return a table's header labels as ``{columns: [...], rows: [...]}``."""
+        self._unsupported("get_table_headers")
+
+    def get_grid_cell(self, row: int = 0, column: int = 0,
+                      name: Optional[str] = None, role: Optional[str] = None,
+                      app_name: Optional[str] = None,
+                      automation_id: Optional[str] = None,
+                      ) -> Optional[Dict[str, Any]]:
+        """Return the cell at ``(row, column)`` as ``{value, row, column,
+        row_span, column_span}`` (GridPattern.GetItem + GridItemPattern)."""
+        self._unsupported("get_grid_cell")
+
     def _unsupported(self, operation: str):
         """Raise a clear error for an action this backend can't perform."""
         raise AccessibilityNotAvailableError(

je_auto_control/utils/accessibility/backends/windows_backend.py

@@ -32,6 +32,8 @@
 _UIA_TEXT_PATTERN_ID = 10014
 _UIA_ITEMCONTAINER_PATTERN_ID = 10019
 _UIA_VIRTUALIZEDITEM_PATTERN_ID = 10020
+_UIA_TABLE_PATTERN_ID = 10012
+_UIA_GRIDITEM_PATTERN_ID = 10007
 _UIA_AUTOMATIONID_PROPERTY = 30011
 _EXPAND_STATES = {0: "collapsed", 1: "expanded", 2: "partial", 3: "leaf"}
 
@@ -305,6 +307,37 @@ def get_properties(self, name=None, role=None, app_name=None,
             return None
         return _read_properties(raw)
 
+    def get_table_headers(self, name=None, role=None, app_name=None,
+                          automation_id=None) -> Optional[Dict[str, Any]]:
+        raw = self._find_raw(name, role, app_name, automation_id)
+        pattern = self._pattern(raw, _UIA_TABLE_PATTERN_ID,
+                                "IUIAutomationTablePattern") if raw else None
+        if pattern is None:
+            return None
+        try:
+            columns = pattern.GetCurrentColumnHeaders()
+            rows = pattern.GetCurrentRowHeaders()
+        except (OSError, AttributeError):
+            return None
+        return {"columns": _header_names(columns), "rows": _header_names(rows)}
+
+    def get_grid_cell(self, row=0, column=0, name=None, role=None,
+                      app_name=None, automation_id=None) -> Optional[Dict[str, Any]]:
+        raw = self._find_raw(name, role, app_name, automation_id)
+        grid = self._pattern(raw, _UIA_GRID_PATTERN_ID,
+                             "IUIAutomationGridPattern") if raw else None
+        if grid is None:
+            return None
+        try:
+            cell = grid.GetItem(int(row), int(column))
+        except (OSError, AttributeError):
+            return None
+        if not cell:
+            return None
+        return _read_cell(self._pattern(cell, _UIA_GRIDITEM_PATTERN_ID,
+                                        "IUIAutomationGridItemPattern"),
+                          cell, int(row), int(column))
+
     def _text_pattern(self, name, role, app_name, automation_id):
         """Find a control and return its IUIAutomationTextPattern, or None."""
         raw = self._find_raw(name, role, app_name, automation_id)
@@ -373,6 +406,45 @@ def _read_row(pattern, row: int, cols: int):
         return cells
 
 
+def _header_names(array) -> List[str]:
+    """Read an IUIAutomationElementArray of header elements into name strings."""
+    names: List[str] = []
+    try:
+        count = int(array.Length or 0)
+    except (OSError, AttributeError):
+        return names
+    for index in range(count):
+        try:
+            names.append(str(array.GetElement(index).CurrentName or ""))
+        except (OSError, AttributeError):
+            names.append("")
+    return names
+
+
+def _read_cell(item_pattern, cell, row: int, column: int) -> Dict[str, Any]:
+    """Build a cell record, enriching with GridItemPattern row/col/span if present."""
+    info: Dict[str, Any] = {
+        "value": _safe_name(cell), "row": row, "column": column,
+        "row_span": 1, "column_span": 1,
+    }
+    if item_pattern is not None:
+        for key, attr in (("row", "CurrentRow"), ("column", "CurrentColumn"),
+                          ("row_span", "CurrentRowSpan"),
+                          ("column_span", "CurrentColumnSpan")):
+            try:
+                info[key] = int(getattr(item_pattern, attr))
+            except (OSError, AttributeError, ValueError, TypeError):
+                pass
+    return info
+
+
+def _safe_name(raw) -> str:
+    try:
+        return str(raw.CurrentName or "")
+    except (OSError, AttributeError):
+        return ""
+
+
 def _as_text(value) -> str:
     return str(value or "")
 

je_auto_control/utils/executor/action_executor.py

@@ -2486,6 +2486,36 @@ def _get_element_properties(name: Optional[str] = None, role: Optional[str] = No
     return {"found": props is not None, "properties": props}
 
 
+def _table_headers(name: Optional[str] = None, role: Optional[str] = None,
+                   app_name: Optional[str] = None,
+                   automation_id: Optional[str] = None) -> Dict[str, Any]:
+    """Adapter: a table's row/column header labels (TablePattern)."""
+    from je_auto_control.utils.table_pattern import table_headers
+    headers = table_headers(name=name, role=role, app_name=app_name,
+                            automation_id=automation_id)
+    return {"found": headers is not None, "headers": headers}
+
+
+def _table_cell(row: Any, column: Any, name: Optional[str] = None,
+                role: Optional[str] = None, app_name: Optional[str] = None,
+                automation_id: Optional[str] = None) -> Dict[str, Any]:
+    """Adapter: the cell at (row, column) with its span (GridItemPattern)."""
+    from je_auto_control.utils.table_pattern import table_cell
+    cell = table_cell(int(row), int(column), name=name, role=role,
+                      app_name=app_name, automation_id=automation_id)
+    return {"found": cell is not None, "cell": cell}
+
+
+def _cell_by_header(row: Any, column_header: str, name: Optional[str] = None,
+                    role: Optional[str] = None, app_name: Optional[str] = None,
+                    automation_id: Optional[str] = None) -> Dict[str, Any]:
+    """Adapter: read the cell at (row, named column) — assert by header."""
+    from je_auto_control.utils.table_pattern import cell_by_header
+    value = cell_by_header(int(row), str(column_header), name=name, role=role,
+                           app_name=app_name, automation_id=automation_id)
+    return {"found": value is not None, "value": value}
+
+
 def _get_control_text(name: Optional[str] = None, role: Optional[str] = None,
                       app_name: Optional[str] = None,
                       automation_id: Optional[str] = None) -> Dict[str, Any]:
@@ -6431,6 +6461,9 @@ def __init__(self):
             "AC_scroll_control_into_view": _scroll_control_into_view,
             "AC_realize_item": _realize_item,
             "AC_get_element_properties": _get_element_properties,
+            "AC_table_headers": _table_headers,
+            "AC_table_cell": _table_cell,
+            "AC_cell_by_header": _cell_by_header,
             "AC_get_control_text": _get_control_text,
             "AC_get_selected_text": _get_selected_text,
             "AC_get_visible_text": _get_visible_text,

je_auto_control/utils/mcp_server/tools/_factories.py

@@ -1188,6 +1188,37 @@ def a11y_control_tools() -> List[MCPTool]:
             handler=h.get_element_properties,
             annotations=READ_ONLY,
         ),
+        MCPTool(
+            name="ac_table_headers",
+            description=("Read a native table's header labels via TablePattern: "
+                         "{found, headers:{columns:[...], rows:[...]}}."),
+            input_schema=schema(dict(_M)),
+            handler=h.table_headers,
+            annotations=READ_ONLY,
+        ),
+        MCPTool(
+            name="ac_table_cell",
+            description=("Read the cell at ('row','column') of a native grid: "
+                         "{found, cell:{value,row,column,row_span,column_span}} "
+                         "(GridPattern.GetItem + GridItemPattern)."),
+            input_schema=schema({"row": {"type": "integer"},
+                                 "column": {"type": "integer"}, **_M},
+                                required=["row", "column"]),
+            handler=h.table_cell,
+            annotations=READ_ONLY,
+        ),
+        MCPTool(
+            name="ac_cell_by_header",
+            description=("Read the cell at row 'row' in the column named "
+                         "'column_header' — assert 'the Status column of row 5 "
+                         "says Shipped' without guessing indices. Returns "
+                         "{found, value}."),
+            input_schema=schema({"row": {"type": "integer"},
+                                 "column_header": {"type": "string"}, **_M},
+                                required=["row", "column_header"]),
+            handler=h.cell_by_header,
+            annotations=READ_ONLY,
+        ),
         MCPTool(
             name="ac_get_control_text",
             description=("Read a control's full text via TextPattern: "

je_auto_control/utils/mcp_server/tools/_handlers.py

@@ -809,6 +809,24 @@ def get_element_properties(name=None, role=None, app_name=None,
     return _get_element_properties(name, role, app_name, automation_id)
 
 
+def table_headers(name=None, role=None, app_name=None, automation_id=None):
+    from je_auto_control.utils.executor.action_executor import _table_headers
+    return _table_headers(name, role, app_name, automation_id)
+
+
+def table_cell(row, column, name=None, role=None, app_name=None,
+               automation_id=None):
+    from je_auto_control.utils.executor.action_executor import _table_cell
+    return _table_cell(row, column, name, role, app_name, automation_id)
+
+
+def cell_by_header(row, column_header, name=None, role=None, app_name=None,
+                   automation_id=None):
+    from je_auto_control.utils.executor.action_executor import _cell_by_header
+    return _cell_by_header(row, column_header, name, role, app_name,
+                           automation_id)
+
+
 def get_selected_text(name=None, role=None, app_name=None, automation_id=None):
     from je_auto_control.utils.executor.action_executor import _get_selected_text
     return _get_selected_text(name, role, app_name, automation_id)

je_auto_control/utils/table_pattern/__init__.py

@@ -0,0 +1,6 @@
+"""Table headers + cell addressing for native grids (UIA TablePattern / GridItem)."""
+from je_auto_control.utils.table_pattern.table_pattern import (
+    cell_by_header, table_cell, table_headers,
+)
+
+__all__ = ["table_headers", "table_cell", "cell_by_header"]