Merge branch 'main' into checker-specific-ignores

Author: adamtheturtle

CHANGELOG.rst

@@ -3,6 +3,24 @@ Changelog
 
 .. towncrier release notes start
 
+2026.09.08.1
+------------
+
+- The URL of a reco counts report now has the shape of the presigned cloud storage URL which real Vuforia returns.
+  The report file is named ``{date}-{hour}.csv`` for the current month and ``{month}.csv`` for the previous month, so two requests for the same month in the same hour name the same file and serve the report which the first request generated.
+  The URL carries the query parameters of a presigned URL, and the mock honors ``X-Amz-Date`` and ``X-Amz-Expires`` as real Vuforia's storage does, so a URL which is out of date or which has no query parameters gives the ``403`` XML error document which Amazon S3 gives.
+  A report which is not yet generated gives the ``404`` ``NoSuchKey`` XML error document which Amazon S3 gives, as real Vuforia has now been observed to do, rather than an empty body.
+
+2026.09.08
+----------
+
+- The response delay and client timeout simulation is now provided by the ``mock-response-delay`` package.
+
+- The mock now returns NGINX's ``400 Request Header Or Cookie Too Large`` response for any request header line longer than 8190 bytes, as real Vuforia does.
+
+- Match real Vuforia's request rate limiting, which was checked against it on 2026-09-08.
+  ``DOCUMENTED_REQUEST_RATE_LIMITS`` now allows two ``GET /targets`` requests per minute rather than one, a rate-limited request gets Envoy's empty-bodied ``429`` response rather than a JSON ``TooManyRequests`` body, and the limits are applied before the request's signature is checked.
+
 2026.09.06
 ----------
 

docs/source/differences-to-vws.rst

@@ -184,16 +184,30 @@ endpoints in general, with 45 requests per second for
 ``GET /targets/{target_id}``, 10 requests per second for
 ``GET /duplicates/{target_id}``, and 1 request per minute for ``GET /targets``.
 
-The mock models these limits separately for each group of endpoints, but it applies no limit by default.
-Applying a limit of 1 request per minute to ``GET /targets`` by default would break the tests of anything which uses the mock.
-
-.. admonition:: Unverified assumption
-
-   :ref:`unverified-request-rate-limits`
+The limits were checked against real Vuforia on 2026-09-08, by sending bursts of requests to read-only endpoints:
+
+* ``GET /targets`` accepts two requests per minute, not one.
+  The window is a fixed clock minute: two requests at 40 seconds past the minute were accepted, a third was rejected, and a request three seconds into the next minute was accepted again.
+* The per-second limits are enforced roughly, not exactly.
+  Bursts of 40 concurrent ``GET /summary`` requests saw between 17 and 37 succeed against the documented 15, and a burst of 120 ``GET /targets/{target_id}`` requests saw 74 succeed against the documented 45, so the limiter appears to be spread over more than one instance or window.
+* A limit is keyed on the server access key in the ``Authorization`` header, so one database's burst does not affect another database.
+  Vuforia applies the limit before checking the signature, so a request with a bad signature counts towards the limit, and a request over the limit gets a ``429`` response whether or not it is signed correctly.
+  Requests without an ``Authorization`` header are not rate limited.
+* A rate-limited request gets a ``429`` (``TOO MANY REQUESTS``) response from Envoy with an empty body, no ``Content-Type`` header and an ``x-envoy-ratelimited: true`` header.
+  Vuforia has an Envoy layer at its edge and another in front of the application, and either may reject the request.
+  Only a rejection by the inner layer carries an ``x-envoy-upstream-service-time`` header, which the mock always includes.
+  The ``TooManyRequests`` result code from Vuforia's result codes table does not appear.
+
+The mock returns the empty Envoy response, applies each limit before checking the request's signature, and tracks each limit separately for each database and each group of endpoints.
+The mock's windows are rolling rather than clock-aligned, so two ``GET /targets`` requests block a third until a minute has passed since the first, and the mock enforces the per-second limits exactly.
+The mock only limits requests whose access key belongs to a database, because the limits are configured on the database.
+
+The mock applies no limit by default.
+Applying a limit of two requests per minute to ``GET /targets`` by default would break the tests of anything which uses the mock.
 
 Set ``request_rate_limits`` to
 :data:`mock_vws.request_rate_limits.DOCUMENTED_REQUEST_RATE_LIMITS` to apply
-the documented limits::
+the limits which real Vuforia applies::
 
     from mock_vws import MockVWS
     from mock_vws.database import CloudDatabase
@@ -205,8 +219,8 @@ the documented limits::
 
     with MockVWS() as mock:
         mock.add_cloud_database(cloud_database=database)
-        # A second ``GET /targets`` request within a minute returns
-        # ``TooManyRequests``.
+        # A third ``GET /targets`` request within a minute gets a ``429``
+        # response.
         ...
 
 ``requests_per_second_limit`` remains available. It applies one limit to all
@@ -457,35 +471,70 @@ As real Vuforia does, the mock returns a 401 response with the
 server keys but which names any other database, including one named by its
 name rather than by its ID.
 
-Real Vuforia returns a presigned URL for cloud storage.
-The mock returns a URL served by the mock itself, without the query
-parameters of a presigned URL, so the mock's URL never expires where a real
-one expires after just under seven days.
+Real Vuforia returns a presigned URL for cloud storage, of this form:
+
+.. code-block:: text
+
+   https://guacamole-targetstore-production-targets.s3.us-west-1.amazonaws.com/reports/{database_id}/{file_name}.csv
+     ?X-Amz-Security-Token=...
+     &X-Amz-Algorithm=AWS4-HMAC-SHA256
+     &X-Amz-Date=20260808T210052Z
+     &X-Amz-SignedHeaders=host
+     &X-Amz-Credential=.../20260808/us-west-1/s3/aws4_request
+     &X-Amz-Expires=604799
+     &X-Amz-Signature=...
+
+The mock returns a URL with the same path and the same query parameters,
+served by the mock itself rather than by cloud storage.
 The URL returned by the Flask and Docker mock is built from the
 :envvar:`VWS_BASE_URL` environment variable.
-The report takes :paramref:`~mock_vws.MockVWS.processing_time_seconds`
-seconds to generate in the mock.
-The documentation says a real report takes between a few seconds and one
-hour, but a report for a database with no recognitions has been observed
-ready within seconds.
+The credential, the security token and the signature are placeholders of
+the right shape.
+The mock does not check the signature, so a URL whose signature or file name
+has been changed, which real Vuforia refuses with a ``SignatureDoesNotMatch``
+error document, is served by the mock as if it were signed.
 
 Real Vuforia names the report file after the requested month, and does so
 differently for each of the two months it accepts.
-A report for the current month is named for the date and the hour, such as
+A report for the current month is named for the UTC date and hour, such as
 ``2026-08-08-21.csv``, and a report for the previous month is named for the
 month, such as ``2026-07.csv``.
-The mock names every report after an opaque report identifier, so the
-requested month cannot be recovered from the mock's URL, and two requests for
-the same month never give the same URL.
-
-The mock's URL returns a 404 response until the report is ready, and requires
-no authorization.
-The lack of authorization matches real Vuforia, whose URL carries its own
-signature.
-
-.. admonition:: Unverified assumption
-
-   :ref:`unverified-reco-counts-report-not-ready`
+The mock does the same, so two requests for the same month in the same hour
+name the same file, and two requests for the previous month always do.
+Real Vuforia does not generate the report again for such a request: the URL
+which the second request returns serves the file which the first request
+generated, unchanged.
+The mock does the same, so recognition counts set between the two requests
+are not in the report which the second URL serves.
+
+The URL expires ``X-Amz-Expires`` seconds after its ``X-Amz-Date``, which
+is one second under seven days.
+Real Vuforia's storage checks that the URL is in date before it checks the
+signature, so a URL whose ``X-Amz-Date`` or ``X-Amz-Expires`` has been
+edited to put it out of date gives the same 403 response as a URL which has
+expired, even though the edit invalidates the signature.
+The mock honors those two parameters in the same way, so code which handles
+a stale URL can be tested by editing them.
+The 403 response is the XML ``AccessDenied`` error document which Amazon S3
+gives, with a ``Request has expired`` message, the expiry time and the
+server time.
+A URL without those parameters gives the ``AccessDenied`` error document
+with an ``Access Denied`` message, as it does on real Vuforia.
+
+Until the report is ready, the URL gives a 404 response with the XML
+``NoSuchKey`` error document which Amazon S3 gives, naming the file's key.
+The mock does the same, and the mock gives the same response for a file
+which no request generated.
+The mock's error documents carry random request identifiers where Amazon's
+carry its own.
+The report takes :paramref:`~mock_vws.MockVWS.processing_time_seconds`
+seconds to generate in the mock.
+The documentation says a real report takes between a few seconds and one
+hour, but a real report has been observed ready within a second of the
+request, and the 404 response has been observed by fetching the URL straight
+after the request.
+The download requires no authorization beyond the query parameters of the
+URL, as on real Vuforia.
 
 Paths which the mock does not serve
 -----------------------------------

docs/source/unverified-behavior.rst

@@ -38,19 +38,6 @@ The status code and the body shape come from Vuforia's documentation and from th
 A real database with an exhausted request quota would verify this.
 No such response has been seen.
 
-.. _unverified-request-rate-limits:
-
-Request rate limits
--------------------
-
-:Category: never-attempted
-:API: VWS Target API
-
-Vuforia documents a limit of 15 requests per second for VWS endpoints in general, 45 per second for ``GET /targets/{target_id}``, 10 per second for ``GET /duplicates/{target_id}`` and one per minute for ``GET /targets``.
-The mock models the limits separately for each group of endpoints, and applies them only when it is asked to.
-
-Sending more than the documented number of requests to a real database, and seeing what it returns, would verify this.
-
 .. _unverified-project-suspended:
 
 A suspended database
@@ -79,8 +66,9 @@ Additional result codes
 :Category: never-attempted
 :API: VWS Target API
 
-``ProjectHasNoApiAccess``, ``TargetQuotaReached`` and ``TooManyRequests`` come from Vuforia's result codes table.
+``ProjectHasNoApiAccess`` and ``TargetQuotaReached`` come from Vuforia's result codes table.
 No response from a real database in any of those states has been seen, which is why the mock's ``ProjectHasNoApiAccess`` casing is the table's casing rather than an observed one.
+The table also lists ``TooManyRequests``, but a rate-limited request to a real database gets a ``429`` response with no body at all, so the mock never returns that result code.
 
 A database put into each state by the Target Manager portal would verify these.
 
@@ -97,19 +85,6 @@ The mock does not implement this, so a user of the mock sees a successful respon
 
 A database with more than a million images would verify this, which a test account cannot hold.
 
-.. _unverified-reco-counts-report-not-ready:
-
-A reco counts report which is not ready
----------------------------------------
-
-:Category: never-attempted
-:API: Reco Counts Report API
-
-The URL which the mock returns for a reco counts report gives a 404 response until the report is ready.
-
-A request for a real report which caught it before it was generated would verify this.
-Every real report requested so far has been ready by the time it was asked for.
-
 .. _unverified-reco-counts-report-row-order:
 
 The order of reco counts report rows

newsfragments/3415.change

@@ -464,8 +464,8 @@ def create_cloud_database() -> Response:
 
     :reqjson int requests_per_second_limit: (Optional) The maximum number of
       VWS requests accepted in a rolling one-second window, across all VWS
-      endpoints. Set this to zero to make VWS endpoints return
-      ``TooManyRequests``.
+      endpoints. Set this to zero to make VWS endpoints return a ``429``
+      response.
 
     :reqjson request_rate_limits: (Optional) Request rate limits for
       individual groups of VWS endpoints. This is an object with the optional

newsfragments/3571.change

@@ -286,7 +286,7 @@ def __init__(self) -> None:
 
     @property
     def reco_counts_reports(self) -> dict[str, RecoCountsReport]:
-        """All reco counts reports, keyed by report identifier."""
+        """All reco counts reports, keyed by report file path."""
         with self._lock:
             return dict(self._reports)
 
@@ -299,7 +299,7 @@ def add_reco_counts_report(
     ) -> None:
         """Add a reco counts report."""
         with self._lock:
-            self._reports[reco_counts_report.uuid_] = reco_counts_report
+            self._reports[reco_counts_report.key] = reco_counts_report
 
 
 _RECO_COUNTS_REPORT_STORE = _InMemoryRecoCountsReportStore()
@@ -364,7 +364,7 @@ def validate_request() -> None:
     if (
         request.path.startswith("/oauth2/")
         or request.path.startswith("/modeltargets/")
-        or request.path.startswith("/reports/recoCounts/")
+        or request.path.startswith("/reports/")
     ):
         return
     _ = run_services_validators(
@@ -707,20 +707,25 @@ def reco_counts_report(database_id: str) -> Response:
 
 
 @VWS_FLASK_APP.route(
-    rule="/reports/recoCounts/<string:report_id>",
+    rule="/reports/<string:database_id>/<string:file_name>",
     methods=[HTTPMethod.GET],
 )
 @beartype
-def download_reco_counts_report(report_id: str) -> Response:
+def download_reco_counts_report(database_id: str, file_name: str) -> Response:
     """Download a generated reco counts report.
 
     This stands in for the presigned URL which real Vuforia returns, so it
-    does not require any authorization.
+    does not require any authorization beyond the query parameters of that
+    URL.
     """
+    # The report is looked up by the whole path, which names the database
+    # and the file.
+    del database_id
+    del file_name
     return _to_flask_response(
         api_response=download_report(
             report_store=_RECO_COUNTS_REPORT_STORE,
-            report_id=report_id,
+            request_path=request.full_path,
         ),
     )
 

src/mock_vws/_flask_server/target_manager.py

@@ -19,8 +19,14 @@
 RECO_COUNTS_REPORT_PATH_PATTERN = (
     f"/imagetargets/databases/{DATABASE_ID_PATTERN}/reports/recoCounts"
 )
-# The path which stands in for a reco counts report presigned URL.
-RECO_COUNTS_DOWNLOAD_PATH_PATTERN = "/reports/recoCounts/[A-Za-z0-9]+"
+# The path which stands in for a reco counts report presigned URL, with the
+# query string of that URL.
+# Any file name is matched, so that a file which no request generated gives
+# the response which cloud storage gives for a missing object rather than
+# leaving the request unmatched.
+RECO_COUNTS_DOWNLOAD_PATH_PATTERN = (
+    f"/reports/{DATABASE_ID_PATTERN}/[^/?]+(\\?.*)?"
+)
 
 
 @beartype