Add security and privacy questionnaire#195
Conversation
|
|
||
| > 10. Do features in this specification enable new script execution/loading mechanisms? | ||
|
|
||
| No. Tool `execute` callbacks are ordinary JavaScript invoked in the registering document's existing realm. |
There was a problem hiding this comment.
| No. Tool `execute` callbacks are ordinary JavaScript invoked in the registering document's existing realm. | |
| No. Tool [`execute`](https://webmachinelearning.github.io/webmcp/#dom-modelcontexttool-execute) callbacks are ordinary JavaScript invoked in the registering document's existing realm. |
If we keep this answer, I have the suggestion above. However, I think there's a chance we might want to say "Yes" to this question, since we're basically introducing a structured postMessage() v2, which allows a set of authorized origins to basically directly schedule callbacks in the tool provider's realm, which is kind of "new" on the platform. Since this is relevant to the TAG review we'll file, let me just ask @jyasskin directly: what do you think our answer should be for this question?
There was a problem hiding this comment.
FWIW, I'd argue in favor of "yes" here.
There was a problem hiding this comment.
I think me too. Let's change this to "yes" then.
|
|
||
| > 10. Do features in this specification enable new script execution/loading mechanisms? | ||
|
|
||
| No. Tool `execute` callbacks are ordinary JavaScript invoked in the registering document's existing realm. |
There was a problem hiding this comment.
FWIW, I'd argue in favor of "yes" here.
|
|
||
| > 04. How do the features in your specification deal with sensitive information? | ||
|
|
||
| WebMCP is not a source of sensitive information. Tools may wrap sensitive or high-privilege operations (e.g., purchases, account changes), but that risk is not WebMCP-specific. We discuss this risk in [Tool Implementation as Attack Targets](https://webmachinelearning.github.io/webmcp/#tool-implementation-targets). |
There was a problem hiding this comment.
One question I anticipate from privacy review is what mitigations for this issue are possible for WebMCP. I agree with Dom: well said to make clear WebMCP isn't creating the problem. However, if there are any locations to tie the agent's hands a bit and make the privacy story better (even on the level of recommendations rather than requirements) that would be an improvement. Alternatively, you should make clear that nothing is actually being done to defend against that attack in the spec with something like ... but that risk is not WebMCP-specific and is therefore not defended against in the design of the API.
There was a problem hiding this comment.
I think we can reference the https://webmachinelearning.github.io/webmcp/#mitigations section to begin with perhaps? We should also update the mitigations to include this #176, as discussed in the CG call.
Beyond that, there will just be things that will be left as recommendations for agent implementers, and thus non-normative in nature. But there should be areas where the API design can help with, and I don't want to close the doors on future mitigations by stating otherwise
There was a problem hiding this comment.
+1 to Victor here, strong normative protection might be difficult, but we should definitely try to add e.g. hints that help the agent manage risky actions.
There was a problem hiding this comment.
Sounds good, @victorhuangwq do you want to link to the mitigations section in the spec from here then?
There was a problem hiding this comment.
Linking to mitigations is a fine start.
Just to clear up a couple of misconceptions:
Beyond that, there will just be things that will be left as recommendations for agent implementers, and thus non-normative in nature.
Normative language can include non MUST RFC 2119 terms, like SHOULD and MAY.
But there should be areas where the API design can help with, and I don't want to close the doors on future mitigations by stating otherwise
Saying that you don't do anything about it is not closing the door on future mitigations, particularly if you say so.
There was a problem hiding this comment.
IMO those are two fair points from Ben here :)
Co-authored-by: Dominic Farolino <domfarolino@gmail.com>
Co-authored-by: Dominic Farolino <domfarolino@gmail.com>
Co-authored-by: Dominic Farolino <domfarolino@gmail.com>
Co-authored-by: Dominic Farolino <domfarolino@gmail.com>
3b63206 to
4e84b15
Compare
Co-authored-by: Dominic Farolino <domfarolino@gmail.com>
…om/victorhuangwq/webmcp into security-and-privacy-questionnaire
Co-authored-by: Dominic Farolino <domfarolino@gmail.com>
|
|
||
| > 07. Do the features in your specification expose information about the underlying platform to origins? | ||
|
|
||
| No. |
There was a problem hiding this comment.
Well, it could, right? It exposes whatever you want to ask of the agent, if the agent considers it appropriate to expose this information.
So you could write a do_thing(os_name, os_version, cpu_model) tool and, say, a local command line agent could easily supply that information.
WebMCP is a "dumb pipe", in that sense.
There was a problem hiding this comment.
I guess I'd slightly push back and say in that case, it's not our "features" that are exposing any information about the underlying platform. It's the agent that's exposing it. Just like it might in a standard HTML form without WebMCP. Maybe we should clarify that here. How about:
No, WebMCP does not expose information about the underlying platform, but an agent with access to the platform could choose to expose such information through a tool call. However, this exposure is dependent on agent behavior, and is not specifically made possible by WebMCP.
There was a problem hiding this comment.
Something like that seems totally fine, yeah. It's still a machine to machine sort of interface, so it's important to call out any data can be transferred.
There was a problem hiding this comment.
It's the agent that's exposing it. Just like it might in a standard HTML form without WebMCP.
This comes up a lot. Having this written out once in a canonical location in the spec's security and privacy considerations seems like it could be helpful.
|
|
||
| > 09. Do features in this specification enable access to device sensors? | ||
|
|
||
| No. |
There was a problem hiding this comment.
Like the above, that depends on how it's used. I think it's fair to say "no" here, but we should note the general "pipe-to-powerful-agent" nature of the API.
There was a problem hiding this comment.
I think in the spirit of treating the agent like an extension of the user, it would be the equivalent of a pre-AI website asking the user to: "Input in this HTML form, the result of running the FOO operation on the Bluetooth device plugged into your laptop". We wouldn't really say that this site "enables access to device sensors" if the user follows these instructions. I think this question is more reserved for deterministic privilege access that wasn't possible via script before.
There was a problem hiding this comment.
Well the difference is that in this case it can be done semi-automatically without the human explicitly sharing the data, depending on the agent implementation. I generally agree with your sentiment here, but as per the above it should be pointed out that agents can do anything.
|
|
||
| > 12. Do features in this specification allow an origin some measure of control over a user agent's native UI? | ||
|
|
||
| [Tool annotations](https://webmachinelearning.github.io/webmcp/#dom-modelcontexttoolannotations) can indirectly influence how an agent presents a tool invocation in its UI (e.g., a `readOnlyHint` may cause the agent to skip a confirmation step). |
There was a problem hiding this comment.
If the user agent runs a built in agent, the tool responses are presumably also shown and/or influence what you would call "native" UI (usually these things are isolated documents in a sidebar of some sort, but it's generally more privileged than regular websites and may appear part of the "native" experience to users).
There was a problem hiding this comment.
+1. Plus even the tool title is exactly designed for this, so it's worth calling out that in addition to outputs.
|
|
||
| > 15. How do the features in this specification work in the context of a browser's Private Browsing or Incognito mode? | ||
|
|
||
| We do not anticipate any differences. |
There was a problem hiding this comment.
We should probably use this as an opportunity to mention Private / Incognito Mode in the S&P considerations - there is a risk that agents weaken that barrier by not caring about the difference between the two modes and leaking user state into private browsing.
There was a problem hiding this comment.
Since you have a PR for this, I think we can resolve this specific comment.
There was a problem hiding this comment.
Unresolving since we landed #201 and should probably link to that section of the spec here actually...
|
Sorry for the delay, a few comments but L (pretty) GTM already! :) |
domfarolino
left a comment
There was a problem hiding this comment.
Overall it looks like we're pretty close, but there are a few open threads to resolve:
Addresses #193
cc: @bwalderman @johannhof @domfarolino