How do we handle federation in SPARQL queries

[mmccool]

SPARQL federation has the following issues:

1. Potential for DoS attack vector, since SPARQL queries are expensive and federation amplifies this
2. SPARQL federation requires that the URLs of other SPARQL endpoints are known in advance
3. If one directory links to another, and the client follows the link, is that considered federation? We need to read the spec carefully; service endpoints vs. documents. Is SPARQL definition appropriate or do we need our own (more constrained) definition?

[mmccool]

My comment on 2: we may get multiple results from an "Introduction" query, and perhaps "federation" could be over these results.

[mmccool]

Proposal:

1. (Limited to delegated/recursive federation, where a directory queries other directories on a client's behalf). SPARQL does not allow "recursion" per se, delegated endpoints must be known in advance. There are also other ways to limit DoS attacks, eg. rate-limiting). Allow SPARQL delegation in a directory implementation, but do not require it, and add an editor's note regarding the DoS concern.
2. A client may do an introduction and get multiple results, send queries to exploration services (eg directories and/or peer-to-peer) in parallel, and merge the results.
3. Returning links to other directories so a client can also query them is permitted. See Add links section to directory information model #34

Not included:

• SPARQL implementations using delegation/federation internally are not included in 1. We are only constraining delegation/federation between WoT TDDs.

To do: create text for 1 and 2. 3 will be handled during resolution of #34
@AndreaCimminoArriaga will handle 1, @mmccool will handle 2.

[mmccool]

So... PR has a SHOULD assertion. For my proposal above (allow it, but do not require it) I was thinking this should be a MAY. SHOULD means it is recommended., which is something we should discuss. Certainly if it is implemented, it must follow the SPARQL spec, and we seem to agree that it should be optional. So the thing we have to decide is "how" optional it is. We should also look very carefully at RFC2119, the definitions of MAY and SHOULD are specific:

SHOULD: This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.

MAY: This word, or the adjective "OPTIONAL", mean that an item is
truly optional. One vendor may choose to include the item because a
particular marketplace requires it or because the vendor feels that
it enhances the product while another vendor may omit the same item.
An implementation which does not include a particular option MUST be
prepared to interoperate with another implementation which does
include the option, though perhaps with reduced functionality. In the
same vein an implementation which does include a particular option
MUST be prepared to interoperate with another implementation which
does not include the option (except, of course, for the feature the
option provides.)

[mmccool]

I also think a bit more of the above discussion should be included. Perhaps we should add a placeholder at least in the security considerations section for DoS mitigations, note that recursive delegation is intentionally not included for this reason, and also note (somewhere) that since SPARQL queries are expensive, they SHOULD (MUST?) be protected by security schemes and protocols supporting authentication and encryption.

[mmccool]

Discussed 07.12.2020, agreed on improved wording in the PR, once ready and merged I will close this issue.