DOCSP-10228 lookup and graphlookup now works on sharded collections

Author: ianf-mongodb

source/includes/5.1-fact-sharded-lookup-graphlookup.rst

@@ -0,0 +1,7 @@
+Starting in MongoDB 5.1, 
+the :pipeline:`$lookup` and :pipeline:`$graphLookup` aggregation stages 
+support :ref:`sharded collections <sharded-cluster>` in the ``from`` 
+parameter. 
+
+In previous versions of MongoDB, ``$lookup`` and ``$graphLookup`` 
+only allowed for unsharded ``from`` collections.

source/reference/operator/aggregation/graphLookup.txt

@@ -82,10 +82,11 @@ Definition
         - Target collection for the :pipeline:`$graphLookup`
           operation to search, recursively matching the
           ``connectFromField`` to the ``connectToField``.
-          The ``from`` collection cannot be
-          sharded and must be in the same
+          The ``from`` collection must be in the same
           database as any other collections used in the operation.
-          For information, see :ref:`graphLookup-sharded-collections`.
+
+          Starting in MongoDB 5.1, the collection specified in the 
+          ``from`` parameter can be sharded.
 
       * - ``startWith``
 
@@ -162,28 +163,8 @@ Considerations
 Sharded Collections
 ~~~~~~~~~~~~~~~~~~~
 
-The collection specified in ``from`` cannot be
-:doc:`sharded</sharding>`. However, the collection on which you run the
-:method:`~db.collection.aggregate()` method can be sharded. That is, in
-the following:
-
-.. code-block:: javascript
-
-   db.collection.aggregate([ 
-      { $graphLookup: { from: "fromCollection", ... } } 
-   ])
-
-- The ``collection`` can be sharded.
-
-- The ``fromCollection`` cannot be sharded.
-
-To join multiple sharded collections, consider:
-
-- Modifying client applications to perform manual lookups instead of
-  using the :pipeline:`$graphLookup` aggregation stage.
-
-- If possible, using an :ref:`embedded data model
-  <data-modeling-embedding>` that removes the need to join collections.
+Starting in MongoDB 5.1, you can specify :doc:`sharded collections </sharding>`
+in the ``from`` parameter of :pipeline:`$graphLookup` stages.
 
 Max Depth
 ~~~~~~~~~

source/reference/operator/aggregation/lookup.txt

@@ -62,8 +62,10 @@ The :pipeline:`$lookup` takes a document with these fields:
      - .. _lookup-eq-from:
 
        Specifies the collection in the *same* database to perform
-       the join with. The ``from`` collection cannot be sharded. For
-       details, see :ref:`lookup-sharded-collections`.
+       the join with.
+
+       Starting in MongoDB 5.1, the collection specified in the ``from``
+       parameter can be sharded.
 
    * - :ref:`localField <lookup-eq-localField>`
 
@@ -178,8 +180,9 @@ The :pipeline:`$lookup` stage accepts a document with these fields:
      - .. _lookup-join-from:
 
        Specifies the collection in the *same* database to perform the
-       join operation. The joined collection cannot be sharded (see
-       :ref:`lookup-sharded-collections`).
+       join operation.
+
+       Starting in MongoDB 5.1, the ``from`` collection can be sharded.
 
    * - :ref:`let <lookup-join-let>`
 
@@ -282,8 +285,9 @@ The :pipeline:`$lookup` accepts a document with these fields:
      - .. _lookup-concise-correlated-subquery-from:
 
        Specifies the foreign collection in the *same* database to join
-       to the local collection. The foreign collection cannot be sharded
-       (see :ref:`lookup-sharded-collections`).
+       to the local collection.
+
+       Starting in MongoDB 5.1, the ``from`` collection can be sharded.
 
    * - :ref:`localField
        <lookup-concise-correlated-subquery-localField>`
@@ -396,47 +400,11 @@ Restrictions
 
 .. _lookup-sharded-collections:
 
-Sharded Collection Restrictions
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-In the :pipeline:`$lookup` stage, the ``from`` collection cannot be
-:doc:`sharded</sharding>`. However, the collection on which you run the
-:method:`~db.collection.aggregate()` method can be sharded.
-Specifically, in this example:
-
-.. code-block:: javascript
-
-   db.collection.aggregate([ 
-      { $lookup: { from: "fromCollection", ... } } 
-   ])
-
-- The ``collection`` can be sharded.
-
-- The ``fromCollection`` cannot be sharded.
-
-To join a sharded collection with an unsharded collection, run the
-aggregation on the sharded collection and lookup the unsharded
-collection. For example:
-
-.. code-block:: javascript
-
-   db.shardedCollection.aggregate([
-      { $lookup: { from: "unshardedCollection", ... } }
-   ])
-
-Alternatively, or to join multiple sharded collections, consider:
-
-- Modifying client applications to perform manual lookups instead of
-  using the :pipeline:`$lookup` aggregation stage.
-
-- If possible, using an :ref:`embedded data model
-  <data-modeling-embedding>` that removes the need to join collections.
+Sharded Collections
+~~~~~~~~~~~~~~~~~~~
 
-- If possible, using an `Atlas Data Lake 
-  <https://docs.mongodb.com/datalake/>`__ 
-  `$lookup 
-  <https://docs.mongodb.com/datalake/reference/pipeline/lookup-stage/>`__ 
-  pipeline stage to lookup a sharded collection.
+Starting in MongoDB 5.1, you can specify :doc:`sharded collections </sharding>`
+in the ``from`` parameter of :pipeline:`$lookup` stages.
 
 Examples
 --------

source/release-notes/5.1-compatibility.txt

@@ -32,6 +32,9 @@ Removed Parameters
 
 .. include:: /includes/5.1/removed-parameters.rst
 
+Aggregation
+-----------
+
 Removed Index Types
 -------------------
 

source/release-notes/5.1.txt

@@ -26,6 +26,11 @@ Aggregation
 
 .. _5.1-rel-notes-new-agg-operators:
 
+$lookup and $graphLookup with sharded collections
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. include:: /includes/5.1-fact-sharded-lookup-graphlookup.rst
+
 New Aggregation Operators
 ~~~~~~~~~~~~~~~~~~~~~~~~~