DOCS-219 merged index advice wiki to manual

source/administration/indexes.txt

@@ -7,17 +7,13 @@ Indexing Operations
 Synopsis
 --------
 
-Indexes allow MongoDB to process and fulfill queries quickly, by
-creating an small and efficient representation of the documents in the
-collection. Fundamentally, indexes in MongoDB are operationally
-similar to indexes in other database systems. Read the
-":doc:`/core/indexes`" documentation for more information on the
-fundamentals of indexing in MongoDB, and the
-":doc:`/applications/indexes`" documentation for practical strategies
-and examples for using indexes in your application.
-
-This document provides operational guidelines and procedures related
-to indexing data in MongoDB collections.
+This document provides operational guidelines and procedures for
+indexing data in MongoDB collections. For the fundamentals of MongoDB
+indexing, see the :doc:`/core/indexes` document. For strategies and
+practical approaches, see the :doc:`/applications/indexes` document.
+
+Indexes allow MongoDB to process and fulfill queries quickly by creating
+small and efficient representations of the documents in a collection.
 
 Operations
 ----------
@@ -334,8 +330,8 @@ following tools:
 
   Append the :method:`explain() <cursor.explain()>` method to any cursor
   (e.g. query) to return a document with statistics about the query
-  process, including the index used, and the number of documents
-  scanned.
+  process, including the index used, the number of documents scanned,
+  and the time the query takes to process in milliseconds.
 
 - :method:`cursor.hint()`
 

source/applications/indexes.txt

@@ -7,20 +7,52 @@ Indexing Strategies
 Synopsis
 --------
 
-Indexes allow MongoDB to process and fulfill queries quickly, by
-creating an small and efficient representation of the documents in the
-collection. Read the ":doc:`/core/indexes`" documentation for more
-information on the fundamentals of indexing in MongoDB, and the
-":doc:`/administration/indexes`" documentation for operational
-guidelines and examples for building and managing indexes.
+This document provides practical approaches and strategies for indexing
+in MongoDB. For the fundamentals of MongoDB indexing, see the
+:doc:`/core/indexes` document. For operational guidelines and
+procedures, see the :doc:`/administration/indexes` document.
 
-This document provides an overview of approaches to indexing with
-MongoDB and a selection of strategies that you can use as you develop
-applications with MongoDB.
+Indexes allow MongoDB to process and fulfill queries quickly
+by creating small and efficient representations of the documents in a
+collection.
 
 Strategies
 ----------
 
+The best indexes for your application are based on a number of important
+factors, including the kinds of queries you expect, the ratio of reads
+to writes, and the amount of free memory on your system. The best
+strategy for designing indexes is always to profile a variety of index
+configurations with data sets similar to the ones you'll be running in
+production and to see which perform best. There's no substitute for good
+empirical analyses.
+
+.. _indexes-create-to-match-queries:
+
+Create Indexes to Match Your Queries
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you query on a single key, a single-key index will do. For
+example, assume you're searching for a blog post's slug:
+
+.. code-block:: javascript
+
+   db.posts.find({ slug : 'state-of-mongodb-2010' })
+
+In this case, a unique index on a single key is best:
+
+.. code-block:: javascript
+
+   db.posts.ensureIndex({ slug: 1 }, {unique: true})
+
+If you query on multiple keys, use a :ref:`compound index
+<index-type-compound>`. With the exception of queries that use the
+:operator:`$or` operator, a query cannot use multiple indexes. A query
+must use only one index. Therefore, to query on multiple keys, use a
+:ref:`compound index <index-type-compound>`.
+
+If you query on multiple keys and sort the results, see :ref:`index-sort`.
+
 .. _covered-queries:
 .. _indexes-covered-queries:
 
@@ -44,7 +76,9 @@ database. To use a covered index you must:
 
 Use the :method:`explain() <cursor.explain()>` to test the query. If
 MongoDB was able to use a covered index, then the value of the
-``indexOnly`` field will be ``true``.
+``indexOnly`` field will be ``true``. For more information on
+:method:`explain() <cursor.explain()>`, see
+:ref:`indexes-measuring-use`.
 
 Covered queries are much faster than other queries, for two reasons:
 indexes are typically stored in RAM *or* located sequentially on
@@ -63,29 +97,25 @@ sort operations without the use of an index, these operations are:
 
 #. Abort when the sort operation consume 32 megabytes of memory.
 
-For the best result, index the field you want sorted query
-results. For example:
-
-- if you have an ``{ username: 1 }`` index, you can use this index to
-  return documents sorted by the ``username`` field.
+For the best result, index the field you want sorted in your query
+results. For example, if you have a ``{ username: 1 }`` index, you can
+use this index to return documents sorted by the ``username`` field.
 
-  *MongoDB can return sorted results in either ascending or descending
-  order using an index in ascending or descending order,* because
-  MongoDB can transverse items in the index in both
-  directions. For more information about index order see the section
-  on ":ref:`Ascending and Descending Index Order
-  <index-ascending-and-descending>`."
+*MongoDB can return sorted results in either ascending or descending
+order using an index in ascending or descending order,* because MongoDB
+can transverse items in the index in both directions. For more
+information about index order see :ref:`Ascending and Descending Index
+Order <index-ascending-and-descending>`.
 
-- In general, MongoDB can use a compound index to return sorted
-  results *if*:
+In general, MongoDB can use a :ref:`compound index
+<index-type-compound>` to return sorted results *if*:
 
-  - the first sorted field is first field in the index.
+- The first sorted field is first field in the index.
 
-  - the last field in the index before the first sorted field is an
-    equality match in the query.
+- The last field in the index before the first sorted field is an
+  equality match in the query.
 
-  Consider the example presented below for an illustration of this
-  concept.
+Consider the example below for an illustration of this concept.
 
 .. example::
 
@@ -107,7 +137,7 @@ results. For example:
       db.collection.find( { b:5 } ).sort( { a:1, b:1 } )
 
       db.collection.find( { a:{ $gt:4 } } ).sort( { a:1, b:1 } )
- 
+
       db.collection.find( { a:5 } ).sort( { a:1, b:1 } )
       db.collection.find( { a:5 } ).sort( { b:1, c:1 } )
 
@@ -125,13 +155,15 @@ results. For example:
       db.collection.find( { b:5 } ).sort( { b:1 } )
       db.collection.find( { b:{ $gt:5 } } ).sort( { a:1, b:1 } )
 
-Store Indexes in Memory
-~~~~~~~~~~~~~~~~~~~~~~~
+.. _indexes-ensure-indexes-fit-ram:
+
+Ensure Indexes Fit RAM
+~~~~~~~~~~~~~~~~~~~~~~
 
 For best results, always ensure that your indexes fit entirely in RAM,
-so the system doesn't need to read the index from disk to
-fulfill a query. If your indexes approach or exceed the total size of
-available RAM, they may not fit in memory.
+so the system doesn't need to read the index from disk to fulfill a
+query. If your indexes approach or exceed the total size of available
+RAM, they may not fit in memory.
 
 You can check the size of your indexes in the :program:`mongo` shell,
 using the :method:`db.collection.totalIndexSize()` helper. You may also
@@ -162,13 +194,13 @@ Considerations
 Above all, when developing your indexing strategy you should have a
 deep understanding of:
 
-- the application's queries.
+- The application's queries.
 
-- the relative frequency of each query in the application.
+- The relative frequency of each query in the application.
 
-- the current indexes created for your collections.
+- The current indexes created for your collections.
 
-- which indexes the most common queries use.
+- Which indexes the most common queries use.
 
 MongoDB can only use *one* index to support any given
 operation. However, each clause of an :operator:`$or` query can use
@@ -237,6 +269,23 @@ with fulfilling the query. There are two aspects of selectivity:
    ``a`` are evenly distributed *and* the query can selects a specific
    document using the index.
 
+.. example::
+
+   Avoid single-key indexes with low selectivity. Suppose you have a
+   field called ``status`` where the possible values are ``new`` and
+   ``processed``. If you add an index on ``status`` you've created a
+   low-selectivity index, meaning that the index will be of little help
+   in locating records and will be just taking up space.
+
+   A better strategy, depending on your queries, would be to create a
+   :ref:`compound index <index-type-compound>` that includes the
+   low-selectivity field. For instance, you could have a compound index
+   on ``status`` and ``created_at.``
+
+   Another option, again depending on your use case, might be to use
+   separate collections, one for each status. Experimentation and
+   benchmarks will help you choose the best approach.
+
 To ensure optimal performance, use indexes that are maximally
 selective relative to your queries. At the same time queries need to
 be appropriately selective relative to your indexed data. If overall
@@ -245,6 +294,16 @@ to return results, then some queries may perform faster without
 indexes. See the :ref:`indexes-measuring-use` section for more
 information on testing information.
 
+Write-heavy Applications
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+If your application is write-heavy, then be careful when creating new
+indexes, since each additional index with impose a small
+write-performance penalty. In general, don't be cavalier about adding
+indexes. Indexes should be added to complement your queries. Always have
+a good reason for adding a new index, and make sure you've benchmarked
+alternative strategies.
+
 Insert Throughput
 ~~~~~~~~~~~~~~~~~
 
@@ -258,10 +317,10 @@ some amount of overhead to these operations. In almost every case, the
 performance gains that indexes realize for read operations are worth
 the insertion penalty; however:
 
-- in some cases, an index to support an infrequent query may incur
+- In some cases, an index to support an infrequent query may incur
   more insert-related costs than saved read-time.
 
-- in some situations, if you have many indexes on a collection with a
+- In some situations, if you have many indexes on a collection with a
   high insert throughput and a number of very similar indexes, you may
   find better overall results by using a slightly less effective index
   on some queries if it means consolidating the total number of
@@ -274,7 +333,8 @@ the insertion penalty; however:
 
 - In some cases a single compound on two or more fields index may
   support all of the queries that index on a single field index, or a
-  smaller compound index. In general, MongoDB can use compound index
+  smaller :ref:`compound index <index-type-compound>`. In general,
+  MongoDB can use compound index
   to support the same queries as any of its prefixes. Consider the
   following example:
 

source/core/indexes.txt

@@ -4,15 +4,21 @@ Indexing Overview
 
 .. default-domain:: mongodb
 
+This document provides an overview of indexes in MongoDB, including
+index types and creation options. For operational guidelines and
+procedures, see the :doc:`/administration/indexes` document. For
+strategies and practical approaches, see the
+:doc:`/applications/indexes` document.
+
 Synopsis
 --------
 
 An index is a data structure that allows you to quickly locate documents
-based on the values stored in certain specified fields. Fundamentally, indexes
-in MongoDB are similar to indexes in other database systems. MongoDB
-supports indexes on any field or sub-field contained in documents
-within a MongoDB collection. Consider the following core features of
-indexes:
+based on the values stored in certain specified fields. Fundamentally,
+indexes in MongoDB are similar to indexes in other database systems.
+MongoDB supports indexes on any field or sub-field contained in
+documents within a MongoDB collection. MongoDB indexes have the
+following core features:
 
 - MongoDB defines indexes on a per-:term:`collection` level.
 
@@ -21,9 +27,9 @@ indexes:
   operation.
 
 - Every query, including update operations, use one and only one
-  index. The query optimizer selects the index empirically, by
-  occasionally running alternate query plans, and selecting the plan
-  with the best response time for each query type.  You can override
+  index. The query optimizer selects the index empirically by
+  occasionally running alternate query plans and by selecting the plan
+  with the best response time for each query type. You can override
   the query optimizer using the :method:`cursor.hint()` method.
 
 - You can create indexes on a single field or on multiple fields using
@@ -39,13 +45,6 @@ indexes:
   documents that MongoDB needs to store in memory, thus maximizing database
   performance and throughput.
 
-Continue reading for a complete overview of indexes in MongoDB,
-including the :ref:`types of indexes <index-types>`, basic
-:ref:`operations with indexes <index-operations>`, and other MongoDB
-:ref:`features <index-features>` implemented using indexes.
-
-.. TODO links to other documents about indexing
-
 .. index:: index types
 .. _index-types:
 

source/faq.txt

@@ -10,5 +10,4 @@ Frequently Asked Questions
    /faq/sharding
    /faq/replica-sets
    /faq/storage
-
-.. seealso:: The :wiki:`Indexing FAQ <Indexing+Advice+and+FAQ>` wiki page.
+   /faq/indexes

source/faq/indexes.txt

@@ -0,0 +1,51 @@
+============
+FAQ: Indexes
+============
+
+.. default-domain:: mongodb
+
+This document addresses common questions regarding MongoDB indexes.
+
+If you don't find the answer you're looking for, you can:
+
+- Check the :doc:`/indexes` documentation.
+
+- Check the :doc:`complete list of FAQs </faq>`.
+
+- Post your question to the `MongoDB User Mailing List
+  <https://groups.google.com/forum/?fromgroups#!forum/mongodb-user>`_.
+
+.. contents:: Frequently Asked Questions:
+   :backlinks: none
+   :local:
+
+Should you run :dbcommand:`ensureIndex` after every insert?
+-----------------------------------------------------------
+
+No. An index needs to be created only once for a collection. After
+initial creation, MongoDB automatically updates the index as data
+changes.
+
+Will building a large index affect database performance?
+--------------------------------------------------------
+
+Building an index can be an IO-intensive operation, especially if you
+have a large collection. This is true on any database system that
+supports secondary indexes, including MySQL. If you need to build an
+index on a large collection, consider building the index in the
+background. See :ref:`index-creation-operations`.
+
+If you build a large index without the background option, and if doing
+so causes the database to stop responding, you have two options:
+
+- Wait for the index to finish building
+
+- Kill the current operation (see :method:`db.killOp()`). The partial
+  index will be deleted.
+
+Using :operator:`$ne` and :operator:`$nin` in a query is slow. Why?
+-------------------------------------------------------------------
+
+The :operator:`$ne` and :operator:`$nin` operators can match much of an
+index. If you need to use these, it is often best to make sure that an
+additional, more selective criterion is part of the query.