This book is a work in progress – new chapters will be added as they are written. The content of this book targets the latest version of Elasticsearch.
We welcome feedback – if you spot any errors or would like to suggest improvements, please open an issue on the GitHub repo.
The world is swimming in data. For years we have been simply overwhelmed by the quantity of data flowing through and produced by our systems. Existing technology has focused on how to store and structure warehouses full of data. That’s all well and good… until you actually need to make decisions in real-time informed by that data.
Elasticsearch is a distributed scalable real-time search and analytics engine. It enables you to search, analyze and explore your data, often in ways that you did not anticipate at the start of a project. It exists because raw data sitting on a hard drive is just not useful.
Whether you need full text search, real-time analytics of structured data, or a combination of the two, this book will introduce you to the fundamental concepts required to start working with Elasticsearch at a basic level. With these foundations laid, it will move on to more advanced search techniques which you will need to shape the search experience to fit your requirements.
Elasticsearch is not just about full text search. We explain structured search, analytics, query percolation, geolocation and autocomplete or did- you-mean suggesters. We will also discuss how best to model your data to take advantage of the horizontal scalability of Elasticsearch, and how to configure and monitor your cluster when moving to production.
This book is for anybody who wants to put their data to work. It doesn’t matter whether you are starting a new project and have the flexibility to design the system from the ground up, or whether you need to give new life to a legacy system. Elasticsearch will help you to solve existing problems and open the way to new features that you haven’t yet considered.
This book is suitable for novices and experienced users alike. We expect you to have some programming background and, although not required, it would help to have used SQL and a relational database. We explain concepts from first principles, helping novices to gain a sure footing in the complex world of search.
The reader with a search background will also benefit from this book. Elasticsearch is a new technology which has some familiar concepts. The more experienced user will gain an understanding of how those concepts have been implemented and how they interact in the context of Elasticsearch. Even in the early chapters, there are nuggets of information that will be useful to the more advanced user.
Finally, maybe you are in DevOps. While the other departments are stuffing data into Elasticsearch as fast as they can, you’re the one charged with stopping their servers from bursting into flames. Elasticsearch scales effortlessly, as long as your users play within the rules. You need to know how to setup a stable cluster before going into production, then be able to recognize the warning signs at three in the morning in order to prevent catastrophe. The earlier chapters may be of less interest to you but the last part of the book is essential reading — all you need to know to avoid meltdown.
We wrote this book because Elasticsearch needs a narrative. The existing reference documentation is excellent… as long as you know what you are looking for. It assumes that you are intimately familiar with information retrieval concepts, distributed systems, the query DSL and a host of other topics.
This book makes no such assumptions. It has been written so that a complete beginner — to both search and distributed systems — can pick it up and start building a prototype within a few chapters.
We have taken a problem based approach: this is the problem, how do I solve it, and what are the trade-offs of the alternative solutions? We start with the basics and each chapter builds on the preceding ones, providing practical examples and explaining the theory where necessary.
The existing reference documentation explains how to use features. We want this book to explain why and when to use various features.
Elasticsearch tries very hard to make the complex simple, and to a large degree it succeeds in this. That said, search and distributed systems are complex and sooner or later you have to get to grips with some of the complexity in order to take full advantage of Elasticsearch.
Complexity, however, is not the same as magic. We tend to view complex systems as magical black boxes that respond to incantations, but there are usually simple processes at work within. Understanding these processes helps to disspell the magic — instead of hoping that the black box will do what you want, understanding gives you certainty and clarity.
This is a definitive guide: not only do we help you to get started with Elasticsearch, we tackle the deeper more interesting topics too. These include chapters such as [distributed-cluster], [distributed-docs] and [distributed-search], which are not essential reading but do give you a solid understanding of the internals.
The first part of the book should be read in order as each chapter builds on the previous chapter (although you can skim over the chapters mentioned above). Later chapters like [partial-matching] and [proximity-matching] are more standalone and can be referred to as needed.
This book is divided into five parts:
-
Chapters 1 through 10 provide an introduction to Elasticsearch. They explain how to get your data in and out of Elasticsearch, how Elasticsearch interprets the data in your documents, how basic search works and how to manage indices. By the end of this section you will already be able to integrate your application with Elasticsearch. Chapters 2, 4 and 9 are supplemental chapters which provide more insight into the distributed processes at work, but are not required reading.
-
Chapters 11 through 21 offer a deep dive into search — how to index and query your data to allow you to take advantage of more advanced concepts like word proximity, partial matching, fuzzy matching and language awareness. You will understand how relevance works and how to control it to ensure that the best results are on the first page.
-
Chapters 22 through 25 tackle subjects such as aggregations or analytics, geolocation, query percolation and scripting.
-
Chapters 26 through 28 teach you how to understand the flow of data through your system and how to model your data to work well with the infrastructure provided by Elasticsearch.
-
Finally, chapters 29 through 31 discuss moving to production: the important configurations, what to monitor, and how to diagnose and prevent problems.
Because this book focuses on problem solving in Elasticsearch rather than syntax, we sometimes reference the existing documentation for a complete list of parameters. The reference documentation can be found here:
The following typographical conventions are used in this book:
- Italic
-
Indicates new terms or concepts.
- Constant width
-
Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.
|
Tip
|
This icon signifies a tip, suggestion, or general note. |
|
Warning
|
This icon indicates a warning or caution. |
Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/oreillymedia/title_title.
This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Elasticsearch - The Definitive Guide by Clinton Gormley and Zachary Tony (O’Reilly). Copyright 2014 Elasticsearch BV, 978-0-596-xxxx-x.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com.
|
Note
|
Safari Books Online is an on-demand digital library that delivers expert content in both book and video form from the world’s leading authors in technology and business. |
Technology professionals, software developers, web designers, and business and creative professionals use Safari Books Online as their primary resource for research, problem solving, learning, and certification training.
Safari Books Online offers a range of product mixes and pricing programs for organizations, government agencies, and individuals. Subscribers have access to thousands of books, training videos, and prepublication manuscripts in one fully searchable database from publishers like O’Reilly Media, Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, Course Technology, and dozens more. For more information about Safari Books Online, please visit us online.
Please address comments and questions concerning this book to the publisher:
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://www.oreilly.com/catalog/0636920028505.
To comment or ask technical questions about this book, send email to bookquestions@oreilly.com.
For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia