Skip to content

Commit

Permalink
Update images and content of untranslated
Browse files Browse the repository at this point in the history
  • Loading branch information
looly committed Feb 23, 2015
1 parent c90fb1d commit 5f6adf4
Show file tree
Hide file tree
Showing 283 changed files with 7,514 additions and 5,608 deletions.
30 changes: 12 additions & 18 deletions 052_Mapping_Analysis/45_Mapping.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,20 @@
[[mapping-intro]]
=== Mapping
## 映射

As explained in <<data-in-data-out>>, each document in an index has a _type_.
Every type has its own _mapping_ or _schema definition_. A mapping
defines the fields within a type, the datatype for each field,
and how the field should be handled by Elasticsearch. A mapping is also used
to configure metadata associated with the type.
正如《数据吞吐》一节所说,索引中每个文档都有一个**类型(type)**
每个类型拥有自己的**映射(mapping)**或者**模式定义(schema definition)**。一个映射定义了字段类型,每个字段的数据类型,以及字段被Elasticsearch处理的方式。映射还用于设置关联到类型上的元数据。

We discuss mappings in detail in <<mapping>>. In this section we're going
to look at just enough to get you started.
在《映射》章节我们将探讨映射的细节。这节我们只是带你入门。

[[core-fields]]
==== Core simple field types
### 核心简单字段类型

Elasticsearch supports the following simple field types:
Elasticsearch支持以下简单字段类型:

[horizontal]
String: :: `string`
Whole number: :: `byte`, `short`, `integer`, `long`
Floating point: :: `float`, `double`
Boolean: :: `boolean`
Date: :: `date`
|类型 | 表示 |
|String: | `string`|
|Whole number: | `byte`, `short`, `integer`, `long`
|Floating point: | `float`, `double`
|Boolean: | `boolean`
|Date: | `date`

When you index a document which contains a new field -- one previously not
seen -- Elasticsearch will use <<dynamic-mapping,_dynamic mapping_>> to try
Expand Down
18 changes: 9 additions & 9 deletions 060_Distributed_Search/00_Intro.asciidoc
Original file line number Diff line number Diff line change
@@ -1,34 +1,34 @@
[[distributed-search]]
== Distributed search execution
== Distributed Search Execution

Before moving on, we are going to take a detour and talk about how search is
executed in a distributed environment. It is a bit more complicated than the
basic _create-read-update-delete_ (CRUD) requests that we discussed in
executed in a distributed environment.((("distributed search execution"))) It is a bit more complicated than the
basic _create-read-update-delete_ (CRUD) requests((("CRUD (create-read-update-delete) operations"))) that we discussed in
<<distributed-docs>>.

.Content warning
.Content Warning
****
The information presented below is for your interest. You are not required to
The information presented in this chapter is for your interest. You are not required to
understand and remember all the detail in order to use Elasticsearch.
Read the section to gain a taste for how things work, and to know where the
Read this chapter to gain a taste for how things work, and to know where the
information is in case you need to refer to it in the future, but don't be
overwhelmed by the detail.
****

A CRUD operation deals with a single document that has a unique combination of
`_index`, `_type` and <<routing-value,`routing` value>> (which defaults to the
`_index`, `_type`, and <<routing-value,`routing` values>> (which defaults to the
document's `_id`). This means that we know exactly which shard in the cluster
holds that document.

Search requires a more complicated execution model because we don't know which
documents will match the query -- they could be on any shard in the cluster. A
documents will match the query: they could be on any shard in the cluster. A
search request has to consult a copy of every shard in the index or indices
we're interested in to see if they have any matching documents.

But finding all matching documents is only half the story. Results from
multiple shards must be combined into a single sorted list before the `search`
API can return a ``page'' of results. For this reason, search is executed in a
two-phase process called "Query then Fetch".
two-phase process called _query then fetch_.
46 changes: 23 additions & 23 deletions 060_Distributed_Search/05_Query_phase.asciidoc
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
=== Query Phase

During the initial _query phase_, the query is broadcast to a shard copy (a
During the initial _query phase_, the((("distributed search execution", "query phase")))((("query phase of distributed search"))) query is broadcast to a shard copy (a
primary or replica shard) of every shard in the index. Each shard executes
the search locally and builds a _priority queue_ of matching documents.
the search locally and ((("priority queue")))builds a _priority queue_ of matching documents.

.Priority queue
.Priority Queue
****
A priority queue is just a sorted list which holds the _Top-N_ matching
A _priority queue_ is just a sorted list that holds the _top-n_ matching
documents. The size of the priority queue depends on the pagination
parameters: `from` and `size`. For example, the following search request
parameters `from` and `size`. For example, the following search request
would require a priority queue big enough to hold 100 documents:
[source,js]
Expand All @@ -25,49 +25,49 @@ GET /_search
The query phase process is depicted in <<img-distrib-search>>.

[[img-distrib-search]]
.Query Phase of distributed search
image::images/06-01_query.png["Query Phase of distributed search"]
.Query phase of distributed search
image::images/elas_0901.png["Query phase of distributed search"]

1. The client sends a `search` request to `Node 3` which creates an empty
The query phase consists of the following three steps:

1. The client sends a `search` request to `Node 3`, which creates an empty
priority queue of size `from + size`.
2. `Node 3` forwards the search request to a primary or replica copy of every
shard in the index. Each shard executes the query locally and adds the
results into a local sorted priority queue of size `from + size`.
3. Each shard returns the doc IDs and sort values of all of the docs in its
3. Each shard returns the doc IDs and sort values of all the docs in its
priority queue to the coordinating node, `Node 3`, which merges these
values into its own priority queue to produce a globally sorted list of
results.
When a search request is sent to a node, that node becomes the coordinating
node. It is the job of this node to broadcast the search request to all
node.((("nodes", "coordinating node for search requests"))) It is the job of this node to broadcast the search request to all
involved shards, and to gather their responses into a globally sorted result
set which it can return to the client.
set that it can return to the client.

The first step is to broadcast the request to a shard copy of every node in
the index. Just like <<distrib-read,document `GET` requests>>, search requests
can be handled by a primary shard or by any of its replicas. This is how more
can be handled by a primary shard or by any of its replicas.((("shards", "handling search requests"))) This is how more
replicas (when combined with more hardware) can increase search throughput.
A coordinating node will round robin through all shard copies on subsequent
A coordinating node will round-robin through all shard copies on subsequent
requests in order to spread the load.

Each shard executes the query locally and builds a sorted priority queue of
length `from + size`, in other words enough results to satisfy the global
length `from + size`&#x2014;in other words, enough results to satisfy the global
search request all by itself. It returns a lightweight list of results to the
coordinating node, which contains just the doc IDs and any values required for
sorting, such as the `_score`.

The coordinating node merges these shard-level results into its own sorted
priority queue which represents the globally sorted result set. Here the query
priority queue, which represents the globally sorted result set. Here the query
phase ends.

.Multi-index search
****
An index can consist of one or more primary shards, so a search request
[NOTE]
====
An index can consist of one or more primary shards,((("indices", "multi-index search"))) so a search request
against a single index needs to be able to combine the results from multiple
shards. A search against *multiple* or *all* indices works in exactly the same
way -- there are just more shards involved.
****
shards. A search against _multiple_ or _all_ indices works in exactly the same
way--there are just more shards involved.
====
36 changes: 19 additions & 17 deletions 060_Distributed_Search/10_Fetch_phase.asciidoc
Original file line number Diff line number Diff line change
@@ -1,60 +1,62 @@
=== Fetch Phase

The query phase identifies which documents satisfy the search request, but we
The query phase identifies which documents satisfy((("distributed search execution", "fetch phase")))((("fetch phase of distributed search"))) the search request, but we
still need to retrieve the documents themselves. This is the job of the fetch
phase.
phase, shown in <<img-distrib-fetch>>.

[[img-distrib-fetch]]
.Fetch Phase of distributed search
image::images/06-02_fetch.png["Fetch Phase of distributed search"]
.Fetch phase of distributed search
image::images/elas_0902.png["Fetch Phase of distributed search"]

The distributed phase consists of the following steps:

1. The coordinating node identifies which documents need to be fetched and
issues a multi `GET` request to the relevant shards.
2. Each shard loads the documents and _enriches_ them, if required, then
2. Each shard loads the documents and _enriches_ them, if required, and then
returns the documents to the coordinating node.
3. Once all documents have been fetched, the coordinating node returns the
results to the client.
The coordinating node first decides which documents *actually* need to be
fetched. For instance, if our query specified `{ "from": 90, "size": 10 }` then
The coordinating node first decides which documents _actually_ need to be
fetched. For instance, if our query specified `{ "from": 90, "size": 10 }`,
the first 90 results would be discarded and only the next 10 results would
need to be retrieved. These documents may come from one, some or all of the
need to be retrieved. These documents may come from one, some, or all of the
shards involved in the original search request.

The coordinating node builds a <<distrib-multi-doc,multi-get request>> for
each shard which holds a pertinent document and sends the request to the same
each shard that holds a pertinent document and sends the request to the same
shard copy that handled the query phase.

The shard loads the document bodies -- the `_source` field -- and, if
The shard loads the document bodies--the `_source` field--and, if
requested, enriches the results with metadata and
<<highlighting-intro,search snippet highlighting>>.
Once the coordinating node receives all results, it assembles them into a
single response which it returns to the client.
single response that it returns to the client.

.Deep pagination
.Deep Pagination
****
The query-then-fetch process supports pagination with the `from` and `size`
parameters, but *within limits*. Remember that each shard must build a priority
parameters, but _within limits_. ((("size parameter")))((("from parameter")))((("pagination", "supported by query-then-fetch process")))((("deep paging, problems with"))) Remember that each shard must build a priority
queue of length `from + size`, all of which need to be passed back to
the coordinating node. And the coordinating node needs to sort through
`number_of_shards * (from + size)` documents in order to find the correct
`size` documents.
Depending on the size of your documents, the number of shards, and the
hardware you are using, paging 10,000 to 50,000 results (1,000 to 5,000 pages)
deep should be perfectly doable. But with big enough `from` values, the
deep should be perfectly doable. But with big-enough `from` values, the
sorting process can become very heavy indeed, using vast amounts of CPU,
memory and bandwidth. For this reason we strongly advise against deep paging.
memory, and bandwidth. For this reason, we strongly advise against deep paging.
In practice, ``deep pagers'' are seldom human anyway. A human will stop
paging after two or three pages and will change their search criteria. The
paging after two or three pages and will change the search criteria. The
culprits are usually bots or web spiders that tirelessly keep fetching page
after page until your servers crumble at the knees.
If you *do* need to fetch large numbers of docs from your cluster, you can
If you _do_ need to fetch large numbers of docs from your cluster, you can
do so efficiently by disabling sorting with the `scan` search type,
which we discuss <<scan-scroll,later in this chapter>>.
Expand Down
63 changes: 30 additions & 33 deletions 060_Distributed_Search/15_Search_options.asciidoc
Original file line number Diff line number Diff line change
@@ -1,46 +1,43 @@
=== Search options
=== Search Options

There are a few optional query-string parameters which can influence the
search process:
A few ((("search options")))optional query-string parameters can influence the search process.

==== `preference`
==== preference

The `preference` parameter allows you to control which shards or nodes are
used to handle the search request. It accepts values like: `_primary`,
`_primary_first`, `_local`, `_only_node:xyz`, `_prefer_node:xyz` and
The `preference` parameter allows((("preference parameter")))((("search options", "preference"))) you to control which shards or nodes are
used to handle the search request. It accepts values such as `_primary`,
`_primary_first`, `_local`, `_only_node:xyz`, `_prefer_node:xyz`, and
`_shards:2,3`, which are explained in detail on the
{ref}/search-request-preference.html[search `preference`]
documentation page.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-preference.html[search `preference`]
documentation page.

However, the most generally useful value is some arbitrary string, to avoid
the _bouncing results_ problem.
the _bouncing results_ problem.((("bouncing results problem")))

[[bouncing-results]]
.Bouncing results
.Bouncing Results
****
Imagine that you are sorting your results by a `timestamp` field and there are
two documents with the same timestamp. Because search requests are
Imagine that you are sorting your results by a `timestamp` field, and
two documents have the same timestamp. Because search requests are
round-robined between all available shard copies, these two documents may be
returned in one order when the request is served by the primary, and in
another order when served by the replica.
This is known as the _bouncing results_ problem: every time the user refreshes
the page they see the results in a different order.
The problem can be avoided by always using the same shards for the same user,
the page, the results appear in a different order. The problem can be avoided by always using the same shards for the same user,
which can be done by setting the `preference` parameter to an arbitrary string
like the user's session ID.
****

==== `timeout`
==== timeout

By default, the coordinating node waits to receive a response from all shards.
By default, the coordinating node waits((("search options", "timeout"))) to receive a response from all shards.
If one node is having trouble, it could slow down the response to all search
requests.

The `timeout` parameter tells the coordinating node how long it should wait
The `timeout` parameter tells((("timeout parameter"))) the coordinating node how long it should wait
before giving up and just returning the results that it already has. It can be
better to return some results than none at all.

Expand All @@ -59,34 +56,34 @@ how many shards responded successfully:
...
--------------------------------------------------
<1> The search request timed out.
<2> One shard out of 5 failed to respond in time.
<2> One shard out of five failed to respond in time.

If all copies of a shard fail for other reasons -- perhaps because of a
hardware failure -- this will also be reflected in the `_shards` section of
If all copies of a shard fail for other reasons--perhaps because of a
hardware failure--this will also be reflected in the `_shards` section of
the response.

[[search-routing]]
==== `routing`
==== routing

In <<routing-value>> we explained how a custom `routing` parameter could be
In <<routing-value>>, we explained how a custom `routing` parameter((("search options", "routing")))((("routing parameter"))) could be
provided at index time to ensure that all related documents, such as the
documents belonging to a single user, are stored on a single shard. At search
time, instead of searching on all of the shards of an index, you can specify
time, instead of searching on all the shards of an index, you can specify
one or more `routing` values to limit the search to just those shards:

[source,js]
--------------------------------------------------
GET /_search?routing=user_1,user2
--------------------------------------------------

This technique comes in useful when designing very large search systems and we
This technique comes in handy when designing very large search systems, and we
discuss it in detail in <<scale>>.

[[search-type]]
==== `search_type`
==== search_type

While `query_then_fetch` is the default search type, other search types can
be specified for particular purposes, as:
While `query_then_fetch` is the default((("query_then_fetch search type")))((("search options", "search_type")))((("search_type"))) search type, other search types can
be specified for particular purposes, for example:

[source,js]
--------------------------------------------------
Expand All @@ -95,27 +92,27 @@ GET /_search?search_type=count

`count`::

The `count` search type only has a `query` phase. It can be used when you
The `count` search type has only a `query` phase.((("count search type"))) It can be used when you
don't need search results, just a document count or
<<aggregations,aggregations>> on documents matching the query.

`query_and_fetch`::

The `query_and_fetch` search type combine the query and fetch phases into a
The `query_and_fetch` search type ((("query_and_fetch serch type")))combines the query and fetch phases into a
single step. This is an internal optimization that is used when a search
request targets a single shard only, such as when a
<<search-routing,`routing`>> value has been specified. While you can choose
to use this search type manually, it is almost never useful to do so.

`dfs_query_then_fetch` and `dfs_query_and_fetch`::

The `dfs` search types have a pre-query phase which fetches the term
The `dfs` search types((("dfs search types"))) have a prequery phase that fetches the term
frequencies from all involved shards in order to calculate global term
frequencies. We discuss this further in <<relevance-is-broken>>.

`scan`::

The `scan` search type is used in conjunction with the `scroll` API to
The `scan` search type is((("scan search type"))) used in conjunction with the `scroll` API ((("scroll API")))to
retrieve large numbers of results efficiently. It does this by disabling
sorting. We discuss _scan-and-scroll_ in the next section.

Expand Down
Loading

0 comments on commit 5f6adf4

Please sign in to comment.