Commit Graph

1376 Commits

Author SHA1 Message Date
Al Lefebvre 94f82368f0 Update templates.asciidoc
I've been attempting to programatically verify that adding index templates via the `{path.conf}/templates/` directory works fine although I was never able to validate this via an API call to the `/_template/`.  It seems that these templates do not appear in that API call, which I discovered in the following mail thread:

http://elasticsearch-users.115913.n3.nabble.com/Loading-of-index-settings-template-from-file-in-config-templates-td4024923.html#d1366317284000-912

My question is why wouldn't the `/_template/*` method return these templates?  This tends to complicate things for those that want to perform automated tests to verify that they are in fact being recognized and used by Elasticsearch.
2015-03-20 14:52:20 -06:00
Michael McCandless 34b397597c Core: increase default rate limiting for snapshot, restore and recovery to 40 MB/sec
This also fixes a possible issue that may cause over-throttling when
there are many small files being copied, which should be rare.

Closed #10185
2015-03-20 16:09:42 -04:00
javanna 9ca74c8217 [DOCS] clarify no-master-block docs
Closes #9739
2015-03-20 15:58:16 +01:00
javanna 4348959f9d Delete api: remove broadcast delete if routing is missing when required
This commit changes the behaviour of the delete api when processing a delete request that refers to a type that has routing set to required in the mapping, and the routing is missing in the request. Up until now the delete api sent a broadcast delete request to all of the shards that belong to the index, making sure that the document could be found although the routing value wasn't specified. This was probably not the best choice: if the routing is set to required, an error should be thrown instead.

A `RoutingMissingException` gets now thrown instead, like it happens in the same situation with every other api (index, update, get etc.). Last but not least, this change allows to get rid of a couple of `TransportAction`s, `Request`s and `Response`s and simplify the codebase.

Closes #9123
Closes #10136
2015-03-20 09:19:43 +01:00
Simon Willnauer 1168347b9d [REPLICATION] Remove `async` replication
Closes #10114
2015-03-19 14:44:21 -07:00
Clinton Gormley aa94ced0ae Remove references to the thrift and memcached transport plugins
as they are no longer supported

Closes #10166
2015-03-19 20:49:58 +01:00
Clinton Gormley 25369f0727 Remove async replication from the docs and REST spec
Relates to #10114
2015-03-19 15:34:12 +01:00
javanna f4691458d5 [DOCS] added note about dynamic scriptings and updated links in getting started page
Closes #10074
2015-03-19 15:22:39 +01:00
javanna ddcecc2bc2 [DOCS] added instructions on how to write parameterized tests
Closes #9423
2015-03-19 12:43:51 +01:00
jaymode 105bdd486a [HTTP] add option to only return simple exception messages
Adds a setting to disable detailed error messages and full exception stack traces
in HTTP responses. When set to false, the error_trace request parameter will result
in a HTTP 400 response. When the error_trace parameter is not present, the message
of the first ElasticsearchException will be output and no nested exception messages
will be output.
2015-03-18 17:49:05 -04:00
Joshua Rich db2caa54cd Small grammar fix. 2015-03-17 11:27:13 -07:00
Boaz Leskes b605184471 Recovery: add total operations to the `_recovery` API
This commit adds the current total number of translog operations to the recovery reporting API. We also expose the recovered / total percentage:

```
"translog": {
   "recovered": 536,
   "total": 986,
   "percent": "54.3%",
   "total_time": "2ms",
   "total_time_in_millis": 2
},
```

Closes #9368
Closes #10042
2015-03-17 07:31:29 -07:00
Martijn van Groningen 4393939f5e inner_hits: Nested parent field should be resolved based on the parent inner hit definition, instead of the nested parent field in the mapping.
The behaviour is better in the case someone has multiple levels of nested object fields defined in the mapping and like to define a single inner_hits definition that is two or more levels deep.

If someone wants inner hits on a nested field that is 2 levels deep the following would need to be defined:

```
{
  ...
  "inner_hits" : {
     "path" : {
        "level1" : {
            "inner_hits" : {
               "path" : {
                  "level2" : {
                     "query" : { .... }
                  }
               }
            }
        }
     }
  }
}
```

With this change the above can be defined as:

```
{
  ...
  "inner_hits" : {
     "path" : {
        "level1.level2" : {
            "query" : { .... }
        }
     }
  }
}
```

Closes #9251
2015-03-16 16:31:03 -07:00
Michael McCandless 0683a66277 Core: remove index.fail_on_merge_failure
Always fail the engine if an unexpected exception is hit during merge.

Closes #10088
2015-03-14 09:53:42 -04:00
Lee Hinman 6aec68cd29 Revert "[QUERY] Remove lowercase_expanded_terms and locale options"
This reverts commit d1f7bd97cb.

Ryan pointed out that this needs to work with the multi term query, so
additional analysis and tests should be added.
2015-03-13 13:51:44 -06:00
Lee Hinman d1f7bd97cb [QUERY] Remove lowercase_expanded_terms and locale options
The analysis chain should be used instead of relying on this, as it is
confusing when dealing with different per-field analysers.

The `locale` option was only used for `lowercase_expanded_terms`, which,
once removed, is no longer needed, so it was removed as well.

Fixes #9978
Relates to #9973
2015-03-13 13:17:27 -06:00
Petr Bela f27cb07eb9 [DOCS] fix typo in scripting docs 2015-03-12 15:28:50 -07:00
olivier bourgain bcb4decca9 [DOCS] add missing comma in percentile_rank aggregation example 2015-03-10 08:21:06 -07:00
olivier bourgain fb7cd2ea9a [DOCS] Adjusted geo_distance aggregation example
unit is not returned in the response, but we have key and an implicit from starting at 0 for the first bucket
2015-03-10 08:20:20 -07:00
olivier bourgain eaeddc6bd4 [DOCS] missing curly brace in ip_range aggregation example 2015-03-10 08:19:57 -07:00
David Pilato d9c19cd846 [doc] Cat API: show open and closed indices in _cat/indices
Related to #7936
2015-03-09 15:45:26 -07:00
Britta Weber 580728dfd6 significant terms: add scriptable significance heuristic
This commit adds scripting capability to significant_terms.
Custom heuristics can be implemented with a script that provides
parameters subset_freq, superset_freq,subset_size, superset_size.

closes #7850
2015-03-06 17:06:04 +01:00
Matias Tealdi cba6dff3ac fixing typo in expDecayFunction and adding offset to all dacay functions
closes #9887
2015-03-05 12:28:08 +01:00
Clinton Gormley 3f9d4f9635 Update query-string-syntax.asciidoc
Closes #9965
2015-03-03 20:03:51 +01:00
Pius 430b091e7d Docs: Added default value
Added default value to `cluster.routing.allocation.node_initial_primaries_recoveries`

Closes #9955
2015-03-03 10:15:11 +01:00
Clinton Gormley c223ed0db4 Update search-type.asciidoc
Changed search_type docs to reflect that the `(dfs_)query_and_fetch` modes are an internal optimization and should not be specified explicitly by the user.

Relates to #9606
2015-03-02 10:55:22 +01:00
Pius a90e5c03b7 Update getting-started.asciidoc
Closes #9932
2015-03-01 21:07:09 +01:00
Pius a182c7428d Added note on max # of docs allowed in a shard 2015-03-01 21:07:07 +01:00
Geoff Bourne 0e09c02c56 Spelling out the sort order options
Closes #9768
2015-03-01 21:05:52 +01:00
cgp b1e6df3b6c Update span-multi-term-query.asciidoc
Added comma - there is no "term range" query

Closes #9855
2015-02-28 03:05:05 +01:00
Clinton Gormley e194fb3a07 Docs: Default distance unit in geo distance agg is metres, not km
Closes #9812
2015-02-28 01:45:29 +01:00
David Pilato 0c8da6bb84 [doc] Link mapper-attachment type documentation to its repo
As explained in elasticsearch/elasticsearch-mapper-attachments#101, we should have consistent documentation.

The best option is to link the  documentation in elasticsearch guide to the most recent README in the plugin repo.

Closes #9756
2015-02-27 22:18:59 +01:00
IsaacHaze ee163e570b Docs: Update snapshots.asciidoc
Adds more determiners.

Closes #9673
2015-02-27 20:42:12 +01:00
Ryan Ernst 9d708e20a0 Mappings: Lock down _size field
This also changes the stored setting for _size to true (for
indexes created in 2.x).

see #8143
closes #9913
2015-02-27 11:09:52 -08:00
Ryan Ernst 3b7928d568 Mappings: Lock down _field_names field
Now that we have an explicit `enabled` flag, we can lock down
the field type so it is not mungeable.

see #8143
closes #9912
2015-02-26 15:15:59 -08:00
Ryan Ernst 7181bbde26 Mappings: Remove _boost field
This has been deprecated since 1.0.0.RC1. It is finally removed here.

closes #8875
2015-02-26 15:07:07 -08:00
Ryan Ernst 78df69e6a0 Mappings: Lock down _routing field
`required` is now the only changeable settings (on indexes created after 1.x).

see #8143
closes #9895
2015-02-26 13:09:41 -08:00
Boaz Leskes e9dbfa9ee6 Transport: added a simple request tracer, logging incoming and outgoing requests
The request tracer logs in TRACE level under the `transport.tracer` log and is dynamically configurable with include and exclude arrays to filter out unneeded info. By default all requests are logged with the exception of fault detection pings (fired every second).

add the notion of tracers in the MockTransportService for testing purposes

Closes #9286
2015-02-25 21:33:57 +01:00
Ryan Ernst 32e042f1c4 Mappings: Lock down _index field
see #8143
closes #9870
2015-02-25 12:24:55 -08:00
Lee Hinman 2e9ea4abaf Add support for `minimum_should_match` to `simple_query_string`
This behaves similar to the way that `minimum_should_match` works for
the `match` query (in fact it is implemented in the exact same way)

Fixes #6449
2015-02-25 11:35:33 -07:00
Boaz Leskes 3e32dd985a Recovery: RecoveryState clean up
To support the `_recovery` API, the recovery process keeps track of current progress in a class called RecoveryState. This class currently have some issues, mostly around concurrency (see #6644 ). This PR cleans it up as well as other issues around it:

- Make the Index subsection API cleaner:
- remove redundant information - all calculation is done based on the underlying file map
- clearer definition of what is what: total files, vs reused files (local files that match the source) vs recovered files (copied over). % based progress is reported based on recovered files only.
- cleaned up json response to match other API (sadly this breaks the structure). We now properly report human values for dates and other units.
- Add more robust unit testing
- Detail flag was passed along as state (it's now a ToXContent param)
- State lookup during reporting is now always done via the IndexShard , no more fall backs to many other classes.
- Cleanup APIs around time and move the little computations to the state class as opposed to doing them out of the API

I also improved error messages out of the REST testing infra for things I run into.

Closes #6644
Closes #9811
2015-02-25 17:34:22 +01:00
Igor Motov c5ebdf11bb Snapshot/Restore: add ability to retrieve currently running snapshots
Together with #8782 it should help in the situations simliar to #8887 by adding an ability to get information about currently running snapshot without accessing the repository itself.

Closes #8887
2015-02-25 11:06:32 -05:00
Boaz Leskes 6953777c3a API: add pending tasks count to cluster health
The number of current pending tasks is useful to detect and overloaded master. This commit adds it to the cluster health API. The complete list can be retrieved from the dedicated pending tasks API.

It also adds rest tests for the cluster health variants.

Closes #9877
2015-02-25 14:58:44 +01:00
Clinton Gormley 5a53ff6f1b Update migrate_2_0.asciidoc
More code formatting in breaking changes
2015-02-25 14:13:25 +01:00
Clinton Gormley e805fe71cc Update migrate_2_0.asciidoc
Code formatting in breaking changes
2015-02-25 14:11:57 +01:00
Clinton Gormley 5146cf6256 Update migrate_2_0.asciidoc
Fixed bad heading levels in breaking changes
2015-02-25 14:10:17 +01:00
Clinton Gormley 0c61ea803d Update migrate_2_0.asciidoc
Fixed bad asciidoc in breaking changes
2015-02-25 14:07:19 +01:00
Colin Goodheart-Smithe 2520dc78ec [DOCS] added a note for the default shard_size value 2015-02-25 11:00:55 +00:00
Ryan Ernst be0cef0c43 Mappings: Lock down _type field
see #8143
closes #9869
2015-02-24 22:37:41 -08:00
Ryan Ernst b96bd201c1 Mappings: Lock down _id field
There are two implications to this change.
First, percolator now uses _uid internally, extracting the id portion
when needed. Second, sorting on _id is no longer possible, since you
can no longer index _id. However, _uid can still be used to sort, and
is better anyways as indexing _id just to make it available to
fielddata for sorting is wasteful.

see #8143
closes #9842
2015-02-24 14:26:22 -08:00
Michael Sander fd6c6058ce Remove Triple Negative!
Double negatives are confusing, but a triple negative (1 no, 2 non, 3 null)? It takes five minutes to understand this little sentence.  Cleaned that up a bit.

Closes #9789
2015-02-23 20:09:05 +01:00
Colin Goodheart-Smithe 2753db4685 Scripting: Removed deprecated script parameter names
This change removes the deprecated script parameter names ('file', 'id', and 'scriptField').
It also removes the ability to load file scripts using the 'script' parameter. File scripts should be loaded using the 'script_file' parameter only.
2015-02-23 13:49:21 +00:00
Colin Goodheart-Smithe 7d3856c9d3 [DOCS] update script docs to use preferred script parameter names 2015-02-23 11:16:28 +00:00
Robert Muir 1e015e6e33 Tests: Remove global shared cluster
This was previously attempted in #8854. I revived that branch and did
some performance testing as was suggested in the comments there.

I fixed all the errors, mostly just the rest tests, which
needed to have http enabled on the node settings (the global cluster
previously had this always enabled). I also addressed the comments from
that issue.

My performance tests involved running the entire test suite on my
desktop which has 6 cores, 16GB of ram, and nothing else was being
run on the box at the time. I ran each set of settings 3 times and
took the average time.

| mode    | master | patch | diff |
| ------- | ------ | ----- | ---- |
| local   | 409s   | 417s  | +2%  |
| network | 368s   | 380s  | +3%  |

This increase in average time is clearly worthwhile to pay to achieve
isolation of tests. One caveat is the way I fixed the rest tests
is still to have one cluster for the entire suite, so all the rest
tests can still potentially affect each other, but this is an
issue for another day.

There were some oddities that I noticed while running these tests
that I would like to point out, as they probably deserve some
investigation (but orthogonal to this PR):
* The total test run times are highly variable (more than a minute between the min and max)
* Running in network mode is on average actually *faster* than local mode. How is this possible!?
2015-02-22 22:04:22 -08:00
Martijn van Groningen daefb4c673 Docs: Document that the fielddata loading defaults to eager on the _parent field.
Closes #9804
2015-02-22 23:15:59 +01:00
markharwood 29b1902cfb New aggregations feature - “PercentageScore” heuristic for significant_terms aggregation provides simple “per-capita” type measures.
Closes #9720
2015-02-20 13:22:08 +00:00
Adrien Grand 4708227ecf Codecs: Remove the ability to have custom per-field postings and doc values formats.
This commit makes the `postings_format` and `doc_values_format` options of
mappings illegal on 2.0 and ignored on 1.x (meaning that the default postings
and doc values formats from the codec will be used in such a case).

This removes a fair amount of code.

Close #8746 #9741
2015-02-19 15:47:25 +01:00
Lee Hinman eb666f7f50 Add shadow replicas for shared filesystems
Squashed commit of the following:

commit 20835037c98e7d2fac4206c372717a05a27c4790
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 15:27:17 2015 -0700

    Use Enum for "_primary" preference

commit 325acbe4585179190a959ba3101ee63b99f1931a
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 14:32:41 2015 -0700

    Use ?preference=_primary automatically for realtime GET operations

commit edd49434af5de7e55928f27a1c9ed0fddb1fb133
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 14:32:06 2015 -0700

    Move engine creation into protected createNewEngine method

commit 67a797a9235d4aa376ff4af16f3944d907df4577
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 13:14:01 2015 -0700

    Factor out AssertingSearcher so it can be used by mock Engines

commit 62b0c28df8c23cc0b8205b33f7595c68ff940e2b
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 11:43:17 2015 -0700

    Use IndexMetaData.isIndexUsingShadowReplicas helper

commit 1a0d45629457578a60ae5bccbeba05acf5d79ddd
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:59:31 2015 -0700

    Rename usesSharedFilesystem -> isOnSharedFilesystem

commit 73c62df4fc7da8a5ed557620a83910d89b313aa1
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:58:02 2015 -0700

    Add MockShadowEngine and hook it up to be used

commit c8e8db473830fce1bdca3c4df80a685e782383bc
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:45:50 2015 -0700

    Clarify comment about pre-defined mappings

commit 60a4d5374af5262bd415f4ef40f635278ed12a03
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:18:22 2015 -0700

    Add a test for shadow replicas that uses field data

commit 7346f9f382f83a21cd2445b3386fe67472bc3184
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:37:14 2015 -0700

    Revert changes to RecoveryTarget.java

commit d90d6980c9b737bd8c0f4339613a5373b1645e95
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:35:44 2015 -0700

    Rename `ownsShard` to `canDeleteShardContent`

commit 23001af834d66278ac84d9a72c37b5d1f3a10a7b
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:35:25 2015 -0700

    Remove ShadowEngineFactory, add .newReadOnlyEngine method in EngineFactory

commit b64fef1d2c5e167713e869b22d388ff479252173
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:25:19 2015 -0700

    Add warning that predefined mappings should be used

commit a1b8b8cf0db49d1bd1aeb84e51491f7f0de43b59
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 14:31:50 2015 -0700

    Remove unused import and fix index creation example in docs

commit 0b1b852365ceafc0df86866ac3a4ffb6988b08e4
Merge: b9d1fed a22bd49
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 10:56:02 2015 -0700

    Merge remote-tracking branch 'refs/remotes/origin/master' into shadow-replicas

commit b9d1fed25ae472a9dce1904eb806702fba4d9786
Merge: 4473e63 41fd4d8
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 09:02:27 2015 -0700

    Merge remote-tracking branch 'refs/remotes/origin/master' into shadow-replicas

commit 4473e630460e2f0ca2a2e2478f3712f39a64c919
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 09:00:39 2015 -0700

    Add asciidoc documentation for shadow replicas

commit eb699c19f04965952ae45e2caf107124837c4654
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 16:15:39 2015 +0100

    remove last nocommit

commit c5ece6d16d423fbdd36f5d789bd8daa5724d77b0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 16:13:12 2015 +0100

    simplify shadow engine

commit 45cd34a12a442080477da3ef14ab2fe7947ea97e
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:32:57 2015 +0100

    fix tests

commit 744f228c192602a6737051571e040731d413ba8b
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:28:12 2015 +0100

    revert changes to IndexShardGateway - these are leftovers from previous iterations

commit 11886b7653dabc23655ec76d112f291301f98f4a
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:26:48 2015 +0100

    Back out non-shared FS code. this will go in in a second iteration

commit 77fba571f150a0ca7fb340603669522c3ed65363
Merge: e8ad614 2e3c6a9
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:16:46 2015 +0100

    Merge branch 'master' into shadow-replicas

    Conflicts:
    	src/main/java/org/elasticsearch/index/engine/Engine.java

commit e8ad61467304e6d175257e389b8406d2a6cf8dba
Merge: 48a700d 1b8d8da
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 10:54:20 2015 +0100

    Merge branch 'master' into shadow-replicas

commit 48a700d23cff117b8e4851d4008364f92b8272a0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 10:50:59 2015 +0100

    add test for failing shadow engine / remove nocommit

commit d77414c5e7b2cde830a8e3f70fe463ccc904d4d0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 10:27:56 2015 +0100

    remove nocommits in IndexMetaData

commit abb696563a9e418d3f842a790fcb832f91150be2
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 17:05:02 2015 +0100

    remove nocommit and simplify delete logic

commit 82b9f0449108cd4741568d9b4495bf6c10a5b019
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 16:45:27 2015 +0100

    reduce the changes compared to master

commit 28f069b6d99a65e285ac8c821e6a332a1d8eb315
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 16:43:46 2015 +0100

    fix primary relocation

commit c4c999dd61a44a7a0db9798275a622f2b85b1039
Merge: 2ae80f9 455a85d
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 15:04:26 2015 +0100

    Merge branch 'master' into shadow-replicas

commit 2ae80f9689346f8fd346a0d3775a6341874d8bef
Author: Lee Hinman <lee@writequit.org>
Date:   Fri Feb 13 16:25:34 2015 -0700

    throw UnsupportedOperationException on write operations in ShadowEngine

commit 740c28dd9ef987bf56b670fa1a8bcc6de2845819
Merge: e5bc047 305ba33
Author: Lee Hinman <lee@writequit.org>
Date:   Fri Feb 13 15:38:39 2015 -0700

    Merge branch 'master' into shadow-replicas

commit e5bc047d7c872ae960d397b1ae7b4b78d6a1ea10
Author: Lee Hinman <lee@writequit.org>
Date:   Fri Feb 13 11:38:09 2015 -0700

    Don't replicate document request when using shadow replicas

commit 213292e0679d8ae1492ea11861178236f4abd8ea
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Feb 13 13:58:05 2015 +0100

    add one more nocommit

commit 83d171cf632f9b77cca9de58505f7db8fcda5599
Merge: aea9692 09eb8d1
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Feb 13 13:52:29 2015 +0100

    Merge branch 'master' into shadow-replicas

commit aea96920d995dacef294e48e719ba18f1ecf5860
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Feb 13 09:56:41 2015 +0100

    revert unneeded changes on Store

commit ea4e3e58dc6959a92c06d5990276268d586735f3
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 14:26:30 2015 -0700

    Add documentation to ShadowIndexShard, remove nocommit

commit 4f71c8d9f706a0c1c39aa3a370efb1604559d928
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 14:17:22 2015 -0700

    Add documentation to ShadowEngine

commit 28a9d1842722acba7ea69e0fa65200444532a30c
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 14:08:25 2015 -0700

    Remove nocommit, document canDeleteIndexContents

commit d8d59dbf6d0525cd823d97268d035820e5727ac9
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 10:34:32 2015 -0700

    Refactor more shared methods into the abstract Engine

commit a7eb53c1e8b8fbfd9281b43ae39eacbe3cd1a0a6
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 17:38:59 2015 +0100

    Simplify shared filesystem recovery by using a dedicated recovery handler that skip
    most phases and enforces shard closing on the soruce before the target opens it's engine

commit a62b9a70adad87d7492c526f4daf868cb05018d9
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 15:59:54 2015 +0100

    fix compile error after upstream changes

commit abda7807bc3328a89fd783ca7ad8c6deac35f16f
Merge: f229719 35f6496
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 15:57:28 2015 +0100

    Merge branch 'master' into shadow-replicas

    Conflicts:
    	src/main/java/org/elasticsearch/index/engine/Engine.java

commit f2297199b7dd5d3f9f1f109d0ddf3dd83390b0d1
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 12:41:32 2015 +0100

    first cut at catchup from primary
    make flush to a refresh
    factor our ShadowIndexShard to have IndexShard be idential to the master and least intrusive

    cleanup abstractions

commit 4a367c07505b84b452807a58890f1cbe21711f27
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 09:50:36 2015 +0100

    fix primary promotion

commit cf2fb807e7e243f1ad603a79bc9d5f31a499b769
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 16:45:41 2015 -0700

    Make assertPathHasBeenCleared recursive

commit 5689b7d2f84ca1c41e4459030af56cb9c0151eff
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:58:19 2015 -0700

    Add testShadowReplicaNaturalRelocation

commit fdbe4133537eaeb768747c2200cfc91878afeb97
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:28:57 2015 -0700

    Use check for shared filesystem in primary -> primary relocation

    Also adds a nocommit

commit 06e2eb4496762130af87ce68a47d360962091697
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:21:32 2015 -0700

    Add a test checking that indices with shadow replicas clean up after themselves

commit e4dbfb09a689b449f0edf6ee24222d7eaba2a215
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:08:18 2015 -0700

    Fix segment info for ShadowEngine, remove test nocommit

commit 80cf0e884c66eda7d59ac5d59235e1ce215af8f5
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 14:30:13 2015 -0700

    Remove nocommit in ShadowEngineTests#testFailStart()

commit 5e33eeaca971807b342f9be51a6a566eee005251
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 14:22:59 2015 -0700

    Remove overly-complex test

commit 2378fbb917b467e79c0262d7a41c23321bbeb147
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 13:45:44 2015 -0700

    Fix missing import

commit 52e9cd1b8334a5dd228d5d68bd03fd0040e9c8e9
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 13:45:05 2015 -0700

    Add a test for replica -> primary promotion

commit a95adbeded426d7f69f6ddc4cbd6712b6f6380b4
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 12:54:14 2015 -0700

    Remove tests that don't apply to ShadowEngine

commit 1896feda9de69e4f9cf774ef6748a5c50e953946
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 10:29:12 2015 -0700

    Add testShadowEngineIgnoresWriteOperations and testSearchResultRelease

commit 67d7df41eac5e10a1dd63ddb31de74e326e9d38b
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 10:06:05 2015 -0700

    Add start of ShadowEngine unit tests

commit ca9beb2d93d9b5af9aa6c75dbc0ead4ef57e220d
Merge: 2d42736 57a4646
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 18:03:53 2015 +0100

    Merge branch 'master' into shadow-replicas

commit 2d42736fed3ed8afda7e4aff10b65d292e1c6f92
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 17:51:22 2015 +0100

    shortcut recovery if we are on a shared FS - no need to compare files etc.

commit 24d36c92dd82adce650e7ac8e9f0b43c83b2dc53
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 17:08:08 2015 +0100

    utilize the new delete code

commit 2a2eed10f58825aae29ffe4cf01aefa5743a97c7
Merge: 343dc0b 173cfc1
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 16:07:41 2015 +0100

    Merge branch 'master' into shadow-replicas

    Conflicts:
    	src/main/java/org/elasticsearch/gateway/GatewayMetaState.java

commit 343dc0b527a7052acdc783ac5abcaad1ef78dbda
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 16:05:28 2015 +0100

    long adder is not available in java7

commit be02cabfeebaea74b51b212957a2a466cfbfb716
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 10 22:04:24 2015 -0700

    Add test that restarts nodes to ensure shadow replicas recover

commit 7fcb373f0617050ca1a5a577b8cf32e32dc612b0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 10 23:19:21 2015 +0100

    make test more evil

commit 38135af0c1991b88f168ece0efb72ffe9498ff59
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 10 22:25:11 2015 +0100

    make tests pass

commit 05975af69e6db63cb95f3e40d25bfa7174e006ea
Author: Lee Hinman <lee@writequit.org>
Date:   Mon Jan 12 18:44:29 2015 +0100

    Add ShadowEngine
2015-02-18 15:34:06 -07:00
Christoph Büscher 30fd70f07b Aggregations: Simplify time zone option in `date_histogram`
Removed the existing `pre_zone` and `post_zone` option in `date_histogram` in favor of
the simpler `time_zone` option. Previously, specifying different values for these could
lead to confusing scenarios where ES would return bucket keys that are not UTC.
Now `time_zone` is the only option setting, the calculation of date buckets to take place in the
preferred time zone, but after rounding converting the bucket key values back to UTC.

Closes #9062
Closes #9637
2015-02-16 16:54:06 +01:00
Blake Niemyjski 8cba6c3abb Fixed an invalid query
Closes #9682
2015-02-13 21:11:42 +01:00
Ryan Ernst 533fdbdf75 Mappings: Remove support for field access by short name
When multiple fields under object fields share the same name, accessing
by short name is ambiguous.  This removes support for short names,
always requiring the full name when used in queries.

closes #8872
2015-02-12 09:58:37 -08:00
Andreas Kohn 01b8479179 Allow configuration of the GC log file via an environment variable
Enabling GC logging works now by setting the environment variable ES_GC_LOG_FILE
to the full path to the GC log file. Missing directories will be created as needed.

The ES_USE_GC_LOGGING environment variable is no longer used.

Closes #8471
Closes #8479
2015-02-12 17:07:57 +01:00
gseng d1deb6bd1e Update update-settings.asciidoc
Updating to the fields as mentioned on http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-fielddata.html

Closes #9657
2015-02-12 13:09:16 +01:00
Clinton Gormley 856b0fa1a0 Docs: Fixed explanation of how the query string query is rewritten 2015-02-12 12:46:44 +01:00
Clinton Gormley 20ece4acb5 Update core-types.asciidoc
Provide an example of how to disable norms

Closes #9641
2015-02-12 12:10:11 +01:00
Ryan Ernst f735baf306 Core: Remove ability to run optimize and upgrade async
This has been very trappy. Rather than continue to allow buggy behavior
of having upgrade/optimize requests sidestep the single shard per node
limits optimize is supposed to be subject to, this removes
the ability to run the upgrade/optimize async.

closes #9638
2015-02-11 11:30:27 -08:00
Clinton Gormley faae98c5d8 Updated latest version in docs 2015-02-11 19:25:10 +01:00
Clinton Gormley 57a4646776 Docs: Added note about groovy sandbox vulnerability to modules/scripting 2015-02-11 17:54:53 +01:00
Clinton Gormley 6fadeeca56 Updated doc annotations for 1.4.3 2015-02-11 17:54:53 +01:00
Ryan Ernst b3474f6b25 Mappings: Remove ability to set path for _id and _routing on 2.0+ indexes
_id and _routing now no longer support the 'path' setting on indexes
created with 2.0.  Indexes created before 2.0 still support this
setting for backcompat.

closes #6730
2015-02-10 10:53:44 -08:00
Alfredo Serafini e607e53591 Update span-multi-term-query.asciidoc
added wildcard to the list of possible nested queries

Closes #9586
2015-02-09 16:01:46 +01:00
Christoph Büscher d2f852a274 Aggregations: Add 'offset' option to date_histogram, replacing 'pre_offset' and 'post_offset'
Add offset option to 'date_histogram' replacing and simplifying the previous 'pre_offset' and 'post_offset' options.
This change is part of a larger clean up task for `date_histogram` from issue #9062.
2015-02-09 14:03:28 +01:00
Christoph Büscher dfc0496fc0 Add warning to settings documentation about setting number_of_replicas on a closed index
Issue #9566 raises the point that setting the number of shards on a closed index can lead to this index not beeing able to open again. This change in documentation is ment to warn the user about this issue.
2015-02-06 12:09:24 +01:00
Ryan Ernst c6968883a7 Mappings: Remove support for new indexes using path setting in
object/nested fields or index_name in any field

Backcompat is still here for indexes created before 2.0.

closes #6677
2015-02-05 12:44:43 -08:00
Adrien Grand 95f46f1212 Docs: Use the new experimental annotation.
We now have a very useful annotation to mark features or parameters as
experimental. Let's use it! This commit replaces some custom text warnings with
this annotation and adds this annotation to some existing features/parameters:
 - inner_hits (unreleased yet)
 - terminate_after (released in 1.4)
 - per-bucket doc count errors in the terms agg (released in 1.4)

I also tagged with this annotation settings which should either be not needed
(like the ability to evict entries from the filter cache based on time) or that
are too deep into the way that Elasticsearch works like the Directory
implementation or merge settings.

Close #9563
2015-02-05 15:29:45 +01:00
Adrien Grand 3a486066fd Docs: Remove the experimental status of the cardinality and percentiles(-ranks) aggregations
These aggregations are not experimental anymore but some of their parameters
still are:
 - `precision_threshold` and `rehash` on `cardinality`
 - `compression` on percentiles(-ranks)

Close #9560
2015-02-05 15:18:40 +01:00
Masaru Hasegawa b4f7d26723 Fielddata: Change threshold value of fielddata.filter.frequency.max/min
Make it consider 1.0 as 100% instead of aboslute count 1.

Closes: #9327
2015-02-05 13:27:42 +09:00
Adam 928ea82188 Docs: Updated documentation for query-string-syntax to include '>' '<' and '=' as reserved characters
Closes #9518
2015-02-04 17:55:15 +01:00
Simon Willnauer 0c5599e1d1 [ENGINE] Remove full flush / FlushType.NEW_WRITER
The `full` option and `FlushType.NEW_WRITER` only exists to allow
realtime changes to two settings (`index.codec` and `index.concurrency`).
Those settings are very expert and don't really need to be updateable
in realtime.
2015-02-04 17:38:05 +01:00
Robert Muir 027730006b core: add 'checksum' option for index.shard.check_on_startup
The current "checkindex" on startup is very very expensive. This is
like running one of the old school hard drive diagnostic checkers and
usually not a good idea.

But we can do a CRC32 verification of files. We don't even need to
open an indexreader to do this, its much more lightweight.

This option (as well as the existing true/false) are randomized in
tests to find problems.

Also fix bug where use of the current option would always leak
an indexwriter lock.

Closes #9183
2015-02-03 00:10:08 -05:00
Ryan Ernst 6079d88d43 Mappings: Remove type prefix support from field names in queries
This is the first part of #8872.
2015-02-02 13:10:56 -08:00
Christoph Büscher 44193e7ba5 Aggregations: Add 'offset' option to histogram aggregation
Histogram aggregation supports an 'offset' option to move bucket boundaries.
In a histogram with buckets of size X these can be moved from 0, X, 2X, 3X,...
by an offset value of Y to Y, X+Y, 2X+Y, 3X+Y... by using the 'offset' option.
The previous 'pre_offset' and 'post_offset' options are removed in favour of
the simplified 'offset' option.

Closes #9417
Closes #9505
2015-02-02 18:23:01 +01:00
Clinton Gormley eea22d7731 Docs: Fixed asciidoc error in snapshots.asciidoc 2015-01-29 20:57:12 +01:00
J Charitopoulos be8d8d658c Docs: minor syntax
Closes #9481
2015-01-29 20:27:20 +01:00
Glen Smith 3d5fbfb997 Docs: Update pattern-replace-charfilter.asciidoc
Remove invalid trailing comma from json

Closes #9477
2015-01-29 20:24:08 +01:00
David Pilato 878e46d7f9 [Docs] fix missing space 2015-01-29 19:17:41 +01:00
Oliver e412dab63a Docs: Fix sample query
Closes #9472
2015-01-29 15:56:24 +01:00
Ryan Ernst afcedb94ed Mappings: Remove `index_analyzer` setting to simplify analyzer logic
The `analyzer` setting is now the base setting, and `search_analyzer`
is simply an override of the search time analyzer.  When setting
`search_analyzer`, `analyzer` must be set.

closes #9371
2015-01-28 13:43:15 -08:00
Zachary Tong a4eb1d5505 Aggregations: Add standard deviation bounds to extended_stats
Extended_stats now displays the upper and lower bounds on standard deviations (e.g. avg +/- std).
Default is to show 2 std above/below, but can be changed using the `sigma` parameter.
Accepts non-negative doubles

Closes #9356
2015-01-28 11:47:20 -05:00
J Charitopoulos b359520849 Docs: Update snapshots.asciidoc
minor syntax

Closes #9457
2015-01-28 15:54:13 +01:00
Clinton Gormley 8978aa5465 Docs: Improved the template query docs
Added the `file` and `id` parameters.

Closes #9458
2015-01-28 14:19:59 +01:00
Lee Hinman 2f6527f491 [DOCS] Update documentation for `max_token_length`
In 1.4 the behavior is different due to
https://issues.apache.org/jira/browse/LUCENE-5897
2015-01-27 13:52:14 -07:00
Colin Goodheart-Smithe 285ef0f06d Aggregations: Clean up response API for Aggregations
This change makes the response API object for Histogram Aggregations the same for all types of Histogram, and does the same for all types of Ranges.
The change removes getBucketByKey() from all aggregations except filters and terms. It also reduces the methods on the Bucket class to just getKey() and getKeyAsString().
The getKey() method returns Object and the actual Type is returns will be appropriate for the type of aggregation being run. e.g. date_histogram will return a DateTime for this method and Histogram will return a Number.
2015-01-27 10:53:44 +00:00
Christian Verkerk 5b31189498 Docs: Update cluster.asciidoc
Clarify the preferencing.

Closes #9434
2015-01-27 10:48:40 +01:00
Ryan Ernst 385c43c141 Mappings: Remove _analyzer
closes #9279
2015-01-26 09:14:17 -08:00
jhtimmins 4aba382358 Docs: Change "There are few concepts" to "There are a few concepts"
Closes #8888
2015-01-21 10:33:33 +01:00
Igor Motov c0da353ef5 Snapshot/Restore: add support for changing index settings during restore process
Closes #7887
2015-01-20 15:49:47 -05:00
Alex Ksikes 615513ee9b Docs: clearer MLT documentation
Closes #9351
2015-01-20 16:42:39 +01:00
David Pilato fb10346953 [Mapper] Add `ignore_missing` option to `timestamp`
Related to #9049.

By default, the default value for `timestamp` is `now` which means the date the document was processed by the indexing chain.

You can now reject documents which not provide a `timestamp` value by setting `ignore_missing` to false (default to `true`):

```js
{
    "tweet" : {
        "_timestamp" : {
            "enabled" : true,
            "ignore_missing" : false
        }
    }
}
```

When you update the cluster to 1.5 or master, this index created with 1.4 we automatically migrate an index created with 1.4 to the 1.5 syntax.

Let say you have defined this in elasticsearch 1.4.x:

```js
DELETE test
PUT test
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  }
}
PUT test/type/_mapping
{
  "type" : {
      "_timestamp" : {
          "enabled" : true,
          "default" : null
      }
  }
}
```

After migration, the mapping become:

```js
{
   "test": {
      "mappings": {
         "type": {
            "_timestamp": {
               "enabled": true,
               "store": false,
               "ignore_missing": false
            },
            "properties": {}
         }
      }
   }
}
```

Closes #8882.
2015-01-20 13:20:05 +01:00
Michael McCandless 3c0d2081cf Core: change default xlog size from 200 MB to 512 MB
Closes #9341
2015-01-19 15:52:29 -05:00
eBuildy 85ef44fd73 Docs: Fix missing comma and boolean true
Closes #9350
2015-01-19 21:31:29 +01:00
Martijn van Groningen 8e0292b1aa docs: fix inner hits snippet 2015-01-19 18:56:45 +01:00
sweetest eaa1674d6d Introduce index option named 'index.percolator.map_unmapped_fields_as_string', that handles unmapped fields in percolator queries as type string.
Closes #9053
Closes #9054
2015-01-19 09:51:10 +01:00
Michael McCandless b9358ccca8 Core: switch to auto IO throttle for merges
This adds a new boolean (index.merge.scheduler.auto_throttle) dynamic
setting, default true (matching Lucene), to adaptively set the IO rate
limit for merges over time.

This is more flexible than the previous fixed rate throttling because
it responds depending on the incoming merge rate, so search-heavy
applications that are not doing much indexing will see merges heavily
throttled while indexing-heavy cases will lighten the throttle so
merges can keep up within incoming indexing.

The fixed rate throttling is still available as a fallback if things
go horribly wrong.

Closes #9243

Closes #9133
2015-01-16 13:00:08 -05:00
Clinton Gormley c644c377ab Update api-conventions.asciidoc
Corrected explanation of fuzzy AUTO

Related to #9278
2015-01-16 14:26:50 +01:00
Clinton Gormley f5b91c374a Update upgrade.asciidoc
Upgrade request needs pretty and human for the demonstrated output.

Closes #9313
2015-01-16 13:55:22 +01:00
David Haney 395960feef Docs: Updated standard token filter docs to indicate true behavior: doing nothing
Closes #9300
2015-01-15 21:33:29 +01:00
Michael McCandless def2d34f80 don't mention fixed throttling in the docs 2015-01-14 10:13:10 -05:00
Michael McCandless 107099affa put back fixed throttling, but off by default 2015-01-14 05:35:09 -05:00
Paul Echeverri 4f938ad37e Updates the command to add the repo to not use add-apt-repository, which automatically adds a non-working deb-src line to sources.list. Command now uses echo to write the correct line to sources.list instead.
Fixes #9261
2015-01-12 21:18:00 +00:00
Tomoya Hirano 15d46988dc Fix typo in sample json
Fixes #9253
2015-01-12 15:58:16 +00:00
David Pilato 052645903a Rest: remove status code from main action
Today we give the HTTP status back within the HTTP response itself and within the JSON response as well:

```sh
curl localhost:9200/
```

```js
{
  "status" : 200,
  "name" : "Red Wolf",
  "version" : {
    "number" : "2.0.0",
    "build_hash" : "6837a61d8a646a2ac7dc8da1ab3c4ab85d60882d",
    "build_timestamp" : "2014-08-19T13:55:56Z",
    "build_snapshot" : true,
    "lucene_version" : "4.9"
  },
  "tagline" : "You Know, for Search"
}
```
2015-01-12 12:37:46 +01:00
David Pilato fc7a0d3a4a [Docs] fix three to four 2015-01-12 12:13:23 +01:00
Michael McCandless 1aad275c55 expose current CMS throttle in merge stats; fix tests, docs; also log per-merge stop/throttle/rate 2015-01-11 05:52:43 -05:00
Michael McCandless 31e6acf3f2 first cut 2015-01-10 16:38:56 -05:00
Christoph Büscher 04cb09f44c [TEST] Add missing docs and tests for '_cat/segments'
The '_cat/segments' api was missing docs and a rest test which are added here.

Closes #5856
2015-01-09 12:29:11 +01:00
Ryan Ernst 060f963a8e Mappings: Remove allow_type_wrapper setting
Before Elasticsearch 1.0, the type was allowed to be passed as the root
element when uploading a document.  However, this was ambiguous if the
mappings also contained a field with the same name as the type.  The
behavior was changed in 1.0 to not allow this, but a setting was added
for backwards compatibility.  This change removes the setting for 2.0.
2015-01-08 09:13:40 -08:00
Martijn van Groningen ca4f27f40e Core: Added `_shards` header to all write responses.
The header indicates to how many shard copies (primary and replicas shards) a write was supposed to go to, to how many
shard copies to write succeeded and potentially captures shard failures if writing into a replica shard fails.

For async writes it also includes the number of shards a write is still pending.

Closes #7994
2015-01-08 18:10:08 +01:00
Martijn van Groningen dedaf9387e Core: Also check if indices resolved via aliases resolution aren't closed and deal with this according to IndicesOptions.
Closes #9057
2015-01-08 16:45:34 +01:00
Martijn van Groningen 20f7be378b Removed parent parameter from update request, because it is just sets the routing.
The routing option should be used instead. The parent a child document points to can't be updated.

Closes #4538
2015-01-07 10:26:20 +01:00
Ryan Ernst f7f99b8dbf Stats: Added verbose option to segments api, with full ram tree as first
additional element per segment.

This commit adds a verbose flag to the _segments api.  Currently the
only additional information returned when set to true is the full
ram tree from lucene for each segment.
2015-01-06 10:04:52 -08:00
Adrien Grand bc86796592 Core: Remove terms filter cache.
This is our only cache which is not 'exact' and might allow for stalled results.
Additionally, a similar cache that we have and needs to perform lookups in other
indices in order to run queries is the script index, and for this index we rely
on the filesystem cache, so we should probably do the same with terms filters
lookups.

Close #9056
2015-01-06 17:21:20 +01:00
Simon Willnauer 236e2491b4 [ALLOCATION] Remove primary balance factor
The `cluster.routing.allocation.balance.primary` setting has caused
a lot of confusion in the past while it has very little benefit form a
shard allocatioon point of view. Users tend to modify this value to
evently distribute primaries across the nodes which is dangerous since
a prmiary flag on it's own can trigger relocations. The primary flag for a shard
is should not have any impact on cluster performance unless the high level feature
suffereing from primary hotspots is buggy. Yet, this setting was intended to be a
tie-breaker which is not necessary anymore since the algorithm is deterministic.

This commit removes this setting entriely.
2015-01-06 16:43:39 +01:00
Simon Willnauer 4900f52619 [ALLOCATION] Weight deltas must be absolute deltas
In some situations the shard balanceing weight delta becomes negative. Yet,
a negative delta is always treated as `well balanced` which is wrong. I wasn't
able to reproduce the issue in any way other than useing the real world data
from issue #9023. This commit adds a fix for absolute deltas as well as a base
test class that allows to build tests or simulations from the cat API output.

Closes #9023
2015-01-06 15:48:44 +01:00
Clinton Gormley 75cc7077c7 Update plugins.asciidoc
Added entity resolution plugin for duplication detection

Related to #9131
2015-01-05 12:53:37 +01:00
Mikhail Korobov 707025fb7a [Docs] fix curl examples in Nodes Stats docs
Closes #9118
2014-12-31 14:01:37 +01:00
Clinton Gormley f83909f7ae Docs: The regexp query defaults to the `ALL` flag, and removed
the `AUTOMATON` flag which is not used in Elasticsearch.

Closes #6180
2014-12-30 19:53:31 +01:00
Clinton Gormley 904f20a41b Update setup.asciidoc
Add a note about using the same JVM version on all nodes and clients
2014-12-30 17:40:51 +01:00
dtpeacock 582d5e8d3c Doc has store "false" not store "true"
Came from 3465e69e83 due to changing "yes" to "false".

Closes #9075
2014-12-29 11:59:22 +01:00
Martijn van Groningen d8054ec299 inner_hits: Added another more compact syntax for inner hits.
Closes #8770
2014-12-24 17:41:35 +01:00
Ryan Ernst 39b3613420 Fix date histogram docs grammar. 2014-12-23 10:19:55 -08:00
Nicholas Knize 77a7ef28b3 [GEO] Add optional left/right parameter to GeoJSON
This feature adds an optional orientation parameter to the GeoJSON document and geo_shape mapping enabling users to explicitly define how they want Elasticsearch to interpret vertex ordering.  The default uses the right-hand rule (counterclockwise for outer ring, clockwise for inner ring) complying with OGC Simple Feature Access standards. The parameter can be explicitly specified for an entire index using the geo_shape mapping by adding "orientation":{"left"|"right"|"cw"|"ccw"|"clockwise"|"counterclockwise"} and/or overridden on each insert by adding the same parameter to the GeoJSON document.

closes #8764
2014-12-22 12:09:45 -06:00
Adrien Grand fb6c3b7c29 [Docs] Improve documentation of the new caching policy for filters. 2014-12-22 17:14:47 +01:00
Adrien Grand ce11e0ee6d Filter cache: add a `_cache: auto` option and make it the default.
Up to now, all filters could be cached using the `_cache` flag that could be
set to `true` or `false` and the default was set depending on the type of the
`filter`. For instance, `script` filters are not cached by default while
`terms` are. For some filters, the default is more complicated and eg. date
range filters are cached unless they use `now` in a non-rounded fashion.

This commit adds a 3rd option called `auto`, which becomes the default for
all filters. So for all filters a cache wrapper will be returned, and the
decision will be made at caching time, per-segment. Here is the default logic:
 - if there is already a cache entry for this filter in the current segment,
   then return the cache entry.
 - else if the doc id set cannot iterate (eg. script filter) then do not cache.
 - else if the doc id set is already cacheable and it has been used twice or
   more in the last 1000 filters then cache it.
 - else if the filter is costly (eg. multi-term) and has been used twice or more
   in the last 1000 filters then cache it.
 - else if the doc id set is not cacheable and it has been used 5 times or more
   in the last 1000 filters, then load it into a cacheable set and cache it.
 - else return the uncached set.

So for instance geo-distance filters and script filters are going to use this
new default and are not going to be cached because of their iterators.

Similarly, date range filters are going to use this default all the time, but
it is very unlikely that those that use `now` in a not rounded fashion will get
reused so in practice they won't be cached.

`terms`, `range`, ... filters produce cacheable doc id sets with good iterators
so they will be cached as soon as they have been used twice.

Filters that don't produce cacheable doc id sets such as the `term` filter will
need to be used 5 times before being cached. This ensures that we don't spend
CPU iterating over all documents matching such filters unless we have good
evidence of reuse.

One last interesting point about this change is that it also applies to compound
filters. So if you keep on repeating the same `bool` filter with the same
underlying clauses, it will be cached on its own while up to now it used to
never be cached by default.

`_cache: true` has been changed to only cache on large segments, in order to not
pollute the cache since small segments should not be the bottleneck anyway.
However `_cache: false` still has the same semantics.

Close #8449
2014-12-18 15:51:36 +01:00
Michael McCandless 242e631e95 Core: ignore known idle threads by default in /_nodes/hot_threads
Add a new ignore_idle_threads boolean option (default true) to
/_nodes/hot_threads, to filter out threads in known idle places like
waiting on a socket select or on pulling the next task from an empty
queue.

Closes #8985

Closes #8908
2014-12-17 11:59:31 -05:00
Yasir Bamarni 5059d6fe1c Update percolate.asciidoc
wrong type used in the -GET request

Closes #8942
2014-12-17 14:05:27 +01:00
Pablo Díaz-López adb1a5b43b Update getting-started.asciidoc
Missing -X flag at the curl template

Closes #8977
2014-12-17 14:03:38 +01:00
Peter Johnson a.k.a. insertcoffee 4b5e6b2de0 [docs] pedantry
Closes #8982
2014-12-17 13:46:39 +01:00
Nicholas Knize ac0e37449e Adding unit test for self intersecting polygons. Relevant to #7751 even/odd discussion
Updating documentation to describe polygon ambiguity and vertex ordering.
2014-12-16 10:54:39 -06:00
Ryan Ernst 37287284e6 Settings: Remove `mapping.date.round_ceil` setting for date math parsing
The setting `mapping.date.round_ceil` (and the undocumented setting
`index.mapping.date.parse_upper_inclusive`) affect how date ranges using
`lte` are parsed.  In #8556 the semantics of date rounding were
solidified, eliminating the need to have different parsing functions
whether the date is inclusive or exclusive.

This change removes these legacy settings and improves the tests
for the date math parser (now at 100% coverage!). It also removes the
unnecessary function `DateMathParser.parseTimeZone` for which
the existing `DateTimeZone.forID` handles all use cases.

Any user previously using these settings can refer to the changed
semantics and change their query accordingly. This is a breaking change
because even dates without datemath previously used the different
parsing functions depending on context.

closes #8598
closes #8889
2014-12-15 13:13:45 -08:00
Timothy Perisho ceafde41e9 Docs: typo on "frequent"
I replaced "high frequent terms" with "high frequency terms" and "low frequent terms" with "low frequency terms".

Alternatively, we could write, "highly frequent terms" and "minimally frequent terms" (or just "rare terms").

Closes #8962
2014-12-15 19:59:50 +01:00
Clinton Gormley fcb83055de Update repositories.asciidoc
Update formatting of PGP key
2014-12-15 18:04:17 +01:00
Simon Willnauer 1247774ff1 Remove Gateway abstraction
We only have a single gatweway since es 1.3. There is no need to keep all
these abstractsion and nested packages. We can fold most of it into simpler
structures.
2014-12-15 15:53:02 +01:00
spapin ad747ba67f Docs: fix a typo in cluster stats documentation example
Closes #8898
2014-12-15 14:14:38 +01:00
Ayush 23dbecf3e7 Update percolate.asciidoc
Updating the `associated` spelling

Closes #8907
2014-12-15 14:12:03 +01:00
Alexander Reelsen 544ef8cb17 Packaging: Add java7/8 java-package paths to debian init script
If you use the java-package tool to create java packages, those
paths also should be added to the debian init script.

Also updated the docs, that it is ok to install java8.

Closes #7383
2014-12-11 16:15:00 +01:00
Peter Fabian Mitchell b2bab05c29 HTTP: Add 'http.publish_port' setting to the HTTP module
This change adds a 'http.publish_port' setting to the HTTP module to configure
the port which HTTP clients should use when communicating with the node. This
is useful when running on a bridged network interface or when running behind
a proxy or firewall.

Closes #8807
Closes #8137
2014-12-11 16:10:07 +01:00
Robert Muir a2ffe494ae [core] add best_compression option for Lucene 5.0
Upgrades lucene to latest, and supports the BEST_COMPRESSION parameter
now supported (with backwards compatibility, etc) in Lucene.
This option uses deflate, tuned for highly compressible data.

index.codec::
The default value compresses stored data with LZ4 compression, but
this can be set to best_compression for a higher compression ratio,
at the expense of slower stored fields performance.

IMO its safest to implement as a named codec here, because ES already
has logic to handle this correctly, and because its unrealistic to have
a plethora of options to Lucene's default codec... we are practically
limited in Lucene to what we can support with back compat, so I don't
think we should overengineer this and add additional unnecessary plumbing.

See also:
https://issues.apache.org/jira/browse/LUCENE-5914
https://issues.apache.org/jira/browse/LUCENE-6089
https://issues.apache.org/jira/browse/LUCENE-6090
https://issues.apache.org/jira/browse/LUCENE-6100

Closes #8863
2014-12-10 22:13:09 -05:00
Alexander Clausen 633905161a Docs: use https to download the gpg public key
Closes #8818
2014-12-10 18:14:07 +01:00
Adam Menges 3a3030e217 Docs: Fix the wording for inner hits a bit
Closes #8747
2014-12-09 13:36:26 +01:00
Ashraf Sarhan 24f8807cb5 Docs: Update repositories.asciidoc
1. Enable the repository using "add-apt-repository" to avoid this error "No command 'deb' found".
2. Adding "sudo" to update and install command.

Closes #8691
2014-12-09 13:23:16 +01:00
Kevin Kluge 63ac4614f4 docs: add pgp key to repositories page 2014-12-08 15:41:09 +01:00
Jun Ohtani d78d2ff93d Docs: add randomizedtesting-runner to testing-framework.asciidoc
Close #8450
2014-12-07 01:30:58 +09:00
Adrien Grand 344bbf2ced Docs: Add instructions to start elasticsearch on bootup on RHEL/Fedora. 2014-12-05 11:14:13 +01:00
tristanbob 0a09f1ea13 Docs: Added a command to start elasticsearch on bootup on Debian.
Close #8600
2014-12-05 11:03:32 +01:00
David Pilato d2a2d1bb53 java: QueryBuilders cleanup: remove deprecated
Related to #8667:

Some QueryBuilders have been deprecated in 1.x branches. We removed them in 2.0.

Removed
-------

* `textPhrase(...)`
* `textPhrasePrefix(...)`
* `textPhrasePrefixQuery(...)`
* `filtered(...)`
* `inQuery(...)`
* `commonTerms(...)`
* `queryString(...)`
* `simpleQueryString(...)`

Closes #8721.
2014-12-03 16:07:34 +01:00
Peter Johnson a.k.a. insertcoffee ac71f1b70a [docs] formatting and general pedantry
I'm not sure if the `distance-units` section is totally clear, when using the 'Geohash Cell Filter' and omitting a unit, the default is to interpret the integer as the 'length of the geohash prefix', not to default it to 'meter'. Maybe I'm being pedantic.

Closes #8744
2014-12-02 19:23:48 +01:00
John Michael Luy 01ef80a33d Update range-filter.asciidoc
Closes #8741
2014-12-02 18:00:38 +01:00
John Michael Luy f20f6ffe22 Docs: Update range-query.asciidoc
Closes #8740
2014-12-02 12:55:44 +01:00
Martijn van Groningen d7e224da04 Added `inner_hits` feature that allows to include nested hits.
Inner hits allows to embed nested inner objects, children documents or the parent document that contributed to the matching of the returned search hit as inner hits, which would otherwise be hidden.

Closes #8153
Closes #3022
Closes #3152
2014-12-02 12:01:01 +01:00
Itamar Syn-Hershko cb042cd662 Fixing typo
Closes #8713
2014-12-01 10:52:00 +01:00
Dan Tuffery 3b5fa9075a Docs: Grammar correction
Closes #8702
2014-11-29 14:06:04 +01:00
Clinton Gormley 88e06cba80 Update daterange-aggregation.asciidoc
Clarified the date-math expressions on date range aggregations

Closes #8703
2014-11-28 16:53:33 +01:00
Alex Ksikes 256712640f MLT Query: Support for ignore docs
Adds a `ignore_like` parameter to the MLT Query, which simply tells the
algorithm to skip all the terms from the given documents. This could be useful
in order to better guide nearest neighbor search by telling the algorithm to
never explore the space spanned by the given `ignore_like` docs. In essence we
are interested about the characteristic of a given item, but not of the ones
provided by `ignore_like`, thereby forcing the algorithm to go deeper in its
selection of terms. Note that this is different than simply performing a must
not boolean query on the unliked items. The syntax is exactly the same as the
`like` parameter.

Closes #8674
2014-11-28 14:48:43 +01:00
pmamat 9e2eaeece4 Docs: Additional info about _score calculation
Description taken from http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/multi-query-strings.html / 110_Multi_Field_Search/05_Multiple_query_strings.asciidoc

Closes #8635
2014-11-28 13:54:45 +01:00
Britta Weber 59507cf793 function_score: match only document with score above custom score threshold
functon_score matched each document regardless of the computed score.
This commit adds a query parameter `min_score` (-Float.MAX_VALUE default).
Documents that have a score lower than this threshold will not be mached.

closes #6952
2014-11-28 12:35:26 +01:00
David Pilato 43a1435d3b [Docs] fix consistency between examples 2014-11-27 20:29:34 +01:00
David Pilato 40f0e07db3 [Docs] Fix missing new line 2014-11-27 19:39:12 +01:00
Britta Weber f00b431c18 [docs] explain default settings for parameters of decay functions
relates to #8624
2014-11-27 19:18:55 +01:00
David Pilato da27c2104a [Docs] Fix missing comma in mapping 2014-11-27 11:03:19 +01:00
Clinton Gormley 818b9b7563 Updated docs to use v1.4.1 as current 2014-11-26 17:18:37 +01:00
Sebastian Ziebell 3a6c6f4b26 Docs: Adds documentation for indices.exists_template
Closes: #8657
2014-11-25 19:36:01 +01:00
tristanbob 807f363d6d Added note that ES packages automatically change vm.max_map_count
Closes #8601
2014-11-25 18:25:46 +01:00
Matt Hughes afba977e80 Docs: Added swift openstack repository
Closes #8583
2014-11-25 13:49:15 +01:00
David Haney 2c429452e9 Typo: changed "5% or the real words" to "5% of the real words"
Closes #8582
2014-11-25 13:15:33 +01:00
Michael McCandless 856b294441 Core: let Lucene kick off merges
Today, Elasticsearch has a separate merge thread pool checking once
per second (by default) if any merges are necessary, but this is no
longer necessary since we can and do now tell Lucene's
ConcurrentMergeScheduler never to "hard pause" threads when merges
fall behind, since we do our own index throttling.

This change goes back to letting Lucene launch merges as needed, and
removes these two expert settings:

  index.merge.force_async_merge
  index.merge.async_interval

Now merges kick off immediately instead of waiting up to 1 second
before running.

Closes #8643
2014-11-25 04:13:57 -05:00
Martijn van Groningen 1d7cdd7d22 Applied PR, changed the way defaults are handled and updated the docs.
Closes #4452
2014-11-24 13:32:41 +01:00
Lee Hinman 45408844e7 Remove NoneGateway, NoneGatewayAllocator, & NoneGatewayModule
Always use the LocalGateway* equivalents

We already check in the LocalGateway whether a node is a client node, or
is not master-eligible, and skip writing the state there. This allows us
to remove this code that was previously used only for tribe nodes (which
are not master eligible anyway and wouldn't write state) and in
tests (which can shake more bugs out)
2014-11-24 12:22:05 +01:00
dw ad408eee85 Docs: Reword note regarding _source for accuracy
Previously it suggested _source was always present, when that is not the case.

Closes #8491
2014-11-24 12:19:44 +01:00
Laurent Broudoux feb465f26f Docs: Update plugins.asciidoc on river plugins section
Adding links to Amazon S3 and Google Drive river plugins

Closes #8544
2014-11-24 12:15:12 +01:00
Michael McCandless dfb6d6081c Core: upgrade to current Lucene 5.0.0 snapshot
Elasticsearch no longer unlocks the Lucene index on startup (this was
dangerous, and could possibly lead to corruption).

Added the new serbian_normalization TokenFilter from Lucene.

NoLockFactory is no longer supported (index.store.fs.fs_lock = none),
and if you have a typo in your fs_lock you'll now hit a StoreException
instead of silently using NoLockFactory.

Closes #8588
2014-11-24 05:08:42 -05:00
Adrien Grand 8346e92ebb Core: Fix script fields to be returned as a multivalued field when they produce a list.
This change is essentially the same as #3015 but on script fields.

Close #8592
2014-11-24 09:41:16 +01:00
mdzor bc52ccfd33 Docs: Update update-settings.asciidoc
Inconsistent indentation

Closes #8525
2014-11-23 14:45:56 +01:00
barbasa fd6c41bfbf Missing quote in the example 2014-11-23 14:03:58 +01:00
Alban Perillat-Merceroz 54466938da Fix error in documentation
Indexation does not fail if no timestamp provided when there is a default value defined in mapping.
2014-11-23 14:02:51 +01:00
dw bb81055c33 Docs: Remove reference to imaginary "no_docs_query"
No reference to it in the source code except this file.

Closes #8566
2014-11-23 13:56:33 +01:00
Mariam Hakobyan 4a1ab6543c Docs: Added new elasticsearch-river-kafka plugin to the documentation (uses latest version of Kafka, EL Bulk API, and supports concurrent requests)
Closes #8518
2014-11-23 12:28:45 +01:00
Alex Ksikes 1959275622 Term Vectors: More consistent naming for term vector[s]
We speak of the term vectors of a document, where each field has an associated
stored term vector. Since by default we are requesting all the term vectors of
a document, the HTTP request endpoint should rather be called `_termvectors`
instead of `_termvector`. The usage of `_termvector` is now deprecated, as
well as the transport client call to termVector and prepareTermVector.

Closes #8484
2014-11-21 14:06:44 +01:00
Robert Muir 9ef69f9f36 Disable bloom filters.
make the "es090" postings format read-only, just to support old segments. There is a test version that subclasses it with write-capability for testing.

Closes #8571
2014-11-20 21:03:23 -05:00
Simon Willnauer 0fcb466555 [STORE] Remove `memory`/ `ram` store
The RAM store is discuraged for production usage anyway and
we don't test it in our randomized infrastructure. This commit
removes it for `2.0`
2014-11-20 14:47:19 +01:00
javanna 06fafa3ed9 [DOCS] document that we support loading multiple logging conf files 2014-11-19 11:34:57 +01:00
Boaz Leskes 1e16375d04 Docs: Update execution hint docs for Significant terms agg
copied over the relevant pieces from the terms agg

Closes #8532
2014-11-18 20:54:26 +01:00
Olivier Favre 4d68d3d053 Provide more context variables in update scripts
In addition to `_source`, the following variables are available through
the `ctx` map: `_index`, `_type`, `_id`, `_version`, `_routing`,
`_parent`, `_timestamp`, `_ttl`.

Some of these fields are more useful still within the context of an
Update By Query, see #1607, #2230, #2231.
2014-11-14 10:14:39 +01:00
Clinton Gormley 32fc657d71 Docs: Fixed a bad ref to docs-bulk-udp which no longer exists in master 2014-11-13 14:34:49 +01:00
Colin Goodheart-Smithe 353574d6af Indices API: Fix GET index API always running all features
Previous to this change all features (_alias,_mapping,_settings,_warmer) are run regardless of which features are actually requested. This change fixes the request object to resolve this bug
2014-11-13 13:22:46 +00:00
Clinton Gormley 6b05b229af Docs: Changed breaking docs in master to correspond with 1.x for easier merging 2014-11-13 13:50:57 +01:00
Colin Goodheart-Smithe 34b37ab7f0 [DOCS] Added documentation for log4j-extras dependency 2014-11-13 12:40:14 +00:00
javanna c1428b5964 [DOCS] Expand logging documentation
Updated log4j link so it doesn't point to log4j 2.0 but version 1.2. Clarified which formats are supported and briefly explained what loggers and appenders are, plus added a link to the log4j docs.

Closes #5305
Closes #8455
2014-11-13 11:08:10 +01:00
Joel Taddei 7e72800c83 [DOCS] Corrected syntax error in search curl cmd
Closes #8447
2014-11-12 17:21:19 +01:00
Mark Walkom bfd1bcd30a Updated threadpool documentation to elaborate/clarify what the pools are for and their values
Closes #8446
2014-11-12 22:33:38 +11:00
Israel Tsadok 7590629531 Docs: note about confusing disk threshold settings 2014-11-12 09:24:03 +01:00
Martijn van Groningen 94c1a7dabe Docs: Fix incorrect documentation for the `index.query.parse.allow_unmapped_fields` setting.
The `index.query.parse.allow_unmapped_fields` setting can't influence whether unmapped fields are allowed
in alias filters and percolator queries.
2014-11-11 15:13:55 +00:00
Michael McCandless 8aebb9656b Core: add max_determinized_states to query_string and regexp query/filter
This prevents too-difficult regular expressions from consuming
excessive RAM/CPU; the default max_determinized_states is 10,000 (same
as Lucene) but query_string and regepx query/filter can override
per-request.

The also upgrades to a new Lucene 5.0.0 snapshot.

Closes #8386

Closes #8357
2014-11-10 13:43:48 -05:00
Clinton Gormley cff544dcc2 Docs: Removed old coming/added tags 2014-11-10 14:41:24 +01:00
Britta Weber c5a4c1d6b4 [docs] add 2d vis for decay functions and parameters
closes #8420
2014-11-10 10:56:41 +01:00
Veres Lajos 4059e4ac86 typo fixes - https://github.com/vlajos/misspell_fixer
Closes #8323
2014-11-08 18:55:57 +01:00
Clinton Gormley 08aa715d2e Update datehistogram-aggregation.asciidoc
Clarified use of fractional time units in the date histo agg.

Closes #7957
2014-11-08 17:49:34 +01:00
Clinton Gormley b9149f836b Docs: Improve the exists/missing filters documentation
Closes #7274
2014-11-08 16:57:41 +01:00
Clinton Gormley f5ad699284 Update multi-get.asciidoc
Documented that the fields parameter can be passed in the query string.

Closes #4006
2014-11-08 13:55:23 +01:00
Kevin Kluge c473976e31 [docs] fix typo in getting-started
Closes #8354
2014-11-06 10:57:56 +01:00
Robert Muir 610ce078fb Upgrade master to lucene 5.0 snapshot
This has a lot of improvements in lucene, particularly around memory usage, merging, safety, compressed bitsets, etc.

On the elasticsearch side, summary of the larger changes:

    API changes: postings API became a "pull" rather than "push", collector API became per-segment, etc.
    packaging changes: add lucene-backwards-codecs.jar as a dependency.
    improvements to boolean filtering: especially ensuring it will not be slow for SparseBitSet.
    use generic BitSet api in plumbing so that concrete bitset type is an implementation detail.
    use generic BitDocIdSetFilter api for dedicated bitset cache, so there is type safety.
    changes to support atomic commits
    implement Accountable.getChildResources (detailed memory usage API) for fielddata, etc
    change handling of IndexFormatTooOld/New, since they no longer extends CorruptIndexException

Closes #8347.

Squashed commit of the following:

commit d90d53f5f21b876efc1e09cbd6d63c538a16cd89
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Nov 5 21:35:28 2014 +0100

    Make default codec/postings/docvalues format constants

commit cb66c22c71cd304a36e7371b199a8c279908ae37
Merge: d4e2f6d ad4ff43
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Nov 5 11:41:13 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit d4e2f6dfe767a5128c9b9ae9e75036378de08f47
Merge: 4e5445c 4111d93
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Nov 5 06:26:32 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 4e5445c775f580730eb01360244e9330c0dc3958
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 16:19:19 2014 -0500

    FixedBitSet -> BitSet

commit 9887ea73e8b857eeda7f851ef3722ef580c92acf
Merge: 1bf8894 fc84666
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 15:26:25 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 1bf8894430de3e566d0dc5623b0cc28b0d674ebb
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 15:22:51 2014 -0500

    remove nocommit

commit a9c2a2259ff79c69bae7806b64e92d5f472c18c8
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 13:48:43 2014 -0500

    turn jenkins red again

commit 067baaaa4d52fce772c81654dcdb5051ea79139f
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 13:18:21 2014 -0500

    unzip from stream

commit 82b6fba33d362aca2313cc0ca495f28f5ebb9260
Merge: b2214bb 6523cd9
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 13:10:59 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit b2214bb093ec2f759003c488c3c403c8931db914
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 13:09:53 2014 -0500

    go back to my URL until we can figure out what is up with jenkins

commit e7d614172240175a51f580aeaefb6460d21cede9
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 10:52:54 2014 -0500

    try this jenkins

commit 337a3c7704efa7c9809bf373152d711ee55f876c
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Nov 4 16:17:49 2014 +0100

    Rename temp-files under lock to prevent metadata reads while renaming

commit 77d5ba80d0a76efa549dd753b9f114b2f2d2d29c
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 10:07:11 2014 -0500

    continue to treat too-old/too-new as corruption for now

commit 98d0fd2f4851bc50e505a94ca592a694d502c51c
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 09:24:21 2014 -0500

    fix last nocommit

commit 643fceed66c8caf22b97fc489d67b4a2a90a1a1c
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Nov 4 14:46:17 2014 +0100

    remove NoSuchDirectoryException

commit 2e43c4feba05cfaf451df70f946c0930cbcc4557
Merge: 93826e4 8163107
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Nov 4 14:38:00 2014 +0100

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 93826e4d56a6a97c2074669014af77ff519bde63
Merge: 7f10129 44e24d3
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Nov 4 12:54:27 2014 +0100

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

    Conflicts:
    	src/main/java/org/elasticsearch/index/store/DistributorDirectory.java
    	src/main/java/org/elasticsearch/index/store/Store.java
    	src/main/java/org/elasticsearch/indices/recovery/RecoveryStatus.java
    	src/test/java/org/elasticsearch/index/store/DistributorDirectoryTest.java
    	src/test/java/org/elasticsearch/index/store/StoreTest.java
    	src/test/java/org/elasticsearch/indices/recovery/RecoveryStatusTests.java

commit 7f10129364623620575c109df725cf54488b3abb
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Nov 4 11:32:24 2014 +0100

    Fix TopHitsAggregator to not ignore the top-level/leaf collector split.

commit 042fadc8603b997bdfdc45ca44fec70dc86774a6
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Nov 4 11:31:20 2014 +0100

    Remove MatchDocIdSet in favor of DocValuesDocIdSet.

commit 7d877581ff5db585a674c95ac391ac78a0282826
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Nov 4 11:10:08 2014 +0100

    Make the and filter use the cost API.

    Lucene 5 ensured that cost() can safely be used, and this will have the benefit
    that the order in which filters are specified is not important anymore (only
    for slow random-access filters in practice).

commit 78f1718aa2cd82184db7c3a8393e6215f43eb4a8
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 23:55:17 2014 -0500

    fix previous eclipse import braindamage

commit 186c40e9258ce32f22a9a714ab442a310b6376e0
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 22:32:34 2014 -0500

    allow child queries to exhaust iterators again

commit b0b1271305e1b6d0c4c4da51a3c54df1aa5c0605
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Nov 3 14:50:44 2014 -0800

    Fix nocommit for mapping output.  index_options will not be printed if
    the field is not indexed.

commit ba223eb85e399c9620a347a983e29bf703953e7a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Nov 3 14:07:26 2014 -0800

    Remove no commit for chinese analyzer provider.  We should have a
    separate issue to address not using this provider on new indexes.

commit ca554b03c4471797682b2fb724f25205cf040c4a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Nov 3 13:41:59 2014 -0800

    Fix stop tests

commit de67c4653ec47dee9c671390536110749d2bb05f
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Nov 3 12:51:17 2014 -0800

    Remove analysis nocommits, switching over to Lucene43*Filters for
    backcompat

commit 50cae9bec72c25c33a1ab8a8931bccb3355171e2
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 15:32:25 2014 -0500

    add ram accounting and TODO lazy-loading (its no worse than master, can be a followup improvement) for suggesters

commit 7a7f0122f138684b312d0f0b03dc2a9c16c15f9c
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 15:11:26 2014 -0500

    bump lucene version

commit cd0cae5c35e7a9e049f49ae45431f658fb86676b
Merge: 446bc09 3c72073
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 14:49:05 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 446bc09b4e8bf4602d3c252b53ddaa0da65cce2f
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 14:46:30 2014 -0500

    remove hack

commit a19d85a968d82e6d00292b49630ef6ff2dbf2f32
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 12:53:11 2014 -0500

    dont create exceptions with circular references on corruption (will open a PR for this)

commit 0beefb9e821d97c37e90ec556d81ac7b00369b8a
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 11:47:14 2014 -0500

    temporarily add craptastic detector for this horrible bug

commit e9f2d298bff75f3d1591f8622441e459c3ce7ac3
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 10:56:01 2014 -0500

    add nocommit

commit e97f1d50a91a7129650b8effc7a9ecf74ca0569a
Merge: c57a3c8 f1f50ac
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 10:12:12 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit c57a3c8341ed61dca62eaf77fad6b8b48aeb6940
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 10:11:46 2014 -0500

    fix nocommit

commit dd0e77e4ec07c7011ab5f6b60b2ead33dc2333d2
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 09:54:09 2014 -0500

    nocommit -> TODO, this is in much more places in the codebase, bigger issue

commit 3cc3bf56d72d642059f8fe220d6f2fed608363e9
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Nov 1 23:59:17 2014 -0700

    Remove nocommit and awaitsfix for edge ngram filter test.

commit 89f115245155511c0fbc0d5ee62e63141c3700c1
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Nov 1 23:57:44 2014 -0700

    Fix EdgeNGramTokenFilter logic for version <= 4.3, and fixed instanceof
    checks in corresponding tests to correctly check for reverse filter when
    applicable.

commit 112df869cd199e36aab0e1a7a288bb1fdb2ebf1c
Author: Robert Muir <rmuir@apache.org>
Date:   Sun Nov 2 00:08:30 2014 -0400

    execute geo disjoint query/filter as intersects

commit e5061273cc685f1252e9a3a9ae4877ec9bce7752
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:58:59 2014 -0400

    remove chinese analyzer from docs

commit ea1af11b8978fcc551f198e24fe21d52806993ef
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:29:00 2014 -0400

    fix ram accounting bug

commit 53c0a42c6aa81aa6bf81d3aa77b95efd513e0f81
Merge: e3bcd3c 6011a18
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:16:29 2014 -0400

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit e3bcd3cc07a4957e12c7b3affc462c31290a9186
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:15:01 2014 -0400

    fix url-email back compat (thanks ryan)

commit 91d6b096a96c357755abee167098607223be1aad
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:11:26 2014 -0400

    bump lucene version

commit d2bb9568df72b37ec7050d25940160b8517394bc
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 20:33:07 2014 -0400

    remove nocommit

commit 1d049c471e19e5c457262c7399c5bad9e023b2e3
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 20:28:58 2014 -0400

    fix eclipse to group org/com imports together: without this, its madness

commit 09d8c1585ee99b6e63be032732c04ef6fed84ed2
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 14:27:41 2014 -0400

    remove nocommit, if you dont liek it, print assembly and tell me how it can be better

commit 8a6a294313fdf33b50c7126ec20c07867ecd637c
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 20:01:55 2014 +0100

    Remove deprecated usage of DocIdSets.newDocIDSet.

commit 601bee60543610558403298124a84b1b3bbd1045
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 14:13:18 2014 -0400

    maybe one of these zillions of annotations will stop thread leaks

commit 9d3f69abc7267c5e455aefa26db95cb554b02d62
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 14:05:39 2014 -0400

    fix some analysis nocommits

commit 312e3a29c77214b8142d21c33a6b2c2b151acf9a
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 18:28:45 2014 +0100

    Remove XConstantScoreQuery/XFilteredQuery/ApplyAcceptedDocsFilter.

commit 5a0cb9f8e167215df7f1b1fad11eec6e6c74940f
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 17:06:45 2014 +0100

    Fix misleading documentation of DocIdSets.toCacheable.

commit 8b4ef2b5b476fff4c79c0c2a0e4769ead26cf82b
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 17:05:59 2014 +0100

    Fix CustomRandomAccessFilterStrategy to override the right method.

commit d7a9a407a615987cfffc651f724fbd8795c9c671
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 16:21:35 2014 +0100

    Better handle the special case when there is a single SHOULD clause.

commit 648ad389f07e92dfc451f345549c9841ba5e4c9a
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 15:53:38 2014 +0100

    Cut over XBooleanFilter to BitDocIdSet.Builder.

    The idea is similar to what happened to Lucene's BooleanFilter.

    Yet XBooleanFilter is a bit more sophisticated and I had to slightly
    change the way it is implemented in order to make it work. The main difference
    with before is that slow filters are now applied lazily, so eg. if you have 3
    MUST clauses, two with a fast iterator and the third with a slow iterator, the
    previous implementation used to apply the fast iterators first and then only
    check the slow filter for bits which were set in the bit set. Now we are
    computing a bit set based on the fast must clauses and then basically returning
    a BitsFilteredDocIdSet.wrap(bitset, slowClause).

    Other than that, BooleanFilter still uses the bitset optimizations when or-ing
    and and-ind filters.

    Another improvement is that BooleanFilter is now aware of the cost API.

commit b2dad312b4bc9f931dc3a25415dd81c0d9deee08
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 10:18:53 2014 -0400

    clear nocommit

commit 4851d2091e744294336dfade33906c75fbe695cd
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 15:15:16 2014 +0100

    cut over to RoaringDocIdSet

commit ca6aec24a901073e65ce4dd6b70964fd3612409e
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:57:30 2014 +0100

    make nocommit more explicit

commit d0742ee2cb7a6c48b0bbb31580b7fbcebdb6ec40
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 09:55:24 2014 -0400

    fix standardtokenizer nocommit

commit 7d6faccafff22a86af62af0384838391d46695ca
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:54:08 2014 +0100

    fix compilation

commit a038a405c1ff6458ad294e6b5bc469e622f699d0
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:53:43 2014 +0100

    fix compilation

commit 30c9e307b1f5d80e2deca3392c0298682241207f
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:52:35 2014 +0100

    fix compilation

commit e5139bc5a0a9abd2bdc6ba0dfbcb7e3c2e7b8481
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 09:52:16 2014 -0400

    clear nocommit here

commit 85dd2cedf7a7994bed871ac421cfda06aaf5c0a5
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:46:17 2014 +0100

    fix CompletionPostingsFormatTest

commit c0f3781f616c9b0ee3b5c4d0998810f595868649
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 09:38:00 2014 -0400

    add tests for these analyzers

commit 51f9999b4ad079c283ae762c862fd0e22d00445f
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:10:26 2014 +0100

    remove nocommit - this is not an issue

commit fd1388fa03e622b0738601c8aeb2dbf7949a6dd2
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Fri Oct 31 14:07:01 2014 +0100

    Remove redundant null check

commit 3d6dd51b0927337ba941a235446b22e8cd500dc3
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Fri Oct 31 14:01:37 2014 +0100

    Removed the work around to prevent p/c error when invoking #iterator() twice, because the custom query filter wrapper now doesn't transform the result to a cache doc id set any more.

    I think the transforming to a cachable doc id set in CustomQueryWrappingFilter isn't needed at all, because we use the DocIdSet only once and because of that is just slowed things down.

commit 821832a537e00cd1216064b379df3e01d2911d3a
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 13:54:33 2014 +0100

    one more nocommit

commit 77eb9ea4c4ea50afb2680c29682ddcb3851a9d4f
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Fri Oct 31 13:52:29 2014 +0100

    Remove cast

commit a400573c034ed602221f801b20a58a9186a06eae
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 13:49:24 2014 +0100

    fix stop filter

commit 51746087cf8ec34c4d20aa05ba8dbff7b3b43eec
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 13:21:36 2014 +0100

    fix changed semantics of FBS.nextSetBit to check for NO_MORE_DOCS

commit 8d0a4e2511310f1293860823fe3ba80ac771bbe3
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 08:13:44 2014 -0400

    do the bogus cast differently

commit 46a5cc5732dea096c0c80ae5ce42911c9c51e44e
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 13:00:16 2014 +0100

    I hate it but P/C now passes

commit 580c0c2f82bbeacf217e594f22312b11d1bdb839
Merge: a9d3c00 1645434
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 06:54:31 2014 -0400

    fix nocommit/classcast

commit a9d3c004d62fe04989f49a897e6ff84973c06eb9
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 08:49:31 2014 +0100

    Update TODO.

commit aa75af0b407792aeef32017f03a6f442ed970baa
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 19:18:25 2014 -0400

    clear obselete nocommits from lucene bump

commit d438534cf41fcbe2d88070e2f27c994625e082c2
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 18:53:20 2014 -0400

    throw classcastexception when ES abuses regular filtercache for nested docs

commit 2c751f3a8feda43ec127c34769b069de21f3d16f
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 18:31:34 2014 -0400

    bump lucene revision, fix tests

commit d6ef7f6304ae262bf6228a7d661b2a452df332be
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 22:37:58 2014 +0100

    fix merge problems

commit de9d361f88a9ce6bb3fba85285de41f223c95767
Merge: 41f6aab f6b37a3
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 22:28:59 2014 +0100

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

    Conflicts:
    	pom.xml
    	src/main/java/org/elasticsearch/Version.java
    	src/main/java/org/elasticsearch/gateway/local/state/meta/MetaDataStateFormat.java

commit 41f6aab388aa80c40b08a2facab2617576203a0d
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 17:48:46 2014 +0100

    fix potiential NPE

commit c4428b12e1ae838b91e847df8b4a8be7f49e10f4
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 17:38:46 2014 +0100

    don't advance iterator in a match(doc) method

commit 28ab948e99e3ea4497c9b1e468384806ba7e1790
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 17:34:58 2014 +0100

    don't advance iterator in a match(doc) method

commit eb0f33f6634fadfcf4b2bf7327400e568f0427bb
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 16:55:54 2014 +0100

    fix GeoUtilsTest

commit 7f711fe3eaf73b6c2268cf42d5a41132a61ad831
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 16:43:16 2014 +0100

    Use a dedicated default index option if field type is not indexed by default

commit 78e3f37ab779e3e1b25b45a742cc86ab5f975149
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 10:56:14 2014 -0400

    disable this test with AwaitsFix to reduce noise

commit 9a590f563c8e03a99ecf0505c92d12d7ab20d11d
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 09:38:49 2014 +0100

    fix lucene version

commit abe3ca1d8bb6b5101b545198f59aec44bacfa741
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 09:35:05 2014 +0100

    fix AnalyzingCompletionLookupProvider to wrok with new codec API

commit 464293b245852d60bde050c6d3feb5907dcfbf5f
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 00:26:00 2014 -0400

    don't try to write stuff to tests class directory

commit 031cc6c19f4fe4423a034b515f77e5a0e282a124
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 00:12:36 2014 -0400

    AwaitsFix these known issues to reduce noise

commit 4600d51891e35847f2d344247d6f915a0605c0d1
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 00:06:53 2014 -0400

    openbitset lives on

commit 8492bae056249e2555d24acd55f1046b66a667c4
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 23:42:54 2014 -0400

    fixes for filter tests

commit 31f24ce4efeda31f97eafdb122346c7047a53bf2
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 23:12:38 2014 -0400

    don't use fieldcache

commit 8480789942fdff14a6d2b2cd8134502fe62f20c8
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 23:04:29 2014 -0400

    ancient index no longer supported

commit 02e78dc7ebdd827533009f542582e8db44309c57
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 23:37:02 2014 +0100

    fix more tests

commit ff746c6df23c50b3f3ec24922413b962c8983080
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 23:08:19 2014 +0100

    fix all mapper

commit e4fb84b517107b25cb064c66f83c9aa814a311b2
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 22:55:54 2014 +0100

    fix distributor tests and cut over to FileStore API

commit 20c850e2cfe3210cd1fb9e232afed8d4ac045857
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 22:42:18 2014 +0100

    use DOCS_ONLY if index=true and current options == null

commit 44169c108418413cfe51f5ce23ab82047463e4c2
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 22:33:36 2014 +0100

    Fix index=yes|no settings in mappers

commit a3c5f77987461a18121156ed345d42ded301c566
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 21:51:41 2014 +0100

    fix several field mappers conversion from setIndexed to indexOptions

commit df84d736908e88a031d710f98e222be68ae96af1
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 21:33:35 2014 +0100

    fix SourceFieldMapper to be not indexed

commit b2bf01d12a8271a31fb2df601162d0e89924c8f5
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 21:23:08 2014 +0100

    Cut over to .liv files in store and corruption tests

commit 619004df436f9ef05d24bef1b6a7f084c6b0ad75
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 17:05:52 2014 +0100

    fix more tests

commit b7ed653a8b464de446e00456bce0a89e47627c38
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 16:19:08 2014 +0100

    [STORE] Add dedicated method to write temporary files

    Recovery writes temporary files which might not end up in the
    right distributor directories today. This commit adds a dedicated
    API that allows specifying the target file name in order to create the
    tempoary file in the correct directory.

commit 7d574659f6ae04adc2b857146ad0d8d56ca66f12
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 10:28:49 2014 -0400

    add some leniency to temporary bogus method

commit f97022ea7c2259f7a5cf97d924c59ed75ab65b32
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 10:24:17 2014 -0400

    fix MultiCollector bug

commit b760533128c2b4eb10ad76e9689ef714293dd819
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:56:08 2014 +0100

    CheckIndex is now closeable we need to close it

commit 9dae9fb6d63546a6c2427be2a2d5c8358f5b1934
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:45:11 2014 +0100

    s/Lucene51/Lucene50

commit 7aea9b86856a8c1b06a08e7c312ede1168af1287
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:42:30 2014 +0100

    fix BloomFilterPostingsFormat

commit 16fea6fe842e88665d59cc091e8224e8dc6ce08c
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:41:16 2014 +0100

    fix some codec format issues

commit 3d77aa97dd2c4012b63befef3f2ba2525965e8a6
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:30:43 2014 +0100

    fix CodecTests

commit 6ef823b1fde25657438ace1aabd9d552d6ae215e
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:26:47 2014 +0100

    make it compile

commit 9991eee1fe99435118d4dd42b297ffc83fce5ec5
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 09:12:43 2014 -0400

    add an ugly hack for TopHitsAggregator for now

commit 03e768a01fcae6b1f4cb50bcceec7d42977ac3e6
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:01:02 2014 +0100

    cut over ES090PostingsFormat

commit 463d281faadb794fdde3b469326bdaada25af048
Merge: 0f8740a 8eac79c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 08:30:36 2014 -0400

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 0f8740a782455a63524a5a82169f6bbbfc613518
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 01:00:15 2014 -0400

    fix/hack remaining filter and analysis issues

commit df534488569da13b31d66e581456dfd4b55156b9
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Oct 28 23:11:47 2014 -0400

    fix ngrams / openbitset usage

commit 11f5dc3b9887f4da80a0fa1818e1350b30599329
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Oct 28 22:42:44 2014 -0400

    hack over sort comparators

commit 4ebdc754350f512596f6a02770d223e9f5f7975a
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Oct 28 21:27:07 2014 -0400

    compiler errors < 100

commit 2d60c9e29de48ccb0347dd87f7201f47b67b83a0
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Oct 28 03:13:08 2014 -0400

    clear some nocommits around ram usage

commit aaf47fe6c0aabcfb2581dd456fc50edf871da758
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 12:27:34 2014 -0400

    migrate fieldinfo handling

commit ef6ed6d15d8def71cd880d97249678136cd29fe3
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 12:07:13 2014 -0400

    more simple fixes

commit f475e1048ae697dd9da5bd9da445102b0b7bc5b3
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 11:58:21 2014 -0400

    more fielddata ram accounting fixes

commit 16b4239eaa9b4262df258257df4f31d39f28a3a2
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 16:47:32 2014 +0100

    add missing file

commit 5b542fa2a6da81e36a0c35b8e891a1d8bc58f663
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 16:43:29 2014 +0100

    cut over completion posting formats - still some nocommits

commit ecdea49404c4ec4e1b78fb54575825f21b4e096e
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 11:21:09 2014 -0400

    fielddata accountable fixes

commit d43da265718917e20c8264abd43342069198fe9c
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 16:19:53 2014 +0100

    cut over BloomFilterPostings to new API

commit 29b192ba621c14820175775d01242162b88bd364
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 10:22:51 2014 -0400

    fix more analyzers

commit 74b4a0c5283e323a7d02490df469497c722780d2
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 09:54:25 2014 -0400

    fix tests

commit 554084ccb4779dd6b1c65fa7212ad1f64f3a6968
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 14:51:48 2014 +0100

    maintain supressed exceptions on CorruptIndexException

commit cf882d9112c5e8ef1e9f2b0f800f7aa59001a4f2
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 14:47:17 2014 +0100

    commitOnClose=false

commit ebb2a9189ab2f459b7c6c9985be610fd90dfe410
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 14:46:06 2014 +0100

    cut over indexwriter closeing in InternalEngine

commit cd21b3d4706f0b562bd37792d077d60832aff65f
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 14:38:10 2014 +0100

    fix constant

commit f93f900c4a1c90af3a21a4af5735a7536423fe28
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 09:50:49 2014 -0400

    fix test

commit a9a752940b1ab4699a6a08ba8b34afca82b843fe
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Mon Oct 27 09:26:18 2014 +0100

    Be explicit about the index options

commit d9ee815babd030fa2ceaec9f467c105ee755bf6b
Author: Simon Willnauer <simonw@apache.org>
Date:   Sun Oct 26 20:03:44 2014 +0100

    cut over store and directory

commit b3f5c8e39039dd8f5caac0c4dd1fc3b1116e64ca
Author: Robert Muir <rmuir@apache.org>
Date:   Sun Oct 26 13:08:39 2014 -0400

    more test fixes

commit 8842f2684e3606aae0860c27f7a4c53e273d47fb
Author: Robert Muir <rmuir@apache.org>
Date:   Sun Oct 26 12:14:52 2014 -0400

    tests manual labor

commit c43de5aec337919a3fdc3638406dff17fc80bc98
Author: Robert Muir <rmuir@apache.org>
Date:   Sun Oct 26 11:04:13 2014 -0400

    BytesRef -> BytesRefBuilder

commit 020c0d087a2f37566a1db390b0e044ebab030138
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Sun Oct 26 15:53:37 2014 +0100

    Moved over to BitSetFilter

commit 48dd1b909e6c52cef733961c9ecebfe4f67109fe
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Sun Oct 26 15:53:11 2014 +0100

    Left over Collector api change in ScanContext

commit 6ec248ef63f262bcda400181b838fd9244752625
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Sun Oct 26 15:47:40 2014 +0100

    Moved indexed() over to indexOptions != null or indexOptions == null

commit 9937aebfd8546ae4bb652cd976b3b43ac5ab7a63
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Sun Oct 26 13:26:31 2014 +0100

    Fixed many compile errors. Mainly around the breaking Collector api change in 5.0.

commit fec32c4abc0e3309cf34260c8816305a6f820c9e
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Oct 25 11:22:17 2014 -0400

    more easy fixes

commit dab22531d801800d17a65dc7c9464148ce8ebffd
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Oct 25 09:33:41 2014 -0400

    more progress

commit 414767e9a955010076b0497cc4f6d0c1850b48d3
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Oct 25 06:33:17 2014 -0400

    more progress

commit ad9d969fddf139a8830254d3eb36a908ba87cc12
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 24 14:28:01 2014 -0400

    current state of fun

commit 464475eecb0be15d7d084135ed16051f76a7e521
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 24 11:42:41 2014 -0400

    bump to 5.0 snapshot
2014-11-05 15:48:51 -05:00
Clinton Gormley 4d3842311f Docs: Updated ES/JVM versions 2014-11-05 12:41:22 +01:00
Adrien Grand 9ea25df649 Switch to murmurhash3 to route documents to shards.
We currently use the djb2 hash function in order to compute the shard a
document should go to. Unfortunately this hash function is not very
sophisticated and you can sometimes hit adversarial cases, such as numeric ids
on 33 shards.

Murmur3 generates hashes with a better distribution, which should avoid the
adversarial cases.

Here are some examples of how 100000 incremental ids are distributed to shards
using either djb2 or murmur3.

5 shards:
Murmur3: [19933, 19964, 19940, 20030, 20133]
DJB:     [20000, 20000, 20000, 20000, 20000]

3 shards:
Murmur3: [33185, 33347, 33468]
DJB:     [30100, 30000, 39900]

33 shards:
Murmur3: [2999, 3096, 2930, 2986, 3070, 3093, 3023, 3052, 3112, 2940, 3036, 2985, 3031, 3048, 3127, 2961, 2901, 3105, 3041, 3130, 3013, 3035, 3031, 3019, 3008, 3022, 3111, 3086, 3016, 2996, 3075, 2945, 2977]
DJB:     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 900, 900, 900, 900, 1000, 1000, 10000, 10000, 10000, 10000, 9100, 9100, 9100, 9100, 9000, 9000, 0, 0, 0, 0, 0, 0]

Even if djb2 looks ideal in some cases (5 shards), the fact that the
distribution of its hashes has some patterns can raise issues with some shard
counts (eg. 3, or even worse 33).

Some tests have been modified because they relied on implementation details of
the routing hash function.

Close #7954
2014-11-04 16:32:42 +01:00
Clinton Gormley 5797682bd0 Update cluster.asciidoc - fix invalid asciidoc 2014-11-04 15:22:36 +01:00
Clinton Gormley 60eaeb5052 Update cluster.asciidoc
Fixed asciidoc on cluster module page
2014-11-04 14:32:05 +01:00
Clinton Gormley b0e5fb7823 Update zen.asciidoc
Tidied up the "No master block" asciidoc
2014-11-04 14:27:22 +01:00
Martijn Laarman 82278bb7bc [Aggregations] Meta data support
This commit adds the ability to associate a bit of state with each
individual aggregation.

The aggregation response can be hard to stitch back together without
having a reference to the aggregation request. In many cases this is not
available, many json serializer frameworks cache types globally or have a
static deserialisation override mechanism. In these cases making the
original request available, if at all possible, would be a hack.

The old facets returned `_type` which was just enough metadata to know
what the originating facet type in the request was.

This PR takes `_type` one step further by introducing ANY arbitrary meta
data. This could be further <strike>ab</strike>used for instance by
generic/automated aggregations that include UI state (color information,
thresholds, user input states, etc) per aggregation.
2014-11-03 22:32:23 +01:00
Ryan Ernst 7ec31abbb7 Fix missing word in upgrade docs. 2014-11-03 11:44:41 -08:00
Alexander Reelsen c04fa43587 Docs: Convert markdown to asciidoc in transport profile docs 2014-11-02 08:25:45 +01:00
Aarni Koskela 6011a18381 Docs: Add mention of `hyphenation_patterns_path`
Refs ElasticSearch's HyphenationCompoundWordTokenFilterFactory.java.

Closes #8305
2014-11-01 15:47:53 +01:00
Alexander Reelsen 5eeac2fdf6 Netty: Add HTTP pipelining support
This adds HTTP pipelining support to netty. Previously pipelining was not
supported due to the asynchronous nature of elasticsearch. The first request
that was returned by Elasticsearch, was returned as first response,
regardless of the correct order.

The solution to this problem is to add a handler to the netty pipeline
that maintains an ordered list and thus orders the responses before
returning them to the client. This means, we will always have some state
on the server side and also requires some memory in order to keep the
responses there.

Pipelining is enabled by default, but can be configured by setting the
http.pipelining property to true|false. In addition the maximum size of
the event queue can be configured.

The initial netty handler is copied from this repo
https://github.com/typesafehub/netty-http-pipelining

Closes #2665
2014-10-31 16:30:11 +01:00
Clinton Gormley e56d85439c Update search-template.asciidoc
Clarified using the conditional clause template example as a string
2014-10-31 15:32:14 +01:00
Clinton Gormley 2569188d25 Update search-template.asciidoc
Fixed asciidoc typo

Closes #8308
2014-10-31 14:40:32 +01:00
astefan 4049154dbc Docs: Document action.replication_type setting
Document action.replication_type setting

Closes #8290
2014-10-31 13:53:34 +01:00
cmpich e57c8b0673 Docs: Update getting-started.asciidoc
Closes #8195
2014-10-29 15:04:13 +01:00
cmpich 36462c0305 Docs: Update getting-started.asciidoc
Closes #8194
2014-10-29 15:01:18 +01:00
Clinton Gormley 8f02c451b8 Update source-field.asciidoc
very minor typofix

Closes #8066
2014-10-29 14:51:05 +01:00
Alex Ksikes 35f55608cc MLT Field Query: remove it from master
The MLT field query is simply replaced by a MLT query set to specififc field.
To simplify code maintenance we should deprecate it in 1.4 and remove it in
2.0.

Closes #8238
2014-10-29 10:19:00 +01:00
Areek Zillur 96f1606cdc Completion Suggester: Fix CompletionFieldMapper to correctly parse weight
- Allows weight to be defined as a string representation of a positive integer

closes #8090
2014-10-28 18:39:02 -04:00
Dmitriy Khvatov 71a90ab4fe Docs: Update multi-get.asciidoc
Duplicate word

Closes #8228
2014-10-28 10:58:47 +01:00
tlrx 8c864cf3f6 Cat Recovery API: Reverting changes introduced with commit e1c75bae87
Adding these 2 headers to the CAT Recovery made the CI tests hanging for a loooong time.

Related to #8041
2014-10-27 20:49:58 +01:00
Zachary Tong f5b2dfd052 Aliases: Throw exception if index is null or missing when creating an alias
Fixes a bug where alias creation would allow `null` for index name, which thereby
applied the alias to _all_ indices.  This patch makes the validator throw an
exception if the index is null.

```bash
POST /_aliases
{
   "actions": [
      {
         "add": {
            "alias": "empty-alias",
            "index": null
         }
      }
   ]
}
```
```json
{
   "error": "ActionRequestValidationException[Validation Failed: 1: Alias action [add]: [index] may not be null;]",
   "status": 400
}
```

The reason this bug wasn't caught by the existing tests is because
the old test for nullness only validated against a cluster which had
zero indices.  The null index is translated into "_all", and since
there are no indices, this fails because the index doesn't exist.
 So the test passes.

However, as soon as you add an index, "_all" resolves and you get the
situation described in the original bug report:  null index is
accepted by the alias, resolves to "_all" and gets applied to everything.

The REST tests, otoh, explicitly tested this bug as a real feature and therefore
passed.  The REST tests were modified to change this behavior.

Fixes #7863
2014-10-27 14:39:01 -04:00
Alex Ksikes 0be5c60bce MLT Query: use ParseField#withAllDeprecated for percent_terms_to_match
Also the parameter was deprecated but not removed so we keep it in the doc and
mark it as deprecated ...

Closes #8241
2014-10-27 17:35:06 +01:00
Alex Ksikes 991f3e2cd3 Docs: fix tags for dfs and new like parameter 2014-10-27 15:42:44 +01:00
Clinton Gormley fbd0403a6f Documented that HTTP pipelining is not supported 2014-10-27 14:49:48 +01:00
Adrien Grand 7ea490dfd1 Aggregations: Return the sum of the doc counts of other buckets.
This commit adds a new field to the response of the terms aggregation called
`sum_other_doc_count` which is equal to the sum of the doc counts of the buckets
that did not make it to the list of top buckets. It is typically useful to have
a sector called eg. `other` when using terms aggregations to build pie charts.

Example query and response:

```json
GET test/_search?search_type=count
{
  "aggs": {
    "colors": {
      "terms": {
        "field": "color",
        "size": 3
      }
    }
  }
}
```

```json
{
   [...],
   "aggregations": {
      "colors": {
         "doc_count_error_upper_bound": 0,
         "sum_other_doc_count": 4,
         "buckets": [
            {
               "key": "blue",
               "doc_count": 65
            },
            {
               "key": "red",
               "doc_count": 14
            },
            {
               "key": "brown",
               "doc_count": 3
            }
         ]
      }
   }
}
```

Close #8213
2014-10-27 12:11:26 +01:00
tlrx e1c75bae87 Cat API: Add node name to _cat/recovery
Add source_node and target_node fields to the recovery cat API. Also fixed and updated the documentation which was not complete concerning fields names.

Closes #8041
2014-10-27 09:47:26 +01:00
Alex Ksikes 4da407a869 MLT Query: versatile 'like' parameter
The MLT query has a lot of parameters. For example, a set of documents is
specified with either `like_text`, `ids` or `docs`, with at least one
parameter required. This commit groups all the document specification
parameters under one called `like`. The syntax is described below and could
easily be extended to allow for new means of specifying document input. The
`like_text`, `ids` and `docs` parameters are deprecated.

As a single piece text:

{
  "query": {
    "more_like_this": {
      "like": "some text here"
    }
  }
}

As a single item:

{
  "query": {
    "more_like_this": {
      "like": {
        "_index": "imdb",
        "_type": "movies",
        "_id": "88247"
      }
    }
  }
}

Or as a mixture of all:

{
  "query": {
    "more_like_this": {
      "like": [
        "Some random text ...",
        {
          "_index": "imdb",
          "_type": "movies",
          "_id": "88247"
        },
        {
          "_index": "imdb",
          "_type": "movies",
          "doc": {
            "title": "Document with an artificial title!"
          }
        }
      ]
    }
  }
}

Closes #8039
2014-10-25 11:04:51 +02:00
David Pilato 62d8b7ab97 Docs: rolling upgrade process seems incorrect
When reading the [rolling upgrade process](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-upgrade.html#rolling-upgrades), you can see that we wrote:

* disable allocation
* upgrade node1
* upgrade node2
* upgrade node3
* ...
* enable allocation

That won't work as after a node has been removed and restarted, no shard will be allocated anymore.
So closing node2 and remaining nodes, won't help to serve index and search request anymore.

We should write:

* disable allocation
* upgrade node1
* enable allocation
* wait for shards being recovered on node1
* disable allocation
* upgrade node2
* enable allocation
* wait for shards being recovered on node2
* disable allocation
* upgrade node3
* enable allocation
* wait for shards being recovered on node3
* disable allocation
* ...
* enable allocation

I think this documentation update should go in 1.3, 1.4, 1.x and master branches.

Closes #8218
Closes #7973.
2014-10-24 16:45:42 +02:00
Marcin Mikosik ed86d925cd Docs: fixed typo in documentation
Closes #8205
2014-10-24 15:27:31 +02:00
Simon Willnauer d5c0a49620 [ROUTING] Add rebalance enabled allocation decider
This commit adds the ability to enable / disable relocations
on an entire cluster or on individual indices for either:

 * `primaries` - only primaries can rebalance
 * `replica` - only replicas can rebalance
 * `all` - everything can rebalance (default)
 * `none` - all rebalances are disabled

similar to the allocation enable / disable functionality.

Relates to #7288
2014-10-23 14:07:13 +02:00
Alex Ksikes c13f5f21de Term Vectors: support for distributed frequencies
Adds distributed frequencies support for the Term Vectors API. A new parameter
called `dfs` is introduced which defaults to `false`.

Closes #8144
2014-10-23 13:59:59 +02:00
Clinton Gormley a8b21f2cd5 Update update-settings.asciidoc
Removed deprecated `cluster.routing.allocation.disable` settings
2014-10-22 12:46:33 +02:00
Clinton Gormley 2d0c440b09 Update cluster.asciidoc
Fixed asciidoc syntax
2014-10-22 12:45:10 +02:00
Brian Kim 58086dd08b Docs: missing quote
fix missing quote

Closes #8176
2014-10-21 12:52:12 +02:00
Peter Dyson b984cb771f Docs: Provide example of deleting a repository
Example of deleting a repository with explanation that snapshots themselves are left untouched.

Closes #8172
2014-10-21 10:10:03 +02:00
Andrei Kolosok c31a783930 Docs: Update filtered-query.asciidoc
Fix mistyping

Closes #8167
2014-10-21 09:45:19 +02:00
Andrei Kolosok 92abfc8e24 Docs: Update minimum-should-match.asciidoc
Add %-sign to examle in the last section

Closes #8157
2014-10-21 09:43:55 +02:00
David Pilato 0ff61e1d6f Add time_zone setting for query_string
Query String query now supports a new `time_zone` option based on JODA time zones.
When using a range on date field, the time zone is applied.

```json
{
"query": {
  "query_string": {
    "text": "date:[2012 TO 2014]",
    "timezone": "Europe/Paris"
  }
 }
}
```

Closes #7880.
2014-10-20 19:09:45 +02:00
Adrien Grand 230c6684a9 Search: Remove partial fields.
Partial fields have been deprecated since 1.0.0Beta1 in favor of _source
filtering. They will be removed in 2.0.
2014-10-20 12:29:30 +02:00
Adrien Grand f4ee3f25e4 Mappings: Store _timestamp by default.
Storing `_timestamp` by default means that under the default configuration, you
would have all the information you need in order to reindex into a different
index.

Close #8139
2014-10-20 12:17:26 +02:00
Clinton Gormley 4da62a33b5 Update plugins.asciidoc
Added Vietnamese Analyser to plugins page

Closes #6647
2014-10-20 11:56:01 +02:00
Ryan Grimm 74586e2867 Docs: Added 'd' to the list of supported units.
Day was missing from the list of supported units in the date math section.

Closes #8151
2014-10-19 21:24:28 +02:00
Michael McCandless 85065f9c8e Core: cutover to Lucene's query rescorer
This is functionally equivalent to before, so there should be no
user-visible impact, except I added a NOTE in the docs warning about
the interaction of pagination and rescoring.

Closes #6232

Closes #7707
2014-10-18 05:25:50 -04:00
Alexander Reelsen 37e606819c Test: Fix netty multiport tests
Ensuring, that even in local mode, netty is started appropriately.
2014-10-17 16:16:04 -07:00
Alexander Reelsen b06b52449f Netty: Support to bind on multiple host/port pairs
This patch allows to create several netty bootstrap, each of which
listening on different ports. This will potentially allow for features
to listen to different network interfaces for node-to-node or node-to-client
communication and is also the base to listen to several interfaces, so that those
can be used to speed up cluster communication in the future.

Closes #8098
2014-10-17 13:55:07 -07:00
Sergii Golubev 028a2b732a Docs: Percolate reference - a typo and a misused word
Closes #8116
2014-10-17 15:26:29 +02:00
Sergii Golubev ae923a81b9 Docs: Percolate `_score` reference
Added missing `_score` word, made the sentence less ambiguous.

Closes #8115
2014-10-17 15:25:02 +02:00
Sergii Golubev 0fb81d6643 Docs: Update id-field.asciidoc
It is strange to provide an example with `"store" : false` when talking about possibility of enabling the field to be stored.
Broke the line in the mapping in two lines for better readability.
More verbose sentence above the mapping.

Closes #7894
2014-10-17 15:17:24 +02:00
Clinton Gormley 646f7acf5c Docs: Documented the http.*host and transport.*host settings 2014-10-17 15:01:42 +02:00
Andrew O'Brien 33097d901b Docs: Typo: s/by/be/
Closes #8114
2014-10-16 20:51:58 +02:00
Patrick Peschlow 9b03fda5d7 Docs: fix typo
Closes #8111
2014-10-16 20:48:57 +02:00
John Michael Luy 86f6e21cdf Docs: Update query-string-query.asciidoc
Update sample

Closes #8024
2014-10-16 20:21:12 +02:00
Josh Kuhn f4d9cce659 Docs: Added RethinkDB River to list of community plugins
Closes #8016
2014-10-16 20:15:33 +02:00
Son 6f3227db01 Docs: Fix order for PUT _mapping docs
Closes #8083
2014-10-16 18:49:36 +02:00
Colin Goodheart-Smithe c7971fb0f9 Docs: adds note about using null_value with dynamic templates
Closes #7874
2014-10-16 14:00:00 +01:00
Jun Ohtani 533c1084ec Docs: add the predefined language-specific stopword lists to stop-tokenfilter.asciidoc 2014-10-16 13:20:38 +09:00
Erik Ringsmuth 516b87e129 Docs: Removing escape character
The escape character `\` doesn't work and should be removed from the documentation.

Closes #8029
2014-10-15 21:42:22 +02:00
Patrick Peschlow 340e0efcaa Docs: cross_type query needs to be cross_fields
Closes #8021
2014-10-15 18:27:48 +02:00
Chris Earle 2d8a140ed8 Add file descriptor details to cat/nodes
cat/nodes currently does not report any details related to file descriptors. This adds the current number in use, the maximum number available as well as their ratio (percentage) to cat/nodes as hidden-by-default metrics. In addition, this also adds current heap usage (as a non-percentage of ts max) and ram usage (as a non-percerntage of its max) to allow tools to provide more granularity.

Closes #7652
2014-10-15 10:18:41 -05:00
Chris Earle 9b84ad3c7b Adding "min" score mode to parent-child queries
Support for "max", "sum", and "avg" already existed.
2014-10-15 09:23:36 -05:00
Nik Everett 5dc5849922 Docs: Add plugin to list
This plugin provides trigram accelerated regular expression search for when
you need reasonably fast arbitrary matching.

Closes #7997
2014-10-15 13:56:35 +02:00
Clinton Gormley 6a180d1803 Docs: Update highlighting.asciidoc
Added note about how to highlight on the `_all` field

Closes #7991
2014-10-15 13:45:56 +02:00
Henrik Nordvik fdbb62b1ab Docs: Fix curl statements in query-cache.asciidoc
Closes #7989
2014-10-15 13:16:20 +02:00
sp836490 517caa0c6f Update cjk-bigram-tokenfilter.asciidoc 2014-10-15 11:54:19 +09:00
Clinton Gormley 7e916d0b8b Update completion-suggest.asciidoc
Documented the `size` parameter in the completion suggester query
2014-10-14 18:47:32 +02:00
Jake Buob cb381b4645 Docs: Fix (another) Idaho spelling
Closes #7912
2014-10-14 13:29:35 +02:00
Hang Su dd62e50861 fix mismatched curly bracket 2014-10-14 13:24:44 +02:00
Clinton Gormley 66cd3ad17e Docs: Update object-type.asciidoc to make object type more explicit
Closes #7646
2014-10-14 13:16:37 +02:00
Suyog Rao 82b16ae0ad Doc: Clarify that index.routing.allocation.total_shards_per_node means both primary and replica shards
Closes #8002
2014-10-13 18:08:23 -07:00
Adrien Grand 87497c92ca Docs: Add missing link to the 2.0 migration guide. 2014-10-12 17:54:21 +02:00
Clinton Gormley 0d3d541d62 Docs: Tidied the upgrade indices docs 2014-10-11 16:22:24 +02:00
Ryan Ernst 6ac094c038 Add forgotten include for upgrade docs.
See #7922
2014-10-10 10:55:45 -07:00
Ryan Ernst e2c50c033a Add link to new upgrade docs from indices page.
See #7922.
2014-10-10 10:52:11 -07:00
Adrien Grand 491a48e55b Docs: Remove the note that fielddata doesn't support filtering.
This particular note was about fielddata filtering but could cause confusion
that fields that have doc values enabled cannot be used for filtering (as in
a `filtered_query`).
2014-10-10 10:50:47 +02:00
Martijn van Groningen 5763b24686 Core: Make fetch phase nested doc aware
By letting the fetch phase understand the nested docs structure we can serve nested docs as hits.
The `top_hits` aggregation can because of this commit be placed in a `nested` or `reverse_nested` aggregation.

Closes #7164
2014-10-08 22:21:30 +02:00
Colin Goodheart-Smithe 9ce7ca21e4 Docs: Added breaking changes docs for Indices APIs
Adds the breaking changes defaults for the change of default indices options for the GET Aliases API
2014-10-08 15:08:25 +01:00
Igor Motov 45d3842214 Docs: clarify wait_for_completion flag in snapshot documentation
Closes #7952
2014-10-08 08:43:06 -04:00
David Pilato 80ca8e5ab6 Admin: add total index memory in `_cat/indices`
This patch adds to `_cat/indices` information about memory usage per index by adding memory used by FieldData, IdCache, Percolate, Segments (memory, index writer, version map).

```
% curl 'localhost:9200/_cat/indices?v&h=i,tm'
i     tm
wiki  8.1gb
test  30.5kb
user  1.9mb
```

Closes #7008
2014-10-08 13:38:00 +02:00
Igor Motov 2359e33b1e Docs: fix formatting in snapshots documentation 2014-10-07 12:33:21 -04:00
Ryan Ernst c06c10bbb0 Remove deprecations from master (follow up to #7922) 2014-10-07 08:35:11 -07:00
Ryan Ernst c021f22523 Add Upgrade API
This commit does the following:
* Add the new API at the rest layer, being backed by the optimize API
  with upgrade flag, and segments api to find upgrade status.
* Add `upgrade` flag to optimize API, and deprecate `force` flag (will
  remove in master)
* Add test for both synchronous and async upgrade

closes #7884
closes #7922
2014-10-07 08:09:50 -07:00
Igor Motov 555bfcb02b [SNAPSHOT] Add repository validation
Fixes #7096
2014-10-07 10:50:16 -04:00
David Pilato 6ae6a078de Search: add `format` support for date range filter and queries
When the date format is defined in mapping, you can not use another format when querying using range date query or filter.

For example, this won't work:

```
DELETE /test

PUT /test/t/1
{
  "date": "2014-01-01"
}

GET /test/_search
{
  "query": {
    "filtered": {
      "filter": {
        "range": {
          "date": {
            "from": "01/01/2014"
          }
        }
      }
    }
  }
}
```

It causes:

```
Caused by: org.elasticsearch.ElasticsearchParseException: failed to parse date field [01/01/2014], tried both date format [dateOptionalTime], and timestamp number
```

It could be nice if we can support at query time another date format just like we support `analyzer` at search time on String fields.

Something like:

```
GET /test/_search
{
  "query": {
    "filtered": {
      "filter": {
        "range": {
          "date": {
            "from": "01/01/2014",
            "format": "dd/MM/yyyy"
          }
        }
      }
    }
  }
}
```

Same for queries:

```
GET /test/_search
{
  "query": {
    "range": {
      "date": {
        "from": "01/01/2014",
        "format": "dd/MM/yyyy"
      }
    }
  }
}
```

Closes #7189.
2014-10-06 15:59:18 +02:00
Colin Goodheart-Smithe 6cf371395a Aggregations: makes script params consistent with other APIs in scripted_metric
This change removes the script_type parameter form the Scripted Metric Aggregation and adds support for _file and _id suffixes to the init_script, map_script, combine_script and reduce_script parameters to make defining the source of the script consistent with the other APIs which use the ScriptService
2014-10-06 09:07:25 +01:00
Alex Ksikes 349b7a3a8b Term Vectors/MLT Query: support for different analyzers than default at field
This adds a `per_field_analyzer` parameter to the Term Vectors API, which
allows to override the default analyzer at the field. If the field already
stores term vectors, then they will be re-generated. Since the MLT Query uses
the Term Vectors API under its hood, this commits also adds the same ability
to the MLT Query, thereby allowing users to fine grain how each field item
should be processed and analyzed.

Closes #7801
2014-10-03 16:40:17 +02:00
Alex Ksikes c4830cf862 Term Vectors: support for realtime
By default term vectors are now realtime, as opposed to previously near
realtime. If they are not found in the index, they will be generated on the
fly. The document is fetched from the transaction log and treated as an
artificial document. One can set `realtime` parameter to `false` in order to
disable this functionality. This consequently makes the MLT query realtime in
fetching documents, as it previsouly used to be before switching from using
the multi get API to the mtv API.

Closes #7846
2014-10-03 09:26:47 +02:00
Clinton Gormley ea7ed51014 Docs: Typoes on the range filter page 2014-09-30 19:28:35 +02:00
Martijn van Groningen fca406415d Docs: Document the most important changes to zen discovery.
Closes #7746
2014-09-30 13:05:45 +02:00
Alex Ksikes b118558962 MLT Query: Support for artificial documents
Previously, the only way to specify a document not present in the index was to
use `like_text`. This would usually lead to complex queries made of multiple
MLT queries per document field. This commit adds the ability to the MLT query
to directly specify documents not present in the index (artificial documents).
The syntax is similar to the Percolator API or to the Multi Term Vector API.

Closes #7725
2014-09-29 15:49:13 +02:00