Commit Graph

1414 Commits

Author SHA1 Message Date
Mark Walkom bfd1bcd30a Updated threadpool documentation to elaborate/clarify what the pools are for and their values
Closes #8446
2014-11-12 22:33:38 +11:00
Israel Tsadok 7590629531 Docs: note about confusing disk threshold settings 2014-11-12 09:24:03 +01:00
Martijn van Groningen 94c1a7dabe Docs: Fix incorrect documentation for the `index.query.parse.allow_unmapped_fields` setting.
The `index.query.parse.allow_unmapped_fields` setting can't influence whether unmapped fields are allowed
in alias filters and percolator queries.
2014-11-11 15:13:55 +00:00
Michael McCandless 8aebb9656b Core: add max_determinized_states to query_string and regexp query/filter
This prevents too-difficult regular expressions from consuming
excessive RAM/CPU; the default max_determinized_states is 10,000 (same
as Lucene) but query_string and regepx query/filter can override
per-request.

The also upgrades to a new Lucene 5.0.0 snapshot.

Closes #8386

Closes #8357
2014-11-10 13:43:48 -05:00
Clinton Gormley cff544dcc2 Docs: Removed old coming/added tags 2014-11-10 14:41:24 +01:00
Britta Weber c5a4c1d6b4 [docs] add 2d vis for decay functions and parameters
closes #8420
2014-11-10 10:56:41 +01:00
Veres Lajos 4059e4ac86 typo fixes - https://github.com/vlajos/misspell_fixer
Closes #8323
2014-11-08 18:55:57 +01:00
Clinton Gormley 08aa715d2e Update datehistogram-aggregation.asciidoc
Clarified use of fractional time units in the date histo agg.

Closes #7957
2014-11-08 17:49:34 +01:00
Clinton Gormley b9149f836b Docs: Improve the exists/missing filters documentation
Closes #7274
2014-11-08 16:57:41 +01:00
Clinton Gormley f5ad699284 Update multi-get.asciidoc
Documented that the fields parameter can be passed in the query string.

Closes #4006
2014-11-08 13:55:23 +01:00
Kevin Kluge c473976e31 [docs] fix typo in getting-started
Closes #8354
2014-11-06 10:57:56 +01:00
Robert Muir 610ce078fb Upgrade master to lucene 5.0 snapshot
This has a lot of improvements in lucene, particularly around memory usage, merging, safety, compressed bitsets, etc.

On the elasticsearch side, summary of the larger changes:

    API changes: postings API became a "pull" rather than "push", collector API became per-segment, etc.
    packaging changes: add lucene-backwards-codecs.jar as a dependency.
    improvements to boolean filtering: especially ensuring it will not be slow for SparseBitSet.
    use generic BitSet api in plumbing so that concrete bitset type is an implementation detail.
    use generic BitDocIdSetFilter api for dedicated bitset cache, so there is type safety.
    changes to support atomic commits
    implement Accountable.getChildResources (detailed memory usage API) for fielddata, etc
    change handling of IndexFormatTooOld/New, since they no longer extends CorruptIndexException

Closes #8347.

Squashed commit of the following:

commit d90d53f5f21b876efc1e09cbd6d63c538a16cd89
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Nov 5 21:35:28 2014 +0100

    Make default codec/postings/docvalues format constants

commit cb66c22c71cd304a36e7371b199a8c279908ae37
Merge: d4e2f6d ad4ff43
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Nov 5 11:41:13 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit d4e2f6dfe767a5128c9b9ae9e75036378de08f47
Merge: 4e5445c 4111d93
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Nov 5 06:26:32 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 4e5445c775f580730eb01360244e9330c0dc3958
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 16:19:19 2014 -0500

    FixedBitSet -> BitSet

commit 9887ea73e8b857eeda7f851ef3722ef580c92acf
Merge: 1bf8894 fc84666
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 15:26:25 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 1bf8894430de3e566d0dc5623b0cc28b0d674ebb
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 15:22:51 2014 -0500

    remove nocommit

commit a9c2a2259ff79c69bae7806b64e92d5f472c18c8
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 13:48:43 2014 -0500

    turn jenkins red again

commit 067baaaa4d52fce772c81654dcdb5051ea79139f
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 13:18:21 2014 -0500

    unzip from stream

commit 82b6fba33d362aca2313cc0ca495f28f5ebb9260
Merge: b2214bb 6523cd9
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 13:10:59 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit b2214bb093ec2f759003c488c3c403c8931db914
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 13:09:53 2014 -0500

    go back to my URL until we can figure out what is up with jenkins

commit e7d614172240175a51f580aeaefb6460d21cede9
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 10:52:54 2014 -0500

    try this jenkins

commit 337a3c7704efa7c9809bf373152d711ee55f876c
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Nov 4 16:17:49 2014 +0100

    Rename temp-files under lock to prevent metadata reads while renaming

commit 77d5ba80d0a76efa549dd753b9f114b2f2d2d29c
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 10:07:11 2014 -0500

    continue to treat too-old/too-new as corruption for now

commit 98d0fd2f4851bc50e505a94ca592a694d502c51c
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Nov 4 09:24:21 2014 -0500

    fix last nocommit

commit 643fceed66c8caf22b97fc489d67b4a2a90a1a1c
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Nov 4 14:46:17 2014 +0100

    remove NoSuchDirectoryException

commit 2e43c4feba05cfaf451df70f946c0930cbcc4557
Merge: 93826e4 8163107
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Nov 4 14:38:00 2014 +0100

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 93826e4d56a6a97c2074669014af77ff519bde63
Merge: 7f10129 44e24d3
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Nov 4 12:54:27 2014 +0100

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

    Conflicts:
    	src/main/java/org/elasticsearch/index/store/DistributorDirectory.java
    	src/main/java/org/elasticsearch/index/store/Store.java
    	src/main/java/org/elasticsearch/indices/recovery/RecoveryStatus.java
    	src/test/java/org/elasticsearch/index/store/DistributorDirectoryTest.java
    	src/test/java/org/elasticsearch/index/store/StoreTest.java
    	src/test/java/org/elasticsearch/indices/recovery/RecoveryStatusTests.java

commit 7f10129364623620575c109df725cf54488b3abb
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Nov 4 11:32:24 2014 +0100

    Fix TopHitsAggregator to not ignore the top-level/leaf collector split.

commit 042fadc8603b997bdfdc45ca44fec70dc86774a6
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Nov 4 11:31:20 2014 +0100

    Remove MatchDocIdSet in favor of DocValuesDocIdSet.

commit 7d877581ff5db585a674c95ac391ac78a0282826
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Nov 4 11:10:08 2014 +0100

    Make the and filter use the cost API.

    Lucene 5 ensured that cost() can safely be used, and this will have the benefit
    that the order in which filters are specified is not important anymore (only
    for slow random-access filters in practice).

commit 78f1718aa2cd82184db7c3a8393e6215f43eb4a8
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 23:55:17 2014 -0500

    fix previous eclipse import braindamage

commit 186c40e9258ce32f22a9a714ab442a310b6376e0
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 22:32:34 2014 -0500

    allow child queries to exhaust iterators again

commit b0b1271305e1b6d0c4c4da51a3c54df1aa5c0605
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Nov 3 14:50:44 2014 -0800

    Fix nocommit for mapping output.  index_options will not be printed if
    the field is not indexed.

commit ba223eb85e399c9620a347a983e29bf703953e7a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Nov 3 14:07:26 2014 -0800

    Remove no commit for chinese analyzer provider.  We should have a
    separate issue to address not using this provider on new indexes.

commit ca554b03c4471797682b2fb724f25205cf040c4a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Nov 3 13:41:59 2014 -0800

    Fix stop tests

commit de67c4653ec47dee9c671390536110749d2bb05f
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Nov 3 12:51:17 2014 -0800

    Remove analysis nocommits, switching over to Lucene43*Filters for
    backcompat

commit 50cae9bec72c25c33a1ab8a8931bccb3355171e2
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 15:32:25 2014 -0500

    add ram accounting and TODO lazy-loading (its no worse than master, can be a followup improvement) for suggesters

commit 7a7f0122f138684b312d0f0b03dc2a9c16c15f9c
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 15:11:26 2014 -0500

    bump lucene version

commit cd0cae5c35e7a9e049f49ae45431f658fb86676b
Merge: 446bc09 3c72073
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 14:49:05 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 446bc09b4e8bf4602d3c252b53ddaa0da65cce2f
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 14:46:30 2014 -0500

    remove hack

commit a19d85a968d82e6d00292b49630ef6ff2dbf2f32
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 12:53:11 2014 -0500

    dont create exceptions with circular references on corruption (will open a PR for this)

commit 0beefb9e821d97c37e90ec556d81ac7b00369b8a
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 11:47:14 2014 -0500

    temporarily add craptastic detector for this horrible bug

commit e9f2d298bff75f3d1591f8622441e459c3ce7ac3
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 10:56:01 2014 -0500

    add nocommit

commit e97f1d50a91a7129650b8effc7a9ecf74ca0569a
Merge: c57a3c8 f1f50ac
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 10:12:12 2014 -0500

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit c57a3c8341ed61dca62eaf77fad6b8b48aeb6940
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 10:11:46 2014 -0500

    fix nocommit

commit dd0e77e4ec07c7011ab5f6b60b2ead33dc2333d2
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Nov 3 09:54:09 2014 -0500

    nocommit -> TODO, this is in much more places in the codebase, bigger issue

commit 3cc3bf56d72d642059f8fe220d6f2fed608363e9
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Nov 1 23:59:17 2014 -0700

    Remove nocommit and awaitsfix for edge ngram filter test.

commit 89f115245155511c0fbc0d5ee62e63141c3700c1
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Nov 1 23:57:44 2014 -0700

    Fix EdgeNGramTokenFilter logic for version <= 4.3, and fixed instanceof
    checks in corresponding tests to correctly check for reverse filter when
    applicable.

commit 112df869cd199e36aab0e1a7a288bb1fdb2ebf1c
Author: Robert Muir <rmuir@apache.org>
Date:   Sun Nov 2 00:08:30 2014 -0400

    execute geo disjoint query/filter as intersects

commit e5061273cc685f1252e9a3a9ae4877ec9bce7752
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:58:59 2014 -0400

    remove chinese analyzer from docs

commit ea1af11b8978fcc551f198e24fe21d52806993ef
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:29:00 2014 -0400

    fix ram accounting bug

commit 53c0a42c6aa81aa6bf81d3aa77b95efd513e0f81
Merge: e3bcd3c 6011a18
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:16:29 2014 -0400

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit e3bcd3cc07a4957e12c7b3affc462c31290a9186
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:15:01 2014 -0400

    fix url-email back compat (thanks ryan)

commit 91d6b096a96c357755abee167098607223be1aad
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 22:11:26 2014 -0400

    bump lucene version

commit d2bb9568df72b37ec7050d25940160b8517394bc
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 20:33:07 2014 -0400

    remove nocommit

commit 1d049c471e19e5c457262c7399c5bad9e023b2e3
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 20:28:58 2014 -0400

    fix eclipse to group org/com imports together: without this, its madness

commit 09d8c1585ee99b6e63be032732c04ef6fed84ed2
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Nov 1 14:27:41 2014 -0400

    remove nocommit, if you dont liek it, print assembly and tell me how it can be better

commit 8a6a294313fdf33b50c7126ec20c07867ecd637c
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 20:01:55 2014 +0100

    Remove deprecated usage of DocIdSets.newDocIDSet.

commit 601bee60543610558403298124a84b1b3bbd1045
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 14:13:18 2014 -0400

    maybe one of these zillions of annotations will stop thread leaks

commit 9d3f69abc7267c5e455aefa26db95cb554b02d62
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 14:05:39 2014 -0400

    fix some analysis nocommits

commit 312e3a29c77214b8142d21c33a6b2c2b151acf9a
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 18:28:45 2014 +0100

    Remove XConstantScoreQuery/XFilteredQuery/ApplyAcceptedDocsFilter.

commit 5a0cb9f8e167215df7f1b1fad11eec6e6c74940f
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 17:06:45 2014 +0100

    Fix misleading documentation of DocIdSets.toCacheable.

commit 8b4ef2b5b476fff4c79c0c2a0e4769ead26cf82b
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 17:05:59 2014 +0100

    Fix CustomRandomAccessFilterStrategy to override the right method.

commit d7a9a407a615987cfffc651f724fbd8795c9c671
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 16:21:35 2014 +0100

    Better handle the special case when there is a single SHOULD clause.

commit 648ad389f07e92dfc451f345549c9841ba5e4c9a
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 15:53:38 2014 +0100

    Cut over XBooleanFilter to BitDocIdSet.Builder.

    The idea is similar to what happened to Lucene's BooleanFilter.

    Yet XBooleanFilter is a bit more sophisticated and I had to slightly
    change the way it is implemented in order to make it work. The main difference
    with before is that slow filters are now applied lazily, so eg. if you have 3
    MUST clauses, two with a fast iterator and the third with a slow iterator, the
    previous implementation used to apply the fast iterators first and then only
    check the slow filter for bits which were set in the bit set. Now we are
    computing a bit set based on the fast must clauses and then basically returning
    a BitsFilteredDocIdSet.wrap(bitset, slowClause).

    Other than that, BooleanFilter still uses the bitset optimizations when or-ing
    and and-ind filters.

    Another improvement is that BooleanFilter is now aware of the cost API.

commit b2dad312b4bc9f931dc3a25415dd81c0d9deee08
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 10:18:53 2014 -0400

    clear nocommit

commit 4851d2091e744294336dfade33906c75fbe695cd
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 15:15:16 2014 +0100

    cut over to RoaringDocIdSet

commit ca6aec24a901073e65ce4dd6b70964fd3612409e
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:57:30 2014 +0100

    make nocommit more explicit

commit d0742ee2cb7a6c48b0bbb31580b7fbcebdb6ec40
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 09:55:24 2014 -0400

    fix standardtokenizer nocommit

commit 7d6faccafff22a86af62af0384838391d46695ca
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:54:08 2014 +0100

    fix compilation

commit a038a405c1ff6458ad294e6b5bc469e622f699d0
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:53:43 2014 +0100

    fix compilation

commit 30c9e307b1f5d80e2deca3392c0298682241207f
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:52:35 2014 +0100

    fix compilation

commit e5139bc5a0a9abd2bdc6ba0dfbcb7e3c2e7b8481
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 09:52:16 2014 -0400

    clear nocommit here

commit 85dd2cedf7a7994bed871ac421cfda06aaf5c0a5
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:46:17 2014 +0100

    fix CompletionPostingsFormatTest

commit c0f3781f616c9b0ee3b5c4d0998810f595868649
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 09:38:00 2014 -0400

    add tests for these analyzers

commit 51f9999b4ad079c283ae762c862fd0e22d00445f
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 14:10:26 2014 +0100

    remove nocommit - this is not an issue

commit fd1388fa03e622b0738601c8aeb2dbf7949a6dd2
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Fri Oct 31 14:07:01 2014 +0100

    Remove redundant null check

commit 3d6dd51b0927337ba941a235446b22e8cd500dc3
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Fri Oct 31 14:01:37 2014 +0100

    Removed the work around to prevent p/c error when invoking #iterator() twice, because the custom query filter wrapper now doesn't transform the result to a cache doc id set any more.

    I think the transforming to a cachable doc id set in CustomQueryWrappingFilter isn't needed at all, because we use the DocIdSet only once and because of that is just slowed things down.

commit 821832a537e00cd1216064b379df3e01d2911d3a
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 13:54:33 2014 +0100

    one more nocommit

commit 77eb9ea4c4ea50afb2680c29682ddcb3851a9d4f
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Fri Oct 31 13:52:29 2014 +0100

    Remove cast

commit a400573c034ed602221f801b20a58a9186a06eae
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 13:49:24 2014 +0100

    fix stop filter

commit 51746087cf8ec34c4d20aa05ba8dbff7b3b43eec
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 13:21:36 2014 +0100

    fix changed semantics of FBS.nextSetBit to check for NO_MORE_DOCS

commit 8d0a4e2511310f1293860823fe3ba80ac771bbe3
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 08:13:44 2014 -0400

    do the bogus cast differently

commit 46a5cc5732dea096c0c80ae5ce42911c9c51e44e
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Oct 31 13:00:16 2014 +0100

    I hate it but P/C now passes

commit 580c0c2f82bbeacf217e594f22312b11d1bdb839
Merge: a9d3c00 1645434
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 31 06:54:31 2014 -0400

    fix nocommit/classcast

commit a9d3c004d62fe04989f49a897e6ff84973c06eb9
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 31 08:49:31 2014 +0100

    Update TODO.

commit aa75af0b407792aeef32017f03a6f442ed970baa
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 19:18:25 2014 -0400

    clear obselete nocommits from lucene bump

commit d438534cf41fcbe2d88070e2f27c994625e082c2
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 18:53:20 2014 -0400

    throw classcastexception when ES abuses regular filtercache for nested docs

commit 2c751f3a8feda43ec127c34769b069de21f3d16f
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 18:31:34 2014 -0400

    bump lucene revision, fix tests

commit d6ef7f6304ae262bf6228a7d661b2a452df332be
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 22:37:58 2014 +0100

    fix merge problems

commit de9d361f88a9ce6bb3fba85285de41f223c95767
Merge: 41f6aab f6b37a3
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 22:28:59 2014 +0100

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

    Conflicts:
    	pom.xml
    	src/main/java/org/elasticsearch/Version.java
    	src/main/java/org/elasticsearch/gateway/local/state/meta/MetaDataStateFormat.java

commit 41f6aab388aa80c40b08a2facab2617576203a0d
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 17:48:46 2014 +0100

    fix potiential NPE

commit c4428b12e1ae838b91e847df8b4a8be7f49e10f4
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 17:38:46 2014 +0100

    don't advance iterator in a match(doc) method

commit 28ab948e99e3ea4497c9b1e468384806ba7e1790
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 17:34:58 2014 +0100

    don't advance iterator in a match(doc) method

commit eb0f33f6634fadfcf4b2bf7327400e568f0427bb
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 16:55:54 2014 +0100

    fix GeoUtilsTest

commit 7f711fe3eaf73b6c2268cf42d5a41132a61ad831
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 16:43:16 2014 +0100

    Use a dedicated default index option if field type is not indexed by default

commit 78e3f37ab779e3e1b25b45a742cc86ab5f975149
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 10:56:14 2014 -0400

    disable this test with AwaitsFix to reduce noise

commit 9a590f563c8e03a99ecf0505c92d12d7ab20d11d
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 09:38:49 2014 +0100

    fix lucene version

commit abe3ca1d8bb6b5101b545198f59aec44bacfa741
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Oct 30 09:35:05 2014 +0100

    fix AnalyzingCompletionLookupProvider to wrok with new codec API

commit 464293b245852d60bde050c6d3feb5907dcfbf5f
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 00:26:00 2014 -0400

    don't try to write stuff to tests class directory

commit 031cc6c19f4fe4423a034b515f77e5a0e282a124
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 00:12:36 2014 -0400

    AwaitsFix these known issues to reduce noise

commit 4600d51891e35847f2d344247d6f915a0605c0d1
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 30 00:06:53 2014 -0400

    openbitset lives on

commit 8492bae056249e2555d24acd55f1046b66a667c4
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 23:42:54 2014 -0400

    fixes for filter tests

commit 31f24ce4efeda31f97eafdb122346c7047a53bf2
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 23:12:38 2014 -0400

    don't use fieldcache

commit 8480789942fdff14a6d2b2cd8134502fe62f20c8
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 23:04:29 2014 -0400

    ancient index no longer supported

commit 02e78dc7ebdd827533009f542582e8db44309c57
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 23:37:02 2014 +0100

    fix more tests

commit ff746c6df23c50b3f3ec24922413b962c8983080
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 23:08:19 2014 +0100

    fix all mapper

commit e4fb84b517107b25cb064c66f83c9aa814a311b2
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 22:55:54 2014 +0100

    fix distributor tests and cut over to FileStore API

commit 20c850e2cfe3210cd1fb9e232afed8d4ac045857
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 22:42:18 2014 +0100

    use DOCS_ONLY if index=true and current options == null

commit 44169c108418413cfe51f5ce23ab82047463e4c2
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 22:33:36 2014 +0100

    Fix index=yes|no settings in mappers

commit a3c5f77987461a18121156ed345d42ded301c566
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 21:51:41 2014 +0100

    fix several field mappers conversion from setIndexed to indexOptions

commit df84d736908e88a031d710f98e222be68ae96af1
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 21:33:35 2014 +0100

    fix SourceFieldMapper to be not indexed

commit b2bf01d12a8271a31fb2df601162d0e89924c8f5
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 21:23:08 2014 +0100

    Cut over to .liv files in store and corruption tests

commit 619004df436f9ef05d24bef1b6a7f084c6b0ad75
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 17:05:52 2014 +0100

    fix more tests

commit b7ed653a8b464de446e00456bce0a89e47627c38
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 16:19:08 2014 +0100

    [STORE] Add dedicated method to write temporary files

    Recovery writes temporary files which might not end up in the
    right distributor directories today. This commit adds a dedicated
    API that allows specifying the target file name in order to create the
    tempoary file in the correct directory.

commit 7d574659f6ae04adc2b857146ad0d8d56ca66f12
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 10:28:49 2014 -0400

    add some leniency to temporary bogus method

commit f97022ea7c2259f7a5cf97d924c59ed75ab65b32
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 10:24:17 2014 -0400

    fix MultiCollector bug

commit b760533128c2b4eb10ad76e9689ef714293dd819
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:56:08 2014 +0100

    CheckIndex is now closeable we need to close it

commit 9dae9fb6d63546a6c2427be2a2d5c8358f5b1934
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:45:11 2014 +0100

    s/Lucene51/Lucene50

commit 7aea9b86856a8c1b06a08e7c312ede1168af1287
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:42:30 2014 +0100

    fix BloomFilterPostingsFormat

commit 16fea6fe842e88665d59cc091e8224e8dc6ce08c
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:41:16 2014 +0100

    fix some codec format issues

commit 3d77aa97dd2c4012b63befef3f2ba2525965e8a6
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:30:43 2014 +0100

    fix CodecTests

commit 6ef823b1fde25657438ace1aabd9d552d6ae215e
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:26:47 2014 +0100

    make it compile

commit 9991eee1fe99435118d4dd42b297ffc83fce5ec5
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 09:12:43 2014 -0400

    add an ugly hack for TopHitsAggregator for now

commit 03e768a01fcae6b1f4cb50bcceec7d42977ac3e6
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Oct 29 14:01:02 2014 +0100

    cut over ES090PostingsFormat

commit 463d281faadb794fdde3b469326bdaada25af048
Merge: 0f8740a 8eac79c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 08:30:36 2014 -0400

    Merge branch 'master' into enhancement/lucene_5_0_upgrade

commit 0f8740a782455a63524a5a82169f6bbbfc613518
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 29 01:00:15 2014 -0400

    fix/hack remaining filter and analysis issues

commit df534488569da13b31d66e581456dfd4b55156b9
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Oct 28 23:11:47 2014 -0400

    fix ngrams / openbitset usage

commit 11f5dc3b9887f4da80a0fa1818e1350b30599329
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Oct 28 22:42:44 2014 -0400

    hack over sort comparators

commit 4ebdc754350f512596f6a02770d223e9f5f7975a
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Oct 28 21:27:07 2014 -0400

    compiler errors < 100

commit 2d60c9e29de48ccb0347dd87f7201f47b67b83a0
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Oct 28 03:13:08 2014 -0400

    clear some nocommits around ram usage

commit aaf47fe6c0aabcfb2581dd456fc50edf871da758
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 12:27:34 2014 -0400

    migrate fieldinfo handling

commit ef6ed6d15d8def71cd880d97249678136cd29fe3
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 12:07:13 2014 -0400

    more simple fixes

commit f475e1048ae697dd9da5bd9da445102b0b7bc5b3
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 11:58:21 2014 -0400

    more fielddata ram accounting fixes

commit 16b4239eaa9b4262df258257df4f31d39f28a3a2
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 16:47:32 2014 +0100

    add missing file

commit 5b542fa2a6da81e36a0c35b8e891a1d8bc58f663
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 16:43:29 2014 +0100

    cut over completion posting formats - still some nocommits

commit ecdea49404c4ec4e1b78fb54575825f21b4e096e
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 11:21:09 2014 -0400

    fielddata accountable fixes

commit d43da265718917e20c8264abd43342069198fe9c
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 16:19:53 2014 +0100

    cut over BloomFilterPostings to new API

commit 29b192ba621c14820175775d01242162b88bd364
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 10:22:51 2014 -0400

    fix more analyzers

commit 74b4a0c5283e323a7d02490df469497c722780d2
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 09:54:25 2014 -0400

    fix tests

commit 554084ccb4779dd6b1c65fa7212ad1f64f3a6968
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 14:51:48 2014 +0100

    maintain supressed exceptions on CorruptIndexException

commit cf882d9112c5e8ef1e9f2b0f800f7aa59001a4f2
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 14:47:17 2014 +0100

    commitOnClose=false

commit ebb2a9189ab2f459b7c6c9985be610fd90dfe410
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 14:46:06 2014 +0100

    cut over indexwriter closeing in InternalEngine

commit cd21b3d4706f0b562bd37792d077d60832aff65f
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Oct 27 14:38:10 2014 +0100

    fix constant

commit f93f900c4a1c90af3a21a4af5735a7536423fe28
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 27 09:50:49 2014 -0400

    fix test

commit a9a752940b1ab4699a6a08ba8b34afca82b843fe
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Mon Oct 27 09:26:18 2014 +0100

    Be explicit about the index options

commit d9ee815babd030fa2ceaec9f467c105ee755bf6b
Author: Simon Willnauer <simonw@apache.org>
Date:   Sun Oct 26 20:03:44 2014 +0100

    cut over store and directory

commit b3f5c8e39039dd8f5caac0c4dd1fc3b1116e64ca
Author: Robert Muir <rmuir@apache.org>
Date:   Sun Oct 26 13:08:39 2014 -0400

    more test fixes

commit 8842f2684e3606aae0860c27f7a4c53e273d47fb
Author: Robert Muir <rmuir@apache.org>
Date:   Sun Oct 26 12:14:52 2014 -0400

    tests manual labor

commit c43de5aec337919a3fdc3638406dff17fc80bc98
Author: Robert Muir <rmuir@apache.org>
Date:   Sun Oct 26 11:04:13 2014 -0400

    BytesRef -> BytesRefBuilder

commit 020c0d087a2f37566a1db390b0e044ebab030138
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Sun Oct 26 15:53:37 2014 +0100

    Moved over to BitSetFilter

commit 48dd1b909e6c52cef733961c9ecebfe4f67109fe
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Sun Oct 26 15:53:11 2014 +0100

    Left over Collector api change in ScanContext

commit 6ec248ef63f262bcda400181b838fd9244752625
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Sun Oct 26 15:47:40 2014 +0100

    Moved indexed() over to indexOptions != null or indexOptions == null

commit 9937aebfd8546ae4bb652cd976b3b43ac5ab7a63
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date:   Sun Oct 26 13:26:31 2014 +0100

    Fixed many compile errors. Mainly around the breaking Collector api change in 5.0.

commit fec32c4abc0e3309cf34260c8816305a6f820c9e
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Oct 25 11:22:17 2014 -0400

    more easy fixes

commit dab22531d801800d17a65dc7c9464148ce8ebffd
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Oct 25 09:33:41 2014 -0400

    more progress

commit 414767e9a955010076b0497cc4f6d0c1850b48d3
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Oct 25 06:33:17 2014 -0400

    more progress

commit ad9d969fddf139a8830254d3eb36a908ba87cc12
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 24 14:28:01 2014 -0400

    current state of fun

commit 464475eecb0be15d7d084135ed16051f76a7e521
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Oct 24 11:42:41 2014 -0400

    bump to 5.0 snapshot
2014-11-05 15:48:51 -05:00
Boaz Leskes 1e87604b36 Docs: minor update to resiliency page 2014-11-05 15:00:53 +01:00
Boaz Leskes ddd06c772e Docs: updated resiliency page for 1.4.0 2014-11-05 14:50:52 +01:00
Martijn van Groningen 960fcf4e61 Update the status of the shard info header work. 2014-11-05 12:53:45 +01:00
Clinton Gormley 4d3842311f Docs: Updated ES/JVM versions 2014-11-05 12:41:22 +01:00
Adrien Grand 9ea25df649 Switch to murmurhash3 to route documents to shards.
We currently use the djb2 hash function in order to compute the shard a
document should go to. Unfortunately this hash function is not very
sophisticated and you can sometimes hit adversarial cases, such as numeric ids
on 33 shards.

Murmur3 generates hashes with a better distribution, which should avoid the
adversarial cases.

Here are some examples of how 100000 incremental ids are distributed to shards
using either djb2 or murmur3.

5 shards:
Murmur3: [19933, 19964, 19940, 20030, 20133]
DJB:     [20000, 20000, 20000, 20000, 20000]

3 shards:
Murmur3: [33185, 33347, 33468]
DJB:     [30100, 30000, 39900]

33 shards:
Murmur3: [2999, 3096, 2930, 2986, 3070, 3093, 3023, 3052, 3112, 2940, 3036, 2985, 3031, 3048, 3127, 2961, 2901, 3105, 3041, 3130, 3013, 3035, 3031, 3019, 3008, 3022, 3111, 3086, 3016, 2996, 3075, 2945, 2977]
DJB:     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 900, 900, 900, 900, 1000, 1000, 10000, 10000, 10000, 10000, 9100, 9100, 9100, 9100, 9000, 9000, 0, 0, 0, 0, 0, 0]

Even if djb2 looks ideal in some cases (5 shards), the fact that the
distribution of its hashes has some patterns can raise issues with some shard
counts (eg. 3, or even worse 33).

Some tests have been modified because they relied on implementation details of
the routing hash function.

Close #7954
2014-11-04 16:32:42 +01:00
Clinton Gormley 5797682bd0 Update cluster.asciidoc - fix invalid asciidoc 2014-11-04 15:22:36 +01:00
Clinton Gormley 60eaeb5052 Update cluster.asciidoc
Fixed asciidoc on cluster module page
2014-11-04 14:32:05 +01:00
Clinton Gormley b0e5fb7823 Update zen.asciidoc
Tidied up the "No master block" asciidoc
2014-11-04 14:27:22 +01:00
Martijn Laarman 82278bb7bc [Aggregations] Meta data support
This commit adds the ability to associate a bit of state with each
individual aggregation.

The aggregation response can be hard to stitch back together without
having a reference to the aggregation request. In many cases this is not
available, many json serializer frameworks cache types globally or have a
static deserialisation override mechanism. In these cases making the
original request available, if at all possible, would be a hack.

The old facets returned `_type` which was just enough metadata to know
what the originating facet type in the request was.

This PR takes `_type` one step further by introducing ANY arbitrary meta
data. This could be further <strike>ab</strike>used for instance by
generic/automated aggregations that include UI state (color information,
thresholds, user input states, etc) per aggregation.
2014-11-03 22:32:23 +01:00
Ryan Ernst 7ec31abbb7 Fix missing word in upgrade docs. 2014-11-03 11:44:41 -08:00
printercu 695cd31678 Docs: Add elastics-rb to the list of community clients
Closes #8319
2014-11-02 13:55:21 +01:00
Alexander Reelsen c04fa43587 Docs: Convert markdown to asciidoc in transport profile docs 2014-11-02 08:25:45 +01:00
Aarni Koskela 6011a18381 Docs: Add mention of `hyphenation_patterns_path`
Refs ElasticSearch's HyphenationCompoundWordTokenFilterFactory.java.

Closes #8305
2014-11-01 15:47:53 +01:00
Alexander Reelsen 5eeac2fdf6 Netty: Add HTTP pipelining support
This adds HTTP pipelining support to netty. Previously pipelining was not
supported due to the asynchronous nature of elasticsearch. The first request
that was returned by Elasticsearch, was returned as first response,
regardless of the correct order.

The solution to this problem is to add a handler to the netty pipeline
that maintains an ordered list and thus orders the responses before
returning them to the client. This means, we will always have some state
on the server side and also requires some memory in order to keep the
responses there.

Pipelining is enabled by default, but can be configured by setting the
http.pipelining property to true|false. In addition the maximum size of
the event queue can be configured.

The initial netty handler is copied from this repo
https://github.com/typesafehub/netty-http-pipelining

Closes #2665
2014-10-31 16:30:11 +01:00
Clinton Gormley e56d85439c Update search-template.asciidoc
Clarified using the conditional clause template example as a string
2014-10-31 15:32:14 +01:00
Oliver Eilhard a239935f90 Docs: Add elastic client for Google Go.
Add elastic, an Elasticsearch client for Google Go.

Closes #8302
2014-10-31 14:42:27 +01:00
Clinton Gormley 2569188d25 Update search-template.asciidoc
Fixed asciidoc typo

Closes #8308
2014-10-31 14:40:32 +01:00
astefan 4049154dbc Docs: Document action.replication_type setting
Document action.replication_type setting

Closes #8290
2014-10-31 13:53:34 +01:00
cmpich e57c8b0673 Docs: Update getting-started.asciidoc
Closes #8195
2014-10-29 15:04:13 +01:00
cmpich 36462c0305 Docs: Update getting-started.asciidoc
Closes #8194
2014-10-29 15:01:18 +01:00
Clinton Gormley 8f02c451b8 Update source-field.asciidoc
very minor typofix

Closes #8066
2014-10-29 14:51:05 +01:00
Clinton Gormley 6a7f55b61d Docs: Updated Perl client page to mention async client
Closes #8012
2014-10-29 14:48:56 +01:00
Alex Ksikes 35f55608cc MLT Field Query: remove it from master
The MLT field query is simply replaced by a MLT query set to specififc field.
To simplify code maintenance we should deprecate it in 1.4 and remove it in
2.0.

Closes #8238
2014-10-29 10:19:00 +01:00
Areek Zillur 96f1606cdc Completion Suggester: Fix CompletionFieldMapper to correctly parse weight
- Allows weight to be defined as a string representation of a positive integer

closes #8090
2014-10-28 18:39:02 -04:00
Dmitriy Khvatov 71a90ab4fe Docs: Update multi-get.asciidoc
Duplicate word

Closes #8228
2014-10-28 10:58:47 +01:00
tlrx 8c864cf3f6 Cat Recovery API: Reverting changes introduced with commit e1c75bae87
Adding these 2 headers to the CAT Recovery made the CI tests hanging for a loooong time.

Related to #8041
2014-10-27 20:49:58 +01:00
Zachary Tong f5b2dfd052 Aliases: Throw exception if index is null or missing when creating an alias
Fixes a bug where alias creation would allow `null` for index name, which thereby
applied the alias to _all_ indices.  This patch makes the validator throw an
exception if the index is null.

```bash
POST /_aliases
{
   "actions": [
      {
         "add": {
            "alias": "empty-alias",
            "index": null
         }
      }
   ]
}
```
```json
{
   "error": "ActionRequestValidationException[Validation Failed: 1: Alias action [add]: [index] may not be null;]",
   "status": 400
}
```

The reason this bug wasn't caught by the existing tests is because
the old test for nullness only validated against a cluster which had
zero indices.  The null index is translated into "_all", and since
there are no indices, this fails because the index doesn't exist.
 So the test passes.

However, as soon as you add an index, "_all" resolves and you get the
situation described in the original bug report:  null index is
accepted by the alias, resolves to "_all" and gets applied to everything.

The REST tests, otoh, explicitly tested this bug as a real feature and therefore
passed.  The REST tests were modified to change this behavior.

Fixes #7863
2014-10-27 14:39:01 -04:00
Alex Ksikes 0be5c60bce MLT Query: use ParseField#withAllDeprecated for percent_terms_to_match
Also the parameter was deprecated but not removed so we keep it in the doc and
mark it as deprecated ...

Closes #8241
2014-10-27 17:35:06 +01:00
Alex Ksikes 991f3e2cd3 Docs: fix tags for dfs and new like parameter 2014-10-27 15:42:44 +01:00
Clinton Gormley fbd0403a6f Documented that HTTP pipelining is not supported 2014-10-27 14:49:48 +01:00
Adrien Grand 7ea490dfd1 Aggregations: Return the sum of the doc counts of other buckets.
This commit adds a new field to the response of the terms aggregation called
`sum_other_doc_count` which is equal to the sum of the doc counts of the buckets
that did not make it to the list of top buckets. It is typically useful to have
a sector called eg. `other` when using terms aggregations to build pie charts.

Example query and response:

```json
GET test/_search?search_type=count
{
  "aggs": {
    "colors": {
      "terms": {
        "field": "color",
        "size": 3
      }
    }
  }
}
```

```json
{
   [...],
   "aggregations": {
      "colors": {
         "doc_count_error_upper_bound": 0,
         "sum_other_doc_count": 4,
         "buckets": [
            {
               "key": "blue",
               "doc_count": 65
            },
            {
               "key": "red",
               "doc_count": 14
            },
            {
               "key": "brown",
               "doc_count": 3
            }
         ]
      }
   }
}
```

Close #8213
2014-10-27 12:11:26 +01:00
tlrx e1c75bae87 Cat API: Add node name to _cat/recovery
Add source_node and target_node fields to the recovery cat API. Also fixed and updated the documentation which was not complete concerning fields names.

Closes #8041
2014-10-27 09:47:26 +01:00
Alex Ksikes 4da407a869 MLT Query: versatile 'like' parameter
The MLT query has a lot of parameters. For example, a set of documents is
specified with either `like_text`, `ids` or `docs`, with at least one
parameter required. This commit groups all the document specification
parameters under one called `like`. The syntax is described below and could
easily be extended to allow for new means of specifying document input. The
`like_text`, `ids` and `docs` parameters are deprecated.

As a single piece text:

{
  "query": {
    "more_like_this": {
      "like": "some text here"
    }
  }
}

As a single item:

{
  "query": {
    "more_like_this": {
      "like": {
        "_index": "imdb",
        "_type": "movies",
        "_id": "88247"
      }
    }
  }
}

Or as a mixture of all:

{
  "query": {
    "more_like_this": {
      "like": [
        "Some random text ...",
        {
          "_index": "imdb",
          "_type": "movies",
          "_id": "88247"
        },
        {
          "_index": "imdb",
          "_type": "movies",
          "doc": {
            "title": "Document with an artificial title!"
          }
        }
      ]
    }
  }
}

Closes #8039
2014-10-25 11:04:51 +02:00
David Pilato 62d8b7ab97 Docs: rolling upgrade process seems incorrect
When reading the [rolling upgrade process](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-upgrade.html#rolling-upgrades), you can see that we wrote:

* disable allocation
* upgrade node1
* upgrade node2
* upgrade node3
* ...
* enable allocation

That won't work as after a node has been removed and restarted, no shard will be allocated anymore.
So closing node2 and remaining nodes, won't help to serve index and search request anymore.

We should write:

* disable allocation
* upgrade node1
* enable allocation
* wait for shards being recovered on node1
* disable allocation
* upgrade node2
* enable allocation
* wait for shards being recovered on node2
* disable allocation
* upgrade node3
* enable allocation
* wait for shards being recovered on node3
* disable allocation
* ...
* enable allocation

I think this documentation update should go in 1.3, 1.4, 1.x and master branches.

Closes #8218
Closes #7973.
2014-10-24 16:45:42 +02:00
Marcin Mikosik ed86d925cd Docs: fixed typo in documentation
Closes #8205
2014-10-24 15:27:31 +02:00
Simon Willnauer d5c0a49620 [ROUTING] Add rebalance enabled allocation decider
This commit adds the ability to enable / disable relocations
on an entire cluster or on individual indices for either:

 * `primaries` - only primaries can rebalance
 * `replica` - only replicas can rebalance
 * `all` - everything can rebalance (default)
 * `none` - all rebalances are disabled

similar to the allocation enable / disable functionality.

Relates to #7288
2014-10-23 14:07:13 +02:00
Alex Ksikes c13f5f21de Term Vectors: support for distributed frequencies
Adds distributed frequencies support for the Term Vectors API. A new parameter
called `dfs` is introduced which defaults to `false`.

Closes #8144
2014-10-23 13:59:59 +02:00
Clinton Gormley a8b21f2cd5 Update update-settings.asciidoc
Removed deprecated `cluster.routing.allocation.disable` settings
2014-10-22 12:46:33 +02:00
Clinton Gormley 2d0c440b09 Update cluster.asciidoc
Fixed asciidoc syntax
2014-10-22 12:45:10 +02:00
Brian Kim 58086dd08b Docs: missing quote
fix missing quote

Closes #8176
2014-10-21 12:52:12 +02:00
Peter Dyson b984cb771f Docs: Provide example of deleting a repository
Example of deleting a repository with explanation that snapshots themselves are left untouched.

Closes #8172
2014-10-21 10:10:03 +02:00
Andrei Kolosok c31a783930 Docs: Update filtered-query.asciidoc
Fix mistyping

Closes #8167
2014-10-21 09:45:19 +02:00
Andrei Kolosok 92abfc8e24 Docs: Update minimum-should-match.asciidoc
Add %-sign to examle in the last section

Closes #8157
2014-10-21 09:43:55 +02:00
David Pilato 0ff61e1d6f Add time_zone setting for query_string
Query String query now supports a new `time_zone` option based on JODA time zones.
When using a range on date field, the time zone is applied.

```json
{
"query": {
  "query_string": {
    "text": "date:[2012 TO 2014]",
    "timezone": "Europe/Paris"
  }
 }
}
```

Closes #7880.
2014-10-20 19:09:45 +02:00
Adrien Grand 230c6684a9 Search: Remove partial fields.
Partial fields have been deprecated since 1.0.0Beta1 in favor of _source
filtering. They will be removed in 2.0.
2014-10-20 12:29:30 +02:00
Adrien Grand f4ee3f25e4 Mappings: Store _timestamp by default.
Storing `_timestamp` by default means that under the default configuration, you
would have all the information you need in order to reindex into a different
index.

Close #8139
2014-10-20 12:17:26 +02:00
Clinton Gormley 4da62a33b5 Update plugins.asciidoc
Added Vietnamese Analyser to plugins page

Closes #6647
2014-10-20 11:56:01 +02:00
Ryan Grimm 74586e2867 Docs: Added 'd' to the list of supported units.
Day was missing from the list of supported units in the date math section.

Closes #8151
2014-10-19 21:24:28 +02:00
Michael McCandless 85065f9c8e Core: cutover to Lucene's query rescorer
This is functionally equivalent to before, so there should be no
user-visible impact, except I added a NOTE in the docs warning about
the interaction of pagination and rescoring.

Closes #6232

Closes #7707
2014-10-18 05:25:50 -04:00
Alexander Reelsen 37e606819c Test: Fix netty multiport tests
Ensuring, that even in local mode, netty is started appropriately.
2014-10-17 16:16:04 -07:00
Alexander Reelsen b06b52449f Netty: Support to bind on multiple host/port pairs
This patch allows to create several netty bootstrap, each of which
listening on different ports. This will potentially allow for features
to listen to different network interfaces for node-to-node or node-to-client
communication and is also the base to listen to several interfaces, so that those
can be used to speed up cluster communication in the future.

Closes #8098
2014-10-17 13:55:07 -07:00
Sergii Golubev 028a2b732a Docs: Percolate reference - a typo and a misused word
Closes #8116
2014-10-17 15:26:29 +02:00
Sergii Golubev ae923a81b9 Docs: Percolate `_score` reference
Added missing `_score` word, made the sentence less ambiguous.

Closes #8115
2014-10-17 15:25:02 +02:00
wmx3ng 940534299d Docs: Some Data Cant't be obtained.
In previous case, the first 100 data can't be obtained.

Closes #8103
2014-10-17 15:20:33 +02:00
Sergii Golubev 0fb81d6643 Docs: Update id-field.asciidoc
It is strange to provide an example with `"store" : false` when talking about possibility of enabling the field to be stored.
Broke the line in the mapping in two lines for better readability.
More verbose sentence above the mapping.

Closes #7894
2014-10-17 15:17:24 +02:00
Clinton Gormley 646f7acf5c Docs: Documented the http.*host and transport.*host settings 2014-10-17 15:01:42 +02:00
Andrew O'Brien 33097d901b Docs: Typo: s/by/be/
Closes #8114
2014-10-16 20:51:58 +02:00
Patrick Peschlow 9b03fda5d7 Docs: fix typo
Closes #8111
2014-10-16 20:48:57 +02:00
John Michael Luy 86f6e21cdf Docs: Update query-string-query.asciidoc
Update sample

Closes #8024
2014-10-16 20:21:12 +02:00
Josh Kuhn f4d9cce659 Docs: Added RethinkDB River to list of community plugins
Closes #8016
2014-10-16 20:15:33 +02:00
Son 6f3227db01 Docs: Fix order for PUT _mapping docs
Closes #8083
2014-10-16 18:49:36 +02:00
Derek Haynes 4a7d85f186 Docs: Adding community-contributed Scout plugins
Scout plugins for reporting key node, cluster, and index health metrics.

Closes #7937
2014-10-16 16:47:08 +02:00
Colin Goodheart-Smithe c7971fb0f9 Docs: adds note about using null_value with dynamic templates
Closes #7874
2014-10-16 14:00:00 +01:00
Jun Ohtani 533c1084ec Docs: add the predefined language-specific stopword lists to stop-tokenfilter.asciidoc 2014-10-16 13:20:38 +09:00
Erik Ringsmuth 516b87e129 Docs: Removing escape character
The escape character `\` doesn't work and should be removed from the documentation.

Closes #8029
2014-10-15 21:42:22 +02:00
Patrick Peschlow 340e0efcaa Docs: cross_type query needs to be cross_fields
Closes #8021
2014-10-15 18:27:48 +02:00
Chris Earle 2d8a140ed8 Add file descriptor details to cat/nodes
cat/nodes currently does not report any details related to file descriptors. This adds the current number in use, the maximum number available as well as their ratio (percentage) to cat/nodes as hidden-by-default metrics. In addition, this also adds current heap usage (as a non-percentage of ts max) and ram usage (as a non-percerntage of its max) to allow tools to provide more granularity.

Closes #7652
2014-10-15 10:18:41 -05:00
Chris Earle 9b84ad3c7b Adding "min" score mode to parent-child queries
Support for "max", "sum", and "avg" already existed.
2014-10-15 09:23:36 -05:00
Nik Everett 5dc5849922 Docs: Add plugin to list
This plugin provides trigram accelerated regular expression search for when
you need reasonably fast arbitrary matching.

Closes #7997
2014-10-15 13:56:35 +02:00
Clinton Gormley 6a180d1803 Docs: Update highlighting.asciidoc
Added note about how to highlight on the `_all` field

Closes #7991
2014-10-15 13:45:56 +02:00
Henrik Nordvik fdbb62b1ab Docs: Fix curl statements in query-cache.asciidoc
Closes #7989
2014-10-15 13:16:20 +02:00
sp836490 517caa0c6f Update cjk-bigram-tokenfilter.asciidoc 2014-10-15 11:54:19 +09:00
Clinton Gormley 7e916d0b8b Update completion-suggest.asciidoc
Documented the `size` parameter in the completion suggester query
2014-10-14 18:47:32 +02:00
Jake Buob cb381b4645 Docs: Fix (another) Idaho spelling
Closes #7912
2014-10-14 13:29:35 +02:00
Hang Su dd62e50861 fix mismatched curly bracket 2014-10-14 13:24:44 +02:00
Clinton Gormley eaa0479261 Docs: Added elasticsearch-dsl-py
Closes #7812
Closes #7813
2014-10-14 13:20:38 +02:00
Clinton Gormley 66cd3ad17e Docs: Update object-type.asciidoc to make object type more explicit
Closes #7646
2014-10-14 13:16:37 +02:00
Suyog Rao 82b16ae0ad Doc: Clarify that index.routing.allocation.total_shards_per_node means both primary and replica shards
Closes #8002
2014-10-13 18:08:23 -07:00
Adrien Grand 87497c92ca Docs: Add missing link to the 2.0 migration guide. 2014-10-12 17:54:21 +02:00
Clinton Gormley 0d3d541d62 Docs: Tidied the upgrade indices docs 2014-10-11 16:22:24 +02:00
Ryan Ernst 6ac094c038 Add forgotten include for upgrade docs.
See #7922
2014-10-10 10:55:45 -07:00
Ryan Ernst e2c50c033a Add link to new upgrade docs from indices page.
See #7922.
2014-10-10 10:52:11 -07:00
Adrien Grand 491a48e55b Docs: Remove the note that fielddata doesn't support filtering.
This particular note was about fielddata filtering but could cause confusion
that fields that have doc values enabled cannot be used for filtering (as in
a `filtered_query`).
2014-10-10 10:50:47 +02:00
Martijn van Groningen 5763b24686 Core: Make fetch phase nested doc aware
By letting the fetch phase understand the nested docs structure we can serve nested docs as hits.
The `top_hits` aggregation can because of this commit be placed in a `nested` or `reverse_nested` aggregation.

Closes #7164
2014-10-08 22:21:30 +02:00
Colin Goodheart-Smithe 9ce7ca21e4 Docs: Added breaking changes docs for Indices APIs
Adds the breaking changes defaults for the change of default indices options for the GET Aliases API
2014-10-08 15:08:25 +01:00
Igor Motov 45d3842214 Docs: clarify wait_for_completion flag in snapshot documentation
Closes #7952
2014-10-08 08:43:06 -04:00
David Pilato 80ca8e5ab6 Admin: add total index memory in `_cat/indices`
This patch adds to `_cat/indices` information about memory usage per index by adding memory used by FieldData, IdCache, Percolate, Segments (memory, index writer, version map).

```
% curl 'localhost:9200/_cat/indices?v&h=i,tm'
i     tm
wiki  8.1gb
test  30.5kb
user  1.9mb
```

Closes #7008
2014-10-08 13:38:00 +02:00
Igor Motov 2359e33b1e Docs: fix formatting in snapshots documentation 2014-10-07 12:33:21 -04:00
Ryan Ernst c06c10bbb0 Remove deprecations from master (follow up to #7922) 2014-10-07 08:35:11 -07:00
Ryan Ernst c021f22523 Add Upgrade API
This commit does the following:
* Add the new API at the rest layer, being backed by the optimize API
  with upgrade flag, and segments api to find upgrade status.
* Add `upgrade` flag to optimize API, and deprecate `force` flag (will
  remove in master)
* Add test for both synchronous and async upgrade

closes #7884
closes #7922
2014-10-07 08:09:50 -07:00
Igor Motov 555bfcb02b [SNAPSHOT] Add repository validation
Fixes #7096
2014-10-07 10:50:16 -04:00
David Pilato 6ae6a078de Search: add `format` support for date range filter and queries
When the date format is defined in mapping, you can not use another format when querying using range date query or filter.

For example, this won't work:

```
DELETE /test

PUT /test/t/1
{
  "date": "2014-01-01"
}

GET /test/_search
{
  "query": {
    "filtered": {
      "filter": {
        "range": {
          "date": {
            "from": "01/01/2014"
          }
        }
      }
    }
  }
}
```

It causes:

```
Caused by: org.elasticsearch.ElasticsearchParseException: failed to parse date field [01/01/2014], tried both date format [dateOptionalTime], and timestamp number
```

It could be nice if we can support at query time another date format just like we support `analyzer` at search time on String fields.

Something like:

```
GET /test/_search
{
  "query": {
    "filtered": {
      "filter": {
        "range": {
          "date": {
            "from": "01/01/2014",
            "format": "dd/MM/yyyy"
          }
        }
      }
    }
  }
}
```

Same for queries:

```
GET /test/_search
{
  "query": {
    "range": {
      "date": {
        "from": "01/01/2014",
        "format": "dd/MM/yyyy"
      }
    }
  }
}
```

Closes #7189.
2014-10-06 15:59:18 +02:00
Colin Goodheart-Smithe 6cf371395a Aggregations: makes script params consistent with other APIs in scripted_metric
This change removes the script_type parameter form the Scripted Metric Aggregation and adds support for _file and _id suffixes to the init_script, map_script, combine_script and reduce_script parameters to make defining the source of the script consistent with the other APIs which use the ScriptService
2014-10-06 09:07:25 +01:00
Alex Ksikes 349b7a3a8b Term Vectors/MLT Query: support for different analyzers than default at field
This adds a `per_field_analyzer` parameter to the Term Vectors API, which
allows to override the default analyzer at the field. If the field already
stores term vectors, then they will be re-generated. Since the MLT Query uses
the Term Vectors API under its hood, this commits also adds the same ability
to the MLT Query, thereby allowing users to fine grain how each field item
should be processed and analyzed.

Closes #7801
2014-10-03 16:40:17 +02:00
Alex Ksikes c4830cf862 Term Vectors: support for realtime
By default term vectors are now realtime, as opposed to previously near
realtime. If they are not found in the index, they will be generated on the
fly. The document is fetched from the transaction log and treated as an
artificial document. One can set `realtime` parameter to `false` in order to
disable this functionality. This consequently makes the MLT query realtime in
fetching documents, as it previsouly used to be before switching from using
the multi get API to the mtv API.

Closes #7846
2014-10-03 09:26:47 +02:00
Clinton Gormley 3267c2a2bf Docs: Updated the resiliency docs to point to the DiscoveryWithServiceDisruptions class 2014-10-02 21:08:32 +02:00
Clinton Gormley 12265aae02 Docs: Fixed issue link in doc values section of resiliency status 2014-10-02 13:34:27 +02:00
Clinton Gormley 1c7f4ca513 Updated resiliency docs to remove improve_zen branch and update link to dakrone's repo 2014-10-01 18:16:13 +02:00
Clinton Gormley fb18e2e9dd Added resiliency page to docs 2014-10-01 16:16:32 +02:00
Clinton Gormley ea7ed51014 Docs: Typoes on the range filter page 2014-09-30 19:28:35 +02:00
Martijn van Groningen fca406415d Docs: Document the most important changes to zen discovery.
Closes #7746
2014-09-30 13:05:45 +02:00
Alex Ksikes b118558962 MLT Query: Support for artificial documents
Previously, the only way to specify a document not present in the index was to
use `like_text`. This would usually lead to complex queries made of multiple
MLT queries per document field. This commit adds the ability to the MLT query
to directly specify documents not present in the index (artificial documents).
The syntax is similar to the Percolator API or to the Multi Term Vector API.

Closes #7725
2014-09-29 15:49:13 +02:00
Alex Ksikes 5014158d6b MLT Query: use minimum should match more extensive syntax
The minimum number of optional should clauses of the generated query to match
can now be set using the more extensive minimum should match syntax. This
makes the `percent_terms_to_match` parameter deprecated, and replaced in favor
to a new `minimum_should_match` parameter.

Closes #7898
2014-09-29 11:14:56 +02:00
Paul Pearcy 336ef4d468 Clarify s3 snapshot compress behavior
Clarify s3 snapshot compress behavior only applies to metadata and no index files.

Closes #7654
2014-09-28 11:11:19 +02:00
Jake Buob 5ba44b479b Fix Idaho spelling
Doesn't have two `d`s.

Closes #7802
2014-09-28 11:09:48 +02:00
HenrikOssipoff 1445dd2308 Remove comma in JSON
Closes #7827
2014-09-28 11:08:09 +02:00
Christian Verkerk 3efa342ee4 Improve sentence structure
Closes #7842
2014-09-28 11:07:15 +02:00
Thom Dixon 2d7264cd03 GeoJSON is lon,lat
Although emphasized in the text, the example was backwards.

Closes #7843
2014-09-28 11:05:42 +02:00
mdzor 4b3f66e585 Update suggesters.asciidoc
A request was malformed

Closes #7867
2014-09-28 11:04:28 +02:00
Clinton Gormley de0cca4cef Docs: Fixed some deleted refs 2014-09-26 21:35:57 +02:00
Clinton Gormley cb00d4a542 Docs: Removed all the added/deprecated tags from 1.x 2014-09-26 21:04:42 +02:00
Clinton Gormley 091578d117 Update stemmer-tokenfilter.asciidoc
Change the `minimal_english` link to a publicly accessible URL
2014-09-25 20:29:12 +02:00
Sergii Golubev 059d9f757a Docs: bad text wrapping
On the page http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-synonym-tokenfilter.html

even on a huge monitor the text is being wrapped the next way
```
mapping:
ipod, i-pod, i pod => ipod, i-pod, i pod
mapping:
ipod, i-pod, i pod => ipod
```

So one can think that "mapping:" is not in comment and is a part of syntax. But the lines are less than 80 chars, so perhaps the problem is in the page layout and there may be some other pages in the reference where the text is also being wrapped in an undesirable way.

Closes #7739
2014-09-25 19:43:23 +02:00
Cristiano Fontes 3a008c7355 Docs: In the code there is the `d` option.
Documenting a behaviour that is present in the code and works.

Closes #7794
2014-09-25 19:08:59 +02:00
corochoone 57b2e8bfef Update getting-started.asciidoc
zip and tar is archive, deb and rpm is package.

Closes #7749
2014-09-25 13:48:53 +02:00
Shay Banon a82d486bda Add a listener thread pool
Today, when executing an action (mainly when using the Java API), a listener threaded flag can be set to true in order to execute the listener on a different thread pool. Today, this thread pool is the generic thread pool, which is cached. This can create problems for Java clients (mainly) around potential thread explosion.
Introduce a new thread pool called listener, that is fixed sized and defaults to the half the cores maxed at 10, and use it where listeners are executed.
relates to #5152
closes #7837
2014-09-25 11:25:13 +02:00
Alex Ksikes 51bf3e6730 MLT Query: fix percent_terms_to_match
The parameter `percent_terms_to_match` (percentage of terms that must match in
the generated query) was wrongly set to the top level boolean query. This
would lead to zero or all results type of situations. This commit ensures that
the parameter is indeed applied to the query of generated terms.

Closes #7754
2014-09-25 09:56:53 +02:00
Martijn van Groningen 70303be50c Docs: Mention that that p/c queries and filters do not work in the delete-by-query api. 2014-09-25 09:42:14 +02:00
corochoone 3e8c49e98e Update upgrade.asciidoc
Fix markup bug and change "deb" to "dpkg". (Package manager called dpkg, not deb in Debian distribution)

Closes #7748
2014-09-24 21:09:13 +02:00
nandakishore15 ee2503d01d Docs: Update update.asciidoc
Closes #7498
2014-09-24 20:54:09 +02:00
Zachary Wasserman e3c7f28003 Docs: Update thrift.asciidoc
Fix typos, add clarifications and link.

Closes #7678
2014-09-24 20:50:30 +02:00
Clinton Gormley df8b949c45 Docs: Typos on range filter
Closes #7713
2014-09-24 20:19:05 +02:00
David Pilato f2be8d83dc Docs: fix typo 2014-09-23 23:41:07 +02:00
David Pilato 046a3a02f5 Docs: fix no callouts refer to list item 3 2014-09-23 13:30:36 +02:00
David Pilato cdd4d00f46 Docs: java add static imports for query and filter builders
Also move and add inline comments to references

Closes #6920.
2014-09-23 12:39:57 +02:00
David Pilato f0dc8a8ffb Document the Java BulkProcessor
Closes #7638.
2014-09-22 15:32:29 +02:00
David Pilato 53c0e92801 Docs: IndexResponse.matches() does not exist anymore
Since 1.0, percolator has been redesigned so percolator is not applied anymore at index time.

Closes #7548.
2014-09-22 15:20:31 +02:00
Elie A 40608ce266 Update java source example
From the version 1.0 FilterBuilders and QueryBuilders are not part from org.elasticsearch.index.query.xcontent package no more.

Closes #7701.

(cherry picked from commit 32d4200)
2014-09-22 14:37:27 +02:00
Lee Hinman 4185566e93 Add option to take currently relocating shards' sizes into account
When using the DiskThresholdDecider, it's possible that shards could
already be marked as relocating to the node being evaluated. This commit
adds a new setting `cluster.routing.allocation.disk.include_relocations`
which adds the size of the shards currently being relocated to this node
to the node's used disk space.

This new option defaults to `true`, however it's possible to
over-estimate the usage for a node if the relocation is already
partially complete, for instance:

A node with a 10gb shard that's 45% of the way through a relocation
would add 10gb + (.45 * 10) = 14.5gb to the node's disk usage before
examining the watermarks to see if a new shard can be allocated.

Fixes #7753
Relates to #6168
2014-09-19 12:36:51 +02:00
Colin Goodheart-Smithe f0e9b7b8ef [DOC] Add GET Alias API note to breaking changes
Note explains that GET Alias API now supports IndicesOptions and will error if a index is missing
2014-09-18 15:09:01 +01:00
Colin Goodheart-Smithe 8a70b115f2 Aggregations: More consistent response format for scripted metrics aggregation
Changes the name of the field in the scripted metrics aggregation from 'aggregation' to 'value' to be more in line with the other metrics aggregations like 'avg'
2014-09-17 11:46:26 +01:00
Jordan Snodgrass 6246aac9ab Docs: Indicate that the Children Aggregation is coming in 1.4.0 2014-09-17 09:22:02 +02:00
Colin Goodheart-Smithe d4e83df3b8 Aggregations: Adds ability to sort on multiple criteria
The terms aggregation can now support sorting on multiple criteria by replacing the sort object with an array or sort object whose order signifies the priority of the sort. The existing syntax for sorting on a single criteria also still works.

Contributes to #6917
Replaces #7588
2014-09-15 11:08:29 +01:00
markharwood 3c8f8cc090 Aggs enhancement - allow Include/Exclude clauses to use array of terms as alternative to a regex
Closes #6782
2014-09-12 15:28:03 +01:00
Clinton Gormley ab250164f0 Update update.asciidoc
The upserts section should be on the same page as updates.
2014-09-12 07:46:03 +02:00
Colin Goodheart-Smithe 8238388bad [DOCS] clarification of breaking changes to 1.4 due to GET index API 2014-09-11 15:04:04 +01:00
Lee Hinman 1dd26888f6 [DOCS] Additional documentation for _score accessing
Closes #7043
2014-09-11 12:53:25 +02:00
Colin Goodheart-Smithe 5fe782b784 Indices API: Added GET Index API
Returns information about settings, aliases, warmers, and mappings. Basically returns the IndexMetadata. This new endpoint replaces the /{index}/_alias|_aliases|_mapping|_mappings|_settings|_warmer|_warmers and /_alias|_aliases|_mapping|_mappings|_settings|_warmer|_warmers endpoints whilst maintaining the same response formats.  The only exception to this is on the /_alias|_aliases|_warmer|_warmers endpoint which will now return a section for 'aliases' or 'warmers' even if no aliases or warmers exist. This backwards compatibility change is documented in the reference docs.

Closes #4069
2014-09-11 11:19:21 +01:00
Ulrich Reffle 4ed52073fa Docs: A simple typo.
Closes #7535
2014-09-11 11:17:49 +02:00
Clinton Gormley d88577d097 Docs: Added recommended Java versions to the docs 2014-09-11 11:13:55 +02:00
Nik Everett 2bc58d5f77 Docs: Fix misnamed setting
The settings is `index.merge.policy.reclaim_deletes_weight` not
`index.reclaim_deletes_weight`.

Closes #7676
2014-09-11 10:41:23 +02:00
Adrien Grand ccb3d21781 Bulk UDP: Removal.
This feature is rarely used. Removing it will help reduce the moving parts
of Elasticsearch and focus on the core.

Close #7595
2014-09-11 09:52:09 +02:00
smayzak 65a0ca021d The description was incorrect
Looked like a copy and paste from another aggregation
2014-09-10 16:05:03 +02:00
smayzak 6416f5d3d0 Fixing some grammar 2014-09-10 16:05:03 +02:00
David Pilato 7fdd3651fa [docs] Fix typo: resonable - reasonable 2014-09-10 15:57:57 +02:00
Martijn van Groningen 52f1ab6e16 Core: Added the `index.query.parse.allow_unmapped_fields` setting to fail queries if they refer to unmapped fields.
The percolator and filters in aliases by default enforce strict query parsing.

Closes #7335
2014-09-09 15:00:47 +02:00
ansj fee832df66 Docs: Update plugins.asciidoc
add search by sql

Closes #7525
2014-09-09 14:57:38 +02:00
Alexander Reelsen bd0eb32d9c CORS: Disable by default
In order to deliver a more secure out-of-the-box configuration this commit
disables cross-origin resource sharing by default.

Closes #7151
2014-09-09 11:09:03 +02:00
Colin Goodheart-Smithe b127b52fd3 Revert "Aggregations: Adds ability to sort on multiple criteria"
This reverts commit bfedd11ffa.
2014-09-08 20:27:19 +01:00
Colin Goodheart-Smithe bfedd11ffa Aggregations: Adds ability to sort on multiple criteria
The terms aggregation can now support sorting on multiple criteria by replacing the sort object with an array or sort object whose order signifies the priority of the sort. The existing syntax for sorting on a single criteria also still works.

Contributes to #6917
2014-09-08 15:20:33 +01:00
Joao Duarte a6e0f02220 Doc: Fixes node client section name on Java API Doc
Closes #7539
2014-09-08 11:20:49 +02:00
Tanvir Alam c7d0c3ea18 Docs: fixed typo
Closes #7544
2014-09-08 10:50:59 +02:00
Clinton Gormley e0684c4f6e Docs: Update regexp-syntax.asciidoc
Closes #7419
2014-09-07 11:42:33 +02:00
Clinton Gormley 46cda7db09 Docs: Clarify that refresh on update just refreshes the relevant shards 2014-09-07 11:41:05 +02:00
Clinton Gormley 3bda779336 Docs: Update scripting.asciidoc
Clarify that doc[...] does not return objects

Closes #7384
2014-09-07 11:39:33 +02:00
Clinton Gormley 51cec43d3c Docs: Clarify that refresh on get/update/delete just refreshes the relevant shards
Closes #7359
2014-09-07 11:36:52 +02:00
Clinton Gormley 0578a703df Docs: Update prefix-query.asciidoc
Typo (missing word)

Closes #7283
2014-09-07 11:32:05 +02:00
Clinton Gormley cf001d6837 Docs: Adds missing explanation to multi-match
Closes #7281
2014-09-07 11:30:52 +02:00
pachanta 8bae96485e Docs: Update span-not-query.asciidoc
The example given does not clearly explain what the query does.

Closes #7049
2014-09-07 11:28:19 +02:00
Clinton Gormley 1bdf79e527 Docs: Added explanation of how to do multi-field terms agg
Closes #5100
2014-09-07 11:09:52 +02:00
Dominik Surowiecki 7c2490b2ad Docs: Wrong class name
Was QFilterBuilders, should be FilterBuilders.

Closes #7611
2014-09-07 08:46:41 +02:00
shrinidhichaudhari 13e3a5e99c Docs: Update cardinality-aggregation.asciidoc
Closes #7516
2014-09-06 20:45:45 +02:00
Simon Willnauer 7f32e8c707 [STORE] Simplify reading / writing from and to BlobContainer
BlobContainer used to provide async APIs which are not used
internally. The implementation of these APIs are also not async
by nature and neither is any of the pluggable BlobContainers. This
commit simplifies the API to a simple input / output stream and
reduces the hierarchy of BlobContainer dramatically.
NOTE: This is a breaking change!

Closes #7551
2014-09-05 21:40:20 +02:00
Alex Ksikes 07d741c2cb Term Vectors: Support for artificial documents
This adds the ability to the Term Vector API to generate term vectors for
artifical documents, that is for documents not present in the index. Following
a similar syntax to the Percolator API, a new 'doc' parameter is used, instead
of '_id', that specifies the document of interest. The parameters '_index' and
'_type' determine the mapping and therefore analyzers to apply to each value
field.

Closes #7530
2014-09-05 07:42:43 +02:00
Clinton Gormley c88f55ab52 Docs: Fixed typo in windows-as-a-service 2014-09-04 16:12:46 +02:00
Nik Everett 7bcd09a134 [docs] fix typo in language analyzer docs 2014-09-04 09:33:00 +02:00
Adrien Grand 4bfad644b3 Aggregations: Forbid usage of aggregations in conjunction with search_type=SCAN.
Aggregations are collection-wide statistics, which is incompatible with the
collection mode of search_type=SCAN since it doesn't collect all matches on
calls to the search API.

Close #7429
2014-09-03 09:03:01 +02:00
Adrien Grand 203e80e650 Aggregations: Only return aggregations on the first page when scrolling.
Aggregations are collection-wide statistics so they would always be the same.
In order to save CPU/bandwidth, we can just return them on the first page.

Same as #1642 but for aggregations.
2014-09-03 09:03:01 +02:00
Robert Muir 395744b0d2 [Analysis] Add missing docs for latvian analysis 2014-09-02 19:22:59 -04:00
Clinton Gormley a059a6574a Update reverse-nested-aggregation.asciidoc
Fixed reverse nested example

Closes #7463
2014-09-02 11:40:41 +01:00
mikemccand 53ce7ad764 fix typo 2014-09-01 16:07:05 -04:00
Britta Weber c5ff70bf43 function_score: add optional weight parameter per function
Weights can be defined per function like this:

```
"function_score": {
    "functions": [
        {
            "filter": {},
            "FUNCTION": {},
            "weight": number
        }
        ...
```
If `weight` is given without `FUNCTION` then `weight` behaves like `boost_factor`.
This commit deprecates `boost_factor`.

The following is valid:

```
POST testidx/_search
{
  "query": {
    "function_score": {
      "weight": 2
    }
  }
}
POST testidx/_search
{
  "query": {
    "function_score": {
      "functions": [
        {
          "weight": 2
        },
        ...
      ]
    }
  }
}
POST testidx/_search
{
  "query": {
    "function_score": {
      "functions": [
        {
          "FUNCTION": {},
          "weight": 2
        },
        ...
      ]
    }
  }
}
POST testidx/_search
{
  "query": {
    "function_score": {
      "functions": [
        {
          "filter": {},
          "weight": 2
        },
        ...
      ]
    }
  }
}
POST testidx/_search
{
  "query": {
    "function_score": {
      "functions": [
        {
          "filter": {},
          "FUNCTION": {},
          "weight": 2
        },
        ...
      ]
    }
  }
}
```

The following is not valid:

```
POST testidx/_search
{
  "query": {
    "function_score": {
      "weight": 2,
      "FUNCTION(including boost_factor)": 2
    }
  }
}

POST testidx/_search
{
  "query": {
    "function_score": {
      "functions": [
        {
          "weight": 2,
          "boost_factor": 2
        }
      ]
    }
  }
}
````

closes #6955
closes #7137
2014-09-01 11:04:40 +02:00
Britta Weber 92937c7770 Docs: fix formula typo in documentation of the gauss decay in function_score
closes #7506
2014-09-01 09:46:13 +02:00
Adrien Grand 8e1d3d56b3 Docs: Replace added[1.4.0] with coming[1.4.0] since 1.4 is not released yet. 2014-08-29 11:57:22 +02:00
londocr 1213eec834 Spelling error of aggregation 2014-08-28 08:57:12 +02:00
Ryan Ernst 65afa1d93b FunctionScore: Refactor RandomScoreFunction to be consistent, and return values in rang [0.0, 1.0]
RandomScoreFunction previously relied on the order the documents were
iterated in from Lucene. This caused changes in ordering, with the same
seed, if documents moved to different segments. With this change, a
murmur32 hash of the _uid for each document is used as the "random"
value. Also, the hash is adjusted so as to only return values between
0.0 and 1.0 to enable easier manipulation to fit into users' scoring
models.

closes #6907, #7446
2014-08-27 08:37:25 -07:00
Peter Johnson @insertcoffee 10197936da Docs: typo?
Closes #7458
2014-08-26 15:26:34 +02:00
Alexander Reelsen f2aa4a38bc Docs: Added link to clarify meaning of filtering in fielddata context 2014-08-26 12:00:06 +02:00
mikemccand 783a9cbb18 Stats: add segments.index_writer_max_memory to see index writer's max RAM usage before buffered documents must be written to a new segment
Closes #7438

Closes #7440
2014-08-25 14:43:09 -04:00
Lee Hinman c5c11b1837 [DOCS] Document NOT as a simple_query_string flag
Fixes #7426
2014-08-25 10:27:24 +02:00
Alex Ksikes e78694ae82 More Like This Query: defaults to all possible fields for items
Items with no specified field now defaults to all the possible fields from the
document source. Previously, we had required 'fields' to be specified either
as a top level parameter or for each item. The default behavior is now similar
to the MLT API.

Closes #7382
2014-08-22 15:07:22 +02:00
Greg Marzouka 887bd0a9c2 [DOCS] Update to the new {index}/_mapping/{type} format 2014-08-21 13:56:11 -04:00
Britta Weber 2a32cf338e Docs: _ttl ignores all parameters except for enabled and default.
Also, add a line about that it can never be disabled after it was enabled.
2014-08-21 16:16:02 +02:00
Alex Ksikes f1a6b4e9fe More Like This Query: Switch to using the multi-termvectors API
The term vector API can now generate term vectors on the fly, if the terms are
not already stored in the index. This commit exploits this new functionality
for the MLT query. Now the terms are directly retrieved using multi-
termvectors API, instead of generating them from the texts retrieved using the
multi-get API.

Closes #7014
2014-08-21 12:18:21 +02:00
Adrien Grand ea96359d82 Facets: Removal from master.
Close #7337
2014-08-21 10:34:39 +02:00
Alex Ksikes 62ef4a30dc Term vector API: return 'found: false' for docs between index and refresh
Closes #7121
2014-08-21 09:58:49 +02:00
Colin Goodheart-Smithe 7f943f0296 Aggregations: Scriptable Metrics Aggregation
A metrics aggregation which runs specified scripts at the init, collect, combine, and reduce phases

Closes #5923
2014-08-20 18:17:27 +01:00
Shay Banon 3a52296358 Warmer (search) to support query cache
allow for search based warmer to support query cache flag on the search request, and use the index level query caching flag if set.
closes #7326
2014-08-20 09:31:29 -07:00
Martijn van Groningen 383e64bd5c Aggregations: Add `children` bucket aggregator that is able to map buckets between parent types and child types using the already builtin parent/child support.
Closes #6936
2014-08-19 12:40:51 +02:00
Britta Weber 639692943f Docs: Document distance type and sort mode for many to many geo_points
closes #7280
2014-08-18 16:15:55 +02:00
Lee Hinman 99b0faed14 [DOCS] document simple_query_string's `lenient` option 2014-08-18 15:56:06 +02:00
Colin Goodheart-Smithe b2286915cd Geo: Adds support for GeoJSON GeometryCollection
Closes #2796
2014-08-18 13:55:39 +01:00
Clinton Gormley 76b7b68605 Docs: Fixed header levels in setup/dir-layout.asciidoc 2014-08-18 13:38:28 +02:00
Clinton Gormley e6b82bb15d Docs: Fixed bad ID 2014-08-18 13:19:36 +02:00
Xiao Yu 4f417aa774 Docs: Update nested-filter.asciidoc
Fix whitespace to standardize on spaces for indents on both code examples for readability.

Closes #6568
2014-08-18 13:09:00 +02:00
Britta Weber aa1dbc0778 Docs: Fix ambigous explanation of the "fields" parameter in `query_string` query
Closes #7292
2014-08-18 13:06:48 +02:00
Carsten Brandt 6a6c45e2d0 Docs: Update snapshots.asciidoc
Added a link so you directly get to the page that tells how to open/close an index.

Closes #7286
2014-08-18 13:04:25 +02:00
Konrad Feldmeier 3b3e2ed5e9 Docs: Remove the 'Factor' paragraph to reflect #6490
The current implementation of 'date_histogram' does not understand
the `factor` parameter. Since the docs shouldn't raise false hopes,
I removed the section.

Closes #7277
2014-08-18 13:02:15 +02:00
Mpampis Kostas bf8c4f1155 Docs: Remove phrase duplication in api-conventions.asciidoc
Closes #7263
2014-08-18 13:01:12 +02:00
Mpampis Kostas 55b642abc5 Docs: Fix typo in phrase-suggest.asciidoc
Closes #7262
2014-08-18 13:00:30 +02:00
rdpatil4 a0b74695b7 Docs: Update frontends.asciidoc
Closes #7224
2014-08-18 12:59:31 +02:00
Krishna Raj 0f3ebc0876 Docs: Update integrations.asciidoc
Closes #7193
2014-08-18 12:57:45 +02:00
Clinton Gormley 24e66597fa Update repositories.asciidoc
Added apt-get update to the commands for clarity.

Closes #7192
2014-08-18 12:57:08 +02:00
Akshay Kapoor 89088678c3 Docs: Updated typo in getting-started.asciidoc
A very small typo in the description.

Closes #7134
2014-08-18 12:42:23 +02:00
Clinton Gormley 9dfede8cbb Update search-template.asciidoc
Remove extra commas in template query ;-)

Closes #7033
2014-08-18 12:35:18 +02:00
Clinton Gormley 6477e13c77 Typo 2014-08-18 12:30:49 +02:00
smayzak 8449128032 error in code
The top-tags and terms were reversed.
2014-08-18 12:28:53 +02:00
Mark Walkom d75aabc5aa Docs: Added tables for directory locations for deb+rpm and zip+tar.gz installs
Closes #6508
2014-08-18 12:21:25 +02:00
Chris Allen f4328dcd45 Docs: Added Bloodhound Haskell client/DSL to docs
Closes #6031
2014-08-18 12:10:45 +02:00
Robert Muir 5c7cefa292 Analysis: Add keep_types for filtering by token type 2014-08-15 09:28:12 -04:00
Colin Goodheart-Smithe f4d75f0212 REST API: Allows all options for expand_wildcards parameter
This change means that the default settings for expand_wildcards are only applied if the expand_wildcards parameter is not specified rather than being set upfront. It also adds the none and all options to the parameter to allow the user to specify no expansion and expansion to all indexes (equivalent to 'open,closed')

Closes #7258
2014-08-15 12:50:11 +01:00
Gheorghita Catalin Bordianu ca77db1c83 Fixed a typo 2014-08-14 16:27:30 +02:00
Areek Zillur 0b6734aa40 [DOCS] Clarify Completion Suggester output deduplication 2014-08-13 11:09:18 -04:00
Lee Hinman 4dc060527c Add GroovyCollections to the sandbox whitelist
Also clarify in the docs that changing the whitelist/blacklist settings
replace the list, they don't add to it.

Fixes #7089
Fixes #7088
2014-08-13 14:47:49 +02:00
Lee Hinman 59c1e94d90 [DOCS] mention the type of caches that warmers load 2014-08-13 11:23:51 +02:00
Colin Goodheart-Smithe 2906d3e6dc Core: Store index creation time in index metadata
This change stores the index creation time in the index metadata when an index is created.  The creation time cannot be changed but can be set as part of the create index request to allow for correct creation times for historical data.

Closes #7119
2014-08-12 21:34:50 +01:00
Colin Goodheart-Smithe 36083cb27f [DOCS] Added section describing how to return only agg results
Closes #5875
2014-08-11 11:31:01 +01:00
Clinton Gormley 34830368d9 Update clearcache.asciidoc
Changed `field_data` to `fielddata` to be consistent with the indices stats output

Closes #2888
2014-08-08 13:03:59 +02:00
Britta Weber d49ed93488 Docs: md -> asciidoc 2014-08-08 11:25:14 +02:00
baldurh 700e50fc6c Update update.asciidoc
Closes #6577
2014-08-07 21:07:04 +02:00
Elie A e1a2d76626 Fixing typo in the first JSON example
Closes #7172
2014-08-07 14:06:09 +02:00
Adrien Grand a242a63817 [DOCS] Remove the section about codecs.
This documentation was dangerous because it felt like it was possible to gain
substantial performance by just switching the codec of the index.

However, non-default codecs are dangerous to use since they are not supported
in terms of backward compatibility, and most improvements that they bring have
been folded into the default codec anyway (for example, the default codec
"pulses" postings lists that contain a single document).
2014-08-07 11:24:44 +02:00
Colin Goodheart-Smithe e6632ec63e [DOCS] fixed title for filters aggregation documentation 2014-08-07 08:37:43 +01:00
Clinton Gormley 7b0b315b71 Tidied up the filters agg docs and added a coming[] tag 2014-08-07 09:03:23 +02:00
Clinton Gormley e7f1aa4f4f Documented the query cache module
Related to #7161 and #7167
2014-08-06 11:55:11 +02:00
markharwood e6b459cb9f Update API enhancement - add support for scripted upserts.
In the case of inserts the UpdateHelper class will now allow the script used to apply updates to run on the upsert doc provided by clients. This allows the logic for managing the internal state of the data item to be managed by the script and is not reliant on clients performing the initialisation of data structures managed by the script.

Closes #7143
2014-08-05 16:52:44 +01:00
Alexander Reelsen 35e67c84fa CORS: Allowed to configure allow-credentials header to work via SSL
This adds support to return the "Access-Control-Allow-Credentials" header
if needed, so CORS will work flawlessly with authenticated applications.

Closes #6380
2014-08-05 17:33:06 +02:00
Konrad Feldmeier 657b954528 Resolve wording inconsistency
AND and OR filter docs talk about different targets for the operators. I
believe that both should be described in terms of modifying other 'filters'.
I also added articles for easier (human) parsing. This fixes #4762

Closes #7165
2014-08-05 17:16:53 +02:00
Jun Ohtani 5be8aecd10 add Proxy setting for using plugin command
Closes #7150
2014-08-05 12:44:02 +02:00
David Pilato 873a45eaba Search: add time zone setting for relative date math in range filter/query
Filters and Queries now supports `time_zone` parameter which defines which time zone should be applied to the query or filter to convert it to UTC time based value.

When applied on `date` fields the `range` filter and queries accept also a `time_zone` parameter.

The `time_zone` parameter will be applied to your input lower and upper bounds and will move them to UTC time based date:

[source,js]
--------------------------------------------------
{
    "constant_score": {
        "filter": {
            "range" : {
                "born" : {
                    "gte": "2012-01-01",
                    "lte": "now",
                    "time_zone": "+1:00"
                }
            }
        }
    }
}

{
    "range" : {
        "born" : {
            "gte": "2012-01-01",
            "lte": "now",
            "time_zone": "+1:00"
        }
    }
}
--------------------------------------------------

In the above examples, `gte` will be actually moved to `2011-12-31T23:00:00` UTC date.

NOTE: if you give a date with a timezone explicitly defined and use the `time_zone` parameter, `time_zone` will be
ignored. For example, setting `from` to `2012-01-01T00:00:00+01:00` with `"time_zone":"+10:00"` will still use `+01:00` time zone.

Closes #3729.
2014-08-04 15:42:03 +02:00
Britta Weber f84dc23b96 Docs: remove duplicate label 2014-08-04 08:43:44 +02:00
Britta Weber 5706858722 Add parameter to GET for checking if generated fields can be retrieved
Fields of type `token_count`, `murmur3`, `_all` and `_field_names` are generated only when indexing.
If a GET requests accesses the transaction log (because no refresh
between indexing and GET request) then these fields cannot be retrieved at all.
Before the behavior was so:

`_all, _field_names`: The field was siletly ignored
`murmur3, token_count`: `NumberFormatException` because GET tried to parse the values from the source.

In addition, if these fields were not stored, the same behavior occured if the fields were
retrieved with GET after a `refresh()` because here also the source was used to get the fields.

Now, GET accepts a parameter `ignore_errors_on_generated_fields` which has
the following effect:
- Throw exception with meaningful error message explaining the problem if set to false (default)
- Ignore the field if set to true
- Always ignore the field if it was not set to stored

This changes the behavior for `_all` and `_field_names` as now an Exception is thrown if a user
tries to GET them before a `refresh()`.

closes #6676
closes #6973
2014-08-04 08:15:34 +02:00
Britta Weber a3cefd919e significant terms: add google normalized distance, add chi square
closes #6858
2014-08-04 08:15:26 +02:00
Shay Banon 95762e8126 Support "default" for tcpNoDelay and tcpKeepAlive
Allow to set the value default to network.tcp.no_delay and network.tcp.keep_alive so they won't be set at all, since on solaris, setting tcpNoDelay can actually cause failure
relates to #7115
2014-08-02 17:32:41 +02:00
uboness 3c9c9f33e2 Aggregations Added Filters aggregation
A multi-bucket aggregation where multiple filters can be defined (each filter defines a bucket). The buckets will collect all the documents that match their associated filter.

This aggregation can be very useful when one wants to compare analytics between different criterias. It can also be accomplished using multiple definitions of the single filter aggregation, but here, the user will only need to define the sub-aggregations only once.

Closes #6118
2014-08-01 16:01:08 +01:00
Adrien Grand d9d5b35be9 Sort: Make `ignore_unmapped` work for cross-index queries.
Close #2255
2014-08-01 15:30:17 +02:00
Stefan Antoni 8e862f15c1 [DOCS] fixed small typo in percolate.asciidoc 2014-08-01 12:38:35 +02:00
Britta Weber d6a18ab2ba Docs: add 1.4.0 label to many to many geo distance sort 2014-08-01 12:30:08 +02:00
Kurt Hurtado 66560acebb Update fielddata-fields.asciidoc 2014-08-01 09:20:19 +02:00
Areek Zillur 1d581e6286 Search Exists API: Checks if any matching documents exist for a given query
Implements a new Exists API allowing users to do fast exists check on any matched documents for a given query.
This API should be faster then using the Count API as it will:
 - early terminate the search execution once any document is found to exist
 - return the response as soon as the first shard reports matched documents

closes #6995
2014-07-31 15:42:30 -04:00
David Pilato 85eb0ea0e7 Generate timestamp when path is null
Index process fails when having `_timestamp` enabled and `path` option is set.
It fails with a `TimestampParsingException[failed to parse timestamp [null]]` message.

Reproduction:

```
DELETE test
PUT  test
{
    "mappings": {
        "test": {
            "_timestamp" : {
                "enabled" : "yes",
                "path" : "post_date"
            }
        }
    }
}
PUT test/test/1
{
  "foo": "bar"
}
```

You can define a default value for when timestamp is not provided
within the index request or in the `_source` document.

By default, the default value is `now` which means the date the document was processed by the indexing chain.

You can disable that default value by setting `default` to `null`. It means that `timestamp` is mandatory:

```
{
    "tweet" : {
        "_timestamp" : {
            "enabled" : true,
            "default" : null
        }
    }
}
```

If you don't provide any timestamp value, indexation will fail.

You can also set the default value to any date respecting timestamp format:

```
{
    "tweet" : {
        "_timestamp" : {
            "enabled" : true,
            "format" : "YYYY-MM-dd",
            "default" : "1970-01-01"
        }
    }
}
```

If you don't provide any timestamp value, indexation will fail.

Closes #4718.
Closes #7036.
2014-07-31 19:48:22 +02:00
Britta Weber fe86c8bc88 _geo_distance sort: allow many to many geo point distance
Add computation of disyance to many geo points. Example request:

```
{
  "sort": [
    {
      "_geo_distance": {
        "location": [
          {
            "lat":1.2,
            "lon":3
          },
          {
             "lat":1.2,
            "lon":3
          }
        ],
        "order": "desc",
        "unit": "km",
        "sort_mode": "max"
      }
    }
  ]
}
```

closes #3926
2014-07-31 17:33:45 +02:00
Clinton Gormley 4b0a89d4fb Update translog.asciidoc
Documented `index.gateway.local.sync`
2014-07-31 14:06:24 +02:00
Clinton Gormley 36e1c7928c Rewrote post-filter.asciidoc
Closes #5166
2014-07-31 12:56:11 +02:00
Nik Everett 34426eb8c2 Docs: Fix syntax on lang-analyzer
Some of the language analyzer documentation contained invalid json.

Closes #7098
2014-07-30 20:17:27 +02:00
Alex Ksikes e3b3b6c055 Term Vectors API: adds support for wildcards in selected fields
This could useful to generate all term vectors or a chosen set of them.

Closes #7061
2014-07-30 17:44:37 +02:00
gabriel-tessier eaac8141cc Docs: Fix typo in scripting.asciidoc
Replace the mvel by groovy in the forgotten place.
I add the previous change in this one.
Sorry for the spam!

Closes #7071
2014-07-29 12:30:09 +02:00
gabriel-tessier c2c2190d27 Docs: Fix typo in scripting.asciidoc
Closes #7070
2014-07-29 12:28:41 +02:00
Greg Marzouka 8f4a48a550 [Docs] Add link to .NET client docs 2014-07-28 23:48:29 -04:00
Greg Marzouka 3c4caea843 Remove NEST from community clients 2014-07-28 22:01:13 -04:00
Adrien Grand 1fe76b891b Docs: Add links to the equivalent aggs in facets documentation. 2014-07-28 15:22:49 +02:00
Lee Hinman 6abe4c951d Add HierarchyCircuitBreakerService
Adds a breaker for request BigArrays, which are used for parent/child
queries as well as some aggregations. Certain operations like Netty HTTP
responses and transport responses increment the breaker, but will not
trip.

This also changes the output of the nodes' stats endpoint to show the
parent breaker as well as the fielddata and request breakers.

There are a number of new settings for breakers now:

`indices.breaker.total.limit`: starting limit for all memory-use breaker,
defaults to 70%

`indices.breaker.fielddata.limit`: starting limit for fielddata breaker,
defaults to 60%
`indices.breaker.fielddata.overhead`: overhead for fielddata breaker
estimations, defaults to 1.03

(the fielddata breaker settings also use the backwards-compatible
setting `indices.fielddata.breaker.limit` and
`indices.fielddata.breaker.overhead`)

`indices.breaker.request.limit`: starting limit for request breaker,
defaults to 40%
`indices.breaker.request.overhead`: request breaker estimation overhead,
defaults to 1.0

The breaker service infrastructure is now generic and opens the path to
adding additional circuit breakers in the future.

Fixes #6129

Conflicts:
	src/main/java/org/elasticsearch/index/fielddata/IndexFieldData.java
	src/main/java/org/elasticsearch/index/fielddata/IndexFieldDataService.java
	src/main/java/org/elasticsearch/index/fielddata/RamAccountingTermsEnum.java
	src/main/java/org/elasticsearch/index/fielddata/ordinals/GlobalOrdinalsBuilder.java
	src/main/java/org/elasticsearch/index/fielddata/ordinals/InternalGlobalOrdinalsBuilder.java
	src/main/java/org/elasticsearch/index/fielddata/plain/AbstractIndexOrdinalsFieldData.java
	src/main/java/org/elasticsearch/index/fielddata/plain/DisabledIndexFieldData.java
	src/main/java/org/elasticsearch/index/fielddata/plain/IndexIndexFieldData.java
	src/main/java/org/elasticsearch/index/fielddata/plain/NonEstimatingEstimator.java
	src/main/java/org/elasticsearch/index/fielddata/plain/PackedArrayIndexFieldData.java
	src/main/java/org/elasticsearch/index/fielddata/plain/ParentChildIndexFieldData.java
	src/main/java/org/elasticsearch/index/fielddata/plain/SortedSetDVOrdinalsIndexFieldData.java
	src/main/java/org/elasticsearch/node/internal/InternalNode.java
	src/test/java/org/elasticsearch/index/aliases/IndexAliasesServiceTests.java
	src/test/java/org/elasticsearch/index/codec/CodecTests.java
	src/test/java/org/elasticsearch/index/fielddata/AbstractFieldDataTests.java
	src/test/java/org/elasticsearch/index/fielddata/IndexFieldDataServiceTests.java
	src/test/java/org/elasticsearch/index/mapper/MapperTestUtils.java
	src/test/java/org/elasticsearch/index/query/IndexQueryParserFilterCachingTests.java
	src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
	src/test/java/org/elasticsearch/index/query/guice/IndexQueryParserModuleTests.java
	src/test/java/org/elasticsearch/index/search/FieldDataTermsFilterTests.java
	src/test/java/org/elasticsearch/index/search/child/ChildrenConstantScoreQueryTests.java
	src/test/java/org/elasticsearch/index/similarity/SimilarityTests.java
2014-07-28 11:27:33 +02:00
Clinton Gormley be86556946 Update request-body.asciidoc
Added link from `timeout` to time-units

Closes #6361
2014-07-28 11:08:59 +02:00
Brian Altenhofel dbd5cbee7f Docs: Add Drupal Search API Elasticsearch module
The module at drupal.org/project/elasticsearch has been abandoned. The Search API Elasticsearch module allows Drupal to use Elasticsearch as a backend for Search API.

Closes #7001
2014-07-28 10:46:56 +02:00
mikemccand 96ecec34d1 Docs: fix documentation for bloom filter defaults 2014-07-27 18:39:29 -04:00
Clinton Gormley c367ae09e3 Update nested-query.asciidoc
Changed score_mode `total` to `sum` to be consistent with parent-child etc
2014-07-26 22:32:28 +02:00
Clinton Gormley 10b4177def Docs: Fixed path to search-shards 2014-07-26 15:05:53 +02:00
Clinton Gormley 88c8754a3c Docs: Removed search-shards from request-body 2014-07-26 14:52:50 +02:00
Clinton Gormley 93d9628975 Docs: Reorganised the search-shards API docs 2014-07-26 14:51:44 +02:00
Colin Goodheart-Smithe 655157c83a Aggregations: Added an option to show the upper bound of the error for the terms aggregation.
This is only applicable when the order is set to _count.  The upper bound of the error in the doc count is calculated by summing the doc count of the last term on each shard which did not return the term.  The implementation calculates the error by summing the doc count for the last term on each shard for which the term IS returned and then subtracts this value from the sum of the doc counts for the last term from ALL shards.

Closes #6696
2014-07-25 14:24:24 +01:00
Justin Honold 593fffc7a1 Docs: Changing ES_MAX_MEM default from '1gb' to '1g'
If you set ES_HEAP_SIZE to '1gb' as suggested, Java will yield an "Invalid initial heap size".

Closes #6824
2014-07-25 12:50:59 +02:00
rendel 50634e6a3d Docs: Added new entry for the SIREn plugin.
Closes #6961
2014-07-25 12:49:50 +02:00
Alexander Reelsen a1e335b1e9 CORS: Support regular expressions for origin to match against
This commit adds regular expression support for the allow-origin
header depending on the value of the request `Origin` header.

The existing HttpRequestBuilder is also extended to support the
OPTIONS HTTP method.

Relates #5601
Closes #6891
2014-07-25 10:51:22 +02:00
Lee Hinman 1fb9f404df [DOCS] correct documentation about groovy/mvel defaults and deprecations 2014-07-25 10:39:33 +02:00
Simon Willnauer bd51d7a07f Add `wait_if_ongoing` option to _flush requests
This commit adds the ability to force blocking on the flush operaition
to make sure all files have been written and synced to disk. Without
this option a flush might be executing at the same time causing the
current flush to fail and return before all files being synced.

Closes #6996
2014-07-24 15:34:53 +02:00
Areek Zillur 5487c56c70 Search & Count: Add option to early terminate doc collection
Allow users to control document collection termination, if a specified terminate_after number is
set. Upon setting the newly added parameter, the response will include a boolean terminated_early
flag, indicating if the document collection for any shard terminated early.

closes #6876
2014-07-23 15:10:15 -04:00
Lee Hinman a1a03a184c [DOCS] Fix nested root object indexing documentation
Types can no longer be specified when indexing, see:
https://github.com/elasticsearch/elasticsearch/pull/4552
2014-07-23 18:34:27 +02:00
Britta Weber 10201d511c [doc] Correct decay function equations in function_score description
Impact of decay and scale was missing from the equations.

Closes #6983
2014-07-23 17:33:22 +02:00
Clinton Gormley 0f943850a0 Update named-queries-and-filters.asciidoc 2014-07-23 17:28:49 +02:00
Simon Willnauer 5bfea56457 [DOCS] move all coming tags to added in master 2014-07-23 16:37:19 +02:00
babeya 81a83aab22 Docs: Update query-string-syntax.asciidoc
Closes #6253
2014-07-23 16:32:32 +02:00
Konrad Feldmeier 48812ff1f2 Reflect that 'field_value_factor' is only in 1.2.x
While the blogpost http://www.elasticsearch.org/blog/2014-04-02-this-week-in-elasticsearch/ states, that feature #5519 was
added to 1.x, the release notes for, e.g. v1.1.2, however tell otherwise.
Only the release notes for 1.2.0 list #5519 as a new feature.

Since the 1.x docs deprecate/discourage from using `_boost`, and seemingly give a migration example at
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-boost-field.html#function-score-instead-of-boost
users of 1.1.x should be warned.
2014-07-23 15:49:03 +02:00
Peter Johnson @insertcoffee 9a4abc2620 Docs: typo
example fails in bash

Closes #6977
2014-07-23 12:43:43 +02:00
mikemccand cc4d7c6272 Core: don't load bloom filters by default
This change just changes the default for index.codec.bloom.load to
false: with recent performance improvements to ID lookup, such as
#6298, bloom filters don't give much of a performance gain anymore,
and they can consume non-trivial RAM when there are many tiny
documents.

For now, we still index the bloom filters, so if a given app wants
them back, it can just update the index.codec.bloom.load to true.

Closes #6959
2014-07-23 05:58:41 -04:00
Clinton Gormley 3f9aea883f Docs: Made current version, branch and jdk into asciidoc attributes 2014-07-23 11:55:35 +02:00
Clinton Gormley 17df714229 Docs: Change public signing key instructions to work with sudo
Closes #6823
2014-07-23 11:13:03 +02:00
Clinton Gormley 254aa71693 Docs: Added Tiki Wiki integration
Closes #6746
2014-07-23 11:00:46 +02:00
Clinton Gormley ecb2e181ae Docs: Added Tiki Wiki integration
Closes #6746
2014-07-23 10:57:09 +02:00
Areek Zillur f39d4e1f89 PhraseSuggester: Collate option should allow returning phrases with no matching docs
A new option `prune` has been added to allow users to control phrase suggestion pruning when `collate`
is set. If the new option is set, the phrase suggestion option will contain a boolean `collate_match`
indicating whether the respective result had hits in collation.

CLoses #6927
2014-07-22 17:17:15 -04:00
Brian Murphy b98f19a54b [DOCS] Fix typo 2014-07-22 14:51:31 +01:00
Brian Murphy 3c5de7d4a1 [DOCS] Fix indentation 2014-07-22 14:49:45 +01:00
Brian Murphy e3b1aed0fc [DOCS] Update examples to groovy. 2014-07-22 14:45:46 +01:00
Nik Everett 79433d23e3 Update: Detect noop updates sent with doc_as_upsert
This should help prevent spurious updates that just cause extra writing
and cache invalidation for no real reason.

Close #6822
2014-07-22 14:55:34 +02:00
Clinton Gormley 8aefaef68a Update scripting.asciidoc
Added an ID for native java scripts
2014-07-22 11:36:40 +02:00
Peter Johnson @insertcoffee 77a2c979ab typo
causes the example to fail in bash
2014-07-21 19:09:22 +02:00
Clinton Gormley a862732434 Docs: Typo 2014-07-21 18:51:49 +02:00
Adrien Grand abeefbddea Docs: Update documentation about execution hints for the terms aggregation. 2014-07-21 11:55:57 +02:00
Clinton Gormley 6a7a77eada Docs: Add links to client helper classes for bulk/scroll/reindexing 2014-07-18 13:55:47 +02:00
Medcl 792d8970e8 [DOCS]: Added link to saltstack module 2014-07-18 12:45:29 +02:00
Alex Ksikes f22f3db30f Term Vectors API: Computes term vectors on the fly if not stored in the index.
Adds the ability to the Term Vector API to generate term vectors for some
chosen fields, even though they haven't been explicitely stored in the index.

Relates to #5184
Closes #6567
2014-07-17 23:29:05 +02:00
Peter Kim 6a25d9b7b5 [DOCS] Fixed typos 2014-07-17 15:25:34 -04:00
Simon Willnauer f9a9348508 [DOCS] Move benchmark API to 1.4 2014-07-16 15:02:20 +02:00
Brian Murphy d6cd2c2b73 [DOCS][FIX] Fix reference check in indexed scripts/templates doc. 2014-07-16 11:24:18 +01:00
Brian Murphy bc570919ee [DOCS][FIX] Fix doc parsing, broken closing block 2014-07-16 11:18:21 +01:00
Brian Murphy cbd2a97abd [DOCS] : Indexed scripts/templates
These are the docs for the indexed scripts/templates feature.
Also moved the namespace for the REST endpoints.

Closes #6851
2014-07-16 10:49:02 +01:00
Nik Everett da5fb34163 Mappings: Add transform to document before index.
Closes #6566
2014-07-15 18:40:46 +02:00
mikemccand 63cab559e3 Docs: explain that SerialMergeScheduler just maps to CMS for back compat
Closes #6878
2014-07-15 11:38:43 -04:00
Ryan Ernst 64ab22816c Scripting: Add script engine for lucene expressions.
These are javascript expressions, which can only access numeric
fielddata, parameters, and _score. They can only be used for searches (not document updates).

closes #6818
2014-07-15 07:49:01 -07:00
Areek Zillur d0d1b98d23 Stats: Expose IndexWriter and VersionMap RAM usage to ShardStats and _cat endpoint
This commit adds the RAM usage of IndexWriter and VersionMap

Closes #6483
2014-07-14 19:46:12 -04:00
Areek Zillur 76343899ea Phrase Suggester: Add collate option to PhraseSuggester
The newly added collate option will let the user provide a template query/filter which will be executed for every phrase suggestions generated to ensure that the suggestion matches at least one document for the filter/query.
The user can also add routing preference `preference` to route the collate query/filter and additional `params` to inject into the collate template.

Closes #3482
2014-07-14 16:07:52 -04:00
Malte Schirnacher 647a2a64a1 Docs: Update query-string-syntax.asciidoc
Closes #6853
2014-07-14 16:35:17 +02:00
Clinton Gormley 6e70edb0a4 Analysis: Improve Hunspell error messages
The Hunspell service would throw a confusing error message if more than
one affix file was present.  This commit distinguishes between the two
error cases: where there are no affix files and when there are too many
affix files.

Also implements lazy dictionary loading, which was used in the tests
but not implemented.

Closes #6850
2014-07-14 12:13:32 +02:00
Britta Weber 74927adced significant terms: infrastructure for changing easily the significance heuristic
This commit adds the infrastructure to allow pluging in different
measures for computing the significance of a term.
Significance measures can be provided externally by overriding

- SignificanceHeuristic
- SignificanceHeuristicBuilder
- SignificanceHeuristicParser

closes #6561
2014-07-14 11:00:50 +02:00
Igor Motov 60b317caa4 Snapshot/Restore: Add ability to restore indices without their aliases
Closes #6457
2014-07-13 17:52:41 +09:00
Florian Hopf 3689f67a76 Docs: Fixed invalid word count in geodistance agg doc
Closes #6838
2014-07-11 18:35:36 +02:00
mikemccand 6c78147f5f Docs: remove orphan comma 2014-07-11 08:26:08 -04:00
mikemccand b4e80999a7 Docs: fix merge docs to match the code (the max_thread_count default is 'aggressive' (favor SSDs)) 2014-07-11 07:00:57 -04:00
Boaz Leskes f480969503 [Gateway] set a default of 5m to `recover_after_time` when any to the `expected*Nodes` is set
The `recovery_after_time` tells the gateway to wait before starting recovery from disk. The goal here is to allow for more nodes to join the cluster and thus not start potentially unneeded replications. The `expectedNodes` setting (and friends) tells the gateway when it can start recovering even if the `recover_after_time` has not yet elapsed. However, `expectedNodes` is useless if one doesn't set `recovery_after_time`. This commit changes that by setting a sensible default of 5m for `recover_after_time` *if* a `expectedNodes` setting is present.

Closes #6742
2014-07-11 11:28:45 +02:00
Iulia Pasov eed3513c37 Docs: Update plugins.asciidoc to fix typo
Changed the name of the European Environment Agency (from European Environmental Agency)

Closes #6807
2014-07-10 14:04:26 +02:00
Karel Minarik 4ddec99703 [DOC] Added comprehensive documentation for the Ruby and Rails integrations 2014-07-10 11:21:27 +02:00
Simon Willnauer 154bd0309c [DOCS] Fix typo in reference 2014-07-10 08:47:18 +02:00
Simon Willnauer d82a434d10 [STORE] Make a hybrid directory default using `mmapfs` and `niofs`
`mmapfs` is really good for random access but can have sideeffects if
memory maps are large depending on the operating system etc. A hybrid
solution where only selected files are actually memory mapped but others
mostly consumed sequentially brings the best of both worlds and
minimizes the memory map impact.
This commit mmaps only the `dvd` and `tim` file for fast random access
on docvalues and term dictionaries.

Closes #6636
2014-07-10 00:01:43 +02:00
Shay Banon 8910e09beb Disable JSONP by default
By default, disable the option to use JSONP in our REST layer
closes #6795
2014-07-09 21:17:17 +02:00
Roman Sanchez 1cca0071eb Docs: New entry/front-end
Adding, Calaca, a simple search client for Elasticsearch.

Closes #6754
2014-07-09 16:36:23 +02:00
Iulia Pasov a79d0744d3 Docs: Update plugins.asciidoc
Closes #6683
2014-07-09 16:15:59 +02:00
Clinton Gormley b6baa4be4a Update preference.asciidoc
Clarify that `preference` is a query string parameter only
and provide an example.
2014-07-09 11:13:17 +02:00
Clinton Gormley 6c30ad1ce6 Docs: Improved the docs for nested mapping
Closes #1643
2014-07-08 15:54:11 +02:00
Naoki Orii e662d3c535 Docs: YAML mappings use colons, not equals sign
Closes #6750
2014-07-08 13:17:53 +02:00
Nick Steele d857a08cb0 Docs: grammatical error
Closes #6716
2014-07-08 13:16:06 +02:00
Clinton Gormley feb81e228b Docs: Rewrote the scroll/scan docs
Closes #6774
2014-07-08 11:54:53 +02:00
Andrii Gakhov 80321d89d9 Docs: Update histogram-aggregation.asciidoc
filter in a filtered query should be under "filter" key

Closes #6738
2014-07-07 10:44:11 +02:00
Carsten Brandt bd4699da7e Docs: fixed a typo in the docs
Closes: #6718
2014-07-07 10:41:36 +02:00
Clinton Gormley e4baa56f4b Docs: Language analyzers
Clarified the use of stem_exclusion and the keyword_marker
token filter

Closes #6613
2014-07-07 10:06:18 +02:00
Clinton Gormley 54790eea10 Update lang-analyzer.asciidoc
Clarified the use of the `stem_exclusion` token filter.

Closes #6613
2014-07-04 17:50:43 +02:00
Shinsuke Sugaya 4bddb4e346 Update plugins.asciidoc 2014-07-05 00:44:02 +09:00
Shikhar Bhushan 1e894111b0 Docs: Link to eskka discovery plugin from doc
Closes #6721
2014-07-04 17:06:51 +02:00
Clinton Gormley d3f8c66e26 Updated cache.asciidoc
The index level filter cache was removed a long time ago

Closes #6455
2014-07-04 14:26:20 +02:00
David Pilato 162c62dbcc [DOCS] Add information regarding _type parameter requirement for _mget
Change ID to `[[mget-type]]`

Closes #6670.
2014-07-03 15:38:06 +02:00
David Pilato de48d7f94c [DOCS] Add information regarding _type parameter requirement for _mget
Closes #6670.
2014-07-03 15:23:35 +02:00
Jun Ohtani 0c6a859357 Docs: fixed ICU plugin documentation
add ICU Normalization CharFilter to docs

Closes #6711
2014-07-03 15:21:51 +02:00
Mikhail Korobov 955473f475 Docs: unescape regexes in Pattern Tokenizer docs
Currently regexes in Pattern Tokenizer docs are escaped (it seems according to Java rules). I think it is better not to escape them because JSON escaping should be automatic in client libraries, and string escaping depends on a client language used. The default pattern is `\W+`, not `\\W+`.

Closes #6615
2014-07-03 13:34:13 +02:00
hanneskaeufler 6e6f4def5d Docs: Fix typo in timestamp-field.asciidoc
Closes #6661
2014-07-03 13:27:37 +02:00
Robert Muir 2935b751e9 Fix doc formatting. Norwegian stemmers and Scandinavian normalizers
were missing commas between entries.
2014-07-03 07:08:33 -04:00
Robert Muir b9a09c2b06 Analysis: Add additional Analyzers, Tokenizers, and TokenFilters from Lucene
Add `irish` analyzer
Add `sorani` analyzer (Kurdish)

Add `classic` tokenizer: specific to english text and tries to recognize hostnames, companies, acronyms, etc.
Add `thai` tokenizer: segments thai text into words.

Add `classic` tokenfilter: cleans up acronyms and possessives from classic tokenizer
Add `apostrophe` tokenfilter: removes text after apostrophe and the apostrophe itself
Add `german_normalization` tokenfilter: umlaut/sharp S normalization
Add `hindi_normalization` tokenfilter: accounts for hindi spelling differences
Add `indic_normalization` tokenfilter: accounts for different unicode representations in Indian languages
Add `sorani_normalization` tokenfilter: normalizes kurdish text
Add `scandinavian_normalization` tokenfilter: normalizes Norwegian, Danish, Swedish text
Add `scandinavian_folding` tokenfilter: much more aggressive form of `scandinavian_normalization`
Add additional languages to stemmer tokenfilter: `galician`, `minimal_galician`, `irish`, `sorani`, `light_nynorsk`, `minimal_nynorsk`

Add support access to default Thai stopword set "_thai_"

Fix some bugs and broken links in documentation.

Closes #5935
2014-07-03 05:47:49 -04:00
Matthew L Daniel 53f2301eea Docs: Add clarifying text about regexp and terms
For the casual reader, the reference to "term queries" may be glossed over, yielding an unexpected result when using `regexp` queries.
This attempts to make that distinction more prominent.

Closes #6698
2014-07-03 11:39:57 +02:00
jnguyenx 1883f74cc0 Docs: Fixed missing comma in multi match query example 2014-07-03 08:17:09 +02:00
Florian Hopf 0a93956d9a Fixed link to native Java client
Closes #6590
2014-07-01 14:01:49 +02:00
Ian Babrou 698eb7de9b Fixed JSON in fielddata docs 2014-07-01 12:53:10 +02:00
Duncan Angus Wilkie 60a8515fb7 Update histogram-facet.asciidoc
Spotted a typo, which I've fixed.
2014-07-01 10:49:43 +02:00
Florian Hopf c5cf283517 Docs: Removed Sense mention 2014-07-01 08:30:34 +02:00
Igor Motov 1425e28639 Add ability to restore partial snapshots
Closes #5742
2014-06-30 20:18:02 -04:00
Lee Hinman b43b56a6a8 Add a transformer to translate constant BigDecimal to double 2014-06-26 10:52:28 +02:00
mahdeto e78f1edca3 DOC:Added field data circuit breaker settings 2014-06-26 10:29:41 +02:00
Clinton Gormley 30c80319c0 Match query with operator and, cutoff_frequency and stacked tokens
If the match query with cutoff_frequency encounters stacked tokens,
like synonyms in the same position, it returns a boolean query instead
of a common terms query.  However, if the original operator was set
to "and", it was ignoring that and resetting the operator to "or".

In fact, if operator is "and" then there is little benefit in using
a common terms query as a must query is already
executed efficiently.
2014-06-25 17:53:43 +02:00
Lee Hinman 5c6d28240f Switch to Groovy as the default scripting language
This is a breaking change to move from MVEL -> Groovy
2014-06-25 12:15:12 +02:00
Clinton Gormley 64a4acc49b Docs: Added IDs to the highlighters for linking 2014-06-22 16:46:42 +02:00
Clinton Gormley cf059378d1 Docs: Updated stop token filter docs 2014-06-21 18:42:38 +02:00
Clinton Gormley fac724cc99 Docs: Updated the explanation about memory usage with parent/child 2014-06-21 16:32:29 +02:00
Clinton Gormley e52364a95a Docs: Updated cluster health docs 2014-06-20 18:05:46 +02:00
Clinton Gormley adf6e794b6 Docs: Rewrote the filtered query docs to be clearer
Closes #1688
2014-06-19 16:34:26 +02:00
Adrien Grand 703dbff83d Index field names of documents.
The `exists` and `missing` filters need to merge postings lists of all existing
terms, which can be very costly, especially on high-cardinality fields. This
commit indexes the field names of a document under `_field_names` and reuses it
to speed up the `exists` and `missing` filters.

This is only enabled for indices that are created on or after Elasticsearch
1.3.0.

Close #5659
2014-06-19 11:50:06 +02:00
Fitblip d18fb8bfbd REST API: Allow to configure JSONP/callback support
Added the http.jsonp.enable option to configure disabling of JSONP responses, as those
might pose a security risk, and can be disabled if unused.

This also fixes bugs in NettyHttpChannel
* JSONP responses were never setting application/javascript as the content-type
* The content-type and content-length headers were being overwritten even if they were set before

Closes #6164
2014-06-19 08:34:38 +02:00
Chris 011e20678d [DOCS] Fixed json example in nested-aggregation.asciidoc 2014-06-18 19:38:02 +02:00
Colin Goodheart-Smithe 7423ce0560 Aggregations: Added percentile rank aggregation
Percentile Rank Aggregation is the reverse of the Percetiles aggregation.  It determines the percentile rank (the proportion of values less than a given value) of the provided array of values.

Closes #6386
2014-06-18 12:02:08 +01:00
Clinton Gormley 69350dc426 Update stemmer-override-tokenfilter.asciidoc 2014-06-18 11:34:20 +02:00
Clinton Gormley 3eb291f334 Docs: tidied configuration.asciidoc 2014-06-17 17:37:07 +02:00
Shay Banon f450c3ea30 update docs to reflect how default write consistency with 1 replica behaves 2014-06-17 14:25:04 +02:00
Matt Janssen 946dde287a [DOCS] Fixed is/if typo in Api Conventions doc 2014-06-16 15:44:47 +02:00
Volker Fröhlich 06192686a2 [DOCS] Fixd typo in http.asciidoc 2014-06-16 10:42:34 +02:00
stephlag 13d910f016 Added missing comma in suggester example 2014-06-13 16:01:04 +02:00
Adrien Grand 7a34702925 [DOCS] Clarify the trade-off of the `disk` doc values format. 2014-06-13 13:24:53 +02:00
Adrien Grand 01327d7136 Facets: deprecation.
Users are encouraged to move to the new aggregation framework that was
introduced in Elasticsearch 1.0.

Close #6485
2014-06-13 13:13:44 +02:00
Clinton Gormley eb6c9fe111 Docs: Linked to fielddata formats from core types
Closes #6489
2014-06-13 12:58:03 +02:00
Daniel Winterstein 99549cab00 Added link to "native" ES client 2014-06-13 12:49:48 +02:00
Boaz Leskes 7fb16c783d Added caching support to geohash_filter
Caching is turned off by default.

Closes #6478
2014-06-12 22:19:34 +02:00
Shay Banon 2330421816 Wait till node is part of cluster state for join process
When a node sends a join request to the master, only send back the response after it has been added to the master cluster state and published.
This will fix the rare cases where today, a join request can return, and the master, since its under load, have not yet added the node to its cluster state, and the node that joined will start a fault detect against the master, failing since its not part of the cluster state.
Since now the join request is longer, also increase the join request timeout default.
closes #6480
2014-06-12 18:15:51 +02:00
Lee Hinman 3a3f81d59b Enable DiskThresholdDecider by default, change default limits to 85/90%
Fixes #6200
Fixes #6201
2014-06-12 16:35:29 +02:00
Clinton Gormley c41e63c2f9 Docs: Updated index-modules/store and setup/configuration
Explain how to set different index storage types, and
added the vm settings required to stop mmapfs from running
out of memory

Closes #6327
2014-06-12 13:56:06 +02:00
shadow000fire 1b45b216fd Update nested-query.asciidoc
Added note that fields inside a nested query must be full qualified.
2014-06-12 12:48:23 +02:00
Luke Fender f9da5259bc [DOCS] Fixed typo in post-filter.asciidoc
Remove 'be' where it is not needed
2014-06-12 12:09:19 +02:00
Igor Motov 56a264cf6d [DOCS] Snapshot/restore: add more information about snapshot and restore monitoring 2014-06-11 20:52:45 -04:00
Clinton Gormley f546662e8f Docs: Hunspell tidied
Tidied some formatting
2014-06-11 21:49:02 +02:00
Clinton Gormley 04dacaaf27 Docs: Use the "stemmer" token filter for the english analyzer, to be consistent 2014-06-11 13:47:07 +02:00
Clinton Gormley 8a94b71b75 Docs: Corrected the use of keyword_marker on the lang analyzers 2014-06-11 13:43:02 +02:00
Clinton Gormley 673ef3db3f The StemmerTokenFilter had a number of issues:
* `english` returned the slow snowball English stemmer
* `porter2` returned the snowball Porter stemmer (v1)
* `portuguese` was used twice, preventing the second version from working

Changes:

* `english` now returns the fast PorterStemmer (for indices created from v1.3.0 onwards)
* `porter2` now returns the snowball English stemmer (for indices created from v1.3.0 onwards)
* `light_english` now returns the `kstem` stemmer (`kstem` still works)
* `portuguese_rslp` returns the PortugueseStemmer
* `dutch_kp` is a synonym for `kp`

Tests and docs updated

Fixes #6345
Fixes #6213
Fixes #6330
2014-06-11 12:30:16 +02:00
Martijn van Groningen 5e408f3d40 Change the top_hits to be a metric aggregation instead of a bucket aggregation (which can't have an sub aggs)
Closes #6395
Closes #6434
2014-06-10 09:09:50 +02:00
Clinton Gormley e323e577e8 Docs: Fixed bad ref on cjk_width/bigram pages 2014-06-09 23:36:58 +02:00
Clinton Gormley 5e40868f44 Docs: Fixed a bad ref on lang analyzers page 2014-06-09 23:03:12 +02:00
Clinton Gormley 5c5c1da06c Docs: Fixed some errors on the language analyzers page 2014-06-09 22:51:28 +02:00
Clinton Gormley 585b0ef730 Docs: Added custom-analyzer equivalents of all the language analyzers 2014-06-09 22:41:25 +02:00
Clinton Gormley bc402d5f87 Docs: Documented the cjk_width and cjk_bigram token filters 2014-06-09 22:40:58 +02:00
Matthew L Daniel b0a85f6ca3 Guard against improper auto_expand_replica values
Previously if the user provided a non-conforming string, it would blow up with
`java.lang.StringIndexOutOfBoundsException: String index out of range: -1`
which is not a *helpful* error message.

Also updated the documentation to make the possible setting values more clear.

Close #5752
2014-06-07 01:19:06 +02:00
markharwood 724129e6ce Aggregations optimisation for memory usage. Added changes to core Aggregator class to support a new mode of deferred collection.
A new "breadth_first" results collection mode allows upper branches of aggregation tree to be calculated and then pruned
to a smaller selection before advancing into executing collection on child branches.

Closes #6128
2014-06-06 15:59:51 +01:00
fransflippo cdbde4a578 [DOCS] Reworded note about shorthand suggest syntax
The existing Note about the shorthand suggest syntax was poorly worded and confusing. Please check whether the way I've phrased it now is still correct as to what the shorthand form actually does and doesn't do: the original wording did not provide me enough information to be sure.
Thanks!
2014-06-06 10:21:01 +02:00
Evgeniy Sokovikov 1383ab77b6 [DOCS] Fixed typo in put-mapping docs
split backwardscompatibility to backwards compatibility
2014-06-05 19:55:11 +02:00
Yervand Aghababyan cb22417cc1 [DOCS] Fixed the fuzzy query docs with correct default value max_expansion option 2014-06-05 19:52:12 +02:00
Steve Fuller e991c1f717 [DOCS] fixed typo in date-format.asciidoc 2014-06-05 19:49:20 +02:00
Jad Naous 5aa84c9aab [DOCS] Fixed typos in aggregations.asciidoc
Fix plural/singular forms.
2014-06-05 19:47:01 +02:00
gseng 7b5807fe4a [DOCS] Fixed typo in object-type.asciidoc 2014-06-05 19:34:50 +02:00
Christian Wolfe 9a11c909f3 [DOCS] Clean up Java-API docs
Fixed some wording and subject/verb agreement while reading through the Java API documentation.
2014-06-05 19:29:17 +02:00
Philip Stevens 4998c0928f [DOCS] Replace facets example with aggregations in warmers docs 2014-06-05 19:22:16 +02:00
Israel Tsadok 1a58016ea1 [DOCS] Add special attributes for indices allocation filtering 2014-06-05 10:38:07 +02:00
Rob Young 07a6143386 [DOCS] Fix grammar in dynamic mappings 2014-06-04 08:56:15 +02:00
Colin Goodheart-Smithe b9f4d44b14 Aggregations: Adds GeoBounds Aggregation
The GeoBounds Aggregation is a new single bucket aggregation which outputs the coordinates of a bounding box containing all the points from all the documents passed to the aggregation as well as the doc count. Geobound Aggregation also use a wrap_logitude parameter which specifies whether the resulting bounding box is permitted to overlap the international date line.  This option defaults to true.

This aggregation introduces the idea of MetricsAggregation which do not return double values and cannot be used for sorting.  The existing MetricsAggregation has been renamed to NumericMetricsAggregation and is a subclass of MetricsAggregation.  MetricsAggregations do not store doc counts and do not support child aggregations.

Closes #5634
2014-06-03 15:59:56 +01:00
violuke 4f99f0c6f1 [DOCS] Improved readability of multi-match query docs 2014-06-03 14:23:34 +02:00
darkwarriors d8765a8f1d [DOCS] fixed urls in nodes-stats docs 2014-06-03 13:48:42 +02:00
stephlag a10bedfa96 [DOCS] Update index docs to match changes in IndexResponse class
IndexResponse.id() ->  IndexResponse.getId()
2014-06-03 13:48:42 +02:00
Patrik Ragnarsson 9a3368b937 [DOCS] Fix minor error in cluster stats example 2014-06-03 13:38:37 +02:00
Gaurav Arora 4a3837acf0 [DOCS] fix typo in network module docs 2014-06-03 13:19:36 +02:00
James Yu 8994eed82b [DOCS] Update elasticsearch version in repositories.asciidoc 2014-06-03 12:30:51 +02:00
Steve Fuller b800be891f [DOCS] fixed typo in fucntion-score query docs 2014-06-03 12:05:59 +02:00
violuke 0020e5fc0a [DOCS] Improved grammar in multi-match query docs 2014-06-03 11:50:41 +02:00
javanna 5a1ad7b42e [DOCS] fixed curl requests in benchmark docs 2014-06-03 11:47:13 +02:00
leonardo menezes f3eca05c3b [DOCS] removed slowest on single query benchmark requests
Relates to #5904
2014-06-03 11:47:13 +02:00
javanna 3fcbe1d6cf [DOCS] reordered cat apis menu 2014-06-03 11:06:35 +02:00
Ivan Brusic 29bc6bce1a [DOCS] Fielddata cat API added in 1.2.0 2014-06-03 11:06:28 +02:00
salyh db9921fc03 [DOCS] Add community supported MSI installer to docs 2014-06-03 10:59:57 +02:00
salyh 27b38818b7 [DOCS] Add imap river and security plugin to docs 2014-06-03 10:59:50 +02:00
Andrew Raines b2d1b3df4b [DOCS] Clarify that only_expunge_deletes doesn't override expunge_deletes_allowed 2014-06-02 17:49:01 -05:00
Clinton Gormley 46a67b638d Parent/Child: Added min_children/max_children to has_child query/filter
Added support for min_children and max_children parameters to
the has_child query and filter. A parent document will only
be considered if a match if the number of matching children
fall between the min/max bounds.

Closes #6019
2014-05-30 19:38:39 +02:00
Shay Banon 9c98bb3554 Have a dedicated join timeout that is higher than ping.timeout for node join
Using ping.timeout, which defaults to 3s, to use as a timeout value on the join request a node makes to the master once its discovered can be too small, specifically when there is a large cluster state involved (and by definition, all the buffers and such on the nio layer will be "cold"). Introduce a dedicated join.timeout setting, that by default is 10x the ping.timeout (so 30s by default).
closes #6342
2014-05-30 12:42:08 +02:00
Clinton Gormley 7fff6f1f43 Docs: Tidied percolate.asciidoc 2014-05-30 11:56:06 +02:00
Martijn van Groningen aab38fb2e6 Aggregations: added pagination support to `top_hits` aggregation by adding `from` option.
Closes #6299
2014-05-30 11:45:31 +02:00
javanna 74eff87dd6 [DOCS] Java 7 is required since 1.2.0 2014-05-30 10:45:22 +02:00
Adrien Grand 328a7e513c [DOCS] Document filtered query strategies. 2014-05-28 17:57:43 +02:00
David Pilato 1dc186a595 [DOCS] fix typo 2014-05-27 15:57:39 +02:00
Itamar Syn-Hershko ac812f72b7 Docs: Adding Hebrew analyzer
Closes #6306
2014-05-27 13:40:53 +02:00
Martijn van Groningen 5fafd2451a Added `top_hits` aggregation that keeps track of the most relevant document being aggregated per bucket.
Closes #6124
2014-05-23 16:01:18 +02:00
Nik Everett 3573822b7e Highlight fields in request order
Because json objects are unordered this also adds an explicit order syntax
that looks like
    "highlight": {
        "fields": [
            {"title":{ /*params*/ }},
            {"text":{ /*params*/ }}
        ]
    }

This is not useful for any of the builtin highlighters but will be useful
in plugins.

Closes #4649
2014-05-22 16:44:14 +02:00
Simon Willnauer 9d5507047f Update Documentation Feature Flags [1.2.0] 2014-05-22 15:06:42 +02:00
Alex Ksikes 2546c06131 More Like This Query: allow for both 'like_text' and 'docs/ids' to be specified.
Closes #6246
2014-05-22 13:50:17 +02:00
Clinton Gormley f950344546 [DOCS] Fixed title levels in context suggester 2014-05-21 20:47:25 +02:00
Alex Ksikes a29b4a800d More Like This Query: replaced 'exclude' with 'include' to avoid double negation when set.
Closes #6248
2014-05-21 18:45:03 +02:00
Simon Willnauer ec3b1c57ac Move Benchmark release to 1.3 2014-05-21 10:17:59 +02:00
Igor Motov 91c7892305 Add ability to snapshot replicating primary shards
This change adds a new cluster state that waits for the replication of a shard to finish before starting snapshotting process. Because this change adds a new snapshot state, an pre-1.2.0 nodes will not be able to join the 1.2.0 cluster that is currently running snapshot/restore operation.

Closes #5531
2014-05-20 08:57:21 -04:00
Simon Willnauer 85a0b76dbb Upgrade to Lucene 4.8.1
This commit upgrades to the latest Lucene 4.8.1 release including the
following bugfixes:

 * An IndexThrottle now kicks in when merges start falling behind
   limiting index threads to 1 until merges caught up. Closes #6066
 * RateLimiter now kicks in at the configured rate where previously
   the limiter was limiting at ~8MB/sec almost all the time. Closes #6018
2014-05-19 20:47:55 +02:00
Itamar Syn-Hershko d1589b3815 Fixing invalid jsons 2014-05-19 15:07:56 +02:00
Andrew Selden 420f2db4cd [DOCS] Cat recovery API update
This is an update for the _cat/recovery API documentation. The examples
have been updated. Removed the bottom paragraph explaining why there
could be values > 100%. This can no longer happen so that had to be
removed.

Closes #6159
2014-05-18 17:43:13 -07:00
Simon Willnauer f79b28375d Add missing coming tag
Relates to #6188
Relates to #5539
2014-05-18 10:54:17 +02:00
Alex Ksikes db991dc3a4 More Like This Query: Added searching for multiple items.
The syntax to specify one or more items is the same as for the Multi GET API.
If only one document is specified, the results returned are the same as when
using the More Like This API.

Relates #4075 Closes #5857
2014-05-17 19:14:56 +02:00
Igor Motov c20713530d Switch to shared thread pool for all snapshot repositories
Closes #6181
2014-05-16 19:03:15 -04:00
Boaz Leskes 9f10547f4b Allow 0 as a valid external version
Until now all version types have officially required the version to be a positive long number. Despite of this has being documented, ES versions <=1.0 did not enforce it when using the `external` version type. As a result people have succesfully indexed documents with 0 as a version. In 1.1. we introduced validation checks on incoming version values and causing indexing request to fail if the version was set to 0. While this is strictly speaking OK, we effectively have a situation where data already indexed does not match the version invariant.

To be lenient and adhere to spirit of our data backward compatibility policy, we have decided to allow 0 as a valid external version type. This is somewhat complicated as 0 is also the internal value of `MATCH_ANY`, which indicates requests should succeed regardles off the current doc version. To keep things simple, this commit changes the internal value of `MATCH_ANY` to `-3` for all version types.

Since we're doing this in a minor release (and because versions are stored in the transaction log), the default `internal` version type still accepts 0 as a `MATCH_ANY` value. This is not a problem for other version types as `MATCH_ANY` doesn't make sense in that context.

Closes #5662
2014-05-16 22:10:16 +02:00
Clinton Gormley f510e25306 [DOCS] Renamed the "cat" chapters to be more searchable 2014-05-16 21:43:35 +02:00
Clinton Gormley bfeb5a7120 added install instruction with apt
Closes #6206
2014-05-16 19:07:05 +02:00
David Pilato bd871f96c2 Check that a plugin is Lucene compatible with the current running node using `lucene` property in `es-plugin.properties` file.
* If plugin does not provide `lucene` property, we consider that the plugin is compatible.
* If plugin provides `lucene` property, we try to load related Enum org.apache.lucene.util.Version. If this fails, it means that the node is too "old" comparing to the Lucene version the plugin was built for.
* We compare then two first digits of current node lucene version against two first digits of plugin Lucene version. If not equal, it means that the plugin is too "old" for the current node.

Plugin developers who wants to launch plugin check only have to add a `lucene` property in `es-plugin.properties` file. If you are using maven to build your plugin, you can do it like this:

In `pom.xml`:

```xml
    <properties>
        <lucene.version>4.6.0</lucene.version>
    </properties>

    <build>
        <resources>
            <resource>
                <directory>src/main/resources</directory>
                <filtering>true</filtering>
            </resource>
        </resources>
    </build>
```

In `es-plugin.properties`, add:

```properties
lucene=${lucene.version}
```

BTW, if you don't already have it, you can add the plugin version as well:

```properties
version=${project.version}
```

You can disable that check using `plugins.check_lucene: false`.
2014-05-16 13:41:20 +02:00
Clinton Gormley 8f0991c14f [DOCS] Rewrote the memory settings section on the configuration page 2014-05-14 16:02:59 +02:00
Britta Weber 08e57890f8 use shard_min_doc_count also in TermsAggregation
This was discussed in issue #6041 and #5998 .

closes #6143
2014-05-14 14:10:04 +02:00
Gaurav Arora e041b5992c Fix typo in docs 2014-05-14 12:36:35 +02:00