Commit Graph

2638 Commits

Author SHA1 Message Date
Zachary Tong f5a992486e [Docs] Add missing annotation, fix blockdef 2015-12-17 17:02:38 -05:00
Zachary Tong 9a70dbb51a Add ability to profile query and collectors
Provides a new flag which can be enabled on a per-request basis.
When `"profile": true` is set, the search request will execute in a
mode that collects detailed timing information for query components.

```
GET /test/test/_search
{
   "profile": true,
   "query": {
      "match": {
         "foo": "bar"
      }
   }
}
```

Closes #14889

Squashed commit of the following:

commit a92db5723d2c61b8449bd163d2f006d12f9889ad
Merge: 117dd99 3f87b08
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Dec 17 09:44:10 2015 -0500

    Merge remote-tracking branch 'upstream/master' into query_profiler

commit 117dd9992e8014b70203c6110925643769d80e62
Merge: 9b29d68 82a64fd
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Dec 15 13:27:18 2015 -0500

    Merge remote-tracking branch 'upstream/master' into query_profiler

    Conflicts:
    	core/src/main/java/org/elasticsearch/search/SearchService.java

commit 9b29d6823a71140ecd872df25ff9f7478e7fe766
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Dec 14 16:13:23 2015 -0500

    [TEST] Profile flag needs to be set, ensure searches go against primary only for consistency

commit 4d602d8ad1f8cbc7b475450921fa3bc7d395b34f
Merge: 8b48e87 7742c1e
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Dec 14 10:56:25 2015 -0500

    Merge remote-tracking branch 'upstream/master' into query_profiler

commit 8b48e876348b163ab730eeca7fa35142165b05f9
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Dec 14 10:56:01 2015 -0500

    Delegate straight to in.matchCost, no need for profiling

commit fde3b0587911f0b5f15e779c671d0510cbd568a9
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Dec 14 10:28:23 2015 -0500

    Documentation tweaks, renaming build_weight -> create_weight

commit 46f5e011ee23fe9bb8a1f11ceb4fa9d19fe48e2e
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Dec 14 10:27:52 2015 -0500

    Profile TwoPhaseIterator should override matchCost()

commit b59f894ddb11b2a7beebba06c4ec5583ff91a7b2
Merge: 9aa1a3a b4e0c87
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Dec 9 14:23:26 2015 -0500

    Merge remote-tracking branch 'upstream/master' into query_profiler

commit 9aa1a3a25c34c9cd9fffaa6114c25a0ec791307d
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Dec 9 13:41:48 2015 -0500

    Revert "Move some of the collector wrapping logic into ProfileCollectorBuilder"

    This reverts commit 02cc31767fb76a7ecd44a302435e93a05fb4220e.

commit 57f7c04cea66b3f98ba2bec4879b98e4fba0b3c0
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Dec 9 13:41:31 2015 -0500

    Revert "Rearrange if/else to make intent clearer"

    This reverts commit 59b63c533fcaddcdfe4656e86a6f6c4cb1bc4a00.

commit 2874791b9c9cd807113e75e38be465f3785c154e
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Dec 9 13:38:13 2015 -0500

    Revert "Move state into ProfileCollectorBuilder"

    This reverts commit 0bb3ee0dd96170b06f07ec9e2435423d686a5ae6.

commit 0bb3ee0dd96170b06f07ec9e2435423d686a5ae6
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Dec 3 11:21:55 2015 -0500

    Move state into ProfileCollectorBuilder

commit 59b63c533fcaddcdfe4656e86a6f6c4cb1bc4a00
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Dec 2 17:21:12 2015 -0500

    Rearrange if/else to make intent clearer

commit 511db0af2f3a86328028b88a6b25fa3dfbab963b
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Dec 2 17:12:06 2015 -0500

    Rename WEIGHT -> BUILD_WEIGHT

commit 02cc31767fb76a7ecd44a302435e93a05fb4220e
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Dec 2 17:11:22 2015 -0500

    Move some of the collector wrapping logic into ProfileCollectorBuilder

commit e69356d3cb4c60fa281dad36d84faa64f5c32bc4
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 30 15:12:35 2015 -0500

    Cleanup imports

commit c1b4f284f16712be60cd881f7e4a3e8175667d62
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 30 15:11:25 2015 -0500

    Review cleanup: Make InternalProfileShardResults writeable

commit 9e61c72f7e1787540f511777050a572b7d297636
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 30 15:01:22 2015 -0500

    Review cleanup: Merge ProfileShardResult, InternalProfileShardResult.  Convert to writeable

commit 709184e1554f567c645690250131afe8568a5799
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 30 14:38:08 2015 -0500

    Review cleanup: Merge ProfileResult, InternalProfileResult.  Convert to writeable

commit 7d72690c44f626c34e9c608754bc7843dd7fd8fe
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 30 14:01:34 2015 -0500

    Review cleanup: use primitive (and default) for profile flag

commit 97d557388541bbd3388cdcce7d9718914d88de6d
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 30 13:09:12 2015 -0500

    Review cleanup: Use Collections.emptyMap() instead of building an empty one explicitly

commit 219585b8729a8b0982e653d99eb959efd0bef84e
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 30 13:08:12 2015 -0500

    Add todo to revisit profiler architecture in the future

commit b712edb2160e032ee4b2f2630fadf131a0936886
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 30 13:05:32 2015 -0500

    Split collector serialization from timing, use writeable instead of streamable

    Previously, the collector timing was done in the same class that was serialized, which required
    leaving the collector null when serializing.  Besides being a bit gross, this made it difficult to
    change the class to Writeable.

    This splits up the timing (InternalProfileCollector + ProfileCollector) and the serialization of
    the times (CollectorResult).  CollectorResult is writeable, and also acts as the public interface.

commit 6ddd77d066262d4400e3d338b11cebe7dd27ca78
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Nov 25 13:15:12 2015 -0500

    Remove dead code

commit 06033f8a056e2121d157654a65895c82bbe93a51
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Nov 25 12:49:51 2015 -0500

    Review cleanup:  Delegate to in.getProfilers()

    Note:  Need to investigate how this is used exactly so we can add a test, it isn't touched by a
    normal inner_hits query...

commit a77e13da21b4bad1176ca2b5d5b76034fb12802f
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Nov 25 11:59:58 2015 -0500

    Review cleanup:  collapse to single `if` statement

commit e97bb6215a5ebb508b0293ac3acd60d5ae479be1
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Nov 25 11:39:43 2015 -0500

    Review cleanup: Return empty map instead of null for profile results

    Note: we still need to check for nullness in SearchPhaseController, since an empty/no-hits result
    won't have profiling instantiated (or any other component like aggs or suggest).  Therefore
    QuerySearchResult.profileResults() is still @Nullable

commit db8e691de2a727389378b459fa76c942572e6015
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Nov 25 10:14:47 2015 -0500

    Review cleanup: renaming, formatting fixes, assertions

commit 9011775fe80ba22c2fd948ca64df634b4e32772d
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Nov 19 20:09:52 2015 -0500

    [DOCS] Add missing annotation

commit 4b58560b06f08d4b99b149af20916ee839baabd7
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Nov 19 20:07:17 2015 -0500

    [DOCS] Update documentation for new format

commit f0458c58e5538ed8ec94849d4baf3250aa9ec841
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Nov 17 10:14:09 2015 +0100

    Reduce visibility of internal classes.

commit d0a7d319098e60b028fa772bf8a99b2df9cf6146
Merge: e158070 1bdf29e
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Nov 17 10:09:18 2015 +0100

    Merge branch 'master' into query_profiler

commit e158070a48cb096551f3bb3ecdcf2b53bbc5e3c5
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Nov 17 10:08:48 2015 +0100

    Fix compile error due to bad html in javadocs.

commit a566b5d08d659daccb087a9afbe908ec3d96cd6e
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 16 17:48:37 2015 -0500

    Remove unused collector

commit 4060cd72d150cc68573dbde62ca7321c47f75703
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 16 17:48:10 2015 -0500

    Comment cleanup

commit 43137952bf74728f5f5d5a8d1bfc073e0f9fe4f9
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Nov 16 17:32:06 2015 -0500

    Fix negative formatted time

commit 5ef3a980266326aff12d4fe380f73455ff28209f
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Nov 13 17:10:17 2015 +0100

    Fix javadocs.

commit 276114d29e4b17a0cc0982cfff51434f712dc59e
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Nov 13 16:25:23 2015 +0100

    Fix: include rewrite time as well...

commit 21d9e17d05487bf4903ae3d2ab6f429bece2ffef
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Nov 13 15:10:15 2015 +0100

    Remove TODO about profiling explain.

commit 105a31e8e570efb879447159c3852871f5cf7db4
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Nov 13 14:59:30 2015 +0100

    Fix nocommit now that the global collector is a root collector.

commit 2e8fc5cf84adb1bfaba296808c329e5f982c9635
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Nov 13 14:53:38 2015 +0100

    Make collector wrapping more explicit/robust (and a bit less magical).

commit 5e30b570b0835e1ce79a57933a31b6a2d0d58e2d
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Nov 13 12:44:03 2015 +0100

    Simplify recording API a bit.

commit 9b453afced6adc0a59ca1d67d90c28796b105185
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Nov 13 10:54:25 2015 +0100

    Fix serialization-related nocommits.

commit ad97b200bb123d4e9255e7c8e02f7e43804057a5
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Nov 13 10:46:30 2015 +0100

    Fix DFS.

commit a6de06986cd348a831bd45e4f524d2e14d9e03c3
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Nov 12 19:29:16 2015 +0100

    Remove forbidden @Test annotation.

commit 4991a28e19501109af98026e14756cb25a56f4f4
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Nov 12 19:25:59 2015 +0100

    Limit the impact of query profiling on the SearchContext API.

    Rule is: we can put as much as we want in the search.profile package but should
    aim at touching as little as possible other areas of the code base.

commit 353d8d75a5ce04d9c3908a0a63d4ca6e884c519a
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Nov 12 18:05:09 2015 +0100

    Remove dead code.

commit a3ffafb5ddbb5a2acf43403c946e5ed128f47528
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Nov 12 15:30:35 2015 +0100

    Remove call to forbidden String.toLowerCase() API.

commit 1fa8c7a00324fa4e32bd24135ebba5ecf07606f1
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Nov 12 15:30:27 2015 +0100

    Fix compilation.

commit 2067f1797e53bef0e1a8c9268956bc5fb8f8ad97
Merge: 22e631f fac472f
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Nov 12 15:21:12 2015 +0100

    Merge branch 'master' into query_profiler

commit 22e631fe6471fed19236578e97c628d5cda401a9
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Nov 3 18:52:05 2015 -0500

    Fix and simplify serialization of shard profile results

commit 461da250809451cd2b47daf647343afbb4b327f2
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Nov 3 18:32:22 2015 -0500

    Remove helper methods, simpler without them

commit 5687aa1c93d45416d895c2eecc0e6a6b302139f2
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Nov 3 18:29:32 2015 -0500

    [TESTS] Fix tests for new rewrite format

commit ba9e82857fc6d4c7b72ef4d962d2102459365299
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Oct 30 15:28:14 2015 -0400

    Rewrites begone! Record all rewrites as a single time metric

commit 5f28d7cdff9ee736651d564f71f713bf45fb1d91
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Oct 29 15:36:06 2015 -0400

    Merge duplicate rewrites into one entry

    By using the Query as the key in a map, we can easily merge rewrites together.  This means
    the preProcess(), assertion and main query rewrites all get merged together.  Downside is that
    rewrites of the same Query (hashcode) but in different places get lumped together.  I think the
    simplicity of the solution is better than the slight loss in output fidelity.

commit 9a601ea46bb21052746157a45dcc6de6bc350e9c
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Oct 29 15:28:27 2015 -0400

    Allow multiple "searches" per profile (e.g. query + global agg)

commit ee30217328381cd83f9e653d3a4d870c1d2bdfce
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Oct 29 11:29:18 2015 -0400

    Update comment, add nullable annotation

commit 405c6463a64e118f170959827931e8c6a1661f13
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Oct 29 11:04:30 2015 -0400

    remove out-dated comment

commit 2819ae8f4cf1bfd5670dbd1c0e06195ae457b58f
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Oct 27 19:50:47 2015 +0100

    Don't render children in the profiles when there are no children.

commit 7677c2ddefef321bbe74660471603d202a4ab66f
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Oct 27 19:50:35 2015 +0100

    Set the profiler on the ContextIndexSearcher.

commit 74a4338c35dfed779adc025ec17cfd4d1c9f66f5
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Oct 27 19:50:01 2015 +0100

    Fix json rendering.

commit 6674d5bebe187b0b0d8b424797606fdf2617dd27
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Oct 27 19:20:19 2015 +0100

    Revert "nocommit - profiling never enabled because setProfile() on ContextIndexSearcher never called"

    This reverts commit d3dc10949024342055f0d4fb7e16c7a43423bfab.

commit d3dc10949024342055f0d4fb7e16c7a43423bfab
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Oct 23 17:20:57 2015 -0400

    nocommit - profiling never enabled because setProfile() on ContextIndexSearcher never called

    Previously, it was enabled by using DefaultSearchContext as a third-party "proxy", but since
    the refactor to make it unit testable, setProfile() needs to be called explicitly.  Unfortunately,
    this is not possible because SearchService only has access to an IndexSearcher.  And it is not
    cast'able to a DefaultIndexSearcher.

commit b9ba9c5d1f93b9c45e97b0a4e35da6f472c9ea53
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Oct 23 16:27:00 2015 -0400

    [TESTS] Fix unit tests

commit cf5d1e016b2b4a583175e07c16c7152f167695ce
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Oct 23 16:22:34 2015 -0400

    Increment token after recording a rewrite

commit b7d08f64034e498533c4a81bff8727dd8ac2843e
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Oct 23 16:14:09 2015 -0400

    Fix NPE if a top-level root doesn't have children

commit e4d3b514bafe2a3a9db08438c89f0ed68628f2d6
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Oct 23 16:05:47 2015 -0400

    Fix NPE when profiling is disabled

commit 445384fe48ed62fdd01f7fc9bf3e8361796d9593
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Oct 23 16:05:37 2015 -0400

    [TESTS] Fix integration tests

commit b478296bb04fece827a169e7522df0a5ea7840a3
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Oct 23 15:43:24 2015 -0400

    Move rewrites to their own section, remove reconciliation

    Big commit because the structural change affected a lot of the wrapping code.  Major changes:

    - Rewrites are now in their own section in the response
    - Reconciliation is gone...we don't attempt to roll the rewrites into the query tree structure
    - InternalProfileShardResults (plural) simply holds a Map<String, InternalProfileShardResult> and
    helps to serialize / ToXContent
    - InternalProfileShardResult (singular) is now the holder for all shard-level profiling details. Currently
    this includes query, collectors and rewrite.  In the future it would hold suggest, aggs, etc
    - When the user requests the profiled results, they get back a Map<String, ProfileShardResult>
    instead of doing silly helper methods to convert to maps, etc
    - Shard details are baked into a string instead of serializing the object

commit 24819ad094b208d0e94f17ce9c3f7c92f7414124
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Oct 23 10:25:38 2015 -0400

    Make Profile results immutable by removing relative_time

commit bfaf095f45fed74194ef78160a8e5dcae1850f9e
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 23 10:54:59 2015 +0200

    Add nocommits.

commit e9a128d0d26d5b383b52135ca886f2c987850179
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 23 10:39:37 2015 +0200

    Move all profile-related classes to the same package.

commit f20b7c7fdf85384ecc37701bb65310fb8c20844f
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Oct 23 10:33:14 2015 +0200

    Reorganize code a bit to ease unit testing of ProfileCollector.

commit 3261306edad6a0c70f59eaee8fe58560f61a75fd
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 18:07:28 2015 +0200

    Remove irrelevant nocommit.

commit a6ac868dad12a2e17929878681f66dbd0948d322
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 18:06:45 2015 +0200

    Make CollectorResult's reason a free-text field to ease bw compat.

commit 5d0bf170781a950d08b81871cd1e403e49f3cc12
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 16:50:52 2015 +0200

    Add unit tests for ProfileWeight/ProfileScorer.

commit 2cd88c412c6e62252504ef69a59216adbb574ce4
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 15:55:17 2015 +0200

    Rename InternalQueryProfiler to Profiler.

commit 84f5718fa6779f710da129d9e0e6ff914fd85e36
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 15:53:58 2015 +0200

    Merge InternalProfileBreakdown into ProfileBreakdown.

commit 135168eaeb8999c8117ea25288104b0961ce9b35
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 13:56:57 2015 +0200

    Make it possible to instantiate a ContextIndexSearcher without SearchContext.

commit 5493fb52376b48460c4ce2dedbe00cc5f6620499
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 11:53:29 2015 +0200

    Move ProfileWeight/Scorer to their own files.

commit bf2d917b9dae3b32dfc29c35a7cac4ccb7556cce
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 11:38:24 2015 +0200

    Fix bug that caused phrase queries to fail.

commit b2bb0c92c343334ec1703a221af24a1b55e36d53
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 11:36:17 2015 +0200

    Parsing happens on the coordinating node now.

commit 416cabb8621acb5cd8dfa77374fd23e428f52fe9
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 11:22:17 2015 +0200

    Fix compilation (in particular remove guava deps).

commit f996508645f842629d403fc2e71c1890c0e2cac9
Merge: 4616a25 bc3b91e
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Oct 22 10:44:38 2015 +0200

    Merge branch 'master' into query_profiler

commit 4616a25afffe9c24c6531028f7fccca4303d2893
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Oct 20 12:11:32 2015 -0400

    Make Java Count API compatible with profiling

commit cbfba74e16083d719722500ac226efdb5cb2ff55
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Oct 20 12:11:19 2015 -0400

    Fix serialization of profile query param, NPE

commit e33ffac383b03247046913da78c8a27e457fae78
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Oct 20 11:17:48 2015 -0400

    TestSearchContext should return null Profiler instead of exception

commit 73a02d69b466dc1a5b8a5f022464d6c99e6c2ac3
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Oct 19 12:07:29 2015 -0400

    [DOCS] Update docs to reflect new ID format

commit 36248e388c354f954349ecd498db7b66f84ce813
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Oct 19 12:03:03 2015 -0400

    Use the full [node][index][shard] string as profile result ID

commit 5cfcc4a6a6b0bcd6ebaa7c8a2d0acc32529a80e1
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Oct 15 17:51:40 2015 -0400

    Add failing test for phrase matching

    Stack trace generated:

    [2015-10-15 17:50:54,438][ERROR][org.elasticsearch.search.profile] shard [[JNj7RX_oSJikcnX72aGBoA][test][2]], reason [RemoteTransportException[[node_s0][local[1]][indices:data/read/search[phase/query]]]; nested: QueryPhaseExecutionException[Query Failed [Failed to execute main query]]; nested: AssertionError[nextPosition() called more than freq() times!]; ], cause [java.lang.AssertionError: nextPosition() called more than freq() times!
    	at org.apache.lucene.index.AssertingLeafReader$AssertingPostingsEnum.nextPosition(AssertingLeafReader.java:353)
    	at org.apache.lucene.search.ExactPhraseScorer.phraseFreq(ExactPhraseScorer.java:132)
    	at org.apache.lucene.search.ExactPhraseScorer.access$000(ExactPhraseScorer.java:27)
    	at org.apache.lucene.search.ExactPhraseScorer$1.matches(ExactPhraseScorer.java:69)
    	at org.elasticsearch.common.lucene.search.ProfileQuery$ProfileScorer$2.matches(ProfileQuery.java:226)
    	at org.apache.lucene.search.ConjunctionDISI$TwoPhaseConjunctionDISI.matches(ConjunctionDISI.java:175)
    	at org.apache.lucene.search.ConjunctionDISI$TwoPhase.matches(ConjunctionDISI.java:213)
    	at org.apache.lucene.search.ConjunctionDISI.doNext(ConjunctionDISI.java:128)
    	at org.apache.lucene.search.ConjunctionDISI.nextDoc(ConjunctionDISI.java:151)
    	at org.apache.lucene.search.ConjunctionScorer.nextDoc(ConjunctionScorer.java:62)
    	at org.elasticsearch.common.lucene.search.ProfileQuery$ProfileScorer$1.nextDoc(ProfileQuery.java:205)
    	at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:224)
    	at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:169)
    	at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
    	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:795)
    	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:509)
    	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:347)
    	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:111)
    	at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:366)
    	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:378)
    	at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:368)
    	at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:365)
    	at org.elasticsearch.transport.local.LocalTransport$2.doRun(LocalTransport.java:280)
    	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    	at java.lang.Thread.run(Thread.java:745)

commit 889fe6383370fe919aaa9f0af398e3040209e40b
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Oct 15 17:30:38 2015 -0400

    [DOCS] More docs

commit 89177965d031d84937753538b88ea5ebae2956b0
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Oct 15 09:59:09 2015 -0400

    Fix multi-stage rewrites to recursively find most appropriate descendant rewrite

    Previously, we chose the first rewrite that matched.  But in situations where a query may
    rewrite several times, this won't build the tree correctly.  Instead we need to recurse
    down all the rewrites until we find the most appropriate "leaf" rewrite

    The implementation of this is kinda gross: we recursively call getRewrittenParentToken(),
    which does a linear scan over the rewriteMap and tries to find rewrites with a larger token
    value (since we know child tokens are always larger).  Can almost certainly find a better
    way to do this...

commit 0b4d782b5348e5d03fd26f7d91bc4a3fbcb7f6a5
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Oct 14 19:30:06 2015 -0400

    [Docs] Documentation checkpoint

commit 383636453f6610fcfef9070c21ae7ca11346793e
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Sep 16 16:02:22 2015 -0400

    Comments

commit a81e8f31e681be16e89ceab9ba3c3e0a018f18ef
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Sep 16 15:48:49 2015 -0400

    [TESTS] Ensure all tests use QUERY_THEN_FETCH, DFS does not profile

commit 1255c2d790d85fcb9cbb78bf2a53195138c6bc24
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Sep 15 16:43:46 2015 -0400

    Refactor rewrite handling to handle identical rewrites

commit 85b7ec82eb0b26a6fe87266b38f5f86f9ac0c44f
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Sep 15 08:51:14 2015 -0400

    Don't update parent when a token is added as root -- Fixes NPE

commit 109d02bdbc49741a3b61e8624521669b0968b839
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Sep 15 08:50:40 2015 -0400

    Don't set the rewritten query if not profiling -- Fixes NPE

commit 233cf5e85f6f2c39ed0a2a33d7edd3bbd40856e8
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Sep 14 18:04:51 2015 -0400

    Update tests to new response format

commit a930b1fc19de3a329abc8ffddc6711c1246a4b15
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Sep 14 18:03:58 2015 -0400

    Fix serialization

commit 69afdd303660510c597df9bada5531b19d134f3d
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Sep 14 15:11:31 2015 -0400

    Comments and cleanup

commit 64e7ca7f78187875378382ec5d5aa2462ff71df5
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Sep 14 14:40:21 2015 -0400

    Move timing into dedicated class, add proper rewrite integration

commit b44ff85ddbba0a080e65f2e7cc8c50d30e95df8e
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Sep 14 12:00:38 2015 -0400

    Checkpoint - Refactoring to use a token-based dependency tree

commit 52cedd5266d6a87445c6a4cff3be8ff2087cd1b7
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Fri Sep 4 19:18:19 2015 -0400

    Need to set context profiling flag before calling queryPhase.preProcess

commit c524670cb1ce29b4b3a531fa2bff0c403b756f46
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Sep 4 18:00:37 2015 +0200

    Reduce profiling overhead a bit.

    This removes hash-table lookups everytime we start/stop a profiling clock.

commit 111444ff8418737082236492b37321fc96041e09
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Sep 4 16:18:59 2015 +0200

    Add profiling of two-phase iterators.

    This is useful for eg. phrase queries or script filters, since they are
    typically consumed through their two-phase iterator instead of the scorer.

commit f275e690459e73211bc8494c6de595c0320f4c0b
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Sep 4 16:03:21 2015 +0200

    Some more improvements.

    I changed profiling to disable bulk scoring, since it makes it impossible to
    know where time is spent. Also I removed profiling of operations that are
    always fast (eg. normalization) and added nextDoc/advance.

commit 3c8dcd872744de8fd76ce13b6f18f36f8de44068
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Sep 4 14:39:50 2015 +0200

    Remove println.

commit d68304862fb38a3823aebed35a263bd9e2176c2f
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Sep 4 14:36:03 2015 +0200

    Fix some test failures introduced by the rebase...

commit 04d53ca89fb34b7a21515d770c32aaffcc513b90
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Sep 4 13:57:35 2015 +0200

    Reconcile conflicting changes after rebase

commit fed03ec8e2989a0678685cd6c50a566cec42ea4f
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Thu Aug 20 22:40:39 2015 -0400

    Add Collectors to profile results

    Profile response element has now been re-arranged so that everything is listed per-shard to
    facilitate grouping elements together.  The new `collector` element looks like this:

    ```
    "profile": {
      "shards": [
         {
            "shard_id": "keP4YFywSXWALCl4m4k24Q",
            "query": [...],
            "collector": [
               {
                  "name": "MultiCollector",
                  "purpose": "search_multi",
                  "time": "16.44504400ms",
                  "relative_time": "100.0000000%",
                  "children": [
                     {
                        "name": "FilteredCollector",
                        "purpose": "search_post_filter",
                        "time": "4.556013000ms",
                        "relative_time": "27.70447437%",
                        "children": [
                           {
                              "name": "SimpleTopScoreDocCollector",
                              "purpose": "search_sorted",
                              "time": "1.352166000ms",
                              "relative_time": "8.222331299%",
                              "children": []
                           }
                        ]
                     },
                     {
                        "name": "BucketCollector: [[non_global_term, another_agg]]",
                        "purpose": "aggregation",
                        "time": "10.40379400ms",
                        "relative_time": "63.26400829%",
                        "children": []
                     },
           ...
    ```

commit 1368b495c934be642c00f6cbf9fc875d7e6c07ff
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Aug 19 12:43:03 2015 -0400

    Move InternalProfiler to profile package

commit 53584de910db6d4a6bb374c9ebb954f204882996
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 18 18:34:58 2015 -0400

    Only reconcile rewrite timing when rewritten query is different from original

commit 9804c3b29d2107cd97f1c7e34d77171b62cb33d0
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 18 16:40:15 2015 -0400

    Comments and cleanup

commit 8e898cc7c59c0c1cc5ed576dfed8e3034ca0967f
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 18 14:19:07 2015 -0400

    [TESTS] Fix comparison test to ensure results sort identically

commit f402a29001933eef29d5a62e81c8563f1c8d0969
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 18 14:17:59 2015 -0400

    Add note about DFS being unable to profile

commit d446e08d3bc91cd85b24fc908e2d82fc5739d598
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 18 14:17:23 2015 -0400

    Implement some missing methods

commit 13ca94fb86fb037a30d181b73d9296153a63d6e4
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 18 13:10:54 2015 -0400

    [TESTS] Comments & cleanup

commit c76c8c771fdeee807761c25938a642612a6ed8e7
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 18 13:06:08 2015 -0400

    [TESTS] Fix profileMatchesRegular to handle NaN scores and nearlyEqual floats

commit 7e7a10ecd26677b2239149468e24938ce5cc18e1
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 18 12:22:16 2015 -0400

    Move nearlyEquals() utility function to shared location

commit 842222900095df4b27ff3593dbb55a42549f2697
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 18 12:04:35 2015 -0400

    Fixup rebase conflicts

commit 674f162d7704dd2034b8361358decdefce1f76ce
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Aug 17 15:29:35 2015 -0400

    [TESTS] Update match and bool tests

commit 520380a85456d7137734aed0b06a740e18c9cdec
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Aug 17 15:28:09 2015 -0400

    Make naming consistent re: plural

commit b9221501d839bb24d6db575d08e9bee34043fc65
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Aug 17 15:27:39 2015 -0400

    Children need to be added to list after serialization

commit 05fa51df940c332fbc140517ee56e849f2d40a72
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Aug 17 15:22:41 2015 -0400

    Re-enable bypass for non-profiled queries

commit f132204d264af77a75bd26a02d4e251a19eb411d
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Aug 17 15:21:14 2015 -0400

    Fix serialization of QuerySearchResult, InternalProfileResult

commit 27b98fd475fc2e9508c91436ef30624bdbee54ba
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Aug 10 17:39:17 2015 -0400

    Start to add back tests, refactor Java api

commit bcfc9fefd49307045108408dc160774666510e85
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Aug 4 17:08:10 2015 -0400

    Checkpoint

commit 26a530e0101ce252450eb23e746e48c2fd1bfcae
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Tue Jul 14 13:30:32 2015 -0400

    Add createWeight() checkpoint

commit f0dd61de809c5c13682aa213c0be65972537a0df
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Mon Jul 13 12:36:27 2015 -0400

    checkpoint

commit 377ee8ce5729b8d388c4719913b48fae77a16686
Author: Zachary Tong <zacharyjtong@gmail.com>
Date:   Wed Mar 18 10:45:01 2015 -0400

    checkpoint
2015-12-17 15:29:00 -05:00
Yannick Welsch 3a442db9bd Allocate primary shards based on allocation ids
Closes #15281
2015-12-17 15:55:50 +01:00
Simon Willnauer eae3da3b54 Merge branch 'master' into settings_prototype 2015-12-17 15:13:41 +01:00
Jason Bryan 9a1133ca50 Fix typo in scroll.asciidoc
Fix scroll request with sort.

Closes #15493
2015-12-16 20:31:46 -05:00
Michael McCandless 319dc8c8ed remove dead code; get one test working again; fix docs; remove nocommits 2015-12-16 16:19:07 -05:00
Simon Willnauer 71b204ea49 Merge branch 'master' into settings_prototype 2015-12-16 20:29:21 +01:00
Jason Tedor d9a24961c5 Fix minor issues in delimited payload token filter docs
This commit addresses a few minor issues in the delimited payload token
filter docs:
  - the provided example reversed the payloads associated with the
    tokens "the" and "fox"
  - two additional typos in the same sentence
    - "per default" -> "by default"
    - "default int to" -> "default into"
  - adds two serial commas
2015-12-16 13:00:20 -05:00
Clinton Gormley 4597a22ace Merge pull request #15473 from jmluy/patch-1
Update sample in sort for consistency
2015-12-16 12:53:09 +01:00
Simon Willnauer 6ea266a89c Merge branch 'master' into settings_prototype 2015-12-15 16:33:01 +01:00
Costin Leau 210657a453 [DOC] escape # in programlisting 2015-12-15 16:44:27 +02:00
umeku 0ce88b5887 Fix inaccurate docs for nested datatype
Closes #15436
2015-12-15 15:15:00 +01:00
Costin Leau 4426ed0a09 [DOCS] Link docs on repository-hdfs plugin
relates #15191
2015-12-15 14:52:36 +02:00
Clinton Gormley 82eb498b29 Docs: Updated plugin author help for Gradle
Relates to #15280
2015-12-15 12:27:47 +01:00
Clinton Gormley 061446b25a Merge pull request #15304 from cjohansen/patch-1
Fix typo
2015-12-15 10:57:38 +01:00
Costin Leau 7bca97bba6 HDFS Snapshot/Restore plugin
Migrated from ES-Hadoop. Contains several improvements regarding:

* Security
Takes advantage of the pluggable security in ES 2.2 and uses that in order
to grant the necessary permissions to the Hadoop libs. It relies on a
dedicated DomainCombiner to grant permissions only when needed only to the
libraries installed in the plugin folder
Add security checks for SpecialPermission/scripting and provides out of
the box permissions for the latest Hadoop 1.x (1.2.1) and 2.x (2.7.1)

* Testing
Uses a customized Local FS to perform actual integration testing of the
Hadoop stack (and thus to make sure the proper permissions and ACC blocks
are in place) however without requiring extra permissions for testing.
If needed, a MiniDFS cluster is provided (though it requires extra
permissions to bind ports)
Provides a RestIT test

* Build system
Picks the build system used in ES (still Gradle)
2015-12-14 21:50:09 +02:00
Clinton Gormley f20f41e02e Merge pull request #15405 from alexg-dev/patch-1
More detailed explanation of some similarity types
2015-12-14 14:28:43 +01:00
Clinton Gormley 6144457f01 Merge pull request #15403 from fforbeck/master
Added description for parameter <local>, FIX #14125
2015-12-14 14:24:49 +01:00
Clinton Gormley 83ee1fc903 Merge pull request #15400 from TheDude05/fix-match_pattern-docs
Fix docs with `match_pattern` in dynamic templates
2015-12-14 14:18:59 +01:00
Martijn van Groningen aaacf096d2 Merge remote-tracking branch 'es/master' into feature/ingest 2015-12-14 11:57:10 +01:00
Felipe Forbeck 708abcc59a Added desc for parameter <local> 2015-12-11 22:26:33 -02:00
Nicholas Knize 5f3d807f61 Update geo_shape/query docs, fix TermStrategy defaults
This commit adds the following:
* SpatialStrategy documentation to the geo-shape reference docs.
* Updates relation documentation to geo-shape-query reference docs.
* Updates GeoShapeFiledMapper to set points_only to true if TERM strategy is used (to be consistent with documentation)
2015-12-11 17:14:22 -06:00
Andrew Williams e7127c9f6f Fix docs with `match_pattern` in dynamic templates 2015-12-11 14:03:54 -06:00
Jim Ferenczi 9ab168dbf6 Removes all the reference of the query in the docs 2015-12-11 20:07:57 +01:00
Adrien Grand 9fd843748e Remove the `format` option of the `_source` field.
This option allows to force the xcontent type to use to store the `_source`
document. The default is to use the same format as the input format.

This commit makes this option ignored for 2.x indices and rejected for 3.0
indices.
2015-12-11 18:06:20 +01:00
Jim Ferenczi d78e24689b Merge pull request #15364 from jimferenczi/missing_query_removal
Remove the MissingQueryBuilder which was deprecated in 2.2.0.
2015-12-11 17:20:47 +01:00
Clinton Gormley 259c6eeb59 Merge pull request #15274 from murnieza/patch-1
[Doc] Redundant indefinite article removed
2015-12-11 14:38:44 +01:00
Martijn van Groningen 503a166b71 Merge remote-tracking branch 'es/master' into feature/ingest 2015-12-11 14:32:16 +01:00
Yannick Welsch db5594a2af Support wildcards for getting repositories or snapshots
Closes #15151
2015-12-11 10:20:27 +01:00
Ryan Ernst 93de1ed606 Merge pull request #15354 from rjernst/just_node
Remove NodeBuilder
2015-12-10 19:19:55 -08:00
Jun Ohtani fab44398d9 Analysis: Add detail response support
add explain option
fix char_filter bug

Closes #11076 #15257
2015-12-10 23:10:51 +09:00
Jim Ferenczi 437488ae64 Remove the MissingQueryBuilder which was deprecated in 2.2.0.
As a replacement use ExistsQueryBuilder inside a mustNot() clause.
So instead of using `new ExistsQueryBuilder(name)` now use:
`new BoolQueryBuilder().mustNot(new ExistsQueryBuilder(name))`.

Closes #14112
2015-12-10 12:55:05 +01:00
Clinton Gormley f43c8476aa Improvements to network docs 2015-12-10 12:01:18 +01:00
Clinton Gormley 9d71c7210b Added "size units" to API conventions docs 2015-12-10 12:01:18 +01:00
Robert Muir fac8d97356 ipv4 -> IPv4 2015-12-10 12:01:18 +01:00
Robert Muir f578254ca7 simplify wording 2015-12-10 12:01:18 +01:00
Robert Muir 3049b14f6b add missing 'of' 2015-12-10 12:01:18 +01:00
Robert Muir 610e9b5436 add missing header 2015-12-10 12:01:18 +01:00
Robert Muir 27c08d452e fix tables 2015-12-10 12:01:18 +01:00
Robert Muir 7595c4a3c8 Improve network docs
This makes some minor improvements (does not fix all problems!)

It reorders unicast disco in elasticsearch.yml to be right after the network host,
for better locality.

It removes the warning (unreleased) about publish addresses, lets try to really discourage setting
that unless you need to (behind a proxy server). Most people should be fine with `network.host`

Finally it reorganizes the network docs page a bit:

We add a table of 4 "basic" settings at the very beginning:

* network.host
* discovery.zen.ping.unicast.hosts
* http.port
* transport.tcp.port

The first two being the most important, which addresses to bind and talk to, and the other two
being the port numbers.

The rest of the stuff I tried to simplify and reorder under "advanced" headers.

This is just a quick stab, I still think we need more effort into this thing, but we gotta start somewhere.
2015-12-10 12:01:18 +01:00
Yannick Welsch 17da7fb0ee Merge pull request #15305 from ywelsch/feature/multi-alias-rest
Add support to _aliases endpoint to specify multiple indices and aliases in one action
2015-12-10 09:44:36 +01:00
Ryan Ernst 6f166dd0ff Remove NodeBuilder
The NodeBuilder is currently used to construct a Node. However, this is
really just yet-another-builder that wraps around a Settings.Builder
witha couple convenience methods. But there are very few uses of these
convenience methods.  This change removes NodeBuilder, in favor of just
using the Node constructor.
2015-12-10 00:01:44 -08:00
David Pilato 414fccb7d1 Merge branch 'fix/15268-proxy-auth' 2015-12-09 23:21:00 +01:00
Yannick Welsch bef0bedba9 Add support to _aliases endpoint to specify multiple indices and aliases in one action
Closes #15305
2015-12-09 19:08:27 +01:00
javanna 283d9c1523 [TEST] adjust processor docs. remove throws exception when the field is not there. 2015-12-09 18:24:26 +01:00
javanna 8240031216 Merge branch 'master' into feature/ingest 2015-12-09 18:14:32 +01:00
Simon Willnauer ce417540c5 apply review from @clintongormley 2015-12-09 12:24:40 +01:00
Adrien Grand f2d8a35888 Dynamically map floating-point numbers as floats instead of doubles.
Close #13851
2015-12-08 18:19:47 +01:00
Britta Weber e0aa481bf5 Merge pull request #15213 from brwe/copy-to-in-multi-fields-exception
throw exception if a copy_to is within a multi field

Copy to within multi field is ignored from 2.0 on, see #10802.
Instead of just ignoring it, we should throw an exception if this
is found in the mapping when a mapping is added. For already
existing indices we should at least log a warning.
We remove the copy_to in any case.

related to #14946
2015-12-08 14:41:07 +01:00
Simon Willnauer 2e27ee393f add rest API to reset settings 2015-12-08 14:39:16 +01:00
David Pilato 7dcb40bcac Add support for proxy authentication for s3 and ec2
When using S3 or EC2, it was possible to use a proxy to access EC2 or S3 API but username and password were not possible to be set.

This commit adds support for this. Also, to make all that consistent, proxy settings for both plugins have been renamed:

* from `cloud.aws.proxy_host` to `cloud.aws.proxy.host`
* from `cloud.aws.ec2.proxy_host` to `cloud.aws.ec2.proxy.host`
* from `cloud.aws.s3.proxy_host` to `cloud.aws.s3.proxy.host`
* from `cloud.aws.proxy_port` to `cloud.aws.proxy.port`
* from `cloud.aws.ec2.proxy_port` to `cloud.aws.ec2.proxy.port`
* from `cloud.aws.s3.proxy_port` to `cloud.aws.s3.proxy.port`

New settings are `proxy.username` and `proxy.password`.

```yml
cloud:
    aws:
        protocol: https
        proxy:
            host: proxy1.company.com
            port: 8083
            username: myself
            password: theBestPasswordEver!
```

You can also set different proxies for `ec2` and `s3`:

```yml
cloud:
    aws:
        s3:
            proxy:
                host: proxy1.company.com
                port: 8083
                username: myself1
                password: theBestPasswordEver1!
        ec2:
            proxy:
                host: proxy2.company.com
                port: 8083
                username: myself2
                password: theBestPasswordEver2!
```

Note that `password` is filtered with `SettingsFilter`.

We also fix a potential issue in S3 repository. We were supposed to accept key/secret either set under `cloud.aws` or `cloud.aws.s3` but the actual code never implemented that.

It was:

```java
account = settings.get("cloud.aws.access_key");
key = settings.get("cloud.aws.secret_key");
```

We replaced that by:

```java
String account = settings.get(CLOUD_S3.KEY, settings.get(CLOUD_AWS.KEY));
String key = settings.get(CLOUD_S3.SECRET, settings.get(CLOUD_AWS.SECRET));
```

Also, we extract all settings for S3 in `AwsS3Service` as it's already the case for `AwsEc2Service` class.

Closes #15268.
2015-12-07 23:10:54 +01:00
Simon Willnauer 4e80a5e099 Merge pull request #15234 from s1monw/remove_ancient_settings
Remove ancient deprecated and alternative recovery settings
2015-12-07 21:15:49 +01:00
Tal Levy e6b287c083 Merge pull request #15133 from talevy/the_one_true_field
[Ingest] Have processors operate one field at time.
2015-12-07 08:31:08 -08:00
Tal Levy 45f48ac126 update all processors to only operate on one field at a time when possible 2015-12-07 08:30:00 -08:00
Martijn van Groningen 6062c4eac9 Merge branch 'master' into feature/ingest 2015-12-07 15:53:39 +01:00
Simon Willnauer c1f7f8c03c add settings to migration guide 2015-12-07 09:54:23 +01:00
Clinton Gormley cea1c465d4 Update rolling_upgrade.asciidoc
`--path.config` -> `--path.conf`

Closes #15242
2015-12-05 13:30:47 +01:00
Clinton Gormley d82a685f57 Update upgrade.asciidoc
Closes #15240
2015-12-05 13:27:25 +01:00
Lee Hinman 46ca2d679d Merge remote-tracking branch 'dakrone/remove-get-forcemerge' 2015-12-04 10:21:57 -07:00
javanna 6c43137413 Merge branch 'master' into feature/ingest 2015-12-04 14:10:01 +01:00
Clinton Gormley 69fff60624 Merge pull request #15231 from bt5e/master
fixed minor typo
2015-12-04 10:01:02 +01:00
Clinton Gormley 543b9440b7 Merge pull request #15227 from popol1991/patch-1
Update scripting.asciidoc
2015-12-04 09:58:03 +01:00
David Pilato 012dfb2988 Merge pull request #15205 from imacube/imacube-plugins-s3-update
Update repository-s3.asciidoc
2015-12-04 08:59:01 +01:00
Ben Tse 3cede749f9 fixed minor typo 2015-12-03 23:53:48 -05:00
Gao Yingkai 47c5da523f Update scripting.asciidoc
Fixed minor typo in the example of native scripting request.
2015-12-03 20:24:20 -05:00
Lee Hinman f709b7283f Remove `GET` option for /_forcemerge
POST should be used to indicate this is not just a retrieval operation.

Resolves #15165
2015-12-03 13:56:15 -07:00
Ryan 94027f1461 Made requested changes
Moved section to before testing section and made requested formatting changes.
2015-12-03 09:27:20 -08:00
Tal Levy 56da7b32ed add ability to define custom grok patterns within processor config 2015-12-03 08:24:07 -08:00
Clinton Gormley e84b61358f Fix version order for breaking changes docs 2015-12-03 14:24:19 +01:00
Ryan 5890c9a813 Update repository-s3.asciidoc
Documentation of AWS VPC public vs. private subnets and their affects on accessing S3.
2015-12-02 16:07:57 -08:00
Martijn van Groningen f427ad2094 docs: undo accidental rename added via: 5e07644788 2015-12-02 12:17:43 +01:00
Martijn van Groningen a9ecde041b Merge branch 'master' into feature/ingest 2015-12-02 11:21:15 +01:00
Xavier Coulon 144225f4e5 Fixing typo
Replace "Too shade or not to shade..." with "To shade or not to shade..."

(cherry picked from commit f44c5a4)
(cherry picked from commit 12d5510)
2015-12-02 09:25:52 +01:00
javanna 5e07644788 [DOCS] add missing comma 2015-12-01 20:07:17 +01:00
javanna 6c0510b01d Make rename processor less error prone
Rename processor now checks whether the field to rename exists and throws exception if it doesn't. It also checks that the new field to rename to doesn't exist yet, and throws exception otherwise. Also we make sure that the rename operation is atomic, otherwise things may break between the remove and the set and we'd leave the document in an inconsistent state.

Note that the requirement for the new field name to not exist simplifies the usecase for e.g. { "rename" : { "list.1": "list.2"} } as such a rename wouldn't be accepted if list is actually a list given that either list.2 already exists or the index is out of bounds for the existing list. If one really wants to replace an existing field, that field needs to be removed first through remove processor and then rename can be used.
2015-12-01 19:58:24 +01:00
Clinton Gormley 9dbda2af62 Update scripting.asciidoc
Fix script syntax for script_score

Closes #15096
2015-11-30 17:07:52 +01:00
Honza Král 02798951ef [docs] Updating the Python client docxs 2015-11-30 16:50:08 +01:00
David Pilato bed9bf19c6 S3 repository: fix spelling error
Reported at https://github.com/elastic/elasticsearch-cloud-aws/pull/221
2015-11-30 16:01:55 +01:00
Martijn van Groningen fdf4543b8e Renamed `add` processor to `set` processor.
This name makes more sense, because if a field already exists it overwrites it.
2015-11-30 15:03:20 +01:00
David Pilato a4e22b44e4 add java-api doc about shading / embedding
Two new sections added

* Dealing with JAR dependency conflicts
* Embedding jar with dependencies

Closes #15071.
2015-11-30 11:47:17 +01:00
Jim Ferenczi e182072b6f Merge pull request #15017 from jimferenczi/fields_option
Refuse to load fields from _source when using the `fields` option and support wildcards.
2015-11-30 11:01:21 +01:00
Jim Ferenczi 731833cfc6 Fixes #14489
Do not to load fields from _source when using the `fields` option.
  Non stored (non existing) fields are ignored by the fields visitor when using the `fields` option.

Fixes #10783
  Support * wildcard to retrieve stored fields when using the `fields` option.
  Supported pattern styles are "xxx*", "*xxx", "*xxx*" and "xxx*yyy".
2015-11-30 11:00:32 +01:00
Martijn van Groningen 467a47670c Merge remote-tracking branch 'es/master' into feature/ingest 2015-11-30 10:24:27 +01:00
Clinton Gormley 72be42d742 Document that _index is a virtual field and only supports term queries
Closes #15070
Closes #15081
2015-11-30 08:43:23 +01:00
Clinton Gormley 1685126bb6 Merge pull request #15085 from kaneshin/docs/modify/post_filter
Remove a trailing comma from an example data of JSON
2015-11-30 08:05:10 +01:00
Boaz Leskes f8a027e591 Merge pull request #14801 from bleskes/resiliency_update
Update the resiliency page to 2.0.0
2015-11-29 12:21:15 +01:00
Johnny Lim a66be6cfb9 Polish doc
Closes #15011
2015-11-28 18:25:42 +01:00
Shintaro Kaneko d7baeb1e7b Remove a trailing comma from an example data of JSON 2015-11-28 16:50:28 +00:00
Clinton Gormley 174b4bacbe Merge pull request #14871 from jamiemccarthy/doc-fix
Fix doc of nested_path sort option
2015-11-28 17:46:47 +01:00
Clinton Gormley 2ab14cb21c Merge pull request #14900 from shikhar/patch-1
link to es-restlog plugin
2015-11-28 15:09:22 +01:00
javanna b4b698f653 Merge branch 'master' into feature/ingest 2015-11-28 11:15:25 +01:00
Martijn van Groningen 9d1fa0d6da ingest: Add `meta` processor that allows to modify the metadata attributes of document being processed 2015-11-26 15:46:32 +01:00
tomoya yokota 82d26c852a property name is not right
`ignore_script` is not right. `ignored_script' is right.

See org.elasticsearch.index.analysis.CJKBigramFilterFactory
2015-11-26 14:22:23 +09:00
Martijn van Groningen a84d35ab3f Merge remote-tracking branch 'es/master' into feature/ingest 2015-11-25 18:45:05 +01:00
javanna 4759a6e50f Merge branch 'master' into feature/ingest 2015-11-25 14:59:10 +01:00
Jimmi Dyson c4ee350c5e Add Kubernetes discovery community plugin 2015-11-25 12:54:29 +00:00
Jason Tedor b6da075505 Fix typo in TTL field docs
Closes #14994
2015-11-24 22:57:35 -05:00
Clinton Gormley 18a8c20cba Merge pull request #14873 from williammwang/master
Clarify where index.similarity.default.type is set
2015-11-24 15:01:43 +01:00
javanna 8f1f5d4da0 Split mutate processor into one processor per function 2015-11-24 14:31:53 +01:00
Clinton Gormley 74d9c8f5cf Docs: Fix :issue: and :pull: asciidoc replacements
Closes #14922
2015-11-24 12:35:28 +01:00
javanna eeb51ce8d0 Merge branch 'master' into feature/ingest 2015-11-24 10:23:53 +01:00
Boaz Leskes 316f07743a feedback 2015-11-23 13:15:22 +01:00
David Pilato 28109a18a2 Fix example for s3 repository bucket name
Closes #13588.
2015-11-23 13:14:02 +01:00
David Pilato 5b0e2823b1 Merge branch 'docs/mapper-attachments' 2015-11-23 12:14:31 +01:00
javanna 36655b688c Merge branch 'master' into feature/ingest 2015-11-23 10:05:17 +01:00
Martijn van Groningen 48771f1a76 field stats: Added `min_value_as_string` and `max_value_as_string` response elements for all number based fields. The existing `min_value` and `max_value` will return the values as numbers instead.
Closes #14404
2015-11-23 08:48:28 +01:00
Christoph Büscher be9dd035e2 Merge pull request #14887 from cbuescher/merge-base-shapebuilders
Merging BaseLineString and BasePolygonBuilder with subclass
2015-11-21 12:52:31 +01:00
Christoph Büscher 991ab030dc Adding notes about ShapeBuilder changes to migration doc. 2015-11-21 12:51:28 +01:00
Clinton Gormley 69c4ed0324 Change the commented out line to a comment block 2015-11-20 20:32:49 +01:00
Clinton Gormley 9960d32549 Comment out release notes in master until they are populated 2015-11-20 20:28:32 +01:00
Clinton Gormley ccde02c930 Add release notes to docs index 2015-11-20 20:23:54 +01:00
Clinton Gormley 9e0ca4a795 Updated the release-notes script to produce AsciiDoc and added placeholders 2015-11-20 20:05:53 +01:00
Clinton Gormley 2293c0d8c8 Update token-count.asciidoc
Fix typo
2015-11-20 19:00:52 +01:00
Shikhar Bhushan c57e006cd5 link to es-restlog plugin 2015-11-20 12:21:50 -05:00
Christoph Büscher 71e734647b Docs: `exists` query does not need to be wrapped inside `constant_score`
In the example we show an `exists` query inside a constant score query. While this is possible, it can mislead users to think it is necessary so we should remove it.
2015-11-20 18:08:51 +01:00
William e042e06a5a Update similarity.asciidoc 2015-11-19 16:41:29 -08:00
Jamie McCarthy ce20337d03 Fix doc of nested_path sort option 2015-11-19 12:22:00 -05:00
David Pilato 33b0e662cc Merge branch 'pr/phonetic_daitch_mokotoff' 2015-11-19 14:48:11 +01:00
Isabel Drost-Fromm a656f90a72 Revert back to type instead of child_type...
for has child queries.

Relates to #14249
2015-11-19 14:47:01 +01:00
David Pilato d48d8ef863 Merge branch 's3-storage-class-support' of https://github.com/socialrank/elasticsearch into socialrank-s3-storage-class-support
# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.
2015-11-19 13:10:08 +01:00
Yannick Welsch 6a2fa73fb5 Merge pull request #14808 from ywelsch/refactor/delayed-allocation
Simplify delayed shard allocation
2015-11-19 09:58:52 +01:00
Yannick Welsch 2084df825f Simplify delayed shard allocation
- moves calculation of the delay to a single place (ReplicaShardAllocator)
- reduces coupling between GatewayAllocator and RoutingService
- in master failover situations, elapsed delay time is forgotten

Closes #14808
2015-11-19 09:53:07 +01:00
Xu Zhang 2e6d72de27 Catch exception when reading corrupted snapshot.
Single corrupted snapshot file shouldn't prevent listing all other
snapshot in repository.
2015-11-18 21:43:46 -08:00
Nicholas Knize 0d349854d3 Add CONTAINS relation to geo_shape query
At the time of geo_shape query conception, CONTAINS was not yet a supported spatial operation in Lucene. Since it is now available this commit adds ShapeRelation.CONTAINS to GeoShapeQuery. Randomized testing is included and documentation is updated.
2015-11-18 14:15:35 -06:00
David Pilato 052285ac88 Revert 3cacec4468
We actually want to keep the test when using deprecated setting in 3.0.
We will keep this setting deprecated as well so people will be able to update in a smoother way.

Also add the deprecating information to the migration documentation.
2015-11-18 19:14:26 +01:00
David Pilato dc17ab31d3 Merge remote-tracking branch 'origin/master' 2015-11-18 16:37:02 +01:00
craigwi 79a4d9ce36 Add support for secondary azure storage account
Follow up for #13228.

This commit adds support for a secondary storage account:

```yml
cloud:
    azure:
        storage:
            my_account1:
                account: your_azure_storage_account1
                key: your_azure_storage_key1
                default: true
            my_account2:
                account: your_azure_storage_account2
                key: your_azure_storage_key2
```

When creating a repository, you can choose which azure account you want to use for it:

```sh
curl -XPUT localhost:9200/_snapshot/my_backup1?pretty -d '{
  "type": "azure"
}'

curl -XPUT localhost:9200/_snapshot/my_backup2?pretty -d '{
  "type": "azure",
  "settings": {
    "account" : "my_account2",
    "location_mode": "secondary_only"
  }
}'
```

`location_mode` supports `primary_only` or `secondary_only`. Defaults to `primary_only`. Note that if you set it
to `secondary_only`, it will force `read_only` to true.
2015-11-18 16:31:48 +01:00
Clinton Gormley 685a0462d8 Merge pull request #13894 from dmland/patch-1
Update query-string-syntax.asciidoc
2015-11-18 15:50:30 +01:00
David Pilato 52bf365013 Add support for `daitch_mokotoff`
[Daitch Mokotoff](https://en.wikipedia.org/wiki/Daitch%E2%80%93Mokotoff_Soundex) support has been added in Lucene 5.
So we can now support it as well.
2015-11-18 15:41:39 +01:00
Clinton Gormley b855e7d14e Merge pull request #14778 from rothvaw/patch-1
Update update.asciidoc
2015-11-18 15:22:02 +01:00
Martijn van Groningen d2ae3ffa36 docs: can't use same call out twice 2015-11-18 15:16:13 +01:00
Martijn van Groningen e443bfc492 docs: fix callouts 2015-11-18 15:13:38 +01:00
Christoph Büscher ecbc7774ca Docs: Add query to Query DSL documentation
We already introduced the MatchNoneQueryBuilder query that does not
return any documents, mainly because we needed it for internal
representation of the NONE option in the IndicesQueryBuilder.
However, the query was requested at least once also for the query dsl,
and since we can parser it already we should document it as
`match_none` query in the relevant reference docs as well.
2015-11-18 14:39:28 +01:00
Martijn van Groningen 8a454dae33 field stats: Added a `format` option to index constraint that allows to specify date index constraint values in a different format then the for specified in the mapping.
Closes #14804
2015-11-18 14:19:07 +01:00
Clinton Gormley 062abf813b Update packaging.asciidoc
Remove mention of following symlinks under `path.scripts`

Closes #14733
2015-11-18 13:48:43 +01:00
Kopylov German 951ddcf4ea Fix missed comma in bool query 2015-11-18 03:02:52 +03:00
J Grubb c491dcc930 Update health.asciidoc
Changing network address of curl commands to "localhost" instead of 192.x.x.x
2015-11-17 14:46:18 -07:00
Tal Levy d0b5fc8bb4 Merge remote-tracking branch 'upstream/master' into feature/ingest 2015-11-17 12:08:26 -08:00
Jason Tedor 6872d545ac Add system CPU percent to OS stats
This commit adds the system CPU percent reflecting the recent CPU usage
for the whole system.
2015-11-17 13:48:46 -05:00
javanna ef4d2a9426 Merge branch 'master' into feature/ingest 2015-11-17 17:42:36 +01:00
Clinton Gormley 728cc5137a Merge pull request #14738 from petmit/patch-1
Update error in documentation for multi-fields
2015-11-17 17:33:53 +01:00
Clinton Gormley a2b92a65e9 Merge pull request #14540 from KangYongKyun/patch-1
colon is added
2015-11-17 15:54:59 +01:00
Boaz Leskes c6f0798a20 Update the resiliency page to 2.0.0 2015-11-17 14:32:33 +01:00
Clinton Gormley 98028419a5 Merge pull request #14610 from yokotaso/patch-1
Update snowball document page.
2015-11-17 14:17:30 +01:00
Krishna Raj f5ca9d0e10 Updated the correct doc for Kafka Consumer
Updated the correct doc for Kafka Consumer with link to latest repository.

Closes #14634
2015-11-17 14:11:03 +01:00
Isabel Drost-Fromm 7698ab7bfc Fix typos in query dsl docs.
When passing the example json snippets through the query parser while working
on #14249 some of the examples could not be parsed. This PR fixes those
examples.

Relates to #14249
2015-11-17 13:40:10 +01:00
Peter Lebedev 1201843d72 Update java.asciidoc
Add few more breaking changes.

Closes #14696
2015-11-17 12:27:44 +01:00
Shaunak Kashyap 636c88d09f Fixing setting name 2015-11-13 13:36:14 -08:00
Jason Tedor a846a257c5 Clarify stale shard issue on Resiliency page 2015-11-13 12:40:51 -05:00
Jason Tedor 7b16e82df4 Add stale shard issue to Resiliency page
This commit adds a simple description of the stale shard issue to the
Resiliency page.

Relates #14671
2015-11-11 08:30:54 -05:00
Martijn van Groningen da87cbf2b4 Merge remote-tracking branch 'es/master' into feature/ingest 2015-11-11 14:14:44 +07:00
Martijn van Groningen 4da05168f4 geoip: renamed `ip_field` option to `source_field`, because it can hold a ip or hostname.
geoip: add a `fields` option to control what fields are added by geoip processor
geoip: instead of adding all fields, only `country_code`, `city_name`, `location`, `continent_name` and `region_name` fields are added.
2015-11-10 11:50:12 +07:00
Tal Levy 8fc5a3d032 introduce mutate processor.
fix forbiddenapis

update

clean up and add rest test

update mutate factory to use configuration utilities

compile gsub pattern

cleanup, update parseBooleans, null tests
2015-11-09 18:49:55 -08:00
Adrien Grand 35c0b50879 Reword some documentation to make it more obvious that doc values are a columnar representation of the data.
Some users may already be familiar with column stores, so saying more explicitly
that doc values are a columnar representation of the data may help them better
and/or more quickly understand what doc values are about.
2015-11-09 23:32:47 +01:00
Lee Hinman 25f8db9aa9 Merge remote-tracking branch 'dakrone/node-limit-decider' 2015-11-09 11:03:19 -07:00
Lee Hinman 145374b762 Add cluster-wide setting for total shard limit
This adds the `cluster.routing.allocation.total_shards_per_node`
setting, which limits the total number of shards across all indices on
each node. It defaults to -1 and can be dynamically configured.

Resolves #14456
2015-11-09 11:03:07 -07:00
David Pilato e993c6a862 Migrate mapper attachements plugin to asciidoc
Followup for #14605
2015-11-09 15:35:06 +01:00
javanna ca980b7a83 [DOCS] document replacement for search exists
Relates to #13910
Closes #14393
2015-11-09 15:05:07 +01:00
Clinton Gormley fe473ed24e Documented passing a local truststore to bin/plugin 2015-11-09 14:33:23 +01:00
Clinton Gormley 40650ad2d5 Merge pull request #14612 from arinto/patch-2
[DOCS] Typo on the tribe node description
2015-11-09 14:00:56 +01:00
Clinton Gormley c49aaa1284 Merge pull request #14608 from jimmyjones2/patch-1
Update all-field.asciidoc
2015-11-09 13:43:25 +01:00
javanna f657a7dbf3 Merge branch 'feature/ingest' into ingest/date 2015-11-09 13:42:07 +01:00
javanna 0c1fa74c3a Merge branch 'master' into feature/ingest 2015-11-09 12:36:51 +01:00
Clinton Gormley bf3f9c778c Docs: Added plugin-descriptor.properties to breaking changes
Closes #14569
2015-11-09 12:17:44 +01:00
Adrien Grand 81767bc639 Merge pull request #14474 from jpountz/doc/search_body_fields_warning
Add a warning about fields vs. source filtering.
2015-11-09 11:52:58 +01:00
Clinton Gormley 0b9b9d7983 Revert "Removed redundant docs/community-clients/index.asciidoc"
These docs weren't redundant after all.

This reverts commit 22d163692c.
2015-11-08 23:24:45 +01:00
Clinton Gormley 4684be3680 Merge pull request #14391 from jurgc11/patch-2
Add community plugin
2015-11-08 22:12:26 +01:00
Clinton Gormley f0fda74083 Moved a breaking change to the correct section. Closes #14510 2015-11-08 21:53:56 +01:00
Clinton Gormley 1849c645f3 Merge pull request #14509 from elastic/markw/typo-parentchild-releasenotes-14508
Fixed minor typo
2015-11-08 21:48:40 +01:00
Clinton Gormley 22d163692c Removed redundant docs/community-clients/index.asciidoc 2015-11-08 21:45:38 +01:00
Clinton Gormley f7a7b192e4 Merge pull request #14504 from sckott/add-elasticdsl
Added new community R client, edited old entry
2015-11-08 21:41:55 +01:00
Clinton Gormley df865a2628 Merge pull request #14431 from arinto/patch-1
[DOCS] Missing tribe nodes' short description
2015-11-08 18:36:31 +01:00
Clinton Gormley 28015d9e03 Docs: Made the list of breaking changes in each version more obvious 2015-11-08 12:59:07 +01:00
Clinton Gormley 1220b89a60 Docs: Fixed bad link in completion suggester 2015-11-08 09:51:14 +01:00
Areek Zillur dd1c687ace Completion Suggester V2
The completion suggester provides auto-complete/search-as-you-type functionality.
This is a navigational feature to guide users to relevant results as they are typing, improving search precision.
It is not meant for spell correction or did-you-mean functionality like the term or phrase suggesters.

The completions are indexed as a weighted FST (finite state transducer) to provide fast Top N prefix-based
searches suitable for serving relevant results as a user types.

closes #10746
2015-11-07 17:46:27 -05:00
javanna a84ce07a90 added docs 2015-11-06 17:48:44 +01:00
Nik Everett fbeade92e5 [docs] Fix download link 2015-11-05 14:18:49 -05:00
Andrzej Wisłowski 7d010195f4 Update plugin-script.asciidoc
It is better not to show lang-groovy plugin in the example, as it is not used since elasticsearch 1.4.0
2015-11-05 10:27:34 -07:00
Clinton Gormley 6fa799e694 Docs: Correct docs around CONF_DIR and -Des.path.conf 2015-11-05 14:05:52 +01:00
KangYongKyun b5d49641fb colon is added
"predict"  10 => "predict" : 10
2015-11-05 11:32:20 +09:00
Yannick Welsch 825d0c64e6 Add duration field to /_cat/snapshots
Closes #14385
2015-11-04 10:34:00 +01:00
Mark Walkom 4f2154b0e0 Removed / from commands 2015-11-04 14:56:06 +11:00
Mark Walkom ab0cba4bbf Removed / from command 2015-11-04 14:54:40 +11:00
Mark Walkom 97a066829c Fixed minor typo 2015-11-04 14:46:37 +11:00
Scott Chamberlain 9d8c026dbc Added new community R client, edited old entry
I maintain two R clients, elastic and elasticdsl
adding entry for elasticdsl and edited elastic entry
2015-11-03 15:46:47 -08:00
xuzha fb1d8bb149 Add os.allocated_processors
Current processors setting is not reflected in nodes info API
("os.available_processors"). Add os.allocated_processors to shows
actual number of processors that we are using.
2015-11-03 09:50:17 -08:00
javanna ce6aa258a9 Remove support for query_binary and filter_binary
query_binary and filter_binary are unused at this point, as we only parse on the coordinating node and the java api only holds structured java objects for queries and filters, meaning they all implement Writeable and get natively serialized.

Relates to #14308
Closes #14433
2015-11-03 14:05:45 +01:00
Adrien Grand c9120c5c2a Docs: Add a warning about fields vs. source filtering.
Close #14470
2015-11-03 11:18:00 +01:00
Tanguy Leroux d819930261 [Doc] Fix correct number of slashes when installing a plugin with zip file 2015-11-03 10:32:44 +01:00
Martijn van Groningen 03387266ca ingest: Added new `geoip` processor, that adds geographical information to documents based on an ip address.
The information is fetched from the Maxmind geolite2 database, that is embedded in the ingest plugin.
2015-11-03 12:06:29 +07:00
Tal Levy 464b46437f Introduce the GrokProcessor
Also moved all processor classes into a subdirectory and introduced a
ConfigException class to be a catch-all for things that can go wrong
when constructing new processors with configurations that possibly throw
exceptions. The GrokProcessor loads patterns from the resources
directory.

fix resource path issue, and add rest-api-spec test for grok

fix rest-spec tests

changes: license, remove configexception, throw IOException

add more tests and fix iso8601-hour pattern

move grok patterns from resources to config

fix tests with pom changes, updated IngestClientIT with grok processor

update gradle build script for grok deps and test configuration

move config files to src/main/packaging

move Env out of Processor, fix test for src/main/packaging change

add docs

clean up test resources task

update Grok to be immutable

- Updated the Grok class to be immutable. This means that all the
  pattern bank loading is handled by an external utility class called
  PatternUtils.
- fixed tabs in the nagios patterns file's comments
2015-11-02 20:27:49 -08:00
Jason Tedor e3b8dc7121 Forbid changing thread pool types
This commit forbids the changing of thread pool types for any thread
pool. The motivation here is that these are expert settings with
little practical advantage.

Closes #14294, relates #2509, relates #2858, relates #5152
2015-11-02 20:52:48 -05:00
Tal Levy 4f5f2ff5e9 Merge remote-tracking branch 'upstream/master' into feature/ingest
Conflicts:
	plugins/pom.xml
	qa/smoke-test-plugins/pom.xml
2015-11-02 10:03:10 -08:00
David Pilato 77521560ed Merge branch 'ec2-improve-disco-nodes' of https://github.com/chaudum/elasticsearch into chaudum-ec2-improve-disco-nodes 2015-11-02 14:52:08 +01:00
Brady Vidovic 7ea5a7ea80 Update mapping PUT should use /_mapping
Instead of /mapping
2015-10-30 11:57:17 -06:00
David Pilato 4bbea60c40 Add FSCrawler 2015-10-30 17:36:25 +01:00
jurgc11 9550392e90 Add community plugin 2015-10-30 16:04:34 +00:00
Nik Everett 28633fae21 [mapping] Remove transform
Removes the mapping transform feature which when used made debugging very
difficult. Users should transform their documents on the way into
Elasticsearch rather than having Elasticsearch do it.

Closes #12674
2015-10-30 11:46:54 -04:00
Clinton Gormley 7f179cdab0 Docs: Added redirect for mapping-nested-type 2015-10-30 10:33:32 +01:00
Mark Walkom 10ad6ae10f Mention changes to repos.
Fixes 14376
2015-10-30 18:46:31 +11:00
Tal Levy 61d5ccbe7d Merge remote-tracking branch 'upstream/master' into feature/ingest 2015-10-29 16:44:37 -07:00
Lee Hinman 3b5058017e Merge branch 'remove-optimize-rest' 2015-10-29 15:18:03 -06:00
Clinton Gormley 16fb84bc34 Merge pull request #14296 from Dwaligon/master
Fixing misspelled words in documentation
2015-10-29 20:24:23 +01:00
xuzha b3090d654a Add AWS Canned ACL support
User could set cannedACL, S3 repository would use this cannedACL to
create s3 object and bucket.
2015-10-29 11:48:02 -07:00
Christian Haudum 3334160003 improved building of disco nodes
* improved retry policy of ec2 client
* cache results for 10s
2015-10-29 18:07:52 +01:00
Martijn van Groningen b0836d5d07 Merge remote-tracking branch 'es/master' into feature/ingest 2015-10-29 12:46:02 +07:00
xuzha 97ecd7bf5a Expose pending cluster state queue size in node stats
Add 3 stats about the queue: total queue size, number of committed cluster
states, and number of pending cluster states.
2015-10-28 10:59:15 -07:00
javanna 93b57089d0 update migrate guide according to recent deprecation removals 2015-10-28 10:50:29 +01:00
javanna 49f5757ae2 Remove support for multiple highlighter names
The only way to refer to the plain highlighter is now `plain`, the only way to refer to the fast vector highlighter is `fvh` and the only way to refer to the postings highlighter is `postings`. The name variants like `highlighter`, `postings-highlighter` and `fast-vector-highlighter` have been removed.
2015-10-28 10:50:29 +01:00
javanna 6076ccb7b2 Remove support for filter element in nested query
Replaced by query.
2015-10-28 10:50:29 +01:00
javanna f5fb669a46 [DOCS] remove tip on indices query elements order
Since we parse queries on the coordinating node, the order of the elements doesn't count anymore.
2015-10-28 10:50:28 +01:00
Lee Hinman 3a458af0b7 Remove /_optimize REST API endpoint
The `/_optimize` endpoint was deprecated in 2.1.0 and can now be removed
entirely.
2015-10-27 10:17:16 -06:00
javanna dc900a08a6 Remove "query" query and fix related parsing bugs
We have two types of parse methods for queries: one for the inner query, to be used once the parser is positioned within the query element, and one for the whole query source, including the query element that wraps the actual query.

With the search refactoring we ended up using the former in count, cat count and delete by query, whereas we should have used the former.  It ends up working properly given that we have a registered (deprecated) query called "query", which used to allow to wrap a filter into a query, but this has the following downsides:
1) prevents us from removing the deprecated "query" query
2) we end up supporting a top level query that is not wrapped within a query element (pre 1.0 syntax iirc that shouldn't be supported anymore)

This commit finally removes the "query" query and fixes the related parsing bugs. We also had some tests that were providing queries in the wrong format, those have been fixed too.

Closes #13326
Closes #14304
2015-10-27 14:54:30 +01:00
Martijn van Groningen 48e0a6745e Merge remote-tracking branch 'es/master' into feature/ingest 2015-10-27 16:16:45 +07:00
Jason O'Donnell 86c21f3c39 Fixing typo 2015-10-26 16:49:05 -04:00
Jason O'Donnell c7060c1b63 Fixing typo 2015-10-26 16:48:20 -04:00
Jason O'Donnell c7d47e0f4f Fixing typo 2015-10-26 16:47:44 -04:00
Jason O'Donnell 42fb690a1c Fixing typo 2015-10-26 16:46:36 -04:00
Jason O'Donnell 73f620907d Fixing typo 2015-10-26 16:43:25 -04:00
Jason O'Donnell 05d5d9bd12 Fixing typo 2015-10-26 16:42:45 -04:00
Yannick Welsch 2f10300a03 Merge pull request #14247 from ywelsch/feature/cat-snapshots
Add cat API for repositories and snapshots
2015-10-26 18:40:39 +01:00
Yannick Welsch ca75b7b6ce Add cat API for repositories and snapshots
Closes #14247
Closes #13919
2015-10-26 18:37:10 +01:00
Hendrik Saly 58f2b6ca47 Add Kerberos/SPNEGO Shield custom realm
Closes #14282
2015-10-26 13:53:04 +01:00
Clinton Gormley d2bb289c64 Merge pull request #14280 from salyh/patch-1
Added IMAP/POP3 Mailimporter to documentation
2015-10-26 13:47:44 +01:00
Robert Muir 6c8e290322 Allow binding to multiple addresses
* Allow for multiple host specifications (e.g. _en0_,192.168.1.2,_site_).
* Add _site_ and _global_ scopes as counterparts to _local_.
* Warn on heuristic selection of publish address.
* Remove the arbitrary _non_loopback_ setting.

Closes #13954
2015-10-23 23:43:37 -04:00
Christoph Büscher ca57763d2c Update indices-query.asciidoc 2015-10-23 22:55:40 +02:00
Christoph Büscher 0362550fd7 Update indices-query.asciidoc
The current description of the query seemed confusing to some people (e.g. https://twitter.com/wielinde/status/654582620630687744), this is just a proposal to simplify it.
2015-10-23 21:14:52 +02:00
Paul Echeverri d8dcf7b81e Backs out early link fix. 2015-10-22 15:00:35 -07:00
Paul Echeverri 8422e4fa10 Merge branch 'master-linkfix' 2015-10-22 14:29:35 -07:00
Simon Willnauer 14aaeea25a Merge pull request #14169 from s1monw/issues/14168
Add option to disable closing indices
2015-10-22 20:09:59 +02:00
debadair c258d8c341 Removed cross document link to field data topic. 2015-10-22 11:05:43 -07:00
Paul Echeverri 129f8f791f Fixes broken links. 2015-10-21 16:44:12 -07:00
javanna 75cedca0da Remove search exists api
Closes #13682
Closes #13911
2015-10-21 17:39:32 +02:00
Lee Hinman e5fa63e692 [DOCS] Fix doc link for optimize documentation 2015-10-20 22:53:14 -06:00
Lee Hinman b3646f9bfc Merge remote-tracking branch 'dakrone/use-the-force-luke' 2015-10-20 21:13:18 -06:00
debadair a5f4e46395 Removed include for not-query. 2015-10-20 14:28:00 -07:00
debadair 69acde33c2 Fixed broken xrefs to query-dsl-not-query, which has been removed. 2015-10-20 13:01:37 -07:00
Christoph Büscher 5d25bc30cd Query DSL: Remove NotQueryBuilder
The NotQueryBuilder has been deprecated on the 2.x branches
and can be removed with the next major version. It can be
replaced by boolean query with added mustNot() clause.

Closes #13761
2015-10-20 19:43:16 +02:00
Lee Hinman 9ea4909035 Add Force Merge API, deprecate Optimize API
This adds an API for force merging lucene segments. The `/_optimize` API is now
deprecated and replaced by the `/_forcemerge` API, which has all the same flags
and action, just a different name.
2015-10-20 09:00:24 -06:00
Jason Tedor 18b32f0ba5 Remove cache concurrency level settings that no longer apply
This commit removes some cache concurrency level settings that were
applicable when the cache was backed by the Guava cache implementation,
but no longer apply with the cache implementation completed in #13717.

Relates #7836, relates #13224, relates #13717
2015-10-20 10:15:03 -04:00
Isabel Drost-Fromm 4a5f9956a9 Remove reference to deleted limit query. 2015-10-20 11:09:25 +02:00
Isabel Drost-Fromm dac88c0281 Updates java query dsl documentation
Closes #13800
2015-10-20 09:37:36 +02:00
Pius ce8d07d9f4 Added max-at thread info
Added max-at thread info for snapshot, warmer and refresh pools.

See https://github.com/elastic/elasticsearch/blob/master/core/src/main/java/org/elasticsearch/threadpool/ThreadPool.java#L137
2015-10-19 12:13:14 -07:00
xuzha 6a6e168b8b Adding backoff from retries on GCE errors
In case of any error while trying to get GCE instances list from GCE
API, elasticsearch will slow down its API calls.
2015-10-19 10:28:08 -07:00
javanna a6e7a5f307 Java api: remove the count api
Closes #14166
Closes #13928
2015-10-19 14:40:52 +02:00
javanna c5152c7ecb [DOCS] terminate_after is not experimental anymore
we are relying on terminate_after more and more, replaced the limit filter with it and soon it will also replace the search_exists api. At that point we should make it a stable api rather than experimental.

Closes #14183
2015-10-19 13:56:42 +02:00
bharvi fd412551e4 Changed boost keyword with bool in bounding box query 2015-10-18 15:12:47 +05:30
bharvi 00db388722 The boost keyword changed to bool 2015-10-18 13:45:10 +05:30
Simon Willnauer db87594bca Add option to disable closing indices
Closed indices are currently out of scope for snapshots and shard migration,
and can cause issues in managed environments – where closing an index does
not necessarily make sense, as it still consumes the managed environment's storage quota.

This commit adds an option to dynamically disable closing indices via node or cluster settings.

Closes #14168
2015-10-16 22:10:51 +02:00
Clinton Gormley bb8097a655 Merge pull request #14162 from GeorgiCodes/patch-2
minor fixes to doc
2015-10-16 18:26:52 +02:00
Robert 43e7c5cd13 Names of indexRouting and searchRouting infos are out of date.
Hi,

I belive indexRouting should be changed to routing.index and searchRouting to routing.search.
This is what my ES 2.0.0-rc1 instance returns when I hit cat alias api:



Let me know if I should change other versions of documentation for this issue as well(Looks to me like it always was incorrect in docs).
2015-10-15 21:55:06 +02:00
Nicholas Knize b31d3ddd3e Adds geo_centroid metric aggregator
This commit adds a new metric aggregator for computing the geo_centroid over a set of geo_point fields. This can be combined with other aggregators (e.g., geohash_grid, significant_terms) for computing the geospatial centroid based on the document sets from other aggregation results.
2015-10-14 16:19:09 -05:00
Robert Muir 5d001d1578 Decentralize plugin security
* Add ability for plugins to declare additional permissions with a custom plugin-security.policy file and corresponding AccessController logic. See the plugin author's guide for more information.
* Add warning messages to users for extra plugin permissions in bin/plugin.
* When bin/plugin is run interactively (stdin is a controlling terminal and -b/--batch not supplied), require user confirmation.
* Improve unit test and IDE support for plugins with additional permissions by exposing plugin's metadata as a maven test resource.

Closes #14108

Squashed commit of the following:

commit cf8ace65a7397aaccd356bf55f95d6fbb8bb571c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 13:36:05 2015 -0400

    fix new unit test from master merge

commit 9be3c5aa38f2d9ae50f3d54924a30ad9cddeeb65
Merge: 2f168b8 7368231
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 12:58:31 2015 -0400

    Merge branch 'master' into off_my_back

commit 2f168b8038e32672f01ad0279fb5db77ba902ae8
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 12:56:04 2015 -0400

    improve plugin author documentation

commit 6e6c2bfda68a418d92733ac22a58eec35508b2d0
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 12:52:14 2015 -0400

    move security confirmation after 'plugin already installed' check, to prevent user from answering unnecessary questions.

commit 08233a2972554afef2a6a7521990283102e20d92
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 05:36:42 2015 -0400

    Add documentation and pluginmanager support

commit 05dad86c51488ba43ccbd749f0164f3fbd3aee62
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 02:22:24 2015 -0400

    Decentralize plugin permissions (modulo docs and pluginmanager work)
2015-10-14 14:46:45 -04:00
Tim Schlechter 7e2f558887 Added link to Swagger for Elasticsearch 2015-10-13 21:29:53 +02:00
Clinton Gormley 3e7201ef63 Merge pull request #14096 from speedplane/patch-2
Fixed a typo ("when when")
2015-10-13 21:17:09 +02:00
Clinton Gormley 14b6314766 Update fuzzy-query.asciidoc
Closes #14092
2015-10-13 21:03:46 +02:00
Adrien Grand fc0a69273d Merge pull request #14066 from sarwarbhuiyan/mapping-spelling-error
changed ben to been
2015-10-13 09:42:54 +02:00
Adrien Grand d3aa3565db Deprecate `index.analysis.analyzer.default_index` in favor of `index.analysis.analyzer.default`.
Close #11861
2015-10-12 22:19:16 +02:00
Sarwar Bhuiyan b876a4d571 changed he to the 2015-10-12 15:16:49 +01:00
Sarwar Bhuiyan 460c74f584 changed ben to been 2015-10-12 14:39:55 +01:00
Martijn van Groningen 5a3c75ebac added put, get and delete pipeline APIs. 2015-10-09 18:28:55 +02:00
Clinton Gormley 1a50ce73d7 Update setup.asciidoc
Added link to the support matrix to Setup docs
2015-10-09 12:27:43 +02:00
Clinton Gormley 499d53c506 Docs: Document ES_CLASSPATH removal in breaking changes
Closes #14008
2015-10-08 18:22:53 +02:00
Clinton Gormley c208add625 Merge pull request #14019 from ankon/patch-1
Fix minor typo
2015-10-08 18:19:00 +02:00
Clinton Gormley 8afddec2c9 Merge pull request #14031 from ankon/patch-3
Make the sentence actually understandable
2015-10-08 18:17:40 +02:00
Clinton Gormley 51dd4b3db5 Merge pull request #13999 from kkirsche/patch-2
Document that Debian 8 and Ubuntu 14 systemd
2015-10-08 18:09:29 +02:00
Andreas Kohn 2ffc2979d7 [doc] Make the sentence actually understandable 2015-10-08 16:37:17 +02:00
Kevin Kirsche 7150e1d17a Document that Debian 8 and Ubuntu 14 systemd
Fix #13865
2015-10-08 08:33:05 -04:00
Andreas Kohn 3bb24921a9 Remove excess comma in code block 2015-10-08 13:23:29 +02:00
Andreas Kohn a119e82ed0 Fix minor typo 2015-10-08 11:47:55 +02:00
David Pilato becaff30a8 Replace cloud-gce with discovery-gce
Related to #13815
2015-10-08 08:53:54 +02:00
David Pilato 4038aed641 Fix internal link name
Related to #13815
2015-10-08 07:04:11 +02:00
David Pilato 8b03d261ff Add missing breaking doc for azure and aws 2015-10-08 06:53:37 +02:00
David Pilato c73ab50df1 Rename cloud-gce plugin to discovery-gce plugin
Follow up azure and aws splits, we need to be consistent and rename `cloud-gce` to `discovery-gce`.
2015-10-08 06:53:37 +02:00
David Pilato 289cd5dcf4 [discovery-gce] add _gce_ network host setting
When running in GCE platform, an instance has access to:

http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip

Which gives back the private IP address, for example `10.240.0.2`.

http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/externalIp

Gives back the public Ip address, for example `130.211.108.21`.

As we have for `ec2`, we can support new network host settings:

* `_gce:privateIp:X_`: The private IP address of the machine for a given network interface.
* `_gce:hostname_`: The hostname of the machine.
* `_gce_`: Same as `_gce:privateIp:0_` (recommended).

Closes #13605.
Closes #13590.

BTW resolveIfPossible now throws IOException so code is also updated for ec2 discovery and
some basic tests have been added.
2015-10-07 22:04:34 +02:00
David Pilato c0363dd56b Update GCE discovery documentation
(cherry picked from commit d4cb81c)
(cherry picked from commit 9af04eb)
(cherry picked from commit b383344)
2015-10-07 21:43:12 +02:00
Nik Everett 3851093483 Merge pull request #13996 from nik9000/native_scripts
Rewrite native script documentation
2015-10-07 13:04:39 -04:00
Nik Everett 475e01f759 [doc] Rewrite native script documentation
Closes #13811
2015-10-07 13:03:51 -04:00
Kevin Kirsche 072f6ae138 Docs: Note that plugin removal usually requires node restart
Closes #13301
Closes #13983
2015-10-07 18:10:29 +02:00
Michele Palmia c67552eabc Docs: Fix example about executing filters
Closes #13979
2015-10-07 17:51:30 +02:00
Nik Everett b32022dce1 Merge pull request #13977 from nik9000/doc_breaking_java
Add a note about shard failure in the api
2015-10-07 10:36:50 -04:00
Alexander Reelsen 7b02e91e29 Docs: Update repository docs with new major version structure 2015-10-07 14:42:54 +02:00
Clinton Gormley dc018cf622 Updated docs for 3.0.0-beta 2015-10-07 13:27:46 +02:00
Boaz Leskes bcb3fab6ac Engine: Remove Engine.Create
The `_create` API is handy way to specify an index operation should only be done if the document doesn't exist. This is currently implemented in explicit code paths all the way down to the engine. However, conceptually this is no different than any other versioned operation - instead of requiring a document is on a specific version, we require it to be deleted (or non-existent). This PR removes Engine.Create in favor of a slight extension in the VersionType logic.

There are however a couple of side effects:
- DocumentAlreadyExistsException is removed and VersionConflictException is used instead (with an improved error message)
- Update will reject version parameters if the upsert option is used (it doesn't compute anyway).
- Translog.Create is also removed infavor of Translog.Index (that's OK because their binary format was the same, so we can just read Translog.Index of the translog file)

Closes #13955
2015-10-07 12:37:34 +02:00
Britta Weber 8a590d46b8 Merge pull request #13772 from brwe/remove-CONF_FILE
Remove option to configure custom config file via CONF_FILE or -Des.default.conf
2015-10-07 10:34:14 +02:00
Nik Everett 36a6211523 [doc] Add a note about shard failure in the api
Closes #13674
2015-10-06 16:24:39 -04:00
David Pilato f0252f34e9 Enhance plugin-descriptor.properties guide
* extract doc from the descriptor
* make obvious that plugin authors will have to release a new version for each elasticsearch version
2015-10-06 22:12:45 +02:00
Britta Weber a80a03f098 Merge pull request #13973 from brwe/conf-file-migration-doc
add paragraph about removal of conf file option
2015-10-06 19:39:15 +02:00
Britta Weber df81497e9f add paragraph about removal of conf file option 2015-10-06 19:36:48 +02:00
Alex 4077a322c5 Docs: Fix typo - datehistogram
date_histogram in place of datehistogram

Closes #13886
2015-10-06 19:22:21 +02:00
Britta Weber 71aefd5a06 remove option to configure custom config file via CONF_FILE or -Des.default.conf
It is rarely used and was not consistently handled by different distributions anyway.
This commit also adds a test for specifying CONF_DIR when installing plugins and
starting elasticsearch.

relates to #12712 and #12954
closes #5329
closes #13715
2015-10-06 19:02:43 +02:00
Alexander Reelsen b7ae59d8af Docs: Removed obsoleted MVEL paragraph 2015-10-06 18:55:34 +02:00
David Pilato 7b431ecc16 Fix cross links
Related to #13602
(cherry picked from commit ff4cdab)
(cherry picked from commit 8b46a4b)
2015-10-06 12:18:32 +02:00
David Pilato 685c1f4a54 [discovery-ec2] network.host must be set
With 2.0, we now bind to `localhost` by default instead of binding to the network card and use its IP address.

 When the discovery plugin gets from AWS API the list of nodes that should form the cluster, this list is pinged then. But as each node is bound to `localhost`, ping does not get an answer and the node elects itself as the master node.

`network.host` must be set.

 Closes #13589.
2015-10-06 11:18:47 +02:00
javanna e8653f5156 Java api: IdsQueryBuilder to accept only non null ids and types
Types are still optional, but if you do provide them, they can't be null. Split the existing constructor that accepted nnull into two, one that accepts no arguments, and another one that accepts the types argument, which must be not null.

Also trimmed down different ways of setting ids, some were misleading as they would always add the ids to the existing ones and not set them, the add prefix makes that clear. Left `addIds` method that accepts a varargs argument. Added check for ids not be null.
2015-10-05 15:10:30 +02:00
xuzha 8434c79429 Update `cat allocation` doc 2015-10-02 10:25:27 -07:00
Clinton Gormley 07b0a63c5f Merge pull request #13890 from veatch/patch-1
Clarify that aliases and indices can't share names
2015-10-02 18:11:54 +02:00
Clinton Gormley 5f1b867158 Update get-settings.asciidoc
Fixed docs for filtering index settings in get-settings API

Closes #13872
2015-10-02 16:46:45 +02:00
Clinton Gormley 8788516e1b Merge pull request #13868 from hafkensite/patch-1
Update example with parameters
2015-10-02 16:24:05 +02:00
Jose Diaz-Gonzalez 8782c8e08d Update link to Jepsen related test class 2015-10-01 16:34:19 -04:00
Jason Veatch cb0003ff6c Clarify that aliases and indices can't share names 2015-10-01 15:21:34 -04:00
Thomas Cucchietti ecc2985b84 Update inner-hits.asciidoc
Fix a glitch in inner_hits feature documentation (though I'm not absolutely sure of the final version)
2015-09-30 11:07:51 +02:00
Clinton Gormley 0c0f530c31 Merge pull request #13781 from KimTaehee/master
fix typo "Seting" to "Setting"
2015-09-25 15:28:04 +02:00
javanna eaafc62f53 [DOCS] added query-refactoring changes to 3.0 migrate guide 2015-09-25 14:55:37 +02:00
javanna 3d3197342d Merge branch 'master' into merge/query-refactoring 2015-09-25 14:38:53 +02:00
Clinton Gormley c2296b1d68 Merge pull request #13757 from andrestc/docs-now-time_zone
"now" is not affected by "time_zone" in range queries
2015-09-25 13:55:23 +02:00
javanna 3ac4da5f84 Merge branch 'master' into feature/query-refactoring 2015-09-25 13:53:06 +02:00
Clinton Gormley e97b731c46 Merge pull request #13748 from bcoughlin/patch-1
Update put-mapping.asciidoc
2015-09-25 12:13:33 +02:00
javanna 629ac8e014 Java api: remove TermsLookupQueryBuilder
TermsLookupQueryBuilder was left around only for bw comp reasons, but TermsQueryBuilder is its replacement. We can remove it now that it is clear query refactoring goes in master (3.0).
2015-09-25 11:44:05 +02:00
javanna 34de79370f Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/ElasticsearchException.java
2015-09-25 09:38:24 +02:00
Taehee Kim 45e0ccd274 Fix typo 2015-09-25 06:42:21 +09:00
André Carvalho 070c836dce Gives more emphasys 2015-09-24 11:30:00 -03:00
André Carvalho d90fc8f7fd Documents that now is not affected by time_zone in range queries 2015-09-23 22:00:49 -03:00
Jason Tedor 4fe243a5ca Merge pull request #13701 from jasontedor/fix-ping-timeout-settings-inconsistency
Fix ping timeout settings inconsistencies
2015-09-23 13:17:46 -04:00
Jason Tedor d8b29f7beb Fix ping timeout settings inconsistencies
This commit fixes ping timeout settings inconsistencies in
ZenDiscovery. In particular, the documentation refers to the ping
timeout setting as discovery.zen.ping_timeout but the code was
ultimately using discovery.zen.ping.timeout if this was set.

This commit also changes all instances of the raw string
“discovery.zen.ping_timeout” to the constant
o.e.d.z.ZenDiscovery.SETTING_PING_TIMEOUT.

Finally, this commit removes the legacy setting
"discovery.zen.initial_ping_timeout".

Closes #6579, #9581, #9908
2015-09-23 12:58:23 -04:00
Christoph Büscher dbb01f5b43 Query Refactoring: Move MoreLikeThisQueryBuilder validation to constructor
The current MoreLikeThisQueryBuilder validation checks for existence of at
least one `like` text or item. This is hard to check in setters, so this PR
tries to change the construction of the query so that we can do these checks
already at construction time.

Changing to using arrays for fieldnames, likeTexts, likeItems, unlikeTexts
and unlikeItems. `likeTexts` and/or `likeItems` need to be specified at
construction time to validate we have at least one item there.

Relates to #10217
2015-09-23 18:20:46 +02:00
xuzha a77c68ba0e Fix position-increment-gap doc example 2015-09-23 08:04:43 -07:00
ulkas e133fdd49f Update phrase-suggest.asciidoc
small sentence fix
2015-09-23 09:24:23 -04:00
Antonio Bonuccelli 0149ed41f4 Update settings.asciidoc
generalising to single dash
2015-09-23 14:54:51 +02:00
Antonio Bonuccelli 8da0f89739 Order of command line flags
specifying the order required for cli -Des notation arguments
2015-09-23 14:54:50 +02:00
Simon Willnauer 75e816400c Remove TranslogService and fold it into synchronous IndexShard API
This commit moves the size and ops based flush into a synchronous API into
IndexShard and removes the time-based flush alltogether since it' basically
covered by the inactive async flush API we have today. The functionality doesn't
need to be covered by scheduled task and async APIs while we can actually make all
the decisions in a sync manner which is way easier to control and to test.

Closes #13707
2015-09-23 12:39:06 +02:00
javanna 7eedd84dc3 Query refactoring: refactor query_string query and score functions
Refactor the function_score query so it can be parsed on the coordinating node, split parse into fromXContent and toQuery, make FunctionScoreQueryBuilder Writeable.

Closes #13653
2015-09-23 11:32:54 +02:00
Nik Everett 4d47015a0c Merge pull request #13579 from nik9000/run_plugin_as_elasticsearch_in_tar
Run bin/plugin as elasticsearch in tar distro
2015-09-22 11:19:19 -04:00
Nik Everett b521c606ff [test] Run bin/plugin as right user
Before this commit he tests always run bin/plugin as root which is somewhat
unrealistic and causes trouble (log files owned by root instead of
elasticsearch). After this commit `bin/plugin` runs as root when elasticsearch
is installed via the repository and as elasticsearch otherwise which is much
more realistic.

This also adds extra timeout to starting elasticsearch which is required
when all the plugins are installed. And it fixes up a problem with logging
elasticsearch's log if elasticsearch doesn't start which came up multiple
time while debugging this problem.

Also adds docs recommending running `bin/plugin` as the user that owns the
Elasticsearch files or root if installed with the packages.

Closes #13557
2015-09-22 11:17:57 -04:00
David Pilato 3049ef3e35 Merge pull request #13712 from mnylen/patch-1
Add docs note about cloud.aws.region (#13668)
2015-09-22 15:15:54 +02:00
Clinton Gormley 67815eec47 Docs: Removed references to java 7 and debian wheezy
Related to #13694
2015-09-22 14:23:30 +02:00
Mikko Nylén a379400228 Add docs note about cloud.aws.region (#13668) 2015-09-22 14:53:54 +03:00
Christoph Büscher 1aae68d2e8 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/CommonTermsQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/GeoPolygonQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/MatchQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/QueryParseContext.java
	core/src/main/java/org/elasticsearch/index/query/QueryParser.java
2015-09-22 12:43:55 +02:00
Christoph Büscher 03035a28a3 Query Refactoring: moving validation to setters and constructors
Moving validation from validate() to constructors and setters for the
following query builders:

* GeoDistanceQueryBuilder
* GeoDistanceRangeQueryBuilder
* GeoPolygonQueryBuilder
* GeoShapeQueryBuilder
* GeohashCellQuery
* TermsQueryBuilder

Relates to #10217
2015-09-22 11:05:56 +02:00
Nik Everett 877754ba89 reword docs 2015-09-21 14:32:22 -04:00
Nik Everett c67ad3fb2a [docs] Note that no http settings are dynamic
Closes #13364
2015-09-21 13:58:07 -04:00
David Pilato f230eabc15 [cloud-azure] Split azure plugin in 3 plugins
Until now we had a cloud-azure plugin which is providing 3 distinct features:

* discovery on Azure
* snapshot/restore on Aure
* SMB store

This commit splits the plugin by feature so people can use either one or the other or both features.

Doc is updated accordingly.
2015-09-21 17:55:23 +02:00
Christoph Büscher cc69de5c5f Merge branch 'master' into feature/query-refactoring 2015-09-21 15:38:43 +02:00
Clinton Gormley 1f76f49003 Update compound-word-tokenfilter.asciidoc
Improved the docs for compound work token filter.

Closes #13670
Closes #13595
2015-09-21 11:22:14 +02:00
Boaz Leskes cbdcdc1b96 Merge pull request #13526 from robertberg/rberg.searchdocs
Edit search scroll docs for syntactic style
2015-09-19 21:55:55 +02:00
Robert Berg 0279b219bb Edit search scroll docs for syntactic style 2015-09-19 14:42:14 -05:00
Clinton Gormley c08d5fdb24 Fixed some asciidoc confusion in index-modules.asciidoc 2015-09-19 18:14:36 +02:00
Clinton Gormley fa77cf6f6f Docs: Always quote "@file" argument to --data-binary
Closes #13500
2015-09-19 17:28:15 +02:00
Thomas Cucchietti 03488e3902 Docs: Update query_dsl.asciidoc
Minor orthographic fix

Closes #13452
2015-09-19 16:09:53 +02:00
Scott Kingsley Clark 50d984b935 SELECT state, COUNT(*) for clarity
Adding state to the SQL syntax to make it more clear, given that the buckets returned include the values of state AND count for each.

Closes #13597
2015-09-19 15:04:21 +02:00
Mark Walkom fa56f1da79 Docs: Mention we use DEFLATE for best_compression
Closes #13549
2015-09-19 14:02:41 +02:00
Simon Willnauer 2da1baf546 Merge branch 'master' into feature/query-refactoring 2015-09-18 20:38:15 +02:00
Simon Willnauer 197313c19b Merge branch 'master' into feature/query-refactoring 2015-09-18 20:11:25 +02:00
David Pilato 34862bc9f8 Merge branch 'doc/azure-network-host' 2015-09-18 19:51:57 +02:00
Britta Weber 542cf66c61 Update nodeattrs.asciidoc 2015-09-18 13:05:19 +02:00
Christoph Büscher bab9523c56 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/ElasticsearchException.java
2015-09-18 12:05:03 +02:00
Christoph Büscher 37b46fb79c Query Refactoring: moving validation to constructors and setters
This PR is the second batch in moving the query validation we started
to collect in the validate() method to the corresponding setters
and constructors.
2015-09-17 23:30:42 +02:00
David Pilato 71ec1f09e3 Add documentation for setting network.host with azure discovery
With 2.0, we now bind to `localhost` by default instead of binding to the network card and use its IP address.

When the discovery plugin gets from Azure API the list of nodes that should form the cluster, this list is pinged then. But as each node is bound to `localhost`, ping does not get an answer and the node elects itself as the master node.

Closes #13591
2015-09-17 16:29:02 +02:00
Christoph Büscher 887399eebf Query Refactoring: moving validation to constructors and setters
This PR is an initial step to move the query validation we started
to collect in the validate() method to the corresponding setters
and constructors.
2015-09-17 13:57:34 +02:00
Shaunak Kashyap 516ba6c8f6 Using gcloud CLI as gcutil CLI has been deprecated 2015-09-16 06:43:47 -07:00
Alex Ksikes a3c68f690b Refactors MultiMatchQueryBuilder and Parser
Relates to #10217

This PR is against the query-refactoring branch.

Closes #13405
2015-09-16 13:41:22 +02:00
Christoph Büscher 8fb1aa9781 Refactors GeoDistanceQueryBuilder/-Parser
Splits parsing and Lucene query generation. Switches from storing lat/lon
separately to using GeoPoint instead.

Relates to #10217
2015-09-16 11:15:32 +02:00
Alex Ksikes a13336da54 Refactors MoreLikeThisQueryBuilder and Parser
Relates to #10217

This PR is against the query-refactoring branch.

Closes #13486
2015-09-16 02:09:38 +02:00
Christoph Büscher 90f24c1a79 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/GeoDistanceRangeQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/GeoPolygonQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/GeohashCellQuery.java
	core/src/main/java/org/elasticsearch/index/query/WrapperQueryBuilder.java
	core/src/test/java/org/elasticsearch/search/child/ChildQuerySearchIT.java
2015-09-14 14:31:01 +02:00
Boaz Leskes 564e1eca42 merge from master 2015-09-14 09:03:45 +02:00
Lee Hinman 016ba3531a Merge remote-tracking branch 'dakrone/remove-disable-decider' 2015-09-11 09:28:13 -06:00
Nik Everett b205875c43 Merge pull request #13515 from elastic/docsfix
Fix for mappings->_source example in docs
2015-09-11 11:02:55 -04:00
Shane Connelly d86c1e8769 Fixes #13417 2015-09-11 07:34:14 -07:00
javanna 73f7df510e Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/FunctionScoreQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/FunctionScoreQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/factor/FactorParser.java
	core/src/test/java/org/elasticsearch/ExceptionSerializationTests.java
	core/src/test/java/org/elasticsearch/benchmark/search/child/ChildSearchBenchmark.java
	core/src/test/java/org/elasticsearch/benchmark/search/child/ChildSearchShortCircuitBenchmark.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
	core/src/test/java/org/elasticsearch/percolator/PercolatorIT.java
	core/src/test/java/org/elasticsearch/search/child/ChildQuerySearchIT.java
	docs/reference/query-dsl/has-parent-query.asciidoc
2015-09-11 14:15:12 +02:00
Alexander Pepper df9d4eca66 [docs] Document meaning of "FST" and "FSTs".
Conflicts:
	docs/reference/index-modules/fielddata.asciidoc
2015-09-11 05:34:41 -04:00
Adrien Grand 86f1b07df0 Docs: Remove docs for the `filtered`, `and`, `or` and `(f)query` queries. 2015-09-11 11:00:54 +02:00
Boaz Leskes 1b8047e51c merge master 2015-09-11 09:56:13 +02:00
Masaru Hasegawa 5ae00a6129 Take initializing shards into consideration during awareness allocation
It makes decision consistent.
Fixes #12522
2015-09-11 13:13:36 +09:00
Paul Echeverri 7dea10c053 Merge branch 'logstash-integration' 2015-09-10 14:35:46 -07:00
Nik Everett e4981968ad [search] Limit the size of the result window
Requesting a million hits, or page 100,000 is always a bad idea, but users
may not be aware of this. This adds a per-index limit on the maximum size +
from that can be requested which defaults to 10,000.

This should not interfere with deep-scrolling.

Closes #9311
2015-09-10 15:38:29 -04:00
Alexander Reelsen 5d6ee7e82e Docs: Update integration tests documentation 2015-09-10 19:05:15 +02:00
Martijn van Groningen ab0847e0df parent/child: several cleanups
* Dropped ScoreType in favour of Lucene's ScoreMode
* Removed `score_type` option from `has_child` and `has_parent` queries in favour for the already existing `score_mode` option.
* Removed the score mode `sum` in favour for the already existing `total` score mode. (`sum` doesn't exist in Lucene's ScoreMode class)
* If `max_children` is set to `0` it now really means that zero children are allowed to match.
2015-09-10 17:15:41 +02:00
javanna 56b3db6ba3 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/AndQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/AndQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/FQueryFilterParser.java
	core/src/main/java/org/elasticsearch/index/query/FilteredQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/FilteredQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/IndicesQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/LimitQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/LimitQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MultiMatchQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/QueryFilterBuilder.java
	core/src/main/java/org/elasticsearch/index/query/QueryFilterParser.java
	core/src/main/java/org/elasticsearch/index/query/QueryParseContext.java
	core/src/main/java/org/elasticsearch/index/query/ScriptQueryParser.java
	core/src/main/java/org/elasticsearch/indices/query/IndicesQueriesRegistry.java
	core/src/main/java/org/elasticsearch/search/sort/GeoDistanceSortParser.java
	core/src/main/java/org/elasticsearch/search/sort/SortParseElement.java
	core/src/test/java/org/elasticsearch/benchmark/search/child/ChildSearchShortCircuitBenchmark.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
2015-09-10 15:52:28 +02:00
Martijn van Groningen 2eadc6d595 nested sorting: If sorting by nested field then the `nested_path` should always be specified.
Closes #13420
2015-09-10 12:21:12 +02:00
Christoph Büscher 90fac17a2d Query refactoring: MatchQueryBuilder
This add equals, hashcode, read/write methods, separates toQuery and JSON parsing and adds tests.
Also moving MatchQueryBuilder.Type to MatchQuery to MatchQuery, adding serialization and hashcode,
equals there.

Relates to #10217
2015-09-10 12:14:55 +02:00
javanna 484fcd49e5 Query refactoring: query_string
Relates to #10217
2015-09-10 10:09:24 +02:00
Paul Echeverri f8c86c7252 Adds links to Logstash plugins under the Integrations page. 2015-09-09 17:57:23 -07:00
javanna db705ab460 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/mapper/internal/ParentFieldMapper.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MoreLikeThisQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/MoreLikeThisQueryParser.java
	core/src/main/java/org/elasticsearch/index/search/child/ChildrenQuery.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
	core/src/test/java/org/elasticsearch/index/search/child/AbstractChildTestCase.java
	core/src/test/java/org/elasticsearch/index/search/child/ChildrenConstantScoreQueryTests.java
	core/src/test/java/org/elasticsearch/index/search/child/ChildrenQueryTests.java
	core/src/test/java/org/elasticsearch/search/child/ChildQuerySearchBwcIT.java
	core/src/test/java/org/elasticsearch/search/child/ChildQuerySearchIT.java
	core/src/test/resources/org/elasticsearch/index/query/simple-query-string.json
2015-09-09 17:08:22 +02:00
Martijn van Groningen 73d84e4797 Merge pull request #13376 from martijnvg/remove_pre2x_slow_parent_child
Removed pre 2.x parent child implementation
2015-09-09 15:20:10 +02:00
David Pilato 35049a05c3 Allocation: add support for filtering by transport IP address
Allocation filtering by IP only works today using the node host address. But in some cases, you might want to filter using the publish address which could be different.
2015-09-09 15:15:53 +02:00
Martijn van Groningen 2fb2a12c52 Removed pre 2.x parent child implementation 2015-09-09 13:21:40 +02:00
Simon Willnauer d2e53e0e0c Split HasParentQueryParser into toQuery and formXContent
This commit splits HasParentQueryParser into toQuery and fromXContent.
This change also deprecates several keys in favor of simplified settings
and adds basic unittests for HasParentQueryParser.

Relates to #10217
2015-09-09 12:06:44 +02:00
Alex Ksikes a45ee273e3 MLT: builder takes a new Item object like its parser
Previously the parser could take any Term Vectors request, but this would be
not the case of the builder which would still use MultiGetRequest.Item. This
introduces a new Item class which is used by both the builder and parser.

Beyond that the rest is mostly cleanups such as:

1) Deprecating the ignoreLike methods, in favor to using unlike.

2) Deprecating and renaming MoreLikeThisBuilder#addItem to addLikeItem.

3) Ordering the methods of MoreLikeThisBuilder more logically.

This change is needed for the upcoming query refactoring of MLT.

Closes #13372
2015-09-09 11:40:27 +02:00
Nicholas Knize e4e71d8a9a add points_only option to GeoShapeFieldMapper for optimizing indexing performance on geo_shape indexes designed to store only points. Includes updated documentation and exception handling for ensuring index integrity on points only data. 2015-09-08 16:17:50 -05:00
Ryan Ernst 55795f8ec2 Merge pull request #13383 from rjernst/go_away_transport_paths
Remove environment from transport client
2015-09-08 13:28:21 -07:00
David Pilato 81223179df [doc] Java add missing breaking changes
Closes #13151
2015-09-08 21:23:20 +02:00
Ryan Ernst 86ffa38ec7 Add migration guide note about transport client changes to
settings/environment
2015-09-08 10:22:46 -07:00
Simon Willnauer 18bec264f9 Split HasChildQueryParser into toQuery and formXContent
This is an intial commit that splits HasChildQueryParser / Builder into
the two seperate steps. This one is particularly nasty since it transports
a pretty wild InnerHits object that needs heavy refactoring. Yet, this commit
has still some nocommits and needs more tests and maybe another cleanup but
it's a start to get the code out there.
2015-09-08 17:11:57 +02:00
javanna f2605b34d6 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/ScriptQueryParser.java
2015-09-08 16:11:22 +02:00
Clinton Gormley 2c20658204 Docs: Added deprecation notice for _timestamp and _ttl 2015-09-07 21:16:19 +02:00
Clinton Gormley 8aba6ce93a Docs: Improved the date histogram docs for time_zone and offset 2015-09-07 19:54:00 +02:00
Martijn van Groningen 11c87106ce docs: inner hits is no longer experimental 2015-09-07 16:58:46 +02:00
Adrien Grand 0c26e7cd83 Remove the scan and count search types.
These search types have been deprecated in 2.1 and 2.0 respectively, and will
be removed in 3.0.
2015-09-07 15:18:45 +02:00
xuzha e69551d8df This commit remove the block for user to set invalid Signer Type.
Instead of throw an exception, this change log a warning message
and accept any signer type.
2015-09-05 22:57:05 -07:00
Zachary Tong 397d5beae1 Aggregations: Add stats_bucket / extended_stats_bucket pipeline aggregations
These are the complements to the stats/extended_stats metric aggregations, and can be used
to calculate a variety of statistics over buckets
2015-09-04 15:23:48 -04:00
javanna be3409f1db Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/AndQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/BoolQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/ExistsQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/GeoShapeQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MissingQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/QueryParseContext.java
	core/src/main/java/org/elasticsearch/index/query/SimpleQueryStringParser.java
	core/src/main/java/org/elasticsearch/index/query/SpanContainingQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/SpanWithinQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/TermsQueryParser.java
	core/src/main/java/org/elasticsearch/search/sort/GeoDistanceSortParser.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
	core/src/test/java/org/elasticsearch/index/search/child/AbstractChildTestCase.java
2015-09-04 19:21:38 +02:00
Zachary Tong c5b39ce85e [DOCS] Fix broken inter-page link 2015-09-03 23:17:01 -04:00
Zachary Tong 1016734b4c Aggregations: Add percentiles_bucket pipeline aggregations
This pipeline will calculate percentiles over a set of sibling buckets.  This is an exact
implementation, meaning it needs to cache a copy of the series in memory and sort it to determine
the percentiles.

This comes with a few limitations: to prevent serializing data around, only the requested percentiles
are calculated (unlike the TDigest version, which allows the java API to ask for any percentile).
It also needs to store the data in-memory, resulting in some overhead if the requested series is
very large.
2015-09-03 22:24:14 -04:00
Robert Muir f216d92d19 Upgrade to lucene 5.4-snapshot r1701068 2015-09-03 15:13:33 -04:00
Lee Hinman ffb2f75305 Remove DisableAllocationDecider
The `EnableAllocatiorDecider` has replaced the
`DisableAllocationDecider`, which has been deprecated. It can now be
removed.
2015-09-03 09:10:48 -06:00
javanna fa93cc7b89 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/test/java/org/elasticsearch/ExceptionSerializationTests.java
2015-09-03 13:02:49 +02:00
Britta Weber 8e4e98099d Merge pull request #13307 from brwe/doc-script-fix
[doc] remove comment about function_score faster than script sort. It…
2015-09-03 12:36:33 +02:00
Britta Weber 2b27bc11b6 [doc] remove comment about function_score faster than script sort. It is not so. 2015-09-03 12:33:00 +02:00
David Pilato 4bffdbfafc [doc] fix cross link between core and plugins doc
For ec2 and s3
2015-09-03 12:11:58 +02:00
David Pilato 30aa231f8e [plugin] split cloud-aws in repository-s3 and discovery-ec2
Until now we had a cloud-aws plugin which is providing 2 disctinct features:

* discovery on EC2
* snapshot/restore on S3

This commit splits the plugin by feature so people can use either one or the other or both features.

Doc is updated accordingly.
2015-09-03 11:12:20 +02:00
Adrien Grand bd44dbe5cd Docs: Insist that setting size=0 will help performance. 2015-09-03 09:36:34 +02:00
javanna 52be313c69 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/BoolQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/IdsQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/SimpleQueryStringParser.java
	core/src/main/java/org/elasticsearch/index/query/TemplateQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/TermsQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/functionscore/fieldvaluefactor/FieldValueFactorFunctionParser.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/GND.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/JLHScore.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/NXYSignificanceHeuristic.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/PercentageScore.java
	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/ScriptHeuristic.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/SignificantTermsSignificanceScoreIT.java
2015-09-02 16:00:35 +02:00
Lee Hinman 118eab5462 Merge pull request #13257 from elastic/docsfix
Fixed non-valid JSON (though ES would accept it)
2015-09-02 07:51:13 -06:00
Colin Goodheart-Smithe 1d9905a798 [DOCS] Added note about valid return types for scripts in the scripted_metric aggregation 2015-09-02 12:13:15 +01:00
David Pilato 34ee4c2d66 [build] remove shaded elasticsearch version
The shaded version of elasticsearch was built at the very beginning to avoid dependency conflicts in a specific case where:

* People use elasticsearch from Java
* People needs to embed elasticsearch jar within their own application (as it's today the only way to get a `TransportClient`)
* People also embed in their application another (most of the time older) version of dependency we are using for elasticsearch, such as: Guava, Joda, Jackson...

This conflict issue can be solved within the projects themselves by either upgrade the dependency version and use the one provided by elasticsearch or by shading elasticsearch project and relocating some conflicting packages.

Example
-------

As an example, let's say you want to use within your project `Joda 2.1` but elasticsearch `2.0.0-beta1` provides `Joda 2.8`.
Let's say you also want to run all that with shield plugin.

Create a new maven project or module with:

```xml
<groupId>fr.pilato.elasticsearch.test</groupId>
<artifactId>es-shaded</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
    <elasticsearch.version>2.0.0-beta1</elasticsearch.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>${elasticsearch.version}</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.plugin</groupId>
        <artifactId>shield</artifactId>
        <version>${elasticsearch.version}</version>
    </dependency>
</dependencies>
```

And now shade and relocate all packages which conflicts with your own application:

```xml
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>2.4.1</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <relocations>
                            <relocation>
                                <pattern>org.joda</pattern>
                                <shadedPattern>fr.pilato.thirdparty.joda</shadedPattern>
                            </relocation>
                        </relocations>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>
```

You can create now a shaded version of elasticsearch + shield by running `mvn clean install`.

In your project, you can now depend on:

```xml
<dependency>
    <groupId>fr.pilato.elasticsearch.test</groupId>
    <artifactId>es-shaded</artifactId>
    <version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
    <groupId>joda-time</groupId>
    <artifactId>joda-time</artifactId>
    <version>2.1</version>
</dependency>
```

Build then your TransportClient as usual:

```java
TransportClient client = TransportClient.builder()
        .settings(Settings.builder()
                        .put("path.home", ".")
                        .put("shield.user", "username:password")
                        .put("plugin.types", "org.elasticsearch.shield.ShieldPlugin")
        )
        .build();
client.addTransportAddress(new InetSocketTransportAddress(new InetSocketAddress("localhost", 9300)));

// Index some data
client.prepareIndex("test", "doc", "1").setSource("foo", "bar").setRefresh(true).get();
SearchResponse searchResponse = client.prepareSearch("test").get();
```

If you want to use your own version of Joda, then import for example `org.joda.time.DateTime`. If you want to access to the shaded version (not recommended though), import `fr.pilato.thirdparty.joda.time.DateTime`.

You can run a simple test to make sure that both classes can live together within the same JVM:

```java
CodeSource codeSource = new org.joda.time.DateTime().getClass().getProtectionDomain().getCodeSource();
System.out.println("unshaded = " + codeSource);

codeSource = new fr.pilato.thirdparty.joda.time.DateTime().getClass().getProtectionDomain().getCodeSource();
System.out.println("shaded = " + codeSource);
```

It will print:

```
unshaded = (file:/path/to/joda-time-2.1.jar <no signer certificates>)
shaded = (file:/path/to/es-shaded-1.0-SNAPSHOT.jar <no signer certificates>)
```

This PR also removes fully-loaded module.

By the way, the project can now build with Maven 3.3.3 so we can relax a bit our maven policy.
2015-09-02 11:57:10 +02:00
Shane Connelly 5e385d5bf2 Fixed non-valid JSON (though ES would accept it) 2015-09-01 13:17:07 -07:00
Robert Muir 930a5ee170 Merge pull request #13244 from rmuir/lt
Lithuanian analysis
2015-09-01 15:42:58 -04:00
Nik Everett af394443e0 Merge pull request #13226 from nik9000/docs_position_increment_gap
[docs] Fix docs for position_increment_gap
2015-09-01 09:11:28 -04:00
Robert Muir 0d3e3f81fc Lithuanian analysis 2015-09-01 08:52:10 -04:00
Clinton Gormley 20921fcc3d Document transport.ping_schedule
Closes #13241
2015-09-01 13:43:17 +02:00
Clinton Gormley 1ee6ea9247 Docs: index.codec is static, not dynamic
The `index.codec` setting can only be set on a closed index, not dynamically
2015-09-01 11:49:42 +02:00
xuzha f46e66e7d0 Remove the experimental indices.fielddata.cache.expire
closes #10781
2015-09-01 00:40:04 -07:00
Nik Everett da16dcf527 [docs] Fix docs for position_increment_gap
Closes #13207
2015-08-31 14:05:55 -04:00
Tanguy Leroux db7aecab4d update list of available os stats
os cpu information is no longer exposed through the nodes stats api
2015-08-31 17:03:45 +02:00
Simon Willnauer 66b78341e4 Add note about multi data path and disk threshold deciders
Prior to 2.0 we summed up the available space on all disk on a node
due to the raid-0 like behavior. Now we don't do this anymore and use the
min & max disk space to make decisions.

Closes #13106
2015-08-31 16:23:54 +02:00
Michael McCandless a49217949f Merge pull request #13199 from mikemccand/remove_merge_docs
Move expert segment merge settings documentation off site into javadocs.
2015-08-31 09:52:19 -04:00
Clinton Gormley aa52c4f712 Docs: Fixed variations of spelling of buckets_path
Closes #13201
2015-08-31 13:47:40 +02:00
javanna faf526dccb Merge branch 'master' into feature/query-refactoring 2015-08-31 13:28:48 +02:00
Ryan Ernst 6295f8e795 Merge branch 'master' into tell_me_your_plugins 2015-08-30 14:20:54 -07:00
Michael McCandless 1c85b68674 Don't document expert segment merge settings 2015-08-29 17:21:46 -04:00
Clinton Gormley 3636563242 Docs: Fixed typo in getting started 2015-08-29 13:15:18 +02:00
Clinton Gormley 40dbba4e64 Update settings.asciidoc
Added removal of merge and merge throttling settings to breaking changes
2015-08-28 18:31:54 +02:00
Clinton Gormley 0331e444e6 Update mapping.asciidoc
Added fielddata format removal to breaking changes docs
2015-08-28 18:12:54 +02:00
Nik Everett 54639478c0 Merge pull request #13180 from nik9000/cat_non_verbose
Default _cat verbose to false
2015-08-28 11:59:29 -04:00
Clinton Gormley a46368310a Update mapping.asciidoc
Add type name length restriction to breaking changes docs.
2015-08-28 17:52:55 +02:00
Nik Everett 8a75d33ee6 Remove migration note about cat verbosity
Its no longer true.
2015-08-28 11:24:58 -04:00
Boaz Leskes d9f6e302b5 doc feedback 2015-08-28 12:31:45 +02:00
Boaz Leskes f70ed876d6 added docs 2015-08-28 12:31:45 +02:00
javanna 78d097de39 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/apache/lucene/queryparser/classic/MapperQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/AndQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/AndQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/BoolQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/DisMaxQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/DisMaxQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/GeoPolygonQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasChildQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/HasParentQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MoreLikeThisQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/MultiMatchQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/NestedQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/OrQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/PrefixQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/RegexpQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/SpanNearQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/SpanOrQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/TermQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/TermsQueryParser.java
	core/src/main/java/org/elasticsearch/index/query/support/InnerHitsQueryParserHelper.java
	core/src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java
2015-08-28 10:49:28 +02:00
Igor Motov 2b87d7d919 Add `readonly` option for repositories
Closes #7831
Closes #11753
2015-08-27 18:21:29 -04:00
Nik Everett 9eb684da51 Default detect_noop to true
detect_noop is pretty cheap and noop updates compartively expensive so this
feels like a sensible default.

Also had to do some testing and documentation around how _ttl works with
detect_noop.

Closes #11282
2015-08-27 10:34:18 -04:00
Dan Tuffery d8298e1d3a Update query_dsl.asciidoc
Fixed typo.'
2015-08-27 12:47:15 +02:00
Ryan Ernst fdf8e67de3 Merge branch 'master' into docs/multicast 2015-08-26 19:42:13 -07:00
Ryan Ernst 7a61adf8c4 Add links to multicast and dbq plugins 2015-08-26 19:40:00 -07:00
xuzha 9bd4a7b72e Fix doc build 2015-08-26 16:02:36 -07:00
xuzha fb2be6d6a1 The name "position_offset_gap" is confusing because Lucene has three
similar sounding things:

* Analyzer#getPositionIncrementGap
* Analyzer#getOffsetGap
* IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS and
* FieldType#storeTermVectorOffsets

Rename position_offset_gap to position_increment_gap
closes #13056
2015-08-26 14:56:35 -07:00
Alex Ksikes 0872a6ea39 removes TermsLookup changes as it was used internally 2015-08-26 14:07:55 +02:00
Alex Ksikes 1af0a39221 Refactors TermsQueryBuilder and Parser
Refactors TermsQueryBuilder and Parser for #10217.

This PR is against the query-refactoring branch.

Closes #12042
2015-08-26 13:58:46 +02:00
Christoph Büscher efadf87371 Merge branch 'master' into feature/query-refactoring 2015-08-26 10:23:03 +02:00
Nik Everett 3f6108ab74 [Mapping] Move position_offset_gap default change
Until a couple of hours ago we expected the position_offset_gap to default
to 0 in 2.0 and 100 in 2.1. We decided it was worth backporting that new
default to 2.0. So now that its backported we need to teach 2.1 that 2.0
also defaults to 100.

Closes #7268
2015-08-25 18:01:10 -04:00
Nik Everett 840b4cf0e1 [docs] Fix typo
grap -> gap
2015-08-25 15:38:14 -04:00
Nik Everett 4b9664beeb Mapping: Default position_offset_gap to 100
This is much more fiddly than you'd expect it to be because of the way
position_offset_gap is applied in StringFieldMapper. Instead of setting
the default to 100 its simpler to make sure that all the analyzers default
to 100 and that StringFieldMapper doesn't override the default unless the
user specifies something different. Unless the index was created before
2.1, in which case the old default of 0 has to take.

Also postition_offset_gaps less than 0 aren't allowed at all.

New tests test that:
1. the new default doesn't match phrases across values with reasonably low
slop (5)
2. the new default doest match phrases across values with reasonably high
slop (50)
3. you can override the value and phrases work as you'd expect
4. if you leave the value undefined in the mapping and define it on a
custom analyzer the the value from the custom analyzer shines through

Closes #7268
2015-08-25 14:21:50 -04:00
Clinton Gormley 7fcfe42f13 Docs: Removed river docs from master 2015-08-25 17:59:07 +02:00
Clinton Gormley 327ceec12d Docs: Added a networking section to migrating changes and mentioned multicast removal 2015-08-25 17:53:44 +02:00
Christoph Büscher 59cb67c7bd Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/support/NestedInnerQueryParseSupport.java
2015-08-25 11:59:40 +02:00
Ryan Ernst d5a2d8a64d Docs: Add migration guide notes for multicast moving to a plugin
See #13027
2015-08-24 14:08:09 -07:00
Ryan Ernst d06be0707c Added link to test framework for plugin authors 2015-08-24 12:05:00 -07:00
Ryan Ernst 297f0f22d7 Merge branch 'master' into tell_me_your_plugins 2015-08-24 12:03:25 -07:00
Nik Everett 2953985659 [docs] Document noop behavior of index api
Relates to #12969
2015-08-24 12:19:07 -04:00
Adrien Grand 7b878b5b5c Docs: Document the `_doc` sort order. 2015-08-24 15:39:50 +02:00