Commit Graph

3117 Commits

Author SHA1 Message Date
Nik Everett da96b6e41d [reindex] Add thottling support
The throttle is applied when starting the next scroll request so that its
timeout can include the throttle time.
2016-03-22 12:34:14 -04:00
javanna eebd0cfccd Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-22 10:34:40 +01:00
Simon Willnauer 47f0e6e8f4 [TEST] Disable InternalClusterInfoService in messy tests, it sends IndicesStatsRequest periodically which messes with the messy test 2016-03-22 10:12:03 +01:00
javanna bf390a935e Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-21 17:18:23 +01:00
Boaz Leskes 2d1152ebac Remove ClusterService interface, in favor of it's only production instance #17183
We current have a ClusterService interface, implemented by InternalClusterService and a couple of test classes. Since the decoupling of the transport service and the cluster service, one can construct a ClusterService fairly easily, so we don't need this extra indirection.

Closes #17183
2016-03-21 13:55:10 +01:00
Martijn van Groningen e3b7e5d75a percolator: Replace percolate api with the new percolator query
Also replaced the PercolatorQueryRegistry with the new PercolatorQueryCache.

The PercolatorFieldMapper stores the rewritten form of each percolator query's xcontext
in a binary doc values field. This make sure that the query rewrite happens only during
indexing (some queries for example fetch shapes, terms in remote indices) and
the speed up the loading of the queries in the percolator query cache.

Because the percolator now works inside the search infrastructure a number of features
(sorting fields, pagination, fetch features) are available out of the box.

The following feature requests are automatically implemented via this refactoring:

Closes #10741
Closes #7297
Closes #13176
Closes #13978
Closes #11264
Closes #10741
Closes #4317
2016-03-21 12:21:50 +01:00
Ryan Ernst f71f0d6010 Revert "Build: Switch to maven-publish plugin"
This reverts commit a90a2b34fc.
2016-03-18 17:22:25 -07:00
Ryan Ernst 6af4c43c4f Merge pull request #17128 from rjernst/maven_publish
Build: Switch to maven-publish plugin
2016-03-17 11:53:50 -07:00
Jack Conradson 800c844ebd Split up Analyzer and Writer into multiple pieces.
Closes #17158
2016-03-17 10:15:13 -07:00
Simon Willnauer e91a141233 Prevent index level setting from being configured on a node level
Today we allow to set all kinds of index level settings on the node level which
is error prone and difficult to get right in a consistent manner.
For instance if some analyzers are setup in a yaml config file some nodes might
not have these analyzers and then index creation fails.

Nevertheless, this change allows some selected settings to be specified on a node level
for instance:
 * `index.codec` which is used in a hot/cold node architecture and it's value is really per node or per index
 * `index.store.fs.fs_lock` which is also dependent on the filesystem a node uses

All other index level setting must be specified on the index level. For existing clusters the index must be closed
and all settings must be updated via the API on each of the indices.

Closes #16799
2016-03-17 14:42:18 +01:00
Christoph Büscher 6ddf9ae92f Merge branch 'master' into feature-suggest-refactoring 2016-03-16 15:27:02 +01:00
Nik Everett 7197172047 [reindex] Properly register status
Without this commit fetching the status of a reindex from a node that isn't
coordinating the reindex will fail. This commit properly registers reindex's
status so this doesn't happen. To do so it moves all task status registration
into NetworkModule and creates a method to register other statuses which the
reindex plugin calls.
2016-03-16 07:40:49 -04:00
Christoph Büscher 39667b5793 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	docs/reference/migration/migrate_5_0/java.asciidoc
2016-03-16 12:06:42 +01:00
Ryan Ernst a90a2b34fc Build: Switch to maven-publish plugin
The build currently uses the old maven support in gradle. This commit
switches to use the newer maven-publish plugin. This will allow future
changes, for example, easily publishing to artifactory.

An additional part of this change makes publishing of build-tools part
of the normal publishing, instead of requiring a separate upload step
from within buildSrc. That also sets us up for a follow up to enable
precomit checks on the buildSrc code itself.
2016-03-15 19:16:37 -07:00
Jason Tedor 618441aea3 Merge pull request #17088 from jasontedor/simplify-bootstrap-settings
Bootstrap does not set system properties
2016-03-15 19:25:16 -04:00
Jason Tedor 66ba044ec5 Use setting in integration test cluster config 2016-03-15 17:45:17 -04:00
Yannick Welsch 7fb07a9a53 Merge pull request #17112 from ywelsch/enhance/forbid-sysout-for-tests
Forbid test sources to use System.out.println and Throwable.printStackTrace
2016-03-15 16:36:34 +01:00
Yannick Welsch f5e6db4090 Remove System.out.println and Throwable.printStackTrace from tests 2016-03-15 15:40:37 +01:00
Christoph Büscher b4b874f0d8 Merge branch 'master' into feature-suggest-refactoring 2016-03-15 12:11:39 +01:00
Christoph Büscher 1f1f6861b7 Merge branch 'master' into sort-serialization-scriptsort
Conflicts:
	core/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java
2016-03-15 11:03:28 +01:00
Simon Willnauer 345e988bbc Merge pull request #17072 from s1monw/add_backwards_rest_tests
Add infrastructure to run REST tests on a multi-version cluster

This change adds the infrastructure to run the rest tests on a multi-node
cluster that users 2 different minor versions of elasticsearch. It doesn't implement
any dedicated BWC tests but rather leverages the existing REST tests.

Since we don't have a real version to test against, the tests uses the current version
until the first minor / RC is released to ensure the infrastructure works.

Given the amount of problems this change already found I think it's worth having this run with our test suite by default. The structure of this infra will likely change over time but for now it's a step into the right direction. We will likely want to split it up into integTests and integBwcTests etc. so each plugin can have it's own bwc tests but that's left for future refactoring.
2016-03-15 09:17:43 +01:00
Ali Beyad f527a034a3 Merge remote-tracking branch 'upstream/master' into feature-suggest-refactoring 2016-03-14 17:25:02 -04:00
Jack Conradson dfec4547ea Added one minor comment for expressions tests. 2016-03-14 13:19:52 -07:00
Simon Willnauer bd96075f7f Merge branch 'master' into add_backwards_rest_tests 2016-03-14 21:18:42 +01:00
Jack Conradson 7665c4fe6f Merge branch 'master' into pr/17091 2016-03-14 12:16:21 -07:00
Christoph Büscher 40f3501d7f Merge branch 'master' into feature-suggest-refactoring 2016-03-14 14:57:57 +01:00
Christoph Büscher f0074668db Merge branch 'master' into sort-serialization-scriptsort 2016-03-14 12:15:53 +01:00
Christoph Büscher 97638c95fc Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	docs/reference/migration/migrate_5_0.asciidoc
2016-03-14 11:13:47 +01:00
Simon Willnauer 31740e279f Resolve index names to Index instances early
Today index names are often resolved lazily, only when they are really
needed. This can be problematic especially when it gets to mapping updates
etc. when a node sends a mapping update to the master but while the request
is in-flight the index changes for whatever reason we would still apply the update
since we use the name of the index to identify the index in the clusterstate.
The problem is that index names can be reused which happens in practice and sometimes
even in a automated way rendering this problem as realistic.
In this change we resolve the index including it's UUID as early as possible in places
where changes to the clusterstate are possible. For instance mapping updates on a node use a
concrete index rather than it's name and the master will fail the mapping update iff
the index can't be found by it's <name, uuid> tuple.

Closes #17048
2016-03-14 11:08:48 +01:00
Alexander Kazakov 8e6b2b3909 Check that _value is used in aggregations script before setting value to specialValue #14262 2016-03-14 12:04:06 +03:00
Adrien Grand 5596e31068 Upgrade to lucene-6.0.0-f0aa4fc. #17075 2016-03-14 07:58:52 +01:00
Jason Tedor 8a05c2a2be Bootstrap does not set system properties
Today, certain bootstrap properties are set and read via system
properties. This action-at-distance way of managing these properties is
rather confusing, and completely unnecessary. But another problem exists
with setting these as system properties. Namely, these system properties
are interpreted as Elasticsearch settings, not all of which are
registered. This leads to Elasticsearch failing to startup if any of
these special properties are set. Instead, these properties should be
kept as local as possible, and passed around as method parameters where
needed. This eliminates the action-at-distance way of handling these
properties, and eliminates the need to register these non-setting
properties. This commit does exactly that.

Additionally, today we use the "-D" command line flag to set the
properties, but this is confusing because "-D" is a special flag to the
JVM for setting system properties. This creates confusion because some
"-D" properties should be passed via arguments to the JVM (so via
ES_JAVA_OPTS), and some should be passed as arguments to
Elasticsearch. This commit changes the "-D" flag for Elasticsearch
settings to "-E".
2016-03-13 20:09:15 -04:00
Simon Willnauer 121e7c8ca4 Add infrastructure to run REST tests on a multi-version cluster
This change adds the infrastructure to run the rest tests on a multi-node
cluster that users 2 different minor versions of elasticsearch. It doesn't implement
any dedicated BWC tests but rather leverages the existing REST tests.

Since we don't have a real version to test against, the tests uses the current version
until the first minor / RC is released to ensure the infrastructure works.

Relates to #14406
Closes #17072
2016-03-13 10:52:39 +01:00
Ryan Ernst 5bd7da5659 Addressed PR feedback
* Fix tests still referring to -E
* add comment about missing classes
* rename writer constant
2016-03-11 11:46:23 -08:00
Ryan Ernst 591fb8f028 Merge branch 'master' into cli-parsing 2016-03-11 10:45:05 -08:00
Ali Beyad 31dcb3e18b Merge pull request #16873 from abeyad/suggester-wiring-refactoring
Change internal representation of suggesters
2016-03-11 12:34:35 -05:00
Christoph Büscher bbcbba1bf5 Fixing some tests and compile problems in reindex module 2016-03-11 17:58:13 +01:00
Christoph Büscher 5107388fe9 Added enum for script sort type 2016-03-11 17:32:39 +01:00
Christoph Büscher daeffb149c Merge branch 'master' into feature-suggest-refactoring 2016-03-11 10:37:28 +01:00
Yannick Welsch 04e55ecf6b Make logging message String constant to allow static checks 2016-03-11 10:30:59 +01:00
Yannick Welsch 718876a941 Fix wrong placeholder usage in logging statements 2016-03-11 10:30:59 +01:00
Nik Everett ebc12690bc [reindex] Move refresh tests to unit test
The refresh tests were failing rarely due to refreshes happening
automatically on indexes with -1 refresh intervals. This commit moves
the refresh test into a unit test where we can check if it was attempted
so we never get false failures from background refreshes.

It also stopped refresh from being run if the reindex request was canceled.
2016-03-10 17:48:22 -05:00
Nik Everett b2eec96045 [reindex] Make search failure cause rest failure
Indexing failures have caused the reindex http request to fail for a while
now. Both search and indexing failures cause it to abort. But search
failures didn't cause a non-200 response code from the http api. This
fixes that.

Also slips in a fix to some infrequently failing rest tests.

Closes #16037
2016-03-10 13:47:49 -05:00
Nik Everett b8d931d23c [reindex] Timeout if sub-requests timeout
Sadly, it isn't easy to simulate a timeout during an integration test, you
just have to cause one. Groovy's sleep should do the job.
2016-03-10 13:05:23 -05:00
Christoph Büscher 69c83b3459 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/action/suggest/TransportSuggestAction.java
        core/src/main/java/org/elasticsearch/search/suggest/SuggestParseElement.java
	core/src/main/java/org/elasticsearch/search/suggest/SuggestionSearchContext.java
	core/src/main/java/org/elasticsearch/search/suggest/phrase/PhraseSuggester.java
2016-03-10 11:44:36 +01:00
Nik Everett 38241a5d8b [reindex] Implement CompositeIndicesRequest
Implements CompositeIndicesRequest on UpdateByQueryRequest and
ReindexRequest so that plugins can reason about the request. In both cases
this implementation is imperfect but useful because instead of listing
all requests that make up the request it instead attempts to make dummy
requests that represent the requests that it will later make.
2016-03-09 16:29:23 -05:00
Simon Willnauer 016bc3ea2b Merge pull request #17001 from s1monw/cut_over_to_uuid
Use index UUID to lookup indices on IndicesService

Today we use the index name to lookup index instances on the IndicesService
which applied to search requests but also to index deletion etc. This commit
moves the interface to expect an Index instance which is a tuple
and looks up the index by uuid rather than by name. This prevents accidental modification
of the wrong index if and index is recreated or searching from the wrong index in such a case.
Accessing an index that has the same name but different UUID will now result in an IndexNotFoundException.
2016-03-09 19:44:36 +01:00
Simon Willnauer e72dac91b3 Use index UUID to lookup indices on IndicesService
Today we use the index name to lookup index instances on the IndicesService
which applied to search reqeusts but also to index deletion etc. This commit
moves the interface to expcet and `Index` instance which is a <name, uuid> tuple
and looks up the index by uuid rather than by name. This prevents accidential modificaiton
of the wrong index if and index is recreated or searching from the _wrong_ index in such a case.
Accessing an index that has the same name but different UUID will now result in an IndexNotFoundException.

Closes #17001
2016-03-09 19:42:15 +01:00
Ryan Ernst 1dafead2eb Fix precommit 2016-03-08 22:55:24 -08:00
Nik Everett 6d0efae713 Teach list tasks api to wait for tasks to finish
_wait_for_completion defaults to false. If set to true then the API will
wait for all the tasks that it finds to stop running before returning. You
can use the timeout parameter to prevent it from waiting forever. If you
don't set a timeout parameter it'll default to 30 seconds.

Also adds a log message to rest tests if any tasks overrun the test. This
is just a log (instead of failing the test) because lots of tasks are run
by the cluster on its own and they shouldn't cause the test to fail. Things
like fetching disk usage from the other nodes, for example.

Switches the request to getter/setter style methods as we're going that
way in the Elasticsearch code base. Reindex is all getter/setter style.

Closes #16906
2016-03-08 11:53:57 -05:00
Christoph Büscher 1264f37a1b Merge branch 'master' into feature-suggest-refactoring 2016-03-08 11:10:08 +01:00
Christoph Büscher 5ff413074a Adding tests for `time_zone` parameter for date range aggregation 2016-03-07 15:38:24 +01:00
Christoph Büscher 7ec5075a87 Merge branch 'master' into feature-suggest-refactoring 2016-03-07 10:45:43 +01:00
Adrien Grand c69fc008d4 Merge pull request #16972 from alexshadow007/fix-16812
Build empty extended stats aggregation if no docs collected for bucket
2016-03-07 10:44:20 +01:00
Robert Muir 54018a5d37 upgrade to lucene 6.0.0-snapshot-bea235f
Closes #16964

Squashed commit of the following:

commit a23f9d2d29220991aa498214530753d7a5a148c6
Merge: eec9c4e 0b0a251
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Mar 7 04:12:02 2016 -0500

    Merge branch 'master' into lucene6

commit eec9c4e5cd11e9c3e0b426f04894bb2a6dae4f21
Merge: bc67205 675d940
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 13:45:00 2016 -0500

    Merge branch 'master' into lucene6

commit bc67205bdfe1526eae277ab7856fc050ecbdb7b2
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 09:56:31 2016 -0500

    fix test bug

commit a60723b007ff12d97b1810cef473bd7b553a0327
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 15:35:35 2016 +0100

    Fix SimpleValidateQueryIT to put braces around boosted terms

commit ae3a49d7ba7ced448d2a5262e5d8ec98671a9090
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 15:27:25 2016 +0100

    fix multimatchquery

commit ae23fdb88a8f6d3fb7ba60fd1aaf3fd72d899aa5
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 15:20:49 2016 +0100

    Rewrite DecayFunctionScoreIT to be independent of the similarity used

    This test relied a lot on the term scoring and compared scores
    that are dependent on the similarity. This commit changes the base query
    to be a predictable constant score query.

commit 366c2d518c35d31251033f1b6f6a93f6e2ae327d
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 14:06:14 2016 +0100

    Fix scoring in tests due to changes to idf calculation.

    Lucene 6 uses a different default similarity as well as a different
    way to calculate IDF. In contrast to older version lucene 6 uses docCount per field
    to calculate the IDF not the # of docs in the index to overcome the sparse field
    cases.

commit dac99fd64ac2fa71b8d8d106fe68825e574c49f8
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 08:21:57 2016 -0500

    don't hardcoded expected termquery score

commit 6e9f340ba49ab10eed512df86d52a121aa775b0f
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 08:04:45 2016 -0500

    suppress deprecation warning until migrated to points

commit 3ac8908424b3fdad44a90a4f7bdb3eff7efd077d
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 07:21:43 2016 -0500

    Remove invalid test: all commits have IDs, and its illegal to do this.

commit c12976288124ad1a26467e7e848fb810548e7eab
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 07:06:14 2016 -0500

    don't test with unsupported back compat

commit 18bbfe76128570bc70883bf91ff4c44c82d27817
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 07:02:18 2016 -0500

    remove now invalid lucene 4 backcompat test

commit 7e730e572886f0ef2d3faba712e4256216ff01ec
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 06:58:52 2016 -0500

    remove now invalid lucene 4 backwards test

commit 244d2ab6868ba5ac9e0bcde3c2833743751a25ec
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 06:47:23 2016 -0500

    use 6.0 codec

commit 5f64d4a431a6fdaa1234adca23f154c2a1de8284
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 06:43:08 2016 -0500

    compile, javadocs, forbidden-apis, etc

commit 1f273cd62a7fe9ca8f8944acbbfc5cbdd3d81ccb
Merge: cd33921 29e3443
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 10:45:29 2016 +0100

    Merge branch 'master' into lucene6

commit cd33921ac742ef9fb351012eff35f3c7dbda7264
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:58:37 2016 -0500

    fix hunspell dictionary loading

commit c7fdbd837b01f7defe9cb1c24e2ec65604b0dc96
Merge: 4d4190f d8948ba
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:41:53 2016 -0500

    Merge branch 'master' into lucene6

commit 4d4190fd82601aaafac6b8254ccb3edf218faa34
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:39:14 2016 -0500

    remove nocommit

commit 77ca69e288b1a41aa9595c921ed166c272a00ea8
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:38:24 2016 -0500

    clean up numericutils vs legacynumericutils

commit a466d696fbaad04b647ffbc0857a9439b583d0bf
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:32:43 2016 -0500

    upgrade spatial4j

commit 5412c747a8cfe638bacedbc8233163cb75cc3dc5
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:19:28 2016 -0500

    move to 6.0.0-snapshot-8eada27

commit b32bfe924626b87e540692375ece09e7c2edb189
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 11:30:09 2016 +0100

    Fix some test compile errors.

commit 6ccde35e9840b03c68d1a2cd47c7923a06edf64a
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 11:25:51 2016 +0100

    Current Lucene version is 6.0.0.

commit f62e1015d931b4cc04c778298a8fa1ba65e97ad9
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 11:20:48 2016 +0100

    Fix compile errors in NGramTokenFilterFactory.

commit 6837c6eabf96075f743649da9b9b52dd39611c58
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:50:59 2016 +0100

    Fix the edge ngram tokenizer/filter.

commit ccd7f070de5efcdfbeb34b9555c65c4990bf1ba6
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:42:44 2016 +0100

    The missing value is now accessible through a getter.

commit bd3b77f9b28e5b05daa3d49683a9922a6baf2963
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:41:51 2016 +0100

    Remove IndexCacheableQuery.

commit 05f3091c347aeae80eeb16349ac51d2b53cf86f7
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:39:43 2016 +0100

    Fix compilation of function_score queries.

commit 81cda79a2431ac78f56b0cc5a5765387f662d801
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:35:02 2016 +0100

    Fix compile errors in BlendedTermQuery.

commit 70994ce8dd1eca0b995870974a38e20f26f96a7b
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 23:33:03 2016 -0500

    add bug ID

commit 29d4f1a71f36f646b5a6060bed3db019564a279d
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 21:02:32 2016 -0500

    easy .store changes

commit 5e1a1e6fd665fa455e88d3a8987362fad5f44bb1
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 20:47:24 2016 -0500

    cleanups mostly around boosting

commit 333a669ec6c305ada5645d13ed1da0e19ec1d053
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 20:27:56 2016 -0500

    more simple fixes

commit bd5cd98a1e089c866b6b4a5e159400b110140ce6
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 19:49:38 2016 -0500

    more easy fixes and removal of ancient cruft

commit a68f419ee47da5f9c9ce5b372f01d707e902474c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 19:35:02 2016 -0500

    cutover numerics

commit 4ca5dc1fa47dd5892db00899032133318fff3116
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 18:34:18 2016 -0500

    fix some constants

commit 88710a17817086e477c6c021ec346d0534b7fb88
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 18:14:25 2016 -0500

    Add spatial-extras jar as a core dependency

commit c8cd6726583e5ce3f546ed355d4eca037164a30d
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 18:03:33 2016 -0500

    update to lucene 6 jars
2016-03-07 04:12:23 -05:00
Alexander Kazakov 3674774b73 Build empty extended stats aggregation if no docs are collected for bucket #16812 2016-03-05 22:50:54 +03:00
javanna da19922489 [TEST] adapt test infra to node.client setting removal
No changes were really needed in our test infra as it didn't use `node.client`. Yet it didn't take into account ingest nodes, what we used to call client nodes in InternalTestCluster were actually ingest only nodes, which now become coordinating only nodes.
Also renamed some method to get rid of the node client terminology as much as possible in favour or coordinating only node.
2016-03-05 10:55:19 +01:00
Nik Everett 4d6cb34417 [reindex] Add ingest support 2016-03-04 10:05:13 -05:00
Christoph Büscher 30a788d87c Suggestions: Make field name mandatory ctor argument
The field name is a required argument for all suggesters, but
it was specified via a field() setter in SuggestionBuilder so far.
This changes field name to being a mandatory constructor argument
and lets suggestion builders throw an error if field name is missing
or the empty string.
2016-03-03 21:35:53 +01:00
Lee Hinman 6adbbff97c Fix organization rename in all files in project
Basically a query-replace of "https://github.com/elasticsearch/" with "https://github.com/elastic/"
2016-03-03 12:04:13 -07:00
Clinton Gormley 529f7cb42c Fixed bad YAML in msearch test 2016-03-03 14:45:02 +01:00
Christoph Büscher ef4db5c1e4 Merge branch 'master' into feature-suggest-refactoring 2016-03-03 12:18:03 +01:00
Adrien Grand 2b545df372 Fix modules/reindex to not use the string field anymore. 2016-03-03 11:11:00 +01:00
Adrien Grand eef19be072 Deprecate string in favor of text/keyword. #16877
This commit removes the ability to use string fields on indices created on or
after 5.0. Dynamic mappings now generate text fields by default for strings
but there are plans to also add a sub keyword field (in a future PR).

Most of the changes in this commit are just about replacing string with
keyword or text. Some tests have been removed because they existed because of
corner cases of string mappings like setting ignore-above on a text field or
enabling term vectors on a keyword field which are now impossible.

The plan is to remove strings entirely in 6.0.
2016-03-03 10:20:56 +01:00
Nik Everett 18e5bb83c5 Disable problematic reindex test
This should get the builds back to normal while we wait on #16914 or
something like it to fix the test properly.
2016-03-02 13:02:29 -05:00
Nik Everett 942eb70956 Revert "Silence reindex's rest tests"
This reverts commit aa0ef84f5a.
2016-03-02 09:17:06 -05:00
Nik Everett aa0ef84f5a Silence reindex's rest tests
They are failing sporadically in CI.
2016-03-02 08:32:30 -05:00
Christoph Büscher be8ed737bc Suggest: Move name of suggestion to SuggestBuilder
Currently each suggestion keeps track of its own name. This has
the disadvantage of having to pass down the parsed name property
in the suggestions fromXContent() and the serialization methods
as an argument, since we need it in the ctor.

This change moves the naming of the suggestions to the surrounding
SuggestBuilder and by this eliminates the need for passind down
the names in the parsing and serialization methods. By making
`name` a required argument in SuggestBuilder#addSuggestion() we
also make sure it is always set and prevent using the same name twice,
which wasn't possible before.
2016-03-01 17:34:47 +01:00
Christoph Büscher aecf51cb42 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/rest/action/search/RestSearchAction.java
2016-03-01 17:20:15 +01:00
Nik Everett c7c8bb357a Merge pull request #16861 from nik9000/reindex_is_ready
Reindex required some parsing changes for search requests to support
differing defaults from the regular search api.
2016-03-01 10:02:48 -05:00
javanna b7597d7aea Rename SearchServiceTransportAction to SearchTransportService
The suffix TransportAction is misleading as it may make think that it extends TransportAction, but it does not. This class makes accessible the different search operations exposed by SearchService through the transport layer. Also resolved few compiler warnings in the class itself.
2016-03-01 11:35:04 +01:00
Nik Everett aeed7ee218 Reindex: rename source to searchRequest
This makes the code easier to read for those familiar with the
Elasticsearch code base.
2016-02-29 14:57:16 -05:00
Nik Everett 16934d50b6 Reindex isn't a plugin
Remove another mention of it being a plugin.
2016-02-29 10:21:10 -05:00
Nik Everett 356b7cd12b Merge reindex to master because it is ready! 2016-02-29 10:08:29 -05:00
Nik Everett d587a74533 Fix reindex to work with master branch
Stuff changed, reindex's got to change.
2016-02-29 10:04:16 -05:00
Christoph Büscher 379bd94c2a Merge branch 'master' into feature-suggest-refactoring 2016-02-29 14:06:30 +01:00
Yannick Welsch ab4e671bfd Remove suppressAccessChecks permission for Groovy script plugin
Closes #16527
2016-02-27 18:58:00 +01:00
Yannick Welsch af918ebf2b Upgrade groovy dependency in lang-groovy module to version 2.4.6
Closes #16830
2016-02-27 17:41:15 +01:00
Nik Everett c38119bae9 Merge branch 'master' into feature/reindex 2016-02-26 16:59:54 -05:00
Christoph Büscher d888b5c267 Merge branch 'master' into feature-suggest-refactoring 2016-02-22 11:22:28 -08:00
Jack Conradson 7986770e5f Moved Painless from a plugin to a module.
Closes #16755
2016-02-21 16:50:54 -08:00
Mike McCandless 5fffede2b0 Upgrade to Lucene 5.5.0 official release 2016-02-20 17:34:16 -05:00
Ali Beyad 5320d538f1 Merge remote-tracking branch 'upstream/master' into feature-suggest-refactoring 2016-02-15 11:03:33 -05:00
Colin Goodheart-Smithe ebc745211d Moved AggregatorBuilder implementations into their own class files
Also renamed histogram.AbstractBuilcer to AbstractHistogramBuilder, range.AbstractBuilder to AbstractRangeBuilder and org.elasticsearch.search.aggregations.pipeline.having to org.elasticsearch.search.aggregations.pipeline.bucketselector
2016-02-15 14:09:36 +00:00
Colin Goodheart-Smithe 1a46628daa Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/builder/SearchSourceBuilderTests.java
2016-02-15 10:37:16 +00:00
Jason Tedor 1fb1ffbac6 Class permission for Groovy references
This commit adds a class permission for groovy.lang.Reference so they
can be used in scripts.

Closes #16660
2016-02-14 09:45:40 -05:00
Ali Beyad beea413820 Merge remote-tracking branch 'upstream/master' into feature-suggest-refactoring 2016-02-12 11:54:44 -05:00
Simon Willnauer 297fb24112 apply fixes after review 2016-02-12 12:53:18 +01:00
Colin Goodheart-Smithe ed3f7903f4 Merge branch 'master' into feature/aggs-refactoring 2016-02-12 09:48:59 +00:00
Simon Willnauer 685bee3081 Merge branch 'master' into add_rewrite_infra 2016-02-12 09:46:01 +01:00
Ali Beyad eed557742f Refactors building query specific objects from the term suggestion
builder and merges changes between the suggestion builders to context
object implementations.
2016-02-11 18:30:58 -05:00
Nik Everett 821a20f582 Merge branch 'master' into feature/reindex 2016-02-11 17:41:05 -05:00
Nik Everett 18808b7576 Move reindex from a plugin to a module 2016-02-11 17:39:49 -05:00
Nicholas Knize 52ee4c7027 upgrade to lucene 5.5.0-snapshot-850c6c2 2016-02-11 14:28:50 -06:00
Adrien Grand bc47c577d2 Add a new `keyword` field.
The `keyword` field is intended to replace `not_analyzed` string fields. It is
indexed and has doc values by default, and doesn't support enabling term
vectors.

Although it doesn't support setting an analyzer for now, there are plans for
it to support basic normalization in the future such as case folding.
2016-02-11 18:19:53 +01:00
Simon Willnauer 8bdc7362a6 Add infrastructure to rewrite query buidlers
QueryBuilders today do all their heavy lifting in toQuery() which
can be too late for several operations. For instance if we want to fetch geo shapes
on the coordinating node we need to do all this before we create the actual lucene query
which happens on the shard itself. Also optimizations for request caching need to be done
to the query builder rather than the query which then in-turn needs to be serialized again.
This commit adds the basic infrastructure for query rewriting and moves the heavy lifting into
the rewrite method for the following queries:

 * `WrapperQueryBuilder`
 * `GeoShapeQueryBuilder`
 * `TermsQueryBuilder`
 * `TemplateQueryBuilder`

Other queries like `MoreLikeThisQueryBuilder` still need to be fixed / converted. The nice
sideeffect of this is that queries like template queries will now also match the request cache
if their non-template equivalent has been cached befoore. In the future this will allow to
add optimizataion like rewriting time-based queries into primitives like `match_all_docs` or `match_no_docs`
based on the currents shards bounds. This is especially appealing for indices that are read-only ie. never change.
2016-02-11 10:45:49 +01:00
Christoph Büscher 9e0f6e3f9c Adding method to build SuggestionContext to PhraseSuggestionBuilder
This adds a build method for the SuggestionContext to the PhraseSuggestionBuilder
and another one that creates the SuggestionSearchContext to the top level
SuggestBuilder. Also adding tests that make sure the current way of parsing
xContent to a SuggestionContext is reflected in the output the builders create.
2016-02-10 18:57:27 +01:00
Colin Goodheart-Smithe 1f760bd1bd Merge branch 'master' into feature/aggs-refactoring 2016-02-10 12:16:26 +00:00
Christoph Büscher 421ed1228b Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/search/suggest/phrase/PhraseSuggestionBuilder.java
2016-02-10 11:37:51 +01:00
Colin Goodheart-Smithe 54e2f31189 Parse Aggregations on Coordinating node 2016-02-10 10:00:16 +00:00
Yannick Welsch 380393a5b7 Add permission to access sun.reflect.MethodAccessorImpl from Groovy scripts
Groovy uses reflection to invoke closures. These reflective calls are optimized by the JVM after "sun.reflect.inflationThreshold" number of invocations.
After inflation, access to sun.reflect.MethodAccessorImpl is required from the security manager.

Closes #16536
2016-02-09 16:47:38 +01:00
Christoph Büscher fab3b5568f Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/search/suggest/phrase/PhraseSuggestionContext.java
2016-02-09 11:35:13 +01:00
Colin Goodheart-Smithe 02ecfd6279 Merge branch 'master' into feature/aggs-refactoring 2016-02-08 15:41:02 +00:00
Colin Goodheart-Smithe a0b004acb5 Split AggregatorFactory into AggregatorBuilder and AggregatorFactory 2016-02-08 14:27:34 +00:00
Isabel Drost-Fromm 0b3cab6da3 Move sorting tests w/o scripting back to core
In preparation of #16127 this moves sorting tests back to core
that don't actually need scripting or groovy to work.
2016-02-08 14:45:33 +01:00
Christoph Büscher 2dec129038 Merge branch 'master' into feature-suggest-refactoring 2016-02-08 09:47:06 +01:00
Simon Willnauer b185209b63 Make IndicesWarmer a private class of IndexService
There is no need for IndicesWarmer to be a global accessible class. All it needs
access to is inside IndexService. It also doesn't need to be mutable once it's not a per node
instance. This commit move IndicesWarmer to IndexWarmer and makes the default impls like field data and
norms warming an impl detail. Also the IndexShard doesn't depend on this class anymore, instead it accepts
an Engine.Warmer as a ctor argument which delegates to the actual warmer from the index.
2016-02-05 15:24:17 +01:00
Colin Goodheart-Smithe 19d9c588e1 more line length fixes 2016-02-05 10:41:06 +00:00
Simon Willnauer 8685a003d9 fix another test 2016-02-05 11:17:53 +01:00
Colin Goodheart-Smithe f06f17f328 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamInput.java
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamOutput.java
2016-02-05 09:22:14 +00:00
Christoph Büscher 225adcfaeb Merge branch 'master' into feature-suggest-refactoring 2016-02-04 23:50:57 +01:00
Martijn van Groningen 7a6adfd93a ingest: Added foreach processor.
This processor is useful when all elements of a json array need to be processed in the same way.
This avoids that a processor needs to be defined for each element in an array.
Also it is very likely that it is unknown how many elements are inside an json array.
2016-02-04 23:44:01 +01:00
Ryan Ernst fe8c9d394d Build: Do not publish modules
Modules are an internal implementation detail of our build, and there is
no need to publish them with gradle. This change disables publishing of
all modules.
2016-02-04 13:03:03 -08:00
Christoph Büscher c0006426a7 Merge branch 'master' into feature-suggest-refactoring 2016-02-04 11:01:24 +01:00
Simon Willnauer cc520b20e1 Merge branch 'master' into fix_settings_filter 2016-02-04 09:28:41 +01:00
Nik Everett a2f07679fd Add task status
Implements a simple task status for superclasses of ReplicationRequest to
show how you can do use the status.
2016-02-03 18:21:42 -05:00
Simon Willnauer e02d2e004e Rewrite SettingsFilter to be immutable
This change rewrites the entire settings filtering mechanism to be immutable.
All filters must be registered up-front in the SettingsModule. Filters that are comma-sparated are
not allowed anymore and check on registration.
This commit also adds settings filtering to the default settings recently added to ensure we don't render
filtered settings.
2016-02-03 20:05:55 +01:00
Ali Beyad 96d7fc858f Fixed a compiler error with the SuggestSearchTests 2016-02-03 12:42:03 -05:00
Ali Beyad 882cd5c6ee Refactored the term suggestion builder for the query refactoring effort
Added the term suggestion builder's serialization/deserialization and
equals/hashCode methods.
2016-02-03 12:00:03 -05:00
Colin Goodheart-Smithe 5d9d91b761 Merge branch 'master' into feature/aggs-refactoring 2016-02-03 14:45:16 +00:00
Isabel Drost-Fromm 7d787ad1d1 Merge pull request #16411 from MaineC/test-fix/geo-sort-messy
Move non-groovy geo tests from messy tests to core
2016-02-03 14:52:01 +01:00
Isabel Drost-Fromm f99570b0f4 Split messy geo distance/ geo distance sorting tests.
This splits the geo distance and geo distance sorting tests marked messy:
Test cases that don't really need Groovy support are moved back to the
core test suite closer to the code they actually test.

Relates to #15178
2016-02-03 13:54:07 +01:00
Simon Willnauer a77344d742 Merge branch 'master' into make_settings_strict 2016-02-03 13:21:49 +01:00
Simon Willnauer 4a4e523357 Merge branch 'master' into make_settings_strict 2016-02-03 11:34:12 +01:00
Christoph Büscher b7e3323ded Merge branch 'master' into feature-suggest-refactoring 2016-02-03 09:51:23 +01:00
Robert Muir d5dc05f69e Upgrade to lucene 5.5.0-snapshot-1725675 2016-02-02 22:53:39 -05:00
Tal Levy 3191fc7347 Merge pull request #16355 from talevy/fix_ingest_exception
revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation
2016-02-02 14:11:24 -08:00
Tal Levy 0a1580eefa revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation 2016-02-02 14:08:22 -08:00
Martijn van Groningen 64ac037a82 Added an ingest qa that tests processor real world like configurations.
Renamed `ingest-with-mustache` to `smoke-test-ingest-with-all-dependencies`
Also renamed `ingest-disabled` to `smoke-test-ingest-disabled` so that the name is more inline with other qa smoke test modules.
2016-02-02 22:37:24 +01:00
Yannick Welsch 8cca0395ef Revert "Make GeoDistanceSortBuilder serializable"
This reverts commit c108a4ce6d.
2016-02-02 14:04:45 +01:00
Yannick Welsch 7ea2e99f91 Revert "Indentation fix for messy SimpleSortTest"
This reverts commit 26f77eb70d.
2016-02-02 14:03:40 +01:00
Isabel Drost-Fromm 26f77eb70d Indentation fix for messy SimpleSortTest 2016-02-02 13:46:25 +01:00
Simon Willnauer 818a9eefb2 Make settings validation strict
This commit enableds strict settings validation on node startup. All settings
passed to elasticsearch either through system properties, yaml files or any other
way to pass settings must be registered and valid. Settings that are unknown ie. due to
typos or due to deprecation or removal will cause the node to NOT start up. Plugins
have to declare all their settings on the `SettingsModule#registerSetting` and settings for
plugins that are not installed must be removed.

This commit also removes the ability to specify the nodes name via `-Des.name` or just `name` in the
configuration files. The node name must be prefixed with the node prexif like `node.name: Boom`. Left over
usage of `name` will also cause startup to fail.
2016-02-02 11:32:44 +01:00
Isabel Drost-Fromm c108a4ce6d Make GeoDistanceSortBuilder serializable
Adds to GeoDistanceSortBuilder:

* equals
* hashcode
* writeto/readfrom
* moves xcontent parsing logic over
* adds roundtrip tests
* fixes roundtrip test for xcontent by keeping points just as geopoints not geohashes internally
* fixes xcontent parsing of ignore_malformed if coerce is set/unset
* adds exception to sortMode setter to avoid setting invalid sort modes

Relates to #15178
2016-02-02 11:24:32 +01:00
Colin Goodheart-Smithe a241983c9d Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/aggregations/AggregationsBinaryIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/FilterIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/FiltersIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/GeoDistanceIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/NestedIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/SamplerIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/pipeline/DerivativeIT.java
#	modules/lang-groovy/src/test/java/org/elasticsearch/messy/tests/BucketSelectorTests.java
#	modules/lang-groovy/src/test/java/org/elasticsearch/messy/tests/TDigestPercentileRanksTests.java
2016-02-01 13:38:33 +00:00
Christoph Büscher 7cae28f96a Merge branch 'master' into feature-suggest-refactoring 2016-02-01 13:38:34 +01:00
Christoph Büscher 61c435e6a9 PhraseSuggestionBuilde: Refactor DirectCandidateGenerator
As a prerequisite for refactoring the whole PhraseSuggestionBuilder
to be able to be parsed and streamed from the coordinating node, the
DirectCandidateGenerator must implement Writeable, be able to parse
a new instance (fromXContent()) and later when transported to the
shard to generate a PhraseSuggestionContext.DirectCandidateGenerator.
Also adding equals/hashCode and tests and moving DirectCandidateGenerator
to its own DirectCandidateGeneratorBuilder class.
2016-02-01 12:22:48 +01:00
Jason Tedor 105411060c Uppercase ells ('L') in long literals
This commit removes and forbids the use of lowercase ells ('l') in long
literals because they are often hard to distinguish from the digit
representing one ('1').

Closes #16329
2016-01-30 22:16:02 -05:00
Tal Levy fca442f4d1 Introduce Pipeline Factory Error Responses in Node Ingest
When there is an exception thrown during pipeline creation within
Rest calls (in put pipeline, and simulate) We now return a structured
error response to the user with details around which processor's
configuration is the cause of the issue, or which configuration property
is misconfigured, etc.
2016-01-29 13:37:27 -08:00
Colin Goodheart-Smithe 859f9e69b7 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/action/search/SearchRequestBuilder.java
#	core/src/main/java/org/elasticsearch/percolator/PercolateContext.java
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/ScriptHeuristic.java
#	core/src/main/java/org/elasticsearch/search/internal/DefaultSearchContext.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificanceHeuristicTests.java
2016-01-29 11:58:19 +00:00
Christoph Büscher 75e93c2eca Merge branch 'master' into feature-suggest-refactoring 2016-01-29 10:52:54 +01:00
Igor Motov e035dabd4d Extend tracking of parent tasks to master node, replication and broadcast actions
Now MasterNodeOperations, ReplicationAllShards, ReplicationSingleShard,  BroadcastReplication and BroadcastByNode actions keep track of their parent tasks.
2016-01-28 12:13:36 -05:00
Christoph Büscher 09c8a21e83 Merge branch 'master' into feature-suggest-refactoring 2016-01-28 10:46:40 +01:00
Boaz Leskes 2a137b5548 Make index uuid available in Index, ShardRouting & ShardId
In the early days Elasticsearch used to use the index name as the index identity. Around 1.0.0 we introduced a unique index uuid which is stored in the index setting. Since then we used that uuid in a few places but it is by far not the main identifier when working with indices, partially because it's not always readily available in all places.

This PR start to make a move in the direction of using uuids instead of name by making sure that the uuid is available on the Index class (currently just a wrapper around the name) and as such also available via ShardRouting and ShardId.

Note that this is by no means an attempt to do the right thing with the uuid in all places. In almost all places it falls back to the name based comparison that was done before. It is meant as a first step towards slowly improving the situation.

Closes #16217
2016-01-28 08:40:10 +01:00
Christoph Büscher c00c0fa020 Initial refactoring for phrase suggester
Adding initial serialization methods (readFrom, writeTo) to the
PhraseSuggestionBuilder, also adding the base test framework for
serialiazation testing, equals and hashCode. Moving SuggestionBuilder
out of the global SuggestBuilder for better readability.
2016-01-27 17:16:37 +01:00
Simon Willnauer 7ff99eb89d Merge branch 'master' into trash_context_and_headers
if (name == 'expamle-fixtures') return
2016-01-27 16:33:50 +01:00
Colin Goodheart-Smithe 187009c12c Merge branch 'master' into feature/aggs-refactoring 2016-01-27 14:54:12 +00:00
Jason Tedor 284cc3a048 Script mode settings as booleans
This commit modifies the accept values for script mode settings from
"on", "off", and "sandbox" to "true", "false", and "sandbox".
2016-01-27 06:26:58 -05:00
Jason Tedor 9944573449 Rename methods on ScriptEngineService
This commit method renames the ScriptEngineService interface methods
types, extensions, and sandboxed to getTypes, getExtensions, and
isSandboxed, respectively.
2016-01-27 06:26:04 -05:00
Jason Tedor 087e55cc51 Script mode settings
This commit converts the script mode settings to the new settings
infrastructure. This is a major refactoring of the handling of script
mode settings. This refactoring is necessary because these settings are
determined at runtime based on the registered script engines and the
registered script contexts.
2016-01-27 06:26:04 -05:00
Simon Willnauer 71c3e57aee Merge branch 'master' into trash_context_and_headers 2016-01-27 11:42:42 +01:00
Adrien Grand 2aaa5e6448 Remove the ability to enable doc values with the `fielddata.format` setting.
Doc values can now only be enabled by setting `doc_values: true` in the
mappings. Removing this feature also means that we can now fail mapping updates
that try to disable doc values.
2016-01-27 09:06:00 +01:00
Adrien Grand 35709f62b6 Be stricter about parsing boolean values in mappings.
Parsing is currently very lenient, which has the bad side-effect that if you
have a typo and pass eg. `store: fasle` this will actually be interpreted as
`store: true`. Since mappings can't be changed after the fact, it is quite bad
if it happens on an index that already contains data.

Note that this does not cover all settings that accept a boolean, but since the
PR was quite hard to build and already covers some main settirgs like `store`
or `doc_values` this would already be a good incremental improvement.
2016-01-27 09:06:00 +01:00
Boaz Leskes 82ae74071e logging: cleanup some old TestLogging annotations and add a few debug level by default 2016-01-26 20:48:07 +01:00
Jason Tedor bdddea2dd0 Security permissions for Groovy closures
This commit adds some permissions that Groovy needs to use closures.

Closes #16196
2016-01-26 12:33:07 -05:00
Colin Goodheart-Smithe 11bafa18e1 Removes Aggregation Builders in place of AggregatorFactory implementations 2016-01-26 15:13:43 +00:00
Colin Goodheart-Smithe 3b35754f59 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/percolator/PercolateDocumentParserTests.java
2016-01-26 13:17:53 +00:00
javanna 9bb4caa6f1 adapt to upstream changes, jvm flag doesn't get returned anymore 2016-01-25 18:01:19 +01:00
javanna 36d98478bf Merge branch 'master' into feature/ingest 2016-01-25 18:01:09 +01:00
Colin Goodheart-Smithe d63b8a25f8 Merge branch 'master' into feature/aggs-refactoring 2016-01-25 10:42:59 +00:00
Colin Goodheart-Smithe cd8320b171 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/filter/FilterAggregator.java
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/filters/FiltersAggregator.java
#	core/src/main/java/org/elasticsearch/search/SearchModule.java
2016-01-25 10:42:20 +00:00
Alexander Reelsen 3a0839de9a Scripting: Allow to get size of array in mustache
This adds support for returning the size of an array
or an collection, in addition to access fields via
`array.0` you can specify `array.size` to get its size.
2016-01-25 07:58:10 +01:00
markharwood d371ef35f4 Settings - change over to o.e.common.settings.Setting for http settings:
http.cors.allow-credentials
http.cors.enabled
http.detailed_errors.enabled
http.enabled
http.pipelining
http.cors.max-age
2016-01-22 15:13:11 +00:00
Daniel Mitterdorfer e9bb3d31a3 Convert "path.*" and "pidfile" to new settings infra 2016-01-22 15:14:13 +01:00
Ryan Ernst df24019261 Merge pull request #16038 from rjernst/remove_site_plugin
Plugins: Remove site plugins
2016-01-21 12:32:22 -08:00
Tal Levy 3a6c2d008e rename processor_tag to tag 2016-01-21 09:05:42 -08:00
Martijn van Groningen 20f634952c mustache: Improve the mustache script engine
* Added a `content_type` option at compile time to decide how variable values are encoded. Possible values are `application/json` and `plain/text`. Defaults to `application/json`.
* Added support for variable placeholders to lookup values from specific slots in arrays/lists.
2016-01-21 18:04:06 +01:00
Tal Levy c55f9bbd9b Merge pull request #16132 from talevy/holdmytag
[Ingest] add an AbstractProcessor to help hold the re-used processorTag variable
2016-01-21 07:30:29 -08:00
Simon Willnauer 142547271e Merge branch 'master' into trash_context_and_headers 2016-01-21 14:42:15 +01:00
Martijn van Groningen 44465c94f8 Merge remote-tracking branch 'es/master' into feature/ingest 2016-01-21 11:46:27 +01:00
Tal Levy f1204cb1ab add an AbstractProcessor to help hold the re-used processorTag variable 2016-01-20 14:34:26 -08:00
Robert Muir 6e7e3a2274 Update lucene to r1725675
Adds DFI (divergence from independence) provider.
Fixes test bugs passing invalid values for BM25 parameters.
2016-01-20 03:32:51 -05:00
Martijn van Groningen e2e207687d Fixes due to changes in master branch. 2016-01-19 22:03:26 +01:00
Martijn van Groningen 602a0f183e Merge remote-tracking branch 'es/master' into feature/ingest 2016-01-19 22:01:38 +01:00
Simon Willnauer 40783b5d32 delete dead code 2016-01-19 21:06:23 +01:00
Simon Willnauer 3d0cedbabb Merge branch 'master' into trash_context_and_headers 2016-01-19 20:47:43 +01:00
Nik Everett e1e73d9914 Create default for ExecutableScript#unwrap
Tons of scripts just return the variable they are passed and that is the
most intuitive behavior so that may as well be the default implementation.
2016-01-19 13:17:11 -05:00
Simon Willnauer fbfa9f4925 Merge branch 'master' into new_index_settings 2016-01-19 10:13:48 +01:00
Nik Everett 53cce3c399 Merge pull request #16071 from nik9000/xlint_lang_groovy
Remove Xlint from lang-groovy and discovery-azure
2016-01-18 21:06:11 -05:00
Ryan Ernst ef4f0a8699 Test: Make rest test framework accept http directly for the test cluster
The rest test framework, because it used to be tightly integrated with
ESIntegTestCase, currently expects the addresses for the test cluster to
be passed using the transport protocol port. However, it only uses this
to then find the http address.

This change makes ESRestTestCase extend from ESTestCase instead of
ESIntegTestCase, and changes the sysprop used to tests.rest.cluster,
which now takes the http address.

closes #15459
2016-01-18 16:44:14 -08:00
Nik Everett bd1324ccc9 Remove Xlint from lang-groovy and discovery-azure
discovery-azure didn't actually need it.

This removes all non-default Xlints from modules.
2016-01-18 18:24:44 -05:00
Nik Everett 63090f5d4b Merge pull request #15761 from nik9000/more_remove_inject
Don't use guice for QueryParsers
2016-01-18 17:39:53 -05:00
Nik Everett 093600cd63 Cleanup ContextAndHeaderTransportTests
We have two similar tests with the same name, ContextAndHeaderTransportTests.
They shared lots of common code so I extracted much of it into
ActionRecordingPlugin, a plugin which records all action requests for later
inspection.

I also removed all the warnings from both tests. That made lang-mustache
compile cleanly without any custom -Xlint so I removed those. To remove
the warnings I had to add type parameters to ActionFilter which seemed
like a good idea anyway.
2016-01-18 16:14:58 -05:00
Nik Everett 15c43b64a1 Don't use guice for QueryParsers
1. Gets guice out of the business of building ScoreFunctionParsers and
QueryParsers.
2. Moves QueryParser registration to SearchModule
3. Moves NamedWriteableRegistry construction out of guice and into Node and
TransportClient.
4. Moves shape registration into SearchModule so now all named writeable
registration is done in the SearchModule.

This is breaking for plugin authors. Instead of declaring new QueryParser
like:

```java
public void onModule(IndicesModule module) {
  module.registerQueryParser(NewQueryParser.class);
}
```

you do it like:

```java
public void onModule(SearchModule module) {
  module.registerQueryParser(NewQueryParser::new);
}
```

The QueryParser's argument no longer come from @Inject, now they come from
the declaration in the plugin. The above example is for a no-arg QueryParser.
Most of the QueryParsers in Elasticsearch are no-arg.

ScoreFunctionParsers have a similar but slightly different change. This:

```java
public void onModule(SearchModule module) {
  module.registerFunctionScoreParser(NewFunctionScoreParser.class);
}
```

becomes

```java
public void onModule(SearchModule module) {
  module.registerFunctionScoreParser(new NewFunctionScoreParser());
}
```

Since all known ScoreFunctionParsers have no arg constructors its simpler to
just build them at registration time rather than specify a supplier that is
used to build them later.
2016-01-18 15:02:47 -05:00
Simon Willnauer c2e8ba8970 Register index.version_created on unittest that set it 2016-01-18 15:04:35 +01:00
Simon Willnauer 7925e2ef84 convert IndexModule settings 2016-01-18 09:23:35 +01:00
Simon Willnauer a8eedd0457 fix all kinds of crazy test failures 2016-01-18 09:23:34 +01:00
Simon Willnauer 2eecb9c02d add tests for AutoExpandReplicas 2016-01-18 09:23:33 +01:00
Nik Everett 742e1da857 Squash more raw type errors
Its a long string to pull
2016-01-17 17:07:32 -05:00
Ryan Ernst 3b78267c71 Plugins: Remove site plugins
Site plugins used to be used for things like kibana and marvel, but
there is no longer a need since kibana (and marvel as a kibana plugin)
uses node.js. This change removes site plugins, as well as the flag for
jvm plugins. Now all plugins are jvm plugins.
2016-01-16 22:45:37 -08:00
Tal Levy 9f48df9736 Add on_failure support for verbose _simulate execution and introduce optional processor_tag to Processors 2016-01-15 14:56:20 -08:00
javanna 050585e89f remove BiFunction<Environment, TemplateService, Processor.Factory> in favour of Function<TemplateService, Processor.Factory>
the environment is now available through NodeModule#getNode#getEnvironment and can be retrieved during onModule(NodeModule), no need for this indirection anymore using the BiFunction
2016-01-15 15:12:53 +01:00
Martijn van Groningen 21cc0b2316 Cleanup ingest initialization code.
* Folded IngestModule into NodeModule
* Renamed IngestBootstrapper to IngestService
* Let NodeService construct IngestService and removed the Guice annotations
* Let IngestService implement Closable
2016-01-15 13:35:06 +01:00
javanna 169b3c75c5 move grok patterns loading to IngestGrokPlugin class out of the GrokProcessor.Factory 2016-01-14 16:24:07 +01:00
Martijn van Groningen f3883343cb Move the pipeline configuration from the dedicated index to the cluster state.
Closes #15842
2016-01-13 22:59:36 +01:00
Simon Willnauer ce42ae4cf3 Merge pull request #15922 from s1monw/feature/ingest
Review feedback and several cleanups
2016-01-13 12:04:04 +01:00
Simon Willnauer 574d1b35b3 Replace ContextAndHeaders with a ThreadPool based ThreadLocal implementation
ContextAndHeaders has a massive impact on the core infrastructure since it has to
be manually passed on to all relevant places across threads/network calls etc. For the same reason
it's also very error prone and easily forgotten on potentially relevant APIs.

The new ThreadContext is associated with a ThreadPool (node or transport client) and ensures that
headers and context registered on a current thread are inherited to new threads spawned, send across
the network to be deserialized on the receiver end as well as restored on the response handling thread
once the response is received.
2016-01-13 11:53:32 +01:00
Tal Levy 4bf12a7dca Merge pull request #15910 from talevy/ingest-fix-grok-nocaptures
[Ingest] fix NPE when Grok matches expression, but no captures defined.
2016-01-12 12:02:55 -08:00
Tal Levy b39a64e49f accept grok matches with no captures instead of NPE 2016-01-12 12:01:55 -08:00
Colin Goodheart-Smithe f429cb08d7 Merge branch 'master' into feature/aggs-refactoring 2016-01-12 12:59:52 +00:00