Commit Graph

17485 Commits

Author SHA1 Message Date
Ryan Ernst 8d78f970b0 Build: Enforce name is set for all plugins
The plugin name currently defaults to the gradle project name. But the
gradle project name for standalone repo (like an external plugin would
be) defaults to the directory name of the repo. This is trappy, since it
depends on how the repo was checked out.

This change enforces the plugin name is always set.

closes #14603
2015-11-07 16:57:11 -08:00
Areek Zillur 6444194dae fix conflicts from merge 2015-11-07 18:22:28 -05:00
Robert Muir f2ee1d79fa Merge pull request #185 from rmuir/fix_security_exceptions
Fix security problems in mapper-attachments.
2015-11-07 18:13:36 -05:00
Robert Muir 6b831e1e8b be explicit about name and link to bugzilla for the POI issue 2015-11-07 18:13:05 -05:00
Areek Zillur d5198b326c fix comments and throw ElasticsearchException when we fail to report fst index size 2015-11-07 17:50:56 -05:00
Areek Zillur d37e011526 add query context builders 2015-11-07 17:50:56 -05:00
Areek Zillur 40022e2168 remove nocommit 2015-11-07 17:50:55 -05:00
Areek Zillur e87b4e00eb add back awaitfix tests 2015-11-07 17:50:55 -05:00
Simon Willnauer 6f9a486071 more parsers 2015-11-07 17:50:55 -05:00
Robert Muir b0fcbb7081 remove unnecessary unboxing, we do that as impl detail in parseTika 2015-11-07 17:48:45 -05:00
Simon Willnauer f5560a3087 add parsers 2015-11-07 17:47:46 -05:00
Simon Willnauer 8911469e63 some basic cleanups 2015-11-07 17:46:28 -05:00
Areek Zillur 75d55e11ad cut over to using ObjectParser for context mappings 2015-11-07 17:46:27 -05:00
Areek Zillur 96c9849f5a added FuzzyOptionsBuilder javadocs and cleanups 2015-11-07 17:46:27 -05:00
Areek Zillur dd1c687ace Completion Suggester V2
The completion suggester provides auto-complete/search-as-you-type functionality.
This is a navigational feature to guide users to relevant results as they are typing, improving search precision.
It is not meant for spell correction or did-you-mean functionality like the term or phrase suggesters.

The completions are indexed as a weighted FST (finite state transducer) to provide fast Top N prefix-based
searches suitable for serving relevant results as a user types.

closes #10746
2015-11-07 17:46:27 -05:00
Robert Muir 3c40ed22b3 Fix security problems in mapper-attachments.
Many of the tests were not running, or did not check the exceptions.
I renamed all tests to meet *Tests* so they run, and assert exception messages.

Also because we must (currently) invoke tika with additional privileges, I added
the security logic, and fixed unit testing to call our static method directly.

This must be package private for security reasons, i simply put everything in
org.elasticsearch.mapper.attachments package.

I upgraded tika to the latest, so we are up to date, and removed logic around
tika == null and old locale issues.
2015-11-07 17:45:16 -05:00
Ryan Ernst cdab360992 Merge pull request #184 from rjernst/update_to_gradle
Build: Switch mapper attachments to gradle
2015-11-07 13:11:05 -08:00
Ryan Ernst d0f5950c80 Merge pull request #14598 from rjernst/integ_test_parity
Improve integ test to match ant behavior
2015-11-07 13:09:04 -08:00
Ryan Ernst 22bcf197b7 Add some comments to particularly nasty groovy code 2015-11-07 13:06:14 -08:00
Robert Muir 5164ec8c20 Merge pull request #14602 from rmuir/buildSrc
add groovy build code (buildSrc/) in 'gradle eclipse'
2015-11-07 15:41:05 -05:00
Robert Muir 866a514785 add groovy build code (buildSrc/) in 'gradle eclipse'
This makes it a groovy project that works in eclipse.
You will have to install a plugin for groovy language support
(I used a snapshot build from https://github.com/groovy/groovy-eclipse/wiki)
2015-11-07 15:14:00 -05:00
Robert Muir e06cae84f3 only allow code to bind to the user's configured port numbers/ranges
Random code shouldn't be listening on sockets elsewhere.

Today its the wild west, but we only need to grant access to what the user configured.

This means e.g. multicast plugin has to declare its intentions in its security.policy

Closes #14549
2015-11-07 12:22:12 -05:00
Ryan Ernst 835994a582 Fix groupid and remove more legacy files 2015-11-07 01:11:23 -08:00
Ryan Ernst 9ee315a9c8 Build: Improve integ test to match ant behavior
Many other improvements:
* Use spaces in ES path
* Use space in path for plugin file installation
* Use a different cwd than ES home
* Use jps to ensure process being stopped is actually elasticsearch
* Stop ES if pid file already exists
* Delete pid file when successfully killed

Also, refactored the cluster formation code to be a little more organized.

closes #14464
2015-11-07 00:45:19 -08:00
William Bowling 202a735d8f Put method addField on TopHitsBuilder 2015-11-07 17:15:02 +11:00
Robert Muir f03196193f Fix Build to correctly treat URLs and not leak a file handle.
Closes #14595

Squashed commit of the following:

commit d0b2b262e9dcdbc2aee163b9a84db082c8b5b96b
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Nov 6 22:36:54 2015 -0500

    Switch to JarInputStream, to contain suppressforbidden. Also add a test that fails if path is not accessible (regardless of whether its a jar)

commit f99c1d240db23ceb2a06987b3bd69eae0229550b
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Nov 6 22:16:16 2015 -0500

    remove leniency in i/o here

commit b160d4303ee81a8c9298729596ecbc893f5f8894
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Nov 6 21:58:21 2015 -0500

    Fix Build to correctly treat URLs and to not leak a file handle
2015-11-06 22:40:58 -05:00
Ryan Ernst afe6fe8fa2 Remove outdated dep exclusion 2015-11-06 14:46:19 -08:00
Ryan Ernst 867f496ac8 Build: Switch mapper attachments to gradle
This change switches the plugin to use gradle as the build system, and
updates the master branch to track elasticsearch master.
2015-11-06 14:07:57 -08:00
Ryan Ernst 59a1cda9bb Merge pull request #14592 from rjernst/eclipse_cycles
Build: Fix eclipse generation to add a core-tests projects
2015-11-06 11:55:47 -08:00
Ryan Ernst ee691e547a Remove outdated projectsPrefix setup 2015-11-06 11:45:32 -08:00
Ryan Ernst 548430072f Build: Fix eclipse generation to add a core-tests projects
Eclipse does not have the ability to differentiate test dependencies
from main dependencies. This causes what looks like a circular
dependency through test-framework. This change sets up an additional
core-tests project for eclipse only, which removes this problem.
2015-11-06 11:41:18 -08:00
Simon Willnauer 01ca95a648 Only allow rebalance operations to run if all shard store data is available
This commit prevents running rebalance operations if the store allocator is
still fetching async shard / store data to prevent pre-mature rebalance decisions
which need to be reverted once shard store data is available. This is typically happening
on rolling restarts which can make those restarts extremely painful.

Closes #14387
2015-11-06 20:18:01 +01:00
Nicholas Knize afaf96732a Add GeoPointV2 Field Mapping
This commit adds the abstraction layer to GeoPointFieldMapper needed to cut over to Lucene 5.4's new GeoPointField type while maintaining backward compatibility with 'legacy' geo_point indexes.
2015-11-06 13:14:04 -06:00
Ryan Ernst 3cabff9880 Merge pull request #14590 from rjernst/use_projects_prefix
Use projectsPrefix in project references for vagrant tests
2015-11-06 11:08:29 -08:00
Ryan Ernst 634c495e3d Build: Use projectsPrefix in project references for vagrant tests
It is important we use projectsPrefix whenever addinga direct project
dependency, so that attachment still works.
2015-11-06 10:52:19 -08:00
javanna 1dfe6f6dcf make sure headers etc. are passed over to inner index and delete requests in put/delete pipeline 2015-11-06 19:21:30 +01:00
Jason Tedor 97644e3046 Merge pull request #14588 from jasontedor/enforce-java-8
Enforce Java 8 in Gradle builds
2015-11-06 12:33:28 -05:00
Jason Tedor b0c7065000 Cleaner version check 2015-11-06 12:28:31 -05:00
Jason Tedor 7d7926e10b Merge pull request #14586 from jasontedor/transport-broadcast-by-node-action-master-failure
Handle shards assigned to nodes that are not in the cluster state
2015-11-06 12:10:36 -05:00
Jason Tedor 699c701b59 Handle shards assigned to nodes that are not in the cluster state
This commit addresses an issue in TransportBroadcastByNodeAction.
Namely, if in between a master leaving the cluster and a new one being
assigned, a request that relies on TransportBroadcastByNodeAction
(e.g., an indices stats request) is issued,
TransportBroadcastByNodeAction might attempt to send a request to a
node that is no longer in the local node’s cluster state.

The exact circumstances that lead to this are as follows. When the
master leaves the cluster and another node’s master fault detection
detects this, the local node will update its local cluster state to no
longer include the master node. However, the routing table will not be
updated. This means that in the preparation for sending the requests in
TransportBroadcastByNodeAction, we need to check that not only is a
shard assigned, but also that it is assigned to a node that is still in
the local node’s cluster state.

This commit adds such a check to the constructor of
TransportBroadcastByNodeAction. A new unit test is added that checks
that no request is sent to the master node in such a situation; this
test fails with a NullPointerException without the fix. Additionally,
the unit test TransportBroadcastByNodeActionTests#testResultAggregation
is updated to also simulate a master failure. This updated test also
fails prior to the fix.

Closes #14584
2015-11-06 12:09:30 -05:00
javanna a84ce07a90 added docs 2015-11-06 17:48:44 +01:00
Jason Tedor faf5f55f41 Enforce Java 8 in Gradle builds
This commit forces builds with Gradle to require at least Java 8.

Closes #14499
2015-11-06 11:39:14 -05:00
Ryan Ernst c71cc0f21d Revert "Build: Fail integ test if elasticsearch fails to start on windows"
This reverts commit f974d5e470.

Apprently spawn does not support failonerror in ant.
2015-11-06 08:22:17 -08:00
Yannick Welsch c6b41bc704 Introduce inner class AsyncSingleAction to store state used across retry invocations 2015-11-06 16:38:33 +01:00
Yannick Welsch af3b126a15 [TEST] InternalCluster.restartRandomDataNode should restart a *data* node 2015-11-06 16:24:27 +01:00
javanna d318990339 added REST test 2015-11-06 16:10:22 +01:00
javanna 682876f7d7 added factory tests 2015-11-06 15:44:20 +01:00
javanna dbf5c96876 Merge branch 'feature/ingest' into ingest/date 2015-11-06 13:04:51 +01:00
javanna 588de6ccff Add generic type to Processor.Factory and rename Geo*Tests to GeoIp*Tests 2015-11-06 12:53:20 +01:00
javanna 1d2e244bac Merge branch 'feature/ingest' into ingest/date 2015-11-06 12:08:14 +01:00