Commit Graph

17939 Commits

Author SHA1 Message Date
Ryan Ernst 866bed0864 Merge pull request #14926 from rjernst/maven_cleanup
Remove old files replaced by gradle
2015-11-24 00:54:22 -08:00
Ryan Ernst af496f5e7f Merge pull request #14925 from rjernst/maven_test_parity
Add back leaveTemporary and ifNoTests
2015-11-24 00:53:25 -08:00
Ryan Ernst 31b7e91e5a Build: Allow customizing wait condition and cluster settings
The current wait condition for an integ test cluster being up is a
simple http get on the root path for elasticsearch. However, it is
useful to allow having arbitrary wait conditions. This change reworks
the wait task to first check that each node process started successfully
and has a socket up, followed by an arbitrary wait condition which
defaults to the current http get.

Also, cluster settings are allowed to be added, and overriden. Finally,
custom setup commands are made relative to the elasticsearch home dir
for each node.
2015-11-24 00:48:58 -08:00
Adrien Grand 999913443d Merge pull request #14896 from jpountz/fix/mapper_plugins
Register field mappers at the node level.
2015-11-24 09:28:06 +01:00
Adrien Grand 5f33fbdb75 Register field mappers at the node level.
This moves the registration of field mappers from the index level to the node
level and also ensures that mappers coming from plugins are treated no
differently from core mappers.
2015-11-24 08:59:37 +01:00
Jason Tedor 7c104fdb39 Merge pull request #14874 from jasontedor/ipv4-parsing
Do not be lenient when parsing CIDRs
2015-11-23 20:51:14 -05:00
Jason Tedor b31f2850c0 Do not be lenient when parsing CIDRs
This commit fixes some leniency in the parsing of CIDRs. The leniency
that existed before includes not validating the octet values nor
validating the network mask; in some cases issues with these values were
silently ignored. Parsing is now done according to the guidelines in RFC
4632, page 6.

Closes #14862
2015-11-23 20:46:23 -05:00
Ryan Ernst 321606d3aa Merge pull request #14952 from rjernst/attach_without_subdir
Rework extra plugins support to be through sibling directories
2015-11-23 15:51:32 -08:00
Robert Muir 54de9252d0 die extra plugins 2015-11-23 18:47:56 -05:00
Ryan Ernst 21b42e2a42 Fix xplugins to not omit the first level project name 2015-11-23 15:15:40 -08:00
Ryan Ernst 529e442e2a Merge branch 'master' into attach_without_subdir 2015-11-23 15:01:55 -08:00
Ryan Ernst d97ed9edc3 Fix testing instructions with x-plugins 2015-11-23 14:54:08 -08:00
Ryan Ernst 113a026e1f Change extra plugins to x-plugins directory instead of extra-plugin- prefix 2015-11-23 14:44:37 -08:00
Michael McCandless 2f315cb859 Merge pull request #14951 from mikemccand/upgrade_lucene_1715952
Upgrade Lucene to 5.4.0-snapshot-1715952
2015-11-23 17:25:48 -05:00
Ryan Ernst 5a62282150 Build: Rework extra plugins support to be through sibling directories
This change removes the subdirectory support for extra-plugins, and
replaces it with an iteration of sibling directories with the prefix
"extra-plugin-".
2015-11-23 14:15:45 -08:00
Michael McCandless e13b0d4bde upgrade lucene to 5.4.0-snapshot-1715952 2015-11-23 17:13:49 -05:00
Jason Tedor dcfdc0dbf3 Remove leftover TODOs in o.e.c.r.RoutingTable 2015-11-23 12:57:06 -05:00
Yannick Welsch 3dcd35d1c8 Fix issue where shard state metadata is written while containing no allocation id
Such a write can happen when upgrading shard state metadata using the MultiDataPathUpgrader

Relates to #14831
2015-11-23 18:14:02 +01:00
Daniel Mitterdorfer b804a0c4a6 Improve stability of UpdateIT
With this commit, we reduce the amount of work
that UpdateIT does and add progress logging.

Closes #14877
2015-11-23 17:54:57 +01:00
Yannick Welsch 7d32c88d7b Merge pull request #14831 from ywelsch/feature/persist-allocid-shardstate
Persist allocation ID with shard state metadata on nodes
2015-11-23 17:44:36 +01:00
Yannick Welsch fccad13029 Persist allocation ID with shard state metadata on nodes
Closes #14831
2015-11-23 17:35:33 +01:00
Isabel Drost-Fromm 37d7629146 Merge pull request #14249 from MaineC/enhancement/8964
Switch query parsers to use ParseField
2015-11-23 14:16:40 +01:00
Boaz Leskes c1264b926f Improve trace logging in TransportReplicationAction and error reporting at RecoveryWhileUnderLoadIT
Things that helped me traced down an issue.

Closes #14931
2015-11-23 13:52:32 +01:00
Boaz Leskes 4a6f3c7840 Make sure the remaining delay of unassigned shard is updated with every reroute
For example: if a node left the cluster and an async store fetch was triggered. In that time no shard is marked as delayed (and strictly speaking it's not yet delayed). This caused test for shard delays post node left to fail. see : http://build-us-00.elastic.co/job/es_core_master_windows-2012-r2/2074/testReport/

 To fix this, the delay update is now done by the Allocation Service, based of a fixed time stamp that is determined at the beginning of the reroute.

 Also, this commit fixes a bug where unassigned info instances were reused across shard routings, causing calculated delays to be leaked.

Closes #14890
2015-11-23 13:16:15 +01:00
Boaz Leskes 316f07743a feedback 2015-11-23 13:15:22 +01:00
David Pilato 28109a18a2 Fix example for s3 repository bucket name
Closes #13588.
2015-11-23 13:14:02 +01:00
David Pilato 5b0e2823b1 Merge branch 'docs/mapper-attachments' 2015-11-23 12:14:31 +01:00
Isabel Drost-Fromm 38e1ad5596 Switches from manual field parsing to ParseField
This switches query parsing from manual field parsing to using ParseField.

Also adds unit tests for each query that check original json can be parsed
into query builders.

Relates to #8964
2015-11-23 11:37:46 +01:00
Christoph Büscher e25f7042b9 Making CircleBuilder writable and adding equals/hashcode 2015-11-23 11:35:44 +01:00
Christoph Büscher 5ca9f3ff8f Geo: Make ShapeBuilders implement Writable
We recently refactored the queries to make them parsable on the
coordinating note and adding serialization and equals/hashCode
capability to them. So far ShapeBuilders nested inside queries
were still transported as a byte array that needs to be parsed
later on the shard receiving the query. To be able to also
serialize geo shapes this way, we also need to make all the
implementations of ShapeBuilder implement Writable.

This PR adds this to PointBuilder and also adds tests for
serialization, equality and hashCode.
2015-11-23 11:24:43 +01:00
Boaz Leskes 6e2e91cf2a Set an newly created IndexShard's ShardRouting before exposing it to operations
The work for #10708 requires tighter integration with the current shard routing of a shard. As such, we need to make sure it is set before the IndexService exposes the shard to external operations.

Closes #14918
2015-11-23 11:22:40 +01:00
Dominik Stadler 72cab4e9b6 Map log-level 'trace' to JDK-Level 'FINEST' to make it possible to actually enable the trace-log via JdkESLogger.setLevel() 2015-11-23 09:49:29 +01:00
Martijn van Groningen 48771f1a76 field stats: Added `min_value_as_string` and `max_value_as_string` response elements for all number based fields. The existing `min_value` and `max_value` will return the values as numbers instead.
Closes #14404
2015-11-23 08:48:28 +01:00
Ryan Ernst b9976ada99 Build: Remove old files replaced by gradle
This change removes files that are no longer needed with the gradle
build. The license checker was already rewritten in groovy. The plugin
descriptor template exists in buildSrc resources. log4j properties was
moved to the test framework. site_en.xml seems to be a legacy file,
there are no references to it anywhere in the maven build that I could
find. The update lucene script was just a helper for running the license
check in update mode, but that can be done with gradle using the
updateShas command. Finally, there was a leftover build.gradle from when
I attempted to make dev-tools a project of its own.
2015-11-22 23:46:24 -08:00
Martijn van Groningen 4b94b50eb1 field stats: Index constraints should remove indices in the response if the field to evaluate s empty
Index constraints should remove indices in the response if the field to evaluate if empty. Index constraints can't work with that and it is the same as if the field doesn't match.
2015-11-23 08:35:15 +01:00
Ryan Ernst 503d836ee4 Build: Add back leaveTemporary and ifNoTests
This change adds back the last of the missing test options to the juni4
wrapper. leaveTemporary is important in that setting it to true (which
is how we had it set in maven) removes the warnings we currently get
about a leftover file that cannot be deleted (from jna).
2015-11-22 22:18:42 -08:00
Ryan Ernst b2ef770f07 Merge pull request #14923 from rjernst/enforce_gradle_idea
Enforce that gradle idea was run before importing in intellij
2015-11-22 19:03:21 -08:00
Jason Tedor 5e2eb7d0de Clarify three possible outcomes in CacheTests#testCachePollution 2015-11-22 19:58:04 -05:00
Jason Tedor 14a635fe4b Use correct equality in CacheTests#testReplaceRecomputesSize.Value 2015-11-22 19:56:27 -05:00
Ryan Ernst 3071de2d33 Build: Enforce that gradle idea was run before importing in intellij
Due to how intellij imports gradle projects, the tasks which setup
resources are not run. This means we must be sure to run gradle idea
from the command line before importing into elasticsearch. This change
adds a simple marker file to indicate we have run from the command line
before importing. It won't help for new projects that add plugin
metadata, but it will at least make sure the initial project is set up
correctly.
2015-11-22 16:07:39 -08:00
Ryan Ernst bc0a840a9c Merge pull request #14921 from rjernst/multi_node_qa
Get multi node smoke tests working
2015-11-22 15:27:33 -08:00
Ryan Ernst 7d3da7e6ee Build: Get multi node smoke tests working
This change adds back the multi node smoke test, as well as making the
cluster formation for any test allow multiple nodes. The main changes in
cluster formation are abstracting out the node specific configuration to
a helper struct, as well as making a single wait task that waits for all
nodes after their start tasks have run. The output on failure was also
improved to log which node's info is being printed.
2015-11-22 14:49:05 -08:00
Ryan Ernst 6b2f3a9ad2 Fix bad merge 2015-11-22 08:51:34 -08:00
Ryan Ernst 8e2b670b28 Merge pull request #14912 from rjernst/extra_plugins
Change project attachment into special extra-plugins dir
2015-11-22 08:45:24 -08:00
Ryan Ernst b8e462b934 Build: Change project attachment into special extra-plugins dir
Currently we use the "gradle project attachment plugin" to support
building elasticsearch as part of another project. However, this plugin
has a number of issues, a large part of which is requiring consistent
use of the projectsPrefix.

This change removes projectsPrefix, and adds support for a special
extra-plugins directory in the root of elasticsearch. Any projects
checked out within this directory will be automatically added to
elasticsearch.
2015-11-22 08:44:33 -08:00
Robert Muir 5337686d8d Merge pull request #14914 from rmuir/lock_down_system_property_writes
Ban write access to system properties
2015-11-22 11:28:55 -05:00
Ryan Ernst 9ee08dbb0b Merge pull request #14916 from rjernst/force_task_order
Enforce similar tasks run for dependencies first
2015-11-22 07:41:42 -08:00
Ryan Ernst f9351b3e0e Build: Enforce similar tasks run for dependencies first
Gradle ensures task dependencies are executed in the correct order.
However, project dependencies only build what is needed for the
dependency. This means the order of higher level tasks are not
guaranteed. This change adds task ordering between test and integTest
for a project and its dependencies.
2015-11-22 01:17:49 -08:00
Robert Muir 9e41d5a738 fix typo in comment 2015-11-21 23:13:05 -05:00
Robert Muir 30529c008d Ban write access to system properties
* Forbid System.setProperties & co in forbidden APIs.
* Ban property write access at runtime with security manager.

Plugins that need to modify system properties will need to request permission in their plugin-security.policy
2015-11-21 22:33:06 -05:00