Commit Graph

15011 Commits

Author SHA1 Message Date
Jason Tedor f4774d17a6 Do not swallow exceptions thrown while parsing settings
This commit fixes an issue that was causing Elasticsearch to silently
ignore settings files that contain garbage. The underlying issue was
swallowing an SettingsException under the assumption that the only
reason that an exception could be throw was due to the settings file
not existing (in this case the IOException would be the cause of the
swallowed SettingsException). This assumption is mistaken as an
IOException could also be thrown due to an access error or a read
error. Additionally, a SettingsException could be thrown exactly
because garbage was found in the settings file. We should instead
explicitly check that the settings file exists, and bomb on an
exception thrown for any reason.

Closes #13028
2015-08-21 10:43:48 -04:00
Jason Tedor 13c1c27d5a Merge pull request #13036 from jasontedor/limit-type-name-length
Limit type name length
2015-08-21 09:19:11 -04:00
Jason Tedor 1579e491d8 Limit type name length
This commit will limit type name length to be at most 255 characters.

Closes #13021
2015-08-21 09:18:30 -04:00
Jason Tedor f0aae96f11 Utility method for getting the first version prior to given version 2015-08-21 09:18:29 -04:00
Simon Willnauer d96e8634ac Suppress rest exceptions by default and log them instead
Today we are very verbose when rendering exceptions on the rest layer.
Yet, this isn't necessarily very easy to read and way too much infromation most
of the time. This change suppresses the stacktrace rendering by default but instead
adds a `rest.suppressed` logger that logs the suppressed stacktrace or rather the entire
exception on the node that renderes the exception
The log message looks like this:

```
[2015-08-19 16:21:58,427][INFO ][rest.suppressed          ] /test/_search/ Params: {index=test}
[test] IndexNotFoundException[no such index]
	at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.resolve(IndexNameExpressionResolver.java:551)
```
2015-08-21 15:17:45 +02:00
javanna ded442952b Java api: add missing support for escape to QueryStringQueryBuilder
QueryStringQueryParser parses `escape`, but java api users had no chance to actually set it.

Closes #13016
2015-08-21 15:04:53 +02:00
Simon Willnauer 650a86e2ce Merge pull request #12985 from s1monw/cleanup_release_script
Refactor script for RC creation
2015-08-21 14:37:32 +02:00
Simon Willnauer 5458d07ea2 Refactor script for RC creation
The script now allows to run all required steps at once and alternatively
prints out manual instructions to run the steps individually. It also has
flags and options to run debug builds from a local checkout.
2015-08-21 14:36:59 +02:00
Jason Tedor 31b80e4f3f Merge pull request #13010 from jasontedor/feature/improve-java-version-comparison
Improve java version comparison and explicitly enforce a version format
2015-08-21 08:30:50 -04:00
Jason Tedor 5ff1f8a058 Enforce version format for java.version for plugins
Currently we implicitly enforce a version format on the java.version
property for plugins via JarHell.checkJavaVersion. We should explicitly
enforce this version format and specify it in the documentation.

This commit adds an explicit enforcement of the version format on the
java.version property for plugins and updates the documentation for
plugin-descriptor.properties accordingly.

Closes #13009
2015-08-21 08:28:41 -04:00
Jason Tedor 126e8e4aee Improve Java version comparison in JarHell
This commit improves Java version comparison in JarHell.

The first improvement is the addition of a method to check the version
format of a target version string. This method will reject target
version strings that are not a sequence of nonnegative decimal integers
separated by “.”s, possibly with leading zeros (0*[0-9]+(\.[0-9]+)?).
This version format is the version format used for Java specification
versioning (cf. Java Product Versioning, 1.5.1 Specification Versioning
and the Javadocs for java.lang.Package.)

The second improvement is a clean method for checking that a target
version is compatible with the runtime version of the JVM. This is done
using the system property java.specification.version and comparing the
versions lexicograpically. This method of comparison has been tested on
JDK 9 builds that include JEP-220 (the Project Jigsaw JEP concerning
modular runtime images) and JEP-223 (the version string JEP). The class
that encapsulates the methods for parsing and comparing versions is
written in a way that can easily be converted to use the Version class
from JEP-223 if that class is ultimately incorporated into mainline JDK
9.

Closes #12441
2015-08-21 08:27:47 -04:00
Jason Tedor 8fca951be0 Merge pull request #12901 from jasontedor/feature/12900
Accumulate validation errors when validating index templates
2015-08-21 08:08:50 -04:00
Jason Tedor 4a5040bb32 Accumulate validation errors when validating index templates
This commit changes the behavior when validating index templates to
accumulate all validation errors before reporting failure to the user.
This addresses a usability issue when creating index templates.

Closes #12900
2015-08-21 08:06:42 -04:00
Colin Goodheart-Smithe 38085cf90a Aggregation: Fix AggregationPath.subPath() to not throw ArrayStoreException
Aggregation.subPath() always threw an ArrayStoreException because we were trying to pass a List into System.arraycopy(). This change fixes that bug and adds a test to prevent regression
2015-08-21 12:50:05 +01:00
Simon Willnauer 71cbcea3c2 [TEST] Add discovery-multicast to PluginManagerIT 2015-08-21 13:46:25 +02:00
xuzha 2276ff6283 Remove `cloud.account` and `cloud.key` settings
closes #12809
2015-08-21 00:56:15 -07:00
Simon Willnauer 47a0296211 Clear ScrollID / NextScrollID even if the first query returned no docs 2015-08-21 09:51:30 +02:00
Adrien Grand d99b5c8bc6 Merge pull request #13032 from jpountz/fix/match_all_caching
Never cache match_all queries.
2015-08-21 09:38:50 +02:00
Adrien Grand c54a4c4f89 Docs: Fix section id for the new discovery-multicast plugin. 2015-08-21 09:36:55 +02:00
Adrien Grand f3265095ab Never cache match_all queries.
This commit backports https://issues.apache.org/jira/browse/LUCENE-6748 to make
sure that we never cache trivial queries like MatchAllDocsQuery.
2015-08-21 09:31:53 +02:00
Ryan Ernst 16e5c7c21c Add multicast plugin to plugin manager official list of plugins
See #13027
2015-08-20 23:45:36 -07:00
Simon Willnauer 3fb2d8e448 Merge pull request #12947 from s1monw/expected_shard_size
Add `expectedShardSize` to ShardRouting and use it in path.data allocation
2015-08-21 08:34:38 +02:00
Ryan Ernst 703a4b3f40 Merge pull request #13027 from rjernst/cast_away_multicast
Move multicast discovery to a plugin
2015-08-20 23:21:12 -07:00
Simon Willnauer 3dd6c4ab80 Use constant to determin if expected size is available 2015-08-21 08:18:59 +02:00
Simon Willnauer d2507c4ac0 Add `expectedShardSize` to ShardRouting and use it in path.data allocation
Today we only guess how big the shard will be that we are allocating on a node.
Yet, we have this information on the master but it's not available on the data nodes
when we pick a data path for the shard. We use some rather simple heuristic based on
existing shard sizes on this node which might be complete bogus. This change adds
the expected shard size to the ShardRouting for RELOCATING and INITIALIZING shards
to be used on the actual node to find the best data path for the shard.

Closes #11271
2015-08-21 08:18:59 +02:00
Robert Muir a2507a8f1c test: fix and simplify logic 2015-08-21 02:06:34 -04:00
Ryan Ernst 0c09bf2b18 Remove outdated comment 2015-08-20 22:30:26 -07:00
Robert Muir d0835715c2 Fix formatting of startup/configuration errors.
Try to strike a balance between usability and debuggability.
2MB stacktrace is not gonna work... we print message and find
"likely root cause" and identify it as such, limit it to 30 stack
frames, and filter out guice. When frames are truncated we identify
that too, and always inform the user full exception is available from
logs.

Closes #13029
2015-08-21 01:19:29 -04:00
Ryan Ernst d0f5ce58d8 Add multicast plugin to help and qa 2015-08-20 22:04:17 -07:00
Ryan Ernst 2b1e4acafe Remove logging message from testing 2015-08-20 17:08:44 -07:00
Ryan Ernst bcd58ad02f Merge branch 'master' into cast_away_multicast 2015-08-20 17:03:02 -07:00
Ryan Ernst 164efaecbe Networking: Move multicast discovery to a plugin
Multicast has known issues (see #12999 and #12993). This change moves
multicast into a plugin, and deprecates it in the docs.  It also allows
for plugging in multiple zen ping implementations.

closes #13019
2015-08-20 16:43:25 -07:00
Robert Muir bd539e7f98 simplify this 2015-08-20 18:56:26 -04:00
Robert Muir 7298be474d fix test failure 2015-08-20 17:31:06 -04:00
Robert Muir 478bf1c9ef Merge pull request #13026 from rmuir/too_noisy
Don't print lots of noise on IPv4 only hosts.
2015-08-20 17:11:40 -04:00
Simon Willnauer aea596241c Merge pull request #13020 from s1monw/issues/13014
Remove support for address resolving in InetSocketTransportAddress
2015-08-20 23:10:50 +02:00
Robert Muir 5beabdd0ec Don't print lots of noise on IPv4 only hosts.
If the machine doesn't support IPv6, or if the user disabled it
with -Djava.net.preferIPv4Stack, or if the user disabled it with
ES_USE_IPV4, we will still try to bind a socket to ::1 out of box,
and we will see lots of exceptions logged to the console.

It is harmless noise but not a great experience.
2015-08-20 17:05:30 -04:00
Simon Willnauer d9089d56aa Don't use networking in InternalTestClusterTests 2015-08-20 22:55:04 +02:00
Simon Willnauer 3eee8b387e only verify both nodes join the cluster 2015-08-20 21:51:13 +02:00
Simon Willnauer c15d4c33b2 Add back private default ctor 2015-08-20 21:42:14 +02:00
Simon Willnauer 214d303e6a Remove support for address resolving in InetSocketTransportAddress
this commit removes all support for reverse host name resolving from
InetSocketTransportAddress. This class now only returns IP addresses.

Closes #13014
2015-08-20 20:57:26 +02:00
Robert Muir e2ab62596f Default to unicast discovery, with default host list of 127.0.0.1, [::1]
Fix unicast discovery to work when a host has multiple addresses.
Ban dangerous methods in java.net with forbidden APIs.
Fix ipv6 bugs and formatting of network addresses everywhere.

Closes #12999
Closes #12993

Squashed commit of the following:

commit 6c1aa001d091c5cf25212a53dc701fb704337f1e
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 14:25:43 2015 -0400

    Fix these to be correct with addresses just in case

commit 648215627e84abf58a71400e7dc9ae775efb71d6
Merge: d00561b 41d8fbe
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 13:23:09 2015 -0400

    Merge branch 'master' into unicast_all_the_way_down

commit d00561b76fd1aa5850699f7901f3dae3d4d402b7
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 16:38:50 2015 +0200

    limit local ports to 5 in UnicastZenPing

commit e2e15c594006746cbe24432694294a71cc99deb8
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 10:32:47 2015 -0400

    fix port limiting

commit 10153cb7adadda81a1f482445e703836b65cf5e2
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 10:18:37 2015 -0400

    don't serialize scopeids: that's broken

commit 2aa63d43db2baec68a2e9bc227cfeb85dfeb4f83
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 16:06:51 2015 +0200

    restore @Network

commit c840f1d1ef438826ae1ecfd5e45942a0e30dc9c0
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 16:02:30 2015 +0200

     Use NetworkAddress.formatAddress where applicable in plugins

commit 374ce878852b35d626b7a29c8c4773545b0e9ddd
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 15:34:06 2015 +0200

    Use NetworkAddress.formatAddress where applicable

commit e7a606d63f1bc43c1b62b6e17adf707c76d43a15
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 10:17:57 2015 +0200

    Add @Multicast annotation to disable multicast tests by default.

    We only run multicast tests now when we explicitly state it. A working
    multicast env is required which is not always the case.

commit 2d7d2d0347179696ab41f71f048b13305014c85b
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 09:51:28 2015 +0200

    Remove extra check for local mode in InternalTestCluster

commit dda59ac39aa136d4687b9274c2692cd77f8b8f66
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 09:37:03 2015 +0200

    Handle node mode across entire test cluster

    We used static methods reading sys properties to define the node mode
    per cluster. this had lots of problems when tests couldn't cope with
    mixed or only local mode. Now we are passing it down to the cluster from the test
    which allows to @SuppressNetworkMode / @SupressLocalMode on the test to force
    consistent node configurations.

commit 058197b7a408318995c88ce7f6762e32348de0de
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 03:19:14 2015 -0400

    really ban InetSocketAddress's trappy method and break build and go to sleep, sorry

commit ac8779185aee1e17e6f5a81766290fdfc9c603ba
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 03:16:52 2015 -0400

    Ban methods that might surprisingly cause DNS lookups

commit e64fe3dff2b11503e5f2831eb9863d64f56c5538
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 02:59:05 2015 -0400

    Add unit test

commit f15434f20fb1a3691b1cc16028597d8fae937e05
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 02:39:02 2015 -0400

    fix ipv6 formatting bugs

commit 05c2c74098052c75fbb79ea1818a295ef2e03e30
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 02:12:05 2015 -0400

    format addresses correctly so I can actually read what comes out of our logs and stats apis

commit 4f9389dcf1e8925f23153c5eb271b4ce2294dbaf
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 21:26:52 2015 -0400

    ban dangerous methods in java.net

commit 6aacd4d9925f324903d1d099a6cf5f862aeaf677
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 20:59:24 2015 -0400

    ban lenient method

commit f466a842c60163d1f4554bdce8a4163edb534c2c
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 00:29:00 2015 +0200

    fix tests to not mix local transport and zen unicast disco

commit 0de007a33b33fb68cf85cd86db4ca4f8ce10bbc9
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 00:10:07 2015 +0200

    fix tests to not mix local transport and zen unicast disco

commit 539f6ca6e5137e0d496239adc8684688dedcc824
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 00:02:01 2015 +0200

    fix tests to not mix local transport and zen unicast disco

commit 004c2881b25467f332acc8c9f9e92b1f0f9d314e
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 17:51:45 2015 -0400

    Fix multinode

commit 54113af325ce31571811c49fdaae89d5687be4ba
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 17:36:45 2015 -0400

    fix integration tests

commit 0156a77a56319d6b9737ec6a531992052e50bd59
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 23:32:18 2015 +0200

    enable multicast in MulticastZenPingIT.java

commit 1791caa35da853ce0122485fa3fd4674c671ec6e
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 17:23:16 2015 -0400

    Fix constant

commit 22820b53e0b2dc9fd47145c2bc29ce912a8fd484
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 22:59:09 2015 +0200

    give it some extra ids for local transport crazyness

commit b2138fafa94a8a085813fd48356df63e57ade5b3
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 22:51:42 2015 +0200

    pass on local addresses from configured transport rather than hard code IP addresses

commit 1bf5de1f457b081e0ce262b57d2b55d39c434156
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 22:04:31 2015 +0200

    fix PluggableTransportModuleIT.java to use local disco and detach port limit for node local disco

commit b6706eddfa04c43947c16551359ae98a463d34aa
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 14:16:03 2015 -0400

    Default to unicast discovery, with default host list of 127.0.0.1, [::1]
2015-08-20 14:26:40 -04:00
Adrien Grand 41d8fbe8f5 Merge pull request #12994 from jpountz/deprecate/scan
Deprecate the `scan` search type.
2015-08-20 12:47:45 +02:00
Adrien Grand 6fa258b8fa Deprecate the `scan` search type.
This commit deprecates the `scan` search type in favour of regular scroll
requests sorted by `_doc`.

Related to #12983
2015-08-20 12:47:23 +02:00
Adrien Grand 833f821171 Merge pull request #12983 from jpountz/enhancement/scroll_doc
Optimize sorted scroll when sorting by `_doc`.
2015-08-20 12:01:04 +02:00
Adrien Grand 78c2f1063a Optimize sorted scroll when sorting by `_doc`.
This change means that we will be able to remove the `SCAN` search type in 3.0
and recommend users to use sorted scrolls instead.
2015-08-20 11:51:59 +02:00
David Pilato 449dfc8f56 Merge branch 'fix/12984-rpm-module' 2015-08-20 09:43:47 +02:00
Adrien Grand 7662705f37 Merge pull request #12997 from jpountz/fix/doc_scroll_close
Fix documentation: scrolls are not closed automatically.
2015-08-20 09:24:02 +02:00
Adrien Grand 551e92ec71 Fix documentation: scrolls are not closed automatically.
The documentation states that scrolls are automatically closed when all
documents are consumed, but this is not the case. I first tried to fix
the code to close scrolls automatically but this made REST tests fail
because clearing a scroll that is already closed returned a 4xx error
instead of a 2xx code, so this has probably been this way for a very long
time.
2015-08-20 09:20:40 +02:00
Jason Tedor a0243200e0 Fix a documentation typo and a code comment typo to path.repo 2015-08-19 22:14:50 -04:00