Commit Graph

14972 Commits

Author SHA1 Message Date
Simon Willnauer c15d4c33b2 Add back private default ctor 2015-08-20 21:42:14 +02:00
Simon Willnauer 214d303e6a Remove support for address resolving in InetSocketTransportAddress
this commit removes all support for reverse host name resolving from
InetSocketTransportAddress. This class now only returns IP addresses.

Closes #13014
2015-08-20 20:57:26 +02:00
Robert Muir e2ab62596f Default to unicast discovery, with default host list of 127.0.0.1, [::1]
Fix unicast discovery to work when a host has multiple addresses.
Ban dangerous methods in java.net with forbidden APIs.
Fix ipv6 bugs and formatting of network addresses everywhere.

Closes #12999
Closes #12993

Squashed commit of the following:

commit 6c1aa001d091c5cf25212a53dc701fb704337f1e
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 14:25:43 2015 -0400

    Fix these to be correct with addresses just in case

commit 648215627e84abf58a71400e7dc9ae775efb71d6
Merge: d00561b 41d8fbe
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 13:23:09 2015 -0400

    Merge branch 'master' into unicast_all_the_way_down

commit d00561b76fd1aa5850699f7901f3dae3d4d402b7
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 16:38:50 2015 +0200

    limit local ports to 5 in UnicastZenPing

commit e2e15c594006746cbe24432694294a71cc99deb8
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 10:32:47 2015 -0400

    fix port limiting

commit 10153cb7adadda81a1f482445e703836b65cf5e2
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 10:18:37 2015 -0400

    don't serialize scopeids: that's broken

commit 2aa63d43db2baec68a2e9bc227cfeb85dfeb4f83
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 16:06:51 2015 +0200

    restore @Network

commit c840f1d1ef438826ae1ecfd5e45942a0e30dc9c0
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 16:02:30 2015 +0200

     Use NetworkAddress.formatAddress where applicable in plugins

commit 374ce878852b35d626b7a29c8c4773545b0e9ddd
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 15:34:06 2015 +0200

    Use NetworkAddress.formatAddress where applicable

commit e7a606d63f1bc43c1b62b6e17adf707c76d43a15
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 10:17:57 2015 +0200

    Add @Multicast annotation to disable multicast tests by default.

    We only run multicast tests now when we explicitly state it. A working
    multicast env is required which is not always the case.

commit 2d7d2d0347179696ab41f71f048b13305014c85b
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 09:51:28 2015 +0200

    Remove extra check for local mode in InternalTestCluster

commit dda59ac39aa136d4687b9274c2692cd77f8b8f66
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 09:37:03 2015 +0200

    Handle node mode across entire test cluster

    We used static methods reading sys properties to define the node mode
    per cluster. this had lots of problems when tests couldn't cope with
    mixed or only local mode. Now we are passing it down to the cluster from the test
    which allows to @SuppressNetworkMode / @SupressLocalMode on the test to force
    consistent node configurations.

commit 058197b7a408318995c88ce7f6762e32348de0de
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 03:19:14 2015 -0400

    really ban InetSocketAddress's trappy method and break build and go to sleep, sorry

commit ac8779185aee1e17e6f5a81766290fdfc9c603ba
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 03:16:52 2015 -0400

    Ban methods that might surprisingly cause DNS lookups

commit e64fe3dff2b11503e5f2831eb9863d64f56c5538
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 02:59:05 2015 -0400

    Add unit test

commit f15434f20fb1a3691b1cc16028597d8fae937e05
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 02:39:02 2015 -0400

    fix ipv6 formatting bugs

commit 05c2c74098052c75fbb79ea1818a295ef2e03e30
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 02:12:05 2015 -0400

    format addresses correctly so I can actually read what comes out of our logs and stats apis

commit 4f9389dcf1e8925f23153c5eb271b4ce2294dbaf
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 21:26:52 2015 -0400

    ban dangerous methods in java.net

commit 6aacd4d9925f324903d1d099a6cf5f862aeaf677
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 20:59:24 2015 -0400

    ban lenient method

commit f466a842c60163d1f4554bdce8a4163edb534c2c
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 00:29:00 2015 +0200

    fix tests to not mix local transport and zen unicast disco

commit 0de007a33b33fb68cf85cd86db4ca4f8ce10bbc9
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 00:10:07 2015 +0200

    fix tests to not mix local transport and zen unicast disco

commit 539f6ca6e5137e0d496239adc8684688dedcc824
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 00:02:01 2015 +0200

    fix tests to not mix local transport and zen unicast disco

commit 004c2881b25467f332acc8c9f9e92b1f0f9d314e
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 17:51:45 2015 -0400

    Fix multinode

commit 54113af325ce31571811c49fdaae89d5687be4ba
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 17:36:45 2015 -0400

    fix integration tests

commit 0156a77a56319d6b9737ec6a531992052e50bd59
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 23:32:18 2015 +0200

    enable multicast in MulticastZenPingIT.java

commit 1791caa35da853ce0122485fa3fd4674c671ec6e
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 17:23:16 2015 -0400

    Fix constant

commit 22820b53e0b2dc9fd47145c2bc29ce912a8fd484
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 22:59:09 2015 +0200

    give it some extra ids for local transport crazyness

commit b2138fafa94a8a085813fd48356df63e57ade5b3
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 22:51:42 2015 +0200

    pass on local addresses from configured transport rather than hard code IP addresses

commit 1bf5de1f457b081e0ce262b57d2b55d39c434156
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 22:04:31 2015 +0200

    fix PluggableTransportModuleIT.java to use local disco and detach port limit for node local disco

commit b6706eddfa04c43947c16551359ae98a463d34aa
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 14:16:03 2015 -0400

    Default to unicast discovery, with default host list of 127.0.0.1, [::1]
2015-08-20 14:26:40 -04:00
Adrien Grand 41d8fbe8f5 Merge pull request #12994 from jpountz/deprecate/scan
Deprecate the `scan` search type.
2015-08-20 12:47:45 +02:00
Adrien Grand 6fa258b8fa Deprecate the `scan` search type.
This commit deprecates the `scan` search type in favour of regular scroll
requests sorted by `_doc`.

Related to #12983
2015-08-20 12:47:23 +02:00
Adrien Grand 833f821171 Merge pull request #12983 from jpountz/enhancement/scroll_doc
Optimize sorted scroll when sorting by `_doc`.
2015-08-20 12:01:04 +02:00
Adrien Grand 78c2f1063a Optimize sorted scroll when sorting by `_doc`.
This change means that we will be able to remove the `SCAN` search type in 3.0
and recommend users to use sorted scrolls instead.
2015-08-20 11:51:59 +02:00
David Pilato 449dfc8f56 Merge branch 'fix/12984-rpm-module' 2015-08-20 09:43:47 +02:00
Adrien Grand 7662705f37 Merge pull request #12997 from jpountz/fix/doc_scroll_close
Fix documentation: scrolls are not closed automatically.
2015-08-20 09:24:02 +02:00
Adrien Grand 551e92ec71 Fix documentation: scrolls are not closed automatically.
The documentation states that scrolls are automatically closed when all
documents are consumed, but this is not the case. I first tried to fix
the code to close scrolls automatically but this made REST tests fail
because clearing a scroll that is already closed returned a 4xx error
instead of a 2xx code, so this has probably been this way for a very long
time.
2015-08-20 09:20:40 +02:00
Jason Tedor a0243200e0 Fix a documentation typo and a code comment typo to path.repo 2015-08-19 22:14:50 -04:00
Jason Tedor f3c7bf0936 Fix documentation typo to path.repo for UNC path example
Closes #13008
2015-08-19 21:55:32 -04:00
Igor Motov dc82262db6 Don't show access_key and filter_key in S3 repository settings
In #11265 we added an ability to filter out sensitive repository settings. This commit uses this change to filter out access_key and filter_key in S3 repository settings.

Closes elastic/elasticsearch-cloud-aws#184
2015-08-19 16:21:42 -04:00
Robert Muir 829b858c1d Merge pull request #12995 from rmuir/dedup_resolver_for_binding
Deduplicate addresses from resolver.
2015-08-19 16:17:45 -04:00
Martijn van Groningen f2f95ea115 Removed CachedDfSource and move the dfs logic into the ContextIndexSearcher 2015-08-19 20:25:41 +02:00
Christoph Büscher 0fc96ede69 Merge pull request #12886 from cbuescher/fix/12278
Make ValueParser.DateMath aware of timezone setting
2015-08-19 18:42:19 +02:00
Robert Muir ce120b9c71 Deduplicate addresses from resolver.
In some cases this may contain duplicates, although its a misconfiguration,
lets not bind to multiple ports. Its no problem for us to dedup, this code
doesn't need to be huper-duper fast since its used only for logic around bind/publish
2015-08-19 12:06:54 -04:00
Britta Weber c0d31a2c9e Merge pull request #12965 from brwe/es-running-already
fail with better error message if elasticsearch was started already
2015-08-19 17:56:56 +02:00
Britta Weber 4cc6359756 fail with better error message if elasticsearch was started already
Before it failed with
.../es/qa/smoke-test-plugins/target/integ-tests/es.pid doesn't exist
which was confusing.
2015-08-19 17:56:05 +02:00
Simon Willnauer 22ae0a161f Log all interfaces we try to bind before binding 2015-08-19 17:20:48 +02:00
David Pilato dba1b52e81 [build] rpm module should be build on machine with /usr/bin/rpmbuild
We build the rpm module automatically if you have either:

* `/usr/bin/rpmbuild`
* `/usr/local/bin/rpmbuild`

available.

If your `rpmbuild` is in another location and available in your path, then run maven with `rpm` profile:

```sh
mvn deploy -Prpm
```

Closes #12984.
2015-08-19 15:37:28 +02:00
Colin Goodheart-Smithe 88d482437a Aggregations: Throw error if cardinality aggregator has a sub aggregation
The cardinality aggregation is a metric aggregation and therefore cannot accept sub-aggregations. It was previously possible to create a rest request with a cardinality aggregation that had sub-aggregations. Now such a request will throw an error in the response.

Close #12988
2015-08-19 14:31:57 +01:00
Simon Willnauer ca28acc373 Merge pull request #12986 from s1monw/fix_host_resolving
Only resolve host if explicitly allowed.
2015-08-19 15:25:55 +02:00
Simon Willnauer e5134317be Merge pull request #12936 from s1monw/add_build_hash_to_headers
Add build short hash to the download manager headers to identify staging builds
2015-08-19 15:10:46 +02:00
Simon Willnauer bd56b0a79e Only resolve host if explicitly allowed.
We have some settings that prevent host name resolution which should
be repected by InetSocketTransportAddress#getHost() to only resolve if
allowed or desired.
2015-08-19 14:54:56 +02:00
Robert Muir bbf669506b Merge pull request #12979 from rmuir/ifconfig
Log network configuration at debug level
2015-08-19 07:14:01 -04:00
Christoph Büscher 4410287f42 Merge pull request #12775 from cbuescher/fix/12715
Use 'name' from plugin descriptor file to determine plugin name
2015-08-19 12:33:34 +02:00
Christoph Büscher 8454d49552 Plugins: Add 'name' property to plugin descriptor file to determine plugin name
At the moment, when installing from an url, a user provides the plugin name on
the command line like:

* bin/plugin install [plugin-name] --url [url]

This can lead to problems when picking an already existing name from another
plugin, and can potentially overwrite plugins already installed with that name.

This, this PR introduces a mandatory `name` property to the plugin descriptor
file which replaces the name formerly provided by the user.

With the addition of the `name` property to the plugin descriptor file, the user
does not need to specify the plugin name any longer when installing from a file
or url. Because of this, all arguments to `plugin install` command are now
either treated as a symbolic name, a URL or a file without the need to specify
this with an explicit option.

The new syntax for `plugin install` is now:

bin/plugin install [name or url]

* downloads official plugin
bin/plugin install analysis-kuromoji

* downloads github plugin
bin/plugin install lmenezes/elasticsearch-kopf

* install from URL or file
bin/plugin install http://link.to/foo.zip
bin/plugin install file:/path/to/foo.zip

If the argument does not parse to a valid URL, it is assumed to be a name and the
download location is resolved like before. Regardless of the source location of
the plugin, it is extracted to a temporary directory and the `name` property from
the descriptor file is used to determine the final install location.

Relates to #12715
2015-08-19 12:13:01 +02:00
Jason Tedor 66b0e7a6e1 Merge pull request #12977 from jasontedor/fix/12873
Add millisecond parser for dynamic date fields mapped from "yyyy/MM/dd"
2015-08-19 05:30:24 -04:00
Simon Willnauer ddd6be1047 Merge pull request #12981 from s1monw/serialize_interrupted_exceptions
Add serialization support for InterruptedException
2015-08-19 11:26:49 +02:00
Simon Willnauer 828e31ce32 Add serialization support for InterruptedException
it's an important exception to serialize and we see it often in tests
etc. but then it's wrapped in NotSerializableExceptionWrapper which is
odd. This commit adds native support for this exception.
2015-08-19 10:01:58 +02:00
Boaz Leskes 43fae91ab9 Test: ClusterDiscoveryConfiguration.UnicastZen should allow for port ranges
The cluster configuration allows to setup a cluster for use with unicast discovery. This means that nodes have to have pre-calculated known addresses which can be used to poplulate the unicast hosts setting. Despite of repeated attempts to select unused ports, we still see test failures where the node can not bind to it's assigned port due to it already being in use (most on CentOS). This commit changes it to allow each node to have a pre-set mutual exclusive range of ports and add all those ports to the unicast hosts list. That's OK because we know the node will only bind to one of those.
2015-08-19 10:01:38 +02:00
Robert Muir fa70162178 Log network configuration at debug level 2015-08-19 01:26:36 -04:00
Robert Muir ef236e3f0f Merge pull request #12976 from rmuir/exceptional_noise
Improve console logging on startup exception
2015-08-18 23:25:02 -04:00
Jason Tedor 4c9327d983 Add millisecond parser for dynamic date fields mapped from "yyyy/MM/dd"
Dynamic date fields mapped from dates of the form "yyyy-MM-dd"
automatically receive the millisecond paresr epoch_millis as an
alternative parsing format. However, dynamic date fields mapped from
dates of the form "yyyy/MM/dd" do not. This is a bug since the migration
documentation currently specifies that a dynamically added date field,
by default, includes the epoch_millis format. This commit adds
epoch_millis as an alternative parser to dynamic date fields mapped from
dates of the form "yyyy/MM/dd".

Closes #12873
2015-08-18 22:30:43 -04:00
Robert Muir f7f7fecafb Improve console logging on startup exception
Today we show the exception twice: once by the logger and then again
by the JVM. This is too noisy, and easy to avoid.
2015-08-18 22:27:39 -04:00
Ryan Ernst 32a097382c Merge pull request #12952 from rjernst/construct_it_yourself
Simplify Plugin API for constructing modules
2015-08-18 14:28:48 -07:00
Ryan Ernst 54ae55662c Remove leftover class and fix compile issues from master merge 2015-08-18 14:28:03 -07:00
Simon Willnauer 32bb49e829 [TEST] Make it clear what address we try to bind 2015-08-18 23:17:43 +02:00
Ryan Ernst 7393068417 Merge branch 'master' into construct_it_yourself 2015-08-18 14:15:08 -07:00
Simon Willnauer 0ffd99cca3 Drop commons-lang dependency
commons-lang really is only used by some core classes to join strings or modiy arrays.
It's not worth carrying the dependency. This commit removes the dependency on commons-lang
entirely.
2015-08-18 22:59:31 +02:00
Robert Muir e07f039659 Workaround JDK bug 8034057
This causes a FileSystemException when trying to retrieve FileStore for a path,
and falsely returns false for Files.isWritable

Squashed commit of the following:

commit d2cc0d966f3bc94aa836b316a42b3c5724bc01ef
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Aug 18 15:49:48 2015 -0400

    fixes for the non-bogus comments

commit 6e0a272f5f8ef7358654ded8ff4ffc31831fa5c7
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Aug 18 15:30:43 2015 -0400

    Fix isWritable too

commit 2a8764ca118fc4c950bfc60d0b97de873e0e82ad
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Aug 18 14:49:50 2015 -0400

    try to workaround filestore bug
2015-08-18 16:44:57 -04:00
Igor Motov 509edefb04 Improve stability of restoreIndexWithMissingShards test
Replaced awaitBusy with assertBusy and increased the timeout.
2015-08-18 14:27:24 -04:00
David Pilato 0599f85d2d [build] simplify ant script for plugins
Now we are using short names for artifactId (see #12879) so we don't need anymore to transform long names `elasticsearch-pluginname` to short names `pluginname` in ant script when we install a plugin.
Modify also convert-plugin-name
Clean up remaining plugins with old format
And fix vagrant tests
2015-08-18 19:41:37 +02:00
Ryan Ernst c57755095c Merge pull request #12967 from rjernst/no_more_toe_stepping
Ensure additionalSettings() do not conflict
2015-08-18 10:36:05 -07:00
Ryan Ernst 2a57539a28 Tweak exception message 2015-08-18 10:35:37 -07:00
Ryan Ernst a0c41218ff Merge branch 'master' into no_more_toe_stepping 2015-08-18 10:34:29 -07:00
Ryan Ernst a3afe57792 Fix compile failure from bad merge after renaming of
ExtensionPoint.TypeExtensionPoint
2015-08-18 10:32:12 -07:00
Ryan Ernst 7db253bbce Merge pull request #12948 from rjernst/module_culling3
Simplify custom repository type setup
2015-08-18 10:17:56 -07:00
Ryan Ernst c0444ccbc6 Merge branch 'master' into module_culling3 2015-08-18 10:16:40 -07:00