Commit Graph

15199 Commits

Author SHA1 Message Date
Ryan Ernst 16e5c7c21c Add multicast plugin to plugin manager official list of plugins
See #13027
2015-08-20 23:45:36 -07:00
Simon Willnauer 3fb2d8e448 Merge pull request #12947 from s1monw/expected_shard_size
Add `expectedShardSize` to ShardRouting and use it in path.data allocation
2015-08-21 08:34:38 +02:00
Ryan Ernst 703a4b3f40 Merge pull request #13027 from rjernst/cast_away_multicast
Move multicast discovery to a plugin
2015-08-20 23:21:12 -07:00
Simon Willnauer 3dd6c4ab80 Use constant to determin if expected size is available 2015-08-21 08:18:59 +02:00
Simon Willnauer d2507c4ac0 Add `expectedShardSize` to ShardRouting and use it in path.data allocation
Today we only guess how big the shard will be that we are allocating on a node.
Yet, we have this information on the master but it's not available on the data nodes
when we pick a data path for the shard. We use some rather simple heuristic based on
existing shard sizes on this node which might be complete bogus. This change adds
the expected shard size to the ShardRouting for RELOCATING and INITIALIZING shards
to be used on the actual node to find the best data path for the shard.

Closes #11271
2015-08-21 08:18:59 +02:00
Robert Muir a2507a8f1c test: fix and simplify logic 2015-08-21 02:06:34 -04:00
Ryan Ernst 0c09bf2b18 Remove outdated comment 2015-08-20 22:30:26 -07:00
Robert Muir d0835715c2 Fix formatting of startup/configuration errors.
Try to strike a balance between usability and debuggability.
2MB stacktrace is not gonna work... we print message and find
"likely root cause" and identify it as such, limit it to 30 stack
frames, and filter out guice. When frames are truncated we identify
that too, and always inform the user full exception is available from
logs.

Closes #13029
2015-08-21 01:19:29 -04:00
Ryan Ernst d0f5ce58d8 Add multicast plugin to help and qa 2015-08-20 22:04:17 -07:00
Ryan Ernst 2b1e4acafe Remove logging message from testing 2015-08-20 17:08:44 -07:00
Ryan Ernst bcd58ad02f Merge branch 'master' into cast_away_multicast 2015-08-20 17:03:02 -07:00
Ryan Ernst 164efaecbe Networking: Move multicast discovery to a plugin
Multicast has known issues (see #12999 and #12993). This change moves
multicast into a plugin, and deprecates it in the docs.  It also allows
for plugging in multiple zen ping implementations.

closes #13019
2015-08-20 16:43:25 -07:00
Robert Muir bd539e7f98 simplify this 2015-08-20 18:56:26 -04:00
Robert Muir 7298be474d fix test failure 2015-08-20 17:31:06 -04:00
Robert Muir 478bf1c9ef Merge pull request #13026 from rmuir/too_noisy
Don't print lots of noise on IPv4 only hosts.
2015-08-20 17:11:40 -04:00
Simon Willnauer aea596241c Merge pull request #13020 from s1monw/issues/13014
Remove support for address resolving in InetSocketTransportAddress
2015-08-20 23:10:50 +02:00
Murilo Pereira a960b3cac4 Here too. 2015-08-20 18:07:51 -03:00
Robert Muir 5beabdd0ec Don't print lots of noise on IPv4 only hosts.
If the machine doesn't support IPv6, or if the user disabled it
with -Djava.net.preferIPv4Stack, or if the user disabled it with
ES_USE_IPV4, we will still try to bind a socket to ::1 out of box,
and we will see lots of exceptions logged to the console.

It is harmless noise but not a great experience.
2015-08-20 17:05:30 -04:00
Murilo Pereira 13f961a3d3 s/bucket_paths/buckets_path/
Using "bucket_paths" makes the server return a 400 with "Unknown key for a VALUE_STRING in [aggregation-name]: [buckets_paths]."
2015-08-20 18:05:02 -03:00
Ian Truslove ae0a74eb1c Couple of typos - various misspellings of `buckets-path` 2015-08-20 14:57:09 -06:00
Simon Willnauer d9089d56aa Don't use networking in InternalTestClusterTests 2015-08-20 22:55:04 +02:00
Simon Willnauer 3eee8b387e only verify both nodes join the cluster 2015-08-20 21:51:13 +02:00
Simon Willnauer c15d4c33b2 Add back private default ctor 2015-08-20 21:42:14 +02:00
Simon Willnauer 214d303e6a Remove support for address resolving in InetSocketTransportAddress
this commit removes all support for reverse host name resolving from
InetSocketTransportAddress. This class now only returns IP addresses.

Closes #13014
2015-08-20 20:57:26 +02:00
Robert Muir e2ab62596f Default to unicast discovery, with default host list of 127.0.0.1, [::1]
Fix unicast discovery to work when a host has multiple addresses.
Ban dangerous methods in java.net with forbidden APIs.
Fix ipv6 bugs and formatting of network addresses everywhere.

Closes #12999
Closes #12993

Squashed commit of the following:

commit 6c1aa001d091c5cf25212a53dc701fb704337f1e
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 14:25:43 2015 -0400

    Fix these to be correct with addresses just in case

commit 648215627e84abf58a71400e7dc9ae775efb71d6
Merge: d00561b 41d8fbe
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 13:23:09 2015 -0400

    Merge branch 'master' into unicast_all_the_way_down

commit d00561b76fd1aa5850699f7901f3dae3d4d402b7
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 16:38:50 2015 +0200

    limit local ports to 5 in UnicastZenPing

commit e2e15c594006746cbe24432694294a71cc99deb8
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 10:32:47 2015 -0400

    fix port limiting

commit 10153cb7adadda81a1f482445e703836b65cf5e2
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 10:18:37 2015 -0400

    don't serialize scopeids: that's broken

commit 2aa63d43db2baec68a2e9bc227cfeb85dfeb4f83
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 16:06:51 2015 +0200

    restore @Network

commit c840f1d1ef438826ae1ecfd5e45942a0e30dc9c0
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 16:02:30 2015 +0200

     Use NetworkAddress.formatAddress where applicable in plugins

commit 374ce878852b35d626b7a29c8c4773545b0e9ddd
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 15:34:06 2015 +0200

    Use NetworkAddress.formatAddress where applicable

commit e7a606d63f1bc43c1b62b6e17adf707c76d43a15
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 10:17:57 2015 +0200

    Add @Multicast annotation to disable multicast tests by default.

    We only run multicast tests now when we explicitly state it. A working
    multicast env is required which is not always the case.

commit 2d7d2d0347179696ab41f71f048b13305014c85b
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 09:51:28 2015 +0200

    Remove extra check for local mode in InternalTestCluster

commit dda59ac39aa136d4687b9274c2692cd77f8b8f66
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 09:37:03 2015 +0200

    Handle node mode across entire test cluster

    We used static methods reading sys properties to define the node mode
    per cluster. this had lots of problems when tests couldn't cope with
    mixed or only local mode. Now we are passing it down to the cluster from the test
    which allows to @SuppressNetworkMode / @SupressLocalMode on the test to force
    consistent node configurations.

commit 058197b7a408318995c88ce7f6762e32348de0de
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 03:19:14 2015 -0400

    really ban InetSocketAddress's trappy method and break build and go to sleep, sorry

commit ac8779185aee1e17e6f5a81766290fdfc9c603ba
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 03:16:52 2015 -0400

    Ban methods that might surprisingly cause DNS lookups

commit e64fe3dff2b11503e5f2831eb9863d64f56c5538
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 02:59:05 2015 -0400

    Add unit test

commit f15434f20fb1a3691b1cc16028597d8fae937e05
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 02:39:02 2015 -0400

    fix ipv6 formatting bugs

commit 05c2c74098052c75fbb79ea1818a295ef2e03e30
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Aug 20 02:12:05 2015 -0400

    format addresses correctly so I can actually read what comes out of our logs and stats apis

commit 4f9389dcf1e8925f23153c5eb271b4ce2294dbaf
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 21:26:52 2015 -0400

    ban dangerous methods in java.net

commit 6aacd4d9925f324903d1d099a6cf5f862aeaf677
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 20:59:24 2015 -0400

    ban lenient method

commit f466a842c60163d1f4554bdce8a4163edb534c2c
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 00:29:00 2015 +0200

    fix tests to not mix local transport and zen unicast disco

commit 0de007a33b33fb68cf85cd86db4ca4f8ce10bbc9
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 00:10:07 2015 +0200

    fix tests to not mix local transport and zen unicast disco

commit 539f6ca6e5137e0d496239adc8684688dedcc824
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Aug 20 00:02:01 2015 +0200

    fix tests to not mix local transport and zen unicast disco

commit 004c2881b25467f332acc8c9f9e92b1f0f9d314e
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 17:51:45 2015 -0400

    Fix multinode

commit 54113af325ce31571811c49fdaae89d5687be4ba
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 17:36:45 2015 -0400

    fix integration tests

commit 0156a77a56319d6b9737ec6a531992052e50bd59
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 23:32:18 2015 +0200

    enable multicast in MulticastZenPingIT.java

commit 1791caa35da853ce0122485fa3fd4674c671ec6e
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 17:23:16 2015 -0400

    Fix constant

commit 22820b53e0b2dc9fd47145c2bc29ce912a8fd484
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 22:59:09 2015 +0200

    give it some extra ids for local transport crazyness

commit b2138fafa94a8a085813fd48356df63e57ade5b3
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 22:51:42 2015 +0200

    pass on local addresses from configured transport rather than hard code IP addresses

commit 1bf5de1f457b081e0ce262b57d2b55d39c434156
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Aug 19 22:04:31 2015 +0200

    fix PluggableTransportModuleIT.java to use local disco and detach port limit for node local disco

commit b6706eddfa04c43947c16551359ae98a463d34aa
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Aug 19 14:16:03 2015 -0400

    Default to unicast discovery, with default host list of 127.0.0.1, [::1]
2015-08-20 14:26:40 -04:00
Adrien Grand 41d8fbe8f5 Merge pull request #12994 from jpountz/deprecate/scan
Deprecate the `scan` search type.
2015-08-20 12:47:45 +02:00
Adrien Grand 6fa258b8fa Deprecate the `scan` search type.
This commit deprecates the `scan` search type in favour of regular scroll
requests sorted by `_doc`.

Related to #12983
2015-08-20 12:47:23 +02:00
Adrien Grand 833f821171 Merge pull request #12983 from jpountz/enhancement/scroll_doc
Optimize sorted scroll when sorting by `_doc`.
2015-08-20 12:01:04 +02:00
Adrien Grand 78c2f1063a Optimize sorted scroll when sorting by `_doc`.
This change means that we will be able to remove the `SCAN` search type in 3.0
and recommend users to use sorted scrolls instead.
2015-08-20 11:51:59 +02:00
David Pilato 449dfc8f56 Merge branch 'fix/12984-rpm-module' 2015-08-20 09:43:47 +02:00
Adrien Grand 7662705f37 Merge pull request #12997 from jpountz/fix/doc_scroll_close
Fix documentation: scrolls are not closed automatically.
2015-08-20 09:24:02 +02:00
Adrien Grand 551e92ec71 Fix documentation: scrolls are not closed automatically.
The documentation states that scrolls are automatically closed when all
documents are consumed, but this is not the case. I first tried to fix
the code to close scrolls automatically but this made REST tests fail
because clearing a scroll that is already closed returned a 4xx error
instead of a 2xx code, so this has probably been this way for a very long
time.
2015-08-20 09:20:40 +02:00
Jason Tedor a0243200e0 Fix a documentation typo and a code comment typo to path.repo 2015-08-19 22:14:50 -04:00
Jason Tedor f3c7bf0936 Fix documentation typo to path.repo for UNC path example
Closes #13008
2015-08-19 21:55:32 -04:00
Igor Motov dc82262db6 Don't show access_key and filter_key in S3 repository settings
In #11265 we added an ability to filter out sensitive repository settings. This commit uses this change to filter out access_key and filter_key in S3 repository settings.

Closes elastic/elasticsearch-cloud-aws#184
2015-08-19 16:21:42 -04:00
Robert Muir 829b858c1d Merge pull request #12995 from rmuir/dedup_resolver_for_binding
Deduplicate addresses from resolver.
2015-08-19 16:17:45 -04:00
Martijn van Groningen f2f95ea115 Removed CachedDfSource and move the dfs logic into the ContextIndexSearcher 2015-08-19 20:25:41 +02:00
Christoph Büscher 0fc96ede69 Merge pull request #12886 from cbuescher/fix/12278
Make ValueParser.DateMath aware of timezone setting
2015-08-19 18:42:19 +02:00
Ryan Ernst 6520395c1c tweaks 2015-08-19 09:37:35 -07:00
Robert Muir ce120b9c71 Deduplicate addresses from resolver.
In some cases this may contain duplicates, although its a misconfiguration,
lets not bind to multiple ports. Its no problem for us to dedup, this code
doesn't need to be huper-duper fast since its used only for logic around bind/publish
2015-08-19 12:06:54 -04:00
Britta Weber c0d31a2c9e Merge pull request #12965 from brwe/es-running-already
fail with better error message if elasticsearch was started already
2015-08-19 17:56:56 +02:00
Britta Weber 4cc6359756 fail with better error message if elasticsearch was started already
Before it failed with
.../es/qa/smoke-test-plugins/target/integ-tests/es.pid doesn't exist
which was confusing.
2015-08-19 17:56:05 +02:00
Simon Willnauer 22ae0a161f Log all interfaces we try to bind before binding 2015-08-19 17:20:48 +02:00
David Pilato dba1b52e81 [build] rpm module should be build on machine with /usr/bin/rpmbuild
We build the rpm module automatically if you have either:

* `/usr/bin/rpmbuild`
* `/usr/local/bin/rpmbuild`

available.

If your `rpmbuild` is in another location and available in your path, then run maven with `rpm` profile:

```sh
mvn deploy -Prpm
```

Closes #12984.
2015-08-19 15:37:28 +02:00
Colin Goodheart-Smithe 88d482437a Aggregations: Throw error if cardinality aggregator has a sub aggregation
The cardinality aggregation is a metric aggregation and therefore cannot accept sub-aggregations. It was previously possible to create a rest request with a cardinality aggregation that had sub-aggregations. Now such a request will throw an error in the response.

Close #12988
2015-08-19 14:31:57 +01:00
Simon Willnauer ca28acc373 Merge pull request #12986 from s1monw/fix_host_resolving
Only resolve host if explicitly allowed.
2015-08-19 15:25:55 +02:00
Simon Willnauer e5134317be Merge pull request #12936 from s1monw/add_build_hash_to_headers
Add build short hash to the download manager headers to identify staging builds
2015-08-19 15:10:46 +02:00
Simon Willnauer bd56b0a79e Only resolve host if explicitly allowed.
We have some settings that prevent host name resolution which should
be repected by InetSocketTransportAddress#getHost() to only resolve if
allowed or desired.
2015-08-19 14:54:56 +02:00
Robert Muir bbf669506b Merge pull request #12979 from rmuir/ifconfig
Log network configuration at debug level
2015-08-19 07:14:01 -04:00
Christoph Büscher 4410287f42 Merge pull request #12775 from cbuescher/fix/12715
Use 'name' from plugin descriptor file to determine plugin name
2015-08-19 12:33:34 +02:00