Commit Graph

27282 Commits

Author SHA1 Message Date
Nicholas Knize 0c4eb0a029 Add new ip_range field type
This commit adds support for indexing and searching a new ip_range field type. Both IPv4 and IPv6 formats are supported. Tests are updated and docs are added.
2017-05-05 09:43:42 -05:00
Simon Willnauer e3766d2828 Expand cross cluster search indices for search requests to the concrete index or to it's aliases (#24502)
This change will expand the shard level request to the actual concrete index or to the aliases
that expanded to the concrete index to ensure shard level requests won't see wildcard expressions as their original indices
2017-05-05 16:40:04 +02:00
Clinton Gormley 9f08a553d9 Fixed docs syntax for for-in loop in painless 2017-05-05 16:07:20 +02:00
Simon Willnauer 5bfb98ade4 [TEST] Reenable disabled tests for _field_caps and _search_shards (#24505) 2017-05-05 16:02:26 +02:00
Clinton Gormley 0174119296 Added release notes for 6.0.0-alpha1 2017-05-05 12:39:50 +02:00
Clinton Gormley c9aecbb8a5 Added removal of JavaScript and Python to breaking changes 2017-05-05 12:39:50 +02:00
Simon Willnauer 8055b14f2e Temporarily disable tests 2017-05-05 12:08:00 +02:00
Simon Willnauer 03267e03da Fix NPE if field caps request has a field that exists not in all indices (#24504)
If a field caps request contains a field name that doesn't exist in all indices
the response will be partial and we hide an NPE. The NPE is now fixed but we still
have the problem that we don't pass on errors on the shard level to the user. This will
be fixed in a followup.
2017-05-05 11:56:03 +02:00
Clinton Gormley 6e970db533 Fixed chunking of breaking changes docs 2017-05-05 11:08:55 +02:00
Simon Willnauer 6b67e0bf2f Include all aliases including non-filtering in `_search_shards` response (#24489)
`_search_shards`API today only returns aliases names if there is an alias
filter associated with one of them. Now it can be useful to see which aliases
have been expanded for an index given the index expressions. This change also includes non-filtering aliases even without a filtering alias being present.
2017-05-05 09:34:12 +02:00
Yannick Welsch c8712e9531 Limit AllocationService dependency injection hack (#24479)
Changes the scope of the AllocationService dependency injection hack so that it is at least contained to the AllocationService and does not leak into the Discovery world.
2017-05-05 08:39:18 +02:00
Jason Tedor 035494fa17 Remove obsolete JVM options from build
We start the test JVMs with various options. There are two that we
should remove, they do not make sense.
 - we require Java 8 yet there was a conditional build option for Java 7
 - we do not set MaxDirectMemorySize in our default JVM options, we
   should not in the test JVMs either

Relates #24501
2017-05-05 00:49:18 -04:00
Jason Tedor 61d5eddbd6 Fix typo in comment in IndexShardTestCase
This commit fixes a silly typo in IndexShardTestCase.java.
2017-05-04 21:04:35 -04:00
Nik Everett a01f846226 CONSOLEify a few more docs
Adds CONSOLE to cross-cluster-search docs but skips them for testing
because we don't have a second cluster set up. This gets us the
`VIEW IN CONSOLE` and `COPY AS CURL` links and makes sure that they
are valid yaml (not json, technically) but doesn't get testing.
Which is better than we had before.

Adds CONSOLE to the dynamic templates docs and ingest-node docs.
The ingest-node docs contain a *ton* of non-console snippets. We
might want to convert them to full examples later, but that can be
a separate thing.

Relates to #18160
2017-05-04 21:01:14 -04:00
Nik Everett 559bec23cc Docs: rewrite the docs/README file
I originally wrote this file when we first added snippets testing
and a lot has changed. We've grown quite fond of the
`// TESTRESPONSE[s/foo/bar/]` construct, for example, but the docs
discouraged its use.

Relates to #18160
2017-05-04 17:57:25 -04:00
Nik Everett 9f431543fc CONSOLEify inner hits docs
Rewrites most of the snippets in the `innert_hits` docs to be
complete examples and enables `VIEW IN CONSOLE`, `COPY AS CURL`,
and automatic testing of the snippets.
2017-05-04 17:30:54 -04:00
Nik Everett 9bc7e210a0 Test: Move flag to painless tests (#24494)
The `-XX:-OmitStackTraceInFastThrow` flag is only required by Painless's
tests so we'll only set it there. This is much simpler.
2017-05-04 13:11:09 -04:00
Igor Motov 6002b41b5f Add StreamInput.readEnum and StreamOutput.writeEnum (#24475)
Implements the common enum serialization/deserialization pattern for enumeration on the StreamInput/StreamOutput.
2017-05-04 12:22:54 -04:00
Clinton Gormley e9547d6a70 Rewrote the github issue template to be shorter and more likely to be read (#24486) 2017-05-04 17:41:21 +02:00
Jason Tedor de65f51d34 Simplify file store
Today we go to heroic lengths to workaround bugs in the JDK or around
issues like BSD jails to get information about the underlying file
store. For example, we went to lengths to work around a JDK bug where
the file store returned would incorrectly report whether or not a path
is writable in certain situations in Windows operating
systems. Another bug prevented getting file store information on
Windows on a virtual drive on Windows. We no longer need to work
around these bugs, we could simply try to write to disk and let an I/O
exception arise if we could not write to the disk or take advantage of
the fact that these bugs are fixed in recent releases of the JDK
(e.g., the file store bug is fixed since 8u72). Additionally, we
collected information about all file stores on the system which meant
that if the user had a stale NFS mount, Elasticsearch could hang and
fail on startup if that mount point was not available. Finally, we
collected information through Lucene about whether or not a disk was a
spinning disk versus an SSD, information that we do not need since we
assume SSDs by default. This commit takes into consideration that we
simply do not need this heroic effort, we do not need information
about all file stores, and we do not need information about whether or
not a disk spins to greatly simplfy file store handling.

Relates #24402
2017-05-04 11:19:41 -04:00
James Baiera f5edd5049a Fixing permission errors for `KERBEROS` security mode for HDFS Repository (#23439)
Added missing permissions required for authenticating with Kerberos to HDFS. Also implemented 
code to support authentication in the form of using a Kerberos keytab file. In order to support 
HDFS authentication, users must install a Kerberos keytab file on each node and transfer it to the 
configuration directory. When a user specifies a Kerberos principal in the repository settings the 
plugin automatically enables security for Hadoop and begins the login process. There will be a 
separate PR and commit for the testing infrastructure to support these changes.
2017-05-04 10:51:31 -04:00
Nik Everett 4b80e1a5ca Docs: fix list of testing images
Fedora-24 is gone. Long live Fedora-25.
2017-05-04 10:49:49 -04:00
Jason Tedor cb46e97a04 Fix reschedule async fsync test
This commit fixes the reschedule async fsync test in index service
tests. This test was passing for the wrong reason. Namely, the test was
trying to set translog durability to async, fire off an indexing
request, and then assert that the translog eventually got fsynced. The
problem here is that in the course of issuing the indexing request, a
mapping update is trigger. The mapping update triggers the index
settings to be refreshed. Since the test did not issue a cluster state
update to change the durability from request to async but instead did
this directly through index service, the mapping update flops the
durability back to async. This means that when the indexing request
executes, an fsync is performed after the request and the assertoin that
an fsync is not needed passes but for the wrong reason (in short: the
test wanted it to pass because an async fsync fired, but instead it
passed because a request async fired). This commit fixes this by going
through the index settings API so that a proper cluster state update is
triggered and so the mapping update does not flop the durability back to
request.
2017-05-04 10:41:20 -04:00
Jason Tedor 50b617f73a Remove global checkpoint assertion in index shard
Due to races, this assertion in index shard can be wrong. This commit
removes that assertion and adjusts the explanatory comment.
2017-05-04 10:33:42 -04:00
Adrien Grand 977016ba25 Do not index `_type` when there is at most one type. (#24363)
This change makes `_type` behave pretty much like `_index` when
`index.mapping.single_type` is true.
2017-05-04 16:29:35 +02:00
James Baiera d928ae210d Add Vagrant based testing fixture (#24249) 2017-05-04 10:17:55 -04:00
Jim Ferenczi 953add8e70 Add 5.4.0 to bwc versions 2017-05-04 15:57:18 +02:00
Jason Tedor 1fc777b6e3 Change logging level on reroute test
We are still chasing a test failure here and increasing the logging
level stopped the failures. We have a theory as to what is going on so
this commit reduces the logging level to hopefully trigger the failure
again and give us the logging that we need to confirm the theory.
2017-05-04 09:23:19 -04:00
Jim Ferenczi c4002e5ca4 Bump the Lucene version for ES V5_4_1 2017-05-04 14:50:05 +02:00
Simon Willnauer 8356df0846 [TEST] Add a test that alias requests are dense for all indices 2017-05-04 14:29:59 +02:00
Simon Willnauer 07f106d39c [TEST] Rollback temporarily disabled field_caps test (#24483) 2017-05-04 14:14:22 +02:00
Dimitrios Liappis ee6deb34d0 Tests: Switch to fedora-25 in test plugin
Use fedora-25 Vagrant box in VagrantTestPlugin, which was missing from
9a3ab3e800 causing packaging test
failures.

Additionally update TESTING.asciidoc
2017-05-04 14:29:35 +03:00
Jim Ferenczi 01872f3649 Add version for 5.4.1 and bwc indices for 5.4.0 2017-05-04 13:29:03 +02:00
Simon Willnauer c4eea85713 Use V_5_5_UNRELEASED constant consitently on both request and response 2017-05-04 11:52:07 +02:00
Simon Willnauer 14e57bf9f8 Add cross cluster support to `_field_caps` (#24463)
To support kibana this commit adds an internal optimization
to support the cross cluster syntax for indices on the `_field_caps`
API.

Closes #24334
2017-05-04 11:44:54 +02:00
Dimitrios Liappis 9a3ab3e800 Tests: Switch to Fedora-25 for packaging tests
Switch the Fedora Vagrant box for packaging tests to Fedora-25 as
Fedora-24 will become EOL one month after Fedora-26 is out.

Relates #24466
2017-05-04 12:04:53 +03:00
Adrien Grand d0e71510ad Enforce at most one type. (#24428)
This is a follow-up to #24317, which did the hard work but was merged in such a
way that it exposes the setting while still allowing indices to have multiple
types by default in order to give time to people who test against master to
add this setting to their index settings.
2017-05-04 10:27:50 +02:00
Ali Beyad 48031a2c5a [DOCS] Fixes the documentation on leading forward slashes in the (#24478)
[DOCS] Fixes the documentation on leading forward slashes in the
base_path of S3 repositories

Closes #23435
2017-05-03 22:58:43 -04:00
Nik Everett 45dd3780e2 CONSOLEify remaining _cat docs
Relates to #18160
2017-05-03 20:59:27 -04:00
Jason Tedor 4862544934 Increase logging in IndexRecoveryIT#rerouteTest
This test started failing but the logging here is insufficient to
discern what is happening. This commit increases the logging level on
this test until the failure can be understood.
2017-05-03 20:11:41 -04:00
Jason Tedor 2edd4d11f2 Revise issue template
This commit revises the issue template, hoping to make it clearer that
following the guidelines is a really good thing to do.

Relates #24470
2017-05-03 18:51:50 -04:00
Jason Tedor 06364cf6f0 You had one job Netty logging guard
In pre-release versions of Elasticsearch 5.0.0, users were subject to
log messages of the form "your platform does not.*reliably.*potential
system instability". This is because we disable Netty from being unsafe,
and Netty throws up this scary info-level message when unsafe is
unavailable, even if it was unavailable because the user requested that
it be unavailabe. Users were rightly confused, and concerned. So, we
contributed a guard to Netty to prevent this log message from showing up
when unsafe was explicitly disabled. This guard shipped with all
versions of Netty that shipped starting with Elasticsearch
5.0.0. Unfortunately, this guard was lost in an unrelated refactoring
and now with the 4.1.10.Final upgrade, users will again see this
message. This commit is a hack around this until we can get a fix
upstream again.

Relates #24469
2017-05-03 18:49:08 -04:00
Igor Motov cacba6bc46 Allow plugins to upgrade templates and index metadata on startup (#24379)
The UpgraderPlugin adds two additional extension points called during cluster upgrade and when old indices are introduced into the cluster state during initial recovery, restore from a snapshot or as a dangling index. One extension point allows plugin to update old templates and another extension points allows to update/check index metadata.
2017-05-03 12:51:41 -04:00
Tim Brooks 855b64b0ee Add non-dispatching listenable action future (#24412)
Currently the only implementation of `ListenableActionFuture` requires
dispatching listener execution to a thread pool. This commit renames
that variant to `DispatchingListenableActionFuture` and converts
`AbstractListenableActionFuture` to be a variant that does not require
dispatching. That class is now named `PlainListenableActionFuture`.
2017-05-03 10:30:54 -05:00
Adrien Grand 7311aaa2eb Fix PercolatorQuerySearchIT to not create multiple types. 2017-05-03 16:44:14 +02:00
Yannick Welsch 7ecb79a8e1 Remove DiscoveryNodesProvider interface (#24461)
The DiscoveryNodesProvider interface provides an unnecessary abstraction and is just used in conjunction with the existing PingContextProvider interface. This commit removes it.
2017-05-03 16:29:51 +02:00
Luca Cavanna 144f96eaeb Open/Close index api to allow_no_indices by default (#24401)
Open/Close index api have allow_no_indices set to false by default, while delete index has it set to true. The flag controls where a wildcard expression that matches no indices will be ignored or an error will be thrown instead. This commit aligns open/close default behaviour to that of delete index.
2017-05-03 16:22:26 +02:00
Jason Tedor 16af0a9ce2 Remove unnecessary field from UnicastZenPing
This field was only ever used in the constructor, where it was set and
then passed around. As such, there's no need for it to be a field and we
can remove it.
2017-05-03 09:22:24 -04:00
javanna a45e2efa00 fix typo in migrate_6_0/java.asciidoc 2017-05-03 15:00:44 +02:00
Dimitrios Liappis 79857357bf Docs: Update production notes for Docker
Add info about the base image used and the github repo of
elasticsearch-docker.

Clarify that setting `memlock=-1:-1` is only a requirement when
`bootstrap_memory_lock=true` and the alternatives we document
elsewhere in docs for disabling swap are valid for Docker as well.

Additionally, with latest versions of docker-ce shipping with
unlimited (or high enough) defaults for `nofile` and `nproc`, clarify
that explicitly setting those per ES container is not required, unless
they are not defined in the Docker daemon.

Finally simplify production `docker-compose.yml` example by removing
unneeded options.

Relates #24389
2017-05-03 14:27:31 +03:00