Commit Graph

27292 Commits

Author SHA1 Message Date
Hendrik Muhs f41ddb3607 Move MockLogAppender to elasticsearch test (#24542)
In order to make MockLogAppender (utility to test logging) available outside
of es-core move MockLogAppender from test core-tests to test framework. As
package names do not change, no need to change clients.
2017-05-08 13:02:27 +02:00
Jason Tedor bbdaf113d4 Remove gap skipping when opening engine
Today when opening the engine we skip gaps in the history, advancing the
local checkpoint until it is equal to the maximum sequence number
contained in the commit. This allows history to advance, but it leaves
gaps. A previous change filled these gaps when recovering from store,
but since we were skipping the gaps while opening the engine, this
change had no effect. This commit removes the gap skipping when opening
the engine allowing the gap filling to do its job.

Relates #24535
2017-05-08 06:38:28 -04:00
glefloch 59dd4d288a documentation of preserve existing settings 2017-05-08 12:05:01 +02:00
javanna e875f7f72e remove duplicated import in AppendProcessor 2017-05-08 10:36:36 +02:00
Jason Tedor 3e485c2ca5 Fix plugin installation permissions
When installing plugin permissions, we try to set the permissions on all
installed files ourselves because a umask from the user could violate
everything needed to get the permissions right. Sadly, directories were
not handled correctly at all and so we were still left with broken
installations with umasks like 0077. This commit fixes this issue, adds
a thorough unit test for the situation, and most importantly, adds a
test that sets the umask before installing the plugin.

Relates #24527
2017-05-06 13:25:10 -04:00
Koen De Groote 13c17c75b5 Remove unneeded empty string concatentation
This commit removes concatenation by empty string in places where it
is simply not needed to obtain a string representation.

Relates #24411
2017-05-06 00:28:53 -04:00
Jason Tedor 4027c9da7b TimeValue#parseTimeValue author is bad, feels bad
I stumbled on this code today and I hated it; I wrote it. I did not like
that you could not tell at a glance whether or not the method parameters
were correct. This commit fixes it.

Relates #24522
2017-05-05 18:37:09 -04:00
Jason Tedor 4e5537d9ed Add workaround so path.data can be set in run task
Previously you could set the system property tests.es.path.data and
start the run task with a custom data directory. A change in core to
detect duplicate settings broke this. That change should stay, but we
can work around this easily by only setting path.data to the data
directory if tests.es.path.data is not set.
2017-05-05 15:57:54 -04:00
Tanguy Leroux 7bd2abe48a Change Terms.Bucket to an interface (#24492)
This commit changes the Terms.Bucket abstract class to an interface, so
that it's easier for the Java High Level Rest Client to provide its own
implementation.

In its current state, the Terms.Bucket abstract class inherits from
InternalMultiBucketAggregation.InternalBucket which forces subclasses to
implement Writeable and exposes a public getProperty() method that relies
on InternalAggregation. This two points make it difficult for the Java
High Level Rest Client to implement the Terms and Terms.Bucket correctly.
This is also different from other MultiBucketsAggregation like Range
which are pure interfaces.

Changing Terms.Bucket to an interface causes a method clashes for the
`getBuckets()` method in InternalTerms. This is because:
 - InternalTerms implements Terms which declared a
 `List<Terms.Bucket> getBuckets()` method
 - InternalTerms extends InternalMultiBucketAggregation which declares a
 `List<? extends InternalBucket> getBuckets()` method
 - both overrides the MultiBucketsAggregation
 `List<? extends Bucket> getBuckets()` method

 There was no clashes before this change because Terms.Bucket extends
 InternalBucket and conformed to both declaration. With Terms.Bucket now
 an interface, the getBuckets() method in the Terms interface is changed
 to avoid method clash. This is a breaking change in the Java API but
 it's a straightforward change and the Terms multi bucket aggregation
 interface is also more coherent with the other Range, Histogram,
 Filters, AdjacencyMatrix etc that all return a `List<? extends Bucket>`.
2017-05-05 20:00:39 +02:00
Anupam 0b36fb052c Update completion-suggest.asciidoc (#24506) 2017-05-05 11:34:41 -04:00
Nicholas Knize 0c4eb0a029 Add new ip_range field type
This commit adds support for indexing and searching a new ip_range field type. Both IPv4 and IPv6 formats are supported. Tests are updated and docs are added.
2017-05-05 09:43:42 -05:00
Simon Willnauer e3766d2828 Expand cross cluster search indices for search requests to the concrete index or to it's aliases (#24502)
This change will expand the shard level request to the actual concrete index or to the aliases
that expanded to the concrete index to ensure shard level requests won't see wildcard expressions as their original indices
2017-05-05 16:40:04 +02:00
Clinton Gormley 9f08a553d9 Fixed docs syntax for for-in loop in painless 2017-05-05 16:07:20 +02:00
Simon Willnauer 5bfb98ade4 [TEST] Reenable disabled tests for _field_caps and _search_shards (#24505) 2017-05-05 16:02:26 +02:00
Clinton Gormley 0174119296 Added release notes for 6.0.0-alpha1 2017-05-05 12:39:50 +02:00
Clinton Gormley c9aecbb8a5 Added removal of JavaScript and Python to breaking changes 2017-05-05 12:39:50 +02:00
Simon Willnauer 8055b14f2e Temporarily disable tests 2017-05-05 12:08:00 +02:00
Simon Willnauer 03267e03da Fix NPE if field caps request has a field that exists not in all indices (#24504)
If a field caps request contains a field name that doesn't exist in all indices
the response will be partial and we hide an NPE. The NPE is now fixed but we still
have the problem that we don't pass on errors on the shard level to the user. This will
be fixed in a followup.
2017-05-05 11:56:03 +02:00
Clinton Gormley 6e970db533 Fixed chunking of breaking changes docs 2017-05-05 11:08:55 +02:00
Simon Willnauer 6b67e0bf2f Include all aliases including non-filtering in `_search_shards` response (#24489)
`_search_shards`API today only returns aliases names if there is an alias
filter associated with one of them. Now it can be useful to see which aliases
have been expanded for an index given the index expressions. This change also includes non-filtering aliases even without a filtering alias being present.
2017-05-05 09:34:12 +02:00
Yannick Welsch c8712e9531 Limit AllocationService dependency injection hack (#24479)
Changes the scope of the AllocationService dependency injection hack so that it is at least contained to the AllocationService and does not leak into the Discovery world.
2017-05-05 08:39:18 +02:00
Jason Tedor 035494fa17 Remove obsolete JVM options from build
We start the test JVMs with various options. There are two that we
should remove, they do not make sense.
 - we require Java 8 yet there was a conditional build option for Java 7
 - we do not set MaxDirectMemorySize in our default JVM options, we
   should not in the test JVMs either

Relates #24501
2017-05-05 00:49:18 -04:00
Jason Tedor 61d5eddbd6 Fix typo in comment in IndexShardTestCase
This commit fixes a silly typo in IndexShardTestCase.java.
2017-05-04 21:04:35 -04:00
Nik Everett a01f846226 CONSOLEify a few more docs
Adds CONSOLE to cross-cluster-search docs but skips them for testing
because we don't have a second cluster set up. This gets us the
`VIEW IN CONSOLE` and `COPY AS CURL` links and makes sure that they
are valid yaml (not json, technically) but doesn't get testing.
Which is better than we had before.

Adds CONSOLE to the dynamic templates docs and ingest-node docs.
The ingest-node docs contain a *ton* of non-console snippets. We
might want to convert them to full examples later, but that can be
a separate thing.

Relates to #18160
2017-05-04 21:01:14 -04:00
Nik Everett 559bec23cc Docs: rewrite the docs/README file
I originally wrote this file when we first added snippets testing
and a lot has changed. We've grown quite fond of the
`// TESTRESPONSE[s/foo/bar/]` construct, for example, but the docs
discouraged its use.

Relates to #18160
2017-05-04 17:57:25 -04:00
Nik Everett 9f431543fc CONSOLEify inner hits docs
Rewrites most of the snippets in the `innert_hits` docs to be
complete examples and enables `VIEW IN CONSOLE`, `COPY AS CURL`,
and automatic testing of the snippets.
2017-05-04 17:30:54 -04:00
Nik Everett 9bc7e210a0 Test: Move flag to painless tests (#24494)
The `-XX:-OmitStackTraceInFastThrow` flag is only required by Painless's
tests so we'll only set it there. This is much simpler.
2017-05-04 13:11:09 -04:00
Igor Motov 6002b41b5f Add StreamInput.readEnum and StreamOutput.writeEnum (#24475)
Implements the common enum serialization/deserialization pattern for enumeration on the StreamInput/StreamOutput.
2017-05-04 12:22:54 -04:00
Clinton Gormley e9547d6a70 Rewrote the github issue template to be shorter and more likely to be read (#24486) 2017-05-04 17:41:21 +02:00
Jason Tedor de65f51d34 Simplify file store
Today we go to heroic lengths to workaround bugs in the JDK or around
issues like BSD jails to get information about the underlying file
store. For example, we went to lengths to work around a JDK bug where
the file store returned would incorrectly report whether or not a path
is writable in certain situations in Windows operating
systems. Another bug prevented getting file store information on
Windows on a virtual drive on Windows. We no longer need to work
around these bugs, we could simply try to write to disk and let an I/O
exception arise if we could not write to the disk or take advantage of
the fact that these bugs are fixed in recent releases of the JDK
(e.g., the file store bug is fixed since 8u72). Additionally, we
collected information about all file stores on the system which meant
that if the user had a stale NFS mount, Elasticsearch could hang and
fail on startup if that mount point was not available. Finally, we
collected information through Lucene about whether or not a disk was a
spinning disk versus an SSD, information that we do not need since we
assume SSDs by default. This commit takes into consideration that we
simply do not need this heroic effort, we do not need information
about all file stores, and we do not need information about whether or
not a disk spins to greatly simplfy file store handling.

Relates #24402
2017-05-04 11:19:41 -04:00
James Baiera f5edd5049a Fixing permission errors for `KERBEROS` security mode for HDFS Repository (#23439)
Added missing permissions required for authenticating with Kerberos to HDFS. Also implemented 
code to support authentication in the form of using a Kerberos keytab file. In order to support 
HDFS authentication, users must install a Kerberos keytab file on each node and transfer it to the 
configuration directory. When a user specifies a Kerberos principal in the repository settings the 
plugin automatically enables security for Hadoop and begins the login process. There will be a 
separate PR and commit for the testing infrastructure to support these changes.
2017-05-04 10:51:31 -04:00
Nik Everett 4b80e1a5ca Docs: fix list of testing images
Fedora-24 is gone. Long live Fedora-25.
2017-05-04 10:49:49 -04:00
Jason Tedor cb46e97a04 Fix reschedule async fsync test
This commit fixes the reschedule async fsync test in index service
tests. This test was passing for the wrong reason. Namely, the test was
trying to set translog durability to async, fire off an indexing
request, and then assert that the translog eventually got fsynced. The
problem here is that in the course of issuing the indexing request, a
mapping update is trigger. The mapping update triggers the index
settings to be refreshed. Since the test did not issue a cluster state
update to change the durability from request to async but instead did
this directly through index service, the mapping update flops the
durability back to async. This means that when the indexing request
executes, an fsync is performed after the request and the assertoin that
an fsync is not needed passes but for the wrong reason (in short: the
test wanted it to pass because an async fsync fired, but instead it
passed because a request async fired). This commit fixes this by going
through the index settings API so that a proper cluster state update is
triggered and so the mapping update does not flop the durability back to
request.
2017-05-04 10:41:20 -04:00
Jason Tedor 50b617f73a Remove global checkpoint assertion in index shard
Due to races, this assertion in index shard can be wrong. This commit
removes that assertion and adjusts the explanatory comment.
2017-05-04 10:33:42 -04:00
Adrien Grand 977016ba25 Do not index `_type` when there is at most one type. (#24363)
This change makes `_type` behave pretty much like `_index` when
`index.mapping.single_type` is true.
2017-05-04 16:29:35 +02:00
James Baiera d928ae210d Add Vagrant based testing fixture (#24249) 2017-05-04 10:17:55 -04:00
Jim Ferenczi 953add8e70 Add 5.4.0 to bwc versions 2017-05-04 15:57:18 +02:00
Jason Tedor 1fc777b6e3 Change logging level on reroute test
We are still chasing a test failure here and increasing the logging
level stopped the failures. We have a theory as to what is going on so
this commit reduces the logging level to hopefully trigger the failure
again and give us the logging that we need to confirm the theory.
2017-05-04 09:23:19 -04:00
Jim Ferenczi c4002e5ca4 Bump the Lucene version for ES V5_4_1 2017-05-04 14:50:05 +02:00
Simon Willnauer 8356df0846 [TEST] Add a test that alias requests are dense for all indices 2017-05-04 14:29:59 +02:00
Simon Willnauer 07f106d39c [TEST] Rollback temporarily disabled field_caps test (#24483) 2017-05-04 14:14:22 +02:00
Dimitrios Liappis ee6deb34d0 Tests: Switch to fedora-25 in test plugin
Use fedora-25 Vagrant box in VagrantTestPlugin, which was missing from
9a3ab3e800 causing packaging test
failures.

Additionally update TESTING.asciidoc
2017-05-04 14:29:35 +03:00
Jim Ferenczi 01872f3649 Add version for 5.4.1 and bwc indices for 5.4.0 2017-05-04 13:29:03 +02:00
Simon Willnauer c4eea85713 Use V_5_5_UNRELEASED constant consitently on both request and response 2017-05-04 11:52:07 +02:00
Simon Willnauer 14e57bf9f8 Add cross cluster support to `_field_caps` (#24463)
To support kibana this commit adds an internal optimization
to support the cross cluster syntax for indices on the `_field_caps`
API.

Closes #24334
2017-05-04 11:44:54 +02:00
Dimitrios Liappis 9a3ab3e800 Tests: Switch to Fedora-25 for packaging tests
Switch the Fedora Vagrant box for packaging tests to Fedora-25 as
Fedora-24 will become EOL one month after Fedora-26 is out.

Relates #24466
2017-05-04 12:04:53 +03:00
Adrien Grand d0e71510ad Enforce at most one type. (#24428)
This is a follow-up to #24317, which did the hard work but was merged in such a
way that it exposes the setting while still allowing indices to have multiple
types by default in order to give time to people who test against master to
add this setting to their index settings.
2017-05-04 10:27:50 +02:00
Ali Beyad 48031a2c5a [DOCS] Fixes the documentation on leading forward slashes in the (#24478)
[DOCS] Fixes the documentation on leading forward slashes in the
base_path of S3 repositories

Closes #23435
2017-05-03 22:58:43 -04:00
Nik Everett 45dd3780e2 CONSOLEify remaining _cat docs
Relates to #18160
2017-05-03 20:59:27 -04:00
Jason Tedor 4862544934 Increase logging in IndexRecoveryIT#rerouteTest
This test started failing but the logging here is insufficient to
discern what is happening. This commit increases the logging level on
this test until the failure can be understood.
2017-05-03 20:11:41 -04:00