Commit Graph

6166 Commits

Author SHA1 Message Date
javanna 68eb58f9e3 [TEST] use randomPositiveLong where possible 2016-09-02 10:23:05 +02:00
javanna 774244a61f ThreadPool.Info and SizeValue to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
javanna 84b8c9de19 PluginInfo to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
javanna 555db744f1 use read/writeOptionalWriteable in NodeInfo serialization code 2016-09-02 10:23:05 +02:00
javanna e98e37295a PluginsAndModules to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
javanna 2b2fb8daed TransportInfo to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
javanna 536d13ff11 ProcessInfo to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
javanna 2370c25fa4 ThreadPoolInfo to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
javanna 27e7fc734c HttpInfo to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
javanna 279f8b27e3 JvmInfo to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
javanna bea863c660 OsInfo to implement Writeable rather than Streamable
This allows to make all instance members final. Also added serialization tests and sorted out inizialization that was scattered in two places.
2016-09-02 10:23:05 +02:00
javanna f6ab4e1078 ByteSizeValue to implement Writeable rather than Streamable
With this we can make ByteSizeValue immutable for real.
2016-09-02 10:23:05 +02:00
Luca Cavanna faa03ad9fa Merge pull request #20255 from javanna/enhancement/cluster_stats_available_memory
Add mem section back to cluster stats
2016-09-02 10:19:51 +02:00
Simon Willnauer 724e8ec39c [TEST] Fix settings keys to be the actual keys rather than the toString() of the Setting 2016-09-02 10:00:31 +02:00
Adrien Grand 5bfab76c96 Source filtering should keep working when the source contains numbers greater than `Long.MAX_VALUE`. #20278
Currently it does not because our parsers do not support big integers/decimals
(on purpose) but we do not have to ask our parser for the number type, we can
just ask the jackson parser for a number representation of the value with the
right type.

Note that I did not add similar tests for big decimals because Jackson seems to
never return big decimals, even for decimal values that are out of the range of
values that can be represented by doubles.

Closes #11508
2016-09-02 08:56:04 +02:00
Jason Tedor 1e80adbfbe Configure test logging with Log4j 2
This commit configures test logging for Log4j 2. The default logger
configuration uses the console appender but at the error level, so most
tests are missing logging. Instead, this commit provides a configuration
for tests which is picked up from the classpath by Log4j 2 when it
initializes. However, this now means that we can no longer initialize
Log4j with a bare-bones configuration when tests run as doing so will
prevent Log4j 2 from attempting to configure logging via the
classpath. Consequently, we move this needed initialization (as
commented, to avoid a message about a status logger not being configured
when we are preparing to configure Log4j from properties files in the
config directory) to only run when we are explicitly configuring Log4j
from properties files.

Relates #20284
2016-09-01 14:00:47 -04:00
javanna 186a5d74b8 [TEST] improve ClusterStatsIT to better check mem values returned
Rather than checking that those values are greater than 0, we can sum up the values gotten from all nodes and check that what is returned is that same value.
2016-09-01 19:22:13 +02:00
Clinton Gormley 0e8a43e826 Elasticsearch 2.4.0 uses Lucene 5.5.2 2016-09-01 12:52:01 +02:00
Martijn van Groningen a110498ad8 settings: Make `action.auto_create_index` setting a dynamic cluster setting.
Closes #7513
2016-09-01 12:33:30 +02:00
Clinton Gormley e5ff3da802 Added version 2.4.0 with bwc indices 2016-09-01 11:36:49 +02:00
javanna 042675432e make sure that mem, cpu and swap are never null in OsStats 2016-09-01 11:26:03 +02:00
javanna 5f299ff46f add mem section back to cluster stats
The mem section was buggy in cluster stats and removed. It is now added back with the same structure as in node stats, containing total memory, available memory, used memory and percentages. All the values are the sum of all the nodes across the cluster (or at least the ones that we were able to get the values from).
2016-09-01 11:26:03 +02:00
javanna 5211b6b4bc OsStats.Cpu, OsStats.Mem & OsStats.Swap to implement ToXContent 2016-09-01 11:24:56 +02:00
javanna 0a7a52a31e OsStats and subobjects to implement Writeable rather than Streamable
We can now have final instance members, also drop some optional values and related null checks that weren't needed.
2016-09-01 11:24:56 +02:00
Adrien Grand 34aaea641d Fix NPE when running a range query on a `scaled_float` with no upper bound. #20253
The null check was there, but on the wrong variable.
2016-09-01 11:23:32 +02:00
Simon Willnauer a0becd26b1 Optimize indexing for the autogenerated ID append-only case (#20211)
If elasticsearch controls the ID values as well as the documents
version we can optimize the code that adds / appends the documents
to the index. Essentially we an skip the version lookup for all
documents unless the same document is delivered more than once.

On the lucene level we can simply call IndexWriter#addDocument instead
of #updateDocument but on the Engine level we need to ensure that we deoptimize
the case once we see the same document more than once.

This is done as follows:

1. Mark every request with a timestamp. This is done once on the first node that
receives a request and is fixed for this request. This can be even the
machine local time (see why later). The important part is that retry
requests will have the same value as the original one.

2. In the engine we make sure we keep the highest seen time stamp of "retry" requests.
This is updated while the retry request has its doc id lock. Call this `maxUnsafeAutoIdTimestamp`

3. When the engine runs an "optimized" request comes, it compares it's timestamp with the
current `maxUnsafeAutoIdTimestamp` (but doesn't update it). If the the request
timestamp is higher it is safe to execute it as optimized (no retry request with the same
timestamp has been run before). If not we fall back to "non-optimzed" mode and run the request as a retry one
and update the `maxUnsafeAutoIdTimestamp` unless it's been updated already to a higher value

Relates to #19813
2016-09-01 10:39:40 +02:00
Simon Willnauer 419627c460 Ensure ESTestCase is initialized before we run tests 2016-09-01 09:39:44 +02:00
Jason Tedor d9064f454e Fix additional exception logging calls
This commit modifies a pair of exception logging calls to use
parameterized messages from Log4j.
2016-08-31 23:14:13 -04:00
Jason Tedor 76ab02e002 Merge branch 'master' into log4j2
* master:
  Avoid NPE in LoggingListener
  Randomly use Netty 3 plugin in some tests
  Skip smoke test client on JDK 9
  Revert "Don't allow XContentBuilder#writeValue(TimeValue)"
  [docs] Remove coming in 2.0.0
  Don't allow XContentBuilder#writeValue(TimeValue)
  [doc] Remove leftover from CONSOLE conversion
  Parameter improvements to Cluster Health API wait for shards (#20223)
  Add 2.4.0 to packaging tests list
  Docs: clarify scale is applied at origin+offest (#20242)
2016-08-31 16:37:55 -04:00
Jason Tedor 487ffe8375 Remove code references to logging.yml
This commit removes code references to logging.yml in TranslogToolCli
and PluginCli.
2016-08-31 15:50:45 -04:00
Nik Everett bd93c7054c Revert "Don't allow XContentBuilder#writeValue(TimeValue)"
This reverts commit 7f70c00dad.
2016-08-31 14:45:03 -04:00
Nik Everett 7f70c00dad Don't allow XContentBuilder#writeValue(TimeValue)
We have specific support for writing `TimeValue`s in the form of
`XContentBuilder#timeValueField`. Writing a `TimeValue` using
`XContentBuilder#writeValue` is a bug waiting to happen.
2016-08-31 13:23:38 -04:00
Ali Beyad 4641254ea6 Parameter improvements to Cluster Health API wait for shards (#20223)
* Params improvements to Cluster Health API wait for shards

Previously, the cluster health API used a strictly numeric value
for `wait_for_active_shards`. However, with the introduction of
ActiveShardCount and the removal of write consistency level for
replication operations, `wait_for_active_shards` is used for
write operations to represent values for ActiveShardCount. This
commit moves the cluster health API's usage of `wait_for_active_shards`
to be consistent with its usage in the write operation APIs.

This commit also changes `wait_for_relocating_shards` from a
numeric value to a simple boolean value `wait_for_no_relocating_shards`
to set whether the cluster health operation should wait for
all relocating shards to complete relocation.

* Addresses code review comments

* Don't be lenient if `wait_for_relocating_shards` is set
2016-08-31 11:58:19 -04:00
Jason Tedor e166459bbe Merge branch 'master' into log4j2
* master:
  Increase visibility of deprecation logger
  Skip transport client plugin installed on JDK 9
  Explicitly disable Netty key set replacement
  percolator: Fail indexing percolator queries containing either a has_child or has_parent query.
  Make it possible for Ingest Processors to access AnalysisRegistry
  Allow RestClient to send array-based headers
  Silence rest util tests until the bogusness can be simplified
  Remove unknown HttpContext-based test as it fails unpredictably on different JVMs
  Tests: Improve rest suite names and generated test names for docs tests
  Add support for a RestClient base path
2016-08-31 10:59:27 -04:00
Jason Tedor 1a805bb675 Increase visibility of deprecation logger
The deprecation logger is an important way to make visible features of
Elasticsearch that are deprecated. Yet, the default logging makes the
log messages for the deprecation logger invisible. We want these log
messages to be visible, so the default logging for the deprecation
logger should enable these log messages. This commit changes the log
level of deprecation log message to warn, and configures the deprecation
logger so that these log messages are visible out of the box.

Relates #20254
2016-08-31 10:51:17 -04:00
Jason Tedor ac8c2e98ab Enable console logging for CLI tools
This commit enables CLI tools to have console logging. For the CLI
tools, we skip configuring the logging infrastructure via the config
file, and instead set the level only via a system property.
2016-08-31 09:05:26 -04:00
Jason Tedor 0fdc5ca587 Remove logger getter from DeprecationLogger
This commit removes an unused getter for the logger field from the
DeprecationLogger.
2016-08-30 21:19:16 -04:00
Igor Motov a68083f5cb Make it possible for Ingest Processors to access AnalysisRegistry
The analysis registry will be used in PMML plugin ingest processor.
2016-08-30 21:09:41 -04:00
Jason Tedor abe3efdfa9 Fix failing max map count check test
This commit fixes failing max map count check test due to the use of a
logging message supplier.
2016-08-30 18:49:39 -04:00
Jason Tedor abf8a1a3f0 Avoid allocating log parameterized messages
This commit modifies the call sites that allocate a parameterized
message to use a supplier so that allocations are avoided unless the log
level is fine enough to emit the corresponding log message.
2016-08-30 18:17:09 -04:00
Jason Tedor 7da0cdec42 Introduce Log4j 2
This commit introduces Log4j 2 to the stack.
2016-08-30 13:31:24 -04:00
Nik Everett df73292256 Add an alias action to delete an index
While removing an index isn't actually an alias action, if we add
an alias action that deletes an index then we can delete and index
and add an alias with the same name as the index atomically, in
the same cluster state update.

Closes #20064
2016-08-30 10:15:21 -04:00
Simon Willnauer 497e7d1054 User lambda instead of annoymous class in SearchPhaseController 2016-08-30 12:58:54 +02:00
Tanguy Leroux b4245c7ad9 Add exclusion filters support to filter_path
This commit adds the support for exclusion filter to the response filtering (filter_path) feature. It changes the XContentBuilder APIs so that it now accepts two types of filters: inclusive and exclusive. Filters are no more String arrays but sets of String instead.
2016-08-30 09:08:30 +02:00
Martijn van Groningen 1925813e09 ingest: Fix rename processor change rename leaf fields into branch fields
Instead of get, set and remove we do get, remove and then set to avoid type conflicts in IngestDocument.
If the set still fails we try to restore the original field in ingest document.

Closes #19892
2016-08-30 07:38:01 +02:00
Ali Beyad a132405642 Ensures that during the restore process, if a file in the snapshot (#20220)
already has a file of the same name in the Store, but is different
in content (different checksum/length), then those files are first
deleted before restoring the files in question.
2016-08-29 17:51:35 -04:00
Ali Beyad 55b91cdc17 Removes unused test helper method to write old blob store format 2016-08-29 12:44:58 -04:00
Areek Zillur 99734ec576 Merge pull request #20034 from areek/cleanup/index_operation
Set created flag in index operation
2016-08-29 12:34:24 -04:00
Nik Everett 9c3f6d58ac Support downgrading keyword/text into string
This changes Elasticsearch to automatically downgrade `text` and
`keyword` fields into appropriate `string` fields when changing the
mapping of indexes imported from 2.x. This allows users to use the
modern, documented syntax against 2.x indexes. It also makes it clear
that reindexing in order to recreate the index in 5.0 is required for
any long lived indexes. This change is useful for the times when you
can't (cluster is just starting, not stable enough for reindex) or
shouldn't (index will only live 90 days or something).
2016-08-29 11:27:37 -04:00
javanna d7ec2db9b0 [TEST] enable cacheKey check in ShardSearchTransportRequestTests
Now that #20081 is merged we can check that cacheKey is consistent across equal search requests, something that wasn't true before due to ordering of map keys when using index boost.

Relates to #19986
2016-08-29 17:20:26 +02:00