Commit Graph

22641 Commits

Author SHA1 Message Date
Jim Ferenczi 439b2a96e5 Add an index setting to limit the maximum number of slices allowed in a scroll request (default to 1024). 2016-06-10 09:43:32 +02:00
Daniel Mitterdorfer 7229c91289 Remove trace logging from NettyHttpRequestSizeLimitIT
With this commit we revert back to normal behavior as the
underlying issue has been fixed with #18627.
2016-06-10 07:46:04 +02:00
Nik Everett e02d9f0945 Squash a race condition in RefreshListeners
It presented as listeners never being called if you refresh at the same
time as the listener is added. It was caught rarely by
testConcurrentRefresh. mostly this is removing code and adding a comment:

```
Note that it is not safe for us to abort early if we haven't advanced the
position here because we set and read lastRefreshedLocation outside of a
synchronized block. We do that so that waiting for a refresh that has
already passed is just a volatile read but the cost is that any check
whether or not we've advanced the position will introduce a race between
adding the listener and the position check. We could work around this by
moving this assignment into the synchronized block below and double
checking lastRefreshedLocation in addOrNotify's synchronized block but
that doesn't seem worth it given that we already skip this process early
if there aren't any listeners to iterate.
```
2016-06-09 13:48:41 -04:00
Areek Zillur 41d31541a6 Allow users to override the name for the rollover index 2016-06-09 13:43:19 -04:00
Nik Everett a0585269be [docs] s/lags/Flags/
Copy and paste lots an `F`.
2016-06-09 13:08:53 -04:00
Areek Zillur 756af81b00 change rollover rest-api placeholder name 2016-06-09 12:53:24 -04:00
Nik Everett 09cc4c449a [docs] Pattern replace char filter now support flags 2016-06-09 12:41:20 -04:00
gfyoung 6f222b5be1 Support flags in pattern replace char filter
Works just like pattern analyzer's flags param.

Closes #18362.
2016-06-09 12:39:23 -04:00
Areek Zillur a9f24ea2dc fail rollover request if rollover index already exists 2016-06-09 12:38:12 -04:00
Nik Everett fb52c258fd [test] Check if RefreshListeners was called immediately
Return a boolean from RefreshListeners, true if we called the listener
inline and false if we didn't, and check it in the test.
2016-06-09 12:08:36 -04:00
Areek Zillur 9027e8a719 renamed simulated mode to dry_run mode 2016-06-09 11:55:10 -04:00
Nik Everett bd276ef5f1 [test] Check for listener calling error
Failing to call a refresh listener is logger at WARN but that'll
cause test failure. This adds explicit assertions that there are
no errors.
2016-06-09 11:26:08 -04:00
Areek Zillur ce211119d0 Add Shrink request source parser to parse create index request body
Follow up to https://github.com/elastic/elasticsearch/pull/18732#discussion_r66407196
2016-06-09 10:41:59 -04:00
Britta Weber 053a615686 [TEST] wait for yellow before query execution
We can remove this once https://github.com/elastic/elasticsearch/pull/18759
is in.
2016-06-09 15:11:48 +02:00
Areek Zillur 94a7978ef6 add documentation 2016-06-08 18:38:02 -04:00
Robert Muir 6202b63ba7 Merge pull request #18789 from rmuir/java9_localdate_you_just_had_to_do_this_didnt_you
painless: exempt LocalDate from bridge method checks on java 9
2016-06-08 17:38:28 -04:00
Jason Tedor e9017f619e Improve performance of applyDeletedShards
This commit addresses a performance issue in
IndicesClusterStateService#applyDeletedShards. Namely, the current
implementation is O(number of indices * number of shards). This is
because of an outer loop over the indices and an inner loop over the
assigned shards, all to check if a shard is in the outer index. Instead,
we can group the shards by index, and then just do a map lookup for each
index.

Testing this on a single-node with 2500 indices, each with 2 shards,
creating an index before this optimization takes 0.90s and after this
optimization takes 0.19s.

Relates #18788
2016-06-08 16:08:00 -04:00
Christoph Wurm cd0473159a Installation and removal instructions for Ingest Plugins (#18769)
* Add install and remove instructions to ingest-geoip and ingest-attachment docs
2016-06-08 21:55:59 +02:00
Robert Muir 9826b64e1f Exempt LocalDate from bridge method checks on java 9
The painless whitelist has a lot of self-checking, in this case, it checks
for missing covariant overrides. It fails on java 9, because LocalDate.getEra()
now returns IsoEra instead of Era: https://bugs.openjdk.java.net/browse/JDK-8072746

To our checker, it thinks we were lazy with whitelisting :)

This means painless works on java 9 again
2016-06-08 15:47:09 -04:00
Simon Willnauer 9497b704bb [TEST] Fix NodeEnvironmentTests on Windows - use Path.resolve instead of platform dependent path seperator 2016-06-08 21:40:35 +02:00
Areek Zillur ae3eb15caa fix rest tests 2016-06-08 15:03:35 -04:00
Areek Zillur 0c6d19c40c add body support for create index request 2016-06-08 14:16:06 -04:00
Nik Everett 06d26635de Packaging tests: ingest-grok is now ingest-common 2016-06-08 14:03:48 -04:00
Nik Everett bd1af34506 [reindex] Extract runnable to inner class
Makes it more readable
2016-06-08 14:00:15 -04:00
Nik Everett 2437313e4e Remove extra logging
The test shouldn't be failing any more.
2016-06-08 13:52:43 -04:00
Nik Everett 5b94c4a25b Fix a race condition in reindex's rethrottle
If you rethrottled the request while is was performing a scroll
request then it wouldn't properly pick up the rethrottle for that
batch. This was causing test failure and might cause issues for
users. The work around is simple though: just issue the rethrottle
again with a slightly faster throttle than the first time.

Caught by:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-os-compatibility/os=centos/525/console
2016-06-08 13:52:43 -04:00
Nik Everett 4b21157906 Remove setRefresh
It has been replaced with `setRefreshPolicy` which has support for
waiting until refresh with `setRefreshPolicy(WAIT_FOR)`.

Related to #1063
2016-06-08 13:50:59 -04:00
Lee Hinman 92349f70e2 Merge remote-tracking branch 'dakrone/igs-false2' 2016-06-08 10:49:20 -06:00
Lee Hinman c637fea84b Change the default of `include_global_state` from true to false for restores
This changes the default value to be false *only* for restore operations.

Resolves #18569
2016-06-08 10:48:36 -06:00
Martijn van Groningen 3dd3ed4905 ingest: Upgrade geoip processor's dependencies and database files
The database files have been doubled in size compared to the previous files being used.
For this reason the database files are now gzip compressed, which required using
`GZIPInputStream` when loading database files.
2016-06-08 18:41:48 +02:00
Nik Everett 5161afe5e3 Support optional ctor args in ConstructingObjectParser
You declare them like
```
static {
  PARSER.declareInt(optionalConstructorArg(), new ParseField("animal"));
}
```

Other than being optional they follow all of the rules of regular
`constructorArg()`s. Parsing an object with optional constructor args
is going to be slightly less efficient than parsing an object with
all required args if some of the optional args aren't specified because
ConstructingObjectParser isn't able to build the target before the
end of the json object.
2016-06-08 12:38:40 -04:00
Areek Zillur dec0dcc30b minor cleanup 2016-06-08 11:33:55 -04:00
Simon Willnauer bec26015b2 [TEST] add a dedicated test for empty files 2016-06-08 15:40:14 +02:00
Christoph Büscher a2372778dd Fix problem with TimeIntervalRounding on DST end
Due to an error in our current TimeIntervalRounding, two dates can
round to the same key, even when they are 1h apart when using
short interval roundings (e.g. 20m) and a time zone with DST change.

Here is an example for the CET time zone:

On 25 October 2015, 03:00:00 clocks are turned backward 1 hour to
02:00:00 local standard time. The dates
"2015-10-25T02:15:00+02:00" (1445732100000) (before DST end) and
"2015-10-25T02:15:00+01:00" (1445735700000) (after DST end)
are thus 1h apart, but currently they round to the same value
"2015-10-25T02:00:00.000+01:00" (1445734800000).

This violates an important invariant of rounding, namely that the
rounded value must be less or equal to the value that is rounded.
It also leads to wrong histogram bucket counts because documents in
[02:00:00+02:00, 02:20:00+02:00) go to the same bucket as documents
from [02:00:00+01:00, 02:20:00+01:00).

The problem happens because in TimeIntervalRounding#roundKey() we
need to perform the rounding operation in local time, but on
converting back to UTC we don't honor the original values time zone
offset. This fix changes that and adds tests both for DST start and
DST end as well as a test that demonstrates what happens to bucket
sizes when the dst change is not evently divisibly by the interval.
2016-06-08 13:05:52 +02:00
Jim Ferenczi 712c77264d Fix ut: make sure that the number of slices is bigger than 1 in the SliceBuilder tests. 2016-06-08 11:51:46 +02:00
Areek Zillur 134a4e5e52 incorporate feedback 2016-06-07 22:38:47 -04:00
Areek Zillur 76c59b7417 remove unused request param 2016-06-07 22:37:10 -04:00
Lee Hinman 762bbdbd0c Revert "Change the default of `include_global_state` from true to false."
This reverts commit 052a62250c.
2016-06-07 15:07:37 -06:00
Lee Hinman 052a62250c Change the default of `include_global_state` from true to false.
Resolves #18569
2016-06-07 15:06:20 -06:00
Jack Conradson be168f528b Merge pull request #18771 from jdconrad/methods
Add Method to Get New MethodWriters
2016-06-07 11:47:29 -07:00
Nik Everett a405c2ba99 Switch QueryBuilders to new MatchPhraseQueryBuilder
It was doing deprecated things with MatchQueryBuilder.
2016-06-07 14:35:23 -04:00
Jack Conradson f9b45107c4 Added method to allow creation of new methods on-the-fly. 2016-06-07 11:24:33 -07:00
Robert Muir 90f2aab9a9 Merge pull request #18748 from rmuir/painless_method_references
painless: method reference support
2016-06-07 14:22:49 -04:00
Robert Muir f23e8c3722 more cleanups 2016-06-07 12:42:19 -04:00
Lee Hinman 32bd869b28 Merge remote-tracking branch 'dakrone/no-cluster-name-in-path' 2016-06-07 10:14:23 -06:00
Lee Hinman feb244c14a Remove cluster name from data path
Previously Elasticsearch used $DATA_DIR/$CLUSTER_NAME/nodes for the path
where data is stored, this commit changes that to be $DATA_DIR/nodes.

On startup, if the old folder structure is detected it will be used.
This behavior will be removed in Elasticsearch 6.0

Resolves #17810
2016-06-07 10:13:48 -06:00
Jason Tedor 370b198f9f Remove explicit parallel new GC flag
Unless explicitly disabled, the parallel new collector is enabled
automatically as soon as the CMS collector is enabled:

  void Arguments::set_cms_and_parnew_gc_flags() {
    assert(
      !UseSerialGC && !UseParallelOldGC && !UseParallelGC,
      "Error");
    assert(UseConcMarkSweepGC, "CMS is expected to be on here");

    // If we are using CMS, we prefer to UseParNewGC,
    // unless explicitly forbidden.
    if (FLAG_IS_DEFAULT(UseParNewGC)) {
      FLAG_SET_ERGO(bool, UseParNewGC, true);
    }

While it's fine to be explicit, the UseParNewGC flag is deprecatd in JDK
8 and produces warning messages in JDK 9:

  Java HotSpot(TM) 64-Bit Server VM warning: Option UseParNewGC was
  deprecated in version 9.0 and will likely be removed in a future
  release.

Thus, we can and should just remove this flag from the default JVM
options.

Relates #18767
2016-06-07 12:05:38 -04:00
Jim Ferenczi 43b419b230 rehash the docvalues in DocValuesSliceQuery using BitMixer.mix instead of the naive Long.hashCode. 2016-06-07 17:58:32 +02:00
Robert Muir 89224dbc3c switch FunctionRef over to methodhandles 2016-06-07 11:56:44 -04:00
Martijn van Groningen f611f1c99e ingest: Move processors from core to ingest-common module.
Folded grok processor into ingest-common module.

The rest tests have been moved to ingest-common module as well, because these tests don't run in the rest-api-spec module but in the distribution:integ-test-zip module
and adding a test plugin there felt just wrong to me. I think this is ok. I left a tiny ingest rest test behind in that tests with an empty pipeline.

Removed messy tests, these tests were already covered in the rest tests

Added ingest test plugin in test infra so that each module testing integration with ingest doesn't need write its own plugin

Moved reindex ingest tests to qa module

Closes #18490
2016-06-07 17:32:52 +02:00