Commit Graph

138 Commits

Author SHA1 Message Date
Boaz Leskes 1df2d3015e Add OS name to _nodes and _cluster/nodes
we currently don't expose this.

This adds the following to the OS section of `_nodes`:

```
"os": {
     "name": "Mac OS X",
     ...
 }
 ```

 and the following to the OS section of `_cluster/stats`:

 ```
      "os": {
         ...
          "names": [
             {
                "name": "Mac OS X",
                "count": 1
             }
          ],
          ...
       },
 ```

Closes #11807
2015-06-22 20:36:29 +02:00
Ryan Ernst 12e7cbe92b Mappings: Lockdown _timestamp
This is a follow up to #8143 and #6730 for _timestamp. It removes
support for `path`, as well as any field type settings, and
enables docvalues for _timestamp, for 2.0.  Users who need to
adjust these settings can use a date field.
2015-06-22 10:21:03 -07:00
Zachary Tong 5d94febcb1 Aggregations: Moving average forecasts should not include current datapoint.
- Fixes tests, and removes a few special snowflake, fragile tests.
- Removes concrete implementation of predict() and moves it into
  each model so that the logic is clearer.  Because there is some
  shared checks/assertions, those remain in predict() and the main
  prediction happens in doPredict()
2015-06-22 11:13:35 -04:00
Alexander Reelsen eb23530e06 Internal: Readd accidental removed functionality in InternalClusterService
The commit about adding cluster health response features also removed
accidentally some functionality, that resulted in wrong instanceof checks
in InternalClusterService and thus in test failures because the cluster
state task that was added via an anonymous was missing the cast.

This commit readds the abstract class with slight renaming.

Commit id was: 88f8d58c8b
2015-06-22 16:34:02 +02:00
Simon Willnauer 1f3670733a Load percolator queries before shard is marked POST_RECOVERY
If we mark the shard as being in POST_RECOVERY before the percolator
is fully set up we might expose it to the user as fully searchable before
all queries are loaded. This can lead to wrong results especially in tests
when a shard is concurrently marked as STARTED.

This commit also removes unneded abstractions on IndexShard where readoperations
should be allowed when the purose is a write.
2015-06-22 16:23:27 +02:00
Shay Banon 55c55677a3 Remove reroute with no reassign
Its not used in our codebase anymore, so no need for it
closes #11804
2015-06-22 15:53:56 +02:00
Alexander Reelsen 88f8d58c8b Cluster Health: Add max wait time for pending task and active shard percentage
In order to get a quick overview using by simply checking the cluster state
and its corresponding cat API, the following two attributes have been added
to the cluster health response:

* task max waiting time, the time value of the first task of the
  queue and how long it has been waiting
* active shards percent: The percentage of the number of shards that are in
  initializing state

This makes the cluster health API handy to check, when a fully restarted
cluster is back up and running.

Closes #10805
2015-06-22 15:04:47 +02:00
Colin Goodheart-Smithe 772d0cc6e7 Build: Make rest-spec-api a project so eclipse build works
The change makes rest-spec-api a project in the same way as we build dev-tools. it packages the tests and api in a bundle using the maven-remote-resources-plugin and uses the same plugin in the plugins and core pom to unpack the rest-api-spec into the target directory and references the rest tests there in the test resources.

The main stimulus for this change is that for those using Eclipse the current build does not work. After running `mvn eclipse:eclipse` the Eclipse IDE errors because the rest-api-spec is outside of the project scope, meaning that every time the command is run (required whenever any dependencies change), the class path of all the projects has to be manually fixed.
2015-06-22 11:41:44 +01:00
Simon Willnauer ec6946e979 [TEST] Enable testcase since #11482 is merged 2015-06-22 12:19:03 +02:00
Alexander Reelsen 38ddc8159c Dates: Allow for negative unix timestamps
This fixes an issue to allow for negative unix timestamps.
An own printer for epochs instead of just having a parser has been added.
Added docs that only 10/13 length unix timestamps are supported
Added docs in upgrade documentation

Fixes #11478
2015-06-22 11:56:31 +02:00
Simon Willnauer 895a7dd03c [TEST] assume assertions are enabled 2015-06-22 11:25:02 +02:00
Simon Willnauer af4b7ee000 Merge pull request #11795 from s1monw/abstract_runnable_ping
Use abstract runnable in scheduled ping
2015-06-22 10:27:19 +02:00
Simon Willnauer c7285d9d19 [TEST] Remove sleeps and latch timeouts from ClusterServiceTests
Tests relying on sleeps and latch timeouts are prone to weird timing issues
and hard to read / understand error messages. This commit moves towards a more
deterministic error model and replaces empty fails with real exceptions.
2015-06-22 09:51:51 +02:00
Simon Willnauer a45c05d907 Use abstract runnable in scheduled ping 2015-06-22 09:40:06 +02:00
Simon Willnauer 2e762f078d fix typo 2015-06-21 13:50:04 +02:00
Ryan Ernst 4c869cdf61 quite noisy test failure, related to #11769 2015-06-19 22:30:55 -07:00
Igor Motov 576b825d1c Improve logging of repository verification exceptions.
Some repository verification exceptions are currently only returned to the users but not logged on the nodes where the exceptions occurred, which makes troubleshooting difficult.

Closes #11760
2015-06-19 22:09:03 -04:00
Ryan Ernst aa4369f55b fix parent field mapper to copy customFieldDataSettings on merge 2015-06-19 16:48:34 -07:00
Ryan Ernst 0a5e763697 Removed hardcoded seed from test 2015-06-19 16:08:51 -07:00
Ryan Ernst de2dea04fb Rename MappedFieldType.validateCompatible to checkCompatibility 2015-06-19 15:59:30 -07:00
Ryan Ernst fa8a300f71 Merge pull request #11783 from rjernst/refactor/field-type-merge
Move merge simulation of fieldtype settings to fieldtype method
2015-06-19 15:45:01 -07:00
Ryan Ernst a71b128738 Address PR comments and fix customFieldDataSettings to still be copied
on merge
2015-06-19 15:41:36 -07:00
Ryan Ernst 434b1c94b3 Mappings: Move merge simulation of fieldtype settings to fieldtype method.
For #8871, we need to be able to check field types are compatible,
without comparing FieldMappers.  This change moves the simulation
checks (which generate merge conflicts) for any properties of
MappedFieldTypes into a new method, validateCompatible.

This also simplifies the merge code which merges settings
between the old and new fieldtypes. Previously, each subclass
of FieldMapper would have to set its own fieldtype settings.
However, now that we have .clone(), which perfectly copies
all properties (with subclasses accounted for), we can now
do a simple clone when merging.

Finally, this fixes a subtle bug in merging, in which if
merging has conflicts, and we were not simulating, we would
still update the field type, even though it was not compatible!

NOTE: there is one test failure I am trying to track down with
timestamp merging. Otherwise, all tests pass.
2015-06-19 13:42:16 -07:00
Areek Zillur a8c2886b3f Internal: Mark store as corrupted instead of deleting state file on engine failure
Currently, we delete the shard _state file on engine failure.
This behaviour does not persist the engine failure reason for later inspection.

This commit marks the shard store as corrupted instead of deleting
the _state file to ensure the store index can not be opened after and
the engine failure is persisted.
2015-06-19 14:46:26 -04:00
Alex Ksikes e26b1763e0 CommonTermsQuery fix for ignored coordination factor
CommonTermsQueryParser does not check for disable_coords, only for
disable_coord. Yet the builder only outputs disable_coords, leading to
disabling the coordination factor to be ignored in the Java API.

Closes #11730
Closes #11780
2015-06-19 10:15:37 -05:00
Ryan Ernst 04c3095f5e Merge pull request #11770 from rjernst/fix/null_value_merge
Hide more fieldType access and cleanup null_value merging
2015-06-19 08:15:33 -07:00
Zachary Tong ae742c4a03 Aggregations: moving_avg model parser should accept any numeric, not just doubles
Also changes the models to throw ParseExceptions instead of SearchParseExceptions, so that
the validation can be unit-tested.

Fixes #11487
2015-06-19 10:36:15 -04:00
Iarwa1n ef8ee12259 Fix exception message in RestRequest
Closes #11305
2015-06-19 11:06:23 +01:00
Ryan Ernst caa40df328 Mappings: Hide more fieldType access and cleanup null_value merging
There were some missed uses of AbstractFieldMapper.fieldType in #11764.
This change also moves null_value merging into AbstractFieldMapper.
2015-06-18 15:58:30 -07:00
Ryan Ernst bfc4b8dd10 Merge pull request #11764 from rjernst/fix/field-type-ref
Replace fieldType access in mappers with getter
2015-06-18 13:01:07 -07:00
Ryan Ernst f09e0d7fb4 Mappings: Replace fieldType access in mappers with getter
A small refactoring to allow changing how the field type is stored later.
2015-06-18 11:47:38 -07:00
Shay Banon 341e980c52 [TEST] Use the correct renamed setting
and make the default value setting private
2015-06-18 20:40:07 +02:00
Simon Willnauer 1b2a3d0af6 Add @Repeat to forbidden APIs
@Repeat should not be committed just like @Seed.
Use -Pdev to run annotated methods.
2015-06-18 20:34:02 +02:00
Shay Banon 6d2ca116b9 [TEST] assertBusy on hasUnassigned
on fast machines, node leave might not move shards to unassigned right away, wait for it
2015-06-18 20:30:25 +02:00
Shay Banon 256f3d21ce Reset registeredNextDelaySetting on reroute
Need to reset the registered setting in order to make sure the nex round will capture the right delay interval

also randomize setting and name the setting properly

closes #11759
2015-06-18 20:20:49 +02:00
Jack Conradson 917aeb7278 Scripting: Allow executable expression scripts for aggregations
Added several classes to support expressions being used for numerical
calculations in aggregations.  Expressions will still not compile
when used with mapping and update script contexts.

Closes #11596
Closes #11689
2015-06-18 10:57:38 -07:00
Simon Willnauer 87946ba3b2 Allow security rule for advanced SSL configutation
if plugins need to install SSL factories etc. we have to allow
to `setFactory` in the security policy.
2015-06-18 16:50:35 +02:00
Shay Banon 792a545633 Optional Delayed Allocation on Node leave
Allow to set delayed allocation timeout on unassigned shards when a node leaves the cluster. This allows to wait for the node to come back for a specific period in order to try and assign the shards back to it to reduce shards movements and unnecessary relocations.

The setting is an index level setting under `index.unassigned.node_left.delayed_timeout` and defaults to 0 (== no delayed allocation). We might want to change the default, but lets do it in a different change to come up with the best value for it. The setting can be updated dynamically.

When shards are delayed, a log message with "info" level will notify how many shards are being delayed.

An implementation note, we really only need to care about delaying allocation on unassigned replica shards. If the primary shard is unassigned, anyhow we are going to wait for a copy of it, so really the only case to delay allocation is for replicas.

close #11712
2015-06-18 16:06:02 +02:00
Simon Willnauer 30cc984565 [TEST] fix TermsAggregationSearchBenchmark 2015-06-18 15:06:49 +02:00
Simon Willnauer 5808102d19 Upgrade AWS dependency to 1.10.0
This also apgrades JodaTime to 2.8.0.
2015-06-18 14:04:38 +02:00
Simon Willnauer 85dccdb8ab Use UTC as timeszone for index creation date 2015-06-18 11:15:40 +02:00
Simon Willnauer 2a63249441 Add DateTime ctors without timezone to forbidden APIs
Using DateTime with default timezone is asking for trouble and should
be added to forbidden APIs
2015-06-18 10:43:45 +02:00
Isabel Drost-Fromm 2fa56899d6 Merge pull request #11696 from MaineC/feature/simple-query-string-addboost
Add support for query boost to SimpleQueryStringBuilder.

Relates to #10217
2015-06-18 10:10:57 +02:00
Shay Banon cc97a4159e remove 1.7 version check 2015-06-18 09:34:39 +02:00
David Pilato 6a466b9a00 Replace Iterators#emptyIterator by JDK one
`com.google.common.collect.Iterators#emptyIterator()` is marked as deprecated and will be removed in May 2016. We should use JDK7 `Collections#emptyIterator()`
2015-06-18 08:47:31 +02:00
Igor Motov b2000a48a4 Snapshot/Restore: Add snapshot name validation logic to all snapshot operation
Make sure snapshot name validation occurs earlier in all snapshot operations.
2015-06-17 19:13:35 -04:00
Igor Motov 36da42c93b Show human readable Elasticsearch version that created index and date when index was created
By setting human parameter to true, it's now possible to see human readable versions of Elasticsearch that created and updated the index as well as the date when the index was created.

Closes #11484
2015-06-17 18:52:01 -04:00
Alex Ksikes 3f6dae1a73 More Like This: renamed `ignore_like` to `unlike`
This changes the parameter name `ignore_like` to the more user friendly name
`unlike`. This later feature generates a query from the terms in `A` but not
from the terms in `B`. This translates to a result set which is like `A` but
unlike `B`. We could have further negatively boosted any documents that have
some `B`, but these documents already do not receive any contribution from
having `B`, and would therefore negatively compete with documents having `A`.

Closes #11117
2015-06-17 17:18:50 -05:00
Simon Willnauer 90f9b5f60d Allow to opt-out of loading packaged REST tests
this is really just a workaround for plugins to run their own
REST tests instead of the core ones. It opts out of the rest test
loading from the core jar file and tries to load from the classpath instead.
Eventually we need to fix this infrastrucutre to move away from parameterized
tests such that subclasses can override behavior.

Closes #11721
2015-06-17 20:37:59 +02:00
Clinton Gormley 05d512f417 Packaging: Add LICENSE, NOTICE, and sha1 files and tests for all core dependencies
Added a licenses/ directory to core which contains a sha1 file for each JAR
dependency, and one or more LICENSE files and one NOTICE file for each
project.

Also adds dev-tools/src/main/resources/license-check/check_license_and_sha.pl
which checks that the licenses/ dir is up to date during a mvn verify,
and which can be used to update the sha1 files when upgrading dependencies.

Closes #2794
Closes #10684
Closes #11705
2015-06-17 18:06:00 +02:00