Commit Graph

12142 Commits

Author SHA1 Message Date
Britta Weber 471cd54e39 fix api hangs if no shards allocated 2015-05-13 17:05:29 +02:00
Simon Willnauer 869baeea0c Don't throw an exception if repositories are unregister with match all
Today we barf if repositories are unregistered with a `*` pattern. This
happens on almost every test and adds weird log messages. I dont' think
we should barf in that case.

Closes #11113
2015-05-13 17:01:04 +02:00
Simon Willnauer 6a43fe348d Fail recovery if retry recovery if resetRecovery fails
This might fail if the shard is closed for instance. This will leak
a shard lock causing the shard being locked on this node forever.
2015-05-13 16:47:18 +02:00
javanna add18a5c99 Java api: remove duplicated buildAsBytes and corresponding toString methods
We have some builders, specifically query builders, `SearchSourceBuilder`, `QuerySourceBuilder` and `SuggestBuilder`, that implement `ToXContent` and also allow to build their content as bytes by simply creating a `BytesReference` that holds their json (or yaml etc.) content (`buildAsBytes` methods). They can also print out their content through `toString`. Made sure that those common methods are in one single place and reused where needed.

Also, merged `QueryBuilder` and `BaseQueryBuilder` and made `QueryBuilder` an abstract class instead of an interface.

Closes #11063
2015-05-13 16:14:16 +02:00
Simon Willnauer 807b3c6b95 [TEST] Add basic unittest like tests for SyncedFlushService
This commit adds a SingleNodeTest that runs through the different fialure szenarios
on a single shard to test the basic operations.
2015-05-13 16:05:34 +02:00
javanna d2765a2e26 Java api: add missing rewrite parameter to FuzzyQueryBuilder
We parse the rewrite field in FuzzyQueryParser but we don't allow to set it via FuzzyQueryBuilder for our java api users. Added missing field and setter.

Closes #11130
Closes #11139
2015-05-13 15:46:57 +02:00
Britta Weber f1948cf95c doc for seal api and doc for syned flush in general 2015-05-13 15:43:05 +02:00
Britta Weber c628d67f9e api for synced flush
closes #11098
2015-05-13 14:45:24 +02:00
Adrien Grand 5c9c4776cd Internal: remove dependency on hppc:esoteric.
The esoteric classifier contains in particular maps that take bytes or doubles
as keys. In the byte case, we can just use integer, and in the double case we
can use their long bits instead.
2015-05-13 14:13:10 +02:00
Dawid Weiss 491b00c4ec Updating to HPPC-0.7.1
Close #11035
2015-05-13 13:53:14 +02:00
Simon Willnauer 43ff544117 Add translog checkpoints to prevent translog corruption
Today we are almost intentionally corrupt the translog if we loose
a node due to powerloss or similary disasters. In the translog reading
code we simply read until we hit an EOF exception ignoring the rest of the
translog file once hit. There is no information stored how many records
we are expecting or what the last written offset was.

This commit restructures the translog to add checkpoints that are written
with every sync operation recording the number of synced operations as well
as the last synced offset. These checkpoints are also used to identify the actual
transaction log file to open instead of relying on directory traversal.

This change adds a significant amount of additional checks and pickyness to the translog
code. For instance is the translog now associated with a specific engine via a UUID that is
written to each translog file as part of it's header. If an engine opens a translog file it
was not associated with the operation will fail.

Closes to #10933
Relates to #11011
2015-05-13 12:40:35 +02:00
Adrien Grand 472cc0af08 Merge pull request #11142 from jpountz/feature/bool_filter
Query DSL: Add `filter` clauses to `bool` queries.
2015-05-13 12:25:32 +02:00
Adrien Grand 630757906a Query DSL: Add `filter` clauses to `bool` queries.
These clauses filter the document space without affecting scoring and map to
Lucene's BooleanClause.Occur.FILTER. The `filtered` query is now deprecated and

```json
{
  "filtered": {
    "query": { //query },
    "filter": { //filter }
  }
}
```
should be replaced with
```json
{
  "bool": {
    "must": { //query },
    "filter": { //filter }
  }
}
```
2015-05-13 12:04:56 +02:00
Shay Banon ba20d4b6ba [TEST] fix unintended relocation 2015-05-13 11:18:28 +02:00
Adrien Grand 36884807b3 Merge pull request #11120 from jpountz/fix/bool_parsing
Query DSL: Fix `bool` parsing.
2015-05-13 11:09:12 +02:00
Adrien Grand 11db3170cd Query DSL: Fix `bool` parsing.
In #10985 I introduced a bug that should clauses are parsed as filters while
must_not clauses should be parsed as filters.
2015-05-13 11:08:36 +02:00
Adrien Grand 44e9b75296 Mappings: Make FieldNameAnalyzer less lenient.
In case FieldNameAnalyzer does not find an explicit analyzer for a given
field name, it returns the default analyzer. This behaviour can hide bugs
where the analyzer fails to be propagated to FieldNameAnalyzer or an
analyzer is requested for a field which is not mapped.
2015-05-13 11:01:38 +02:00
Simon Willnauer 6046ecbab3 Prevent PercolateResponse from serializing negative VLong
We are using a a VLong to serialize the PercolateResponse#tookInMillis. This
can due to several `System.currentTimeMillis()` implemenation details be negative.
We should prevent the negavite value for being serialized as a VLong and make sure
we use a valid value for this in the first place

Closes #11138
2015-05-13 10:22:08 +02:00
Adrien Grand f22935b1a8 Query DSL: Make geo filters queries.
This commit replaces geo filters with queries that support two-phase iteration.
2015-05-13 09:10:49 +02:00
Ryan Ernst 1b15333f34 Merge pull request #11074 from rjernst/pr/include-in-object-removal
Mappings: Remove ability to set meta fields inside documents
2015-05-12 23:24:36 -07:00
Ryan Ernst f766b260ba Add tests for includeInObject backcompat 2015-05-12 23:11:15 -07:00
Ryan Ernst 565ffb16f1 Mappings: Remove ability to set meta fields inside documents
A few meta fields can currently be set within a document's source.
However, the recommended way to set meta fields like this is through
the api, and setting within the document can be a performance trap
(e.g. needing to find _id in order to route the document).

This change removes the ability to set meta fields within
a document source for 2.0+ indexes.

closes #11051
closes #11074
2015-05-12 23:09:03 -07:00
Robert Muir 5deba7264c Add tests.config support to BootstrapForTesting
Several plugins (e.g. elasticsearch-cloud-aws, elasticsearch-cloud-azure, elasticsearch-cloud-gce)
have integration tests that run with actual credentials to a remote service, so test runs
need access to this file.

These all require the tester (or jenkins) to supply the file with -Dtests.config.
2015-05-12 21:23:17 -04:00
Igor Motov d6efe1e508 Docs: Add information about restoring to a different cluster 2015-05-12 20:59:24 -04:00
Robert Muir 79023c1e61 Merge pull request #11132 from rmuir/lockdown6
Remove unnecessary permissions.
2015-05-12 20:03:20 -04:00
Robert Muir 4473bb4275 Remove unnecessary permissions. 2015-05-12 19:15:59 -04:00
Lee Hinman 990c3a9093 Merge remote-tracking branch 'dakrone/acquire-write-lock-on-delete' 2015-05-12 13:23:09 -06:00
Lee Hinman f7e8d7c024 [CORE] Acquire IndexWriter's `write.lock` lock before shard deletion
In `NodeEnvironment.deleteShardDirectoryUnderLock`, we will now attempt
to acquire, then release, the `write.lock` file for the Lucene index in
question to ensure that no other `IndexWriter` has the directory open
before deleting the data.

Note that the `write.lock` file must be released before the actual
deletion in order to allow the directory to be deleted.

Fixes #11097
2015-05-12 13:19:29 -06:00
Robert Muir 262555079c Merge pull request #11106 from rmuir/symlink
Improve path mgmt on init, better error messages, symlink support
2015-05-12 14:21:19 -04:00
Adrien Grand 2999ea9558 Merge pull request #11126 from jpountz/fix/script_query
Query DSL: Make the script filter a query.
2015-05-12 19:33:17 +02:00
Adrien Grand 6936577c6e Query DSL: Make the script filter a query.
This change changes the script filter so that it produces scorers with two-phase
iteration support instead of doc id sets with random-access.
2015-05-12 19:20:37 +02:00
Adrien Grand 842608c396 Merge pull request #11125 from jpountz/upgrade/lucene-5.2.0-snapshot-1678978
Upgrade to lucene-5.2.0-snapshot-1678978.
2015-05-12 18:53:56 +02:00
Adrien Grand 290fefcdf8 Upgrade to lucene-5.2.0-snapshot-1678978. 2015-05-12 18:49:51 +02:00
Britta Weber e6f5fb82f0 Merge remote-tracking branch 'origin/master' into feature/synced_flush 2015-05-12 14:50:42 +02:00
Britta Weber 24ffcc28ac fix merge conflict in javadoc 2015-05-12 14:40:56 +02:00
Simon Willnauer a7abe0a324 Make synced flush attemp async to run it easily from a TransportAction
Today we enforce blocking which doesnt' really fit in the elasticsearch model
this commit adds async execution to the synced flush service by passing a
ActinListener to the service returing immediately.
2015-05-12 11:54:56 +02:00
Britta Weber 272babd9dc [TEST] add comment bout security manager to bwc test description 2015-05-12 10:26:42 +02:00
markharwood 89b95dccc8 Aggregations enhancement - remove pointless term frequency lookups.
If the user has set a shard_min_doc_count setting then avoid looking up background frequencies if the term fails to meet the foreground threshold on a shard.

Closes #11093
2015-05-12 08:45:17 +01:00
Simon Willnauer 236f6ccad7 [TEST] Reduce likelyness of sync on every operation 2015-05-12 09:32:12 +02:00
Robert Muir d1defef69d Properly handle the case where symlinks are supported, but
the user is not a windows administrator (can throw IOE in this case)
2015-05-12 00:53:51 -04:00
Robert Muir 019a9410a3 SecurityBootstrap -> BootstrapForTesting, and make less things public 2015-05-12 00:34:02 -04:00
Robert Muir 2cea5f2b38 Improve path management on init:
* Properly support symlinks (e.g. /tmp -> /mnt/tmp)
* Check all configured paths up front and deliver the best exception we can when things are wrong
* Initialize securitymanager earlier
* Fix too-loud error logging of Natives root check
2015-05-12 00:20:52 -04:00
Robert Muir 7c2a8dfb9e Merge pull request #11104 from rmuir/generate_paths
Generate access to tests paths like other paths.
2015-05-11 18:32:26 -04:00
Robert Muir 3ccdde7e2a Generate access to tests paths like other paths. 2015-05-11 18:04:14 -04:00
Ryan Ernst e7618b8528 Settings: Remove file based index templates
As a follow up to #10870, this removes support for
index templates on disk. It also removes a missed
place still allowing disk based mappings.

closes #11052
2015-05-11 12:51:22 -07:00
gmarz 6dcd5bf618 [TEST] Mlockall: Removing assertTrue until we have the proper test infra 2015-05-11 13:51:59 -04:00
Britta Weber 571feec451 Merge remote-tracking branch 'origin/master' into feature/synced_flush 2015-05-11 18:21:36 +02:00
gmarz 852482844c [TEST] Only check if mlockall succeeds on Windows for now 2015-05-11 12:12:32 -04:00
Michael McCandless 08f7caa56b Use System.nanoTime when measuring elapsed time
System.currentTimeMillis is more vulnerable when the clock shifts.

Closes #11058
2015-05-11 11:24:04 -04:00
gmarz cc3f02c0a0 [TEST] Fix mlockall test when running on Windows 2015-05-11 11:09:15 -04:00