Commit Graph

907 Commits

Author SHA1 Message Date
Nik Everett da42838cff Add wait_for_completion to reindex
It defaults to false and when false it returns a task identifier. Right
now all you can do is get the task to see if it is still running. Once
the task finishes it vanishes and you can't get any information about it.
2016-01-19 14:01:36 -05:00
Nik Everett 0d6d77328d Merge branch 'master' into feature/reindex 2016-01-19 13:27:24 -05:00
Jack Conradson 2249a640bb Plan A Update...
New Features:

Exceptions (throw, try/catch)
Infinite Loop Checking
Iterators added to the API

Fixes:

Improved loop path analysis
Fixed set/add issue with shortcuts in lists
Fixed minor promotion issue with incorrect type
Fixed score issue related to score extending Number

Documentation:

Added JavaDocs for some files
2016-01-19 09:49:03 -08:00
David Pilato 725d7f968b Merge branch 'fix/12567-azure-timeout' 2016-01-19 18:02:33 +01:00
David Pilato 6db5b5033c Change exception message for wrong timeout in azure repository settings
Backport in master this change: https://github.com/elastic/elasticsearch/pull/15950#discussion-diff-50128378

Related to #15080
Related to #15950
2016-01-19 18:01:54 +01:00
David Pilato d6e6ef4ec4 Merge branch 'pr/tests-azure-repo' 2016-01-19 17:07:59 +01:00
Nik Everett 6b6d01c4fe Merge pull request #16072 from nik9000/xlint_plugins
Remove remaining xlints from plugins
2016-01-19 10:54:15 -05:00
Nik Everett 4efa8c4ff5 Remove remaining xlints from plugins 2016-01-19 10:53:48 -05:00
David Pilato 8d35bca4db Add more tests for Azure Repository client selection
One test we forgot in #14843 and #13779 is the default client selection.

Most of the time, users won't define explicitly which client they want to use because they are providing only one connection to Azure storage:

```yml
cloud:
    azure:
        storage:
            my_account:
                account: your_azure_storage_account
                key: your_azure_storage_key
```

Then using the default client like this:

```sh
# This one will use the default account (my_account1)
curl -XPUT localhost:9200/_snapshot/my_backup1?pretty -d '{
  "type": "azure"
}'
```

This commit adds tests to check that the right client is still selected when no client is explicitly set when creating the snapshot.
2016-01-19 14:23:08 +01:00
David Pilato 11e7a746ae Replace server side timeout by client side timeout
This commit replaces server side timeout (which is BTW not correctly implemented in azure client 2.0.0 but fixed later #16084) with a client side timeout.

As a consequence, for each request sent to azure, azure client will raise an exception after a given amount of time (timeout).

Closes #12567
2016-01-19 13:56:08 +01:00
Simon Willnauer fbfa9f4925 Merge branch 'master' into new_index_settings 2016-01-19 10:13:48 +01:00
Simon Willnauer 8e0390b09e Register index.version_created for several analysis plugin tests 2016-01-19 09:35:32 +01:00
Adrien Grand d6cbd6f2f0 Merge pull request #16059 from jpountz/enhancement/mapping_merge_reason
Expose the reason why a mapping merge is issued.
2016-01-19 09:27:32 +01:00
Simon Willnauer 20fe8ac854 Register index.version.created in Kuromoji unittest since it's a node level setting here 2016-01-19 09:14:41 +01:00
Nik Everett 53cce3c399 Merge pull request #16071 from nik9000/xlint_lang_groovy
Remove Xlint from lang-groovy and discovery-azure
2016-01-18 21:06:11 -05:00
Ryan Ernst ef4f0a8699 Test: Make rest test framework accept http directly for the test cluster
The rest test framework, because it used to be tightly integrated with
ESIntegTestCase, currently expects the addresses for the test cluster to
be passed using the transport protocol port. However, it only uses this
to then find the http address.

This change makes ESRestTestCase extend from ESTestCase instead of
ESIntegTestCase, and changes the sysprop used to tests.rest.cluster,
which now takes the http address.

closes #15459
2016-01-18 16:44:14 -08:00
Nik Everett bd1324ccc9 Remove Xlint from lang-groovy and discovery-azure
discovery-azure didn't actually need it.

This removes all non-default Xlints from modules.
2016-01-18 18:24:44 -05:00
Nik Everett 08ca9d7645 Fix warnings in lang-plan-a
Removes all Xlint skips in lang-plan-a. Warnings were just some extra
casts which were removed, some raw types which we just needed to <?>, and
a couple of unchecked casts in data that we know to be Map<String, Object>
because that structure is super-ultra-common in scripts.
2016-01-18 15:27:51 -05:00
Adrien Grand 055953d6b3 Expose the reason why a mapping merge is issued.
This would be useful in order to only perform some validations in the case of
a mapping update and in cases when a mapping is restored eg. after a restart,
such as discussed in #15989.

This replaces the current `applyDefault` parameter which can be derived from
the mapping merge reason: the default mapping should be applied only in case of
a mapping update, if the mapping does not exist yet and if this is not the
default mapping.
2016-01-18 17:41:23 +01:00
Simon Willnauer a61723b538 convert index.store.fs.fs_lock 2016-01-18 10:17:11 +01:00
Simon Willnauer 7925e2ef84 convert IndexModule settings 2016-01-18 09:23:35 +01:00
Simon Willnauer 79f4697f3e Register MockFSDirectoryService settings 2016-01-18 09:23:34 +01:00
Simon Willnauer a8eedd0457 fix all kinds of crazy test failures 2016-01-18 09:23:34 +01:00
Simon Willnauer ceef6d7a42 fix Murmur3FieldMapperTests 2016-01-18 09:23:34 +01:00
Nik Everett 645b053061 Merge branch 'master' into feature/reindex 2016-01-15 14:04:37 -05:00
David Pilato ed45ad6327 Fix Azure repository with only one primary account
Using a single azure account is now rejected.
This commit fixes this issue and adds a test for it.

This regression was introduced with #13779. Hopefully no elasticsearch version has been released since then.

Needs to be merged in 2.2, 2.x and master branches.
2016-01-14 13:50:02 +01:00
Nik Everett 85797aeb50 Basic reindex and update_by_query
This creates an reindex plugin with a very basic implementation that is
very like delete-by-query. New we'll integrate it with the task managament
work but for now this works.
2016-01-13 09:45:49 -05:00
Nik Everett 01ce49e94e Ban Serializable
1. Uses forbidden patterns to prevent things from referencing
java.io.Serializable or from mentioning serialVersionUID.
2. Uses -Xlint:-serial so we don't have to hear from javac that we aren't
declaring serialVersionUID on any classes that we make that happen to extend
Serializable.
3. Remove Serializable and serialVersionUID declarations.

I didn't use forbidden apis because it doesn't look like it has a way to ban
explicitly implementing Serializable. If you try to ban Serializable with
forbidden apis you end up banning all Exceptions and all Strings.

Closes #15847
2016-01-11 16:57:31 -05:00
Nik Everett b29c416b7c Merge pull request #15848 from nik9000/xlint3
Remove -Xlint:-deprecation from all but core
2016-01-11 16:55:13 -05:00
Nik Everett 6250f4dbaa Remove deprecated azure settings 2016-01-08 13:13:14 -05:00
Jason Tedor 871d1b4885 Remove and forbid use of j.u.c.ThreadLocalRandom
This commit removes and now forbids all uses of
java.util.concurrent.ThreadLocalRandom across the codebase. The
underlying issue with ThreadLocalRandom is that it can not be
seeded. This means that if ThreadLocalRandom is used in production code,
then tests that cover any code path containing ThreadLocalRandom will be
prevented from being reproducible by use of ThreadLocalRandom. Instead,
using org.elasticsearch.common.random.Randomness#get will give
reproducible sources of random when running under tests and otherwise
still give an instance of ThreadLocalRandom when running as production
code.
2016-01-08 12:23:48 -05:00
Nik Everett 98fdb39d3d Remove deprecated settings 2016-01-08 11:17:56 -05:00
Nik Everett 81a7607256 Remove -Xlint:-deprecation from plugins
Instead we suppress warnings about using deprecated stuff near the usage
site with a comment about why its ok.
2016-01-07 20:44:46 -05:00
Nik Everett 42cb2a5f02 Merge pull request #15811 from nik9000/def_cleanup
Cleanups for Def
2016-01-07 11:03:01 -05:00
Nik Everett 52f28888d5 Merge pull request #15813 from nik9000/xlint1
Remove Xlint:-override,-fallthrough,-static
2016-01-07 08:34:40 -05:00
Nik Everett 244120a065 Remove more Xlint skips 2016-01-06 23:53:05 -05:00
Nik Everett 0786c506dc Remove a few more Xlint skips 2016-01-06 23:28:13 -05:00
Nik Everett 20e7fa97db Remove Xlint:-override,-fallthrough,-static
Adds `@SuppressWarnings("fallthrough")` in two places where the fallthrough
is used to implement well known hashing algorithms.
2016-01-06 22:27:14 -05:00
Nik Everett 74c132afc6 Standardize some methods on varargs
Right now we define the same sort of methods as taking String arrays and
string varargs. We should standardize on one and varargs is easier to
call so lets use varargs!
2016-01-06 21:01:58 -05:00
Nik Everett 32605ecb4f Cleanups for Def
Manually I:
1. Added some missing raw types warnings suppressions.
2. Removed some unused unchecked cast warning suppressions.
3. Added <?> to Class.

I let my IDE:
1. Remove unneeded casts.
2. Reorder imports (just ignore these, everyone does).
2016-01-06 20:28:49 -05:00
Martijn van Groningen 2d6adf6428 Percolator refactoring:
* Added percolator field mapper that extracts the query terms and indexes these terms with the percolator query.
* At percolate time these extracted terms are used to query percolator queries that are like to be evaluated. This can significantly cut down the time it takes to percolate. Whereas before all percolator queries were evaluated if they matches with the document being percolated.
* Changes made to percolator queries are no longer immediately visible, a refresh needs to happen before the changes are visible.
* By default the percolate api only returns upto 10 matches instead of returning all matching percolator queries.
* Made percolate more modular, so that it is easier to add unit tests.
* Added unit tests for the percolator.

Closes #12664
Closes #13646
2016-01-06 16:08:10 +01:00
Igor Motov a89dba27c2 Task Management: Add framework for registering and communicating with tasks
Adds task manager class and enables all activities to register with the task manager. Currently, the immutable Transport*Activity class represents activity itself shared across all requests. This PR adds and an additional structure Task that keeps track of currently running requests and can be used to communicate with these requests using TransportTaskAction.

Related to #15117
2016-01-05 12:24:43 -05:00
Adrien Grand 1a47226d9a Merge pull request #15663 from jpountz/remove/mapping_backcompat
Remove mapping backward compatibilit with pre-2.0.
2016-01-04 10:05:39 +01:00
Robert Muir 25914ae879 Merge pull request #15688 from rmuir/thirdPartyAudit3
Improve thirdPartyAudit check, round 3
2015-12-29 09:24:51 -05:00
David Pilato 96b3166c6d Add timeout settings (default to 5 minutes)
By default, azure does not timeout. This commit adds support for a timeout settings which defaults to 5 minutes.
It's a timeout **per request** not a global timeout for a snapshot request.

It can be defined globally, per account or both. Defaults to `5m`.

```yml
cloud:
    azure:
        storage:
            timeout: 10s
            my_account1:
                account: your_azure_storage_account1
                key: your_azure_storage_key1
                default: true
            my_account2:
                account: your_azure_storage_account2
                key: your_azure_storage_key2
                timeout: 30s
```

In this example, timeout will be 10s for `my_account1` and 30s for `my_account2`.

Closes #14277.
2015-12-29 11:40:48 +01:00
David Pilato a49fe189b0 Support global `repositories.azure.` settings
All those repository settings can also be defined globally in `elasticsearch.yml` file using prefix `repositories.azure.`. For example:

```yml
repositories.azure:
    container: backup-container
    base_path: backups
    chunk_size: 32m
    compress": true
```

Closes #13776.
2015-12-29 10:43:01 +01:00
Robert Muir 180ab2493e Improve thirdPartyAudit check, round 3 2015-12-28 22:38:55 -05:00
Adrien Grand 5eb7555ffb Make text parsing less lenient.
It now requires that the parser is on a value.
2015-12-28 16:06:42 +01:00
Adrien Grand af122f4151 Remove mapping backward compatibilit with pre-2.0.
This removes the backward compatibility layer with pre-2.0 indices, notably
the extraction of _id, _routing or _timestamp from the source document when a
path is defined.
2015-12-24 13:47:37 +01:00
Robert Muir d144ba24a5 Merge pull request #15588 from rmuir/hdfs2-only
merge current hdfs improvements to master
2015-12-23 18:17:22 -05:00