836 Commits

Author SHA1 Message Date
Jason Tedor
6292bc07f9 Merge pull request #12892 from jasontedor/fix/12865
Validate settings specified in index templates at template creation time
2015-08-14 15:43:16 -06:00
Jason Tedor
b88d2f6255 Validate settings specified in index templates at template creation time
Previously settings specified in index templates were not validated upon
template creation. Creating an index from an index template with invalid
settings could lead to cluster stability issues because creation of such
indexes would bypass index settings validation.

This commit adds validation of settings specified in index templates at
template creation time. This works by routing the index template
settings through the index settings validation mechanism.

Closes #12865
2015-08-14 15:23:10 -06:00
Ryan Ernst
33690b990a Merge pull request #12872 from rjernst/resolve_your_own_config
Remove Environment.resolveConfig
2015-08-14 13:56:32 -07:00
Simon Willnauer
03ceabb1da Merge pull request #12894 from s1monw/fix_repro_line
Fix reproduction line to include project filters
2015-08-14 22:52:55 +02:00
Martijn van Groningen
942d040f45 inner hits: Reset the ShardTargetType after serializing inner hits.
This fixes a bug where only the first top level search hit has a shard target and any subsequent search hits don't.
2015-08-14 22:38:12 +02:00
Lee Hinman
1b9877bb65 Use Java 7 version of Files.readAllLines instead of Java 8 version 2015-08-14 14:36:17 -06:00
Lee Hinman
d35a3a37eb Also catch NoSuchFileException 2015-08-14 13:54:14 -06:00
Lee Hinman
33f118e9c8 Print out the name of the sum that failed 2015-08-14 13:51:47 -06:00
Simon Willnauer
bba34de6b3 Fix reproduction line to include project filters
Today on a failure the reproduce line printed out by the test framework
will build all projects and might fail if the test class is not present.
This commit adds a reactor filter to the reproduction line to ensure
unrelated projects are skipped.

Closes #12838
2015-08-14 21:36:49 +02:00
Ryan Ernst
867f056cf6 Simplify random name index and move method to its only user 2015-08-14 11:22:20 -07:00
Christoph Büscher
407781e76a Adding comments to test 2015-08-14 18:42:45 +02:00
Lee Hinman
41d8b552e6 Validate checksums for plugins if available
When a plugin is downloaded, this change additionally tries to download
`${pluginurl}.sha1` and verify the SHA1 checksum for the file. If no
.sha1 file is found, it tries `${pluginurl}.md5`.

Note that if neither checksum file is found, a notice is printed but the
plugin can still be installed. If the checksum check fails, the plugin
install is aborted.

Example output if no checksums are available:

```
bin/plugin install elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT
-> Installing elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT...
Trying http://download.elastic.co/elasticsearch/elasticsearch-analysis-icu/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ...
Trying http://search.maven.org/remotecontent?filepath=elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ...
Trying https://oss.sonatype.org/service/local/repositories/releases/content/elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ...
Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/2.6.0-SNAPSHOT.zip ...
Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip ...
Downloading .....................................DONE
Verifying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
```

Example output if checksums are available:

```
bin/plugin install elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT
-> Installing elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT...
Trying http://download.elastic.co/elasticsearch/elasticsearch-analysis-icu/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ...
Trying http://search.maven.org/remotecontent?filepath=elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ...
Trying https://oss.sonatype.org/service/local/repositories/releases/content/elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ...
Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/2.6.0-SNAPSHOT.zip ...
Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip ...
Downloading .....................................DONE
Verifying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip checksums if available ...
Downloading .DONE
```

Example output if checksums fail:

```
bin/plugin install elasticsearch/elasticsearch-analysis-kuromoji/2.5.0 -url http://localhost:8000/elasticsearch-analysis-kuromoji-2.5.0.zip
-> Installing elasticsearch/elasticsearch-analysis-kuromoji/2.5.0...
Trying http://localhost:8000/elasticsearch-analysis-kuromoji-2.5.0.zip ...
Downloading .............................................DONE
Verifying http://localhost:8000/elasticsearch-analysis-kuromoji-2.5.0.zip checksums if available ...
Downloading .DONE
ERROR: incorrect hash, file hash: [dbdc9c2cd32782054497a21fbdcae3ca1ff23c80], expected: [dbdc9c2cd32782054497a21fbdcae3ca1ff23c80-bad]
```

Resolves #12750
2015-08-14 10:40:40 -06:00
Alexander Reelsen
0240b581e7 PluginManager: Fix automatically generated URLs for official plugins
In order to match the paths of official plugins, we need to fix
the broken test by removing the elasticsearch prefix from the official
plugin names before testing.
2015-08-14 18:28:54 +02:00
Igor Motov
e44991d2ef Mute test SharedClusterSnapshotRestoreIT#renameOnRestoreTest
Working on the fix
2015-08-14 12:12:53 -04:00
Christoph Büscher
e01c0927a6 Aggregations: Make ValueParser.DateMath aware of timezone setting
This PR adds a timezone field to ValueParser.DateMath that is
set to UTC by default but can be set using the existing constructors.
This makes it possible for extended bounds setting in DateHistogram
to also use date math expressions that e.g. round by day and apply
this rounding in the time zone specified in the date histogram
aggregation request.

Closes #12278
2015-08-14 18:11:39 +02:00
Simon Willnauer
4c1ef2c943 Fix test to also include the mapper-size 2015-08-14 18:11:27 +02:00
Simon Willnauer
30692fe917 Add mapper-size to plugin list 2015-08-14 18:04:45 +02:00
Martijn van Groningen
0688dddebf Also delegate rewrite the the wrapped IndexSearcher.
In the the AssertingIndexSearcher is used we then also have extra validation when rewritting the query.
2015-08-14 16:34:04 +02:00
Adrien Grand
2fecc7e5c9 Merge pull request #12875 from jpountz/fix/dumber_context_indexsearcher
Simplify ContextIndexSearcher.
2015-08-14 15:42:23 +02:00
Alexander Reelsen
cedbd20f2c PluginManager: Change staging URL to reflect S3 bucket
In order to create releases without actually changing the version
as part of a commit, we also need to reflect the path of the potentially
changing S3 repo.
2015-08-14 15:23:31 +02:00
Adrien Grand
b3e7146b22 Simplify ContextIndexSearcher.
In particular this commit moves collector wrapping logic from
ContextIndexSearcher to QueryPhase.
2015-08-14 15:03:38 +02:00
Ryan Ernst
f3d63095db Merge branch 'master' into resolve_your_own_config
Conflicts:
	core/src/main/java/org/elasticsearch/env/Environment.java
	core/src/test/java/org/elasticsearch/index/analysis/commongrams/CommonGramsTokenFilterFactoryTests.java
	core/src/test/java/org/elasticsearch/index/analysis/synonyms/SynonymsAnalysisTest.java
	plugins/analysis-kuromoji/src/test/java/org/elasticsearch/index/analysis/KuromojiAnalysisTests.java
2015-08-14 03:34:26 -07:00
Ryan Ernst
be638fb6ef Internal: Remove Environment.resolveConfig
This method has multiple modes of resolving config files by
first looking in the config directory, then on the classpath,
and finally by prefixing with "config/" on the classpath.

Most of the places taking advantage of this were tests, so they
did not have to setup a real home dir with config. The only place
that was really relying on it was the code which loads names.txt
to randomly choose a node name.

This change fixes test to setup fake home dirs with their config
files. It also makes the logic for finding names.txt explicit:
look in config dir, and if it doesn't exist, load /config/names.txt
from the classpath.
2015-08-14 03:03:47 -07:00
javanna
4010e7e9a7 Java api: restore support for minimumShouldMatch and disableCoord in TermsQueryBuilder
TermsQueryParser still parses those values although deprecated. These need to be present in the java api as well to get ready for the query refactoring, where the builders are the intermediate query format that we parse our json queries into. Whatever the parser supports need to be supported by the builder as well.

Closes #12870
2015-08-14 11:17:19 +02:00
javanna
2e0b548b06 Java api: remove support for lookup cache in TermsLooukpBuilder
TermsQueryParser doesn't support the cache field anymore, so if it gets set through java api, the subsequent parsing of that query will throw error

Relates to #12870
2015-08-14 11:16:55 +02:00
Ryan Ernst
470f5370b9 Merge pull request #12868 from rjernst/bye_bye_classloaders
Remove ClassLoader from Settings
2015-08-14 02:15:07 -07:00
Alexander Reelsen
0f3ada159e Release: Create pre release script
Refactored a part out of the release script, so the user can
change the version locally as well as move the documentation
and change the Version.java

The background of this change is to have a very simple release
process that puts stuff into a staging environment, so the beta
release can be tested, before it is officially released.

This means the build_release script can be removed soon.
2015-08-14 11:04:32 +02:00
Ryan Ernst
6dcfda99e8 Internal: Remove ClassLoader from Settings
Settings currently has a classloader member, which any user (plugin
or core ES code) can access to load classes/resources. This is extremely
error prone as setting the classloder on the Settings instance is a
public method. Furthermore, it is not really necessary. Classes that
need resources should load resources using normal means
(getClass().getResourceAsStream). Those that need classes
should use Class.forName, which will load the class with the
same classloader as the calling class. This means, in the few
places where classes are loaded by string name, they will use
the appropriate loader: either the default classloader which loads
core ES code, or a child classloader for each plugin.

This change removes the classloader member from Settings, as
well as other classloader related uses (except for a handful
of cases which must use a classloader, at least for now).
2015-08-13 23:55:27 -07:00
Ryan Ernst
c16772b0fc Undo accidental commit of crap
This reverts commit 589eecf55d26e6fe9831910ee45fb21b5bdedf9b.
2015-08-13 19:39:55 -07:00
Ryan Ernst
589eecf55d Fourth time's a charm, one more mock class to add to test jar 2015-08-13 19:36:53 -07:00
Ryan Ernst
dc1a2d2e5a Merge branch 'master' into fix/12784 2015-08-13 14:35:48 -07:00
Ryan Ernst
9f64c75391 Remove leftover classes 2015-08-13 14:33:11 -07:00
Ryan Ernst
ccef0551ef Tests: Refactor classes only plugged in by tests to use package private extension points
We previous used something like Class.forName to load mock classes,
where tests would set a setting that was *supposed* to only be used by
tests. This change make these impls package private so that only tests
can change out these implementations, through test plugins.

closes #12784
2015-08-13 13:52:55 -07:00
Simon Willnauer
1f41a8c682 Don't cache percolator query on loading percolators
No need to load catch this query since it's cheap and not reused.
If we cache it, it can cause assertions to be tripped since this
method is executed during postRecovery phase and might still run while
nodes are shutdown in tests.
2015-08-13 22:27:14 +02:00
Adrien Grand
75d84631da Merge pull request #12833 from saschamarkus/saschamarkus_issue_11510_rebase
Throw Exception for missing settings file
2015-08-13 14:26:03 +02:00
Simon Willnauer
74f18d8c16 Introduce a formal ExtensionPoint class to stream line extensions
This commit tries to add some infrastructure to streamline how extension
points should be strucutred. It's a simple approache with 4 implementations
for `highlighter`, `suggester`, `allocation_decider` and `shards_allocator`.
It simplifies adding new extension points and forces to register classes instead
of strings.
2015-08-13 13:47:11 +02:00
Colin Goodheart-Smithe
a9a4903a0a Aggregations: Removed unused factor parameter in DateHistogramBuilder
DateHistogramParser does not recognise this parameter and therefore setting it would throw an exception as noted in https://github.com/elastic/elasticsearch/issues/6490. It is also not documented.
2015-08-13 09:47:05 +01:00
Lee Hinman
ae7a73dc1c [TEST] Skip counting ExtraFS files when asserting path is cleared 2015-08-12 16:47:27 -06:00
Lee Hinman
b58970a3fe [TEST] Remove custom data path tests, replace with better tests
Replace with a better, "unit-y" tests that couple the shared data path
with the correctly set `path.shared_data` configuration setting
2015-08-12 16:05:38 -06:00
Lee Hinman
0cb3e21a3e [TEST] mark tests in IndicesCustomDataPathIT with AwaitsFix 2015-08-12 15:44:19 -06:00
Simon Willnauer
605253a39f Cut over master to 2.0.0-SNAPSHOT 2015-08-12 21:16:08 +02:00
Lee Hinman
79d1568b58 Remove the node.enable_custom_paths setting
This setting is useless now that we have the `path.shared_data` setting.

Resolves #12776
2015-08-12 12:23:05 -06:00
Lee Hinman
ff5ad39c7a Add path.shared_data
This allows `path.shared_data` to be added to the security manager while
still allowing a custom `data_path` for indices using shadow replicas.

For example, configuring `path.shared_data: /tmp/foo`, then created an
index with:

```
POST /myindex
{
  "index": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "data_path": "/tmp/foo/bar/baz",
    "shadow_replicas": true
  }
}
```

The index will then reside in `/tmp/foo/bar/baz`.

`path.shared_data` defaults to `null` if not specified.

Resolves #12714
Relates to #11065
2015-08-12 10:51:46 -06:00
Simon Willnauer
fe3179d9cc Return 408 REQUEST_TIMEOUT if _cluster/health times out
Today we return `200 OK` which is misleading since we really didn't
produce a valid response / didn't wait long enough.
2015-08-12 17:51:20 +02:00
Martijn van Groningen
777921dfb0 test: simplified percolator recovery tests 2015-08-12 16:51:27 +02:00
Simon Willnauer
ed9eef92dd [TEST] Fix off-by-one bug in corruptFile util function
The upper bound must be 0-based since we are corrupting an offset into
the file but it can be a 1-based length of the file which results in an
uncorrupted file.
2015-08-12 16:43:31 +02:00
Nik Everett
e4e60774c1 Merge pull request #12787 from nik9000/configured_plugin
QA: Add configuration to the jvm-example plugin
2015-08-12 07:36:34 -07:00
saschamarkus
e59bbf8e31 Issue 11510: Throw Exception for missing settings file
An Exception will be thrown when the givven YAML-file doesn't exist.
2015-08-12 15:13:03 +02:00
Tanguy Leroux
03c327ff12 Expose ClassloadingMXBean in Node Stats
Closes #12738
2015-08-12 14:29:13 +02:00
Konrad Beiske
44c667dac6 Merge pull request #12828 from s1monw/allow_master_elect_extension
Expose zen ElectMasterService as a Discovery extension point
2015-08-12 13:40:13 +02:00