1286 Commits

Author SHA1 Message Date
Alexander Reelsen
0d4711c2fc Ingest: Add attachment processor
This is a simple port of the mapper attachment plugin to the ingest
functionality, no new features. The only option is to limit
the number of chars to prevent indexing of huge documents.

Fields can be selected in the processor as well.

Close #16303
2016-02-09 17:03:30 +01:00
Yannick Welsch
380393a5b7 Add permission to access sun.reflect.MethodAccessorImpl from Groovy scripts
Groovy uses reflection to invoke closures. These reflective calls are optimized by the JVM after "sun.reflect.inflationThreshold" number of invocations.
After inflation, access to sun.reflect.MethodAccessorImpl is required from the security manager.

Closes #16536
2016-02-09 16:47:38 +01:00
Simon Willnauer
96bcb47fc9 Detach QueryShardContext from IndexShard and remove obsolete threadlocals
IndexShard currently holds an arbitraritly used `getQueryShardContext` that comes
out of a ThreadLocal. It's usage is undefined and arbitraty since there is also
such a method with different semantics on `IndexService` This commit removes the threadLocal on
IndexShard as well as on the context itself. It's types are now a member and the QueryShardContext
lifecycle is managed byt SearchContext which passes the types on from the SearchRequest.
2016-02-08 15:05:03 +01:00
Jack Conradson
f6f2d40fd5 Minor clean up.
* Minor clean up of Writer constants.
* Removed synthetic attribute from the generated constructor and method.
* Added a safeguard for maximum script length.

Closes #16457
2016-02-05 12:00:18 -08:00
Robert Muir
dcec4791c4 fix eclipse to compile at all. why does the build not fail? 2016-02-04 18:57:53 -05:00
Martijn van Groningen
7a6adfd93a ingest: Added foreach processor.
This processor is useful when all elements of a json array need to be processed in the same way.
This avoids that a processor needs to be defined for each element in an array.
Also it is very likely that it is unknown how many elements are inside an json array.
2016-02-04 23:44:01 +01:00
Simon Willnauer
cc520b20e1 Merge branch 'master' into fix_settings_filter 2016-02-04 09:28:41 +01:00
Nik Everett
a2f07679fd Add task status
Implements a simple task status for superclasses of ReplicationRequest to
show how you can do use the status.
2016-02-03 18:21:42 -05:00
Simon Willnauer
e1cf5e745d some plugins share settings - make it easy to filter them 2016-02-03 21:50:06 +01:00
Simon Willnauer
baacabf9fa convert more filters 2016-02-03 20:31:16 +01:00
Simon Willnauer
f770164d9a fix imports 2016-02-03 20:14:29 +01:00
Simon Willnauer
e02d2e004e Rewrite SettingsFilter to be immutable
This change rewrites the entire settings filtering mechanism to be immutable.
All filters must be registered up-front in the SettingsModule. Filters that are comma-sparated are
not allowed anymore and check on registration.
This commit also adds settings filtering to the default settings recently added to ensure we don't render
filtered settings.
2016-02-03 20:05:55 +01:00
Lee Hinman
3c7f578010 Merge remote-tracking branch 'dakrone/fix-smb-plugin-master' 2016-02-03 07:50:14 -07:00
Simon Willnauer
a77344d742 Merge branch 'master' into make_settings_strict 2016-02-03 13:21:49 +01:00
Simon Willnauer
4a4e523357 Merge branch 'master' into make_settings_strict 2016-02-03 11:34:12 +01:00
Robert Muir
d5dc05f69e Upgrade to lucene 5.5.0-snapshot-1725675 2016-02-02 22:53:39 -05:00
Lee Hinman
3918072362 Fix calling ensureOpen() on the wrong directory
Also removes ensureCanWrite since this already passes the
TRUNCATE_EXISTING flag when opening.

Adds a REST test that fails without this fix due to the classloader
isolation.

Additionally, move SmbDirectoryWrapper into an elasticsearch package instead of a lucene one, so this would be found at compile time instead of runtime.

Forward-port of #16383
2016-02-02 16:31:16 -07:00
Tal Levy
3191fc7347 Merge pull request #16355 from talevy/fix_ingest_exception
revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation
2016-02-02 14:11:24 -08:00
Tal Levy
0a1580eefa revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation 2016-02-02 14:08:22 -08:00
Martijn van Groningen
64ac037a82 Added an ingest qa that tests processor real world like configurations.
Renamed `ingest-with-mustache` to `smoke-test-ingest-with-all-dependencies`
Also renamed `ingest-disabled` to `smoke-test-ingest-disabled` so that the name is more inline with other qa smoke test modules.
2016-02-02 22:37:24 +01:00
Jack Conradson
54011da950 Fix imports. 2016-02-02 12:26:21 -08:00
Jack Conradson
58ad172492 Merge branch 'master' into tests 2016-02-02 11:48:27 -08:00
Jack Conradson
791417ca89 Fix test file name. 2016-02-02 11:48:19 -08:00
Tanguy Leroux
865bbc2096 Remove string formatting from Terminal print methods
This can be trappy and wrong formating strings can throws format exceptions and hide the real message.
2016-02-02 19:57:16 +01:00
Simon Willnauer
818a9eefb2 Make settings validation strict
This commit enableds strict settings validation on node startup. All settings
passed to elasticsearch either through system properties, yaml files or any other
way to pass settings must be registered and valid. Settings that are unknown ie. due to
typos or due to deprecation or removal will cause the node to NOT start up. Plugins
have to declare all their settings on the `SettingsModule#registerSetting` and settings for
plugins that are not installed must be removed.

This commit also removes the ability to specify the nodes name via `-Des.name` or just `name` in the
configuration files. The node name must be prefixed with the node prexif like `node.name: Boom`. Left over
usage of `name` will also cause startup to fail.
2016-02-02 11:32:44 +01:00
Ryan Ernst
a2c37c0989 CliTool: Allow unexpected exceptions to propagate
Cli tools currently catch all exceptions, and only print the exception
message, except when a special system property is set. Even with this
flag set, certain exceptions, like IOException, are captured and their
stack trace is always lost.

This change adds a UserError class, which can be used a cli tools to
specify a message to the user, as well as an exit status. All other
exceptions are propagated out of main, so java will exit with non-zero
and print the stack trace.
2016-02-01 16:35:22 -08:00
Jack Conradson
e3fd6c2a00 Removed ..= token from the Lexer. Fixed related tests. 2016-02-01 16:16:04 -08:00
Daniel Mitterdorfer
3bee2d3195 Migrate Azure settings to new settings infrastructure
With this commit we migrate all Azure related settings to
the new settings infrastructure.
2016-02-01 16:34:28 +01:00
Ryan Ernst
3787f437ec Merge branch 'master' into remove_multicast 2016-02-01 07:25:45 -08:00
Simon Willnauer
af0e40ec7d Merge pull request #16316 from s1monw/isseus/13685_2nd_try
Ensure all resoruces are closed on Node#close()
2016-02-01 10:17:57 +01:00
Jason Tedor
105411060c Uppercase ells ('L') in long literals
This commit removes and forbids the use of lowercase ells ('l') in long
literals because they are often hard to distinguish from the digit
representing one ('1').

Closes #16329
2016-01-30 22:16:02 -05:00
Ryan Ernst
b8f08c35ec Plugin: Remove multicast plugin
closes #16310.
2016-01-29 18:41:31 -08:00
Tal Levy
fca442f4d1 Introduce Pipeline Factory Error Responses in Node Ingest
When there is an exception thrown during pipeline creation within
Rest calls (in put pipeline, and simulate) We now return a structured
error response to the user with details around which processor's
configuration is the cause of the issue, or which configuration property
is misconfigured, etc.
2016-01-29 13:37:27 -08:00
Simon Willnauer
51745d7272 Call latch in a finally block 2016-01-29 17:38:33 +01:00
Simon Willnauer
e24fac644a Fix AzureRepositoryF to handle exceptions on close
Fix TribeUnitTests to handle exceptions on close
2016-01-29 17:34:02 +01:00
Martijn van Groningen
f5e89f7242 mappings: remove fly weight 2016-01-29 10:12:39 +01:00
Simon Willnauer
fea8676a6c remove dead code 2016-01-28 17:14:52 +01:00
Simon Willnauer
a149ebdb7b remove blanks 2016-01-28 17:01:01 +01:00
Simon Willnauer
687d1d83fa Convert multcast plugin settings to the new infra 2016-01-28 17:01:01 +01:00
Boaz Leskes
2a137b5548 Make index uuid available in Index, ShardRouting & ShardId
In the early days Elasticsearch used to use the index name as the index identity. Around 1.0.0 we introduced a unique index uuid which is stored in the index setting. Since then we used that uuid in a few places but it is by far not the main identifier when working with indices, partially because it's not always readily available in all places.

This PR start to make a move in the direction of using uuids instead of name by making sure that the uuid is available on the Index class (currently just a wrapper around the name) and as such also available via ShardRouting and ShardId.

Note that this is by no means an attempt to do the right thing with the uuid in all places. In almost all places it falls back to the name based comparison that was done before. It is meant as a first step towards slowly improving the situation.

Closes #16217
2016-01-28 08:40:10 +01:00
Jack Conradson
5b836dbb11 Renamed the scripting language Plan A to Painless.
Closes #16245
2016-01-27 10:37:34 -08:00
Simon Willnauer
7ff99eb89d Merge branch 'master' into trash_context_and_headers
if (name == 'expamle-fixtures') return
2016-01-27 16:33:50 +01:00
Jason Tedor
284cc3a048 Script mode settings as booleans
This commit modifies the accept values for script mode settings from
"on", "off", and "sandbox" to "true", "false", and "sandbox".
2016-01-27 06:26:58 -05:00
Jason Tedor
9944573449 Rename methods on ScriptEngineService
This commit method renames the ScriptEngineService interface methods
types, extensions, and sandboxed to getTypes, getExtensions, and
isSandboxed, respectively.
2016-01-27 06:26:04 -05:00
Jason Tedor
087e55cc51 Script mode settings
This commit converts the script mode settings to the new settings
infrastructure. This is a major refactoring of the handling of script
mode settings. This refactoring is necessary because these settings are
determined at runtime based on the registered script engines and the
registered script contexts.
2016-01-27 06:26:04 -05:00
Simon Willnauer
71c3e57aee Merge branch 'master' into trash_context_and_headers 2016-01-27 11:42:42 +01:00
Adrien Grand
35709f62b6 Be stricter about parsing boolean values in mappings.
Parsing is currently very lenient, which has the bad side-effect that if you
have a typo and pass eg. `store: fasle` this will actually be interpreted as
`store: true`. Since mappings can't be changed after the fact, it is quite bad
if it happens on an index that already contains data.

Note that this does not cover all settings that accept a boolean, but since the
PR was quite hard to build and already covers some main settirgs like `store`
or `doc_values` this would already be a good incremental improvement.
2016-01-27 09:06:00 +01:00
Martijn van Groningen
2b2552c301 test: cleanup static test resources 2016-01-26 21:17:05 +01:00
javanna
61630c2b27 migrate node.local and node.mode to new Setting infra 2016-01-26 14:40:46 +01:00
javanna
36d98478bf Merge branch 'master' into feature/ingest 2016-01-25 18:01:09 +01:00