Commit Graph

234 Commits

Author SHA1 Message Date
javanna 83fb0a10e5 Scripting: remove support for script.disable_dynamic setting
Now that fine-grained script settings are supported (#10116) we can remove support for the script.disable_dynamic setting.

Same result as `script.disable_dynamic: false` can be obtained as follows:

```
script.inline: on
script.indexed: on
```
An exception is thrown at startup when the old setting is set, so we make sure we tell users they have to change it rather than ignoring the setting.

Closes #10286
2015-03-31 13:24:52 +02:00
Patrick Peschlow be93884538 Update scripting.asciidoc
change description to better fit the flag name
2015-03-31 09:26:40 +02:00
javanna 0beda40069 [DOCS] added table with supported scripting languages to scripting docs 2015-03-29 11:28:50 +02:00
javanna 425ea5bca6 [DOCS] removed coming tags from scripting docs 2015-03-26 20:22:20 +01:00
javanna d9d1e6a67a Scripting: add support for fine-grained settings
Allow to on/off scripting based on their source (where they get loaded from), the  operation that executes them and their language.

The settings cover the following combinations:

- mode: on, off, sandbox
- source: indexed, dynamic, file
- engine: groovy, expressions, mustache, etc
- operation: update, search, aggs, mapping

The following settings are supported for every engine:

script.engine.groovy.indexed.update:    sandbox/on/off
script.engine.groovy.indexed.search:    sandbox/on/off
script.engine.groovy.indexed.aggs:      sandbox/on/off
script.engine.groovy.indexed.mapping:   sandbox/on/off
script.engine.groovy.dynamic.update:    sandbox/on/off
script.engine.groovy.dynamic.search:    sandbox/on/off
script.engine.groovy.dynamic.aggs:      sandbox/on/off
script.engine.groovy.dynamic.mapping:   sandbox/on/off
script.engine.groovy.file.update:       sandbox/on/off
script.engine.groovy.file.search:       sandbox/on/off
script.engine.groovy.file.aggs:         sandbox/on/off
script.engine.groovy.file.mapping:      sandbox/on/off

For ease of use, the following more generic settings are supported too:

script.indexed: sandbox/on/off
script.dynamic: sandbox/on/off
script.file:    sandbox/on/off

script.update:  sandbox/on/off
script.search:  sandbox/on/off
script.aggs:    sandbox/on/off
script.mapping: sandbox/on/off

These will be used to calculate the more specific settings, using the stricter setting of each combination. Operation based settings have precedence over conflicting source based ones.

Note that the `mustache` engine is affected by generic settings applied to any language, while native scripts aren't as they are static by definition.

Also, the previous `script.disable_dynamic` setting can now be deprecated.

Closes #6418
Closes #10116
Closes #10274
2015-03-26 19:56:55 +01:00
Glen Smith 5a475d21e5 [DOCS] Added explicit "lang" field to documentation of script score definition 2015-03-25 10:58:59 +01:00
Michael McCandless 34b397597c Core: increase default rate limiting for snapshot, restore and recovery to 40 MB/sec
This also fixes a possible issue that may cause over-throttling when
there are many small files being copied, which should be rare.

Closed #10185
2015-03-20 16:09:42 -04:00
javanna 9ca74c8217 [DOCS] clarify no-master-block docs
Closes #9739
2015-03-20 15:58:16 +01:00
Clinton Gormley aa94ced0ae Remove references to the thrift and memcached transport plugins
as they are no longer supported

Closes #10166
2015-03-19 20:49:58 +01:00
jaymode 105bdd486a [HTTP] add option to only return simple exception messages
Adds a setting to disable detailed error messages and full exception stack traces
in HTTP responses. When set to false, the error_trace request parameter will result
in a HTTP 400 response. When the error_trace parameter is not present, the message
of the first ElasticsearchException will be output and no nested exception messages
will be output.
2015-03-18 17:49:05 -04:00
Petr Bela f27cb07eb9 [DOCS] fix typo in scripting docs 2015-03-12 15:28:50 -07:00
Pius 430b091e7d Docs: Added default value
Added default value to `cluster.routing.allocation.node_initial_primaries_recoveries`

Closes #9955
2015-03-03 10:15:11 +01:00
IsaacHaze ee163e570b Docs: Update snapshots.asciidoc
Adds more determiners.

Closes #9673
2015-02-27 20:42:12 +01:00
Boaz Leskes e9dbfa9ee6 Transport: added a simple request tracer, logging incoming and outgoing requests
The request tracer logs in TRACE level under the `transport.tracer` log and is dynamically configurable with include and exclude arrays to filter out unneeded info. By default all requests are logged with the exception of fault detection pings (fired every second).

add the notion of tracers in the MockTransportService for testing purposes

Closes #9286
2015-02-25 21:33:57 +01:00
Igor Motov c5ebdf11bb Snapshot/Restore: add ability to retrieve currently running snapshots
Together with #8782 it should help in the situations simliar to #8887 by adding an ability to get information about currently running snapshot without accessing the repository itself.

Closes #8887
2015-02-25 11:06:32 -05:00
Colin Goodheart-Smithe 7d3856c9d3 [DOCS] update script docs to use preferred script parameter names 2015-02-23 11:16:28 +00:00
Clinton Gormley 57a4646776 Docs: Added note about groovy sandbox vulnerability to modules/scripting 2015-02-11 17:54:53 +01:00
Clinton Gormley eea22d7731 Docs: Fixed asciidoc error in snapshots.asciidoc 2015-01-29 20:57:12 +01:00
J Charitopoulos be8d8d658c Docs: minor syntax
Closes #9481
2015-01-29 20:27:20 +01:00
J Charitopoulos b359520849 Docs: Update snapshots.asciidoc
minor syntax

Closes #9457
2015-01-28 15:54:13 +01:00
Christian Verkerk 5b31189498 Docs: Update cluster.asciidoc
Clarify the preferencing.

Closes #9434
2015-01-27 10:48:40 +01:00
Igor Motov c0da353ef5 Snapshot/Restore: add support for changing index settings during restore process
Closes #7887
2015-01-20 15:49:47 -05:00
Michael McCandless def2d34f80 don't mention fixed throttling in the docs 2015-01-14 10:13:10 -05:00
Michael McCandless 107099affa put back fixed throttling, but off by default 2015-01-14 05:35:09 -05:00
Michael McCandless 1aad275c55 expose current CMS throttle in merge stats; fix tests, docs; also log per-merge stop/throttle/rate 2015-01-11 05:52:43 -05:00
Clinton Gormley 75cc7077c7 Update plugins.asciidoc
Added entity resolution plugin for duplication detection

Related to #9131
2015-01-05 12:53:37 +01:00
Peter Johnson a.k.a. insertcoffee 4b5e6b2de0 [docs] pedantry
Closes #8982
2014-12-17 13:46:39 +01:00
Simon Willnauer 1247774ff1 Remove Gateway abstraction
We only have a single gatweway since es 1.3. There is no need to keep all
these abstractsion and nested packages. We can fold most of it into simpler
structures.
2014-12-15 15:53:02 +01:00
Peter Fabian Mitchell b2bab05c29 HTTP: Add 'http.publish_port' setting to the HTTP module
This change adds a 'http.publish_port' setting to the HTTP module to configure
the port which HTTP clients should use when communicating with the node. This
is useful when running on a bridged network interface or when running behind
a proxy or firewall.

Closes #8807
Closes #8137
2014-12-11 16:10:07 +01:00
Matt Hughes afba977e80 Docs: Added swift openstack repository
Closes #8583
2014-11-25 13:49:15 +01:00
Lee Hinman 45408844e7 Remove NoneGateway, NoneGatewayAllocator, & NoneGatewayModule
Always use the LocalGateway* equivalents

We already check in the LocalGateway whether a node is a client node, or
is not master-eligible, and skip writing the state there. This allows us
to remove this code that was previously used only for tribe nodes (which
are not master eligible anyway and wouldn't write state) and in
tests (which can shake more bugs out)
2014-11-24 12:22:05 +01:00
Laurent Broudoux feb465f26f Docs: Update plugins.asciidoc on river plugins section
Adding links to Amazon S3 and Google Drive river plugins

Closes #8544
2014-11-24 12:15:12 +01:00
Mariam Hakobyan 4a1ab6543c Docs: Added new elasticsearch-river-kafka plugin to the documentation (uses latest version of Kafka, EL Bulk API, and supports concurrent requests)
Closes #8518
2014-11-23 12:28:45 +01:00
Mark Walkom bfd1bcd30a Updated threadpool documentation to elaborate/clarify what the pools are for and their values
Closes #8446
2014-11-12 22:33:38 +11:00
Clinton Gormley cff544dcc2 Docs: Removed old coming/added tags 2014-11-10 14:41:24 +01:00
Veres Lajos 4059e4ac86 typo fixes - https://github.com/vlajos/misspell_fixer
Closes #8323
2014-11-08 18:55:57 +01:00
Clinton Gormley 5797682bd0 Update cluster.asciidoc - fix invalid asciidoc 2014-11-04 15:22:36 +01:00
Clinton Gormley 60eaeb5052 Update cluster.asciidoc
Fixed asciidoc on cluster module page
2014-11-04 14:32:05 +01:00
Clinton Gormley b0e5fb7823 Update zen.asciidoc
Tidied up the "No master block" asciidoc
2014-11-04 14:27:22 +01:00
Alexander Reelsen c04fa43587 Docs: Convert markdown to asciidoc in transport profile docs 2014-11-02 08:25:45 +01:00
Alexander Reelsen 5eeac2fdf6 Netty: Add HTTP pipelining support
This adds HTTP pipelining support to netty. Previously pipelining was not
supported due to the asynchronous nature of elasticsearch. The first request
that was returned by Elasticsearch, was returned as first response,
regardless of the correct order.

The solution to this problem is to add a handler to the netty pipeline
that maintains an ordered list and thus orders the responses before
returning them to the client. This means, we will always have some state
on the server side and also requires some memory in order to keep the
responses there.

Pipelining is enabled by default, but can be configured by setting the
http.pipelining property to true|false. In addition the maximum size of
the event queue can be configured.

The initial netty handler is copied from this repo
https://github.com/typesafehub/netty-http-pipelining

Closes #2665
2014-10-31 16:30:11 +01:00
Clinton Gormley fbd0403a6f Documented that HTTP pipelining is not supported 2014-10-27 14:49:48 +01:00
Simon Willnauer d5c0a49620 [ROUTING] Add rebalance enabled allocation decider
This commit adds the ability to enable / disable relocations
on an entire cluster or on individual indices for either:

 * `primaries` - only primaries can rebalance
 * `replica` - only replicas can rebalance
 * `all` - everything can rebalance (default)
 * `none` - all rebalances are disabled

similar to the allocation enable / disable functionality.

Relates to #7288
2014-10-23 14:07:13 +02:00
Clinton Gormley 2d0c440b09 Update cluster.asciidoc
Fixed asciidoc syntax
2014-10-22 12:45:10 +02:00
Peter Dyson b984cb771f Docs: Provide example of deleting a repository
Example of deleting a repository with explanation that snapshots themselves are left untouched.

Closes #8172
2014-10-21 10:10:03 +02:00
Clinton Gormley 4da62a33b5 Update plugins.asciidoc
Added Vietnamese Analyser to plugins page

Closes #6647
2014-10-20 11:56:01 +02:00
Alexander Reelsen 37e606819c Test: Fix netty multiport tests
Ensuring, that even in local mode, netty is started appropriately.
2014-10-17 16:16:04 -07:00
Alexander Reelsen b06b52449f Netty: Support to bind on multiple host/port pairs
This patch allows to create several netty bootstrap, each of which
listening on different ports. This will potentially allow for features
to listen to different network interfaces for node-to-node or node-to-client
communication and is also the base to listen to several interfaces, so that those
can be used to speed up cluster communication in the future.

Closes #8098
2014-10-17 13:55:07 -07:00
Clinton Gormley 646f7acf5c Docs: Documented the http.*host and transport.*host settings 2014-10-17 15:01:42 +02:00
Patrick Peschlow 9b03fda5d7 Docs: fix typo
Closes #8111
2014-10-16 20:48:57 +02:00
Josh Kuhn f4d9cce659 Docs: Added RethinkDB River to list of community plugins
Closes #8016
2014-10-16 20:15:33 +02:00
Nik Everett 5dc5849922 Docs: Add plugin to list
This plugin provides trigram accelerated regular expression search for when
you need reasonably fast arbitrary matching.

Closes #7997
2014-10-15 13:56:35 +02:00
Igor Motov 45d3842214 Docs: clarify wait_for_completion flag in snapshot documentation
Closes #7952
2014-10-08 08:43:06 -04:00
Igor Motov 2359e33b1e Docs: fix formatting in snapshots documentation 2014-10-07 12:33:21 -04:00
Igor Motov 555bfcb02b [SNAPSHOT] Add repository validation
Fixes #7096
2014-10-07 10:50:16 -04:00
Martijn van Groningen fca406415d Docs: Document the most important changes to zen discovery.
Closes #7746
2014-09-30 13:05:45 +02:00
Paul Pearcy 336ef4d468 Clarify s3 snapshot compress behavior
Clarify s3 snapshot compress behavior only applies to metadata and no index files.

Closes #7654
2014-09-28 11:11:19 +02:00
Christian Verkerk 3efa342ee4 Improve sentence structure
Closes #7842
2014-09-28 11:07:15 +02:00
Clinton Gormley cb00d4a542 Docs: Removed all the added/deprecated tags from 1.x 2014-09-26 21:04:42 +02:00
Shay Banon a82d486bda Add a listener thread pool
Today, when executing an action (mainly when using the Java API), a listener threaded flag can be set to true in order to execute the listener on a different thread pool. Today, this thread pool is the generic thread pool, which is cached. This can create problems for Java clients (mainly) around potential thread explosion.
Introduce a new thread pool called listener, that is fixed sized and defaults to the half the cores maxed at 10, and use it where listeners are executed.
relates to #5152
closes #7837
2014-09-25 11:25:13 +02:00
Zachary Wasserman e3c7f28003 Docs: Update thrift.asciidoc
Fix typos, add clarifications and link.

Closes #7678
2014-09-24 20:50:30 +02:00
Lee Hinman 1dd26888f6 [DOCS] Additional documentation for _score accessing
Closes #7043
2014-09-11 12:53:25 +02:00
ansj fee832df66 Docs: Update plugins.asciidoc
add search by sql

Closes #7525
2014-09-09 14:57:38 +02:00
Alexander Reelsen bd0eb32d9c CORS: Disable by default
In order to deliver a more secure out-of-the-box configuration this commit
disables cross-origin resource sharing by default.

Closes #7151
2014-09-09 11:09:03 +02:00
Clinton Gormley 3bda779336 Docs: Update scripting.asciidoc
Clarify that doc[...] does not return objects

Closes #7384
2014-09-07 11:39:33 +02:00
Simon Willnauer 7f32e8c707 [STORE] Simplify reading / writing from and to BlobContainer
BlobContainer used to provide async APIs which are not used
internally. The implementation of these APIs are also not async
by nature and neither is any of the pluggable BlobContainers. This
commit simplifies the API to a simple input / output stream and
reduces the hierarchy of BlobContainer dramatically.
NOTE: This is a breaking change!

Closes #7551
2014-09-05 21:40:20 +02:00
Adrien Grand ea96359d82 Facets: Removal from master.
Close #7337
2014-08-21 10:34:39 +02:00
Carsten Brandt 6a6c45e2d0 Docs: Update snapshots.asciidoc
Added a link so you directly get to the page that tells how to open/close an index.

Closes #7286
2014-08-18 13:04:25 +02:00
Lee Hinman 4dc060527c Add GroovyCollections to the sandbox whitelist
Also clarify in the docs that changing the whitelist/blacklist settings
replace the list, they don't add to it.

Fixes #7089
Fixes #7088
2014-08-13 14:47:49 +02:00
Alexander Reelsen 35e67c84fa CORS: Allowed to configure allow-credentials header to work via SSL
This adds support to return the "Access-Control-Allow-Credentials" header
if needed, so CORS will work flawlessly with authenticated applications.

Closes #6380
2014-08-05 17:33:06 +02:00
Jun Ohtani 5be8aecd10 add Proxy setting for using plugin command
Closes #7150
2014-08-05 12:44:02 +02:00
Shay Banon 95762e8126 Support "default" for tcpNoDelay and tcpKeepAlive
Allow to set the value default to network.tcp.no_delay and network.tcp.keep_alive so they won't be set at all, since on solaris, setting tcpNoDelay can actually cause failure
relates to #7115
2014-08-02 17:32:41 +02:00
gabriel-tessier eaac8141cc Docs: Fix typo in scripting.asciidoc
Replace the mvel by groovy in the forgotten place.
I add the previous change in this one.
Sorry for the spam!

Closes #7071
2014-07-29 12:30:09 +02:00
gabriel-tessier c2c2190d27 Docs: Fix typo in scripting.asciidoc
Closes #7070
2014-07-29 12:28:41 +02:00
rendel 50634e6a3d Docs: Added new entry for the SIREn plugin.
Closes #6961
2014-07-25 12:49:50 +02:00
Alexander Reelsen a1e335b1e9 CORS: Support regular expressions for origin to match against
This commit adds regular expression support for the allow-origin
header depending on the value of the request `Origin` header.

The existing HttpRequestBuilder is also extended to support the
OPTIONS HTTP method.

Relates #5601
Closes #6891
2014-07-25 10:51:22 +02:00
Lee Hinman 1fb9f404df [DOCS] correct documentation about groovy/mvel defaults and deprecations 2014-07-25 10:39:33 +02:00
Simon Willnauer 5bfea56457 [DOCS] move all coming tags to added in master 2014-07-23 16:37:19 +02:00
Brian Murphy b98f19a54b [DOCS] Fix typo 2014-07-22 14:51:31 +01:00
Brian Murphy 3c5de7d4a1 [DOCS] Fix indentation 2014-07-22 14:49:45 +01:00
Brian Murphy e3b1aed0fc [DOCS] Update examples to groovy. 2014-07-22 14:45:46 +01:00
Clinton Gormley 8aefaef68a Update scripting.asciidoc
Added an ID for native java scripts
2014-07-22 11:36:40 +02:00
Peter Kim 6a25d9b7b5 [DOCS] Fixed typos 2014-07-17 15:25:34 -04:00
Brian Murphy cbd2a97abd [DOCS] : Indexed scripts/templates
These are the docs for the indexed scripts/templates feature.
Also moved the namespace for the REST endpoints.

Closes #6851
2014-07-16 10:49:02 +01:00
Ryan Ernst 64ab22816c Scripting: Add script engine for lucene expressions.
These are javascript expressions, which can only access numeric
fielddata, parameters, and _score. They can only be used for searches (not document updates).

closes #6818
2014-07-15 07:49:01 -07:00
Igor Motov 60b317caa4 Snapshot/Restore: Add ability to restore indices without their aliases
Closes #6457
2014-07-13 17:52:41 +09:00
Boaz Leskes f480969503 [Gateway] set a default of 5m to `recover_after_time` when any to the `expected*Nodes` is set
The `recovery_after_time` tells the gateway to wait before starting recovery from disk. The goal here is to allow for more nodes to join the cluster and thus not start potentially unneeded replications. The `expectedNodes` setting (and friends) tells the gateway when it can start recovering even if the `recover_after_time` has not yet elapsed. However, `expectedNodes` is useless if one doesn't set `recovery_after_time`. This commit changes that by setting a sensible default of 5m for `recover_after_time` *if* a `expectedNodes` setting is present.

Closes #6742
2014-07-11 11:28:45 +02:00
Iulia Pasov eed3513c37 Docs: Update plugins.asciidoc to fix typo
Changed the name of the European Environment Agency (from European Environmental Agency)

Closes #6807
2014-07-10 14:04:26 +02:00
Iulia Pasov a79d0744d3 Docs: Update plugins.asciidoc
Closes #6683
2014-07-09 16:15:59 +02:00
Shinsuke Sugaya 4bddb4e346 Update plugins.asciidoc 2014-07-05 00:44:02 +09:00
Shikhar Bhushan 1e894111b0 Docs: Link to eskka discovery plugin from doc
Closes #6721
2014-07-04 17:06:51 +02:00
Igor Motov 1425e28639 Add ability to restore partial snapshots
Closes #5742
2014-06-30 20:18:02 -04:00
Lee Hinman b43b56a6a8 Add a transformer to translate constant BigDecimal to double 2014-06-26 10:52:28 +02:00
Lee Hinman 5c6d28240f Switch to Groovy as the default scripting language
This is a breaking change to move from MVEL -> Groovy
2014-06-25 12:15:12 +02:00
Volker Fröhlich 06192686a2 [DOCS] Fixd typo in http.asciidoc 2014-06-16 10:42:34 +02:00
Shay Banon 2330421816 Wait till node is part of cluster state for join process
When a node sends a join request to the master, only send back the response after it has been added to the master cluster state and published.
This will fix the rare cases where today, a join request can return, and the master, since its under load, have not yet added the node to its cluster state, and the node that joined will start a fault detect against the master, failing since its not part of the cluster state.
Since now the join request is longer, also increase the join request timeout default.
closes #6480
2014-06-12 18:15:51 +02:00
Igor Motov 56a264cf6d [DOCS] Snapshot/restore: add more information about snapshot and restore monitoring 2014-06-11 20:52:45 -04:00
Gaurav Arora 4a3837acf0 [DOCS] fix typo in network module docs 2014-06-03 13:19:36 +02:00
salyh 27b38818b7 [DOCS] Add imap river and security plugin to docs 2014-06-03 10:59:50 +02:00
Shay Banon 9c98bb3554 Have a dedicated join timeout that is higher than ping.timeout for node join
Using ping.timeout, which defaults to 3s, to use as a timeout value on the join request a node makes to the master once its discovered can be too small, specifically when there is a large cluster state involved (and by definition, all the buffers and such on the nio layer will be "cold"). Introduce a dedicated join.timeout setting, that by default is 10x the ping.timeout (so 30s by default).
closes #6342
2014-05-30 12:42:08 +02:00