Commit Graph

397 Commits

Author SHA1 Message Date
Jack Conradson aa968f6b65 Scripting: Add Field Methods
Added infrastructure to allow basic member methods in the expressions
language to be called.  The methods must have a signature with no arguments.  Also
added the following member methods for date fields (and it should be easy to add more)
* getYear
* getMonth
* getDayOfMonth
* getHourOfDay
* getMinutes
* getSeconds

Allow fields to be accessed without using the member variable [value].
(Note that both ways can be used to access fields for back-compat.)

closes #10890
2015-04-30 15:36:46 -07:00
minde-eagleeye a1289b4ad5 Docs: Update cluster.asciidoc
added a missing comma in one of examples

Closes #10834
2015-04-28 11:48:08 +02:00
javanna c914134355 Scripting: remove groovy sandbox
Groovy sandboxing was disabled by default from 1.4.3 on though since we found out that it could be worked around, so it makes little sense to keep it and maintain it.

Closes #10156
Closes #10480
2015-04-28 11:27:50 +02:00
Clinton Gormley 089914dede Docs: Document `http.max_header_size`
Closes #10752
2015-04-27 15:59:27 +02:00
Clinton Gormley 37ed61807f Docs: Updated the experimental annotations in the docs as follows:
* Removed the docs for `index.compound_format` and `index.compound_on_flush` - these are expert settings which should probably be removed (see https://github.com/elastic/elasticsearch/issues/10778)
* Removed the docs for `index.index_concurrency` - another expert setting
* Labelled the segments verbose output as experimental
* Marked the `compression`, `precision_threshold` and `rehash` options as experimental in the cardinality and percentile aggs
* Improved the experimental text on `significant_terms`, `execution_hint` in the terms agg, and `terminate_after` param on count and search
* Removed the experimental flag on the `geobounds` agg
* Marked the settings in the `merge` and `store` modules as experimental, rather than the modules themselves

Closes #10782
2015-04-26 18:49:15 +02:00
Alexander dbbfe39415 [Docs] fix typo in scripting module
Closes #10622
2015-04-23 14:00:44 +02:00
Clinton Gormley abc7de96ae Docs: Updated version annotations in master 2015-04-09 14:50:11 +02:00
David Pilato 88ee7a5dca Deprecate rivers
* In code, we mark `River`, `AbstractRiverComponent`, `RiverComponent` and `RiverName` classes as deprecated
* We log that information when a cluster is still using it
* We add this information in the plugins list as well
2015-04-09 14:29:16 +02:00
javanna 7bd7ea8f13 Scripting: allow plugins to define custom operations that they use scripts for
Plugins can now define multiple operations/contexts that they use scripts for. Fine-grained settings can then be used to enable/disable scripts based on each single registered context.

Also added a new generic category called `plugin`, which will be used as a default when the context is not specified. This allows us to restore backwards compatibility for plugins on `ScriptService` by restoring the old methods that don't require the script context and making them internally use the `plugin` context, as they can only be called from plugins.

Closes #10347
Closes #10419
2015-04-08 11:57:00 +02:00
Lee Hinman eed7c8af6d [DOCS] Document `indices.recovery.concurrent_small_file_streams` 2015-04-06 11:16:50 -06:00
Dustin Shiver ae60144123 Update for clarification
Make it clear which nodes in the cluster should have `http.enabled` set to `false`.

Closes #10305
2015-04-05 18:05:14 +02:00
Tanguy Leroux eeec90be79 [DOCS] Add verify parameter to snapshot documentation
Add verify parameter to snapshot documentation and remove 'verify' setting at FS repository level (not supported)
2015-04-02 14:23:31 +02:00
David Wittman 3acf4ccb33 Fix typos for gateway.recover_after_time
There were a few references to the setting
`gateway.recovery_after_time`, which should instead be
`gateway.recover_after_time`.
2015-03-31 14:00:08 -05:00
javanna 83fb0a10e5 Scripting: remove support for script.disable_dynamic setting
Now that fine-grained script settings are supported (#10116) we can remove support for the script.disable_dynamic setting.

Same result as `script.disable_dynamic: false` can be obtained as follows:

```
script.inline: on
script.indexed: on
```
An exception is thrown at startup when the old setting is set, so we make sure we tell users they have to change it rather than ignoring the setting.

Closes #10286
2015-03-31 13:24:52 +02:00
Patrick Peschlow be93884538 Update scripting.asciidoc
change description to better fit the flag name
2015-03-31 09:26:40 +02:00
javanna 0beda40069 [DOCS] added table with supported scripting languages to scripting docs 2015-03-29 11:28:50 +02:00
javanna 425ea5bca6 [DOCS] removed coming tags from scripting docs 2015-03-26 20:22:20 +01:00
javanna d9d1e6a67a Scripting: add support for fine-grained settings
Allow to on/off scripting based on their source (where they get loaded from), the  operation that executes them and their language.

The settings cover the following combinations:

- mode: on, off, sandbox
- source: indexed, dynamic, file
- engine: groovy, expressions, mustache, etc
- operation: update, search, aggs, mapping

The following settings are supported for every engine:

script.engine.groovy.indexed.update:    sandbox/on/off
script.engine.groovy.indexed.search:    sandbox/on/off
script.engine.groovy.indexed.aggs:      sandbox/on/off
script.engine.groovy.indexed.mapping:   sandbox/on/off
script.engine.groovy.dynamic.update:    sandbox/on/off
script.engine.groovy.dynamic.search:    sandbox/on/off
script.engine.groovy.dynamic.aggs:      sandbox/on/off
script.engine.groovy.dynamic.mapping:   sandbox/on/off
script.engine.groovy.file.update:       sandbox/on/off
script.engine.groovy.file.search:       sandbox/on/off
script.engine.groovy.file.aggs:         sandbox/on/off
script.engine.groovy.file.mapping:      sandbox/on/off

For ease of use, the following more generic settings are supported too:

script.indexed: sandbox/on/off
script.dynamic: sandbox/on/off
script.file:    sandbox/on/off

script.update:  sandbox/on/off
script.search:  sandbox/on/off
script.aggs:    sandbox/on/off
script.mapping: sandbox/on/off

These will be used to calculate the more specific settings, using the stricter setting of each combination. Operation based settings have precedence over conflicting source based ones.

Note that the `mustache` engine is affected by generic settings applied to any language, while native scripts aren't as they are static by definition.

Also, the previous `script.disable_dynamic` setting can now be deprecated.

Closes #6418
Closes #10116
Closes #10274
2015-03-26 19:56:55 +01:00
Glen Smith 5a475d21e5 [DOCS] Added explicit "lang" field to documentation of script score definition 2015-03-25 10:58:59 +01:00
Michael McCandless 34b397597c Core: increase default rate limiting for snapshot, restore and recovery to 40 MB/sec
This also fixes a possible issue that may cause over-throttling when
there are many small files being copied, which should be rare.

Closed #10185
2015-03-20 16:09:42 -04:00
javanna 9ca74c8217 [DOCS] clarify no-master-block docs
Closes #9739
2015-03-20 15:58:16 +01:00
Clinton Gormley aa94ced0ae Remove references to the thrift and memcached transport plugins
as they are no longer supported

Closes #10166
2015-03-19 20:49:58 +01:00
jaymode 105bdd486a [HTTP] add option to only return simple exception messages
Adds a setting to disable detailed error messages and full exception stack traces
in HTTP responses. When set to false, the error_trace request parameter will result
in a HTTP 400 response. When the error_trace parameter is not present, the message
of the first ElasticsearchException will be output and no nested exception messages
will be output.
2015-03-18 17:49:05 -04:00
Petr Bela f27cb07eb9 [DOCS] fix typo in scripting docs 2015-03-12 15:28:50 -07:00
Pius 430b091e7d Docs: Added default value
Added default value to `cluster.routing.allocation.node_initial_primaries_recoveries`

Closes #9955
2015-03-03 10:15:11 +01:00
IsaacHaze ee163e570b Docs: Update snapshots.asciidoc
Adds more determiners.

Closes #9673
2015-02-27 20:42:12 +01:00
Boaz Leskes e9dbfa9ee6 Transport: added a simple request tracer, logging incoming and outgoing requests
The request tracer logs in TRACE level under the `transport.tracer` log and is dynamically configurable with include and exclude arrays to filter out unneeded info. By default all requests are logged with the exception of fault detection pings (fired every second).

add the notion of tracers in the MockTransportService for testing purposes

Closes #9286
2015-02-25 21:33:57 +01:00
Igor Motov c5ebdf11bb Snapshot/Restore: add ability to retrieve currently running snapshots
Together with #8782 it should help in the situations simliar to #8887 by adding an ability to get information about currently running snapshot without accessing the repository itself.

Closes #8887
2015-02-25 11:06:32 -05:00
Colin Goodheart-Smithe 7d3856c9d3 [DOCS] update script docs to use preferred script parameter names 2015-02-23 11:16:28 +00:00
Clinton Gormley 57a4646776 Docs: Added note about groovy sandbox vulnerability to modules/scripting 2015-02-11 17:54:53 +01:00
Clinton Gormley eea22d7731 Docs: Fixed asciidoc error in snapshots.asciidoc 2015-01-29 20:57:12 +01:00
J Charitopoulos be8d8d658c Docs: minor syntax
Closes #9481
2015-01-29 20:27:20 +01:00
J Charitopoulos b359520849 Docs: Update snapshots.asciidoc
minor syntax

Closes #9457
2015-01-28 15:54:13 +01:00
Christian Verkerk 5b31189498 Docs: Update cluster.asciidoc
Clarify the preferencing.

Closes #9434
2015-01-27 10:48:40 +01:00
Igor Motov c0da353ef5 Snapshot/Restore: add support for changing index settings during restore process
Closes #7887
2015-01-20 15:49:47 -05:00
Michael McCandless def2d34f80 don't mention fixed throttling in the docs 2015-01-14 10:13:10 -05:00
Michael McCandless 107099affa put back fixed throttling, but off by default 2015-01-14 05:35:09 -05:00
Michael McCandless 1aad275c55 expose current CMS throttle in merge stats; fix tests, docs; also log per-merge stop/throttle/rate 2015-01-11 05:52:43 -05:00
Clinton Gormley 75cc7077c7 Update plugins.asciidoc
Added entity resolution plugin for duplication detection

Related to #9131
2015-01-05 12:53:37 +01:00
Peter Johnson a.k.a. insertcoffee 4b5e6b2de0 [docs] pedantry
Closes #8982
2014-12-17 13:46:39 +01:00
Simon Willnauer 1247774ff1 Remove Gateway abstraction
We only have a single gatweway since es 1.3. There is no need to keep all
these abstractsion and nested packages. We can fold most of it into simpler
structures.
2014-12-15 15:53:02 +01:00
Peter Fabian Mitchell b2bab05c29 HTTP: Add 'http.publish_port' setting to the HTTP module
This change adds a 'http.publish_port' setting to the HTTP module to configure
the port which HTTP clients should use when communicating with the node. This
is useful when running on a bridged network interface or when running behind
a proxy or firewall.

Closes #8807
Closes #8137
2014-12-11 16:10:07 +01:00
Matt Hughes afba977e80 Docs: Added swift openstack repository
Closes #8583
2014-11-25 13:49:15 +01:00
Lee Hinman 45408844e7 Remove NoneGateway, NoneGatewayAllocator, & NoneGatewayModule
Always use the LocalGateway* equivalents

We already check in the LocalGateway whether a node is a client node, or
is not master-eligible, and skip writing the state there. This allows us
to remove this code that was previously used only for tribe nodes (which
are not master eligible anyway and wouldn't write state) and in
tests (which can shake more bugs out)
2014-11-24 12:22:05 +01:00
Laurent Broudoux feb465f26f Docs: Update plugins.asciidoc on river plugins section
Adding links to Amazon S3 and Google Drive river plugins

Closes #8544
2014-11-24 12:15:12 +01:00
Mariam Hakobyan 4a1ab6543c Docs: Added new elasticsearch-river-kafka plugin to the documentation (uses latest version of Kafka, EL Bulk API, and supports concurrent requests)
Closes #8518
2014-11-23 12:28:45 +01:00
Mark Walkom bfd1bcd30a Updated threadpool documentation to elaborate/clarify what the pools are for and their values
Closes #8446
2014-11-12 22:33:38 +11:00
Clinton Gormley cff544dcc2 Docs: Removed old coming/added tags 2014-11-10 14:41:24 +01:00
Veres Lajos 4059e4ac86 typo fixes - https://github.com/vlajos/misspell_fixer
Closes #8323
2014-11-08 18:55:57 +01:00
Clinton Gormley 5797682bd0 Update cluster.asciidoc - fix invalid asciidoc 2014-11-04 15:22:36 +01:00
Clinton Gormley 60eaeb5052 Update cluster.asciidoc
Fixed asciidoc on cluster module page
2014-11-04 14:32:05 +01:00
Clinton Gormley b0e5fb7823 Update zen.asciidoc
Tidied up the "No master block" asciidoc
2014-11-04 14:27:22 +01:00
Alexander Reelsen c04fa43587 Docs: Convert markdown to asciidoc in transport profile docs 2014-11-02 08:25:45 +01:00
Alexander Reelsen 5eeac2fdf6 Netty: Add HTTP pipelining support
This adds HTTP pipelining support to netty. Previously pipelining was not
supported due to the asynchronous nature of elasticsearch. The first request
that was returned by Elasticsearch, was returned as first response,
regardless of the correct order.

The solution to this problem is to add a handler to the netty pipeline
that maintains an ordered list and thus orders the responses before
returning them to the client. This means, we will always have some state
on the server side and also requires some memory in order to keep the
responses there.

Pipelining is enabled by default, but can be configured by setting the
http.pipelining property to true|false. In addition the maximum size of
the event queue can be configured.

The initial netty handler is copied from this repo
https://github.com/typesafehub/netty-http-pipelining

Closes #2665
2014-10-31 16:30:11 +01:00
Clinton Gormley fbd0403a6f Documented that HTTP pipelining is not supported 2014-10-27 14:49:48 +01:00
Simon Willnauer d5c0a49620 [ROUTING] Add rebalance enabled allocation decider
This commit adds the ability to enable / disable relocations
on an entire cluster or on individual indices for either:

 * `primaries` - only primaries can rebalance
 * `replica` - only replicas can rebalance
 * `all` - everything can rebalance (default)
 * `none` - all rebalances are disabled

similar to the allocation enable / disable functionality.

Relates to #7288
2014-10-23 14:07:13 +02:00
Clinton Gormley 2d0c440b09 Update cluster.asciidoc
Fixed asciidoc syntax
2014-10-22 12:45:10 +02:00
Peter Dyson b984cb771f Docs: Provide example of deleting a repository
Example of deleting a repository with explanation that snapshots themselves are left untouched.

Closes #8172
2014-10-21 10:10:03 +02:00
Clinton Gormley 4da62a33b5 Update plugins.asciidoc
Added Vietnamese Analyser to plugins page

Closes #6647
2014-10-20 11:56:01 +02:00
Alexander Reelsen 37e606819c Test: Fix netty multiport tests
Ensuring, that even in local mode, netty is started appropriately.
2014-10-17 16:16:04 -07:00
Alexander Reelsen b06b52449f Netty: Support to bind on multiple host/port pairs
This patch allows to create several netty bootstrap, each of which
listening on different ports. This will potentially allow for features
to listen to different network interfaces for node-to-node or node-to-client
communication and is also the base to listen to several interfaces, so that those
can be used to speed up cluster communication in the future.

Closes #8098
2014-10-17 13:55:07 -07:00
Clinton Gormley 646f7acf5c Docs: Documented the http.*host and transport.*host settings 2014-10-17 15:01:42 +02:00
Patrick Peschlow 9b03fda5d7 Docs: fix typo
Closes #8111
2014-10-16 20:48:57 +02:00
Josh Kuhn f4d9cce659 Docs: Added RethinkDB River to list of community plugins
Closes #8016
2014-10-16 20:15:33 +02:00
Nik Everett 5dc5849922 Docs: Add plugin to list
This plugin provides trigram accelerated regular expression search for when
you need reasonably fast arbitrary matching.

Closes #7997
2014-10-15 13:56:35 +02:00
Igor Motov 45d3842214 Docs: clarify wait_for_completion flag in snapshot documentation
Closes #7952
2014-10-08 08:43:06 -04:00
Igor Motov 2359e33b1e Docs: fix formatting in snapshots documentation 2014-10-07 12:33:21 -04:00
Igor Motov 555bfcb02b [SNAPSHOT] Add repository validation
Fixes #7096
2014-10-07 10:50:16 -04:00
Martijn van Groningen fca406415d Docs: Document the most important changes to zen discovery.
Closes #7746
2014-09-30 13:05:45 +02:00
Paul Pearcy 336ef4d468 Clarify s3 snapshot compress behavior
Clarify s3 snapshot compress behavior only applies to metadata and no index files.

Closes #7654
2014-09-28 11:11:19 +02:00
Christian Verkerk 3efa342ee4 Improve sentence structure
Closes #7842
2014-09-28 11:07:15 +02:00
Clinton Gormley cb00d4a542 Docs: Removed all the added/deprecated tags from 1.x 2014-09-26 21:04:42 +02:00
Shay Banon a82d486bda Add a listener thread pool
Today, when executing an action (mainly when using the Java API), a listener threaded flag can be set to true in order to execute the listener on a different thread pool. Today, this thread pool is the generic thread pool, which is cached. This can create problems for Java clients (mainly) around potential thread explosion.
Introduce a new thread pool called listener, that is fixed sized and defaults to the half the cores maxed at 10, and use it where listeners are executed.
relates to #5152
closes #7837
2014-09-25 11:25:13 +02:00
Zachary Wasserman e3c7f28003 Docs: Update thrift.asciidoc
Fix typos, add clarifications and link.

Closes #7678
2014-09-24 20:50:30 +02:00
Lee Hinman 1dd26888f6 [DOCS] Additional documentation for _score accessing
Closes #7043
2014-09-11 12:53:25 +02:00
ansj fee832df66 Docs: Update plugins.asciidoc
add search by sql

Closes #7525
2014-09-09 14:57:38 +02:00
Alexander Reelsen bd0eb32d9c CORS: Disable by default
In order to deliver a more secure out-of-the-box configuration this commit
disables cross-origin resource sharing by default.

Closes #7151
2014-09-09 11:09:03 +02:00
Clinton Gormley 3bda779336 Docs: Update scripting.asciidoc
Clarify that doc[...] does not return objects

Closes #7384
2014-09-07 11:39:33 +02:00
Simon Willnauer 7f32e8c707 [STORE] Simplify reading / writing from and to BlobContainer
BlobContainer used to provide async APIs which are not used
internally. The implementation of these APIs are also not async
by nature and neither is any of the pluggable BlobContainers. This
commit simplifies the API to a simple input / output stream and
reduces the hierarchy of BlobContainer dramatically.
NOTE: This is a breaking change!

Closes #7551
2014-09-05 21:40:20 +02:00
Adrien Grand ea96359d82 Facets: Removal from master.
Close #7337
2014-08-21 10:34:39 +02:00
Carsten Brandt 6a6c45e2d0 Docs: Update snapshots.asciidoc
Added a link so you directly get to the page that tells how to open/close an index.

Closes #7286
2014-08-18 13:04:25 +02:00
Lee Hinman 4dc060527c Add GroovyCollections to the sandbox whitelist
Also clarify in the docs that changing the whitelist/blacklist settings
replace the list, they don't add to it.

Fixes #7089
Fixes #7088
2014-08-13 14:47:49 +02:00
Alexander Reelsen 35e67c84fa CORS: Allowed to configure allow-credentials header to work via SSL
This adds support to return the "Access-Control-Allow-Credentials" header
if needed, so CORS will work flawlessly with authenticated applications.

Closes #6380
2014-08-05 17:33:06 +02:00
Jun Ohtani 5be8aecd10 add Proxy setting for using plugin command
Closes #7150
2014-08-05 12:44:02 +02:00
Shay Banon 95762e8126 Support "default" for tcpNoDelay and tcpKeepAlive
Allow to set the value default to network.tcp.no_delay and network.tcp.keep_alive so they won't be set at all, since on solaris, setting tcpNoDelay can actually cause failure
relates to #7115
2014-08-02 17:32:41 +02:00
gabriel-tessier eaac8141cc Docs: Fix typo in scripting.asciidoc
Replace the mvel by groovy in the forgotten place.
I add the previous change in this one.
Sorry for the spam!

Closes #7071
2014-07-29 12:30:09 +02:00
gabriel-tessier c2c2190d27 Docs: Fix typo in scripting.asciidoc
Closes #7070
2014-07-29 12:28:41 +02:00
rendel 50634e6a3d Docs: Added new entry for the SIREn plugin.
Closes #6961
2014-07-25 12:49:50 +02:00
Alexander Reelsen a1e335b1e9 CORS: Support regular expressions for origin to match against
This commit adds regular expression support for the allow-origin
header depending on the value of the request `Origin` header.

The existing HttpRequestBuilder is also extended to support the
OPTIONS HTTP method.

Relates #5601
Closes #6891
2014-07-25 10:51:22 +02:00
Lee Hinman 1fb9f404df [DOCS] correct documentation about groovy/mvel defaults and deprecations 2014-07-25 10:39:33 +02:00
Simon Willnauer 5bfea56457 [DOCS] move all coming tags to added in master 2014-07-23 16:37:19 +02:00
Brian Murphy b98f19a54b [DOCS] Fix typo 2014-07-22 14:51:31 +01:00
Brian Murphy 3c5de7d4a1 [DOCS] Fix indentation 2014-07-22 14:49:45 +01:00
Brian Murphy e3b1aed0fc [DOCS] Update examples to groovy. 2014-07-22 14:45:46 +01:00
Clinton Gormley 8aefaef68a Update scripting.asciidoc
Added an ID for native java scripts
2014-07-22 11:36:40 +02:00
Peter Kim 6a25d9b7b5 [DOCS] Fixed typos 2014-07-17 15:25:34 -04:00
Brian Murphy cbd2a97abd [DOCS] : Indexed scripts/templates
These are the docs for the indexed scripts/templates feature.
Also moved the namespace for the REST endpoints.

Closes #6851
2014-07-16 10:49:02 +01:00
Ryan Ernst 64ab22816c Scripting: Add script engine for lucene expressions.
These are javascript expressions, which can only access numeric
fielddata, parameters, and _score. They can only be used for searches (not document updates).

closes #6818
2014-07-15 07:49:01 -07:00
Igor Motov 60b317caa4 Snapshot/Restore: Add ability to restore indices without their aliases
Closes #6457
2014-07-13 17:52:41 +09:00
Boaz Leskes f480969503 [Gateway] set a default of 5m to `recover_after_time` when any to the `expected*Nodes` is set
The `recovery_after_time` tells the gateway to wait before starting recovery from disk. The goal here is to allow for more nodes to join the cluster and thus not start potentially unneeded replications. The `expectedNodes` setting (and friends) tells the gateway when it can start recovering even if the `recover_after_time` has not yet elapsed. However, `expectedNodes` is useless if one doesn't set `recovery_after_time`. This commit changes that by setting a sensible default of 5m for `recover_after_time` *if* a `expectedNodes` setting is present.

Closes #6742
2014-07-11 11:28:45 +02:00
Iulia Pasov eed3513c37 Docs: Update plugins.asciidoc to fix typo
Changed the name of the European Environment Agency (from European Environmental Agency)

Closes #6807
2014-07-10 14:04:26 +02:00
Iulia Pasov a79d0744d3 Docs: Update plugins.asciidoc
Closes #6683
2014-07-09 16:15:59 +02:00
Shinsuke Sugaya 4bddb4e346 Update plugins.asciidoc 2014-07-05 00:44:02 +09:00
Shikhar Bhushan 1e894111b0 Docs: Link to eskka discovery plugin from doc
Closes #6721
2014-07-04 17:06:51 +02:00
Igor Motov 1425e28639 Add ability to restore partial snapshots
Closes #5742
2014-06-30 20:18:02 -04:00
Lee Hinman b43b56a6a8 Add a transformer to translate constant BigDecimal to double 2014-06-26 10:52:28 +02:00
Lee Hinman 5c6d28240f Switch to Groovy as the default scripting language
This is a breaking change to move from MVEL -> Groovy
2014-06-25 12:15:12 +02:00
Volker Fröhlich 06192686a2 [DOCS] Fixd typo in http.asciidoc 2014-06-16 10:42:34 +02:00
Shay Banon 2330421816 Wait till node is part of cluster state for join process
When a node sends a join request to the master, only send back the response after it has been added to the master cluster state and published.
This will fix the rare cases where today, a join request can return, and the master, since its under load, have not yet added the node to its cluster state, and the node that joined will start a fault detect against the master, failing since its not part of the cluster state.
Since now the join request is longer, also increase the join request timeout default.
closes #6480
2014-06-12 18:15:51 +02:00
Igor Motov 56a264cf6d [DOCS] Snapshot/restore: add more information about snapshot and restore monitoring 2014-06-11 20:52:45 -04:00
Gaurav Arora 4a3837acf0 [DOCS] fix typo in network module docs 2014-06-03 13:19:36 +02:00
salyh 27b38818b7 [DOCS] Add imap river and security plugin to docs 2014-06-03 10:59:50 +02:00
Shay Banon 9c98bb3554 Have a dedicated join timeout that is higher than ping.timeout for node join
Using ping.timeout, which defaults to 3s, to use as a timeout value on the join request a node makes to the master once its discovered can be too small, specifically when there is a large cluster state involved (and by definition, all the buffers and such on the nio layer will be "cold"). Introduce a dedicated join.timeout setting, that by default is 10x the ping.timeout (so 30s by default).
closes #6342
2014-05-30 12:42:08 +02:00
David Pilato 1dc186a595 [DOCS] fix typo 2014-05-27 15:57:39 +02:00
Itamar Syn-Hershko ac812f72b7 Docs: Adding Hebrew analyzer
Closes #6306
2014-05-27 13:40:53 +02:00
Simon Willnauer 9d5507047f Update Documentation Feature Flags [1.2.0] 2014-05-22 15:06:42 +02:00
Igor Motov 91c7892305 Add ability to snapshot replicating primary shards
This change adds a new cluster state that waits for the replication of a shard to finish before starting snapshotting process. Because this change adds a new snapshot state, an pre-1.2.0 nodes will not be able to join the 1.2.0 cluster that is currently running snapshot/restore operation.

Closes #5531
2014-05-20 08:57:21 -04:00
Igor Motov c20713530d Switch to shared thread pool for all snapshot repositories
Closes #6181
2014-05-16 19:03:15 -04:00
David Pilato bd871f96c2 Check that a plugin is Lucene compatible with the current running node using `lucene` property in `es-plugin.properties` file.
* If plugin does not provide `lucene` property, we consider that the plugin is compatible.
* If plugin provides `lucene` property, we try to load related Enum org.apache.lucene.util.Version. If this fails, it means that the node is too "old" comparing to the Lucene version the plugin was built for.
* We compare then two first digits of current node lucene version against two first digits of plugin Lucene version. If not equal, it means that the plugin is too "old" for the current node.

Plugin developers who wants to launch plugin check only have to add a `lucene` property in `es-plugin.properties` file. If you are using maven to build your plugin, you can do it like this:

In `pom.xml`:

```xml
    <properties>
        <lucene.version>4.6.0</lucene.version>
    </properties>

    <build>
        <resources>
            <resource>
                <directory>src/main/resources</directory>
                <filtering>true</filtering>
            </resource>
        </resources>
    </build>
```

In `es-plugin.properties`, add:

```properties
lucene=${lucene.version}
```

BTW, if you don't already have it, you can add the plugin version as well:

```properties
version=${project.version}
```

You can disable that check using `plugins.check_lucene: false`.
2014-05-16 13:41:20 +02:00
Clinton Gormley b331aa1670 [DOCS] Changed coming[1.1.0] to added in snapshot status 2014-05-13 11:19:28 +02:00
mikemccand 00fcf4d560 #6081: set IO throttling back to 20 MB/sec now that #6018 is fixed 2014-05-12 14:42:26 -04:00
violuke 9ed34b5a9e Correcting gramma 2014-05-06 18:00:19 +02:00
mikemccand 07563379dc fix docs for merging and throttling 2014-05-05 16:22:00 -04:00
javanna 16468f9ca3 [DOCS] Fixed scripting example 2014-04-28 11:39:12 +02:00
Lee Hinman 81e83cca74 Disable dynamic scripting by default
Closes #5853
2014-04-25 15:08:26 -06:00
Simon Willnauer b36ef995bb Change default recovery throttling to 50MB / sec
The current setting of 20MB/sec seems to be too conservative given
the capabilities of modern hardware / network throughput.
A 50MB default should provide better out of the box performance.
2014-04-23 15:40:21 +02:00
Simon Willnauer b4f0603169 Change default merge throttling to 50MB / sec
The current setting of 20MB/sec seems to be too conservative given
the capabilities of modern hardware. Even on cloud infrastructure this
seems to be too lowish. A 50MB default should provide better out of the box
performance
2014-04-22 21:08:40 +02:00
Xiao Yu 4b5e8cec8e Add a site plugin into list
Howdy,

Not sure if this is kosher but I would like to add my site plugin to the list in the docs.
2014-04-17 19:28:37 +02:00
Igor Motov 4c3027729e [DOCS] Make snapshot repository examples consistent 2014-04-16 17:28:43 -04:00
Peter Dyson f8537183b9 [DOCS] update old status of plugins 2014-04-13 20:18:19 -04:00
Nik Everett 40f1913cf3 [Docs] Add experimental highlighter plugin 2014-04-10 13:32:34 -04:00
Costin Leau 960d353dbd Remove plugin isolation feature for a future version
relates #5261
2014-04-09 17:28:11 +03:00
wittyameta 94278d81e3 Update advanced-scripting.asciidoc 2014-04-07 07:20:13 -06:00
Peter Dyson 233279bb64 [DOCS] Fixed typo 2014-04-04 17:37:56 +02:00
Alex Brasetvik cd8ed388d9 Document http.cors-settings 2014-03-31 11:34:46 +02:00
Shay Banon 6fce15beec Tribe: Index level blocks, index conflict settings
allow to configure on the index level which blocks can optionally be applied using tribe.blocks.indices prefix settings.
allow to control what will be done when a conflict is detected on index names coming from several clusters using the tribe.on_conflict setting. Defaults remains "any", but now support also "drop" and "prefer_[tribeName]".
closes #5501
2014-03-27 09:45:20 -07:00
Igor Motov 3ffd0a1dfa Remove deprecated gateways
Closes #5422
2014-03-26 18:10:51 -04:00
bleskes 5d832374dd Update Documentation Feature Flags [1.1.0] 2014-03-25 17:51:30 +01:00
Igor Motov 3414deb215 [DOCS] Mark snapshot status API as coming in 1.1.0 2014-03-24 21:55:19 -04:00
Florian Schilling c0a092aa92 [Doc] Updated docs for distance scripting
Updated docs for distance scripting and
added missing geohash distance functions
Closes #5397
2014-03-20 12:18:25 +01:00
Igor Motov a1192044f2 Add ability to get snapshot status for running snapshots
Closes #4946
2014-03-17 20:13:49 -04:00
David Pilato 0805c01984 [DOCS] Add Azure storage repositories 2014-03-17 19:40:28 +01:00
Justin Etheredge 36219a1786 [DOCS] Updating scripting docs for geo functions
Added a few functions are corrected the default unit where necessary
2014-03-17 11:59:02 +01:00
Clinton Gormley fb934aff57 [DOCS] Documented gateway.local.auto_import_dangled
Relates to #4996
2014-03-15 12:07:17 +01:00
Costin Leau 9624b215fb Add docs for plugin isolation 2014-03-11 12:32:58 +02:00
Konrad Feldmeier d7b0d547d4 [DOCS] Multiple doc fixes
Closes #5047
2014-03-07 14:24:58 +01:00
Kevin 2c7a3a49c5 [DOCS] add Elasticsearch Image Plugin 2014-03-05 14:16:56 +01:00
Kevin 99942089a8 [DOCS] add DynamoDB river plugin 2014-02-13 10:38:04 +01:00
Luca Cavanna 1d8d58391f [DOCS] added coming tags for `zen.discovery.publish_timeout` made dynamic 2014-02-12 15:24:38 +01:00
Luca Cavanna 16e4ac8713 [DOCS] Documented `discovery.zen.publish_timeout` setting 2014-02-12 10:45:37 +01:00
Igor Motov 02ebe33758 [DOCS] Fix typo in rename_pattern in snapshot/restore documentation 2014-02-11 09:23:07 -05:00
Mihnea Dobrescu-Balaur 1f7efb5471 [DOCS] Add GitHub community river plugin 2014-02-11 11:55:24 +01:00
David Pilato c214acc5e7 [DOCS] Add GridFS repository community plugin 2014-02-08 10:43:54 +01:00
Clinton Gormley 93930d6dc7 Removed 0.90.* deprecation and addition notifications
Closes #5052
2014-02-07 20:52:49 +01:00
Kevin d9b704fd86 add redis transport plugin 2014-02-06 18:19:54 +01:00
Costin Leau f5a8de6321 [DOCS] organize a bit the repository plugins
(cherry picked from commit 88e1c20c4581885db7e5e65edf7eb3629c2d31ca)
2014-02-06 19:01:58 +02:00
Clinton Gormley 6238d406b5 [DOCS] Removed the experimental label from Tribe, Hot Threads
and Completion Suggester
2014-02-06 14:19:17 +01:00
David Pilato 583f148334 [DOCS] add azure and gce discovery plugins
Clean EC2 disco doc
Add Azure disco doc
Add Google Compute Engine doc
Fix Zen doc (add `enabled` in `multicast` parameters list) - Fix #5032.
2014-02-06 09:18:42 +01:00
David Pilato 8b1a6fc5b6 Add S3 and HDFS repositories 2014-02-05 17:53:37 +01:00
Igor Motov 2755eecf65 Add throttling to snaphost and restore operations
Closes #4855
2014-01-29 10:33:59 -05:00
Luca Cavanna b61ca9932a [DOCS] Clarified docs for cluster.routing.allocation.same_shard.host cluster setting
Clarified also javadocs for SameShardAllocationDecider
2014-01-28 12:32:37 +01:00
Costin Leau 2690019e95 update link to Hadoop Snapshot/Restore plugin 2014-01-25 18:27:14 +02:00
Brusic d9b71a8083 [DOCS] various docs fixes
Removed unused misc.asciidoc file
Added plugins directory to directory layout
Fixed transport.tcp.connect_timeout value to match the code found in NetworkService.TcpSettings
Clarified that phrase query does not preserve order of terms
Clarified merge page
Added instructions on how to build documentation to docs/README
2014-01-23 10:52:13 +01:00
Shay Banon e29659e36d add internal force local flag, used by tribe node
tribe node to set it to true so all master read operations will automatically execute on the local tribe node
2014-01-20 22:40:26 +01:00
Clinton Gormley 5003ca9278 [DOCS] Fixed file:/// URL for installing plugins 2014-01-20 01:34:12 +01:00
Andy Goldstein 8f659bccb1 Add documentation for transport.publish_port 2014-01-17 22:06:22 +01:00
Igor Motov 4643f78098 [DOCS] Add documentation for URL repository 2014-01-15 13:13:16 -05:00
Clinton Gormley a0b993e2dc [DOCS] Tidied up cluster settings docs 2014-01-15 14:51:18 +01:00
Igor Motov ba7699a38b Add documentation for index.routing.allocation.*._name and index.routing.allocation.*._id options 2014-01-14 16:20:46 -05:00
Igor Motov b987615f5e Improve support for partial snapshots
Fixes #4701. Changes behavior of the snapshot operation. The operation now fails if not all primary shards are available at the beginning of the snapshot operation. The restore operation no longer tries to restore indices with shards that failed or were missing during snapshot operation.
2014-01-13 16:59:21 -05:00
Clinton Gormley 2e79246c1a [DOCS] Added docs for tribe node
Related #4708
2014-01-13 21:53:53 +01:00
Alexander Reelsen 1652767ec8 [DOCS] Added documentation for SameShardAllocationDecider
Closes #4615
2014-01-09 11:24:12 +01:00
Martijn van Groningen e6f83248a2 Deprecated disable allocation decider which has the following options:
`allocation.disable_new_allocation`, `allocation.disable_allocation`, `allocation.disable_replica_allocation`,
in favour for the enable allocation decider which has a single option `allocation.enable` wich can be set to the following values:
`none`, `new_primaries`, `primaries` and `all` (default).

Closes #4488
2014-01-09 10:01:46 +01:00
Igor Motov 5d98341d11 Fix typo in snapshot/restore documentation 2014-01-07 14:03:12 -05:00
Simon Willnauer fa16969360 Cleanup comments and class names s/ElasticSearch/Elasticsearch
* Clean up s/ElasticSearch/Elasticsearch on docs/*
 * Clean up s/ElasticSearch/Elasticsearch on src/* bin/* & pom.xml
 * Clean up s/ElasticSearch/Elasticsearch on NOTICE.txt and README.textile

Closes #4634
2014-01-07 11:21:51 +01:00
David Pilato 0c7b494bb8 plugin manager: new `timeout` option
When testing plugin manager with real downloads, it could happen that the test run forever. Fortunately, test suite will be interrupted after 20 minutes, but it could be useful not to fail the whole test suite but only warn in that case.

By default, plugin manager still wait indefinitely but it can be modified using new `--timeout` option:

```sh
bin/plugin --install elasticsearch/kibana --timeout 30s

bin/plugin --install elasticsearch/kibana --timeout 1h
```

Closes #4603.
Closes #4600.
2014-01-03 16:48:18 +01:00
Britta Weber 9f54e9782d rename _shard -> _index and also rename classes and variables
closes #4584
2014-01-03 14:00:23 +01:00
Martijn van Groningen f4bf0d5112 Replaced `ignore_indices` with `ignore_unavailable`, `expand_wildcards` and `allow_no_indices`.
* `ignore_unavailable` - Controls whether to ignore if any specified indices are unavailable, this includes indices that don't exist or closed indices. Either `true` or `false` can be specified.
* `allow_no_indices` - Controls whether to fail if a wildcard indices expressions results into no concrete indices. Either `true` or `false` can be specified. For example if the wildcard expression `foo*` is specified and no indices are available that start with `foo` then depending on this setting the request will fail. This setting is also applicable when `_all`, `*` or no index has been specified.
* `expand_wildcards` - Controls to what kind of concrete indices wildcard indices expression expand to. If `open` is specified then the wildcard expression if expanded to only open indices and if `closed` is specified then the wildcard expression if expanded only to closed indices. Also both values (`open,closed`) can be specified to expand to all indices.

Closes to #4436
2014-01-02 12:19:45 +01:00
Britta Weber 1ede9a5730 make term statistics accessible in scripts
term statistics can be accessed via the _shard variable.

Below is a minimal example. See documentation on details.

```

DELETE paytest

PUT paytest
{
    "mappings": {
        "test": {
            "_all": {
                "auto_boost": true,
                "enabled": true
            },
            "properties": {
                "text": {
                    "index_analyzer": "fulltext_analyzer",
                    "store": "yes",
                    "type": "string"
                }
            }
        }
    },
    "settings": {
        "analysis": {
            "analyzer": {
                "fulltext_analyzer": {
                    "filter": [
                        "my_delimited_payload_filter"
                    ],
                    "tokenizer": "whitespace",
                    "type": "custom"
                }
            },
            "filter": {
                "my_delimited_payload_filter": {
                    "delimiter": "+",
                    "encoding": "float",
                    "type": "delimited_payload_filter"
                }
            }
        },
        "index": {
            "number_of_replicas": 0,
            "number_of_shards": 1
        }
    }
}

POST paytest/test/1
{
    "text": "the+1 quick+2 brown+3 fox+4 is quick+10"
}

POST paytest/test/2
{
    "text": "the+1 quick+2 red+3 fox+4"
}

POST paytest/_refresh

POST paytest/_search
{
    "script_fields": {
       "ttf": {
          "script": "_shard[\"text\"][\"quick\"].ttf()"
       }
    }
}

POST paytest/_search
{
    "script_fields": {
       "freq": {
          "script": "_shard[\"text\"][\"quick\"].freq()"
       }
    }
}
POST paytest/test/2/_termvector
POST paytest/_search
{
    "script_fields": {
       "payloads": {
          "script": "term = _shard[\"text\"].get(\"red\",_PAYLOADS);payloads = []; for(pos : term){payloads.add(pos.payloadAsFloat(-1));} return payloads;"
       }
    }
}

POST paytest/_search
{
   "script_fields": {
      "tv": {
         "script": "_shard[\"text\"][\"quick\"].freq()"
      }
   },
   "query": {
      "function_score": {
         "functions": [
            {
               "script_score": {
                  "script": "_shard[\"text\"][\"quick\"].freq()"
               }
            }
         ]
      }
   }
}

```

closes #3772
2014-01-02 11:17:33 +01:00
Jun Ohtani 7bbe453273 [DOCS] Added elasticsearch-extended-analyze plugin 2013-11-21 09:48:00 +01:00
Igor Motov 510397aecd Initial implementation of Snapshot/Restore API
Closes #3826
2013-11-10 18:26:56 -05:00
Clinton Gormley 5af4e02d6c [DOCS] Fix link to statsd plugin
Fixes #4128
2013-11-08 20:29:51 +01:00
Ben McCann 46edfc484a [DOCS] Add some documentation about the performance of `_source` usage in scripts. 2013-11-04 11:05:55 +01:00
Igor Motov c724f0de5d Initial implementation of ResourceWatcherService
Closes #4062
2013-11-03 21:55:54 -05:00
Clinton Gormley d67331b554 [DOCS] Added script.disable_dynamic to the scripting page 2013-10-15 12:25:07 +02:00
Shay Banon 420b3396f4 Set queue sizes by default on bulk/index thread pools
Now that we properly fixed the ability to set the queue size on the index / bulk thread pool, we should actually set them to a somehow reasonable value to protect from users potentially overflowing our system.

I suggest defaults to be 50 for bulk, and 200 for indexing.

Also, set the thread pool for get, which we should set (in a similar value to a "read" queue size we have today).
closes #3888
2013-10-12 21:51:37 +02:00
Lee Hinman dede6ee874 Remove extra 'processors' anchor in threadpool docs 2013-10-09 01:56:49 -06:00
Lee Hinman ba40aa374e Uniquify anchor links to fix asciidoc/docbook generation 2013-09-30 15:32:00 -06:00
Lee Hinman 0442b737be Add more anchor links to documentation
Related to #3679
2013-09-30 13:13:16 -06:00
Shay Banon 359d14ddc5 doc processors setting 2013-09-20 14:55:35 +02:00
Shay Banon 29c0f27a9e fix thread pool docs to remove blocking 2013-09-20 12:31:17 +02:00
Shay Banon 20745adadd Add dedicated Suggest Thread Pool
Add a dedicated suggest thread pool for the suggest API. With the new completion suggest type, which is purely CPU bounded, it makes more sense to have a dedicated thread pool for suggest compared to having it share the search thread pool and "competing" against other search operations.
closes #3698
2013-09-15 01:54:27 +02:00
David Pilato fafc4eef98 Plugin Manager: add silent mode.
Now with have proper exit codes for elasticsearch plugin manager (see #3463), we can add a silent mode to plugin manager.

```sh
bin/plugin --install karmi/elasticsearch-paramedic --silent
```

Closes #3628.
2013-09-10 18:31:35 +02:00
David Pilato 764aa54f2d Plugin Manager should support -remove group/artifact/version naming
When installing a plugin, we use:

```sh
bin/plugin --install groupid/artifactid/version
```

But when removing the plugin, we only support:

```sh
bin/plugin --remove dirname
```

where `dirname` is the directory name of the plugin under `/plugins` dir.

Closes #3421.
2013-09-09 21:17:16 +02:00
Clinton Gormley 9d0dd545cb [DOCS] Tidied up the plugins page and added Graphite and Statsd 2013-09-04 23:20:55 +02:00
Clinton Gormley 393c28bee4 [DOCS] Removed outdated new/deprecated version notices 2013-09-03 21:28:31 +02:00
Clinton Gormley 822043347e Migrated documentation into the main repo 2013-08-29 01:24:34 +02:00