Commit Graph

2150 Commits

Author SHA1 Message Date
Martijn van Groningen f2900f71c5 test: unmuted slow watches test.
Removed the dependency on groovy and added a script impl. that allows to run Thread.sleep(...) in order to simulate a slow watch.

Relates to elastic/elasticsearch#724

Original commit: elastic/x-pack-elasticsearch@b18dd89b46
2015-11-20 14:02:17 +01:00
Tanguy Leroux a47adfb270 Marvel: Index Nodes in Time Based Index and Data Index
Closes elastic/elasticsearch#946

Original commit: elastic/x-pack-elasticsearch@0f5721ffbf
2015-11-19 21:09:23 +01:00
Tanguy Leroux ff1e6ec776 Marvel: Fix memory leak when something goes wrong in LocalExporter
When the bulk request executed by the LocalExporter has at least 1 failure, an exception is thrown and the BulkRequestBuilder is not reset. Then the next time Marvel collects data it will contain all the previous requests plus the new ones... leading to an OOM. IThis behavior was reproduced with a closed marvel index but it also happen in case of unassigned shards.

Original commit: elastic/x-pack-elasticsearch@9c3caa08eb
2015-11-19 16:07:01 +01:00
debadair 5809bbdb5d Docs: Deleting unused Marvel doc files. Marvel docs are currently in elasticsearch-marvel/docs.
Original commit: elastic/x-pack-elasticsearch@be7a7e2243
2015-11-18 17:46:21 -08:00
debadair fb4ba6f89d Watcher Docs: Edited chain input topic & added chain to customizing watches.
Original commit: elastic/x-pack-elasticsearch@9e1d7f09c6
2015-11-18 11:15:24 -08:00
jaymode 5b72d1768d test: add throws InterruptedException for InternalTestCluster#beforeTest calls
Original commit: elastic/x-pack-elasticsearch@b89a58a408
2015-11-18 13:48:54 -05:00
Tanguy Leroux 36dd391d4d Marvel: mute NodeStats tests
Original commit: elastic/x-pack-elasticsearch@f32464f864
2015-11-18 19:04:41 +01:00
Tanguy Leroux 37921036cb Marvel: ignore IndexNotFoundException for _all index when Shield is enabled
When Shield and Marvel are installed together and no indices exist in the cluster yet, Shield returns an IndexNotFoundException. This commit ignores and logs at DEBUG level any IndexNotFoundException iff a) Shield is enabled and 2) marvel.agent.indices setting is empty.

Closes elastic/elasticsearch#887

Original commit: elastic/x-pack-elasticsearch@5d227d775d
2015-11-18 16:58:08 +01:00
jaymode 16848c6043 shield: disable document and field level security by default
This change disables document and field level security by default so that we are able to maintain
bulk update functionality. Users that enable DLS/FLS will not have this functionality. Additionally,
if a user tries to configure DLS/FLS in a role without enabling it, the role will be skipped during
parsing and a log message will be logged at the error level.

See elastic/elasticsearch#938

Original commit: elastic/x-pack-elasticsearch@60c7519092
2015-11-18 08:02:29 -05:00
jaymode cc2096b4f9 add the option to disable DLS and FLS completely
This commit reverts a previous change where searcher were not wrapped when the RequestContext
could not be found. If DLS/FLS is enabled, which is the default, any bulk request that contains an
update request will not be permitted. This change also exposes the ability to completely disable DLS
and FLS so that users who are not using these features can still use bulk updates.

See elastic/elasticsearch#938

Original commit: elastic/x-pack-elasticsearch@513782db1c
2015-11-17 12:55:26 -05:00
Ryan Ernst 722bc3713e Merge pull request elastic/elasticsearch#987 from rjernst/clean_more
Build: Clean buildSrc on gradle clean

Original commit: elastic/x-pack-elasticsearch@a672ad7d33
2015-11-17 09:10:21 -08:00
Ryan Ernst cfba6af6c9 Merge pull request elastic/elasticsearch#988 from rjernst/disable_snapshot_cache
Build: Do not cache snapshots

Original commit: elastic/x-pack-elasticsearch@0fcbaad1ab
2015-11-16 13:43:17 -08:00
Ryan Ernst fc1d043c0d Build: Do not cache snapshots
Gradle marks dependencies with a version containing "SNAPSHOT" as
"changing", and caches them for 24 hours by default. This updates the
configuration to not cache them at all.

Original commit: elastic/x-pack-elasticsearch@ac95e30a96
2015-11-16 13:04:46 -08:00
Ryan Ernst 0d67caa468 Build: Clean buildSrc on gradle clean
Same change as elastic/elasticsearchelastic/elasticsearch#14785 but for xplugins.

Original commit: elastic/x-pack-elasticsearch@02e8247686
2015-11-16 12:45:03 -08:00
Tanguy Leroux 891d348dd1 Checks the number of actions in bulk request
Original commit: elastic/x-pack-elasticsearch@33678eb7f6
2015-11-16 15:57:18 +01:00
Tanguy Leroux a901aeef86 Marvel: reset buffer in HttpExporter
The BytesStreamOutput buffer is not reset when exporting multiple documents. It causes the same marvel doc to be indexed many times and it creates very large bulk requests. Bug introduced in elastic/x-pack@8575fd91c4

Original commit: elastic/x-pack-elasticsearch@adac96677d
2015-11-16 15:56:10 +01:00
Tanguy Leroux 0b50bbb5e5 Fix compilation errors with org.elasticsearch.cluster.health.ClusterHealthStatus
Original commit: elastic/x-pack-elasticsearch@68988b2163
2015-11-16 15:34:26 +01:00
Martijn van Groningen 46044a4fe0 test: use one IndexSetting instance
Original commit: elastic/x-pack-elasticsearch@1f35455ebe
2015-11-16 12:01:42 +01:00
Martijn van Groningen 101e4ff7ce test: added test that ensures that field data cache is not bypassed for segment and global ordinal caching when field subset reader hides the requested field.
Original commit: elastic/x-pack-elasticsearch@c4864d18ac
2015-11-16 11:38:02 +01:00
Martijn van Groningen 6e482d1a3d shield: Also prohibit update requests inside bulk requests if FLS is enabled.
We do this already for update requests, but this was forgotten to be checked for bulk requests.

Original commit: elastic/x-pack-elasticsearch@8d864a7c98
2015-11-16 05:49:33 +01:00
Chris Earle 6ef51d5dc0 Merge branch 'master' of https://github.com/elastic/x-plugins
Original commit: elastic/x-pack-elasticsearch@22a9ade30e
2015-11-13 14:44:59 -05:00
Lee Hinman 557f0d4f83 Fix compilation for `newIndexSettings` arity change
Original commit: elastic/x-pack-elasticsearch@f94fabfcc9
2015-11-13 09:35:46 -07:00
Chris Earle 8c5fdc7023 Fixing references to bin/plugin -i
Original commit: elastic/x-pack-elasticsearch@708d381742
2015-11-12 13:45:38 -05:00
Tanguy Leroux 3aa76795f9 Marvel: Ensure ClusterStatsTests waits enough for marvel documents
The should wait a bit more and refreshes index between each assertBusy()

Closes elastic/elasticsearch#729

Original commit: elastic/x-pack-elasticsearch@8ec702c1dc
2015-11-12 14:11:26 +01:00
Lee Hinman 7f0863ce83 Merge remote-tracking branch 'dakrone/fix-nullable'
Original commit: elastic/x-pack-elasticsearch@4ce4f5c3eb
2015-11-11 12:27:41 -07:00
jaymode 1601ce4fba add guava as a test dependency for jimfs
Original commit: elastic/x-pack-elasticsearch@331f834729
2015-11-11 13:40:03 -05:00
jaymode d74de5acc8 shield: only wrap readers if the RequestContext can be located
Previously, when the RequestContext could not be located a FieldSubsetReader was returned that only
allowed meta fields to be read. This was done for safety in case there was an API missed so we did not
leak data. However, this causes issues because some requests in elasticsearch execute on a different
thread than the one with the RequestContext so we effectively lose this context and prevent access to
the fields in the document. This is especially problematic with update requests, because that means that
fields that aren't included in the updated document will be lost.

This commit removes the wrapping of the readers in this case and adds tests for bulk updates.

Closes elastic/elasticsearch#938

Original commit: elastic/x-pack-elasticsearch@74c8059da0
2015-11-11 13:26:49 -05:00
Ryan Ernst 182f5c251d Merge pull request elastic/elasticsearch#968 from rjernst/transitive_goes_poof
Remove transitive dependencies

Original commit: elastic/x-pack-elasticsearch@3b1ad25cf1
2015-11-11 09:28:43 -08:00
Ryan Ernst 26c8e949a4 Build: Remove transitive dependencies
In elastic/elasticsearchelastic/elasticsearch#14668, transitive dependencies were removed
from elasticsearch, and plugins now no longer get transitive deps
either. This commit fixes xplugins to remove transitive deps.

Original commit: elastic/x-pack-elasticsearch@5c41657b6e
2015-11-11 09:18:24 -08:00
Lee Hinman c9dd618a0b Fix @Nullable import in watcher
Original commit: elastic/x-pack-elasticsearch@017ead8ff7
2015-11-11 10:00:24 -07:00
Ryan Ernst 838cb54b9e Merge pull request elastic/elasticsearch#959 from rjernst/version_cleanup
Remove repositories section now that it is included by build-tools

Original commit: elastic/x-pack-elasticsearch@213a117290
2015-11-10 08:10:24 -08:00
Martijn van Groningen 39e58b03d5 Fix upstream changes to IndexSearcherWrapper: https://github.com/elastic/elasticsearch/pull/14654
Original commit: elastic/x-pack-elasticsearch@b32ba2ad78
2015-11-10 23:10:04 +07:00
Ryan Ernst 3a66aeb6a4 Build: Remove repositories section now that it is included by build-tools
With elastic/elasticsearchelastic/elasticsearch#14645, repositories, including the lucene
snapshot repo, are now handled by the build plugin itself. This change
removes this from xplugins.

Original commit: elastic/x-pack-elasticsearch@8e0c7ef894
2015-11-10 07:54:19 -08:00
Martijn van Groningen c916c5e35b watcher: If during startup a watch record couldn't be saved because of a version conflict then overwrite it.
In certain cases this could cause Watcher to prevent from being started (either automatically or manually):
* The watch of the triggered watch doesn't exist
* The execution service rejected the execution the watch. (due to not enough threads in the thread pool)

If a watch record gets overwritten this means it has been executed before when watcher was running, but we were unable to delete the triggered watch. So we executed the a watch twice with the same trigger event. This can happen if watcher was stopped before because of the node running Watcher unexpectedly stopped (master left, power loss etc.)

In stead of failing to write the watch record, we overwrite it set the state 'executed_multiple_times'.

Closes elastic/elasticsearch#695

Original commit: elastic/x-pack-elasticsearch@20f805cb99
2015-11-10 09:24:15 +07:00
Areek Zillur dcae5fb5f0 ensure no operation is blocked before initial license notification
closes elastic/elasticsearch#906

update comment

Original commit: elastic/x-pack-elasticsearch@0bd788720f
2015-11-09 11:31:06 -05:00
Ryan Ernst 28326a2c4c Fix build to still work when not attached to elasticsearch
Original commit: elastic/x-pack-elasticsearch@e7056c2305
2015-11-06 14:03:08 -08:00
Ryan Ernst dca6b57256 Merge pull request elastic/elasticsearch#945 from rjernst/attach_improvements
Build: Improve behavior and setup for attachment to elasticsearch

Original commit: elastic/x-pack-elasticsearch@1457b07952
2015-11-06 12:02:59 -08:00
Ryan Ernst 33785e5446 Build: Improve behavior and setup for attachment to elasticsearch
Currently, running any task while attached to elasticsearch will also
try to run that task in all elasticsearch projects. This change causes
x-plugins to be moved into a subproject when attached. It does this by
dynamically creating a fake root project, and making this root project
think it is attached to the existing attachments, along with x-plugins
itself.

This also fixes buildSrc so attachment to elasticsearch's buildSrc
actually works.

Original commit: elastic/x-pack-elasticsearch@da0adf1da5
2015-11-06 11:08:03 -08:00
Tanguy Leroux 9fc92950c4 Fix Shield zip package
Original commit: elastic/x-pack-elasticsearch@eaf9dfc9ba
2015-11-06 16:17:03 +01:00
Tanguy Leroux 70f1b5c8f2 Marvel: Client nodes should be able to send metrics
closes elastic/elasticsearch#897

Original commit: elastic/x-pack-elasticsearch@1ebcc9de14
2015-11-06 15:32:13 +01:00
Tanguy Leroux 7e27abca98 Marvel: Add test for unsupported cluster version in HttpExporterTests
Original commit: elastic/x-pack-elasticsearch@556a05e86f
2015-11-06 15:12:12 +01:00
Your Full Name bf4c42a40f XContentExtraction: Fix state bug
This fixes a nasty bug, when trying to extract arrays, that contain objects
from a searchinput. Reason for this is, that we reset the state to early, while
still being in an array and not having read all of the embedded objects.

With Jackson 2.6 we should reuse jacksons capabilities to not only filter for
xcontent builders, but also for parsers. Not the nicest code ever.

Closes elastic/elasticsearch#852

Original commit: elastic/x-pack-elasticsearch@2682254644
2015-11-06 09:48:07 +01:00
debadair 6106128272 Docs: Added uninstall topics to Watcher & Shield. Closes elastic/elasticsearch#801.
Original commit: elastic/x-pack-elasticsearch@b990c64647
2015-11-05 17:54:35 -08:00
Simon Willnauer a9028da5d1 Merge pull request elastic/elasticsearch#933 from s1monw/catchup/14518
Fix API usage after index-level guice was removed

Original commit: elastic/x-pack-elasticsearch@06400cc58d
2015-11-05 18:39:12 +01:00
Robert Muir f517fc8680 Merge pull request elastic/elasticsearch#935 from rmuir/nuke_poms
Remove confusing pom.xml files

Original commit: elastic/x-pack-elasticsearch@87c28d8cdc
2015-11-05 12:07:40 -05:00
Robert Muir cfb77bf572 Remove confusing pom.xml files
Original commit: elastic/x-pack-elasticsearch@772a1eb5d3
2015-11-05 10:39:28 -05:00
Simon Willnauer ad2835f6d3 Fix API usage after index-level guice was removed
See https://github.com/elastic/elasticsearch/pull/14518

Original commit: elastic/x-pack-elasticsearch@2428ee47c2
2015-11-05 13:18:58 +01:00
Robert Muir 5311da746b Add jimfs as shield test dep and conditionalize file permissions changing in tests.
See elastic/elasticsearch#928 for followup work

Original commit: elastic/x-pack-elasticsearch@d435cf9000
2015-11-04 14:25:20 -05:00
Ryan Ernst 09b6e3cfdc Add attachment docs to README
Original commit: elastic/x-pack-elasticsearch@49cbc9c66a
2015-11-04 11:19:13 -08:00
Ryan Ernst dc2fcf22b0 Add jcenter to buildSrc repos so that when elasticsearch/buildSrc is
attached it finds it for the dummy perforce dependency.

Original commit: elastic/x-pack-elasticsearch@6bd377ab6a
2015-11-04 10:58:27 -08:00