Commit Graph

1930 Commits

Author SHA1 Message Date
uboness a86bb7b140 Introduced `ExportBulk`
- bulk abstraction on top of exporter
- removed queuing and bulk processor from local exporter
- started to adjust tests to shield (require a lot of helper methods in `MarvelInegTestCase`)
- moved index name resolution to `Exporter` (functionality shared between all exporters)

Original commit: elastic/x-pack-elasticsearch@86b495622c
2015-09-26 04:30:34 +02:00
uboness 3162097ce8 disabled licensing tests until license refactoring is done
Original commit: elastic/x-pack-elasticsearch@e5673fedd9
2015-09-26 04:30:34 +02:00
uboness 075707e12c cleanup for local exporter
Original commit: elastic/x-pack-elasticsearch@b715ab7ca1
2015-09-26 04:30:34 +02:00
Tanguy Leroux e2719ab213 Add tests for LocalExporter
Original commit: elastic/x-pack-elasticsearch@a1866a9c39
2015-09-26 04:30:34 +02:00
Tanguy Leroux da4b259c48 Remove start/stop in Exporter(s)
Original commit: elastic/x-pack-elasticsearch@9ddcb8119b
2015-09-26 04:30:34 +02:00
Tanguy Leroux 415f6eda79 Add LocalExporter implementation
Original commit: elastic/x-pack-elasticsearch@36a683294c
2015-09-26 04:30:34 +02:00
uboness 1c5e03a239 fixed tests
Original commit: elastic/x-pack-elasticsearch@bea8d61278
2015-09-26 04:30:34 +02:00
uboness 57561f9a03 cleaned up `HttpExporterUtils`
Original commit: elastic/x-pack-elasticsearch@2db95799ea
2015-09-26 04:30:34 +02:00
uboness c85f0f4564 Fixed the integration tests
- prepared the ground for shield integration tests.
- for now, `shield` is disabled for all integration tests (once we have the `local` exporter we'll be able to turn shield on.

Original commit: elastic/x-pack-elasticsearch@cf5d660f58
2015-09-26 04:30:34 +02:00
uboness 345efb38d6 Cleaned up warnings in `SecuredClient`
Original commit: elastic/x-pack-elasticsearch@b8f52f1518
2015-09-26 04:30:34 +02:00
uboness 53b2fb9826 cleaned up `Exporters`
- also added additional unit tests for this class

Original commit: elastic/x-pack-elasticsearch@c16891a50a
2015-09-26 04:30:33 +02:00
uboness 7206de9c2a Adding multiple exporter types
- an exporter is no longer fixed construct. Instead, this commit creates an `Exporter` abstraction. An exporter has a type and is responsible for exporting marvel documents.
- multiple exporters can be configured, each can be of a different type.
- an exporter can be of a "singleton" type, indicating that there can only be one exporter of that type at any point in time.
- exporters are configured in the settings. Each exporters is identified by a unique name/id and has its own settings (the type of the exporter defines what settings that exporter expects)
- exporters are loaded at start up time, but it is also possible to update the exporter settings at runtime (all exporter settings are registered as dynamic cluster settings). When updated, all the exporters will be rebuild & reloaded.
- there are two types of exporters: `http` and `local`
- `http` exporter exports documents to external elasticsearch clusters using the REST API
- the new `http` exporter doesn't support user info as part of the URL (it's pron to security info leak). Instead a new `auth.username` and `auth.password` settings were added that can be set per `http` exporter (when shield is installed, the passwords are filtered out from the node info API)
- `local` exporter exports documents to the same cluster the agent is installed on, using the internal node client.
- when no exporter is configured, a default `local` exporter is created (for best ootb experience)

Original commit: elastic/x-pack-elasticsearch@d1d7c8aefd
2015-09-26 04:30:33 +02:00
uboness 334e090902 Shield Integration
- Adds internal `__marvel_user`
- Adds a `SecuredClient` that binds the internal marvel user to all requests it sends

Original commit: elastic/x-pack-elasticsearch@77c4716261
2015-09-26 04:30:33 +02:00
Nik Everett bee764b9ee Merge pull request elastic/elasticsearch#713 from elastic/immutable_set_be_gone
Remove ImmutableSet

Original commit: elastic/x-pack-elasticsearch@fe06937bc7
2015-09-25 21:46:22 -04:00
jaymode 44cd14d5f1 test: cleanup usages of node builder where we actually need Shield loaded
Some tests are still using a node builder for nodes that need to load Shield, which is not currently
supported by elasticsearch. This changes some to use a MockNode and awaits fix a tribe test
that was previously testing nothing.

Original commit: elastic/x-pack-elasticsearch@57d0c902b6
2015-09-25 13:05:07 -04:00
Nik Everett c6e7c9c1d6 Merge branch 'master' into immutable_set_be_gone
Original commit: elastic/x-pack-elasticsearch@faa212ffbd
2015-09-25 10:47:16 -04:00
javanna 22edda4044 [TEST] fix method clashes due to randomTimeValue addition to ESTestCase
Original commit: elastic/x-pack-elasticsearch@74bd522bf8
2015-09-25 16:45:11 +02:00
jaymode dd8ce6017b test: update certificates to have more localhost subject alternative names
This updates our test certificates to have more subject alternative names and adds a script
to make regeneration easier in the future. Also added are the private keys in PEM format and
a .p12 version of the keystore. In the future we may want to start supporting PKCS12 keystores
and this will be a starting point for that.

Original commit: elastic/x-pack-elasticsearch@92c94c9f28
2015-09-25 10:36:28 -04:00
Nik Everett 81970a564f Fix Set->String conversion
Original commit: elastic/x-pack-elasticsearch@c34f2d252d
2015-09-25 10:16:34 -04:00
javanna 90a11b20e6 [TEST] fix method clash due to randomTimeValue addition to ESTestCase
Original commit: elastic/x-pack-elasticsearch@cbcdfa641b
2015-09-25 16:13:32 +02:00
javanna ffc359b91d [TEST] fix compile errors due to breaking java api changes to QueryBuilders
Original commit: elastic/x-pack-elasticsearch@4e2f6f89c7
2015-09-25 16:13:32 +02:00
Martijn van Groningen e16894fb9d Removed dynamic index names in favour for ES' date math index names.
Original commit: elastic/x-pack-elasticsearch@267084f163
2015-09-25 15:55:22 +02:00
Nik Everett 3ce9d903e8 Fix test that relied on iteration order
Original commit: elastic/x-pack-elasticsearch@5523c877ab
2015-09-25 09:02:15 -04:00
Nik Everett 7fe90ba03c Switch from Set to String comparison
Original commit: elastic/x-pack-elasticsearch@385ae531ee
2015-09-25 09:00:26 -04:00
Nik Everett 7fb98b60b4 Merge pull request elastic/elasticsearch#712 from elastic/remove_eclipse
Remove eclipse configuration files

Original commit: elastic/x-pack-elasticsearch@2d217b3e57
2015-09-25 08:24:06 -04:00
Martijn van Groningen 1ce6f9ddba http input: The http input should only url encode once
Java's URI class already does url encoding, so we don't need to do this on out side.

Closes elastic/elasticsearch#699

Original commit: elastic/x-pack-elasticsearch@612dc99945
2015-09-25 12:20:58 +02:00
Nik Everett e3b3ef3ef2 Remove ImmutableSet
Yet another step in the de-guava-ification process.

Original commit: elastic/x-pack-elasticsearch@d8c597c7ba
2015-09-24 16:51:45 -04:00
Nik Everett 974ae21899 Remove eclipse configuration files
These files are generated automatically with eclipse:eclipse so they shouldn't
be tracked in the repository.

Original commit: elastic/x-pack-elasticsearch@3869ddebfa
2015-09-24 14:08:49 -04:00
jaymode 0d1602255b test: make the remote cluster a test cluster in the RemoteIndexAuditTrailStartingTests
This also stops the remote IndexAuditTrail services prior to wiping the cluster to prevent possible
writes after a template has been deleted.

Original commit: elastic/x-pack-elasticsearch@30ef226250
2015-09-24 08:33:58 -04:00
jaymode 6a7462be3e update the IPFilter to always allow traffic from a bound address
This change updates the IPFilter to always allow traffic from the bound addresses of the node
even if they have been explicitly disabled. This behavior can be disabled through a setting but
that could be dangerous if the blocking rule is added via a persistent setting stored in the cluster
state.

Closes elastic/elasticsearch#487

Original commit: elastic/x-pack-elasticsearch@4c1cf9455f
2015-09-23 10:30:43 -04:00
jaymode b219e9c496 remove lucene expressions from qa pom
Original commit: elastic/x-pack-elasticsearch@389cf7b564
2015-09-23 10:13:51 -04:00
jaymode 2e0b87f6c6 audit tampered request when verifying the user header fails
When we receive a message and cannot decode the signed user header, we should audit
this as a tampered request. The request may not be tampered if nodes are configured
differently in terms of system keys, but this at least provides some evidence of the bad
request on the nodes that receives it.

Closes elastic/elasticsearch#647

Original commit: elastic/x-pack-elasticsearch@2f3411d9f9
2015-09-23 08:51:56 -04:00
Nik Everett 56bb508254 Merge pull request elastic/elasticsearch#694 from nik9000/immutable_map_entry_set_be_gone
Stop using ImmutableMap#entrySet

Original commit: elastic/x-pack-elasticsearch@43d6598d27
2015-09-23 08:29:17 -04:00
Nik Everett f5398a739a Stop using ImmutableMap#entrySet
This is the first in a series of changes that will support banning Guava's
ImmutableSet.

Original commit: elastic/x-pack-elasticsearch@7f95900186
2015-09-22 16:49:10 -04:00
debadair 4ed8ed4ced Watcher Docs: Fixed LS config example to work with latest ES output plugin.
Original commit: elastic/x-pack-elasticsearch@57b37e8d5f
2015-09-22 12:22:55 -07:00
jaymode 7c8bad6b53 create basic interface for a caching realm
This extracts an interface for methods that a caching realm must implement. The idea for this
interface is to allow for custom realms to integrate easily with the clear cache api. With this change
a custom realm can implement this API and the clear cache action can call the evict or evictAll
methods on the realm.

Closes elastic/elasticsearch#544

Original commit: elastic/x-pack-elasticsearch@62b0029689
2015-09-22 13:28:39 -04:00
jaymode 138582db2a fix compilation error due to changed XContentParser interface
Original commit: elastic/x-pack-elasticsearch@386c8d357c
2015-09-22 07:29:45 -04:00
Robert Muir f610a347f0 Fix all doclint errors and warnings (so compile does not break with https://github.com/elastic/elasticsearch/pull/13702)
Original commit: elastic/x-pack-elasticsearch@6a05dcd6a2
2015-09-22 00:09:11 -04:00
Tanguy Leroux a5dcb52ba8 Marvel: Add Fielddata stats to index_stats
Closes elastic/elasticsearch#671

Original commit: elastic/x-pack-elasticsearch@eabd7eab20
2015-09-21 16:27:02 +02:00
Tanguy Leroux 34937604e1 Marvel: Remove marvel_ prefix from Marvel document types
Closes elastic/elasticsearch#669

Original commit: elastic/x-pack-elasticsearch@2e04ce186a
2015-09-21 15:58:39 +02:00
uboness 006364279e Fixed bad defaults for html sanitization
- Corrected the `_tables` element list. now it contains the following elements: `<table>`, `<th>`, `<tr>`, `<td>`, `<caption>`, `<col>`, `<colgroup>`, `<thead>`, `<tbody>`, `<tfoot>`

- Added support for empty `<span>` elements

Closes elastic/elasticsearch#660

Original commit: elastic/x-pack-elasticsearch@ce5e57c4aa
2015-09-18 17:39:25 +02:00
Jason Tedor 3ecc12963b Merge pull request elastic/elasticsearch#653 from jasontedor/ints-be-gone
Remove use of com.google.common.primitives.Ints

Original commit: elastic/x-pack-elasticsearch@35a7b4a6ec
2015-09-18 09:45:50 -04:00
Jason Tedor 1a14f7f1a3 Reflect migration of method in core
Original commit: elastic/x-pack-elasticsearch@0039f35646
2015-09-18 08:55:08 -04:00
Jason Tedor c91b5d9a5a Replace Integers.checkedCast with built-in method
Original commit: elastic/x-pack-elasticsearch@6c69849863
2015-09-18 08:55:08 -04:00
Jason Tedor 517b9eaa91 Remove use of com.google.common.primitives.Ints
This commit removes all uses of com.google.common.primitives.Ints
across the codebase.

Relates elastic/elasticsearchelastic/elasticsearch#13224

Original commit: elastic/x-pack-elasticsearch@9500282387
2015-09-18 08:55:08 -04:00
Tanguy Leroux 4d829951b4 Marvel: Disable indexing for nodes attribute in cluster state documents
Closes elastic/elasticsearch#658

Original commit: elastic/x-pack-elasticsearch@bb501aa5b8
2015-09-18 14:24:52 +02:00
Tanguy Leroux db13d99cdf Marvel: Mute IndexStatsCollectorTests
Original commit: elastic/x-pack-elasticsearch@c8746d7ea2
2015-09-18 14:24:28 +02:00
Martijn van Groningen 288274c3c2 test: removed test logging
Original commit: elastic/x-pack-elasticsearch@9ce8f5e7d8
2015-09-18 11:07:35 +02:00
Martijn van Groningen 1e5d2cac11 test: make sure the watch doesn't get throttled
Closes elastic/elasticsearch#640

Original commit: elastic/x-pack-elasticsearch@f1041f6929
2015-09-18 11:02:30 +02:00
Tanguy Leroux fdae10ff74 Marvel: Additional metrics for marvel_node_stats
indices.indexing.index_total
indices.indexing.index_time_in_millis
indices.search.query_total
indices.search.query_time_in_millis

closes elastic/elasticsearch#661

Original commit: elastic/x-pack-elasticsearch@f9892e5b85
2015-09-18 10:25:56 +02:00