Commit Graph

3781 Commits

Author SHA1 Message Date
Chris Earle 6f115a532d [Monitoring] Remove "agent" package
This removes the "agent" package from org.elasticsearch.xpack.monitoring.agent.*, so that now everything is simply org.elasticsearch.xpack.monitoring.*.

Follow-on work will be refactoring some of the other code, but this is a first step now that it's always the agent (in effect).

Original commit: elastic/x-pack-elasticsearch@14025cb17c
2016-09-01 10:56:27 -04:00
jaymode ad9a7c9b96 Migrate xpack to use the common ssl configuration
This change migrates xpack (security, watcher, and monitoring) to use the common ssl
configuration for the elastic stack. As part of this work, several aspects of how we deal
with SSL has been modified.

From a functionality perspective, an xpack wide configuration for SSL was added and
all of the code that needs SSL uses the SSLService now. The following is a list of all
of the aspects of xpack that can have their own SSL configuration, which are separate
from the xpack wide configuration:

* Transport
* Transport profiles
* HTTP Transport
* Realms
* Monitoring Exporters
* HTTP Client

In terms of the code, some cleanups were made with these changes. SSLConfiguration is
now a concrete class and SSLConfiguration.Custom and SSLConfiguration.Global have been
removed. The validate method on key and trust configurations has been removed and these
classes will now throw exceptions when they are constructed with bad values. The
OptionalSettings helper class has been removed as it was just a file with one line functions
that made the code harder to understand. The SSL configuration and service classes have
been moved from the security source directories to the main xpack source set. The SSLService
now handles more of the configuration of the SSLEngine it returns to prevent callers from
having to handle those aspects. The settings that get registered for SSL have been moved to
XPackSettings.

Also included in this PR is a update to the docs around SSL. This includes a large simplification to
the documentation in that the certificate authority configuration section has been removed and the
process that is documented for generating certificates only includes the CLI tool that we bundle.

Closes elastic/elasticsearch#3104
Closes elastic/elasticsearch#2971
Closes elastic/elasticsearch#3164

Original commit: elastic/x-pack-elasticsearch@5bd9e5ef38
2016-09-01 10:51:41 -04:00
jaymode 54103127d0 fix line length
Original commit: elastic/x-pack-elasticsearch@caca0fc0b7
2016-09-01 09:37:05 -04:00
jaymode c5cde120b7 test: add 2.4.0 bwc index
Original commit: elastic/x-pack-elasticsearch@58bcf3abaf
2016-09-01 09:09:48 -04:00
jaymode 836e1d3a28 security: add a reserved role for the ingest feature in Kibana
Closes elastic/elasticsearch#1667

Original commit: elastic/x-pack-elasticsearch@da2f6d9c4e
2016-09-01 08:09:36 -04:00
Simon Willnauer 3638c8639f [TEST] Don't set create=true if no ID is specified
Original commit: elastic/x-pack-elasticsearch@874def17e8
2016-09-01 10:35:44 +02:00
Jason Tedor d348d4781a Merge pull request elastic/elasticsearch#3266 from jasontedor/log4j2
Introduce Log4j 2

Original commit: elastic/x-pack-elasticsearch@b1cee13ac8
2016-08-31 23:34:45 -04:00
Jason Tedor b29cdc0877 Fix additional exception logging call
This commit modifies an exception logging calls to use a parameterized
message from Log4j.

Original commit: elastic/x-pack-elasticsearch@7b0df7d056
2016-08-31 23:16:07 -04:00
Jason Tedor 7a3d416558 Merge branch 'master' into log4j2
* master:
  only lint .js and .jsx files
  Designating list and count APIs as system APIs
  Inverting logic
  Use system API module function exported by Kibana plugin
  monitoring ui/cluster row: IsClusterSupported helper
  monitoring ui: Initial Test Automation Hooks
  monitoring ui/license: toaster content update
  monitoring ui: onClick handler syntax polish
  monitoring ui/license: updating wording on unsupported cluster toaster
  monitoring ui:fix default min shard replication to `N/A`
  monitoring ui: empty state cleanups
  monitoring ui/license: fix redirect issue with license expiry page + back button
  monitoring ui:cluster listing treatment for clusters w/ invalid and unsupported license
  monitoring ui: remove “health” check for cluster listing
  monitoring ui: show clusters that have had license deleted
  Designate certain API calls as system APIs + treat them as special

Original commit: elastic/x-pack-elasticsearch@95865f89ac
2016-08-31 22:35:02 -04:00
Jason Tedor cd0f17c372 Merge branch 'master' into log4j2
* master:
  Changes tests to conform with new cluster health API, calling setWaitForNoRelocatingShards(true) instead of setWaitForRelocatingShards(0)

Original commit: elastic/x-pack-elasticsearch@bde6ad8c8a
2016-08-31 16:17:28 -04:00
Ali Beyad 8757c2f6e1 Changes tests to conform with new cluster health API, calling
setWaitForNoRelocatingShards(true) instead of setWaitForRelocatingShards(0)

Original commit: elastic/x-pack-elasticsearch@c7c12fe64c
2016-08-31 11:56:38 -04:00
Jason Tedor a89f0110f4 Merge branch 'master' into log4j2
* master:
  Use releasable locks in NativeRolesStore
  security: limit the size of the role store cache
  security: remove explicit handshake wait in netty4 transport
  test: smoke-test-plugins-ssl no longer relies on logging to start
  kibana monitoring/uuid config key reference update
  Docs: Updated release date for 2.4 in RNs.
  Update README.md
  Build: Add apijar task to assemble so it gets built with other artifacts
  monitoring ui/license: cluster listing status cell treatment for basic/unsupported cluster
  monitoring ui:fix cluster overview when cluster has no indices/shards
  monitoring ui/license: logic cleanup per feedback
  monitoring ui/license: primary cluster asterisk styling
  monitoring ui/license: allow clicking into primary cluster if all are basic
  monitoring ui: add isPrimary property to cluster listing response
  Security: throw exception if we cannot extract indices from an indices request
  Security: add tests for delete and update by query

Original commit: elastic/x-pack-elasticsearch@3cb41739ee
2016-08-31 10:26:30 -04:00
Jason Tedor 82079185c2 Use releasable locks in NativeRolesStore
This commit replaces the use of try/finally blocks to handle safe
locking/unlocking for role cache read/write lock with releasable locks
in try-with-resources blocks.

Relates elastic/elasticsearch#3278

Original commit: elastic/x-pack-elasticsearch@fbd659cd85
2016-08-31 10:18:09 -04:00
jaymode 7d78911082 security: limit the size of the role store cache
Previously the roles store cache was unbounded as it was a just using a ConcurrentHashMap,
which could lead to excessive memory usage in cases where there are a large number of roles
as we tried to eagerly load the roles into the cache if they were not present. The roles store now
loads roles on demand and caches them for a finite period of time.

Additionally, the background polling of roles has been removed to reduce complexity. A best effort
attempt is made to clear the roles cache upon modification and if necessary the cache can be
cleared manually.

See elastic/elasticsearch#1837

Original commit: elastic/x-pack-elasticsearch@450dd779c8
2016-08-31 09:22:25 -04:00
jaymode 06ff97f63d security: remove explicit handshake wait in netty4 transport
Netty 4's SslHandler does not require the application to wait for the handshake to
be completed before data is written. This change removes the explicit wait on each
handshake future.

Original commit: elastic/x-pack-elasticsearch@c19bcebb83
2016-08-31 08:23:51 -04:00
jaymode 942a70328c test: smoke-test-plugins-ssl no longer relies on logging to start
This change adds a HTTPS check for smoke-test-plugins-ssl so it no longer has to wait for
a debug level log message.

Closes elastic/elasticsearch#2303

Original commit: elastic/x-pack-elasticsearch@f3eaaad5d4
2016-08-31 08:08:41 -04:00
Luca Cavanna 82649355a0 Merge pull request elastic/elasticsearch#3091 from javanna/fix/reindex_repleaceable
Security: add tests for delete and update by query

Original commit: elastic/x-pack-elasticsearch@6f6786a26e
2016-08-31 09:48:34 +02:00
Jason Tedor b1f97ea5ed Remove unnecessary calls to Path#toString
This commit removes some unnecessary calls to Path#toString when logging
a path in OperationFileModeWatcher#onChage. The calls to Path#toString
are not necessary since the logging infrastructure will do this anyway.

Original commit: elastic/x-pack-elasticsearch@da1bc30d60
2016-08-30 21:08:20 -04:00
Jason Tedor 4cd3e02206 Remove empty Javadocs in ExecutableSearchTransform
This commit removes some inadvertent blank Javadocs for
ExecutableSearchTransform.

Original commit: elastic/x-pack-elasticsearch@b73a5860cb
2016-08-30 21:05:18 -04:00
Jason Tedor 71b7818ead Remove unnecessary logging of exception message
This commit removes an unnecessary logging of an exception message from
HttpEmailAttachementParser#toAttachment since the full exception is
logged anyway.

Original commit: elastic/x-pack-elasticsearch@55d4bada00
2016-08-30 21:03:46 -04:00
Tim Sullivan e5f0a38613 Merge pull request elastic/elasticsearch#3149 from tsullivan/monitoring-ui-identify-primary-cluster
Monitoring UI: Identify Primary Cluster

Original commit: elastic/x-pack-elasticsearch@ddd104635b
2016-08-30 15:29:19 -07:00
Jason Tedor 0a285a9a0c Avoid allocating log parameterized messages
This commit modifies the call sites that allocate a parameterized
message to use a supplier so that allocations are avoided unless the log
level is fine enough to emit the corresponding log message.

Original commit: elastic/x-pack-elasticsearch@955ab89b8b
2016-08-30 18:20:15 -04:00
Ryan Ernst 1bfef646b8 Build: Add apijar task to assemble so it gets built with other artifacts
Original commit: elastic/x-pack-elasticsearch@ab9ce9c8d6
2016-08-30 11:23:22 -07:00
Jason Tedor 702d55f0be Introduce Log4j 2
This commit introduces Log4j 2 to the stack.

Original commit: elastic/x-pack-elasticsearch@02b2bb1c89
2016-08-30 13:29:28 -04:00
Lukas Olson 9086e3c43e Merge pull request elastic/elasticsearch#3153 from lukasolson/fix/account-page-link
[security-ui] Fix link to account page so it works from other apps

Original commit: elastic/x-pack-elasticsearch@6032bd90f3
2016-08-30 07:52:48 -07:00
Nik Everett da8540b10b Handle removing more ctors
Original commit: elastic/x-pack-elasticsearch@dd1b62ba71
2016-08-30 10:27:45 -04:00
Nik Everett 2049a45f52 Handle core chaning IndexRequest
Original commit: elastic/x-pack-elasticsearch@8cd311b3e8
2016-08-30 10:27:45 -04:00
javanna 603db388d7 Security: throw exception if we cannot extract indices from an indices request
This used to be an assertion but we move it to an exception to be able to catch this at all times without requiring assertion enabled

Original commit: elastic/x-pack-elasticsearch@fcb5fbe852
2016-08-30 15:23:02 +02:00
javanna fdfc66a8ba Security: add tests for delete and update by query
Original commit: elastic/x-pack-elasticsearch@e85877d03f
2016-08-30 15:22:55 +02:00
jaymode 88abfcfea9 security: fix audit logging.yml to have the correct prefix
When migrating the package and class names from shield to xpack.security, the logging.yml
file was updated incorrectly which resulted in audit entries going to the wrong file. This change
specifies the correct package name so audit logs go to the correct file.

Closes elastic/elasticsearch#3063

Original commit: elastic/x-pack-elasticsearch@5c07108392
2016-08-30 07:39:25 -04:00
jaymode bb4777b7d6 allow extensions to define settings filter
This change allows extensions to define their own filtered settings. This is used when there
may be sensitive credentials stored in a custom realm that should be filtered out of some
API calls.

Closes elastic/elasticsearch#2847

Original commit: elastic/x-pack-elasticsearch@952474daba
2016-08-30 07:20:00 -04:00
jaymode 473728afb4 security: extract indices from CompositeIndicesRequests and exclude duplicates
This change allows us to log the indices from a composite indices request by getting the indices
from all of the sub requests. Additionally, indices are no longer duplicated when auditing.

Closes elastic/elasticsearch#2623
Closes elastic/elasticsearch#2618

Original commit: elastic/x-pack-elasticsearch@f1d3d87bcf
2016-08-30 07:06:04 -04:00
Tanguy Leroux d3dff6659b Returns empty set of filters in MonitoringIndexNameResolver and fix tests
Original commit: elastic/x-pack-elasticsearch@e901347de9
2016-08-30 10:10:08 +02:00
Tanguy Leroux 203ad03d26 Use sets of String in XContentBuilders
With elastic/elasticsearchelastic/elasticsearch#19865 the XContentBuilder has changed to support both inclusive and exclusive filters and now uses Set<String> instead of arrays of Strings. This change updates the various places in x-plugins where string arrays were used.

Original commit: elastic/x-pack-elasticsearch@1f8d4485f4
2016-08-30 09:08:49 +02:00
Nik Everett 8f22eaf1b7 Security: Add tests for indexes created in 2.x (elastic/elasticsearch#3203)
The actual backwards compatibility support is handled by core's ability to downgrade `keyword` and `text` into `string` for indexes created in 2.x.

Original commit: elastic/x-pack-elasticsearch@6b615d9a45
2016-08-29 12:00:19 -04:00
Yannick Welsch cdc41f6082 Add recovery source to ShardRouting
Relates to elastic/elasticsearchelastic/elasticsearch#19516

Original commit: elastic/x-pack-elasticsearch@051fb25bfc
2016-08-27 15:19:49 +02:00
jaymode 40c2672f12 test: remove the version compatibility tests
The x-plugins version is now kept in sync with the elasticsearch version and the
VersionCompatibilityTests just adds noise when updating versions.

Closes elastic/elasticsearch#3079
See elastic/elasticsearch#3212

Original commit: elastic/x-pack-elasticsearch@5998aa2ec0
2016-08-26 13:52:35 -04:00
jaymode e07ae87cf4 security: do not allow security APIs to execute when unlicensed
Although most of the security functionality was disabled when a basic license was applied,
some of the functionality still could be executed such as using the authenticate API or using
a transport client. The issue here is the UI calls the authenticate API and this gives the impression
that security is really in use when it is not.

Original commit: elastic/x-pack-elasticsearch@881453fc4c
2016-08-26 13:46:56 -04:00
Nik Everett 9c2b3d79ad Fix bulk update tests
It was relying on fields extracting fields from the source but it
doesn't do that any more.

Original commit: elastic/x-pack-elasticsearch@23b534c068
2016-08-26 12:52:06 -04:00
Tyler Smalley 7c231a1edc Merge pull request elastic/elasticsearch#3192 from elastic/3132-node-6
Upgrade to Node 6.4.0

Original commit: elastic/x-pack-elasticsearch@e1101ed947
2016-08-25 13:53:11 -07:00
Mike McCandless ab074da873 Don't illegally reuse scorer in this test
Closes elastic/elasticsearch#3216

Original commit: elastic/x-pack-elasticsearch@63654c5585
2016-08-25 11:42:48 -04:00
Court Ewing 6d361d0097 Merge pull request elastic/elasticsearch#3215 from epixa/fix-kibanalinting
monitoring-ui: fix outstanding linting errors

Original commit: elastic/x-pack-elasticsearch@c7c459937e
2016-08-25 11:10:02 -04:00
jaymode 098e61fbc1 security: remove the realtime request interceptor
The realtime request interceptor was added to stop realtime requests from bypassing DLS
or FLS as the request could read a document from the translog. After
elastic/elasticsearchelastic/elasticsearch#20102 we no longer read documents from the translog so we can
allow realtime requests even when DLS or FLS is enabled.

Original commit: elastic/x-pack-elasticsearch@069b501500
2016-08-25 10:22:46 -04:00
jaymode dfdf77c536 security: upgrade to bouncy castle 1.55
Bouncy castle 1.55 was released earlier today and this commit updates the version
we use to 1.55.

Original commit: elastic/x-pack-elasticsearch@95e002431b
2016-08-25 09:41:12 -04:00
Lukas Olson a02114729f Merge pull request elastic/elasticsearch#3172 from w33ble/shield-api-route-tags
Check route tags for apis, don't redirect

Original commit: elastic/x-pack-elasticsearch@5414e126bf
2016-08-23 18:02:46 -07:00
jaymode 26c1da4230 security: roles.yml is empty and add built in monitoring users
This change removes all default roles from the roles.yml file that is distributed with xpack and
adds built in roles for monitoring users and remote monitoring agents.

Closes elastic/elasticsearch#3122

Original commit: elastic/x-pack-elasticsearch@b04508bd56
2016-08-23 15:11:28 -04:00
jaymode 334aa94946 security: simplify lookup of files inside the config dir
This commit simplifies the code used for resolving the files used for security to always
resolve against the `config` file. Elasticsearch no longer offers a way to disable the security
manager, so the files read by x-pack should not really be configurable and only exist in their
default locations since that is what can be read by the process.

As part of this, the documentation was updated to indicate that these files should always be in
the default location and the settings to change the locations have been removed. Also, a bug
was fixed in a few places where settings were still using `shield.` instead of `xpack.security.`.
Finally, some outdated and unused files were deleted from the repository.

Closes elastic/elasticsearch#305

Original commit: elastic/x-pack-elasticsearch@3884f080a0
2016-08-23 14:54:18 -04:00
Ryan Ernst 1aa72336a0 Merge pull request elastic/elasticsearch#3191 from rjernst/api_jar
Build: Add api jar and client jar for xpack

Original commit: elastic/x-pack-elasticsearch@7743f43e23
2016-08-23 11:51:05 -07:00
jaymode 7536acdc9f security: cleanup logging and other minor enhancements/fixes
This change cleans up some of the log messages and levels that could now be considered misleading.

While performing these cleanups, the following was done:

* remove creation of dummy user for gradle run as we have the `elastic` user
* Request interceptors are not bound if field and document level security is disabled
* FLS/DLS interceptors skip execution if document and field level security is disabled by the license state
* The roles store that loaded the role is logged at the TRACE level
* The TransportXPackUsageAction was using the incorrect action name when registering a handler

Closes elastic/elasticsearch#2096
Closes elastic/elasticsearch#1861
Closes elastic/elasticsearch#2229
See elastic/elasticsearch#1879

Original commit: elastic/x-pack-elasticsearch@ac16b21c0c
2016-08-23 14:35:23 -04:00
jaymode 507196dca5 security: allow superusers access to the security index
This change allows users with the superuser role to access the security index. We previously allowed
the XPackUser to access this with the intent that the XPackUser was also the `elastic` user. When the
`elastic` user was split out into the ElasticUser, we did not update the check to allow this user access
to the security index.

Original commit: elastic/x-pack-elasticsearch@fa556d9845
2016-08-23 14:24:43 -04:00