This commit cleans most of the methods of XContentBuilder so that:
- Jackson's convenience methods are used instead of our custom ones (ie field(String,long) now uses Jackson's writeNumberField(String, long) instead of calling writeField(String) then writeNumber(long))
- null checks are added for all field names and values
- methods are grouped by type in the class source
- methods have the same parameters names
- duplicated methods like field(String, String...) and array(String, String...) are removed
- varargs methods now have the "array" name to reflect that it builds arrays
- unused methods like field(String,BigDecimal) are removed
- all methods now follow the execution path: field(String,?) -> field(String) then value(?), and value(?) -> writeSomething() method. Methods to build arrays also follow the same execution path.
Original commit: elastic/x-pack-elasticsearch@d83f3aa6e2
This fixes a bug I found with a customer when he updated from 1.x to 2.x.
Due to an BWC incompatible change in the watch history mapping and a thread
pool rejection during execution a watch was not removed from the triggered
watches and tried to be executed again.
While trying to fix it it turned out that the execution of the failure
test case was still done in the transport thread and thus required some
offloading to another thread pool.
Original commit: elastic/x-pack-elasticsearch@df04ce31f2
If someone deletes the watch index (i.e. by deleting all indices), the watcher
in memory store still contains all the watches and tries to execute watches -
which results in exceptions as the watch itself cannot be updated anymore.
In order to minimize this problem (it cant be get rid of completely), we should
act accordingly if the watch index goes missing (either deleted or closed) and
clear out the memory representation of watches in the watchstore as well as trying
to finish all the current executions.
Closeselastic/elasticsearch#2794
Original commit: elastic/x-pack-elasticsearch@12d98cd566
This change moves the logfile audit output from determining what to log based on the
logger level to a enum based configuration that is used by the index output.
A few notable changes were made:
* We alway log all the information we have except for the request body
* The request body is no longer logged by default for REST events; the user needs to
explicitly opt in as there could be sensitive data in the body
* Added a `realm_authentication_failed` event that separates overall authentication
failure from that of an individual realm
Original commit: elastic/x-pack-elasticsearch@343a2bcdd9
This change adds support for disabling users. Users can be disabled by setting the enabled
property to false and the AuthenticationService will check to make sure that the user is enabled.
If the user is not enabled, this will be audited as an authentication failure.
Also as part of this work, the AnonymousUser was cleaned up to remove having a static instance
that caused issues with tests.
Finally, the poller of users was removed to simplify the code in the NativeUsersStore. In our other
realms we rely on the clear cache APIs and the timeout of the user cache. We should have the
same semantics for the native realm.
Closeselastic/elasticsearch#2172
Original commit: elastic/x-pack-elasticsearch@0820e40183
This rewrites the HTTP Exporter to use the REST client underneath. Functionality is improved in resource blocking (templates and pipelines existing) and the majority of the code fundamentall simplified by removing direct HTTP calls.
This is blocked by the SSLService pull request. After that is merged, the I will update this PR to reflect those changes and it could possibly allow us to remove the security privileges required for monitoring.
Original commit: elastic/x-pack-elasticsearch@1ad25f17f8
Basic backwards compatibility support for watcher.
Closeselastic/elasticsearch#3230
Relates to elastic/elasticsearch#3231 - this actually should fix all the failures caused
by fractional time values but it does so by being able to parse them.
Being able to parse them is important for 2.x compatibility but 5.0
watches shouldn't produce fractional time values. This fixes the
particular way of making fractional time values mentioned in elastic/elasticsearch#3231
but I expect there are a half dozen more places to fix. The actual
watcher tests are fairly basic.
Original commit: elastic/x-pack-elasticsearch@328717455c
This publishes X-Pack usage data to the cluster info from the elected master node. This allows phone home to retrieve this data from the index, rather than fetching it live from the connected cluster (thereby not getting it from any n - 1 clusers that are not connceted).
Original commit: elastic/x-pack-elasticsearch@79bfaaaf0b
This removes the "agent" package from org.elasticsearch.xpack.monitoring.agent.*, so that now everything is simply org.elasticsearch.xpack.monitoring.*.
Follow-on work will be refactoring some of the other code, but this is a first step now that it's always the agent (in effect).
Original commit: elastic/x-pack-elasticsearch@14025cb17c
This change migrates xpack (security, watcher, and monitoring) to use the common ssl
configuration for the elastic stack. As part of this work, several aspects of how we deal
with SSL has been modified.
From a functionality perspective, an xpack wide configuration for SSL was added and
all of the code that needs SSL uses the SSLService now. The following is a list of all
of the aspects of xpack that can have their own SSL configuration, which are separate
from the xpack wide configuration:
* Transport
* Transport profiles
* HTTP Transport
* Realms
* Monitoring Exporters
* HTTP Client
In terms of the code, some cleanups were made with these changes. SSLConfiguration is
now a concrete class and SSLConfiguration.Custom and SSLConfiguration.Global have been
removed. The validate method on key and trust configurations has been removed and these
classes will now throw exceptions when they are constructed with bad values. The
OptionalSettings helper class has been removed as it was just a file with one line functions
that made the code harder to understand. The SSL configuration and service classes have
been moved from the security source directories to the main xpack source set. The SSLService
now handles more of the configuration of the SSLEngine it returns to prevent callers from
having to handle those aspects. The settings that get registered for SSL have been moved to
XPackSettings.
Also included in this PR is a update to the docs around SSL. This includes a large simplification to
the documentation in that the certificate authority configuration section has been removed and the
process that is documented for generating certificates only includes the CLI tool that we bundle.
Closeselastic/elasticsearch#3104Closeselastic/elasticsearch#2971Closeselastic/elasticsearch#3164
Original commit: elastic/x-pack-elasticsearch@5bd9e5ef38
* master:
Changes tests to conform with new cluster health API, calling setWaitForNoRelocatingShards(true) instead of setWaitForRelocatingShards(0)
Original commit: elastic/x-pack-elasticsearch@bde6ad8c8a
* master:
Use releasable locks in NativeRolesStore
security: limit the size of the role store cache
security: remove explicit handshake wait in netty4 transport
test: smoke-test-plugins-ssl no longer relies on logging to start
kibana monitoring/uuid config key reference update
Docs: Updated release date for 2.4 in RNs.
Update README.md
Build: Add apijar task to assemble so it gets built with other artifacts
monitoring ui/license: cluster listing status cell treatment for basic/unsupported cluster
monitoring ui:fix cluster overview when cluster has no indices/shards
monitoring ui/license: logic cleanup per feedback
monitoring ui/license: primary cluster asterisk styling
monitoring ui/license: allow clicking into primary cluster if all are basic
monitoring ui: add isPrimary property to cluster listing response
Security: throw exception if we cannot extract indices from an indices request
Security: add tests for delete and update by query
Original commit: elastic/x-pack-elasticsearch@3cb41739ee
Previously the roles store cache was unbounded as it was a just using a ConcurrentHashMap,
which could lead to excessive memory usage in cases where there are a large number of roles
as we tried to eagerly load the roles into the cache if they were not present. The roles store now
loads roles on demand and caches them for a finite period of time.
Additionally, the background polling of roles has been removed to reduce complexity. A best effort
attempt is made to clear the roles cache upon modification and if necessary the cache can be
cleared manually.
See elastic/elasticsearch#1837
Original commit: elastic/x-pack-elasticsearch@450dd779c8
Netty 4's SslHandler does not require the application to wait for the handshake to
be completed before data is written. This change removes the explicit wait on each
handshake future.
Original commit: elastic/x-pack-elasticsearch@c19bcebb83
This commit removes some unnecessary calls to Path#toString when logging
a path in OperationFileModeWatcher#onChage. The calls to Path#toString
are not necessary since the logging infrastructure will do this anyway.
Original commit: elastic/x-pack-elasticsearch@da1bc30d60
This commit removes an unnecessary logging of an exception message from
HttpEmailAttachementParser#toAttachment since the full exception is
logged anyway.
Original commit: elastic/x-pack-elasticsearch@55d4bada00
This commit modifies the call sites that allocate a parameterized
message to use a supplier so that allocations are avoided unless the log
level is fine enough to emit the corresponding log message.
Original commit: elastic/x-pack-elasticsearch@955ab89b8b
This used to be an assertion but we move it to an exception to be able to catch this at all times without requiring assertion enabled
Original commit: elastic/x-pack-elasticsearch@fcb5fbe852
When migrating the package and class names from shield to xpack.security, the logging.yml
file was updated incorrectly which resulted in audit entries going to the wrong file. This change
specifies the correct package name so audit logs go to the correct file.
Closeselastic/elasticsearch#3063
Original commit: elastic/x-pack-elasticsearch@5c07108392
This change allows extensions to define their own filtered settings. This is used when there
may be sensitive credentials stored in a custom realm that should be filtered out of some
API calls.
Closeselastic/elasticsearch#2847
Original commit: elastic/x-pack-elasticsearch@952474daba
With elastic/elasticsearchelastic/elasticsearch#19865 the XContentBuilder has changed to support both inclusive and exclusive filters and now uses Set<String> instead of arrays of Strings. This change updates the various places in x-plugins where string arrays were used.
Original commit: elastic/x-pack-elasticsearch@1f8d4485f4
The actual backwards compatibility support is handled by core's ability to downgrade `keyword` and `text` into `string` for indexes created in 2.x.
Original commit: elastic/x-pack-elasticsearch@6b615d9a45
Although most of the security functionality was disabled when a basic license was applied,
some of the functionality still could be executed such as using the authenticate API or using
a transport client. The issue here is the UI calls the authenticate API and this gives the impression
that security is really in use when it is not.
Original commit: elastic/x-pack-elasticsearch@881453fc4c
The realtime request interceptor was added to stop realtime requests from bypassing DLS
or FLS as the request could read a document from the translog. After
elastic/elasticsearchelastic/elasticsearch#20102 we no longer read documents from the translog so we can
allow realtime requests even when DLS or FLS is enabled.
Original commit: elastic/x-pack-elasticsearch@069b501500
This commit simplifies the code used for resolving the files used for security to always
resolve against the `config` file. Elasticsearch no longer offers a way to disable the security
manager, so the files read by x-pack should not really be configurable and only exist in their
default locations since that is what can be read by the process.
As part of this, the documentation was updated to indicate that these files should always be in
the default location and the settings to change the locations have been removed. Also, a bug
was fixed in a few places where settings were still using `shield.` instead of `xpack.security.`.
Finally, some outdated and unused files were deleted from the repository.
Closeselastic/elasticsearch#305
Original commit: elastic/x-pack-elasticsearch@3884f080a0
This change cleans up some of the log messages and levels that could now be considered misleading.
While performing these cleanups, the following was done:
* remove creation of dummy user for gradle run as we have the `elastic` user
* Request interceptors are not bound if field and document level security is disabled
* FLS/DLS interceptors skip execution if document and field level security is disabled by the license state
* The roles store that loaded the role is logged at the TRACE level
* The TransportXPackUsageAction was using the incorrect action name when registering a handler
Closeselastic/elasticsearch#2096Closeselastic/elasticsearch#1861Closeselastic/elasticsearch#2229
See elastic/elasticsearch#1879
Original commit: elastic/x-pack-elasticsearch@ac16b21c0c
This change allows users with the superuser role to access the security index. We previously allowed
the XPackUser to access this with the intent that the XPackUser was also the `elastic` user. When the
`elastic` user was split out into the ElasticUser, we did not update the check to allow this user access
to the security index.
Original commit: elastic/x-pack-elasticsearch@fa556d9845
This commit changes how we get the file and directory name for certificates in the tool. The
tool now prompts the user for the filename. If the provided instance name will result in a
valid filename, this is provided as a default. Otherwise the user must provide a valid
filename.
Closeselastic/elasticsearch#2854
Original commit: elastic/x-pack-elasticsearch@3c923d736b
This adds back (again) building a transport client plugin jar for
x-pack, and also adds producing an "api" jar which extension authors can
build against. For now, both these jars are exactly the same, but
eventually they could differ, and be reduced to less than the real
x-pack jar.
see elastic/stackelastic/elasticsearch#7
Original commit: elastic/x-pack-elasticsearch@0a989de18b
This reverts building a separate client jar for xpack. It is not
necessary because we already build x-pack as a jar and publish to
elastic maven, since extension authors need that.
Original commit: elastic/x-pack-elasticsearch@2fab06b42c
SecurityTransportModule handled binding two things in guice. First, IPFilter,
for which createComponents already had the necessary dependencies. The
second was ClientTransportFilter. For transport clients, this was a
noop and could be removed. For nodes, this just attaches the system
user, which could be done directly from SecurityServerTransportService.
Original commit: elastic/x-pack-elasticsearch@da327de476
In some cases, the random ascii value could match a forbidden name, which will cause this
test to fail as it expects valid names. This changes ensures the names are valid.
Original commit: elastic/x-pack-elasticsearch@c79f8fc4cc
This commit marks WatcherUtilsTests#testDeserializeSearchRequest which
is failing with seed 2203D3AD59DB5223 as awaits fix.
Original commit: elastic/x-pack-elasticsearch@4b6cbe544b
Simon Willnauer
Jason Tedor
Tanguy Leroux
jaymode
Alexander Reelsen
javanna
Nik Everett
Martijn van Groningen
Chris Earle
Jack Conradson
Ali Beyad
Luca Cavanna
Ryan Ernst
Yannick Welsch
Mike McCandless
Areek Zillur