SecurityTransportModule handled binding two things in guice. First, IPFilter,
for which createComponents already had the necessary dependencies. The
second was ClientTransportFilter. For transport clients, this was a
noop and could be removed. For nodes, this just attaches the system
user, which could be done directly from SecurityServerTransportService.
Original commit: elastic/x-pack-elasticsearch@da327de476
In some cases, the random ascii value could match a forbidden name, which will cause this
test to fail as it expects valid names. This changes ensures the names are valid.
Original commit: elastic/x-pack-elasticsearch@c79f8fc4cc
This commit marks WatcherUtilsTests#testDeserializeSearchRequest which
is failing with seed 2203D3AD59DB5223 as awaits fix.
Original commit: elastic/x-pack-elasticsearch@4b6cbe544b
This commit marks SearchTransformTests#testParser which is failing with
seed 97BC9E2543410D55 as awaits fix.
Original commit: elastic/x-pack-elasticsearch@3d69e9648c
This commit marks WatcherUtilsTests#testSerializeSearchRequest which is
failing with seed 97BC9E2543410D55 as awaits fix.
Original commit: elastic/x-pack-elasticsearch@45a174c1f2
With the latest clean ups and changes in Watcher, the default search template lang has been switched to WatcherScript.DEFAULT_LANG which points to "groovy" but it should be "mustache" instead.
Original commit: elastic/x-pack-elasticsearch@1d9ef1963e
This adds a "condition" to every action (via the ActionWrapper) that prevents execution of the action if the condition fails. An action-level condition is only useful when there is more than one action, but nothing checks to ensure that it's only used in that scenario.
Original commit: elastic/x-pack-elasticsearch@704cfb1a86
Instead watcher search template should be agnostic of SearchSourceBuilder and just work with BytesReference, so that serializing to a string before templating isn't needed.
Original commit: elastic/x-pack-elasticsearch@36d21ec819
This roughly matches what was done in 2.x for the license tools (except
without any dependency exclusions).
Original commit: elastic/x-pack-elasticsearch@4e1f07b5fa
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#20000, using the new
SearchRequestParsers.
Original commit: elastic/x-pack-elasticsearch@d7cee93a8f
This adds an empty _ingest/pipeline named after the _current_ version of the Monitoring API (currently 2) from both exporters.
This allows us to eventually change the API version (to 3, 4, etc.) and go _back_ and modify the pipeline that exists. The modified pipeline would then "fix" the documents as best as possible and rename the index. As a result, different versions (starting with 5.0) will be able to report to the same monitoring cluster regardless of the running API version.
Note: This has no impact on stale data (e.g., the day before the upgrade) _and_ it implies that the monitoring cluster should always be updated first. A simple reindexing script can be supplied for old data, which can be done at the discretion of the user.
Original commit: elastic/x-pack-elasticsearch@45df5ee87b
The SSLConfigurationReloaderTests rarely failed during some local runs. This turned out to be due to
signaling that the reload happened before we actually reloaded. This led to a race condition where we
attempted to validate the config was reloaded properly and actually reloading. This change fixes the
ordering of operations and uses a CountDownLatch instead of a AtomicInteger and awaitBusy.
Original commit: elastic/x-pack-elasticsearch@9615f225d6
The XPackFeatureSet.Usage writeTo method failed to write the name to the
StreamOutput whereas the deserialization code expected it to be the first value
present in the StreamInput. This causes xpack usage requests made to nodes
other than the master to fail.
This change fixes the serialization and adds tests to all of the feature sets to ensure
that the usage is serialized properly.
Closeselastic/elasticsearch#3072
Original commit: elastic/x-pack-elasticsearch@eccab616ca
This change refactors the ssl services into a single class and reduces the number of
public methods that are exposed by the class. As part of this change, ssl configurations
must be loaded during construction of the service otherwise an exception will be thrown
when trying to retrieve the configuration. Additionally, the reloading of key material and
trust material is now handled by the SSLService instead of the SSLConfiguration. The
SSLConfiguration class has been made package private as its usage has been contained
to the ssl package entirely.
See elastic/elasticsearch#2971
Original commit: elastic/x-pack-elasticsearch@46e4535bb1
Currently there are a number of helper methods, all used at various
stages of initializing xpack, for reading "enabled" settings. Almost all
of these read the setting directly rather than use the new settings
infra. This change adds an XPackSettings class to contain these settings
constants, and adds all the enabled settings to it.
Note: While there is an existing MonitoringSettings, I think we do not
have enough settings (especially for all features) to warrant a separate
class per "feature" (eg graph would have nothing). We should decide
if/when this is necessary on a case by case basis, but I think the
enabled settings should all go here in XPackSettings.
Original commit: elastic/x-pack-elasticsearch@086152d5a5
When security is disabled, we currently set the transport and HTTP type
to Netty 4. However, this is unnecessary as these are the default
settings so this commit removes these explicit settings.
Relates elastic/elasticsearch#3065
Original commit: elastic/x-pack-elasticsearch@afa360ceec
This commit reenable some Watcher tests that were muted in elastic/elasticsearch#724. It removes the Groovy dependency for tests that don't really use Groovy features and replace scripts with mocked scripts. It converts a GroovyScriptConditionIT test into a REST test and moves it in the smoke-test-watcher-with-groovy project.
closeselastic/elasticsearch#724
Original commit: elastic/x-pack-elasticsearch@f4c8161946
These mock Netty plugins were needed to assist in setting
sun.nio.ch.bugLevel. Yet, settings this property should no longer be
needed on JDK 8 and the missing privileged block when writing this
property can be fixed upstream.
Relates elastic/elasticsearch#3047
Original commit: elastic/x-pack-elasticsearch@e2aa4f6fb5
This commit updates Jackson to the 2.8.1 version, which is more strict when it comes to build objects.
Original commit: elastic/x-pack-elasticsearch@fb23208e7d
This change removes guice from creating notification componenents. It
also removes the Notification helper class, as it just makes looking at
what stuff xpack adds more obfuscated.
Original commit: elastic/x-pack-elasticsearch@69b8ea4735
When a watch is executed, but the condition is not met, it is not automatically marked as dirty.
This results in a divergence of what is displayed when someone is just getting a watch via the get
watch API (due to using the in memory store) compare to when someone is using the GET API to get the
watch via the .watches index call.
This commit sets the watch status always to dirty after a condition check, thus forcing an update.
Closeselastic/elasticsearch#2699
Original commit: elastic/x-pack-elasticsearch@e6a00260f7
Users are allowed to create query objects with an empty string value as we do
not currently validate the input against a query parser. In this case, we can ignore
the empty value when parsing the role. If we pass an empty BytesReference in
the role then trying to determine the XContentType will fail in the
SecurityIndexSearcherWrapper.
Closeselastic/elasticsearch#2997
Original commit: elastic/x-pack-elasticsearch@fc593943c4
This commit collapes some duplicated setup code in every test in the
Netty 4 security transport tests to a common method.
Original commit: elastic/x-pack-elasticsearch@0a088db78d
* master: (25 commits)
docs: remove unused file and add link to invalid role examples
Remove interfaces for notification services
Redirect to URL specified by next parameter, if it is set
Fix fixture and tests so they pass again
Update error message to be more actionable
Switch to NamedWriteable pull based extension in xpack
Fixing issue with infinite redirect loop
Toggle display of security nav controls more dynamically
Pass in xpackMainPlugin instead of xpackMainPlugin.info
Wrap the return in a Promise
Only unset the cookie if it is currently set
Clarifying intent of code
Updating tests fixtures + adding assertion for client cookie deletion
If security is disabled, do not attempt to call the authenticate ES API
Disambiguate between resolve function names
Revert to not using xpackMainPlugin.info until the xpackMainPlugin is ready
Redirect /login => / if security is disabled in ES
Register/deregister security management items depending on whether there's an auth'd user
Show/hide the username + logout button depending on whether there is an auth'd user
If security is disabled, continue without auth + delete client cookie
...
Original commit: elastic/x-pack-elasticsearch@16b92a1a59
We have 4 types of notification services, and all of them have an
interface with a single implementation class. They also all
unnecessarily are lifecycle componenets, but the only thing start does
is read the settings.
This change converts all 4 notification services to classes, and makes
them regular components instead of lifecycle services.
Original commit: elastic/x-pack-elasticsearch@897115ae65
After this commit, we have "security3" and "security4" as possible
transport and HTTP settings, we default to "security4" if it is not set,
and we randomize the selection in some of the integration tests.
Original commit: elastic/x-pack-elasticsearch@e56718911a
This commit cleans up the Netty handshake waiting handlers. We rename
the Netty 3 implementation to include "Netty3" in the name, the Netty 4
implementation is not needed, and we improve the handling of waiting for
the handshakes to complete when connecting.
Original commit: elastic/x-pack-elasticsearch@f736fdc8f0
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#19764. It converts
the one use of registering custom NamedWriteable readers in xpack for
xpack feature sets to register them up front with the new pull based
registration.
Original commit: elastic/x-pack-elasticsearch@48e2020816
Since elastic/elasticsearch#19621 MockScriptEngine is stricter and expects scripts to be defined before being used in tests. Because watcher makes heavy use of scripts without really need of custom logic, this commit changed the MockMustacheScriptEngine implementation so that it always returns the script's source as a result.
Original commit: elastic/x-pack-elasticsearch@09cffa9517
This commit moves the reloading and monitoring of files from the trust/key configuration
classes into a separate class that will reload for the whole SSLConfiguration object.
SSLContexts are loaded lazily by most of security, so a listener interface was added to
notify the reloader that there may be other paths to monitor.
Original commit: elastic/x-pack-elasticsearch@1633cc14a7
The tests have been moved back to xpack, turned into a rest test or removed.
For example testing specific inline, file or stored mustache template functionality is already covered in the `lang-mustache` module. The smoke-test-watcher-with-mustache should tests watcher mustach specific things like the if the watcher variables are available.
Original commit: elastic/x-pack-elasticsearch@e434bcd3fa
This makes use of the `registerAsDeprecatedHandler` method to automatically warn users when they're using deprecated functionality.
This will also automatically provide a `Warning` header for anyone using HTTP clients (though they have to be looking for it...).
Graph portion only
Original commit: elastic/x-pack-elasticsearch@d6452a75c1
The value of the setting name had a typo.
Also added a message to show the value of globally configured proxy on startup
to help admins spot if this is configured.
Original commit: elastic/x-pack-elasticsearch@bdc41ff733
This changes the usage stats for the realms to not use nested objects but to instead group
values by the realm type. Additionally, the realms now return the actual size of the users that
are contained or have recently logged in (in case of LDAP/AD).
Additionally, the audit section will always contain an enabled flag and a new field has been
added to capture whether anonymous access is enabled or not.
See elastic/elasticsearch#2210
Original commit: elastic/x-pack-elasticsearch@9cc2852585
There are still tests left to be cleanup, but these tests need to be turned into a rest test and moved the smoke test xpack with mustache module.
Original commit: elastic/x-pack-elasticsearch@3b88b15b97
We currently have the license:base project, as well as license:licensor.
Xpack depends on license base. This change moves all the base license
code into xpack, and makes the license-tools (previously called
licensor) depenend on x-pack.
Original commit: elastic/x-pack-elasticsearch@7fd462ad6a
In 2.x, 1.x license format in cluster state was upgraded
to the 2.x format. This commit removes the code to read
1.x license format from cluster state in 5.x, as 2.x clusters
will already upgrade the license format to 2.x format.
Original commit: elastic/x-pack-elasticsearch@77f18ffc76
This change moves all license related code in xpack under
org.elasticsearch.license. Some classes become package private (more can
become package private later with further deguicing). It also adds
package level javadocs. Note, I did not move anything from the actual
licensing jar. This should be cleaned up in a followup.
Original commit: elastic/x-pack-elasticsearch@2413c4f288
This change removes guice from construction of monitoring classes.
Additionally, it removes disk threshold watermark and enabled setting
from node stats collector. These were not node stats, just cluster
settings. If we want to add back actual percentage of disk threshold
used, it should be in node stats directly.
Original commit: elastic/x-pack-elasticsearch@4cd49557cf
If the result of a search actions fails (i.e. because the index you queried does not exist yet),
the watch record failed to store into the Watch History because of a mapping issue, as the
template path match regular expression did not match properly.
Closeselastic/elasticsearch#2913
Original commit: elastic/x-pack-elasticsearch@3c2d4b3ca9
This is dependent on elastic/elasticsearchelastic/elasticsearch#19589. It also fixes the
existing files that either were missing the license header or had the
open source license header.
Original commit: elastic/x-pack-elasticsearch@2642c20381
The response status code was stored in the result of an http input,
but inaccessible in the payload itself and could not be used in
scripts.
This puts the status code in the payload under the name '_status_code',
similar to the '_headers' variable, which already stores the headers.
Original commit: elastic/x-pack-elasticsearch@dff2a39535
Background: When a watch has been acked, but the condition evaluates to false again,
the watch must be marked as dirty - which means it needs to be persisted to the watches
index - so in case of a master node switch this information is not lost.
This commit fixes the setting of the `dirty` field in the watch status, in case
the condition is not met, but some actions have been acked.
Original commit: elastic/x-pack-elasticsearch@1a55a45b14
The license code currently has a Licensee implementation per feature,
which is updated by the license service. This meant maintaining a
listener type feature for the license service, and having an abstract
listener and such. The licensee class also mixed in stuff only needed by
the license service (acknowledgement messages).
This change collapses all the methods from licensees into
XPackLicenseState. The naming was inconsistent across licensee
implementations, so here it is standardized on `is*Allowed()`. There are
also a number of tests which should be consolidated for testing the
license service but that is left for a future change. I also removed
collector tests that were testing license: that is better left for the
direct tests of the license state in XPackLicenseStateTests.
Original commit: elastic/x-pack-elasticsearch@734871e870
The LicenseState class exists to distinguish when a license is enabled,
vs being inside its grace period. However, the consumers of this state
do not care whether the license is in the grace period, they view that
and an active license as the same thing. The only part that cares about
the grace period is in the license service which logs a warning when a
license begins its grace period.
This change removes the LicenseState enum in favor of a simple boolean
indicating whether the license is active.
Original commit: elastic/x-pack-elasticsearch@5a90a0e3d4
As the specified integration tests were better suited as REST tests, I changed them into
those and removed the (partially already unused) java classes.
Original commit: elastic/x-pack-elasticsearch@f26d8d94e8
We only have one license in 5.x. This change renames the service that
mantains the license state on each node to reflect that fact.
Original commit: elastic/x-pack-elasticsearch@bb241b30cb
This change removes guice from most of the rest of security. It also
converts the last use of onModule in xpack extensions to a pull based
extension.
Original commit: elastic/x-pack-elasticsearch@9de072550e
Both AuthenticationService and AuthorizationService are currently
interfaces with single implementations. This is unnecessary, and makes
it harder to deguice. This change removes the abstractions and leaves
just AuthenticationService and AuthorizationService.
Original commit: elastic/x-pack-elasticsearch@d04c897ae4
AuditTrailService
We currently have a number of actions and components which try to write
to the audit trail. But they do so by expecting a bound AuditTrail
object. In reality, this should always be AuditTrailService, except when
security is disabled. This change removes the use of the AuditTrail
interface for that purpose, and instead makes the AuditTrailService
allow an empty list of trails, so that it is always bound when running
on a node.
Original commit: elastic/x-pack-elasticsearch@9559dbd6c1
This commit removes the SelfReschedulingRunnable and changes the native stores
to use the threadpool for scheduling again since we have now fixed the issue in core.
Original commit: elastic/x-pack-elasticsearch@50030e31ff
This is useful if an index is shared with many small customers, which are to small to have their own index or shard,
and in order to share an index safely they will need to use document level security. This processor can then automatically
add the username or roles of the current authenticated user to the documents being indexed, so that the DLS query can be simple. (`username: abc` only return data inserted by user abc)
Closeselastic/elasticsearch#2738
Original commit: elastic/x-pack-elasticsearch@f4df2f6d6f
Currently the security context is an object passed around to code
needing to check the user for the current request. Like recent
InternalClient changes, it current depends on the AuthenticationService,
but can be simplified by only knowing about the thread context and
crypto service. This change makes SecurityContext a class, instead of an
interface, and removes the dependency on AuthenticationService.
Original commit: elastic/x-pack-elasticsearch@b8af75e8cb
Followup of elastic/elasticsearchelastic/elasticsearch#19435
Relates to elastic/elasticsearchelastic/elasticsearch#19412
Original commit: elastic/x-pack-elasticsearch@60f7047ea9
This commit is a combination of enhancements and fixes to the active directory
and ldap realms. The active directory realm has been enhanced to add support
for authentication against multiple domains in a forest. The ldap realm has
been updated so that:
* attributes required for group resolution are loaded eagerly if possible
* user search can now be executed using unpooled connections
* the default search filter for groups now includes posixGroup and memberUid
to avoid users needed to understand ldap filters
Finally, the UnboundID LDAP SDK was upgraded to the latest version and some
long standing AwaitsFix were addressed.
Closeselastic/elasticsearch#20Closeselastic/elasticsearch#26Closeselastic/elasticsearch#1950Closeselastic/elasticsearch#2145Closeselastic/elasticsearch#2363
Original commit: elastic/x-pack-elasticsearch@63c9be2337
This commit adds the ability to define metadata for roles. This metadata is currently
only used for the API and to indicate that a role is reserved. We can continue passing
on the metadata as needed, when necessary.
Closeselastic/elasticsearch#2036
Original commit: elastic/x-pack-elasticsearch@8b5f606138
This commit adds a CLI tool that can be used to generate a CA and signed certificates in PEM
format. The tool only requires a name of an instance to be provided by the user; ip and dns values
are supported but optional. By default, the tool is interactive and will prompt the user for input but
an option exists to provide a yaml file that contains the necessary information to generate certificates
or signing requests.
The output is in the form of a zip file with subfolders for each instance. Neither the zip file or the PEM
files are encrypted as some parts of our stack do not support encrypted PEM files.
Original commit: elastic/x-pack-elasticsearch@3dc0f8d495
When a HTTP input has a configured response content, then this should
always be treated as preferred over the content type that is returned
by the server in order to give the user the power to decide.
This also refactors the code a bit to make it more readable.
Closeselastic/elasticsearch#2211
Original commit: elastic/x-pack-elasticsearch@ecdb4f931c
ScriptServiceProxy is a thin wrapper around the ScriptService which does
a runAs the xpack user when compiling. But script services know nothing
about xpack users, so this has no real effect. I believe this is a
remnant of when we had indexed scripts, where the compilation may have
done a get on the scripts index.
This change removes the ScriptServiceProxy. It also renames Script in
watcher to WatcherScript, to remove confusion between elasticsearch's
Script and watchers Script.
Original commit: elastic/x-pack-elasticsearch@4e2fdbc518
This change makes the internal realms factories, as well as those added
by extensions, constructed directly instead of via guice. Adding realms
in extensions is now pull based. Finally, all of the generics for realms
and realm factories have been removed.
Original commit: elastic/x-pack-elasticsearch@f0de9d2340
"cloud_internal" license type enables dynamically updating license operation mode via a config file.
When the installed license is "cloud_internal", the node level operation mode can be updated by writing
a `license_mode` file in the x-pack config directory (config/x-pack/license_mode). The file is expected
to have a string representing the desired license mode (e.g. "gold", "basic"). In case of a failure to
read a valid license mode from the `license_mode` file, the operation mode for "cloud_internal" license
defaults to PLATINUM.
This change also ensures that the correct operation mode is reported via the _xpack endpoint.
closeselastic/elasticsearch#2042
Original commit: elastic/x-pack-elasticsearch@6a2d788e45
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#19440. It adds a
getter on XPackExtension for extensions that add custom rest headers, in
addition to the headers registered for xpack itself.
Original commit: elastic/x-pack-elasticsearch@bd142b88c6
This commit moves an awaits fix from
MonitoringIndexNameResolverTestCase#testResolver to
MonitoringIndexNameResolverTestCase#testSource as the previous commit
elastic/x-pack@98e76642ea marked the incorrect test as
awaits fix.
Original commit: elastic/x-pack-elasticsearch@62e9aaa628
The InternalClient is used by xpack code making requests to other nodes,
to add the xpack user to the thread context. To do this, it uses has a
dependency on the AuthenticationService, which in turn transitively
depends on InternalClient (eg IndexAuditTrail). But to add the xpack
user, the full authentication service is not necessary. Only having the
crypto service is needed in order to encrypt the header.
This change simplifes construction of InternalClient both making it a
real class instead of an interface, and removing the dependency on the
AuthenticationService. It also removes the use of
Provider<InternalClient> in users of the client.
Original commit: elastic/x-pack-elasticsearch@10f633bdf3
Both logfile and index audit trails currently depend on injection of
Transport in order to find the bound address of the local node. However,
the ClusterService provides access to information about the local node,
including the bound addresses. This change makes the audit trails use
the cluster service, and also makes the logging audit trail not use a
lifecycle.
Original commit: elastic/x-pack-elasticsearch@d747d64ee1
This provides the same -- more useful -- index memory chart to the index page, instead of just "Lucene Memory", it now breaks down what it shows just like the Node page as as more generalized "Index Memory" chart. It also rolls Fielddata into the Index Memory. With Fielddata now in the Index Memory chart, I decided to add the Segment Count chart to the Index page.
Original commit: elastic/x-pack-elasticsearch@4e3490ce9c
elastic/elasticsearchelastic/elasticsearch#19428 removes `node.mode` and `node.local` this PR
fixes xplugins to configure networking explicitly.
Original commit: elastic/x-pack-elasticsearch@ee8daa36dd
This change removes some of the complexity around licensing classes in
xpack. It removes the interfaces for registration and management so the
remaining LicensesService class is the thing that components wanting to
interact with the license should use. It also removes complexity around
the Licensee interface, removing generics and the registration at
construction time, as well as making the licensees no longer have a
lifecycle. There is still more to be done with simplification of license
classes construction, but this is a step towards a simpler world.
Original commit: elastic/x-pack-elasticsearch@5307d67b5b
This makes use of the registerAsDeprecatedHandler method to automatically warn users when they're using deprecated functionality.
This will also automatically provide a Warning header for anyone using HTTP clients (though they have to be looking for it...).
- This also adds deprecated `/_licenses` variants of the endpoint. Users are consistently making that mistake, and it's easy enough to support this way. Can remove it if people disagree though.
License portion only
Original commit: elastic/x-pack-elasticsearch@431c871fcf
This makes use of the registerAsDeprecatedHandler method to automatically warn users when they're using deprecated functionality.
This will also automatically provide a Warning header for anyone using HTTP clients (though they have to be looking for it...).
- This also changes from PUT _start, _restart, _stop (Watcher endpoints) to POST _start, _restart, _stop
- The deprecated variant still honors PUT
- Nothing about the hijack endpoints was deprecated because they did not change from 2.x
Watcher portion only
Original commit: elastic/x-pack-elasticsearch@36f87a6526
When the monitoring tests are run in isolation, they succeed. However, when the whole suite of
REST tests is being run at the same time, the "Bulk indexing of monitoring data" intermittently
fails with a timeout. Therefore, a timeout of 60 seconds has been added.
Closeselastic/elasticsearch#2809
Original commit: elastic/x-pack-elasticsearch@d11dc7a2be
This particular test requires Elasticsearch to run on port 9400, which is not
guaranteed if the clients run their own tests, as it is a matter of configuration
in gradle.
Therefore these tests need to run in their project.
Original commit: elastic/x-pack-elasticsearch@da38407766
The callback replaces the ability to fully replace the http client instance. By doing that, one used to lose any default that the RestClient had set for the underlying http client. Given that you'd usually override one or two things only, like a couple of timeout values, the ssl factory or the default credentials providers, it is not uder friendly if by doing that users end up replacing the whole http client instance and lose any default set by us.
Original commit: elastic/x-pack-elasticsearch@03adca6f62
Currently each node monitors the cluster state for a license, and if it
does not find one, it sends a request to the master to generate a trial
license. However, the master node has this same logic. Since the master
node is the only thing that can change the cluster state, we know that
once some node becomes master, it will notice the lack of license,
generate a trial license, and send a cluster state update. The trigger
from every node telling the master to generate the trial license is not
needed.
This change removes the register_trial_license action that the non
master nodes used. It removes the need for the TransportService in the
LicensesService, which will help with deguicing.
Original commit: elastic/x-pack-elasticsearch@a71656847e
This is the first deguicing for xplugins to use the new
createComponents(). The removal was very straightforward. One thing to
note is HttpAuthFactory only has one implementation (basic auth), but I
kept the registry and such for now. Also, HttpRequest.Parser is only
used in 2 tests, not at all in main code, it should probably be removed.
Original commit: elastic/x-pack-elasticsearch@839e7e4900
Since the "Should throw 404 after license deletion" test deletes the license,
a subsequent test run fails for many tests in other components.
This patch adds a `teardown` section to install back a trial development license.
Closeselastic/elasticsearch#2782
Original commit: elastic/x-pack-elasticsearch@45ec5d4244
This change creates the CryptoService during xpack plugin construction,
and also collapses the only implementation of CryptoService into a
class instead of an interface.
Original commit: elastic/x-pack-elasticsearch@7f00a5d6ef
The delete test has been relying on the license already being installed
in the cluster from the other test. However, many runners run tests
in arbitrary order, so it has been failing.
This patch adds a "Trial Development" license as the first action
in the test.
Related: elastic/elasticsearch#2777Closeselastic/elasticsearch#2778
Original commit: elastic/x-pack-elasticsearch@cd198cf67a
The actions in the `do` sections of tests definitions have not been indented, which
confused the Ruby YAML parser.
Also, the calls to `license.*` have not been prefixed with the `xpack` API.
Related: elastic/elasticsearch#2777
Original commit: elastic/x-pack-elasticsearch@b2da7d2703
The check is failing, because even when the document is deleted before
the test, the version of the document in Elasticsearch sticks around
for a while.
I have changed the test to use the `is_true` operator to check for
the presence of the field only.
Original commit: elastic/x-pack-elasticsearch@308ffe34f0
The shorthand format has been failing in the Ruby runner. Not sure if
this is particular to the Ruby YAML library, but I think there's no
harm in making the syntax standard.
Original commit: elastic/x-pack-elasticsearch@d753cad64a
This changes the xpack.monitoring.agent.* settings so that they fall under xpack.monitoring.collection.*.
- This also changes Kibana's equivalent data_collection to collection.
Original commit: elastic/x-pack-elasticsearch@a69bc18055
Added missing IOException and simplified filtering out mock transports
for security integration tests.
Relates to elastic/elasticseachelastic/elasticsearch#19332
Original commit: elastic/x-pack-elasticsearch@8a5c015c3d
The SecretService implements a subset of the api provided by the
CryptoService. Currently the CryptoService is simply bound to null when
in the transport client or when security is disabled. This change
converts all users of the SecretService to the CryptoService, and any
tests using SecretService.Insecure to simply using null.
Original commit: elastic/x-pack-elasticsearch@8251c65ffe
The CryptoService currently has a lot of variations of methods that are
unused. It really only uses sign/unsign, encrypt/decrypt. This change
trims the api down to those needed methods.
Original commit: elastic/x-pack-elasticsearch@92e83efeb7
The Clock interface, which basically allows testing in watcher to "time
warp" is currently constructed using guice. This change constructs it
using a protected method on XPackPlugin which can be overriden in tests.
This allows removing the ClockModule. For now, the Clock still needs to
be bound in guice, but this at least removes one guice construction, and
shows how other things can be overriden for tests.
Original commit: elastic/x-pack-elasticsearch@7addaea086
This makes use of the registerAsDeprecatedHandler method to automatically warn users when they're using deprecated functionality.
This will also automatically provide a Warning header for anyone using HTTP clients (though they have to be looking for it...).
Security portion only
Original commit: elastic/x-pack-elasticsearch@ab1a50fe06
This adds the following stats to the usage stats
* Across all active watches
* Number of triggers per type
* Number of inputs per type
* Number of transforms per type
* Number of conditions per type
* Number of actions per type
* Total number of watches
* Total number of active watches
* Per action
* Execution time total
* Execution time mean
* Invocation count
* Total use of watch metadata
Internally this uses a `Counters` helper class, so that creating counters does not require
you to decide about the map structure.
Relates elastic/elasticsearch#2210
Original commit: elastic/x-pack-elasticsearch@e6f95ba290
This changes the way that Kibana (and future applications) send their monitoring stats to Elasticsearch.
Instead of sending their payloads with the System ID (e.g., "kibana") and System Version (e.g., "5.0.0-alpha4"), it now expects the System ID and System _API_ Version (e.g., "2"). This means a few things:
- Future releases are automatically compatible with previous releases as long as the API version doesn't change.
- Users don't have to update Kibana at the exact same time as their cluster (which technically means rolling updates were temporarily blockers of Kibana monitoring before).
- We can accept old API versions (if we need to make a breaking change) and automatically up-convert them to the latest API version. (We are in full control of how far back we choose to accept)
In general, this change implies that users should be updating their Monitoring cluster before their _monitored_ cluster(s) to get the best opportunity of monitoring backwards compatibility. That way if any API change does occur, then it can up-convert as needed. Then, any ES node should be updated, and only then should Kibana be updated. This is not required in any way, but it will give the smoothest experience.
Original commit: elastic/x-pack-elasticsearch@d3c24936e1
While testing another PR, I noticed that when Security is disabled, the _xpack/usage API throws a NullPointerException.
This checks for null before using it, and adds tests to verify behavior.
Original commit: elastic/x-pack-elasticsearch@6474af6569
When a painless exception is raised in the script condition, it was not bubbled up due to
catching exceptions on during execution. This removes the different catching of exceptions
and allows the watch record construct to contain an exception that is also serialized correctly
so that it can be stored in the watch history but also returned in the execute watch API.
This also updates the watch history template, so that exceptions are not indexed, but logged.
Relates elastic/elasticsearch#2587
Original commit: elastic/x-pack-elasticsearch@4dffb672bf
The top-level class Throwable represents all errors and exceptions in
Java. This hierarchy is divided into Error and Exception, the former
being serious problems that applications should not try to catch and the
latter representing exceptional conditions that an application might
want to catch and handle. This commit renames
org.elasticsearch.cli.UserError to org.elasticsearch.UserException to
make its name consistent with where it falls in this hierarchy.
Relates elastic/elasticsearch#2701
Original commit: elastic/x-pack-elasticsearch@589e159ec0
Today throughout the codebase, catch throwable is used with reckless
abandon. This is dangerous because the throwable could be a fatal
virtual machine error resulting from an internal error in the JVM, or an
out of memory error or a stack overflow error that leaves the virtual
machine in an unstable and unpredictable state. This commit removes
catch throwable from the codebase and removes the temptation to use it
by modifying listener APIs to receive instances of Exception instead of
the top-level Throwable.
Relates elastic/elasticsearch#2694
Original commit: elastic/x-pack-elasticsearch@7ecdd7d978
This looks like it predates settings validation in core, and only had a
single use inside the watcher ExecutionService. This change moves the
settings inside ExecutionService to be validated settings, and removes
the watcher specific validation.
Original commit: elastic/x-pack-elasticsearch@82843ce56c
When a license exception is raised, we returned 401 as HTTP error code
in there. However this seems to have triggered some browsers to actually
ask for login credentials, which wont have any impact here.
Closeselastic/elasticsearch#1863
Original commit: elastic/x-pack-elasticsearch@cc63abdac8
We are going to parse the body anyways whenever it's in json format as it is going to be stashed. It is not useful to lazily parse it anymore. Also this allows us to not rely on automatic detection of the xcontent type based on the content of the response, but rather read the content type from the response headers.
Original commit: elastic/x-pack-elasticsearch@11be4684ae
The internal representation of the object that JsonPath gives access to is a map. That is independent of the initial input format, which is json but could also be yaml etc.
This commit renames JsonPath to ObjectPath and adds a static method to create an ObjectPath from an XContent
Original commit: elastic/x-pack-elasticsearch@bc84c68161
Currently, the REST tests for security (and possibly others) don't clean up the environment
after they have run, eg. they don't delete the users and roles they create. This leads to
test failures, because in a subsequent run, a user or role already exists, so eg. a test
like `match: { role: { created: true } }` fails.
This patch adds a `teardown` section to the test, with `do` actions which are to be
executed _after_ the test runs.
This patch assumes that REST tests runners for all languages support the `teardown` directive
in a xUnimt nomenclature -- similarly to the `setup` directive, which they already support.
Original commit: elastic/x-pack-elasticsearch@70d0ff4ee9
This is the xplugins side of moving the client dependency for rest
handlers to the handleRequest method
Original commit: elastic/x-pack-elasticsearch@ce66e35e7b
This commit modifies the construction of the poll interval setting in
the native realm tests in response to upstream change
elastic/elasticsearchelastic/elasticsearch#2f638b5a23597967a98b1ced1deac91d64af5a44.
Original commit: elastic/x-pack-elasticsearch@c6f60f51f4
This adds the `bin/shield/migrate` tool that allows migrating users and
roles from the files to the native (API-based) store.
It looks like this:
```
λ bin/shield/migrate native -U http://localhost:9200 -u test_user -p changeme -n lee,foo -r role1,role2,role3,role4,foo
starting migration of users and roles...
importing users from [/home/hinmanm/scratch/elasticsearch-2.4.0-SNAPSHOT/config/shield/users]...
found existing users: [test_user, joe3, joe2]
migrating user [lee]
{"user":{"created":true}}
no user [foo] found, skipping
importing roles from [/home/hinmanm/scratch/elasticsearch-2.4.0-SNAPSHOT/config/shield/roles.yml]...
found existing roles: [marvel_user, role_query_fields, admin_role, role3, admin, remote_marvel_agent, power_user, role_new_format_name_array, role_run_as, logstash, role_fields, role_run_as1, role_new_format, kibana4_server, user, transport_client, role1.ab, role_query]
migrating role [role1]
{"role":{"created":true}}
migrating role [role2]
{"role":{"created":true}}
role [role3] already exists, skipping
migrating role [role4]
failed to migrate role [role4] with body: {"indices":[{"names":["idx2"]},{"names":["idx2"]},{"names":["idx1"]}]}
java.io.IOException: {"error":{"root_cause":[{"type":"parse_exception","reason":"failed to parse indices privileges for role [role4]. missing required [privileges] field"}],"type":"parse_exception","reason":"failed to parse indices privileges for role [role4]. missing required [privileges] field"},"status":400}
at org.elasticsearch.shield.authc.esusers.tool.ESNativeRealmMigrateTool$MigrateUserOrRoles.postURL(ESNativeRealmMigrateTool.java:206)
at org.elasticsearch.shield.authc.esusers.tool.ESNativeRealmMigrateTool$MigrateUserOrRoles.importRoles(ESNativeRealmMigrateTool.java:389)
at org.elasticsearch.shield.authc.esusers.tool.ESNativeRealmMigrateTool$MigrateUserOrRoles.execute(ESNativeRealmMigrateTool.java:171)
at org.elasticsearch.common.cli.CliTool.execute(CliTool.java:153)
at org.elasticsearch.shield.authc.esusers.tool.ESNativeRealmMigrateTool.main(ESNativeRealmMigrateTool.java:91)
Caused by: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:9200/_shield/role/role4
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at org.elasticsearch.shield.authc.esusers.tool.ESNativeRealmMigrateTool$MigrateUserOrRoles.postURL(ESNativeRealmMigrateTool.java:192)
... 4 more
no role [foo] found, skipping
users and roles imported.
```
Original commit: elastic/x-pack-elasticsearch@3ce47c0ffd
If an attachment is configured of disposition type INLINE, and is referred to
in HTML body parts, then some email clients can display images inside of an HTML
email and refer to those attachments.
Watcher already had support for inlined attachments, however this could not be configured
from a watch, but just via the Java API. Also it was not tested.
This commit changes the attachment to decide on creation if it should be inline or a regular
attachment and adds a test.
Relates elastic/elasticsearch#2381
Relates elastic/elasticsearch#2464Closeselastic/elasticsearch#2557
Original commit: elastic/x-pack-elasticsearch@84935ffb18
When parsing the privileges, we now no longer throw an exception if
there haven't been any names parsed out. This is not an issue though,
because we validate that the `names` array is not empty when we parse
it, and that it's not `null` before returning from the function.
Adds a rest test that sends things out of order to test this still
works.
Resolveselastic/elasticsearch#2606
Original commit: elastic/x-pack-elasticsearch@62a38bea8f
Add new REST API endpoint to acknoweldging actions. The old endpoints
have not been removed as part of this PR, but can be in the next major
version.
Update the documentation to remove the parameter based example, and
mention that the old endpoints are going to be removed in the future.
Closeselastic/elasticsearch#2517
Original commit: elastic/x-pack-elasticsearch@e2558e9e1f
The search type is overridden from default to query_and_fetch by the search action if the number of shards to request is equal to 1
Original commit: elastic/x-pack-elasticsearch@a9552c697c
Template support is going to be removed from the Search API to its own Search Template API in the lang-mustache module (see elastic/elasticsearch#17906, elastic/elasticsearch#18765). This commit changes Watcher's SearchInput and SearchTransform classes so that it now uses a WatcherSearchTemplateRequest that contains both the search request and the template. Search request and template are rendered using WatcherSearchTemplateRequestService before being executed.
Original commit: elastic/x-pack-elasticsearch@bfa16ab80f
We were ignoring the response code which is always 401 because the license is not good to start watcher. Plus all tests run fine without these methods.
Original commit: elastic/x-pack-elasticsearch@f93e1c2777
The watch index action was using the _timestamp field by default.
This functionality now needs to be configured explicitely for a special
field that is part of that document which is going to be indexed.
Relates elastic/elasticsearchelastic/elasticsearch#18980
Original commit: elastic/x-pack-elasticsearch@dfa4cf2296
This class should have never existed, I fixed all places where we messed aroudn with this
and resolved dependencies or let guice deal with it.
Original commit: elastic/x-pack-elasticsearch@6a42c4153d
This commit handles the use of `*` as a field in a role as effectively disabling field level
security. We do this to take advantage of caches that we disable when field level security
is active.
See elastic/elasticsearch#2407
Original commit: elastic/x-pack-elasticsearch@d96e18d57c
This change makes the default role check the authenticating realm when authorizing
a request for the current user (or run as user) where the user is trying to change their
own password. We need to do this, otherwise we open up the potential of a user in one
realm changing the password of a user in another realm.
As part of this work, the authentication service has been refactored and simplified. A
new object, Authentication, is now returned when authenticating. Currently, this object
contains the user, authenticating realm information, and if it is a run as request the
information of the realm that looked up the user.
Closeselastic/elasticsearch#2089
Original commit: elastic/x-pack-elasticsearch@3fd9c37a16
The WWW-Authenticate header can optionally specify the charset that the server uses after
decoding credentials. If this is not specified, most clients will limit the available characters to
ISO-8859-1, which causes issues for certain characters.
See RFC 7617
Closeselastic/elasticsearch#2290
Original commit: elastic/x-pack-elasticsearch@44411eebe7
Ryan Ernst
Tanguy Leroux
Chris Earle
Jason Tedor
jaymode
Martijn van Groningen
Nik Everett
Lee Hinman
Adrien Grand
Britta Weber
Britta Weber
Boaz Leskes
Alexander Reelsen
Daniel Mitterdorfer
Ali Beyad
Daniel Mitterdorfer
Areek Zillur
javanna
Jim Ferenczi
Yannick Welsch
Simon Willnauer
Karel Minarik
Simon Willnauer
Robert Muir
Robert Muir
Colin Goodheart-Smithe
Jay Modi
Luca Cavanna
Tim Sullivan
Timothy Sullivan
Jonathan Budzenski