This adds support for field level security to SQL by creating a new type of flow for securing requests that look like sql requests. `AuthorizationService` verifies that the user can execute the request but doesn't check the indices in the request because they are not yet ready. Instead, it adds a `BiFunction` to the context that can be used to check permissions for an index while servicing the request. This allows requests to cooperatively secure themselves. SQL does this by implementing filtering on top of its `Catalog` abstraction and backing that filtering with security's filters. This minimizes the touch points between security and SQL.
Stuff I'd like to do in followups:
What doesn't work at all still:
1. `SHOW TABLES` is still totally unsecured
2. `DESCRIBE TABLE` is still totally unsecured
3. JDBC's metadata APIs are still totally unsecured
What kind of works but not well:
1. The audit trail doesn't show the index being authorized for SQL.
Original commit: elastic/x-pack-elasticsearch@86f88ba2f5
This commit re-enables the OpenLDAP tests that were previously running against a one-off instance
in AWS but now run against a vagrant fixture. There were some IntegTests that would run against the
OpenLDAP instance randomly but with this change they no longer run against OpenLDAP. This is ok as
the functionality that is tested by these has coverage elsewhere.
relates elastic/x-pack-elasticsearch#1823
Original commit: elastic/x-pack-elasticsearch@ac9bc82297
record_count is no longer written to new results, but is still tolerated
for backwards compatibility. However, in the backwards compatibility case
the results index must already contain the required mapping. There's no
need to add this mapping to newly created results indices.
Original commit: elastic/x-pack-elasticsearch@e586f3ba96
Indices discovery actively ignores indices with more than one type.
However queries made such indices throw an exception (assuming the user
by mistake or not, selects such an index).
Original commit: elastic/x-pack-elasticsearch@16855c7b8f
Fix TemplateTransformMappingTests to work, even if date rolls over
during execution.
Reenable test in BootStrapTests, was forgotten.
Remove the SecurityF/MonitoringF/WatcherF classes, as there is a gradle
command to easily start elasticsearch with xpack
Remove HasherBenchmark, as it is not a test and relies on RandomContext
that is not available anymore (also I think a JMH benchmark would be
needed here).
Remove ManualPublicSmtpServersTester, was not usable anymore.
Remove OldWatcherIndicesBackwardsCompatibilityTests, now in dedicated
rolling upgrade tests.
Remove unused EvalCron class.
Original commit: elastic/x-pack-elasticsearch@100fa9e9b0
SQL relies on being able to fetch information about fields from
the cluster state and it'd be disasterous if that information
wasn't available. This should catch that.
Original commit: elastic/x-pack-elasticsearch@1a62747332
This commit enhances job/datafeed APIs that support acting
on multiple jobs/datafeeds at once so that they accept
expressions that may contain comma-separated lists or
wildcard patterns.
The APIs that are enhances are:
- get jobs API
- get job stats API
- close job API
- get datafeeds API
- get datafeed stats API
- stop datafeed API
relates elastic/x-pack-elasticsearch#1876
Original commit: elastic/x-pack-elasticsearch@45a1139d97
Running the sql rest action test inside the server caused a dependency
loop which was failing the build.
Original commit: elastic/x-pack-elasticsearch@43283671d8
This change makes the setup password tool build the default URL from the settings provided by the
environment. This will ease the amount of work a user would have to do in order to run the tool as
http vs https will be selected automatically and the port/host will as well.
Original commit: elastic/x-pack-elasticsearch@79affe4a79
This commit moves the reading of the token passphrase to the creation of the bootstrap check to
avoid issues with the secure settings keystore already being closed and thus causing issues during
startup.
Original commit: elastic/x-pack-elasticsearch@bba1cc832d
* Don’t set detector field names to empty strings
* Check summary count field and categorisation field names are not empty strings
* Check a detector has a by field when using multivariate by fields
Original commit: elastic/x-pack-elasticsearch@461be12f9f
If the native controller dies or is killed then requests to open jobs
now immediately return with an error that says what the problem is.
The error that is logged also now clearly records the problem.
Previously open job requests would time out if the native controller
was not running, and logged errors were not easy to understand without
in-depth knowledge of the ML code.
relates elastic/x-pack-elasticsearch#2140
Original commit: elastic/x-pack-elasticsearch@fc7f074d4a
This is a hack to remove a dependency cycle I added in elastic/x-pack-elasticsearch#2109. I think
it'd be cleaner to remove the cycle by making sql its own plugin that
doesn't depend on the rest of x-pack-elasticsearch but is still
included within x-pack-elasticsearch. But that is a broader change.
Original commit: elastic/x-pack-elasticsearch@47b7d69d80
This commit adds detection of support for AES 256 bit ciphers and enables their use when the JVM
supports them. For OpenJDK, this is often the case without any changes but for the Oracle JVM, the
unlimited policy file needs to be installed. In order to simplify the work a user would need to do
we can detect this support and automatically enable the AES 256 bit versions of the ciphers we
already enable.
Original commit: elastic/x-pack-elasticsearch@5f23b18a1e
Replacing integration tests with rest tests and unit tests, thus removing integration tests that require start of a new cluster. Removing unused testing methods
Original commit: elastic/x-pack-elasticsearch@265966d80c
The deprecated handlers should have been removed earlier, but are now
going to to away finally.
Also the watcher restart action has been removed, mainly because users
should not blindly restart, but always make sure, that watcher is
stopped correctly before restarting. This had been removed from the
transport action previously.
Original commit: elastic/x-pack-elasticsearch@78a5ec3c05
This flaky tests was using sleep, latches and a custom script plugin,
causing issues with stopping/starting tests.
This removes the integration tests and replaces it with a unit test.
Also removed a couple of unused ctor/method parameters as cleanup.
relates elastic/x-pack-elasticsearch#1639
Original commit: elastic/x-pack-elasticsearch@2a42faf2db
As all the triggering is now done on the shards, people can use
bulk, *-by-query and regular index/delete operations on the watcher
index and do not need to use the dedicated APIs.
We can now remove the long existing rest operation hijacking code.
Original commit: elastic/x-pack-elasticsearch@08f4f4c3de
Some standalone tests set the max script compilation limit. However,
this setting is now set in the main cluster setup in core so it is no
longer needed here. This commit removes these obviated settings in
standalone tests.
Original commit: elastic/x-pack-elasticsearch@089328c8d7
This commit fixes the handling of some exceptions when we attempt to create the security index and
alias. The issue here is provoked by a test that is currently muted with an AwaitsFix,
GroupMappingTests, which will be unmuted in another change.
Original commit: elastic/x-pack-elasticsearch@55f6b656cb
Nik Everett
Alexander Reelsen
Luca Cavanna
David Roberts
Igor Motov
Chris Earle
Lisa Cawley
Jay Modi
Costin Leau
Dimitris Athanasiou
David Kyle
Tim Brooks
Ryan Ernst
Jason Tedor