I'm really really sad to be removing the cli-fixture but I've had
trouble with it leaking recently it is pretty slow. Beyond that, we'd
prefer that our test fixture only fixture things that are external
depndencies.
So, yeah, I'm removing it. So we get faster tests and no chance of
leaking processes. We lose some "realness" in the tests. Instead of
interacting with the CLI like a real user we embed it in the test
process. That means we don't test the forking, we don't test the
executable jar, and we don't test the jLine console detection stuff. On
the other hand we were kind of forcing the jLine console detection stuff
in a funky way with the fixture anyway. And we test the executable jar
in the packaging tests. And that'll have to do.
I haven't renamed `RemoteCli` because it'd bloat this commit with
mechanical changes that'd make it hard to review. I'll rename it in a
followup commit.
This also updates jLine so we can disable blinking to matching
parentheses during testing. I have no clue why, but this wasn't
happening when we used the fixture. The trouble with the blinking is
that it is based on *time* so it slows things down. Worse, it works
inconsistently! Sometimes it spits out sensible ascii codes and
sometimes it, well, spits out weird garbage. When you use it in person
it works fine though. So we keep it on when not testing.
Cleans up some redundancy in when testing CLI errors. Less copy and
paste good.
I was tempted to disable the xterm emulation entirely while working on
this because upgrading jLine changed a few things and it was a real pain
to update. But If we turned that off then we'd have *nothing* testing
the colors and such. That'd be a shame because we use color in the
output to commicate stuff. I like it so I don't want to break it.
While I was there, I replaces the cli connector's `PrintWriter` with a
`BufferedWriter`. The `PrintWriter` was kind of a trap because `println`
would fail to work properly on windows because we force the terminal
into xterm mode and it doesn't know what to do with windows line
endings. Windows.....
Additionally I fixed a race condition between disabling echo when
reading passwords and fast writers. We were disabling the echo shortly
after sending the prompt. A fast enough writer could send us text before
the echo disable kicked in. Now I delegate to `LineReader#readLine`
with a special echo mask that disables echo. This is both easier to test
and doesn't seem to have the race condition. This race condition was
failing the tests because they are so much faster now. Yay!
Original commit: elastic/x-pack-elasticsearch@d0ec027396
Add basic support for catalog parameters in SYS COLUMN
Pass an empty string instead of a null inside the prepared statement
Don't use pattern for catalog in getColumns
Original commit: elastic/x-pack-elasticsearch@17e9e851a0
* foo.bar can mean either: no table field foo.bar or table foo, field bar
The resolution rule has been extended to include the latter case as
fallback.
* Always check field ambiguity
* Since field with dots can create confusion (when not qualified), the
analyzer always now always checks for both qualified and not-qualified
fields and throws an ambiguity message with the potential candidates.
This forces the use of qualifiers or quotes to better indicate the
desired field.
* Add example of aliasing the table to remove ambiguity
Original commit: elastic/x-pack-elasticsearch@8b70b9c4ee
Incorrect latch caused this test to run slowly (until the await
finished), and could probably cause failure due to incorrect ordering
Original commit: elastic/x-pack-elasticsearch@ebeb8655da
The latches were not placed correctly, allowing the aborts
to be set before we checked the state for Indexing the first time.
This was due to using the DelayingIndexer's built in latch, which
isn't placed quite where we needed it.
Original commit: elastic/x-pack-elasticsearch@590cfa07b0
* Pass InputStream when creating XContent parser
Rather than passing the raw `BytesReference` in when creating the xcontent
parser, this passes the StreamInput (which is an InputStream), this allows us to
decouple XContent from BytesReference.
This is the x-pack side of https://github.com/elastic/elasticsearch/pull/28754
* Use the streamInput variant, not sourceAsString
Original commit: elastic/x-pack-elasticsearch@dd5d8b1654
This adds a new Rollup module to XPack, which allows users to configure periodic "rollup jobs" to pre-aggregate data. That data is then available later for search through a special RollupSearch API, which mimics the DSL and functionality of regular search.
Rollups are used to drastically reduce the on-disk footprint of metric-based data (e.g. timestamped document with numeric and keyword fields). It can also be used to speed up aggregations over large datasets, since the rolled data will be considerably smaller and fewer documents to search.
The PR adds seven new endpoints to interact with Rollups; create/get/delete job, start/stop job, a capabilities API similar to field-caps, and a Rollup-enabled search.
Original commit: elastic/x-pack-elasticsearch@dcde91aacf
The watcher thread pool is scaled by the number of CPUs and has by
default up to 5x the number of cores. This is needed because we assumme
I/O based waiting workloads for watcher. However if the node is not a
data node, there will not be any execution of watches with the exception
of a user calling the execute watch API on that node.
This means, we can get away with just one thread, so that there is no
need for the JVM to manage more threads on master/client or ingest only
nodes.
Original commit: elastic/x-pack-elasticsearch@b5899401d3
This commit updates the IndexAuditUpgradeIT test after backporting. The
backport fixes a bug that caused the node name field to be null, which
affected the expected number of nodes in the tests.
Original commit: elastic/x-pack-elasticsearch@9df99d8800
This adds a new setting, `xpack.monitoring.collection.enabled`, and
disables it by default (`false`).
Original commit: elastic/x-pack-elasticsearch@4b3a5a1161
Although not frequently used in production, we make heavy use of the
FileRolesStore within integration tests. This change adds a little bit
more logging at INFO level when the roles.yml file is (re)loaded.
Original commit: elastic/x-pack-elasticsearch@bbacd46e28
In order to check for the REST tests if triggering of watches with
security enabled works as expected, we have to add a watch and wait for
its background execution. In the REST tests the only wait is to wait for
this with a timeout. If the timeout is reached but the watch has not
been executed yet, the test will fail.
This commit replaces the YAML with a java based REST test, so that
helper methods like assertBusy() can be used and waiting for a watch to
be executed now works as expected.
relates elastic/x-pack-elasticsearch#3753
Original commit: elastic/x-pack-elasticsearch@fc39636ef7
The current toXContent serialization of a failed hipchat message writes
the same field called status twice and thus cannot be stored in the
watch history.
This commit ensures the field gets only written once.
relates elastic/x-pack-elasticsearch#3919
Original commit: elastic/x-pack-elasticsearch@fb499e8055
This change makes the IndexAuditTrail update mappings for the current
index even when indexing locally. This is needed in a couple of
scenarios; the first being upgrades from 5x to 6x where the document
type was changed and the second is when upgrading minor versions and
a new field has been introduced. A new field was introduced in 6.2.
Additionally, writing tests for the rolling upgrade revealed a bug
where the actual node name was never being set as it was using the name
key instead of the node.name key.
relates elastic/x-pack-elasticsearch#3901
relates elastic/x-pack-elasticsearch#3718
Original commit: elastic/x-pack-elasticsearch@7374a62ff1
This commit removes some outdated logic in the SecurityTestUtils class
for writing files and creating directories. A long, long time ago there
was a global cluster for tests that was used across suites and because
of this there were calls to delete files if they already existed. The
global cluster has been removed, so we no longer need the code that
deletes the content of a directory if it already exists.
Additionally, the file writing used in SecurityTestUtils did not use
atomic moves when possible and this commit changes the code such that a
temp file is written and we try to atomically move it to the correct
path; if atomic moves are not supported a regular move is performed.
relates elastic/x-pack-elasticsearch#3912
Original commit: elastic/x-pack-elasticsearch@973fcfe2e1
... yet support updates. This commit introduces a few changes of how
watches are put.
The GET Watch API will never return credentials like basic auth
passwords, but a placeholder instead now. If the watcher is enabled to
encrypt sensitive settings, then the original encrypted value is
returned otherwise a "::es_redacted::" place holder.
There have been several Put Watch API changes.
The API now internally uses the Update API and versioning. This has
several implications. First if no version is supplied, we assume an
initial creation. This will work as before, however if a credential is
marked as redacted we will reject storing the watch, so users do not
accidentally store the wrong watch.
The watch xcontent parser now has an additional methods to tell the
caller if redacted passwords have been found. Based on this information
an error can be thrown.
If the user now wants to store a watch that contains a password marked
as redacted, this password will not be part of the toXContent
representation of the watch and in combinatination with update request
the existing password will be merged in. If the encrypted password is
supplied this one will be stored.
The serialization for GetWatchResponse/PutWatchRequest has changed.
The version checks for this will be put into the 6.x branch.
The Watcher UI now needs specify the version, when it wants to store a
watch. This also prevents last-write-wins scenarios and is the reason
why the put/get watch response now contains the internal version.
relates elastic/x-pack-elasticsearch#3089
Original commit: elastic/x-pack-elasticsearch@bb63be9f79
This test recently failed and it appears that the cause was that
roles.yml was deleted while the cluster was still running.
This turns on logging for the FileRolesStore so that changes to file
based roles can traced more easily.
Original commit: elastic/x-pack-elasticsearch@fbce5a8027
* Additional settings for SAML NameID policy
We should not be populating SPNameQualifier by default as it is
intended to be used to specify an alternate SP EntityID rather than
our own. Some IdPs (ADFS) fail when presented with this value.
This commit
- makes the SPNameQualifier a setting that defaults to blank
- adds a setting for "AllowCreate"
- documents the above
Original commit: elastic/x-pack-elasticsearch@093557e88f
Nik Everett
Chris Earle
David Roberts
David Kyle
Costin Leau
Zachary Tong
Ryan Ernst
Lee Hinman
Zachary Tong
Chris Earle
Lisa Cawley
Alexander Reelsen
Luca Cavanna
jaymode
Tim Brooks
javanna
Aaron Bull Schaefer
Tim Vernum
Jay Modi
Tim Vernum