Commit Graph

45646 Commits

Author SHA1 Message Date
Alexander Reelsen 6996739a09 Update community client and integration docs (#41513)
Added integrations for a couple of frameworks.

Removed community clients where the last commit was more than three
years ago. Also added the official go client link and removed the
official groovy client, as it is outdated.
2019-04-26 08:57:14 +02:00
Tim Vernum 13fa72cae3
Fix broken test on FIPS for specific seed (#41230)
Under random seed 4304ED44CB755610 the generated byte pattern causes
BC-FIPS to throw

    java.io.IOException: DER length more than 4 bytes: 101

Rather than simply returning an empty list (as it does for most random
values).

Backport of: #40939
2019-04-26 15:43:48 +10:00
Tim Brooks 1f8ff052a1
Revert "Remove dedicated SSL network write buffer (#41283)"
This reverts commit f65a86c258.
2019-04-25 18:39:25 -06:00
Nhat Nguyen c4cb0507b4 Mute testDriverConfigurationWithSSLInURL
Tracked at #41557
2019-04-25 17:52:57 -04:00
Costin Leau b288b88ba0 SQL: Use field caps inside DESCRIBE TABLE as well (#41377)
Thanks to #34071, there is enough information in field caps to infer
the table structure and thus use the same API consistently across the
IndexResolver.

(cherry picked from commit f99946943a3350206b6bca774b2f060f41a787b3)
2019-04-25 23:41:17 +03:00
Tim Brooks f65a86c258
Remove dedicated SSL network write buffer (#41283)
This is related to #27260. Currently for the SSLDriver we allocate a
dedicated network write buffer and encrypt the data into that buffer one
buffer at a time. This requires constantly switching between encrypting
and flushing. This commit adds a dedicated outbound buffer for SSL
operations that will internally allocate new packet sized buffers as
they are need (for writing encrypted data). This allows us to totally
encrypt an operation before writing it to the network. Eventually it can
be hooked up to buffer recycling.
2019-04-25 14:30:54 -06:00
James Rodewig 08c5d3b912 [DOCS] Explicitly set section IDs for Asciidoctor migration (#41547)
* [DOCS] Explicitly set section ID for faster phrase queries

* [DOCS] Explicitly set section ID for faster prefix queries
2019-04-25 15:07:52 -04:00
Christoph Büscher 52495843cc [Docs] Fix common word repetitions (#39703) 2019-04-25 20:47:47 +02:00
Tim Brooks 6d7110edf5
SSLDriver can transition to CLOSED in handshake (#41458)
TLS 1.3 changes to the SSLEngine introduced a scenario where a UNWRAP
call during a handshake can consume a close notify alerty without
throwing an exception. This means that we continue down a codepath where
we assert that we are still in handshaking mode. Transitioning to closed
from handshaking is a valid scenario. This commit removes this
assertion.
2019-04-25 12:02:17 -06:00
James Rodewig 2e255a6446 [DOCS] Note TESTRESPONSE can't be used immediately after TESTSETUP (#41542) 2019-04-25 13:59:49 -04:00
RomainGeffraye a02f5544fe Update configuring-ldap-realm.asciidoc (#40427) 2019-04-25 13:29:46 -04:00
Jash Gala 20bdd42a2d Fixed very small typo in date (#41398) 2019-04-25 13:05:35 -04:00
Armin Braun 23b3741618
Remove Exists Check from S3 Repository Deletes (#40931) (#41534)
* The check doesn't add much if anything practically, since the S3 repository is eventually consistent and we only log the non-existence of a blob anyway
  * We don't do the check on writes for this very reason and documented it as such
  * Removing the check saves one API call per single delete speeding up the deletion process and lowering costs
2019-04-25 18:25:03 +02:00
Benjamin Trent 08843ba62b
[ML] Adds progress reporting for transforms (#41278) (#41529)
* [ML] Adds progress reporting for transforms

* fixing after master merge

* Addressing PR comments

* removing unused imports

* Adjusting afterKey handling and percentage to be 100*

* Making sure it is a linked hashmap for serialization

* removing unused import

* addressing PR comments

* removing unused import

* simplifying code, only storing total docs and decrementing

* adjusting for rewrite

* removing initial progress gathering from executor
2019-04-25 11:23:12 -05:00
Jim Ferenczi 6184efaff6
Handle unmapped fields in _field_caps API (#34071) (#41426)
Today the `_field_caps` API returns the list of indices where a field
is present only if this field has different types within the requested indices.
However if the request is an index pattern (or an alias, or both...) there
is no way to infer the indices if the response contains only fields that have
the same type in all indices. This commit changes the response to always return
the list of indices in the response. It also adds a way to retrieve unmapped field
in a specific section per field called `unmapped`. This section is created for each field
that is present in some indices but not all if the parameter `include_unmapped` is set to
true in the request (defaults to false).
2019-04-25 18:13:48 +02:00
Armin Braun 40aef2b8aa
Introduce Delegating ActionListener Wrappers (#40129) (#41527)
* Introduce Delegating ActionListener Wrappers
* Dry up use cases of ActionListener that simply pass through the response or exception to another listener
2019-04-25 16:05:04 +02:00
Ignacio Vera d119abdf96
Improve accuracy for Geo Centroid Aggregation (#41514)
keeps the partial results as doubles and uses Kahan summation to help reduce floating point errors.
2019-04-25 15:25:48 +02:00
Armin Braun cd830b53e2
Name Snapshot Data Blobs by UUID (#40652) (#41523)
* Name Snapshot Data Blobs by UUID

* There is no functional reason why we need incremental naming for these files but
  * As explained in #38941 it is a possible source of corrupting the repository
  * It wastes API calls for the list operation
  * Is just needless complication
* Since we store the exact names of the data blobs in all the metadata anyway, we can make this change without any BwC considerations
  * Even on the worst case scenario of a downgrade the functionality would continue working since the incremental names wouldn't conflict with the uuids and the number parsing for finding the next incremental name suppresses the exception when encountring a non-numeric value after the double underscore prefix
2019-04-25 13:18:03 +02:00
David Kyle 31d5a5aa36 Revert "Clean up clusters between tests (#41187)"
This reverts commit 44a0c468cf.
2019-04-25 12:08:14 +01:00
Luca Cavanna 8a0e5f7b87
Deprecate support for first line empty in msearch API (#41442)
In order to support empty action metadata in the first msearch item,
we need to remove support for prepending msearch request body with an
empty line, which prevents us from parsing the empty line as action
metadata for the first search item.

Relates to #41011
2019-04-25 12:45:18 +02:00
Przemyslaw Gomulka 906f88029b
Remove the test which is testing java and joda api backport(#41493) #41518
The test is testing the java time API and fails in case it hits daylight saving time changes.
Java time has the right implementation and we don't need to test this.
more details on how the test was affected by the DST change on this comment
closes #39617
backport(#41493)
2019-04-25 12:21:01 +02:00
Albert Zaharovits fe5789ada1 Fix Has Privilege API check on restricted indices (#41226)
The Has Privileges API allows to tap into the authorization process, to validate
privileges without actually running the operations to be authorized. This commit
fixes a bug, in which the Has Privilege API returned spurious results when checking
for index privileges over restricted indices (currently .security, .security-6,
.security-7). The actual authorization process is not affected by the bug.
2019-04-25 12:03:27 +03:00
Ryan Ernst 7e3875d781 Upgrade hamcrest to 2.1 (#41464)
hamcrest has some improvements in newer versions, like FileMatchers
that make assertions regarding file exists cleaner. This commit upgrades
to the latest version of hamcrest so we can start using new and improved
matchers.
2019-04-24 23:40:03 -07:00
Alexander Reelsen 257682918f Remove stale integrations in docs (#40764)
Removed all integrations in documentation, who have not seen a 
commit for more than three years when the website 
returned an  error.
2019-04-25 05:37:20 +02:00
Armin Braun 7c819fd2aa
Fix BulkRejectionIT (#41446) (#41500)
* Due to #40866 one of the two parallel bulk requests can randomly be
rejected outright when the write queue is full already, we can catch
this situation and ignore it since we can still have the rejection for
the dynamic mapping udate for the other reuqest and it's somewhat rare
to run into this anyway
* Closes #41363
2019-04-24 20:46:21 +02:00
Nik Everett bb6ca25f70
Docs: Specify an ID in migration docs (#41501)
We link to these migraiton docs but we don't specify the id. This
isn't great practice in general and is preventing us from migrating to
Asciidoctor because it generates ids in a slightly different way.
2019-04-24 14:44:11 -04:00
Zachary Tong ec5dd0594f Disallow null/empty or duplicate composite sources (#41359)
Adds some validation to prevent duplicate source names from being
used in the composite agg.

Also refactored to use a ConstructingObjectParser and removed the
private ctor and setter for sources, making it mandatory.
2019-04-24 13:23:31 -04:00
Armin Braun 1db9166ea0
Fix Broken Index Shard Snapshot File Preventing Snapshot Creation (#41310) (#41473)
* The problem here is that if we run into a corrupted index-N file, instead of generating a new index-(N+1) file, we instead set the newest index generation to -1 and thus tried to create `index-0`
   * If `index-0` is corrupt, this prevents us from ever creating a new snapshot using the broken shard, because we are unable to create `index-0` since it already exists
   * Fixed by still using the index generation for naming the next index file, even if it was a broken index file
* Added test that makes sure restoring as well as snapshotting on top of the broken shard index file work as expected
* closes #41304
2019-04-24 18:39:17 +02:00
James Rodewig 37ebefb553 [DOCS] Standardize docs for `url` setting (#41117) 2019-04-24 12:21:02 -04:00
Hendrik Muhs 13d720f4a7 [ML-DataFrame] Replace Streamable w/ Writeable (#41477)
replace usages of Streamable with Writeable

Relates to #36176
2019-04-24 17:17:34 +02:00
Benjamin Trent 07d36fdb23
[ML] refactoring the ML plugin to use the common auditor code (#41419) (#41485) 2019-04-24 09:56:59 -05:00
Armin Braun 381b8e2ece
Fix BulkProcessor Retry ITs (#41338) (#41472)
* The test fails for the retry backoff enabled case because the retry handler in the bulk processor hasn't been adjusted to account for #40866 which now might lead to an outright rejection of the request instead of its items individually
   * Fixed by adding retry functionality to the top level request as well
* Also fixed the duplicate test for the HLRC that wasn't handling the non-backoff case yet the same way the non-client IT did
* closes #41324
2019-04-24 13:46:32 +02:00
Armin Braun 2e4ac178e2
Fix Repository Base Path Matching in Azure ITs (#41457) (#41469)
* Added quotes so that "regexy" base paths like `7.0` that we use on CI
don't break matching
* closes #41405
2019-04-24 12:24:05 +02:00
Alpar Torok 58dd88329f Dix the packer cache script 2019-04-24 11:35:27 +03:00
Jack Conradson a74ba7d5ba Task to generate Painless API's per context (#41233)
This adds a gradle task called generateContextDoc in the Painless module. The 
task will start a cluster, issue commands against the context rest api for 
Painless, and generate documentation for each API per context. Each context 
has a first page of classes sorted by package first and class name second, 
along with a page per package with each classes' constructors, methods, and 
fields. A link is generated for each constructor, method, and field to a JavaDoc 
page when possible.
2019-04-23 11:28:27 -07:00
Nik Everett d1a3ae2268 Docs: Drop last inline callouts
Drops some inline callouts that snuck into 7.x. We're doings this in
preparation for switching the elasticsearch reference to asciidoctor
which doesn't support them.
2019-04-23 14:20:22 -04:00
Guilherme Ferreira 48a17d5768 [Docs] Correct default stop list constant (#41342) 2019-04-23 19:13:51 +02:00
James Rodewig 122e727d28 [DOCS] Fix broken link to elasticsearch-php security page 2019-04-23 12:43:40 -04:00
Jason Tedor 65af47eb31
Introduce aliases version (#41397)
This commit introduces aliases versions to index metadata. This will be
useful in CCR when we replicate aliases.
2019-04-23 12:19:11 -04:00
Nik Everett 7c2e151bf2 Docs: Rework close ml section (#41435)
Reworks the close ml section to work properly in Asciidoctor. It
renders a little funny in AsciiDoc but AsciiDoc is on its way out
anyway....
2019-04-23 11:05:08 -04:00
David Roberts 7e2aec022d [TEST] Mute BulkRejectionIT.testBulkRejectionAfterDynamicMappingUpdate
Due to https://github.com/elastic/elasticsearch/issues/41363
2019-04-23 15:58:38 +01:00
David Turner 411994b489 Mention the cost of tracking live docs in scrolls (#41375)
Relates #41337, in which a heap dump shows hundreds of MBs allocated on the
heap for tracking the live docs for each scroll.
2019-04-23 15:37:41 +01:00
Benjamin Trent e2f8ffdde8
[ML][Data Frame] Moving destination creation to _start (#41416) (#41433)
* [ML][Data Frame] Moving destination creation to _start

* slight refactor of DataFrameAuditor constructor
2019-04-23 09:32:57 -05:00
Martijn Laarman 85b9dc18a7 fix #35262 define deprecations of API's as a whole and urls (#39063)
* fix #35262 define deprecations of API's as a whole and urls

* document hot threads deprecated paths

* deprecate scroll_id as part of the URL, documented only as part of the body which is a safer behaviour as well

* use version numbers up to patch version

* rest spec parser picks up deprecated paths as paths too

(cherry picked from commit 7e06023e7603b7584bfd9ee4e8a1ccd82c208ce7)
2019-04-23 14:28:36 +02:00
David Roberts d8a2970fa4 [TEST] Mute RemoteClusterServiceTests.testCollectNodes
Due to https://github.com/elastic/elasticsearch/issues/41067
2019-04-23 15:13:01 +01:00
Marios Trivyzas e991175776
SQL: Implement IIF(<cond>, <result1>, <result2>) (#41420)
Implement a more trivial case of the CASE expression which is
expressed as a traditional function with 2 or 3 arguments. e.g.:

IIF(a = 1, 'one', 'many')
IIF(a > 0, 'positive')
Closes: #40917

(cherry picked from commit add02f4f553ad472026dcc1eaa84245a0558a4b0)
2019-04-23 16:31:25 +03:00
Hendrik Muhs 31c609db91 [DOCS] change source and dest to be nested and add a query to the example (#41293)
update source and dest to reflect the changes done in #40396 and add a query to the example.
2019-04-23 13:39:43 +02:00
Jason Tedor 4a288af85f
Avoid concurrent modification in mock log appender (#41424)
It can be the case that while we are setting up expectations that also a
log message is appended. For example, if we are setting up these
expectations after a cluster has formed and messages start being sent
around the cluster. In this case, we would hit a concurrent modification
exception while we are mutating the expectations, and also while the
expectations are being iterated over as a message is appended. This
commit avoids this by using a copy-on-write array list which is safe for
concurrent modification and iteration. Note that another possible
approach here is to use synchronized, but that seems unnecessary since
we don't appear to rely on messages that are sent while we are setting
up expectations. Rather, we are setting up some expectations and some
situation that we think will cause those expectations to be met. Using
copy-on-write array list here is nice since we avoid bottlenecking these
tests on synchronizing these methods.
2019-04-22 21:47:16 -04:00
Melori Arellano e6c24e3d2d [DOCS] Add missing setting skip_unavailable to example
The example to delete a remote cluster is missing the `skip_unavailable` setting which results in an error:
```
        "type": "illegal_argument_exception",
        "reason": "missing required setting [cluster.remote.tiny-test.seeds] for setting [cluster.remote.tiny-test.skip_unavailable]"
```
2019-04-22 16:53:18 -04:00
Ryan Ernst cc867e8a5a
Remove /var/run/elasticsearch from packages (#41102)
The pid dir for both systemd and init.d is already managed by those
respective systems (tmpfiles.d and the init script, respectively). Since
the /var/run dir is often mounted as tmpfs, it does not make sense to
have the elasticsearch pid dir added by the package installation. This
commit removes that empty dir from deb and rpm.
2019-04-22 09:57:51 -07:00