[[release-notes-7.0.0]] == {es} version 7.0.0 These release notes include all changes made in the alpha, beta, and RC releases of 7.0.0. Also see <>. [float] === Known issues * Applying deletes or updates on an index after it has been shrunk may corrupt the index. In order to prevent this issue, it is recommended to stop shrinking read-write indices. For read-only indices, it is recommended to force-merge indices after shrinking, which significantly reduces the likeliness of this corruption in the case that deletes/updates would be applied by mistake. This bug is fixed in {es} 7.7 and later versions. More details can be found on the https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue]. * Indices created in 6.x with <> and <> fields using formats that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. https://github.com/elastic/elasticsearch/pull/52555 This is fixed in {es} 7.7 and later versions. [[breaking-7.0.0]] [float] === Breaking changes Aggregations:: * Remove support for deprecated params._agg/_aggs for scripted metric aggregations {es-pull}32979[#32979] (issues: {es-issue}29328[#29328], {es-issue}31597[#31597]) * Percentile/Ranks should return null instead of NaN when empty {es-pull}30460[#30460] (issue: {es-issue}29066[#29066]) * Render sum as zero if count is zero for stats aggregation {es-pull}27193[#27193] (issue: {es-issue}26893[#26893]) Analysis:: * Remove `delimited_payload_filter` {es-pull}27705[#27705] (issues: {es-issue}26625[#26625], {es-issue}27704[#27704]) * Limit the number of tokens produced by _analyze {es-pull}27529[#27529] (issue: {es-issue}27038[#27038]) * Add limits for ngram and shingle settings {es-pull}27211[#27211] (issue: {es-issue}25887[#25887]) Audit:: * Logfile auditing settings remove after deprecation {es-pull}35205[#35205] * Remove index audit output type {es-pull}37707[#37707] (issues: {es-issue}29881[#29881], {es-issue}37301[#37301]) Authentication:: * Security: remove wrapping in put user response {es-pull}33512[#33512] (issue: {es-issue}32332[#32332]) * Remove bwc logic for token invalidation {es-pull}36893[#36893] (issue: {es-issue}36727[#36727]) Authorization:: * Remove aliases resolution limitations when security is enabled {es-pull}31952[#31952] (issue: {es-issue}31516[#31516]) * Remove implicit index monitor privilege {es-pull}37774[#37774] Circuit Breakers:: * Lower fielddata circuit breaker's default limit {es-pull}27162[#27162] (issue: {es-issue}27130[#27130]) CRUD:: * Version conflict exception message enhancement {es-pull}29432[#29432] (issue: {es-issue}21278[#21278]) * Using ObjectParser in UpdateRequest {es-pull}29293[#29293] (issue: {es-issue}28740[#28740]) * Remove support for internal versioning for concurrency control {es-pull}38254[#38254] (issue: {es-issue}1078[#1078]) Distributed:: * Remove undocumented action.master.force_local setting {es-pull}29351[#29351] * Remove tribe node support {es-pull}28443[#28443] * Forbid negative values for index.unassigned.node_left.delayed_timeout {es-pull}26828[#26828] * Remove cluster state size {es-pull}40061[#40061] (issues: {es-issue}39806[#39806], {es-issue}39827[#39827], {es-issue}39951[#39951], {es-issue}40016[#40016]) Features/Features:: * Remove Migration Upgrade and Assistance APIs {es-pull}40075[#40075] (issue: {es-issue}40014[#40014]) Features/Indices APIs:: * Indices Exists API should return 404 for empty wildcards {es-pull}34499[#34499] * Default to one shard {es-pull}30539[#30539] * Limit the number of nested documents {es-pull}27405[#27405] (issue: {es-issue}26962[#26962]) Features/Ingest:: * Add Configuration Except. Data to Metdata {es-pull}32322[#32322] (issue: {es-issue}27728[#27728]) * Add ECS schema for user-agent ingest processor (#37727) {es-pull}37984[#37984] (issues: {es-issue}37329[#37329], {es-issue}37727[#37727]) * Remove special handling for ingest plugins {es-pull}36967[#36967] (issues: {es-issue}36898[#36898], {es-issue}36956[#36956]) Features/Java Low Level REST Client:: * Drop support for the low-level REST client on JDK 7 {es-pull}38540[#38540] (issue: {es-issue}29607[#29607]) Features/Watcher:: * Remove Watcher Account "unsecure" settings {es-pull}36736[#36736] (issue: {es-issue}36403[#36403]) Features/Stats:: * Remove the suggest metric from stats APIs {es-pull}29635[#29635] (issue: {es-issue}29589[#29589]) * Align cat thread pool info to thread pool config {es-pull}29195[#29195] (issue: {es-issue}29123[#29123]) * Align thread pool info to thread pool configuration {es-pull}29123[#29123] (issue: {es-issue}29113[#29113]) Geo:: * Use geohash cell instead of just a corner in geo_bounding_box {es-pull}30698[#30698] (issue: {es-issue}25154[#25154]) Index APIs:: * Always enforce cluster-wide shard limit {es-pull}34892[#34892] (issues: {es-issue}20705[#20705], {es-issue}34021[#34021]) Infra/Circuit Breakers:: * Introduce durability of circuit breaking exception {es-pull}34460[#34460] (issue: {es-issue}31986[#31986]) * Circuit-break based on real memory usage {es-pull}31767[#31767] Infra/Core:: * Default node.name to the hostname {es-pull}33677[#33677] * Remove bulk fallback for write thread pool {es-pull}29609[#29609] * CCS: Drop http address from remote cluster info {es-pull}29568[#29568] (issue: {es-issue}29207[#29207]) * Remove the index thread pool {es-pull}29556[#29556] * Main response should not have status 503 when okay {es-pull}29045[#29045] (issue: {es-issue}8902[#8902]) * Automatically prepare indices for splitting {es-pull}27451[#27451] * Don't refresh on `_flush` `_force_merge` and `_upgrade` {es-pull}27000[#27000] (issue: {es-issue}26972[#26972]) Infra/Logging:: * Elasticsearch json logging {es-pull}36833[#36833] (issue: {es-issue}32850[#32850]) Infra/Packaging:: * Packaging: Remove windows bin files from the tar distribution {es-pull}30596[#30596] * Package ingest-user-agent as a module {es-pull}36956[#36956] * Package ingest-geoip as a module {es-pull}36898[#36898] Infra/REST API:: * Remove GET support for clear cache indices {es-pull}29525[#29525] * Clear Indices Cache API remove deprecated url params {es-pull}29068[#29068] Infra/Scripting:: * Remove support for deprecated StoredScript contexts {es-pull}31394[#31394] (issues: {es-issue}27612[#27612], {es-issue}28939[#28939]) * Remove getDate methods from ScriptDocValues {es-pull}30690[#30690] * Drop `ScriptDocValues#date` and `ScriptDocValues#dates` in 7.0.0 {es-pull}30690[#30690] (issue: {es-issue}23008[#23008]) Infra/Settings:: * Remove config prompting for secrets and text {es-pull}27216[#27216] Machine Learning:: * Remove types from datafeed {es-pull}36538[#36538] (issue: {es-issue}34265[#34265]) Mapping:: * Match phrase queries against non-indexed fields should throw an exception {es-pull}31060[#31060] * Remove legacy mapping code. {es-pull}29224[#29224] * Reject updates to the `_default_` mapping. {es-pull}29165[#29165] (issues: {es-issue}15613[#15613], {es-issue}28248[#28248]) * Remove the `update_all_types` option. {es-pull}28288[#28288] * Remove the `_default_` mapping. {es-pull}28248[#28248] * Reject the `index_options` parameter for numeric fields {es-pull}26668[#26668] (issue: {es-issue}21475[#21475]) * Update the default for include_type_name to false. {es-pull}37285[#37285] * Support 'include_type_name' in RestGetIndicesAction {es-pull}37149[#37149] Network:: * Remove http.enabled setting {es-pull}29601[#29601] (issue: {es-issue}12792[#12792]) * Remove HTTP max content length leniency {es-pull}29337[#29337] * Remove TLS 1.0 as a default SSL protocol {es-pull}37512[#37512] (issue: {es-issue}36021[#36021]) * Security: remove SSL settings fallback {es-pull}36846[#36846] (issue: {es-issue}29797[#29797]) Percolator:: * Remove deprecated percolator map_unmapped_fields_as_string setting {es-pull}28060[#28060] Ranking:: * Add minimal sanity checks to custom/scripted similarities. {es-pull}33564[#33564] (issue: {es-issue}33309[#33309]) * Scroll queries asking for rescore are considered invalid {es-pull}32918[#32918] (issue: {es-issue}31775[#31775]) * Forbid negative scores in function_score query {es-pull}35709[#35709] (issue: {es-issue}33309[#33309]) * Forbid negative field boosts in analyzed queries {es-pull}37930[#37930] (issue: {es-issue}33309[#33309]) Scripting:: * Delete deprecated getValues from ScriptDocValues {es-pull}36183[#36183] (issue: {es-issue}22919[#22919]) Search:: * Remove deprecated url parameters `_source_include` and `_source_exclude` {es-pull}35097[#35097] (issues: {es-issue}22792[#22792], {es-issue}33475[#33475]) * Disallow negative query boost {es-pull}34486[#34486] (issue: {es-issue}33309[#33309]) * Forbid negative `weight` in Function Score Query {es-pull}33390[#33390] (issue: {es-issue}31927[#31927]) * In the field capabilities API, remove support for providing fields in the request body. {es-pull}30185[#30185] * Remove deprecated options for query_string {es-pull}29203[#29203] (issue: {es-issue}25551[#25551]) * Fix Laplace scorer to multiply by alpha (and not add) {es-pull}27125[#27125] * Remove _primary and _replica shard preferences {es-pull}26791[#26791] (issue: {es-issue}26335[#26335]) * Limit the number of expanded fields it query_string and simple_query_string {es-pull}26541[#26541] (issue: {es-issue}25105[#25105]) * Make purely negative queries return scores of 0. {es-pull}26015[#26015] (issue: {es-issue}23449[#23449]) * Remove the deprecated _termvector endpoint. {es-pull}36131[#36131] (issues: {es-issue}36098[#36098], {es-issue}8484[#8484]) * Remove deprecated Graph endpoints {es-pull}35956[#35956] * Validate metadata on `_msearch` {es-pull}35938[#35938] (issue: {es-issue}35869[#35869]) * Make hits.total an object in the search response {es-pull}35849[#35849] (issue: {es-issue}33028[#33028]) * Remove the distinction between query and filter context in QueryBuilders {es-pull}35354[#35354] (issue: {es-issue}35293[#35293]) * Throw a parsing exception when boost is set in span_or query (#28390) {es-pull}34112[#34112] (issue: {es-issue}28390[#28390]) * Track total hits up to 10,000 by default {es-pull}37466[#37466] (issue: {es-issue}33028[#33028]) * Use mappings to format doc-value fields by default. {es-pull}30831[#30831] (issues: {es-issue}26948[#26948], {es-issue}29639[#29639]) Security:: * Remove heuristics that enable security on trial licenses {es-pull}38075[#38075] (issue: {es-issue}38009[#38009]) Snapshot/Restore:: * Include size of snapshot in snapshot metadata {es-pull}30890[#30890] (issue: {es-issue}18543[#18543]) * Remove azure deprecated settings {es-pull}26099[#26099] (issue: {es-issue}23405[#23405]) Store:: * Drop elasticsearch-translog for 7.0 {es-pull}33373[#33373] (issues: {es-issue}31389[#31389], {es-issue}32281[#32281]) * completely drop `index.shard.check_on_startup: fix` for 7.0 {es-pull}33194[#33194] Suggesters:: * Fix threshold frequency computation in Suggesters {es-pull}34312[#34312] (issue: {es-issue}34282[#34282]) * Make Geo Context Mapping Parsing More Strict {es-pull}32821[#32821] (issues: {es-issue}32202[#32202], {es-issue}32412[#32412]) * Remove the ability to index or query context suggestions without context {es-pull}31007[#31007] (issue: {es-issue}30712[#30712]) ZenDiscovery:: * Best-effort cluster formation if unconfigured {es-pull}36215[#36215] * Remove DiscoveryPlugin#getDiscoveryTypes {es-pull}38414[#38414] (issue: {es-issue}38410[#38410]) [[breaking-java-7.0.0]] [float] === Breaking Java changes Aggregations:: * Change GeoHashGrid.Bucket#getKey() to return String {es-pull}31748[#31748] (issue: {es-issue}30320[#30320]) Analysis:: * Remove deprecated AnalysisPlugin#requriesAnalysisSettings method {es-pull}32037[#32037] (issue: {es-issue}32025[#32025]) Features/Java High Level REST Client:: * Drop deprecated methods from Retry {es-pull}33925[#33925] * Cluster health to default to cluster level {es-pull}31268[#31268] (issue: {es-issue}29331[#29331]) * Remove deprecated API methods {es-pull}31200[#31200] (issue: {es-issue}31069[#31069]) Features/Java Low Level REST Client:: * Drop deprecated methods {es-pull}33223[#33223] (issues: {es-issue}29623[#29623], {es-issue}30315[#30315]) * Remove support for maxRetryTimeout from low-level REST client {es-pull}38085[#38085] (issues: {es-issue}25951[#25951], {es-issue}31834[#31834], {es-issue}33342[#33342]) Geo:: * Decouple geojson parse logic from ShapeBuilders {es-pull}27212[#27212] Infra/Core:: * Remove RequestBuilder from Action {es-pull}30966[#30966] * Handle scheduler exceptions {es-pull}38014[#38014] (issues: {es-issue}28667[#28667], {es-issue}36137[#36137], {es-issue}37708[#37708]) Infra/Transport API:: * Java api clean up: remove deprecated `isShardsAcked` {es-pull}28311[#28311] (issues: {es-issue}27784[#27784], {es-issue}27819[#27819]) ZenDiscovery:: * Make node field in JoinRequest private {es-pull}36405[#36405] [[deprecation-7.0.0]] [float] === Deprecations Aggregations:: * Deprecate dots in aggregation names {es-pull}31468[#31468] (issues: {es-issue}17600[#17600], {es-issue}19040[#19040]) Analysis:: * Replace parameter unicodeSetFilter with unicode_set_filter {es-pull}29215[#29215] (issue: {es-issue}22823[#22823]) * Replace delimited_payload_filter by delimited_payload {es-pull}26625[#26625] (issue: {es-issue}21978[#21978]) * Deprecate Standard Html Strip Analyzer in master {es-pull}26719[#26719] (issue: {es-issue}4704[#4704]) * Remove `nGram` and `edgeNGram` token filter names (#38911) {es-pull}39070[#39070] (issues: {es-issue}30209[#30209], {es-issue}38911[#38911]) Audit:: * Deprecate index audit output type {es-pull}37301[#37301] (issue: {es-issue}29881[#29881]) Core:: * Deprecate use of scientific notation in epoch time parsing {es-pull}36691[#36691] * Add backcompat for joda time formats {es-pull}36531[#36531] Cluster Coordination:: * Deprecate size in cluster state response {es-pull}39951[#39951] (issue: {es-issue}39806[#39806]) Features/Indices APIs:: * Default copy settings to true and deprecate on the REST layer {es-pull}30598[#30598] * Reject setting index.optimize_auto_generated_id after version 7.0.0 {es-pull}28895[#28895] (issue: {es-issue}27600[#27600]) Features/Ingest:: * Deprecate `_type` in simulate pipeline requests {es-pull}37949[#37949] (issue: {es-issue}37731[#37731]) Features/Java High Level REST Client:: * Deprecate HLRC security methods {es-pull}37883[#37883] (issues: {es-issue}36938[#36938], {es-issue}37540[#37540]) * Deprecate HLRC EmptyResponse used by security {es-pull}37540[#37540] (issue: {es-issue}36938[#36938]) Features/Watcher:: * Deprecate xpack.watcher.history.cleaner_service.enabled {es-pull}37782[#37782] (issue: {es-issue}32041[#32041]) * deprecate types for watcher {es-pull}37594[#37594] (issue: {es-issue}35190[#35190]) Graph:: * Deprecate types in `_graph/explore` calls. {es-pull}40466[#40466] Infra/Core:: * Deprecate negative epoch timestamps {es-pull}36793[#36793] * Deprecate use of scientific notation in epoch time parsing {es-pull}36691[#36691] Infra/Packaging:: * Deprecate fallback to java on PATH {es-pull}37990[#37990] Infra/Scripting:: * Add types deprecation to script contexts {es-pull}37554[#37554] * Deprecate _type from LeafDocLookup {es-pull}37491[#37491] * Remove deprecated params.ctx {es-pull}36848[#36848] (issue: {es-issue}34059[#34059]) Infra/Transport API:: * Deprecate the transport client in favour of the high-level REST client {es-pull}27085[#27085] Machine Learning:: * Deprecate X-Pack centric ML endpoints {es-pull}36315[#36315] (issue: {es-issue}35958[#35958]) * Adding ml_settings entry to HLRC and Docs for deprecation_info {es-pull}38118[#38118] * Datafeed deprecation checks {es-pull}38026[#38026] (issue: {es-issue}37932[#37932]) * Remove "8" prefixes from file structure finder timestamp formats {es-pull}38016[#38016] * Adjust structure finder for Joda to Java time migration {es-pull}37306[#37306] * Resolve 7.0.0 TODOs in ML code {es-pull}36842[#36842] (issue: {es-issue}29963[#29963]) Mapping:: * Deprecate type exists requests. {es-pull}34663[#34663] * Deprecate types in index API {es-pull}36575[#36575] (issues: {es-issue}35190[#35190], {es-issue}35790[#35790]) * Deprecate uses of _type as a field name in queries {es-pull}36503[#36503] (issue: {es-issue}35190[#35190]) * Deprecate types in update_by_query and delete_by_query {es-pull}36365[#36365] (issue: {es-issue}35190[#35190]) * For msearch templates, make sure to use the right name for deprecation logging. {es-pull}36344[#36344] * Deprecate types in termvector and mtermvector requests. {es-pull}36182[#36182] * Deprecate types in update requests. {es-pull}36181[#36181] * Deprecate types in document delete requests. {es-pull}36087[#36087] * Deprecate types in get, exists, and multi get. {es-pull}35930[#35930] * Deprecate types in search and multi search templates. {es-pull}35669[#35669] * Deprecate types in explain requests. {es-pull}35611[#35611] * Deprecate types in validate query requests. {es-pull}35575[#35575] * Deprecate types in count and msearch. {es-pull}35421[#35421] (issue: {es-issue}34041[#34041]) * Deprecate types in rollover index API {es-pull}38039[#38039] (issue: {es-issue}35190[#35190]) * Deprecate types in get field mapping API {es-pull}37667[#37667] (issue: {es-issue}35190[#35190]) * Deprecate types in the put mapping API. {es-pull}37280[#37280] (issues: {es-issue}29453[#29453], {es-issue}37285[#37285]) * Support include_type_name in the field mapping and index template APIs. {es-pull}37210[#37210] * Deprecate types in create index requests. {es-pull}37134[#37134] (issues: {es-issue}29453[#29453], {es-issue}37285[#37285]) * Deprecate use of the _type field in aggregations. {es-pull}37131[#37131] (issue: {es-issue}36802[#36802]) * Deprecate reference to _type in lookup queries {es-pull}37016[#37016] (issue: {es-issue}35190[#35190]) * Deprecate the document create endpoint. {es-pull}36863[#36863] * Deprecate types in index API {es-pull}36575[#36575] (issues: {es-issue}35190[#35190], {es-issue}35790[#35790]) * Deprecate types in update APIs {es-pull}36225[#36225] Migration:: * Deprecate X-Pack centric Migration endpoints {es-pull}35976[#35976] (issue: {es-issue}35958[#35958]) Monitoring:: * Deprecate /_xpack/monitoring/* in favor of /_monitoring/* {es-pull}36130[#36130] (issue: {es-issue}35958[#35958]) Rollup:: * Re-deprecate xpack rollup endpoints {es-pull}36451[#36451] (issue: {es-issue}36044[#36044]) * Deprecate X-Pack centric rollup endpoints {es-pull}35962[#35962] (issue: {es-issue}35958[#35958]) Scripting:: * Adds deprecation logging to ScriptDocValues#getValues. {es-pull}34279[#34279] (issue: {es-issue}22919[#22919]) * Conditionally use java time api in scripting {es-pull}31441[#31441] Search:: * Deprecate filtering on `_type`. {es-pull}29468[#29468] (issue: {es-issue}15613[#15613]) * Remove X-Pack centric graph endpoints {es-pull}36010[#36010] (issue: {es-issue}35958[#35958]) * Deprecate use of type in reindex request body {es-pull}36823[#36823] * Add typless endpoints for get_source and exist_source {es-pull}36426[#36426] Security:: * Deprecate X-Pack centric license endpoints {es-pull}35959[#35959] (issue: {es-issue}35958[#35958]) * Deprecate /_xpack/security/* in favor of /_security/* {es-pull}36293[#36293] (issue: {es-issue}35958[#35958]) SQL:: * Deprecate X-Pack SQL translate endpoint {es-pull}36030[#36030] * Deprecate X-Pack centric SQL endpoints {es-pull}35964[#35964] (issue: {es-issue}35958[#35958]) Watcher:: * Deprecate X-Pack centric watcher endpoints {es-pull}36218[#36218] (issue: {es-issue}35958[#35958]) [[feature-7.0.0]] [float] === New features Allocation:: * Node repurpose tool {es-pull}39403[#39403] (issues: {es-issue}37347[#37347], {es-issue}37748[#37748]) Analysis:: * Relax TermVectors API to work with textual fields other than TextFieldType {es-pull}31915[#31915] (issue: {es-issue}31902[#31902]) * Add support for inlined user dictionary in Nori {es-pull}36123[#36123] (issue: {es-issue}35842[#35842]) * Add a prebuilt ICU Analyzer {es-pull}34958[#34958] (issue: {es-issue}34285[#34285]) Authentication:: * Add support for API keys to access Elasticsearch {es-pull}38291[#38291] (issue: {es-issue}34383[#34383]) * OIDC realm authentication flows {es-pull}37787[#37787] * OIDC Realm JWT+JWS related functionality {es-pull}37272[#37272] (issues: {es-issue}35339[#35339], {es-issue}37009[#37009]) * OpenID Connect Realm base functionality {es-pull}37009[#37009] (issue: {es-issue}35339[#35339]) Authorization:: * Allow custom authorization with an authorization engine {es-pull}38358[#38358] (issues: {es-issue}32435[#32435], {es-issue}36245[#36245], {es-issue}37328[#37328], {es-issue}37495[#37495], {es-issue}37785[#37785], {es-issue}38137[#38137], {es-issue}38219[#38219]) * Wildcard IndicesPermissions don't cover .security {es-pull}36765[#36765] CCR:: * Generalize search.remote settings to cluster.remote {es-pull}33413[#33413] * Add ccr follow info api {es-pull}37408[#37408] (issue: {es-issue}37127[#37127]) Distributed:: * Log messages from allocation commands {es-pull}25955[#25955] (issues: {es-issue}22821[#22821], {es-issue}25325[#25325]) Features/ILM:: * Add unfollow action {es-pull}36970[#36970] (issue: {es-issue}34648[#34648]) Features/Ingest:: * Revert "Introduce a Hashing Processor (#31087)" {es-pull}32178[#32178] * Add ingest-attachment support for per document `indexed_chars` limit {es-pull}28977[#28977] (issue: {es-issue}28942[#28942]) Features/Java High Level REST Client:: * GraphClient for the high level REST client and associated tests {es-pull}32366[#32366] Features/Monitoring:: * Collect only display_name (for now) {es-pull}35265[#35265] (issue: {es-issue}8445[#8445]) Geo:: * Integrate Lucene's LatLonShape (BKD Backed GeoShapes) as default `geo_shape` indexing approach {es-pull}36751[#36751] (issue: {es-issue}35320[#35320]) * Integrate Lucene's LatLonShape (BKD Backed GeoShapes) as default `geo_shape` indexing approach {es-pull}35320[#35320] (issue: {es-issue}32039[#32039]) * geotile_grid implementation {es-pull}37842[#37842] (issue: {es-issue}30240[#30240]) * Fork Lucene's LatLonShape Classes to local lucene package {es-pull}36794[#36794] * Integrate Lucene's LatLonShape (BKD Backed GeoShapes) as default `geo_shape` indexing approach {es-pull}36751[#36751] (issue: {es-issue}35320[#35320]) * Integrate Lucene's LatLonShape (BKD Backed GeoShapes) as default `geo_shape` indexing approach {es-pull}35320[#35320] (issue: {es-issue}32039[#32039]) Infra/Core:: * Skip shard refreshes if shard is `search idle` {es-pull}27500[#27500] Infra/Logging:: * Logging: Unify log rotation for index/search slow log {es-pull}27298[#27298] Infra/Plugins:: * Reload secure settings for plugins {es-pull}31383[#31383] (issue: {es-issue}29135[#29135]) Infra/REST API:: * Add an `include_type_name` option. {es-pull}29453[#29453] (issue: {es-issue}15613[#15613]) Java High Level REST Client:: * Add rollup search {es-pull}36334[#36334] (issue: {es-issue}29827[#29827]) Java Low Level REST Client:: * Make warning behavior pluggable per request {es-pull}36345[#36345] * Add PreferHasAttributeNodeSelector {es-pull}36005[#36005] Machine Learning:: * Filter undefined job groups from update job calendar actions {es-pull}30757[#30757] * Add delayed datacheck to the datafeed job runner {es-pull}35387[#35387] (issue: {es-issue}35131[#35131]) * Adds set_upgrade_mode API endpoint {es-pull}37837[#37837] Mapping:: * Add a `feature_vector` field. {es-pull}31102[#31102] (issue: {es-issue}27552[#27552]) * Expose Lucene's FeatureField. {es-pull}30618[#30618] * Make typeless APIs usable with indices whose type name is different from `_doc` {es-pull}35790[#35790] (issue: {es-issue}35190[#35190]) * Give precedence to index creation when mixing typed templates with typeless index creation and vice-versa. {es-pull}37871[#37871] (issue: {es-issue}37773[#37773]) * Add nanosecond field mapper {es-pull}37755[#37755] (issues: {es-issue}27330[#27330], {es-issue}32601[#32601]) Ranking:: * Add ranking evaluation API {es-pull}27478[#27478] (issue: {es-issue}19195[#19195]) Recovery:: * Allow to trim all ops above a certain seq# with a term lower than X, … {es-pull}31211[#31211] (issue: {es-issue}10708[#10708]) SQL:: * Add basic support for ST_AsWKT geo function {es-pull}34205[#34205] * Add support for SYS GEOMETRY_COLUMNS {es-pull}30496[#30496] (issue: {es-issue}29872[#29872]) * Introduce HISTOGRAM grouping function {es-pull}36510[#36510] (issue: {es-issue}36509[#36509]) * DATABASE() and USER() system functions {es-pull}35946[#35946] (issue: {es-issue}35863[#35863]) * Introduce INTERVAL support {es-pull}35521[#35521] (issue: {es-issue}29990[#29990]) * Allow sorting of groups by aggregates {es-pull}38042[#38042] (issue: {es-issue}35118[#35118]) * Implement FIRST/LAST aggregate functions {es-pull}37936[#37936] (issue: {es-issue}35639[#35639]) * Introduce SQL DATE data type {es-pull}37693[#37693] (issue: {es-issue}37340[#37340]) Search:: * Add “took” timing info to response for _msearch/template API {es-pull}30961[#30961] (issue: {es-issue}30957[#30957]) * Add allow_partial_search_results flag to search requests with default setting true {es-pull}28440[#28440] (issue: {es-issue}27435[#27435]) * Enable adaptive replica selection by default {es-pull}26522[#26522] (issue: {es-issue}24915[#24915]) * Add intervals query {es-pull}36135[#36135] (issues: {es-issue}29636[#29636], {es-issue}32406[#32406]) * Added soft limit to open scroll contexts #25244 {es-pull}36009[#36009] (issue: {es-issue}25244[#25244]) * Introduce ability to minimize round-trips in CCS {es-pull}37828[#37828] (issues: {es-issue}32125[#32125], {es-issue}37566[#37566]) * Add script filter to intervals {es-pull}36776[#36776] * Add the ability to set the number of hits to track accurately {es-pull}36357[#36357] (issue: {es-issue}33028[#33028]) * Add a maximum search request size. {es-pull}26423[#26423] * Make IntervalQuery available via the Query DSL {es-pull}36135[#36135] (issue: {es-issue}29636[#29636]) Security:: * Switch internal security index to ".security-7" {es-pull}39337[#39337] (issue: {es-issue}39284[#39284]) Suggesters:: * Serialize suggestion responses as named writeables {es-pull}30284[#30284] (issue: {es-issue}26585[#26585]) [[enhancement-7.0.0]] [float] === Enhancements Aggregations:: * Uses MergingDigest instead of AVLDigest in percentiles agg {es-pull}28702[#28702] (issue: {es-issue}19528[#19528]) * Added keyed response to pipeline percentile aggregations 22302 {es-pull}36392[#36392] (issue: {es-issue}22302[#22302]) * Enforce max_buckets limit only in the final reduction phase {es-pull}36152[#36152] (issues: {es-issue}32125[#32125], {es-issue}35921[#35921]) * Histogram aggs: add empty buckets only in the final reduce step {es-pull}35921[#35921] * Handles exists query in composite aggs {es-pull}35758[#35758] * Added parent validation for auto date histogram {es-pull}35670[#35670] * Add Composite to AggregationBuilders {es-pull}38207[#38207] (issue: {es-issue}38020[#38020]) * Allow nested fields in the composite aggregation {es-pull}37178[#37178] (issue: {es-issue}28611[#28611]) * Remove single shard optimization when suggesting shard_size {es-pull}37041[#37041] (issue: {es-issue}32125[#32125]) * Use List instead of priority queue for stable sorting in bucket sort aggregator {es-pull}36748[#36748] (issue: {es-issue}36322[#36322]) * Keys are compared in BucketSortPipelineAggregation so making key type… {es-pull}36407[#36407] Allocation:: * Fail start on obsolete indices documentation {es-pull}37786[#37786] (issue: {es-issue}27073[#27073]) * Fail start on invalid index metadata {es-pull}37748[#37748] (issue: {es-issue}27073[#27073]) * Fail start of non-data node if node has data {es-pull}37347[#37347] (issue: {es-issue}27073[#27073]) Analysis:: * Allow word_delimiter_graph_filter to not adjust internal offsets {es-pull}36699[#36699] (issues: {es-issue}33710[#33710], {es-issue}34741[#34741]) * Ensure TokenFilters only produce single tokens when parsing synonyms {es-pull}34331[#34331] (issue: {es-issue}34298[#34298]) * Allow word_delimiter_graph_filter to not adjust internal offsets {es-pull}36699[#36699] (issues: {es-issue}33710[#33710], {es-issue}34741[#34741]) Audit:: * Add "request.id" to file audit logs {es-pull}35536[#35536] * Security Audit includes HTTP method for requests {es-pull}37322[#37322] (issue: {es-issue}29765[#29765]) * Add X-Forwarded-For to the logfile audit {es-pull}36427[#36427] Authentication:: * Invalidate Token API enhancements - HLRC {es-pull}36362[#36362] (issue: {es-issue}35388[#35388]) * Add DEBUG/TRACE logs for LDAP bind {es-pull}36028[#36028] * Add Tests for findSamlRealm {es-pull}35905[#35905] * Add realm information for Authenticate API {es-pull}35648[#35648] * Formal support for "password_hash" in Put User {es-pull}35242[#35242] (issue: {es-issue}34729[#34729]) * Propagate auth result to listeners {es-pull}36900[#36900] (issue: {es-issue}30794[#30794]) * Reorder realms based on last success {es-pull}36878[#36878] * Improve error message for 6.x style realm settings {es-pull}36876[#36876] (issues: {es-issue}30241[#30241], {es-issue}36026[#36026]) * Change missing authn message to not mention tokens {es-pull}36750[#36750] * Invalidate Token API enhancements - HLRC {es-pull}36362[#36362] (issue: {es-issue}35388[#35388]) * Enhance Invalidate Token API {es-pull}35388[#35388] (issues: {es-issue}34556[#34556], {es-issue}35115[#35115]) Authorization:: * Improve exact index matching performance {es-pull}36017[#36017] * `manage_token` privilege for `kibana_system` {es-pull}35751[#35751] * Grant .tasks access to kibana_system role {es-pull}35573[#35573] * Add apm_user reserved role {es-pull}38206[#38206] * Permission for restricted indices {es-pull}37577[#37577] (issue: {es-issue}34454[#34454]) * Remove kibana_user and kibana_dashboard_only_user index privileges {es-pull}37441[#37441] * Create snapshot role {es-pull}35820[#35820] (issue: {es-issue}34454[#34454]) Build:: * Sounds like typo in exception message {es-pull}35458[#35458] * Allow set section in setup section of REST tests {es-pull}34678[#34678] CCR:: * Add time since last auto follow fetch to auto follow stats {es-pull}36542[#36542] (issues: {es-issue}33007[#33007], {es-issue}35895[#35895]) * Clean followed leader index UUIDs in auto follow metadata {es-pull}36408[#36408] (issue: {es-issue}33007[#33007]) * Change AutofollowCoordinator to use wait_for_metadata_version {es-pull}36264[#36264] (issues: {es-issue}33007[#33007], {es-issue}35895[#35895]) * Refactor AutoFollowCoordinator to track leader indices per remote cluster {es-pull}36031[#36031] (issues: {es-issue}33007[#33007], {es-issue}35895[#35895]) * Concurrent file chunk fetching for CCR restore {es-pull}38495[#38495] * Tighten mapping syncing in ccr remote restore {es-pull}38071[#38071] (issues: {es-issue}36879[#36879], {es-issue}37887[#37887]) * Do not allow put mapping on follower {es-pull}37675[#37675] (issue: {es-issue}30086[#30086]) * Added ccr to xpack usage infrastructure {es-pull}37256[#37256] (issue: {es-issue}37221[#37221]) * FollowingEngine should fail with 403 if operation has no seqno assigned {es-pull}37213[#37213] * Added auto_follow_exception.timestamp field to auto follow stats {es-pull}36947[#36947] * Add time since last auto follow fetch to auto follow stats {es-pull}36542[#36542] (issues: {es-issue}33007[#33007], {es-issue}35895[#35895]) * Reduce retention lease sync intervals {es-pull}40302[#40302] * Renew retention leases while following {es-pull}39335[#39335] (issues: {es-issue}37165[#37165], {es-issue}38718[#38718]) * Reduce refresh when lookup term in FollowingEngine {es-pull}39184[#39184] * Integrate retention leases to recovery from remote {es-pull}38829[#38829] (issue: {es-issue}37165[#37165]) * Enable removal of retention leases {es-pull}38751[#38751] (issue: {es-issue}37165[#37165]) * Introduce forget follower API {es-pull}39718[#39718] (issue: {es-issue}37165[#37165]) Client:: * Fixed required fields and paths list {es-pull}39358[#39358] Cluster Coordination:: * Remove timeout task after completing cluster state publication {es-pull}40411[#40411] * Use default discovery implementation for single-node discovery {es-pull}40036[#40036] * Do not log unsuccessful join attempt each time {es-pull}39756[#39756] Core:: * Override the JVM DNS cache policy {es-pull}36570[#36570] * Replace usages of AtomicBoolean based block of code by the RunOnce class {es-pull}35553[#35553] (issue: {es-issue}35489[#35489]) * Added wait_for_metadata_version parameter to cluster state api. {es-pull}35535[#35535] * Extract RunOnce into a dedicated class {es-pull}35489[#35489] * Introduce elasticsearch-core jar {es-pull}28191[#28191] (issue: {es-issue}27933[#27933]) * Rename core module to server {es-pull}28180[#28180] (issue: {es-issue}27933[#27933]) CRUD:: * Rename seq# powered optimistic concurrency control parameters to ifSeqNo/ifPrimaryTerm {es-pull}36757[#36757] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Expose Sequence Number based Optimistic Concurrency Control in the rest layer {es-pull}36721[#36721] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Add doc's sequence number + primary term to GetResult and use it for updates {es-pull}36680[#36680] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Add seq no powered optimistic locking support to the index and delete transport actions {es-pull}36619[#36619] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Add Seq# based optimistic concurrency control to UpdateRequest {es-pull}37872[#37872] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Introduce ssl settings to reindex from remote {es-pull}37527[#37527] (issues: {es-issue}29755[#29755], {es-issue}37287[#37287]) * Use Sequence number powered OCC for processing updates {es-pull}37308[#37308] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Document Seq No powered optimistic concurrency control {es-pull}37284[#37284] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Enable IPv6 URIs in reindex from remote {es-pull}36874[#36874] * Rename seq# powered optimistic concurrency control parameters to ifSeqNo/ifPrimaryTerm {es-pull}36757[#36757] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Expose Sequence Number based Optimistic Concurrency Control in the rest layer {es-pull}36721[#36721] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Add doc's sequence number + primary term to GetResult and use it for updates {es-pull}36680[#36680] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Add seq no powered optimistic locking support to the index and delete transport actions {es-pull}36619[#36619] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Set acking timeout to 0 on dynamic mapping update {es-pull}31140[#31140] (issues: {es-issue}30672[#30672], {es-issue}30844[#30844]) Discovery-Plugins:: * Adds connect and read timeouts to discovery-gce {es-pull}28193[#28193] (issue: {es-issue}24313[#24313]) Distributed:: * [Close Index API] Mark shard copy as stale if needed during shard verification {es-pull}36755[#36755] * [Close Index API] Refactor MetadataIndexStateService {es-pull}36354[#36354] (issue: {es-issue}36249[#36249]) * [Close Index API] Add TransportShardCloseAction for pre-closing verifications {es-pull}36249[#36249] * TransportResyncReplicationAction should not honour blocks {es-pull}35795[#35795] (issues: {es-issue}35332[#35332], {es-issue}35597[#35597]) * Expose all permits acquisition in IndexShard and TransportReplicationAction {es-pull}35540[#35540] (issue: {es-issue}33888[#33888]) * [RCI] Check blocks while having index shard permit in TransportReplicationAction {es-pull}35332[#35332] (issue: {es-issue}33888[#33888]) * Recover retention leases during peer recovery {es-pull}38435[#38435] (issue: {es-issue}37165[#37165]) * Lift retention lease expiration to index shard {es-pull}38380[#38380] (issues: {es-issue}37165[#37165], {es-issue}37963[#37963], {es-issue}38070[#38070]) * Introduce retention lease background sync {es-pull}38262[#38262] (issue: {es-issue}37165[#37165]) * Allow shards of closed indices to be replicated as regular shards {es-pull}38024[#38024] (issue: {es-issue}33888[#33888]) * Expose retention leases in shard stats {es-pull}37991[#37991] (issue: {es-issue}37165[#37165]) * Introduce retention leases versioning {es-pull}37951[#37951] (issue: {es-issue}37165[#37165]) * Soft-deletes policy should always fetch latest leases {es-pull}37940[#37940] (issues: {es-issue}37165[#37165], {es-issue}37375[#37375]) * Sync retention leases on expiration {es-pull}37902[#37902] (issue: {es-issue}37165[#37165]) * Ignore shard started requests when primary term does not match {es-pull}37899[#37899] (issue: {es-issue}33888[#33888]) * Move update and delete by query to use seq# for optimistic concurrency control {es-pull}37857[#37857] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148], {es-issue}37639[#37639]) * Introduce retention lease serialization {es-pull}37447[#37447] (issues: {es-issue}37165[#37165], {es-issue}37398[#37398]) * Add run under primary permit method {es-pull}37440[#37440] (issue: {es-issue}37398[#37398]) * Introduce retention lease syncing {es-pull}37398[#37398] (issue: {es-issue}37165[#37165]) * Introduce retention lease persistence {es-pull}37375[#37375] (issue: {es-issue}37165[#37165]) * Add validation for retention lease construction {es-pull}37312[#37312] (issue: {es-issue}37165[#37165]) * Introduce retention lease expiration {es-pull}37195[#37195] (issue: {es-issue}37165[#37165]) * Introduce shard history retention leases {es-pull}37167[#37167] (issue: {es-issue}37165[#37165]) * [Close Index API] Add unique UUID to ClusterBlock {es-pull}36775[#36775] * [Close Index API] Mark shard copy as stale if needed during shard verification {es-pull}36755[#36755] * [Close Index API] Propagate tasks ids between Freeze, Close and Verify Shard actions {es-pull}36630[#36630] * Always initialize the global checkpoint {es-pull}34381[#34381] * Introduce retention lease actions {es-pull}38756[#38756] (issue: {es-issue}37165[#37165]) * Add dedicated retention lease exceptions {es-pull}38754[#38754] (issue: {es-issue}37165[#37165]) * Copy retention leases when trim unsafe commits {es-pull}37995[#37995] (issue: {es-issue}37165[#37165]) * Allow retention lease operations under blocks {es-pull}39089[#39089] (issues: {es-issue}34648[#34648], {es-issue}37165[#37165]) * Remove retention leases when unfollowing {es-pull}39088[#39088] (issues: {es-issue}34648[#34648], {es-issue}37165[#37165]) * Introduce retention lease state file {es-pull}39004[#39004] (issues: {es-issue}37165[#37165], {es-issue}38588[#38588], {es-issue}39032[#39032]) * Enable soft-deletes by default for 7.0+ indices {es-pull}38929[#38929] (issue: {es-issue}36141[#36141]) Engine:: * Remove versionType from translog {es-pull}31945[#31945] * Do retry if primary fails on AsyncAfterWriteAction {es-pull}31857[#31857] (issues: {es-issue}31716[#31716], {es-issue}31755[#31755]) * handle AsyncAfterWriteAction exception before listener is registered {es-pull}31755[#31755] (issue: {es-issue}31716[#31716]) * Use IndexWriter#flushNextBuffer to free memory {es-pull}27753[#27753] * Remove pre 6.0.0 support from InternalEngine {es-pull}27720[#27720] * Add sequence numbers based optimistic concurrency control support to Engine {es-pull}36467[#36467] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Require soft-deletes when access changes snapshot {es-pull}36446[#36446] * Use delCount of SegmentInfos to calculate numDocs {es-pull}36323[#36323] * Always configure soft-deletes field of IndexWriterConfig {es-pull}36196[#36196] (issue: {es-issue}36141[#36141]) * Enable soft-deletes by default on 7.0.0 or later {es-pull}36141[#36141] * Always return false from `refreshNeeded` on ReadOnlyEngine {es-pull}35837[#35837] (issue: {es-issue}35785[#35785]) * Add a `_freeze` / `_unfreeze` API {es-pull}35592[#35592] (issue: {es-issue}34352[#34352]) * [RCI] Add IndexShardOperationPermits.asyncBlockOperations(ActionListener) {es-pull}34902[#34902] (issue: {es-issue}33888[#33888]) * Specialize pre-closing checks for engine implementations {es-pull}38702[#38702] * Ensure that max seq # is equal to the global checkpoint when creating ReadOnlyEngines {es-pull}37426[#37426] * Enable Bulk-Merge if all source remains {es-pull}37269[#37269] * Rename setting to enable mmap {es-pull}37070[#37070] (issue: {es-issue}36668[#36668]) * Add hybridfs store type {es-pull}36668[#36668] * Introduce time-based retention policy for soft-deletes {es-pull}34943[#34943] (issue: {es-issue}34908[#34908]) * Handle AsyncAfterWriteAction failure on primary in the same way as failures on replicas {es-pull}31969[#31969] (issues: {es-issue}31716[#31716], {es-issue}31755[#31755]) * Explicitly advance max_seq_no before indexing {es-pull}39473[#39473] (issue: {es-issue}38879[#38879]) * Also mmap cfs files for hybridfs {es-pull}38940[#38940] (issue: {es-issue}36668[#36668]) Features/CAT APIs:: * Expose `search.throttled` on `_cat/indices` {es-pull}37073[#37073] (issue: {es-issue}34352[#34352]) Features/Features:: * Run Node deprecation checks locally (#38065) {es-pull}38250[#38250] (issue: {es-issue}38065[#38065]) Features/ILM:: * Ensure ILM policies run safely on leader indices {es-pull}38140[#38140] (issue: {es-issue}34648[#34648]) * Skip Shrink when numberOfShards not changed {es-pull}37953[#37953] (issue: {es-issue}33275[#33275]) * Inject Unfollow before Rollover and Shrink {es-pull}37625[#37625] (issue: {es-issue}34648[#34648]) * Add set_priority action to ILM {es-pull}37397[#37397] (issue: {es-issue}36905[#36905]) * Add Freeze Action {es-pull}36910[#36910] (issue: {es-issue}34630[#34630]) Features/Indices APIs:: * Add cluster-wide shard limit {es-pull}32856[#32856] (issue: {es-issue}20705[#20705]) * Remove RestGetAllAliasesAction {es-pull}31308[#31308] (issue: {es-issue}31129[#31129]) * Add rollover-creation-date setting to rolled over index {es-pull}31144[#31144] (issue: {es-issue}30887[#30887]) * add is-write-index flag to aliases {es-pull}30942[#30942] * Make index and bulk APIs work without types. {es-pull}29479[#29479] * Simplify deprecation issue levels {es-pull}36326[#36326] * New mapping signature and mapping string source fixed. {es-pull}37401[#37401] Features/Ingest:: * Add ignore_missing property to foreach filter (#22147) {es-pull}31578[#31578] (issue: {es-issue}22147[#22147]) * Compile mustache template only if field includes '{{'' {es-pull}37207[#37207] (issue: {es-issue}37120[#37120]) * Move ingest-geoip default databases out of config {es-pull}36949[#36949] (issue: {es-issue}36898[#36898]) * Make the ingest-geoip databases even lazier to load {es-pull}36679[#36679] * Updates the grok patterns to be consistent with the logstash {es-pull}27181[#27181] Features/Java High Level REST Client:: * HLRC API for _termvectors {es-pull}32610[#32610] (issue: {es-issue}27205[#27205]) * Fix strict setting exception handling {es-pull}37247[#37247] (issue: {es-issue}37090[#37090]) * Use nonblocking entity for requests {es-pull}32249[#32249] Features/Monitoring:: * Make Exporters Async {es-pull}35765[#35765] (issue: {es-issue}35743[#35743]) * Adding mapping for hostname field {es-pull}37288[#37288] * Remove types from internal monitoring templates and bump to api 7 {es-pull}39888[#39888] (issue: {es-issue}38637[#38637]) Features/Stats:: * Stats to record how often the ClusterState diff mechanism is used successfully {es-pull}26973[#26973] * Add JVM dns cache expiration config to JvmInfo {es-pull}36372[#36372] Features/Watcher:: * Validate email adresses when storing a watch {es-pull}34042[#34042] (issue: {es-issue}33980[#33980]) * Move watcher to use seq# and primary term for concurrency control {es-pull}37977[#37977] (issues: {es-issue}10708[#10708], {es-issue}37872[#37872]) * Use ILM for Watcher history deletion {es-pull}37443[#37443] (issue: {es-issue}32041[#32041]) * Add whitelist to HttpClient {es-pull}36817[#36817] (issue: {es-issue}29937[#29937]) * Remove the index type from internal watcher indexes {es-pull}39761[#39761] (issue: {es-issue}38637[#38637]) Geo:: * Adds a name of the field to geopoint parsing errors {es-pull}36529[#36529] (issue: {es-issue}15965[#15965]) * Add support to ShapeBuilders for building Lucene geometry {es-pull}35707[#35707] (issue: {es-issue}35320[#35320]) * Add ST_WktToSQL function {es-pull}35416[#35416] (issue: {es-issue}29872[#29872]) Index APIs:: * Add cluster-wide shard limit warnings {es-pull}34021[#34021] (issues: {es-issue}20705[#20705], {es-issue}32856[#32856]) Infra/Circuit Breakers:: * Have circuit breaker succeed on unknown mem usage {es-pull}33125[#33125] (issue: {es-issue}31767[#31767]) * Account for XContent overhead in in-flight breaker {es-pull}31613[#31613] * Script Stats: Add compilation limit counter to stats {es-pull}26387[#26387] Infra/Core:: * Add RunOnce utility class that executes a Runnable exactly once {es-pull}35484[#35484] * Improved IndexNotFoundException's default error message {es-pull}34649[#34649] (issue: {es-issue}34628[#34628]) * fix a few versionAdded values in ElasticsearchExceptions {es-pull}37877[#37877] * Add simple method to write collection of writeables {es-pull}37448[#37448] (issue: {es-issue}37398[#37398]) * Date/Time parsing: Use java time API instead of exception handling {es-pull}37222[#37222] * [API] spelling: interruptible {es-pull}37049[#37049] (issue: {es-issue}37035[#37035]) * Enhancements to IndicesQueryCache. {es-pull}39099[#39099] (issue: {es-issue}37117[#37117]) * Change zone formatting for all printers {es-pull}39568[#39568] (issue: {es-issue}38471[#38471]) Infra/Logging:: * Trim the JSON source in indexing slow logs {es-pull}38081[#38081] (issue: {es-issue}38080[#38080]) * Optimize warning header de-duplication {es-pull}37725[#37725] (issues: {es-issue}35754[#35754], {es-issue}37530[#37530], {es-issue}37597[#37597], {es-issue}37622[#37622]) * Remove warn-date from warning headers {es-pull}37622[#37622] (issues: {es-issue}35754[#35754], {es-issue}37530[#37530], {es-issue}37597[#37597]) * Add some deprecation optimizations {es-pull}37597[#37597] (issues: {es-issue}35754[#35754], {es-issue}37530[#37530]) * Only update response headers if we have a new one {es-pull}37590[#37590] (issues: {es-issue}35754[#35754], {es-issue}37530[#37530]) Infra/Packaging:: * Choose JVM options ergonomically {es-pull}30684[#30684] * Add OS/architecture classifier to distributions {es-pull}37881[#37881] * Change file descriptor limit to 65535 {es-pull}37537[#37537] (issue: {es-issue}35839[#35839]) * Exit batch files explictly using ERRORLEVEL {es-pull}29583[#29583] (issue: {es-issue}29582[#29582]) * Add no-jdk distributions {es-pull}39882[#39882] * Allow AVX-512 on JDK 11+ {es-pull}40828[#40828] (issue: {es-issue}32138[#32138]) Infra/REST API:: * Remove hand-coded XContent duplicate checks {es-pull}34588[#34588] (issues: {es-issue}22073[#22073], {es-issue}22225[#22225], {es-issue}22253[#22253]) * Add the `include_type_name` option to the search and document APIs. {es-pull}29506[#29506] (issue: {es-issue}15613[#15613]) * Validate `op_type` for `_create` {es-pull}27483[#27483] Infra/Scripting:: * Tests: Add support for custom contexts to mock scripts {es-pull}34100[#34100] * Reflect factory signatures in painless classloader {es-pull}34088[#34088] * Handle missing values in painless {es-pull}32207[#32207] (issue: {es-issue}29286[#29286]) * Add getZone to JodaCompatibleZonedDateTime {es-pull}37084[#37084] * [Painless] Add boxed type to boxed type casts for method/return {es-pull}36571[#36571] Infra/Packaging:: * Use bundled JDK in Docker images {es-pull}40238[#40238] * Upgrade bundled JDK and Docker images to JDK 12 {es-pull}40229[#40229] * Bundle java in distributions {es-pull}38013[#38013] (issue: {es-issue}31845[#31845]) Infra/Settings:: * Settings: Add keystore creation to add commands {es-pull}26126[#26126] * Separate out validation of groups of settings {es-pull}34184[#34184] * Provide a clearer error message on keystore add {es-pull}39327[#39327] (issue: {es-issue}39324[#39324]) Infra/Transport API:: * Change BWC version for VerifyRepositoryResponse {es-pull}30796[#30796] (issue: {es-issue}30762[#30762]) Ingest:: * Grok fix duplicate patterns JAVACLASS and JAVAFILE {es-pull}35886[#35886] * Implement Drop Processor {es-pull}32278[#32278] (issue: {es-issue}23726[#23726]) Java High Level REST Client:: * Add get users action {es-pull}36332[#36332] (issue: {es-issue}29827[#29827]) * Add delete template API {es-pull}36320[#36320] (issue: {es-issue}27205[#27205]) * Implement get-user-privileges API {es-pull}36292[#36292] * Get Deprecation Info API {es-pull}36279[#36279] (issue: {es-issue}29827[#29827]) * Add support for Follow Stats API {es-pull}36253[#36253] (issue: {es-issue}33824[#33824]) * Add support for CCR Stats API {es-pull}36213[#36213] (issue: {es-issue}33824[#33824]) * Put Role {es-pull}36209[#36209] (issue: {es-issue}29827[#29827]) * Add index templates exist API {es-pull}36132[#36132] (issue: {es-issue}27205[#27205]) * Add support for CCR Get Auto Follow Pattern apis {es-pull}36049[#36049] (issue: {es-issue}33824[#33824]) * Add support for CCR Delete Auto Follow Pattern API {es-pull}35981[#35981] (issue: {es-issue}33824[#33824]) * Remove fromXContent from IndexUpgradeInfoResponse {es-pull}35934[#35934] * Add delete expired data API {es-pull}35906[#35906] (issue: {es-issue}29827[#29827]) * Execute watch API {es-pull}35868[#35868] (issue: {es-issue}29827[#29827]) * Add ability to put user with a password hash {es-pull}35844[#35844] (issue: {es-issue}35242[#35242]) * Add ML find file structure API {es-pull}35833[#35833] (issue: {es-issue}29827[#29827]) * Add support for get roles API {es-pull}35787[#35787] (issue: {es-issue}29827[#29827]) * Added support for CCR Put Auto Follow Pattern API {es-pull}35780[#35780] (issue: {es-issue}33824[#33824]) * XPack ML info action {es-pull}35777[#35777] (issue: {es-issue}29827[#29827]) * ML Delete event from Calendar {es-pull}35760[#35760] (issue: {es-issue}29827[#29827]) * Add ML revert model snapshot API {es-pull}35750[#35750] (issue: {es-issue}29827[#29827]) * ML Get Calendar Events {es-pull}35747[#35747] (issue: {es-issue}29827[#29827]) * Add high-level REST client API for `_freeze` and `_unfreeze` {es-pull}35723[#35723] (issue: {es-issue}34352[#34352]) * Fix issue in equals impl for GlobalOperationPrivileges {es-pull}35721[#35721] * ML Delete job from calendar {es-pull}35713[#35713] (issue: {es-issue}29827[#29827]) * ML Add Event To Calendar API {es-pull}35704[#35704] (issue: {es-issue}29827[#29827]) * Add ML update model snapshot API (#35537) {es-pull}35694[#35694] (issue: {es-issue}29827[#29827]) * Add support for CCR Unfollow API {es-pull}35693[#35693] (issue: {es-issue}33824[#33824]) * Clean up PutLicenseResponse {es-pull}35689[#35689] (issue: {es-issue}35547[#35547]) * Clean up StartBasicResponse {es-pull}35688[#35688] (issue: {es-issue}35547[#35547]) * Add support for put privileges API {es-pull}35679[#35679] * ML Add Job to Calendar API {es-pull}35666[#35666] (issue: {es-issue}29827[#29827]) * Add support for CCR Resume Follow API {es-pull}35638[#35638] (issue: {es-issue}33824[#33824]) * Add support for get application privileges API {es-pull}35556[#35556] (issue: {es-issue}29827[#29827]) * Clean up XPackInfoResponse class and related tests {es-pull}35547[#35547] * Add parameters to stopRollupJob API {es-pull}35545[#35545] (issue: {es-issue}34811[#34811]) * Add ML delete model snapshot API {es-pull}35537[#35537] (issue: {es-issue}29827[#29827]) * Add get watch API {es-pull}35531[#35531] (issue: {es-issue}29827[#29827]) * Add ML Update Filter API {es-pull}35522[#35522] (issue: {es-issue}29827[#29827]) * Add ml get filters api {es-pull}35502[#35502] (issue: {es-issue}29827[#29827]) * Add ML get model snapshots API {es-pull}35487[#35487] (issue: {es-issue}29827[#29827]) * Add "_has_privileges" API to Security Client {es-pull}35479[#35479] (issue: {es-issue}29827[#29827]) * Add Delete Privileges API to HLRC {es-pull}35454[#35454] (issue: {es-issue}29827[#29827]) * Add support for CCR Put Follow API {es-pull}35409[#35409] * Add ML delete filter action {es-pull}35382[#35382] (issue: {es-issue}29827[#29827]) * Add delete user action {es-pull}35294[#35294] (issue: {es-issue}29827[#29827]) * HLRC for _mtermvectors {es-pull}35266[#35266] (issues: {es-issue}27205[#27205], {es-issue}33447[#33447]) * Reindex API with wait_for_completion false {es-pull}35202[#35202] (issue: {es-issue}27205[#27205]) * Add watcher stats API {es-pull}35185[#35185] (issue: {es-issue}29827[#29827]) * HLRC support for getTask {es-pull}35166[#35166] (issue: {es-issue}27205[#27205]) * Add GetRollupIndexCaps API {es-pull}35102[#35102] (issue: {es-issue}29827[#29827]) * HLRC: migration api - upgrade {es-pull}34898[#34898] (issue: {es-issue}29827[#29827]) * Add stop rollup job support to HL REST Client {es-pull}34702[#34702] (issue: {es-issue}29827[#29827]) * Bulk Api support for global parameters {es-pull}34528[#34528] (issue: {es-issue}26026[#26026]) * Add delete rollup job support to HL REST Client {es-pull}34066[#34066] (issue: {es-issue}29827[#29827]) * Add support for get license basic/trial status API {es-pull}33176[#33176] (issue: {es-issue}29827[#29827]) * Add machine learning open job {es-pull}32860[#32860] (issue: {es-issue}29827[#29827]) * Add ML HLRC wrapper and put_job API call {es-pull}32726[#32726] * Add Get Snapshots High Level REST API {es-pull}31537[#31537] (issue: {es-issue}27205[#27205]) Java Low Level REST Client:: * On retry timeout add root exception {es-pull}25576[#25576] License:: * Require acknowledgement to start_trial license {es-pull}30135[#30135] (issue: {es-issue}30134[#30134]) * Handle malformed license signatures {es-pull}37137[#37137] (issue: {es-issue}35340[#35340]) Machine Learning:: * Create the ML annotations index {es-pull}36731[#36731] (issues: {es-issue}26034[#26034], {es-issue}33376[#33376]) * Split in batches and migrate all jobs and datafeeds {es-pull}36716[#36716] (issue: {es-issue}32905[#32905]) * Add cluster setting to enable/disable config migration {es-pull}36700[#36700] (issue: {es-issue}32905[#32905]) * Add audits when deprecation warnings occur with datafeed start {es-pull}36233[#36233] * Add lazy parsing for DatafeedConfig:Aggs,Query {es-pull}36117[#36117] * Add support for lazy nodes (#29991) {es-pull}34538[#34538] (issue: {es-issue}29991[#29991]) * Move ML Optimistic Concurrency Control to Seq No {es-pull}38278[#38278] (issues: {es-issue}10708[#10708], {es-issue}36148[#36148]) * Add explanation so far to file structure finder exceptions {es-pull}38191[#38191] (issue: {es-issue}29821[#29821]) * Add reason field in JobTaskState {es-pull}38029[#38029] (issue: {es-issue}34431[#34431]) * Add _meta information to all ML indices {es-pull}37964[#37964] * Add upgrade mode docs, hlrc, and fix bug {es-pull}37942[#37942] * Tighten up use of aliases rather than concrete indices {es-pull}37874[#37874] * Add support for single bucket aggs in Datafeeds {es-pull}37544[#37544] (issue: {es-issue}36838[#36838]) * Create the ML annotations index {es-pull}36731[#36731] (issues: {es-issue}26034[#26034], {es-issue}33376[#33376]) * Merge the Jindex master feature branch {es-pull}36702[#36702] (issue: {es-issue}32905[#32905]) * Add cluster setting to enable/disable config migration {es-pull}36700[#36700] (issue: {es-issue}32905[#32905]) * Allow stop unassigned datafeed and relax unset upgrade mode wait {es-pull}39034[#39034] Mapping:: * Log document id when MapperParsingException occurs {es-pull}37800[#37800] (issue: {es-issue}37658[#37658]) * [API] spelling: unknown {es-pull}37056[#37056] (issue: {es-issue}37035[#37035]) * Make SourceToParse immutable {es-pull}36971[#36971] * Use index-prefix fields for terms of length min_chars - 1 {es-pull}36703[#36703] * Introduce a parameter suppress_types_warnings. {es-pull}38923[#38923] Network:: * Add cors support to NioHttpServerTransport {es-pull}30827[#30827] (issue: {es-issue}28898[#28898]) * Reintroduce mandatory http pipelining support {es-pull}30820[#30820] * Make http pipelining support mandatory {es-pull}30695[#30695] (issues: {es-issue}28898[#28898], {es-issue}29500[#29500]) * Add nio http server transport {es-pull}29587[#29587] (issue: {es-issue}28898[#28898]) * Add class for serializing message to bytes {es-pull}29384[#29384] (issue: {es-issue}28898[#28898]) * Selectors operate on channel contexts {es-pull}28468[#28468] (issue: {es-issue}27260[#27260]) * Unify nio read / write channel contexts {es-pull}28160[#28160] (issue: {es-issue}27260[#27260]) * Create nio-transport plugin for NioTransport {es-pull}27949[#27949] (issue: {es-issue}27260[#27260]) * Add elasticsearch-nio jar for base nio classes {es-pull}27801[#27801] (issue: {es-issue}27802[#27802]) * Unify transport settings naming {es-pull}36623[#36623] * Add sni name to SSLEngine in netty transport {es-pull}33144[#33144] (issue: {es-issue}32517[#32517]) * Add cors support to NioHttpServerTransport {es-pull}30827[#30827] (issue: {es-issue}28898[#28898]) * Reintroduce mandatory http pipelining support {es-pull}30820[#30820] * Make http pipelining support mandatory {es-pull}30695[#30695] (issues: {es-issue}28898[#28898], {es-issue}29500[#29500]) * Add nio http server transport {es-pull}29587[#29587] (issue: {es-issue}28898[#28898]) * Selectors operate on channel contexts {es-pull}28468[#28468] (issue: {es-issue}27260[#27260]) * Unify nio read / write channel contexts {es-pull}28160[#28160] (issue: {es-issue}27260[#27260]) * Create nio-transport plugin for NioTransport {es-pull}27949[#27949] (issue: {es-issue}27260[#27260]) * Add elasticsearch-nio jar for base nio classes {es-pull}27801[#27801] (issue: {es-issue}27802[#27802]) * Add NioGroup for use in different transports {es-pull}27737[#27737] (issue: {es-issue}27260[#27260]) * Add read timeouts to http module {es-pull}27713[#27713] * Implement byte array reusage in `NioTransport` {es-pull}27696[#27696] (issue: {es-issue}27563[#27563]) * Introduce resizable inbound byte buffer {es-pull}27551[#27551] (issue: {es-issue}27563[#27563]) * Decouple nio constructs from the tcp transport {es-pull}27484[#27484] (issue: {es-issue}27260[#27260]) * Remove manual tracking of registered channels {es-pull}27445[#27445] (issue: {es-issue}27260[#27260]) * Remove tcp profile from low level nio channel {es-pull}27441[#27441] (issue: {es-issue}27260[#27260]) * Decouple `ChannelFactory` from Tcp classes {es-pull}27286[#27286] (issue: {es-issue}27260[#27260]) * Enable TLSv1.3 by default for JDKs with support {es-pull}38103[#38103] (issue: {es-issue}32276[#32276]) Packaging:: * Introduce Docker images build {es-pull}36246[#36246] * Move creation of temporary directory to Java {es-pull}36002[#36002] (issue: {es-issue}31003[#31003]) Percolator:: * Make the `type` parameter optional when percolating existing documents. {es-pull}39987[#39987] (issue: {es-issue}39963[#39963]) * Add support for selecting percolator query candidate matches containing geo_point based queries {es-pull}26040[#26040] Plugins:: * Plugin install: don't print download progress in batch mode {es-pull}36361[#36361] Ranking:: * Add k parameter to PrecisionAtK metric {es-pull}27569[#27569] * Vector field {es-pull}33022[#33022] (issue: {es-issue}31615[#31615]) Recovery:: * SyncedFlushService.getShardRoutingTable() should use metadata to check for index existence {es-pull}37691[#37691] (issue: {es-issue}33888[#33888]) * Make prepare engine step of recovery source non-blocking {es-pull}37573[#37573] (issue: {es-issue}37174[#37174]) * Make recovery source send operations non-blocking {es-pull}37503[#37503] (issue: {es-issue}37458[#37458]) * Prepare to make send translog of recovery non-blocking {es-pull}37458[#37458] (issue: {es-issue}37291[#37291]) * Make finalize step of recovery source non-blocking {es-pull}37388[#37388] (issue: {es-issue}37291[#37291]) * Make recovery source partially non-blocking {es-pull}37291[#37291] (issue: {es-issue}36195[#36195]) * Do not mutate RecoveryResponse {es-pull}37204[#37204] (issue: {es-issue}37174[#37174]) * Don't block on peer recovery on the target side {es-pull}37076[#37076] (issue: {es-issue}36195[#36195]) * Reduce recovery time with compress or secure transport {es-pull}36981[#36981] (issue: {es-issue}33844[#33844]) * Translog corruption marker {es-pull}33415[#33415] (issue: {es-issue}31389[#31389]) * Do not wait for advancement of checkpoint in recovery {es-pull}39006[#39006] (issues: {es-issue}38949[#38949], {es-issue}39000[#39000]) Rollup:: * Add non-X-Pack centric rollup endpoints {es-pull}36383[#36383] (issues: {es-issue}35958[#35958], {es-issue}35962[#35962]) * Add more diagnostic stats to job {es-pull}35471[#35471] * Add `wait_for_completion` option to StopRollupJob API {es-pull}34811[#34811] (issue: {es-issue}34574[#34574]) * Replace the TreeMap in the composite aggregation {es-pull}36675[#36675] Recovery:: * Exposed engine must include all operations below global checkpoint during rollback {es-pull}36159[#36159] (issue: {es-issue}32867[#32867]) Scripting:: * Update joda compat methods to use compat class {es-pull}36654[#36654] * [Painless] Add boxed type to boxed type casts for method/return {es-pull}36571[#36571] * [Painless] Add def to boxed type casts {es-pull}36506[#36506] Settings:: * Add user-defined cluster metadata {es-pull}33325[#33325] (issue: {es-issue}33220[#33220]) Search:: * Make limit on number of expanded fields configurable {es-pull}35284[#35284] (issues: {es-issue}26541[#26541], {es-issue}34778[#34778]) * Search: Simply SingleFieldsVisitor {es-pull}34052[#34052] * Don't count hits via the collector if the hit count can be computed from index stats. {es-pull}33701[#33701] * Limit the number of concurrent requests per node {es-pull}31206[#31206] (issue: {es-issue}31192[#31192]) * Default max concurrent search req. numNodes * 5 {es-pull}31171[#31171] (issues: {es-issue}30783[#30783], {es-issue}30994[#30994]) * Change ScriptException status to 400 (bad request) {es-pull}30861[#30861] (issue: {es-issue}12315[#12315]) * Change default value to true for transpositions parameter of fuzzy query {es-pull}26901[#26901] * Introducing "took" time (in ms) for `_msearch` {es-pull}23767[#23767] (issue: {es-issue}23131[#23131]) * Add copy constructor to SearchRequest {es-pull}36641[#36641] (issue: {es-issue}32125[#32125]) * Add raw sort values to SearchSortValues transport serialization {es-pull}36617[#36617] (issue: {es-issue}32125[#32125]) * Add sort and collapse info to SearchHits transport serialization {es-pull}36555[#36555] (issue: {es-issue}32125[#32125]) * Add default methods to DocValueFormat {es-pull}36480[#36480] * Respect indices options on _msearch {es-pull}35887[#35887] * Allow efficient can_match phases on frozen indices {es-pull}35431[#35431] (issues: {es-issue}34352[#34352], {es-issue}34357[#34357]) * Add a new query type - ScriptScoreQuery {es-pull}34533[#34533] (issues: {es-issue}23850[#23850], {es-issue}27588[#27588], {es-issue}30303[#30303]) * Tie break on cluster alias when merging shard search failures {es-pull}38715[#38715] (issue: {es-issue}38672[#38672]) * Add finalReduce flag to SearchRequest {es-pull}38104[#38104] (issues: {es-issue}37000[#37000], {es-issue}37838[#37838]) * Streamline skip_unavailable handling {es-pull}37672[#37672] (issue: {es-issue}32125[#32125]) * Expose sequence number and primary terms in search responses {es-pull}37639[#37639] * Add support for merging multiple search responses into one {es-pull}37566[#37566] (issue: {es-issue}32125[#32125]) * Allow field types to optimize phrase prefix queries {es-pull}37436[#37436] (issue: {es-issue}31921[#31921]) * Add support for providing absolute start time to SearchRequest {es-pull}37142[#37142] (issue: {es-issue}32125[#32125]) * Ensure that local cluster alias is never treated as remote {es-pull}37121[#37121] (issues: {es-issue}32125[#32125], {es-issue}36997[#36997]) * [API] spelling: cacheable {es-pull}37047[#37047] (issue: {es-issue}37035[#37035]) * Add ability to suggest shard_size on coord node rewrite {es-pull}37017[#37017] (issues: {es-issue}32125[#32125], {es-issue}36997[#36997], {es-issue}37000[#37000]) * Skip final reduction if SearchRequest holds a cluster alias {es-pull}37000[#37000] (issues: {es-issue}32125[#32125], {es-issue}36997[#36997]) * Add support for local cluster alias to SearchRequest {es-pull}36997[#36997] (issue: {es-issue}32125[#32125]) * Use SearchRequest copy constructor in ExpandSearchPhase {es-pull}36772[#36772] (issue: {es-issue}36641[#36641]) * Add raw sort values to SearchSortValues transport serialization {es-pull}36617[#36617] (issue: {es-issue}32125[#32125]) * Avoid BytesRef's copying in ScriptDocValues's Strings {es-pull}29581[#29581] (issue: {es-issue}29567[#29567]) Security:: * Make credentials mandatory when launching xpack/migrate {es-pull}36197[#36197] (issues: {es-issue}29847[#29847], {es-issue}33972[#33972]) * Move CAS operations in TokenService to sequence numbers {es-pull}38311[#38311] (issues: {es-issue}10708[#10708], {es-issue}37872[#37872]) * Cleanup construction of interceptors {es-pull}38294[#38294] * Add passphrase support to elasticsearch-keystore {es-pull}37472[#37472] (issue: {es-issue}32691[#32691]) * Types removal security index template {es-pull}39705[#39705] (issue: {es-issue}38637[#38637]) * Types removal security index template {es-pull}39542[#39542] (issue: {es-issue}38637[#38637]) Snapshot/Restore:: * #31608 Add S3 Setting to Force Path Type Access {es-pull}34721[#34721] (issue: {es-issue}31608[#31608]) * Allow Parallel Restore Operations {es-pull}36397[#36397] * Repo Creation out of ClusterStateTask {es-pull}36157[#36157] (issue: {es-issue}9488[#9488]) * Add read-only repository verification {es-pull}35731[#35731] (issue: {es-issue}35703[#35703]) * RestoreService should update primary terms when restoring shards of existing indices {es-pull}38177[#38177] (issue: {es-issue}33888[#33888]) * Allow open indices to be restored {es-pull}37733[#37733] * Create specific exception for when snapshots are in progress {es-pull}37550[#37550] (issue: {es-issue}37541[#37541]) * Make Atomic Blob Writes Mandatory {es-pull}37168[#37168] (issues: {es-issue}37011[#37011], {es-issue}37066[#37066]) * Speed up HDFS Repository Writes {es-pull}37069[#37069] * Implement Atomic Blob Writes for HDFS Repository {es-pull}37066[#37066] (issue: {es-issue}37011[#37011]) * [API] spelling: repositories {es-pull}37053[#37053] (issue: {es-issue}37035[#37035]) * Use CancellableThreads to Abort {es-pull}35901[#35901] (issue: {es-issue}21759[#21759]) * S3 client encryption {es-pull}30513[#30513] (issues: {es-issue}11128[#11128], {es-issue}16843[#16843]) * Mark Deleted Snapshot Directories with Tombstones {es-pull}40228[#40228] (issue: {es-issue}39852[#39852]) Stats:: * Handle OS pretty name on old OS without OS release {es-pull}35453[#35453] (issue: {es-issue}35440[#35440]) Store:: * Add RemoveCorruptedShardDataCommand {es-pull}32281[#32281] (issues: {es-issue}31389[#31389], {es-issue}32279[#32279]) * Add option to force load term dict into memory {es-pull}39741[#39741] SQL:: * Introduce support for NULL values {es-pull}34573[#34573] (issue: {es-issue}32079[#32079]) * Extend the ODBC metric by differentiating between 32 and 64bit platforms {es-pull}36753[#36753] (issue: {es-issue}36740[#36740]) * Fix wrong appliance of StackOverflow limit for IN {es-pull}36724[#36724] (issue: {es-issue}36592[#36592]) * Introduce NOW/CURRENT_TIMESTAMP function {es-pull}36562[#36562] (issue: {es-issue}36534[#36534]) * Move requests' parameters to requests JSON body {es-pull}36149[#36149] (issue: {es-issue}35992[#35992]) * Make INTERVAL millis optional {es-pull}36043[#36043] (issue: {es-issue}36032[#36032]) * Implement data type verification for conditionals {es-pull}35916[#35916] (issue: {es-issue}35907[#35907]) * Implement GREATEST and LEAST functions {es-pull}35879[#35879] (issue: {es-issue}35878[#35878]) * Implement null safe equality operator `<=>` {es-pull}35873[#35873] (issue: {es-issue}35871[#35871]) * SYS COLUMNS returns ODBC specific schema {es-pull}35870[#35870] (issue: {es-issue}35376[#35376]) * Polish grammar for intervals {es-pull}35853[#35853] * Add filtering to SYS TYPES {es-pull}35852[#35852] (issue: {es-issue}35342[#35342]) * Implement NULLIF(expr1, expr2) function {es-pull}35826[#35826] (issue: {es-issue}35818[#35818]) * Lock down JDBC driver {es-pull}35798[#35798] (issue: {es-issue}35437[#35437]) * Implement NVL(expr1, expr2) {es-pull}35794[#35794] (issue: {es-issue}35782[#35782]) * Implement ISNULL(expr1, expr2) {es-pull}35793[#35793] (issue: {es-issue}35781[#35781]) * Implement IFNULL variant of COALESCE {es-pull}35762[#35762] (issue: {es-issue}35749[#35749]) * XPack FeatureSet functionality {es-pull}35725[#35725] (issue: {es-issue}34821[#34821]) * Perform lazy evaluation of mismatched mappings {es-pull}35676[#35676] (issues: {es-issue}35659[#35659], {es-issue}35675[#35675]) * Improve validation of unsupported fields {es-pull}35675[#35675] (issue: {es-issue}35673[#35673]) * Move internals from Joda to java.time {es-pull}35649[#35649] (issue: {es-issue}35633[#35633]) * Allow look-ahead resolution of aliases for WHERE clause {es-pull}38450[#38450] (issue: {es-issue}29983[#29983]) * Implement CURRENT_DATE {es-pull}38175[#38175] (issue: {es-issue}38160[#38160]) * Generate relevant error message when grouping functions are not used in GROUP BY {es-pull}38017[#38017] (issue: {es-issue}37952[#37952]) * Skip the nested and object field types in case of an ODBC request {es-pull}37948[#37948] (issue: {es-issue}37801[#37801]) * Add protocol tests and remove jdbc_type from drivers response {es-pull}37516[#37516] (issues: {es-issue}36635[#36635], {es-issue}36882[#36882]) * Remove slightly used meta commands {es-pull}37506[#37506] (issue: {es-issue}37409[#37409]) * Describe aliases as views {es-pull}37496[#37496] (issue: {es-issue}37422[#37422]) * Make `FULL` non-reserved keyword in the grammar {es-pull}37377[#37377] (issue: {es-issue}37376[#37376]) * Use declared source for error messages {es-pull}37161[#37161] * Improve error message when unable to translate to ES query DSL {es-pull}37129[#37129] (issue: {es-issue}37040[#37040]) * [API] spelling: subtract {es-pull}37055[#37055] (issue: {es-issue}37035[#37035]) * [API] spelling: similar {es-pull}37054[#37054] (issue: {es-issue}37035[#37035]) * [API] spelling: input {es-pull}37048[#37048] (issue: {es-issue}37035[#37035]) * Enhance message for PERCENTILE[_RANK] with field as 2nd arg {es-pull}36933[#36933] (issue: {es-issue}36903[#36903]) * Preserve original source for each expression {es-pull}36912[#36912] (issue: {es-issue}36894[#36894]) * Extend the ODBC metric by differentiating between 32 and 64bit platforms {es-pull}36753[#36753] (issue: {es-issue}36740[#36740]) * Fix wrong appliance of StackOverflow limit for IN {es-pull}36724[#36724] (issue: {es-issue}36592[#36592]) * Enhance checks for inexact fields {es-pull}39427[#39427] (issue: {es-issue}38501[#38501]) * Change the default precision for CURRENT_TIMESTAMP function {es-pull}39391[#39391] (issue: {es-issue}39288[#39288]) * Add "fuzziness" option to QUERY and MATCH function predicates {es-pull}40529[#40529] (issue: {es-issue}40495[#40495]) * Add "validate.properties" property to JDBC's allowed list of settings {es-pull}39050[#39050] (issue: {es-issue}38068[#38068]) Suggesters:: * Remove unused empty constructors from suggestions classes {es-pull}37295[#37295] * [API] spelling: likelihood {es-pull}37052[#37052] (issue: {es-issue}37035[#37035]) Task Management:: * Periodically try to reassign unassigned persistent tasks {es-pull}36069[#36069] (issue: {es-issue}35792[#35792]) * Only require task permissions {es-pull}35667[#35667] (issue: {es-issue}35573[#35573]) * Retry if task can't be written {es-pull}35054[#35054] (issue: {es-issue}33764[#33764]) ZenDiscovery:: * Introduce vote withdrawal {es-pull}35446[#35446] * Add basic Zen1 transport-level BWC {es-pull}35443[#35443] * Add diff-based publishing {es-pull}35290[#35290] * Introduce auto_shrink_voting_configuration setting {es-pull}35217[#35217] * Introduce transport API for cluster bootstrapping {es-pull}34961[#34961] * Reconfigure cluster as its membership changes {es-pull}34592[#34592] (issue: {es-issue}33924[#33924]) * Fail fast on disconnects {es-pull}34503[#34503] * Add storage-layer disruptions to CoordinatorTests {es-pull}34347[#34347] * Add low-level bootstrap implementation {es-pull}34345[#34345] * Gather votes from all nodes {es-pull}34335[#34335] * Add Cluster State Applier {es-pull}34257[#34257] * Add safety phase to CoordinatorTests {es-pull}34241[#34241] * Integrate FollowerChecker with Coordinator {es-pull}34075[#34075] * Integrate LeaderChecker with Coordinator {es-pull}34049[#34049] * Trigger join when active master detected {es-pull}34008[#34008] * Update PeerFinder term on term bump {es-pull}33992[#33992] * Calculate optimal cluster configuration {es-pull}33924[#33924] * Introduce FollowersChecker {es-pull}33917[#33917] * Integrate publication pipeline into Coordinator {es-pull}33771[#33771] * Add DisruptableMockTransport {es-pull}33713[#33713] * Implement basic cluster formation {es-pull}33668[#33668] * Introduce LeaderChecker {es-pull}33024[#33024] * Add leader-side join handling logic {es-pull}33013[#33013] * Add PeerFinder#onFoundPeersUpdated {es-pull}32939[#32939] * Introduce PreVoteCollector {es-pull}32847[#32847] * Introduce ElectionScheduler {es-pull}32846[#32846] * Introduce ElectionScheduler {es-pull}32709[#32709] * Add HandshakingTransportAddressConnector {es-pull}32643[#32643] (issue: {es-issue}32246[#32246]) * Add UnicastConfiguredHostsResolver {es-pull}32642[#32642] (issue: {es-issue}32246[#32246]) * Cluster state publication pipeline {es-pull}32584[#32584] (issue: {es-issue}32006[#32006]) * Introduce gossip-like discovery of master nodes {es-pull}32246[#32246] * Add core coordination algorithm for cluster state publishing {es-pull}32171[#32171] (issue: {es-issue}32006[#32006]) * Add term and config to cluster state {es-pull}32100[#32100] (issue: {es-issue}32006[#32006]) * Add discovery types to cluster stats {es-pull}36442[#36442] * Introduce `zen2` discovery type {es-pull}36298[#36298] * Persist cluster states the old way on non-master-eligible nodes {es-pull}36247[#36247] (issue: {es-issue}3[#3]) * Storage layer WriteStateException propagation {es-pull}36052[#36052] * Implement Tombstone REST APIs {es-pull}36007[#36007] * Update default for USE_ZEN2 to true {es-pull}35998[#35998] * Add warning if cluster fails to form fast enough {es-pull}35993[#35993] * Allow Setting a List of Bootstrap Nodes to Wait for {es-pull}35847[#35847] * VotingTombstone class {es-pull}35832[#35832] * PersistedState interface implementation {es-pull}35819[#35819] * Support rolling upgrades from Zen1 {es-pull}35737[#35737] * Add lag detector {es-pull}35685[#35685] * Move ClusterState fields to be persisted to ClusterState.Metadata {es-pull}35625[#35625] * Introduce ClusterBootstrapService {es-pull}35488[#35488] * Introduce vote withdrawal {es-pull}35446[#35446] * Add basic Zen1 transport-level BWC {es-pull}35443[#35443] * Add elasticsearch-node detach-cluster tool {es-pull}37979[#37979] * Deprecate minimum_master_nodes {es-pull}37868[#37868] * Step down as master when configured out of voting configuration {es-pull}37802[#37802] (issue: {es-issue}37712[#37712]) * Enforce cluster UUIDs {es-pull}37775[#37775] * Bubble exceptions up in ClusterApplierService {es-pull}37729[#37729] * Use m_m_nodes from Zen1 master for Zen2 bootstrap {es-pull}37701[#37701] * Add tool elasticsearch-node unsafe-bootstrap {es-pull}37696[#37696] * Report terms and version if cluster does not form {es-pull}37473[#37473] * Bootstrap a Zen2 cluster once quorum is discovered {es-pull}37463[#37463] * Zen2: Add join validation {es-pull}37203[#37203] * Publish cluster states in chunks {es-pull}36973[#36973] [[bug-7.0.0]] [float] === Bug fixes Aggregations:: * Fix InternalAutoDateHistogram reproducible failure {es-pull}32723[#32723] (issue: {es-issue}32215[#32215]) * fix MultiValuesSourceFieldConfig toXContent {es-pull}36525[#36525] (issue: {es-issue}36474[#36474]) * Cache the score of the parent document in the nested agg {es-pull}36019[#36019] (issues: {es-issue}34555[#34555], {es-issue}35985[#35985]) * Correct implemented interface of ParsedReverseNested {es-pull}35455[#35455] (issue: {es-issue}35449[#35449]) * Handle IndexOrDocValuesQuery in composite aggregation {es-pull}35392[#35392] * Don't load global ordinals with the `map` execution_hint {es-pull}37833[#37833] (issue: {es-issue}37705[#37705]) * Issue #37303 - Invalid variance fix {es-pull}37384[#37384] (issue: {es-issue}37303[#37303]) * Skip sibling pipeline aggregators reduction during non-final reduce {es-pull}40101[#40101] (issue: {es-issue}40059[#40059]) * Extend nextDoc to delegate to the wrapped doc-value iterator for date_nanos {es-pull}39176[#39176] (issue: {es-issue}39107[#39107]) * Only create MatrixStatsResults on final reduction {es-pull}38130[#38130] (issue: {es-issue}37587[#37587]) Allocation:: * Fix _host based require filters {es-pull}38173[#38173] * ALLOC: Fail Stale Primary Alloc. Req. without Data {es-pull}37226[#37226] (issue: {es-issue}37098[#37098]) Analysis:: * Close #26771: beider_morse phonetic encoder failure when languageset unspecified {es-pull}26848[#26848] (issue: {es-issue}26771[#26771]) * Fix PreConfiguredTokenFilters getSynonymFilter() implementations {es-pull}38839[#38839] (issue: {es-issue}38793[#38793]) Audit:: * Fix origin.type for connection_* events {es-pull}36410[#36410] * Fix IndexAuditTrail rolling restart on rollover edge {es-pull}35988[#35988] (issue: {es-issue}33867[#33867]) * Fix NPE in Logfile Audit Filter {es-pull}38120[#38120] (issue: {es-issue}38097[#38097]) * LoggingAuditTrail correctly handle ReplicatedWriteRequest {es-pull}39925[#39925] (issue: {es-issue}39555[#39555]) Authorization:: * Empty GetAliases authorization fix {es-pull}34444[#34444] (issue: {es-issue}31952[#31952]) Authentication:: * Fix kerberos setting registration {es-pull}35986[#35986] (issues: {es-issue}30241[#30241], {es-issue}35942[#35942]) * Add support for Kerberos V5 Oid {es-pull}35764[#35764] (issue: {es-issue}34763[#34763]) * Enhance parsing of StatusCode in SAML Responses {es-pull}38628[#38628] * Limit token expiry to 1 hour maximum {es-pull}38244[#38244] * Fix expired token message in Exception header {es-pull}37196[#37196] * Fix NPE in CachingUsernamePasswordRealm {es-pull}36953[#36953] (issue: {es-issue}36951[#36951]) * Allow non super users to create API keys {es-pull}40028[#40028] (issue: {es-issue}40029[#40029]) * Use consistent view of realms for authentication {es-pull}38815[#38815] (issue: {es-issue}30301[#30301]) * Correct authenticate response for API key {es-pull}39684[#39684] * Fix security index auto-create and state recovery race {es-pull}39582[#39582] Build:: * Use explicit deps on test tasks for check {es-pull}36325[#36325] * Fix jdbc jar pom to not include deps {es-pull}36036[#36036] (issue: {es-issue}32014[#32014]) * Fix official plugins list {es-pull}35661[#35661] (issue: {es-issue}35623[#35623]) CCR:: * Fix follow stats API's follower index filtering feature {es-pull}36647[#36647] * AutoFollowCoordinator should tolerate that auto follow patterns may be removed {es-pull}35945[#35945] (issue: {es-issue}35937[#35937]) * Only auto follow indices when all primary shards have started {es-pull}35814[#35814] (issue: {es-issue}35480[#35480]) * Avoid NPE in follower stats when no tasks metadata {es-pull}35802[#35802] * Fix the names of CCR stats endpoints in usage API {es-pull}35438[#35438] * Prevent CCR recovery from missing documents {es-pull}38237[#38237] * Fix file reading in ccr restore service {es-pull}38117[#38117] * Correct argument names in update mapping/settings from leader {es-pull}38063[#38063] * Ensure changes requests return the latest mapping version {es-pull}37633[#37633] * Do not set fatal exception when shard follow task is stopped. {es-pull}37603[#37603] * Add fatal_exception field for ccr stats in monitoring mapping {es-pull}37563[#37563] * Do not add index event listener if CCR disabled {es-pull}37432[#37432] * When removing an AutoFollower also mark it as removed. {es-pull}37402[#37402] (issue: {es-issue}36761[#36761]) * Make shard follow tasks more resilient for restarts {es-pull}37239[#37239] (issue: {es-issue}37231[#37231]) * Resume follow Api should not require a request body {es-pull}37217[#37217] (issue: {es-issue}37022[#37022]) * Report error if auto follower tries auto follow a leader index with soft deletes disabled {es-pull}36886[#36886] (issue: {es-issue}33007[#33007]) * Remote cluster license checker and no license info. {es-pull}36837[#36837] (issue: {es-issue}36815[#36815]) * Make CCR resilient against missing remote cluster connections {es-pull}36682[#36682] (issues: {es-issue}36255[#36255], {es-issue}36667[#36667]) * AutoFollowCoordinator and follower index already created {es-pull}36540[#36540] (issue: {es-issue}33007[#33007]) * Safe publication of AutoFollowCoordinator {es-pull}40153[#40153] (issue: {es-issue}38560[#38560]) * Enable reading auto-follow patterns from x-content {es-pull}40130[#40130] (issue: {es-issue}40128[#40128]) * Stop auto-followers on shutdown {es-pull}40124[#40124] * Protect against the leader index being removed {es-pull}39351[#39351] (issue: {es-issue}39308[#39308]) * Handle the fact that `ShardStats` instance may have no commit or seqno stats {es-pull}38782[#38782] (issue: {es-issue}38779[#38779]) * Fix LocalIndexFollowingIT#testRemoveRemoteConnection() test {es-pull}38709[#38709] (issue: {es-issue}38695[#38695]) * Fix shard follow task startup error handling {es-pull}39053[#39053] (issue: {es-issue}38779[#38779]) * Filter out upgraded version index settings when starting index following {es-pull}38838[#38838] (issue: {es-issue}38835[#38835]) Circuit Breakers:: * Modify `BigArrays` to take name of circuit breaker {es-pull}36461[#36461] (issue: {es-issue}31435[#31435]) Core:: * Fix CompositeBytesReference#slice to not throw AIOOBE with legal offsets. {es-pull}35955[#35955] (issue: {es-issue}35950[#35950]) * Suppress CachedTimeThread in hot threads output {es-pull}35558[#35558] (issue: {es-issue}23175[#23175]) * Upgrade to Joda 2.10.1 {es-pull}35410[#35410] (issue: {es-issue}33749[#33749]) CRUD:: * Fix Reindex from remote query logic {es-pull}36908[#36908] * Synchronize WriteReplicaResult callbacks {es-pull}36770[#36770] * Cascading primary failure lead to MSU too low {es-pull}40249[#40249] * Store Pending Deletions Fix {es-pull}40345[#40345] (issue: {es-issue}40249[#40249]) * ShardBulkAction ignore primary response on primary {es-pull}38901[#38901] Cluster Coordination:: * Fix node tool cleanup {es-pull}39389[#39389] * Avoid serialising state if it was already serialised {es-pull}39179[#39179] * Do not perform cleanup if Manifest write fails with dirty exception {es-pull}40519[#40519] (issue: {es-issue}39077[#39077]) * Cache compressed cluster state size {es-pull}39827[#39827] (issue: {es-issue}39806[#39806]) * Drop node if asymmetrically partitioned from master {es-pull}39598[#39598] * Fixing the custom object serialization bug in diffable utils. {es-pull}39544[#39544] * Clean GatewayAllocator when stepping down as master {es-pull}38885[#38885] Distributed:: * Combine the execution of an exclusive replica operation with primary term update {es-pull}36116[#36116] (issue: {es-issue}35850[#35850]) * ActiveShardCount should not fail when closing the index {es-pull}35936[#35936] * TransportVerifyShardBeforeCloseAction should force a flush {es-pull}38401[#38401] (issues: {es-issue}33888[#33888], {es-issue}37961[#37961]) * Fix limit on retaining sequence number {es-pull}37992[#37992] (issue: {es-issue}37165[#37165]) * Close Index API should force a flush if a sync is needed {es-pull}37961[#37961] (issues: {es-issue}33888[#33888], {es-issue}37426[#37426]) * Force Refresh Listeners when Acquiring all Operation Permits {es-pull}36835[#36835] * Replaced the word 'shards' with 'replicas' in an error message. (#36234) {es-pull}36275[#36275] (issue: {es-issue}36234[#36234]) * Ignore waitForActiveShards when syncing leases {es-pull}39224[#39224] (issue: {es-issue}39089[#39089]) * Fix synchronization in LocalCheckpointTracker#contains {es-pull}38755[#38755] (issues: {es-issue}33871[#33871], {es-issue}38633[#38633]) * Enforce retention leases require soft deletes {es-pull}39922[#39922] (issue: {es-issue}39914[#39914]) * Treat TransportService stopped error as node is closing {es-pull}39800[#39800] (issue: {es-issue}39584[#39584]) * Use cause to determine if node with primary is closing {es-pull}39723[#39723] (issue: {es-issue}39584[#39584]) * Don’t ack if unable to remove failing replica {es-pull}39584[#39584] (issue: {es-issue}39467[#39467]) * Fix NPE on Stale Index in IndicesService {es-pull}38891[#38891] (issue: {es-issue}38845[#38845]) Engine:: * Set Lucene version upon index creation. {es-pull}36038[#36038] (issue: {es-issue}33826[#33826]) * Wrap can_match reader with ElasticsearchDirectoryReader {es-pull}35857[#35857] * Copy checkpoint atomically when rolling generation {es-pull}35407[#35407] * Subclass NIOFSDirectory instead of using FileSwitchDirectory {es-pull}37140[#37140] (issues: {es-issue}36668[#36668], {es-issue}37111[#37111]) * Bubble up exception when processing NoOp {es-pull}39338[#39338] (issue: {es-issue}38898[#38898]) * ReadOnlyEngine should update translog recovery state information {es-pull}39238[#39238] * Advance max_seq_no before add operation to Lucene {es-pull}38879[#38879] (issue: {es-issue}31629[#31629]) Features/Features:: * Only count some fields types for deprecation check {es-pull}40166[#40166] * Deprecation check for indices with very large numbers of fields {es-pull}39869[#39869] (issue: {es-issue}39851[#39851]) Features/ILM:: * Preserve ILM operation mode when creating new lifecycles {es-pull}38134[#38134] (issues: {es-issue}38229[#38229], {es-issue}38230[#38230]) * Retry ILM steps that fail due to SnapshotInProgressException {es-pull}37624[#37624] (issues: {es-issue}37541[#37541], {es-issue}37552[#37552]) * Remove `indexing_complete` when removing policy {es-pull}36620[#36620] * Handle failure to release retention leases in ILM {es-pull}39281[#39281] (issue: {es-issue}39181[#39181]) * Correct ILM metadata minimum compatibility version {es-pull}40569[#40569] (issue: {es-issue}40565[#40565]) * Handle null retention leases in WaitForNoFollowersStep {es-pull}40477[#40477] * Allow ILM to stop if indices have nonexistent policies {es-pull}40820[#40820] (issue: {es-issue}40824[#40824]) Features/Indices APIs:: * Validate top-level keys for create index request (#23755) {es-pull}23869[#23869] (issue: {es-issue}23755[#23755]) * Reject delete index requests with a body {es-pull}37501[#37501] (issue: {es-issue}8217[#8217]) * Fix duplicate phrase in shrink/split error message {es-pull}36734[#36734] (issue: {es-issue}36729[#36729]) * Get Aliases with wildcard exclusion expression {es-pull}34230[#34230] (issues: {es-issue}33518[#33518], {es-issue}33805[#33805], {es-issue}34144[#34144]) Features/Ingest:: * Fix Deprecation Warning in Script Proc. {es-pull}32407[#32407] * Support unknown fields in ingest pipeline map configuration {es-pull}38352[#38352] (issue: {es-issue}36938[#36938]) * Ingest node - user_agent, move device parsing to an object {es-pull}38115[#38115] (issues: {es-issue}37329[#37329], {es-issue}38094[#38094]) * Fix on_failure with Drop processor {es-pull}36686[#36686] (issue: {es-issue}36151[#36151]) * Support default pipelines + bulk upserts {es-pull}36618[#36618] (issue: {es-issue}36219[#36219]) * Ingest ingest then create index {es-pull}39607[#39607] (issues: {es-issue}32758[#32758], {es-issue}32786[#32786], {es-issue}36545[#36545]) Features/Java High Level REST Client:: * Drop extra level from user parser {es-pull}34932[#34932] * Update IndexTemplateMetadata to allow unknown fields {es-pull}38448[#38448] (issue: {es-issue}36938[#36938]) * `if_seq_no` and `if_primary_term` parameters aren't wired correctly in REST Client's CRUD API {es-pull}38411[#38411] * Update Rollup Caps to allow unknown fields {es-pull}38339[#38339] (issue: {es-issue}36938[#36938]) * Fix ILM explain response to allow unknown fields {es-pull}38054[#38054] (issue: {es-issue}36938[#36938]) * Fix ILM status to allow unknown fields {es-pull}38043[#38043] (issue: {es-issue}36938[#36938]) * Fix ILM Lifecycle Policy to allow unknown fields {es-pull}38041[#38041] (issue: {es-issue}36938[#36938]) * Update authenticate to allow unknown fields {es-pull}37713[#37713] (issue: {es-issue}36938[#36938]) * Update verify repository to allow unknown fields {es-pull}37619[#37619] (issue: {es-issue}36938[#36938]) * Update get users to allow unknown fields {es-pull}37593[#37593] (issue: {es-issue}36938[#36938]) * Update Execute Watch to allow unknown fields {es-pull}37498[#37498] (issue: {es-issue}36938[#36938]) * Update Put Watch to allow unknown fields {es-pull}37494[#37494] (issue: {es-issue}36938[#36938]) * Update Delete Watch to allow unknown fields {es-pull}37435[#37435] (issue: {es-issue}36938[#36938]) * Fix rest reindex test for IPv4 addresses {es-pull}37310[#37310] * Fix weighted_avg parser not found for RestHighLevelClient {es-pull}37027[#37027] (issue: {es-issue}36861[#36861]) Features/Java Low Level REST Client:: * Remove I/O pool blocking sniffing call from onFailure callback, add some logic around host exclusion {es-pull}27985[#27985] (issue: {es-issue}27984[#27984]) * Fix potential IllegalCapacityException in LLRC when selecting nodes {es-pull}37821[#37821] Features/Monitoring:: * Allow built-in monitoring_user role to call GET _xpack API {es-pull}38060[#38060] (issue: {es-issue}37970[#37970]) * Don't emit deprecation warnings on calls to the monitoring bulk API. {es-pull}39805[#39805] (issue: {es-issue}39336[#39336]) Features/Watcher:: * Ignore system locale/timezone in croneval CLI tool {es-pull}33215[#33215] * Support merge nested Map in list for JIRA configurations {es-pull}37634[#37634] (issue: {es-issue}30068[#30068]) * Watcher accounts constructed lazily {es-pull}36656[#36656] * Ensures watch definitions are valid json {es-pull}30692[#30692] (issue: {es-issue}29746[#29746]) * Use non-ILM template setting up watch history template & ILM disabled {es-pull}39325[#39325] (issue: {es-issue}38805[#38805]) * Only flush Watcher's bulk processor if Watcher is enabled {es-pull}38803[#38803] (issue: {es-issue}38798[#38798]) * Fix Watcher stats class cast exception {es-pull}39821[#39821] (issue: {es-issue}39780[#39780]) * Use any index specified by .watches for Watcher {es-pull}39541[#39541] (issue: {es-issue}39478[#39478]) * Resolve concurrency with watcher trigger service {es-pull}39092[#39092] (issue: {es-issue}39087[#39087]) * Metric on watcher stats is a list not an enum {es-pull}39114[#39114] Geo:: * Test `GeoShapeQueryTests#testPointsOnly` fails {es-pull}27454[#27454] * More robust handling of ignore_malformed in geoshape parsing {es-pull}35603[#35603] (issues: {es-issue}34047[#34047], {es-issue}34498[#34498]) * Better handling of malformed geo_points {es-pull}35554[#35554] (issue: {es-issue}35419[#35419]) * Enables coerce support in WKT polygon parser {es-pull}35414[#35414] (issue: {es-issue}35059[#35059]) * Fix GeoHash PrefixTree BWC {es-pull}38584[#38584] (issue: {es-issue}38494[#38494]) * Do not normalize the longitude with value -180 for Lucene shapes {es-pull}37299[#37299] (issue: {es-issue}37297[#37297]) * Geo Point parse error fix {es-pull}40447[#40447] (issue: {es-issue}17617[#17617]) Highlighting:: * Bug fix for AnnotatedTextHighlighter - port of 39525 {es-pull}39750[#39750] (issue: {es-issue}39525[#39525]) Infra/Core:: * Ensure shard is refreshed once it's inactive {es-pull}27559[#27559] (issue: {es-issue}27500[#27500]) * Bubble-up exceptions from scheduler {es-pull}38317[#38317] (issue: {es-issue}38014[#38014]) * Revert back to joda's multi date formatters {es-pull}36814[#36814] (issues: {es-issue}36447[#36447], {es-issue}36602[#36602]) * Propagate Errors in executors to uncaught exception handler {es-pull}36137[#36137] (issue: {es-issue}28667[#28667]) * Correct name of basic_date_time_no_millis {es-pull}39367[#39367] * Allow single digit milliseconds in strict date parsing {es-pull}40676[#40676] (issue: {es-issue}40403[#40403]) * Parse composite patterns using ClassicFormat.parseObject {es-pull}40100[#40100] (issue: {es-issue}39916[#39916]) * Bat scripts to work with JAVA_HOME with parantheses {es-pull}39712[#39712] (issues: {es-issue}30606[#30606], {es-issue}33405[#33405], {es-issue}38578[#38578], {es-issue}38624[#38624]) * Change licence expiration date pattern {es-pull}39681[#39681] (issue: {es-issue}39136[#39136]) * Fix DateFormatters.parseMillis when no timezone is given {es-pull}39100[#39100] (issue: {es-issue}39067[#39067]) * Don't close caches while there might still be in-flight requests. {es-pull}38958[#38958] (issue: {es-issue}37117[#37117]) * Allow single digit milliseconds in strict date parsing {es-pull}40676[#40676] (issue: {es-issue}40403[#40403]) Infra/Packaging:: * Remove NOREPLACE for /etc/elasticsearch in rpm and deb {es-pull}37839[#37839] * Packaging: Update marker used to allow ELASTIC_PASSWORD {es-pull}37243[#37243] (issue: {es-issue}37240[#37240]) * Remove permission editing in postinst {es-pull}37242[#37242] (issue: {es-issue}37143[#37143]) * Some elasticsearch-cli tools could not be run not from ES_HOME {es-pull}39937[#39937] * Obsolete pre 7.0 noarch package in rpm {es-pull}39472[#39472] (issue: {es-issue}39414[#39414]) * Suppress error message when `/proc/sys/vm/max_map_count` is not exists. {es-pull}35933[#35933] * Use TAR instead of DOCKER build type before 6.7.0 {es-pull}40723[#40723] (issues: {es-issue}39378[#39378], {es-issue}40511[#40511]) * Source additional files correctly in elasticsearch-cli {es-pull}40890[#40890] (issue: {es-issue}40889[#40889]) Infra/REST API:: * Reject all requests that have an unconsumed body {es-pull}37504[#37504] (issues: {es-issue}30792[#30792], {es-issue}37501[#37501], {es-issue}8217[#8217]) * Fix #38623 remove xpack namespace REST API {es-pull}38625[#38625] * Remove the "xpack" namespace from the REST API {es-pull}38623[#38623] * Update spec files that erroneously documented parts as optional {es-pull}39122[#39122] * ilm.explain_lifecycle documents human again {es-pull}39113[#39113] * Index on rollup.rollup_search.json is a list {es-pull}39097[#39097] Infra/Scripting:: * Fix Painless void return bug {es-pull}38046[#38046] * Correct bug in ScriptDocValues {es-pull}40488[#40488] Infra/Settings:: * Change format how settings represent lists / array {es-pull}26723[#26723] * Fix setting by time unit {es-pull}37192[#37192] * Fix handling of fractional byte size value settings {es-pull}37172[#37172] * Fix handling of fractional time value settings {es-pull}37171[#37171] Infra/Transport API:: * Remove version read/write logic in Verify Response {es-pull}30879[#30879] (issue: {es-issue}30807[#30807]) * Enable muted Repository test {es-pull}30875[#30875] (issue: {es-issue}30807[#30807]) * Bad regex in CORS settings should throw a nicer error {es-pull}29108[#29108] Index APIs:: * Fix duplicate phrase in shrink/split error message {es-pull}36734[#36734] (issue: {es-issue}36729[#36729]) * Raise a 404 exception when document source is not found (#33384) {es-pull}34083[#34083] (issue: {es-issue}33384[#33384]) Ingest:: * Fix on_failure with Drop processor {es-pull}36686[#36686] (issue: {es-issue}36151[#36151]) * Support default pipelines + bulk upserts {es-pull}36618[#36618] (issue: {es-issue}36219[#36219]) * Support default pipeline through an alias {es-pull}36231[#36231] (issue: {es-issue}35817[#35817]) License:: * Update versions for start_trial after backport {es-pull}30218[#30218] (issue: {es-issue}30135[#30135]) * Do not serialize basic license exp in x-pack info {es-pull}30848[#30848] * Update versions for start_trial after backport {es-pull}30218[#30218] (issue: {es-issue}30135[#30135]) Machine Learning:: * Interrupt Grok in file structure finder timeout {es-pull}36588[#36588] * Prevent stack overflow while copying ML jobs and datafeeds {es-pull}36370[#36370] (issue: {es-issue}36360[#36360]) * Adjust file structure finder parser config {es-pull}35935[#35935] * Fix find_file_structure NPE with should_trim_fields {es-pull}35465[#35465] (issue: {es-issue}35462[#35462]) * Prevent notifications being created on deletion of a non existent job {es-pull}35337[#35337] (issues: {es-issue}34058[#34058], {es-issue}35336[#35336]) * Clear Job#finished_time when it is opened (#32605) {es-pull}32755[#32755] * Fix thread leak when waiting for job flush (#32196) {es-pull}32541[#32541] (issue: {es-issue}32196[#32196]) * Fix CPoissonMeanConjugate sampling error. {ml-pull}335[#335] * Report index unavailable instead of waiting for lazy node {es-pull}38423[#38423] * Fix error race condition on stop _all datafeeds and close _all jobs {es-pull}38113[#38113] (issue: {es-issue}37959[#37959]) * Update ML results mappings on process start {es-pull}37706[#37706] (issue: {es-issue}37607[#37607]) * Prevent submit after autodetect worker is stopped {es-pull}37700[#37700] (issue: {es-issue}37108[#37108]) * Fix ML datafeed CCS with wildcarded cluster name {es-pull}37470[#37470] (issue: {es-issue}36228[#36228]) * Update error message for process update {es-pull}37363[#37363] * Wait for autodetect to be ready in the datafeed {es-pull}37349[#37349] (issues: {es-issue}36810[#36810], {es-issue}37227[#37227]) * Stop datafeeds running when their jobs are stale {es-pull}37227[#37227] (issue: {es-issue}36810[#36810]) * Order GET job stats response by job id {es-pull}36841[#36841] (issue: {es-issue}36683[#36683]) * Make GetJobStats work with arbitrary wildcards and groups {es-pull}36683[#36683] (issue: {es-issue}34745[#34745]) * Fix datafeed skipping first bucket after lookback when aggs are … {es-pull}39859[#39859] (issue: {es-issue}39842[#39842]) * Refactoring lazy query and agg parsing {es-pull}39776[#39776] (issue: {es-issue}39528[#39528]) * Stop the ML memory tracker before closing node {es-pull}39111[#39111] (issue: {es-issue}37117[#37117]) * Scrolling datafeed should clear scroll contexts on error {es-pull}40773[#40773] (issue: {es-issue}40772[#40772]) Mapping:: * Ensure that field aliases cannot be used in multi-fields. {es-pull}32219[#32219] * Treat put-mapping calls with `_doc` as a top-level key as typed calls. {es-pull}38032[#38032] * Correct deprec log in RestGetFieldMappingAction {es-pull}37843[#37843] (issue: {es-issue}37667[#37667]) * Restore a noop _all metadata field for 6x indices {es-pull}37808[#37808] (issue: {es-issue}37429[#37429]) * Make sure PutMappingRequest accepts content types other than JSON. {es-pull}37720[#37720] * Make sure to use the resolved type in DocumentMapperService#extractMappings. {es-pull}37451[#37451] (issue: {es-issue}36811[#36811]) * Improve Precision for scaled_float {es-pull}37169[#37169] (issue: {es-issue}32570[#32570]) * Make sure to accept empty unnested mappings in create index requests. {es-pull}37089[#37089] * Stop automatically nesting mappings in index creation requests. {es-pull}36924[#36924] * Rewrite SourceToParse with resolved docType {es-pull}36921[#36921] (issues: {es-issue}35790[#35790], {es-issue}36769[#36769]) * Optimise rejection of out-of-range `long` values {es-pull}40325[#40325] (issues: {es-issue}26137[#26137], {es-issue}40323[#40323]) * Make sure to reject mappings with type _doc when include_type_name is false. {es-pull}38270[#38270] (issue: {es-issue}38266[#38266]) Network:: * Adjust SSLDriver behavior for JDK11 changes {es-pull}32145[#32145] (issues: {es-issue}32122[#32122], {es-issue}32144[#32144]) * Netty4SizeHeaderFrameDecoder error {es-pull}31057[#31057] * Fix memory leak in http pipelining {es-pull}30815[#30815] (issue: {es-issue}30801[#30801]) * Fix issue with finishing handshake in ssl driver {es-pull}30580[#30580] * Do not resolve addresses in remote connection info {es-pull}36671[#36671] (issue: {es-issue}35658[#35658]) * Always compress based on the settings {es-pull}36522[#36522] (issue: {es-issue}36399[#36399]) * http.publish_host Should Contain CNAME {es-pull}32806[#32806] (issue: {es-issue}22029[#22029]) * Adjust SSLDriver behavior for JDK11 changes {es-pull}32145[#32145] (issues: {es-issue}32122[#32122], {es-issue}32144[#32144]) * Add TRACE, CONNECT, and PATCH http methods {es-pull}31035[#31035] (issue: {es-issue}31017[#31017]) * Transport client: Don't validate node in handshake {es-pull}30737[#30737] (issue: {es-issue}30141[#30141]) * Fix issue with finishing handshake in ssl driver {es-pull}30580[#30580] * Remove potential nio selector leak {es-pull}27825[#27825] * Fix issue where the incorrect buffers are written {es-pull}27695[#27695] (issue: {es-issue}27551[#27551]) * Do not set SO_LINGER on server channels {es-pull}26997[#26997] * Do not set SO_LINGER to 0 when not shutting down {es-pull}26871[#26871] (issue: {es-issue}26764[#26764]) * Release pipelined http responses on close {es-pull}26226[#26226] * Reload SSL context on file change for LDAP {es-pull}36937[#36937] (issues: {es-issue}30509[#30509], {es-issue}36923[#36923]) * Do not resolve addresses in remote connection info {es-pull}36671[#36671] (issue: {es-issue}35658[#35658]) Packaging:: * Fix error message when package install fails due to missing Java {es-pull}36077[#36077] (issue: {es-issue}31845[#31845]) * Add missing entries to conffiles {es-pull}35810[#35810] (issue: {es-issue}35691[#35691]) Plugins:: * Ensure that azure stream has socket privileges {es-pull}28751[#28751] (issue: {es-issue}28662[#28662]) Ranking:: * QueryRescorer should keep the window size when rewriting {es-pull}36836[#36836] Recovery:: * Register ResyncTask.Status as a NamedWriteable {es-pull}36610[#36610] * RecoveryMonitor#lastSeenAccessTime should be volatile {es-pull}36781[#36781] * Create retention leases file during recovery {es-pull}39359[#39359] (issue: {es-issue}37165[#37165]) * Recover peers from translog, ignoring soft deletes {es-pull}38904[#38904] (issue: {es-issue}37165[#37165]) * Retain history for peer recovery using leases {es-pull}38855[#38855] * Resync should not send operations without sequence number {es-pull}40433[#40433] Rollup:: * Fix rollup search statistics {es-pull}36674[#36674] * Fix Rollup's metadata parser {es-pull}36791[#36791] (issue: {es-issue}36726[#36726]) * Fix rollup search statistics {es-pull}36674[#36674] * Remove timezone validation on rollup range queries {es-pull}40647[#40647] * Rollup ignores time_zone on date histogram {es-pull}40844[#40844] Scripting:: * Properly support no-offset date formatting {es-pull}36316[#36316] (issue: {es-issue}36306[#36306]) * [Painless] Generate Bridge Methods {es-pull}36097[#36097] * Fix serialization bug in painless execute api request {es-pull}36075[#36075] (issue: {es-issue}36050[#36050]) * Actually add joda time back to whitelist {es-pull}35965[#35965] (issue: {es-issue}35915[#35915]) * Add back joda to whitelist {es-pull}35915[#35915] (issue: {es-issue}35913[#35913]) Settings:: * Correctly Identify Noop Updates {es-pull}36560[#36560] (issue: {es-issue}36496[#36496]) Search:: * Ensure realtime `_get` and `_termvectors` don't run on the network thread {es-pull}33814[#33814] (issue: {es-issue}27500[#27500]) * [bug] fuzziness custom auto {es-pull}33462[#33462] (issue: {es-issue}33454[#33454]) * Fix inner hits retrieval when stored fields are disabled (_none_) {es-pull}33018[#33018] (issue: {es-issue}32941[#32941]) * Set maxScore for empty TopDocs to Nan rather than 0 {es-pull}32938[#32938] * Handle leniency for cross_fields type in multi_match query {es-pull}27045[#27045] (issue: {es-issue}23210[#23210]) * Raise IllegalArgumentException instead if query validation failed {es-pull}26811[#26811] (issue: {es-issue}26799[#26799]) * Inner hits fail to propagate doc-value format. {es-pull}36310[#36310] * Fix custom AUTO issue with Fuzziness#toXContent {es-pull}35807[#35807] (issue: {es-issue}33462[#33462]) * Fix analyzed prefix query in query_string {es-pull}35756[#35756] (issue: {es-issue}31702[#31702]) * Fix problem with MatchNoDocsQuery in disjunction queries {es-pull}35726[#35726] (issue: {es-issue}34708[#34708]) * Fix phrase_slop in query_string query {es-pull}35533[#35533] (issue: {es-issue}35125[#35125]) * Add a More Like This query routing requirement check (#29678) {es-pull}33974[#33974] * Look up connection using the right cluster alias when releasing contexts {es-pull}38570[#38570] * Fix fetch source option in expand search phase {es-pull}37908[#37908] (issue: {es-issue}23829[#23829]) * Change `rational` to `saturation` in script_score {es-pull}37766[#37766] (issue: {es-issue}37714[#37714]) * Throw if two inner_hits have the same name {es-pull}37645[#37645] (issue: {es-issue}37584[#37584]) * Ensure either success or failure path for SearchOperationListener is called {es-pull}37467[#37467] (issue: {es-issue}37185[#37185]) * `query_string` should use indexed prefixes {es-pull}36895[#36895] * Avoid duplicate types deprecation messages in search-related APIs. {es-pull}36802[#36802] * Serialize top-level pipeline aggs as part of InternalAggregations {es-pull}40177[#40177] (issues: {es-issue}40059[#40059], {es-issue}40101[#40101]) * CCS: Skip empty search hits when minimizing round-trips {es-pull}40098[#40098] (issues: {es-issue}32125[#32125], {es-issue}40067[#40067]) * CCS: Disable minimizing round-trips when dfs is requested {es-pull}40044[#40044] (issue: {es-issue}32125[#32125]) * Fix Fuzziness#asDistance(String) {es-pull}39643[#39643] (issue: {es-issue}39614[#39614]) * Fix alias resolution runtime complexity. {es-pull}40263[#40263] (issue: {es-issue}40248[#40248]) Security:: * Handle 6.4.0+ BWC for Application Privileges {es-pull}32929[#32929] * Remove license state listeners on closeables {es-pull}36308[#36308] (issues: {es-issue}33328[#33328], {es-issue}35627[#35627], {es-issue}35628[#35628]) * Fix exit code for Security CLI tools {es-pull}37956[#37956] (issue: {es-issue}37841[#37841]) * Fix potential NPE in UsersTool {es-pull}37660[#37660] * Remove dynamic objects from security index {es-pull}40499[#40499] (issue: {es-issue}35460[#35460]) * Fix libs:ssl-config project setup {es-pull}39074[#39074] * Do not create the missing index when invoking getRole {es-pull}39039[#39039] Snapshot/Restore:: * Upgrade GCS Dependencies to 1.55.0 {es-pull}36634[#36634] (issues: {es-issue}35229[#35229], {es-issue}35459[#35459]) * Improve Resilience SnapshotShardService {es-pull}36113[#36113] (issue: {es-issue}32265[#32265]) * Keep SnapshotsInProgress State in Sync with Routing Table {es-pull}35710[#35710] * Ensure that gcs client creation is privileged {es-pull}25938[#25938] (issue: {es-issue}25932[#25932]) * Make calls to CloudBlobContainer#exists privileged {es-pull}25937[#25937] (issue: {es-issue}25931[#25931]) * Fix Concurrent Snapshot Ending And Stabilize Snapshot Finalization {es-pull}38368[#38368] (issue: {es-issue}38226[#38226]) * Fix Two Races that Lead to Stuck Snapshots {es-pull}37686[#37686] (issues: {es-issue}32265[#32265], {es-issue}32348[#32348]) * Fix Race in Concurrent Snapshot Delete and Create {es-pull}37612[#37612] (issue: {es-issue}37581[#37581]) * Streamline S3 Repository- and Client-Settings {es-pull}37393[#37393] * Blob store compression fix {es-pull}39073[#39073] SQL:: * Fix translation of LIKE/RLIKE keywords {es-pull}36672[#36672] (issues: {es-issue}36039[#36039], {es-issue}36584[#36584]) * Scripting support for casting functions CAST and CONVERT {es-pull}36640[#36640] (issue: {es-issue}36061[#36061]) * Fix translation to painless for conditionals {es-pull}36636[#36636] (issue: {es-issue}36631[#36631]) * Concat should be always not nullable {es-pull}36601[#36601] (issue: {es-issue}36169[#36169]) * Fix MOD() for long and integer arguments {es-pull}36599[#36599] (issue: {es-issue}36364[#36364]) * Fix issue with complex HAVING and GROUP BY ordinal {es-pull}36594[#36594] (issue: {es-issue}36059[#36059]) * Be lenient for tests involving comparison to H2 but strict for csv spec tests {es-pull}36498[#36498] (issue: {es-issue}36483[#36483]) * Non ISO 8601 versions of DAY_OF_WEEK and WEEK_OF_YEAR functions {es-pull}36358[#36358] (issue: {es-issue}36263[#36263]) * Do not ignore all fields whose names start with underscore {es-pull}36214[#36214] (issue: {es-issue}36206[#36206]) * Fix issue with wrong data type for scripted Grouping keys {es-pull}35969[#35969] (issue: {es-issue}35662[#35662]) * Fix translation of math functions to painless {es-pull}35910[#35910] (issue: {es-issue}35654[#35654]) * Fix jdbc jar to include deps {es-pull}35602[#35602] * Fix query translation for scripted queries {es-pull}35408[#35408] (issue: {es-issue}35232[#35232]) * Clear the cursor if nested inner hits are enough to fulfill the query required limits {es-pull}35398[#35398] (issue: {es-issue}35176[#35176]) * Introduce IsNull node to simplify expressions {es-pull}35206[#35206] (issues: {es-issue}34876[#34876], {es-issue}35171[#35171]) * The SSL default configuration shouldn't override the https protocol if used {es-pull}34635[#34635] (issue: {es-issue}33817[#33817]) * Minor fix for javadoc {es-pull}32573[#32573] (issue: {es-issue}32553[#32553]) * Prevent grouping over grouping functions {es-pull}38649[#38649] (issue: {es-issue}38308[#38308]) * Relax StackOverflow circuit breaker for constants {es-pull}38572[#38572] (issue: {es-issue}38571[#38571]) * Fix issue with IN not resolving to underlying keyword field {es-pull}38440[#38440] (issue: {es-issue}38424[#38424]) * Change the Intervals milliseconds precision to 3 digits {es-pull}38297[#38297] (issue: {es-issue}37423[#37423]) * Fix esType for DATETIME/DATE and INTERVALS {es-pull}38179[#38179] (issue: {es-issue}38051[#38051]) * Added SSL configuration options tests {es-pull}37875[#37875] (issue: {es-issue}37711[#37711]) * Fix casting from date to numeric type to use millis {es-pull}37869[#37869] (issue: {es-issue}37655[#37655]) * Fix BasicFormatter NPE {es-pull}37804[#37804] * Return Intervals in SQL format for CLI {es-pull}37602[#37602] (issues: {es-issue}29970[#29970], {es-issue}36186[#36186], {es-issue}36432[#36432]) * Fix object extraction from sources {es-pull}37502[#37502] (issue: {es-issue}37364[#37364]) * Fix issue with field names containing "." {es-pull}37364[#37364] (issue: {es-issue}37128[#37128]) * Fix bug regarding alias fields with dots {es-pull}37279[#37279] (issue: {es-issue}37224[#37224]) * Proper handling of COUNT(field_name) and COUNT(DISTINCT field_name) {es-pull}37254[#37254] (issue: {es-issue}30285[#30285]) * Fix COUNT DISTINCT filtering {es-pull}37176[#37176] (issue: {es-issue}37086[#37086]) * Fix issue with wrong NULL optimization {es-pull}37124[#37124] (issue: {es-issue}35872[#35872]) * Fix issue with complex expression as args of PERCENTILE/_RANK {es-pull}37102[#37102] (issue: {es-issue}37099[#37099]) * Handle the bwc Joda ZonedDateTime scripting class in Painless {es-pull}37024[#37024] (issue: {es-issue}37023[#37023]) * Fix bug regarding histograms usage in scripting {es-pull}36866[#36866] * Fix issue with always false filter involving functions {es-pull}36830[#36830] (issue: {es-issue}35980[#35980]) * Protocol returns ISO 8601 String formatted dates instead of Long for JDBC/ODBC requests {es-pull}36800[#36800] (issue: {es-issue}36756[#36756]) * Enhance Verifier to prevent aggregate or grouping functions from {es-pull}36799[#36799] (issue: {es-issue}36798[#36798]) * Fix translation of LIKE/RLIKE keywords {es-pull}36672[#36672] (issues: {es-issue}36039[#36039], {es-issue}36584[#36584]) * Scripting support for casting functions CAST and CONVERT {es-pull}36640[#36640] (issue: {es-issue}36061[#36061]) * Concat should be always not nullable {es-pull}36601[#36601] (issue: {es-issue}36169[#36169]) * Fix issue with complex HAVING and GROUP BY ordinal {es-pull}36594[#36594] (issue: {es-issue}36059[#36059]) * Add missing handling of IP field in JDBC {es-pull}40384[#40384] (issue: {es-issue}40358[#40358]) * Fix metric aggs on date/time to not return double {es-pull}40377[#40377] (issues: {es-issue}39492[#39492], {es-issue}40376[#40376]) * CAST supports both SQL and ES types {es-pull}40365[#40365] (issue: {es-issue}40282[#40282]) * Fix RLIKE bug and improve testing for RLIKE statement {es-pull}40354[#40354] (issues: {es-issue}34609[#34609], {es-issue}39931[#39931]) * Unwrap the first value in an array in case of array leniency {es-pull}40318[#40318] (issue: {es-issue}40296[#40296]) * Preserve original source for cast/convert function {es-pull}40271[#40271] (issue: {es-issue}40239[#40239]) * Fix LIKE function equality by considering its pattern as well {es-pull}40260[#40260] (issue: {es-issue}39931[#39931]) * Fix issue with optimization on queries with ORDER BY/LIMIT {es-pull}40256[#40256] (issue: {es-issue}40211[#40211]) * Rewrite ROUND and TRUNCATE functions with a different optional parameter handling method {es-pull}40242[#40242] (issue: {es-issue}40001[#40001]) * Fix issue with getting DATE type in JDBC {es-pull}40207[#40207] * Fix issue with date columns returned always in UTC {es-pull}40163[#40163] (issue: {es-issue}40152[#40152]) * Add multi_value_field_leniency inside FieldHitExtractor {es-pull}40113[#40113] (issue: {es-issue}39700[#39700]) * Fix incorrect ordering of groupings (GROUP BY) based on orderings (ORDER BY) {es-pull}40087[#40087] (issue: {es-issue}39956[#39956]) * Fix bug with JDBC timezone setting and DATE type {es-pull}39978[#39978] (issue: {es-issue}39915[#39915]) * Use underlying exact field for LIKE/RLIKE {es-pull}39443[#39443] (issue: {es-issue}39442[#39442]) * Fix display size for DATE/DATETIME {es-pull}40669[#40669] * Have LIKE/RLIKE use wildcard and regexp queries {es-pull}40628[#40628] (issue: {es-issue}40557[#40557]) * Fix getTime() methods in JDBC {es-pull}40484[#40484] * SYS TABLES: enumerate tables of requested types {es-pull}40535[#40535] (issue: {es-issue}40348[#40348]) * Passing an input to the CLI "freezes" the CLI after displaying an error message {es-pull}40164[#40164] (issue: {es-issue}40557[#40557]) * Wrap ZonedDateTime parameters inside scripts {es-pull}39911[#39911] (issue: {es-issue}39877[#39877]) * ConstantProcessor can now handle NamedWriteable {es-pull}39876[#39876] (issue: {es-issue}39875[#39875]) * Extend the multi dot field notation extraction to lists of values {es-pull}39823[#39823] (issue: {es-issue}39738[#39738]) * Values in datetime script aggs should be treated as long {es-pull}39773[#39773] (issue: {es-issue}37042[#37042]) * Don't allow inexact fields for MIN/MAX {es-pull}39563[#39563] (issue: {es-issue}39427[#39427]) * Fix merging of incompatible multi-fields {es-pull}39560[#39560] (issue: {es-issue}39547[#39547]) * Fix COUNT DISTINCT column name {es-pull}39537[#39537] (issue: {es-issue}39511[#39511]) * Enable accurate hit tracking on demand {es-pull}39527[#39527] (issue: {es-issue}37971[#37971]) * Ignore UNSUPPORTED fields for JDBC and ODBC modes in 'SYS COLUMNS' {es-pull}39518[#39518] (issue: {es-issue}39471[#39471]) * Enforce JDBC driver - ES server version parity {es-pull}38972[#38972] (issue: {es-issue}38775[#38775]) * Fall back to using the field name for column label {es-pull}38842[#38842] (issue: {es-issue}38831[#38831]) Suggesters:: * Fix duplicate removal when merging completion suggestions {es-pull}36996[#36996] (issue: {es-issue}35836[#35836]) Task Management:: * Un-assign persistent tasks as nodes exit the cluster {es-pull}37656[#37656] Watcher:: * Watcher accounts constructed lazily {es-pull}36656[#36656] * Only trigger a watch if new or schedule/changed {es-pull}35908[#35908] * Fix Watcher NotificationService's secure settings {es-pull}35610[#35610] (issue: {es-issue}35378[#35378]) * Fix integration tests to ensure correct start/stop of Watcher {es-pull}35271[#35271] (issues: {es-issue}29877[#29877], {es-issue}30705[#30705], {es-issue}33291[#33291], {es-issue}34448[#34448], {es-issue}34462[#34462]) ZenDiscovery:: * Remove duplicate discovered peers {es-pull}35505[#35505] * Respect the no_master_block setting {es-pull}36478[#36478] * Cancel GetDiscoveredNodesAction when bootstrapped {es-pull}36423[#36423] (issues: {es-issue}36380[#36380], {es-issue}36381[#36381]) * Only elect master-eligible nodes {es-pull}35996[#35996] * Remove duplicate discovered peers {es-pull}35505[#35505] * Fix size of rolling-upgrade bootstrap config {es-pull}38031[#38031] * Always return metadata version if metadata is requested {es-pull}37674[#37674] * Elect freshest master in upgrade {es-pull}37122[#37122] (issue: {es-issue}40[#40]) * Fix cluster state persistence for single-node discovery {es-pull}36825[#36825] [[regression-7.0.0]] [float] === Regressions Infra/Core:: * Restore date aggregation performance in UTC case {es-pull}38221[#38221] (issue: {es-issue}37826[#37826]) * Speed up converting of temporal accessor to zoned date time {es-pull}37915[#37915] (issue: {es-issue}37826[#37826]) Mapping:: * Performance fix. Reduce deprecation calls for the same bulk request {es-pull}37415[#37415] (issue: {es-issue}37411[#37411]) Scripting:: * Use Number as a return value for BucketAggregationScript {es-pull}35653[#35653] (issue: {es-issue}35351[#35351]) [[upgrade-7.0.0]] [float] === Upgrades Discovery-Plugins:: * Bump jackson-databind version for AWS SDK {es-pull}39183[#39183] Engine:: * Upgrade to lucene-8.0.0-snapshot-83f9835. {es-pull}37668[#37668] * Upgrade to Lucene 8.0.0-snapshot-ff9509a8df {es-pull}39350[#39350] * Upgrade to Lucene 8.0.0 {es-pull}39992[#39992] (issue: {es-issue}39640[#39640]) Geo:: * Upgrade JTS to 1.14.0 {es-pull}29141[#29141] (issue: {es-issue}29122[#29122]) Ingest:: * Update geolite2 database in ingest geoip plugin {es-pull}33840[#33840] * Bump jackson-databind version for ingest-geoip {es-pull}39182[#39182] Infra/Core:: * Upgrade to a Lucene 8 snapshot {es-pull}33310[#33310] (issues: {es-issue}32899[#32899], {es-issue}33028[#33028], {es-issue}33309[#33309]) Security:: * Upgrade the bouncycastle dependency to 1.61 {es-pull}40017[#40017] (issue: {es-issue}40011[#40011]) Search:: * Upgrade to Lucene 8.0.0 GA {es-pull}39992[#39992] (issue: {es-issue}39640[#39640]) Snapshot/Restore:: * plugins/repository-gcs: Update google-cloud-storage/core to 1.59.0 {es-pull}39748[#39748] (issue: {es-issue}39366[#39366]) Network:: * Fix Netty Leaks by upgrading to 4.1.28 {es-pull}32511[#32511] (issue: {es-issue}32487[#32487]) * Upgrade Netty 4.3.32.Final {es-pull}36102[#36102] (issue: {es-issue}35360[#35360]) Machine Learning:: * No need to add state doc mapping on job open in 7.x {es-pull}37759[#37759]