[[release-notes-7.4.2]] == {es} version 7.4.2 [discrete] [[bug-7.4.2]] === Bug fixes Transform:: * Prevent assignment if any node is older than 7.4 {es-pull}48055[#48055] (issue: {es-issue}48019[#48019]) [[release-notes-7.4.1]] == {es} version 7.4.1 Also see <>. [[enhancement-7.4.1]] [discrete] === Enhancements Engine:: * Avoid unneeded refresh with concurrent realtime gets {es-pull}47895[#47895] * sync before trimUnreferencedReaders to improve index preformance {es-pull}47790[#47790] (issues: {es-issue}46201[#46201], {es-issue}46203[#46203]) * Limit number of retaining translog files for peer recovery {es-pull}47414[#47414] Infra/Circuit Breakers:: * Emit log message when parent circuit breaker trips {es-pull}47000[#47000] Machine Learning:: * Throttle the delete-by-query of expired results {es-pull}47177[#47177] (issues: {es-issue}47003[#47003]) * The {ml} native processes are now arranged in a `.app` directory structure on macOS, to allow for notarization on macOS Catalina. {ml-pull}593[#593] [[bug-7.4.1]] [discrete] === Bug fixes Aggregations:: * DocValueFormat implementation for date range fields {es-pull}47472[#47472] (issues: {es-issue}47323[#47323], {es-issue}47469[#47469]) Allocation:: * Dangling indices strip aliases {es-pull}47581[#47581] Analysis:: * Reset Token position on reuse in `predicate_token_filter` {es-pull}47424[#47424] (issue: {es-issue}47197[#47197]) Authentication:: * Fix AD realm additional metadata {es-pull}47179[#47179] (issue: {es-issue}45848[#45848]) Authorization:: * Use 'should' clause instead of 'filter' when querying native privileges {es-pull}47019[#47019] CCR:: * Do not auto-follow closed indices {es-pull}47721[#47721] (issue: {es-issue}47582[#47582]) * Relax maxSeqNoOfUpdates assertion in FollowingEngine {es-pull}47188[#47188] (issue: {es-issue}47137[#47137]) Cluster Coordination:: * Omit writing index metadata for non-replicated closed indices on data-only node {es-pull}47285[#47285] (issue: {es-issue}47276[#47276]) Features/ILM+SLM:: * Throw error retrieving non-existent SLM policy {es-pull}47679[#47679] (issue: {es-issue}47664[#47664]) Features/Indices APIs:: * Fix Rollover error when alias has closed indices {es-pull}47148[#47148] (issue: {es-issue}47146[#47146]) Features/Java High Level REST Client:: * Fix ILM HLRC Javadoc->Documentation links {es-pull}48083[#48083] Features/Monitoring:: * [Monitoring] Add new cluster privilege now necessary for the stack monitoring ui {es-pull}47871[#47871] Infra/Scripting:: * Drop stored scripts with the old style-id {es-pull}48078[#48078] (issue: {es-issue}47593[#47593]) Machine Learning:: * Fix detection of syslog-like timestamp in find_file_structure {es-pull}47970[#47970] * Reinstate ML daily maintenance actions {es-pull}47103[#47103] (issue: {es-issue}47003[#47003]) * Fix possibility of crash when calculating forecasts that overflow to disk {ml-pull}688[#688] Network:: * Fix es.http.cname_in_publish_address Deprecation Logging {es-pull}47451[#47451] (issue: {es-issue}47436[#47436]) SQL:: * SQL: Fix issue with negative literels and parentheses {es-pull}48113[#48113] (issue: {es-issue}48009[#48009]) * SQL: add "format" for "full" date range queries {es-pull}48073[#48073] (issue: {es-issue}48033[#48033]) * SQL: Allow whitespaces in escape patterns {es-pull}47577[#47577] (issue: {es-issue}47401[#47401]) * SQL: fix multi full-text functions usage with aggregate functions {es-pull}47444[#47444] (issue: {es-issue}47365[#47365]) * SQL: wrong number of values for columns {es-pull}42122[#42122] Search:: * Fix alias field resolution in match query {es-pull}47369[#47369] Snapshot/Restore:: * Fix Bug in Azure Repo Exception Handling {es-pull}47968[#47968] * Fix Snapshot Corruption in Edge Case {es-pull}47552[#47552] (issues: {es-issue}46250[#46250], {es-issue}47550[#47550]) Store:: * Allow truncation of clean translog {es-pull}47866[#47866] Transform:: * Fix bwc serialization with 7.3 {es-pull}48021[#48021] [[release-notes-7.4.0]] == {es} version 7.4.0 Also see <>. [discrete] === Known issues * Applying deletes or updates on an index after it has been shrunk may corrupt the index. In order to prevent this issue, it is recommended to stop shrinking read-write indices. For read-only indices, it is recommended to force-merge indices after shrinking, which significantly reduces the likeliness of this corruption in the case that deletes/updates would be applied by mistake. This bug is fixed in {es} 7.7 and later versions. More details can be found on the https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue]. * Activating the <> should be avoided in this version. Any attempt to log a slow search can throw an AIOOBE due to a bug that performs concurrent modifications on a shared byte array. (issue: {es-issue}/48358[#48358]) * Indices created in 6.x with <> and <> fields using formats that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. https://github.com/elastic/elasticsearch/pull/52555 This is fixed in {es} 7.7 and later versions. [[breaking-7.4.0]] [discrete] === Breaking changes Infra/REST API:: * Update the schema for the REST API specification {es-pull}42346[#42346] (issue: {es-issue}35262[#35262]) Machine Learning:: * Improve progress reporting for data frame analytics {es-pull}45856[#45856] Ranking:: * Forbid empty doc values on vector functions {es-pull}43944[#43944] Search:: * Use float instead of double for query vectors. {es-pull}46004[#46004] Snapshot/Restore:: * Provide an Option to Use Path-Style-Access with S3 Repo {es-pull}41966[#41966] (issue: {es-issue}41816[#41816]) Transforms:: * Combine task_state and indexer_state in _stats {es-pull}45276[#45276] (issue: {es-issue}45201[#45201]) * Improve response format of transform stats endpoint {es-pull}44350[#44350] (issue: {es-issue}43767[#43767]) [[breaking-java-7.4.0]] [discrete] === Breaking Java changes Geo:: * Geo: Change order of parameter in Geometries to lon, lat {es-pull}45332[#45332] (issue: {es-issue}45048[#45048]) Network:: * Stop Recreating Wrapped Handlers in RestController {es-pull}44964[#44964] [[deprecation-7.4.0]] [discrete] === Deprecations Geo:: * Geo: add Geometry-based query builders to QueryBuilders {es-pull}45058[#45058] (issues: {es-issue}44715[#44715], {es-issue}45048[#45048]) Infra/Core:: * Bundle AdoptOpenJDK 13 {es-pull}46860[#46860] * Add deprecation check for pidfile setting {es-pull}45939[#45939] (issues: {es-issue}45938[#45938], {es-issue}45940[#45940]) * Deprecate the pidfile setting {es-pull}45938[#45938] * Add node.processors setting in favor of processors {es-pull}45855[#45855] * Deprecate setting processors to more than available {es-pull}44889[#44889] Infra/Settings:: * Add deprecation check for processors {es-pull}45925[#45925] (issues: {es-issue}45855[#45855], {es-issue}45905[#45905]) Machine Learning:: * Only emit deprecation warning if there was actual change of a datafeed's job_id. {es-pull}44755[#44755] * Deprecate the ability to update datafeed's job_id. {es-pull}44691[#44691] (issue: {es-issue}44615[#44615]) [[feature-7.4.0]] [discrete] === New features Aggregations:: * Support Range Fields in Histogram and Date Histogram {es-pull}45395[#45395] * Add Cumulative Cardinality agg (and Data Science plugin) {es-pull}43661[#43661] (issue: {es-issue}43550[#43550]) Analysis:: * Add support for inlined user dictionary in the Kuromoji plugin {es-pull}45489[#45489] (issue: {es-issue}25343[#25343]) Authentication:: * PKI realm authentication delegation {es-pull}45906[#45906] (issue: {es-issue}34396[#34396]) * PKI Authentication Delegation in new endpoint {es-pull}43796[#43796] (issue: {es-issue}34396[#34396]) Authorization:: * Add granular privileges for API keys {es-pull}42020[#42020] Features/ILM:: * Add Snapshot Lifecycle Management {es-pull}43934[#43934] (issue: {es-issue}38461[#38461]) Features/Watcher:: * Add max_iterations configuration to watcher action with foreach execution {es-pull}45715[#45715] (issues: {es-issue}41997[#41997], {es-issue}45169[#45169]) Geo:: * [SPATIAL] New ShapeQueryBuilder for querying indexed cartesian geometry {es-pull}45108[#45108] (issue: {es-issue}44980[#44980]) * [GEO] New ShapeFieldMapper for indexing cartesian geometries {es-pull}44980[#44980] * Add Circle Processor {es-pull}43851[#43851] (issue: {es-issue}43554[#43554]) * New `shape` field type for indexing Cartesian Geometries {es-pull}43644[#43644] Machine Learning:: * Allow the user to specify 'query' in Evaluate Data Frame request {es-pull}45775[#45775] (issue: {es-issue}45729[#45729]) * Call the new _estimate_memory_usage API endpoint on data frame analytics _start {es-pull}45536[#45536] (issues: {es-issue}44699[#44699], {es-issue}45544[#45544]) * HLRC for memory usage estimation API {es-pull}45531[#45531] (issues: {es-issue}44699[#44699], {es-issue}45188[#45188]) * Implement ml/data_frame/analytics/_estimate_memory_usage API endpoint {es-pull}45188[#45188] (issue: {es-issue}44699[#44699]) [[enhancement-7.4.0]] [discrete] === Enhancements Aggregations:: * Add more flexibility to MovingFunction window alignment {es-pull}44360[#44360] (issue: {es-issue}42181[#42181]) * Optimize Min and Max BKD optimizations {es-pull}44315[#44315] (issue: {es-issue}44290[#44290]) * Allow pipeline aggs to select specific buckets from multi-bucket aggs {es-pull}44179[#44179] Allocation:: * Defer reroute when starting shards {es-pull}44433[#44433] (issues: {es-issue}42105[#42105], {es-issue}42738[#42738]) * Allow RerouteService to reroute at lower priority {es-pull}44338[#44338] * Auto-release of read-only-allow-delete block when disk utilization fa… {es-pull}42559[#42559] (issue: {es-issue}39334[#39334]) Analysis:: * Allow all token/char filters in normalizers {es-pull}43803[#43803] (issue: {es-issue}43758[#43758]) Authentication:: * Allow Transport Actions to indicate authN realm {es-pull}45767[#45767] (issue: {es-issue}45331[#45331]) * Explicitly fail if a realm only exists in keystore {es-pull}44471[#44471] (issue: {es-issue}44207[#44207]) Authorization:: * Add `manage_own_api_key` cluster privilege {es-pull}45897[#45897] (issue: {es-issue}40031[#40031]) * Consider `owner` flag when retrieving/invalidating keys with API key service {es-pull}45421[#45421] (issue: {es-issue}40031[#40031]) * REST API changes for manage-own-api-key privilege {es-pull}44936[#44936] (issue: {es-issue}40031[#40031]) * Simplify API key service API {es-pull}44935[#44935] (issue: {es-issue}40031[#40031]) CCR:: * Include leases in error message when operations no longer available {es-pull}45681[#45681] CRUD:: * Return seq_no and primary_term for noop update {es-pull}44603[#44603] (issue: {es-issue}42497[#42497]) Cluster Coordination:: * Improve slow logging in MasterService {es-pull}45086[#45086] (issue: {es-issue}45007[#45007]) * More logging for slow cluster state application {es-pull}45007[#45007] * Ignore unknown fields if overriding node metadata {es-pull}44689[#44689] * Allow pending tasks before state recovery {es-pull}44685[#44685] (issue: {es-issue}44652[#44652]) Distributed:: * Do not create engine under IndexShard#mutex {es-pull}45263[#45263] (issue: {es-issue}43699[#43699]) Docs Infrastructure:: * add clarification around TESTSETUP docu and error message {es-pull}43306[#43306] Engine:: * Flush engine after big merge {es-pull}46066[#46066] * Do sync before closeIntoReader when rolling generation to improve index performance {es-pull}45765[#45765] (issue: {es-issue}45371[#45371]) * Refactor index engines to manage readers instead of searchers {es-pull}43860[#43860] * Async IO Processor release before notify {es-pull}43682[#43682] * Enable indexing optimization using sequence numbers on replicas {es-pull}43616[#43616] (issue: {es-issue}34099[#34099]) Features/ILM:: * Add node setting for disabling SLM {es-pull}46794[#46794] (issue: {es-issue}38461[#38461]) * Include in-progress snapshot for a policy with get SLM policy API {es-pull}45245[#45245] * Add option to filter ILM explain response {es-pull}44777[#44777] (issue: {es-issue}44189[#44189]) * Expose index age in ILM explain output {es-pull}44457[#44457] (issue: {es-issue}38988[#38988]) Features/Indices APIs:: * Add Clone Index API {es-pull}44267[#44267] (issue: {es-issue}44128[#44128]) * Add description to force-merge tasks {es-pull}41365[#41365] (issue: {es-issue}15975[#15975]) Features/Ingest:: * Fix IngestService to respect original document content type {es-pull}45799[#45799] * Ingest Attachment: Upgrade tika to v1.22 {es-pull}45575[#45575] * Retrieve processors instead of checking existence {es-pull}45354[#45354] * Add ingest processor existence helper method {es-pull}45156[#45156] * Change the ingest simulate api to not include dropped documents {es-pull}44161[#44161] (issue: {es-issue}36150[#36150]) Features/Java High Level REST Client:: * Add XContentType as parameter to HLRC ART#createServerTestInstance {es-pull}46036[#46036] (issue: {es-issue}45970[#45970]) * Add CloseIndexResponse to HLRC {es-pull}44349[#44349] (issue: {es-issue}39687[#39687]) * Add mapper-extras and the RankFeatureQuery in the hlrc {es-pull}43713[#43713] (issue: {es-issue}43634[#43634]) * removing background state update of Request object by RequestConverte… {es-pull}40156[#40156] (issue: {es-issue}39666[#39666]) * Add delete aliases API to the high-level REST client {es-pull}32909[#32909] (issue: {es-issue}27205[#27205]) Features/Watcher:: * Add SSL/TLS settings for watcher email {es-pull}45272[#45272] (issue: {es-issue}30307[#30307]) * Watcher reporting: add email warning if CSV attachment contains values that may be interperted as formulas {es-pull}44460[#44460] * Watcher add stopped listener {es-pull}43939[#43939] (issue: {es-issue}42409[#42409]) * Improve CryptoService error message on missing secure file {es-pull}43623[#43623] (issue: {es-issue}43619[#43619]) Geo:: * Support WKT point conversion to geo_point type {es-pull}44107[#44107] (issue: {es-issue}41821[#41821]) Infra/Circuit Breakers:: * Fix G1 GC default IHOP {es-pull}46169[#46169] Infra/Core:: * Add OCI annotations and adjust existing annotations {es-pull}45167[#45167] (issues: {es-issue}45162[#45162], {es-issue}45166[#45166]) * Use the full hash in build info {es-pull}45163[#45163] (issue: {es-issue}45162[#45162]) Infra/Packaging:: * Remove redundant Java check from Sys V init {es-pull}45793[#45793] (issue: {es-issue}45593[#45593]) * Notify systemd when Elasticsearch is ready {es-pull}44673[#44673] Infra/Plugins:: * Make plugin verification FIPS 140 compliant {es-pull}44224[#44224] (issue: {es-issue}41263[#41263]) Infra/Scripting:: * Whitelist randomUUID in Painless {es-pull}45148[#45148] (issue: {es-issue}39080[#39080]) * Add missing ZonedDateTime methods for joda compat layer {es-pull}44829[#44829] (issue: {es-issue}44411[#44411]) * Remove stale permissions from untrusted policy {es-pull}44783[#44783] Infra/Settings:: * Add more meaningful keystore version mismatch errors {es-pull}46291[#46291] (issue: {es-issue}44624[#44624]) * Lift the restrictions that uppercase is not allowed in Setting Name. {es-pull}45222[#45222] (issue: {es-issue}43835[#43835]) * Normalize environment paths {es-pull}45179[#45179] (issue: {es-issue}45176[#45176]) Machine Learning:: * Support boolean fields for data frame analytics {es-pull}46037[#46037] * Add description to data frame analytics {es-pull}45774[#45774] * Add regression analysis to data frame analytics {es-pull}45292[#45292] * Introduce formal node ML role {es-pull}45174[#45174] (issues: {es-issue}29943[#29943], {es-issue}43175[#43175]) * Improve CSV header row detection in find_file_structure {es-pull}45099[#45099] (issue: {es-issue}45047[#45047]) * Outlier detection should only fetch docs that have the analyzed fields {es-pull}44944[#44944] * Persist DatafeedTimingStats with RefreshPolicy.NONE by default {es-pull}44940[#44940] (issue: {es-issue}44792[#44792]) * Add result_type field to TimingStats and DatafeedTimingStats documents {es-pull}44812[#44812] * Implement exponential average search time per hour statistics. {es-pull}44683[#44683] (issue: {es-issue}29857[#29857]) * Add r_squared eval metric to regression {es-pull}44248[#44248] * Adds support for regression.mean_squared_error to eval API {es-pull}44140[#44140] * Add DatafeedTimingStats.average_search_time_per_bucket_ms and TimingStats.total_bucket_processing_time_ms stats {es-pull}44125[#44125] (issue: {es-issue}29857[#29857]) * Add DatafeedTimingStats to datafeed GetDatafeedStatsAction.Response {es-pull}43045[#43045] (issue: {es-issue}29857[#29857]) Network:: * Better logging for TLS message on non-secure transport channel {es-pull}45835[#45835] (issue: {es-issue}32688[#32688]) * Asynchronously connect to remote clusters {es-pull}44825[#44825] (issue: {es-issue}40150[#40150]) * Improve errors when TLS files cannot be read {es-pull}44787[#44787] (issue: {es-issue}43079[#43079]) * Add per-socket keepalive options {es-pull}44055[#44055] * Move ConnectionManager to async APIs {es-pull}42636[#42636] Ranking:: * Search enhancement: pinned queries {es-pull}44345[#44345] (issue: {es-issue}44074[#44074]) * Fix parameter value for calling data.advanceExact {es-pull}44205[#44205] * Add l1norm and l2norm distances for vectors {es-pull}44116[#44116] (issue: {es-issue}37947[#37947]) Recovery:: * Ignore translog retention policy if soft-deletes enabled {es-pull}45473[#45473] (issue: {es-issue}45136[#45136]) * Only retain reasonable history for peer recoveries {es-pull}45208[#45208] (issue: {es-issue}41536[#41536]) * Use index for peer recovery instead of translog {es-pull}45136[#45136] (issues: {es-issue}38904[#38904], {es-issue}41536[#41536], {es-issue}42211[#42211]) * Trim local translog in peer recovery {es-pull}44756[#44756] * Make peer recovery send file chunks async {es-pull}44468[#44468] (issues: {es-issue}36195[#36195], {es-issue}44040[#44040]) SQL:: * SQL: Support queries with HAVING over SELECT {es-pull}46709[#46709] (issue: {es-issue}37051[#37051]) * SQL: Break TextFormatter/Cursor dependency {es-pull}45613[#45613] (issue: {es-issue}45516[#45516]) * SQL: remove deprecated use of "interval" from date_histogram usage {es-pull}45501[#45501] (issue: {es-issue}43922[#43922]) * SQL: use hasValue() methods from Elasticsearch's InspectionHelper classes {es-pull}44745[#44745] (issue: {es-issue}35745[#35745]) * Switch from using docvalue_fields to extracting values from _source {es-pull}44062[#44062] (issue: {es-issue}41852[#41852]) Search:: * Adds usage stats for vectors: {es-pull}44512[#44512] * Associate sub-requests to their parent task in multi search API {es-pull}44492[#44492] * Cancel search task on connection close {es-pull}43332[#43332] Security:: * Set security index refresh interval to 1s {es-pull}45434[#45434] (issue: {es-issue}44934[#44934]) Snapshot/Restore:: * add disable_chunked_encoding configuration {es-pull}44052[#44052] * Repository Cleanup Endpoint {es-pull}43900[#43900] Task Management:: * Remove task null check in TransportAction {es-pull}45014[#45014] * TaskListener#onFailure to accept Exception instead of Throwable {es-pull}44946[#44946] * Move child task cancellation to TaskManager {es-pull}44573[#44573] (issue: {es-issue}44494[#44494]) Transforms:: * Add update transform api endpoint {es-pull}45154[#45154] (issue: {es-issue}43438[#43438]) * Add support for bucket_selector {es-pull}44718[#44718] (issues: {es-issue}43744[#43744], {es-issue}44557[#44557]) * Add force delete {es-pull}44590[#44590] (issue: {es-issue}43961[#43961]) * Add dynamic cluster setting for failure retries {es-pull}44577[#44577] * Add optional defer_validation param to PUT {es-pull}44455[#44455] (issue: {es-issue}43439[#43439]) * Add support for geo_bounds aggregation {es-pull}44441[#44441] * Add a frequency option to transform config, default 1m {es-pull}44120[#44120] [[bug-7.4.0]] [discrete] === Bug fixes Aggregations:: * Fix early termination of aggregators that run with breadth-first mode {es-pull}44963[#44963] (issue: {es-issue}44909[#44909]) * Support BucketScript paths of type string and array. {es-pull}44694[#44694] (issue: {es-issue}44385[#44385]) Allocation:: * Avoid overshooting watermarks during relocation {es-pull}46079[#46079] (issue: {es-issue}45177[#45177]) * Cluster health should await events plus other things {es-pull}44348[#44348] * Do not copy initial recovery filter during split {es-pull}44053[#44053] (issue: {es-issue}43955[#43955]) Analysis:: * Enable reloading of synonym_graph filters {es-pull}45135[#45135] (issue: {es-issue}45127[#45127]) * Fix AnalyzeAction response serialization {es-pull}44284[#44284] (issue: {es-issue}44078[#44078]) Authentication:: * Fallback to realm authc if ApiKey fails {es-pull}46538[#46538] * Enforce realm name uniqueness {es-pull}46253[#46253] * Allow empty token endpoint for implicit flow {es-pull}45038[#45038] * Do not use scroll when finding duplicate API key {es-pull}45026[#45026] * Fix broken short-circuit in getUnlicensedRealms {es-pull}44399[#44399] * Fix X509AuthenticationToken principal {es-pull}43932[#43932] (issues: {es-issue}34396[#34396], {es-issue}43796[#43796]) Authorization:: * Do not rewrite aliases on remove-index from aliases requests {es-pull}46989[#46989] * Give kibana user privileges to create APM agent config index {es-pull}46765[#46765] (issue: {es-issue}45610[#45610]) * Add `manage_own_api_key` cluster privilege {es-pull}45696[#45696] (issue: {es-issue}40031[#40031]) * Sparse role queries can throw an NPE {es-pull}45053[#45053] CCR:: * Clean up ShardFollowTasks for deleted indices {es-pull}44702[#44702] (issue: {es-issue}34404[#34404]) * Skip update if leader and follower settings identical {es-pull}44535[#44535] (issue: {es-issue}44521[#44521]) * Avoid stack overflow in auto-follow coordinator {es-pull}44421[#44421] (issue: {es-issue}43251[#43251]) * Avoid NPE when checking for CCR index privileges {es-pull}44397[#44397] (issue: {es-issue}44172[#44172]) CRUD:: * Ignore replication for noop updates {es-pull}46458[#46458] (issues: {es-issue}41065[#41065], {es-issue}44603[#44603], {es-issue}46366[#46366]) * Allow _update on write alias {es-pull}45318[#45318] (issue: {es-issue}31520[#31520]) * Do not allow version in Rest Update API {es-pull}43516[#43516] (issue: {es-issue}42497[#42497]) Cluster Coordination:: * Assert no exceptions during state application {es-pull}47090[#47090] (issue: {es-issue}47038[#47038]) * Avoid counting votes from master-ineligible nodes {es-pull}43688[#43688] Distributed:: * Fix false positive out of sync warning in synced-flush {es-pull}46576[#46576] (issues: {es-issue}28464[#28464], {es-issue}30244[#30244]) * Suppress warning logs from background sync on relocated primary {es-pull}46247[#46247] (issues: {es-issue}40800[#40800], {es-issue}42241[#42241]) * Ensure AsyncTask#isScheduled remain false after close {es-pull}45687[#45687] (issue: {es-issue}45576[#45576]) * Update translog checkpoint after marking operations as persisted {es-pull}45634[#45634] (issue: {es-issue}29161[#29161]) * Fix clock used in update requests {es-pull}45262[#45262] (issue: {es-issue}45254[#45254]) * Restore DefaultShardOperationFailedException's reason during deserialization {es-pull}45203[#45203] * Use IndicesModule named writables in elasticsearch-shard tool {es-pull}45036[#45036] (issue: {es-issue}44628[#44628]) Engine:: * Handle delete document level failures {es-pull}46100[#46100] (issue: {es-issue}46083[#46083]) * Handle no-op document level failures {es-pull}46083[#46083] * Remove leniency during replay translog in peer recovery {es-pull}44989[#44989] * Throw TranslogCorruptedException in more cases {es-pull}44217[#44217] * Fail engine if hit document failure on replicas {es-pull}43523[#43523] (issues: {es-issue}40435[#40435], {es-issue}43228[#43228]) Features/ILM:: * Handle partial failure retrieving segments in SegmentCountStep {es-pull}46556[#46556] * Fixes for API specification {es-pull}46522[#46522] * Fix SnapshotLifecycleMetadata xcontent serialization {es-pull}46500[#46500] (issue: {es-issue}46499[#46499]) * Updated slm API spec parameters and URL {es-pull}44797[#44797] * Fix swapped variables in error message {es-pull}44300[#44300] Features/Indices APIs:: * Fix a bug where mappings are dropped from rollover requests. {es-pull}45411[#45411] (issue: {es-issue}45399[#45399]) * Create index with typeless mapping {es-pull}45120[#45120] * Check shard limit after applying index templates {es-pull}44619[#44619] (issues: {es-issue}34021[#34021], {es-issue}44567[#44567], {es-issue}44619[#44619]) * Validate index settings after applying templates {es-pull}44612[#44612] (issues: {es-issue}34021[#34021], {es-issue}44567[#44567]) Features/Ingest:: * Allow dropping documents with auto-generated ID {es-pull}46773[#46773] (issue: {es-issue}46678[#46678]) Features/Java High Level REST Client:: * [HLRC] Send min_score as query string parameter to the count API {es-pull}46829[#46829] (issue: {es-issue}46474[#46474]) * HLRC multisearchTemplate forgot params {es-pull}46492[#46492] (issue: {es-issue}46488[#46488]) * terminateAfter added to the RequestConverter {es-pull}46474[#46474] (issue: {es-issue}46446[#46446]) * [Closes #44045] Added 'slices' parameter when submitting reindex request via Java high level REST client {es-pull}45690[#45690] (issue: {es-issue}44045[#44045]) * HLRC: Fix '+' Not Correctly Encoded in GET Req. {es-pull}33164[#33164] (issue: {es-issue}33077[#33077]) Features/Watcher:: * Fix class used to initialize logger in Watcher {es-pull}46467[#46467] * Fix wrong URL encoding in watcher HTTP client {es-pull}45894[#45894] (issue: {es-issue}44970[#44970]) * Fix watcher HttpClient URL creation {es-pull}45207[#45207] (issue: {es-issue}44970[#44970]) * Log write failures for watcher history document. {es-pull}44129[#44129] Geo:: * Geo: fix geo query decomposition {es-pull}44924[#44924] (issue: {es-issue}44891[#44891]) * Geo: add validator that only checks altitude {es-pull}43893[#43893] Highlighting:: * Fix highlighting for script_score query {es-pull}46507[#46507] (issue: {es-issue}46471[#46471]) Infra/Core:: * Always check that cgroup data is present {es-pull}45606[#45606] (issue: {es-issue}45396[#45396]) * Safe publication of DelayedAllocationService and SnapshotShardsService {es-pull}45517[#45517] (issue: {es-issue}38560[#38560]) * Add default CLI JVM options {es-pull}44545[#44545] (issues: {es-issue}219[#219], {es-issue}42021[#42021]) * Fix decimal point parsing for date_optional_time {es-pull}43859[#43859] (issue: {es-issue}43730[#43730]) Infra/Logging:: * Fix types field in JSON Search Slow Logs {es-pull}44641[#44641] * Add types field to JSON slow logs in 7.x {es-pull}44592[#44592] (issues: {es-issue}41354[#41354], {es-issue}44178[#44178]) Infra/Packaging:: * Add destructiveDistroTest meta task {es-pull}45762[#45762] * Use bundled JDK in Sys V init {es-pull}45593[#45593] (issue: {es-issue}45542[#45542]) * Restore setting up temp dir for windows service {es-pull}44541[#44541] Infra/Plugins:: * Do not checksum all bytes at once in plugin install {es-pull}44649[#44649] (issue: {es-issue}44545[#44545]) Infra/REST API:: * Improve error message when index settings are not a map {es-pull}45588[#45588] (issue: {es-issue}45126[#45126]) * Add is_write_index column to cat.aliases {es-pull}44772[#44772] * Fix URL documentation in API specs {es-pull}44487[#44487] Infra/Scripting:: * Fix bugs in Painless SCatch node {es-pull}45880[#45880] * Fix JodaCompatibleZonedDateTime casts in Painless {es-pull}44874[#44874] Infra/Settings:: * bug fix about elasticsearch.common.settings.Settings.processSetting {es-pull}44047[#44047] (issue: {es-issue}43791[#43791]) Machine Learning:: * Fix two datafeed flush lockup bugs {es-pull}46982[#46982] * Throw an error when a datafeed needs CCS but it is not enabled for the node {es-pull}46044[#46044] (issue: {es-issue}46025[#46025]) * Handle "null" value of Estimate memory usage API response gracefully. {es-pull}45726[#45726] (issue: {es-issue}44699[#44699]) * Remove timeout on waiting for data frame analytics result processor to complete {es-pull}45724[#45724] (issue: {es-issue}45723[#45723]) * Check dest index is empty when starting data frame analytics {es-pull}45094[#45094] * Catch any error thrown while closing data frame analytics process {es-pull}44958[#44958] * Treat PostDataActionResponse.DataCounts.bucketCount as incremental rather than absolute (total). {es-pull}44803[#44803] (issue: {es-issue}44792[#44792]) * Treat big changes in searchCount as significant and persist the document after such changes {es-pull}44413[#44413] (issues: {es-issue}44196[#44196], {es-issue}44335[#44335]) * Update .ml-config mappings before indexing job, datafeed or data frame analytics config {es-pull}44216[#44216] (issue: {es-issue}44263[#44263]) * Wait for .ml-config primary before assigning persistent tasks {es-pull}44170[#44170] (issue: {es-issue}44156[#44156]) * Fix ML memory tracker lockup when inner step fails {es-pull}44158[#44158] (issue: {es-issue}44156[#44156]) * Fix datafeed checks when a concrete remote index is present {es-pull}43923[#43923] (issue: {es-issue}42113[#42113]) * Rename outlier detection method values `knn` and `tnn` to `distance_kth_nn` and `distance_knn` respectively to match the API. {ml-pull}598[#598] * Fix occasional (non-deterministic) reinitialisation of modeling for the `lat_long` function. {ml-pull}641[#641] Mapping:: * Make sure to validate the type before attempting to merge a new mapping. {es-pull}45157[#45157] (issues: {es-issue}29316[#29316], {es-issue}43012[#43012]) * Ensure field caps doesn't error on rank feature fields. {es-pull}44370[#44370] (issue: {es-issue}44330[#44330]) * Prevent types deprecation warning for indices.exists requests {es-pull}43963[#43963] (issue: {es-issue}43905[#43905]) * Add include_type_name in indices.exitst REST API spec {es-pull}43910[#43910] (issue: {es-issue}43905[#43905]) Network:: * Fix Broken HTTP Request Breaking Channel Closing {es-pull}45958[#45958] (issues: {es-issue}43362[#43362], {es-issue}43850[#43850]) * Fix plaintext on TLS port logging {es-pull}45852[#45852] (issue: {es-issue}32688[#32688]) * transport.publish_address should contain CNAME {es-pull}45626[#45626] (issues: {es-issue}32806[#32806], {es-issue}39970[#39970]) * Fix bug in copying bytes for socket write {es-pull}45463[#45463] (issue: {es-issue}45444[#45444]) Recovery:: * Never release store using CancellableThreads {es-pull}45409[#45409] (issues: {es-issue}45136[#45136], {es-issue}45237[#45237]) * Remove leniency in reset engine from translog {es-pull}44711[#44711] Rollup:: * Fix Rollup job creation to work with templates {es-pull}43943[#43943] SQL:: * SQL: Properly handle indices with no/empty mapping {es-pull}46775[#46775] (issue: {es-issue}46757[#46757]) * SQL: improve ResultSet behavior when no rows are available {es-pull}46753[#46753] (issue: {es-issue}46750[#46750]) * SQL: use the correct data type for types conversion {es-pull}46574[#46574] (issue: {es-issue}46090[#46090]) * SQL: Fix issue with common type resolution {es-pull}46565[#46565] (issue: {es-issue}46551[#46551]) * SQL: fix scripting for grouped by datetime functions {es-pull}46421[#46421] (issue: {es-issue}40241[#40241]) * SQL: Use null schema response {es-pull}46386[#46386] (issue: {es-issue}46381[#46381]) * SQL: Fix issue with IIF function when condition folds {es-pull}46290[#46290] (issue: {es-issue}46268[#46268]) * SQL: Fix issue with DataType for CASE with NULL {es-pull}46173[#46173] (issue: {es-issue}46032[#46032]) * SQL: adds format parameter to range queries for constant date comparisons {es-pull}45326[#45326] (issue: {es-issue}45139[#45139]) * SQL: uniquely named inner_hits sections for each nested field condition {es-pull}45039[#45039] (issues: {es-issue}33080[#33080], {es-issue}44544[#44544]) * SQL: fix URI path being lost in case of hosted ES scenario {es-pull}44776[#44776] (issue: {es-issue}44721[#44721]) * SQL: change the size of the list of concrete indices when resolving multiple indices {es-pull}43878[#43878] (issue: {es-issue}43876[#43876]) * SQL: handle double quotes escaping {es-pull}43829[#43829] (issue: {es-issue}43810[#43810]) * SQL: add pretty printing to JSON format {es-pull}43756[#43756] * SQL: handle SQL not being available in a more graceful way {es-pull}43665[#43665] (issue: {es-issue}41279[#41279]) Search:: * Multi-get requests should wait for search active {es-pull}46283[#46283] (issue: {es-issue}27500[#27500]) * Ensure top docs optimization is fully disabled for queries with unbounded max scores. {es-pull}46105[#46105] (issue: {es-issue}45933[#45933]) * Disallow partial results when shard unavailable {es-pull}45739[#45739] (issue: {es-issue}42612[#42612]) * Prevent Leaking Search Tasks on Exceptions in FetchSearchPhase and DfsQueryPhase {es-pull}45500[#45500] * Fix an NPE when requesting inner hits and _source is disabled. {es-pull}44836[#44836] (issue: {es-issue}43517[#43517]) * Don't use index_phrases on graph queries {es-pull}44340[#44340] (issue: {es-issue}43976[#43976]) Security:: * Initialize document subset bit set cache used for DLS {es-pull}46211[#46211] (issue: {es-issue}45147[#45147]) * Fix .security-* indices auto-create {es-pull}44918[#44918] * SecurityIndexManager handle RuntimeException while reading mapping {es-pull}44409[#44409] * Do not swallow I/O exception getting authentication {es-pull}44398[#44398] (issues: {es-issue}44172[#44172], {es-issue}44397[#44397]) * Use system context for looking up connected nodes {es-pull}43991[#43991] (issue: {es-issue}43974[#43974]) Snapshot/Restore:: * Fix Bug in Snapshot Status Response Timestamps {es-pull}46919[#46919] (issue: {es-issue}46913[#46913]) * GCS deleteBlobsIgnoringIfNotExists should catch StorageException {es-pull}46832[#46832] (issue: {es-issue}46772[#46772]) * Fix TransportSnapshotsStatusAction ThreadPool Use {es-pull}45824[#45824] * Stop Executing SLM Policy Transport Action on Snapshot Pool {es-pull}45727[#45727] (issue: {es-issue}45594[#45594]) * Check again on-going snapshots/restores of indices before closing {es-pull}43873[#43873] * Make Timestamps Returned by Snapshot APIs Consistent {es-pull}43148[#43148] (issue: {es-issue}43074[#43074]) * Recursively Delete Unreferenced Index Directories {es-pull}42189[#42189] (issue: {es-issue}13159[#13159]) Task Management:: * Catch AllocatedTask registration failures {es-pull}45300[#45300] Transforms:: * Use field_caps API for mapping deduction {es-pull}46703[#46703] (issue: {es-issue}46694[#46694]) * Fix off-by-one error in checkpoint operations_behind {es-pull}46235[#46235] * Moves failure state transition for MT safety {es-pull}45676[#45676] (issue: {es-issue}45664[#45664]) * Fix _start?force=true bug {es-pull}45660[#45660] * Fix failure state transitions and race condition {es-pull}45627[#45627] (issues: {es-issue}45562[#45562], {es-issue}45609[#45609]) * Fix starting a batch data frame after stopping at runtime {es-pull}45340[#45340] (issues: {es-issue}44219[#44219], {es-issue}45339[#45339]) * Fix null aggregation handling in indexer {es-pull}45061[#45061] (issue: {es-issue}44906[#44906]) * Unify validation exceptions between PUT and _preview {es-pull}44983[#44983] (issue: {es-issue}44953[#44953]) * Treat bulk index failures as an indexing failure {es-pull}44351[#44351] (issue: {es-issue}44101[#44101]) * Prevent task from attempting to run when failed {es-pull}44239[#44239] (issue: {es-issue}44121[#44121]) * Respond with 409 status code when failing _stop {es-pull}44231[#44231] (issue: {es-issue}44103[#44103]) * Add index validations to _start data frame transform {es-pull}44191[#44191] (issue: {es-issue}44104[#44104]) * Data frame task failure does not make a 500 response {es-pull}44058[#44058] (issue: {es-issue}44011[#44011]) * Audit message missing for autostop {es-pull}43984[#43984] (issue: {es-issue}43977[#43977]) [[regression-7.4.0]] [discrete] === Regressions Aggregations:: * Implement rounding optimization for fixed offset timezones {es-pull}46670[#46670] (issue: {es-issue}45702[#45702]) [[upgrade-7.4.0]] [discrete] === Upgrades Infra/Core:: * Update joda to 2.10.3 {es-pull}45495[#45495] Infra/Packaging:: * Upgrade to JDK 12.0.2 {es-pull}45172[#45172] Network:: * Upgrade to Netty 4.1.38 {es-pull}45132[#45132] Search:: * Upgrade to lucene snapshot 8.3.0-snapshot-8dd116a6158 {es-pull}45604[#45604] (issue: {es-issue}43976[#43976])