[[release-notes-7.4.0]] == {es} version 7.4.0 Also see <>. [[breaking-7.4.0]] [float] === Breaking changes Data Frame:: * [ML-DataFrame] Combine task_state and indexer_state in _stats {pull}45276[#45276] (issue: {issue}45201[#45201]) * [ML] Improve response format of data frame stats endpoint {pull}44350[#44350] (issue: {issue}43767[#43767]) Infra/REST API:: * Update the schema for the REST API specification {pull}42346[#42346] (issue: {issue}35262[#35262]) Machine Learning:: * [ML] Improve progress reportings for DF analytics {pull}45856[#45856] Ranking:: * Forbid empty doc values on vector functions {pull}43944[#43944] Search:: * Use float instead of double for query vectors. {pull}46004[#46004] Snapshot/Restore:: * Provide an Option to Use Path-Style-Access with S3 Repo {pull}41966[#41966] (issue: {issue}41816[#41816]) [[breaking-java-7.4.0]] [float] === Breaking Java changes Geo:: * Geo: Change order of parameter in Geometries to lon, lat {pull}45332[#45332] (issue: {issue}45048[#45048]) Network:: * Stop Recreating Wrapped Handlers in RestController {pull}44964[#44964] [[deprecation-7.4.0]] [float] === Deprecations Geo:: * Geo: add Geometry-based query builders to QueryBuilders {pull}45058[#45058] (issues: {issue}44715[#44715], {issue}45048[#45048]) Infra/Core:: * Add deprecation check for pidfile setting {pull}45939[#45939] (issues: {issue}45938[#45938], {issue}45940[#45940]) * Deprecate the pidfile setting {pull}45938[#45938] * Add node.processors setting in favor of processors {pull}45855[#45855] * Deprecate setting processors to more than available {pull}44889[#44889] Infra/Settings:: * Add deprecation check for processors {pull}45925[#45925] (issues: {issue}45855[#45855], {issue}45905[#45905]) Machine Learning:: * [ML] [7.x] Only emit deprecation warning if there was actual change of a datafeed's job_id. {pull}44755[#44755] * Deprecate the ability to update datafeed's job_id. {pull}44691[#44691] (issue: {issue}44615[#44615]) [[feature-7.4.0]] [float] === New features Aggregations:: * Support Range Fields in Histogram and Date Histogram {pull}45395[#45395] * Add Cumulative Cardinality agg (and Data Science plugin) {pull}43661[#43661] (issue: {issue}43550[#43550]) Analysis:: * Add support for inlined user dictionary in the Kuromoji plugin {pull}45489[#45489] (issue: {issue}25343[#25343]) Authentication:: * PKI realm authentication delegation {pull}45906[#45906] (issue: {issue}34396[#34396]) * PKI Authentication Delegation in new endpoint {pull}43796[#43796] (issue: {issue}34396[#34396]) Authorization:: * Add granular privileges for API keys {pull}42020[#42020] Features/ILM:: * Add Snapshot Lifecycle Management {pull}43934[#43934] (issue: {issue}38461[#38461]) Features/Watcher:: * Add max_iterations configuration to watcher action with foreach execution {pull}45715[#45715] (issues: {issue}41997[#41997], {issue}45169[#45169]) Geo:: * [SPATIAL] New ShapeQueryBuilder for querying indexed cartesian geometry {pull}45108[#45108] (issue: {issue}44980[#44980]) * [GEO] New ShapeFieldMapper for indexing cartesian geometries {pull}44980[#44980] * Add Circle Processor {pull}43851[#43851] (issue: {issue}43554[#43554]) * New `shape` field type for indexing Cartesian Geometries {pull}43644[#43644] Machine Learning:: * Allow the user to specify 'query' in Evaluate Data Frame request {pull}45775[#45775] (issue: {issue}45729[#45729]) * Call the new _estimate_memory_usage API endpoint on df analytics _start {pull}45536[#45536] (issues: {issue}44699[#44699], {issue}45544[#45544]) * HLRC for memory usage estimation API {pull}45531[#45531] (issues: {issue}44699[#44699], {issue}45188[#45188]) * Implement ml/data_frame/analytics/_estimate_memory_usage API endpoint {pull}45188[#45188] (issue: {issue}44699[#44699]) [[enhancement-7.4.0]] [float] === Enhancements Aggregations:: * Add more flexibility to MovingFunction window alignment {pull}44360[#44360] (issue: {issue}42181[#42181]) * Optimize Min and Max BKD optimizations {pull}44315[#44315] (issue: {issue}44290[#44290]) * Allow pipeline aggs to select specific buckets from multi-bucket aggs {pull}44179[#44179] Allocation:: * Defer reroute when starting shards {pull}44433[#44433] (issues: {issue}42105[#42105], {issue}42738[#42738]) * Allow RerouteService to reroute at lower priority {pull}44338[#44338] * Auto-release of read-only-allow-delete block when disk utilization fa… {pull}42559[#42559] (issue: {issue}39334[#39334]) Analysis:: * Allow all token/char filters in normalizers {pull}43803[#43803] (issue: {issue}43758[#43758]) Authentication:: * Allow Transport Actions to indicate authN realm {pull}45767[#45767] (issue: {issue}45331[#45331]) * Explicitly fail if a realm only exists in keystore {pull}44471[#44471] (issue: {issue}44207[#44207]) Authorization:: * Add `manage_own_api_key` cluster privilege {pull}45897[#45897] (issue: {issue}40031[#40031]) * Consider `owner` flag when retrieving/invalidating keys with API key service {pull}45421[#45421] (issue: {issue}40031[#40031]) * REST API changes for manage-own-api-key privilege {pull}44936[#44936] (issue: {issue}40031[#40031]) * Simplify API key service API {pull}44935[#44935] (issue: {issue}40031[#40031]) CCR:: * Include leases in error message when operations no longer available {pull}45681[#45681] CRUD:: * Return seq_no and primary_term for noop update {pull}44603[#44603] (issue: {issue}42497[#42497]) Cluster Coordination:: * Improve slow logging in MasterService {pull}45086[#45086] (issue: {issue}45007[#45007]) * More logging for slow cluster state application {pull}45007[#45007] * Ignore unknown fields if overriding node metadata {pull}44689[#44689] * Allow pending tasks before state recovery {pull}44685[#44685] (issue: {issue}44652[#44652]) Data Frame:: * [ML][Data Frame] Add update transform api endpoint {pull}45154[#45154] (issue: {issue}43438[#43438]) * [ML][Data Frame] add support for bucket_selector {pull}44718[#44718] (issues: {issue}43744[#43744], {issue}44557[#44557]) * [ML][Data Frame] adding force delete {pull}44590[#44590] (issue: {issue}43961[#43961]) * [ML][Data Frame] adding dynamic cluster setting for failure retries {pull}44577[#44577] * [ML][Data Frame] Add optional defer_validation param to PUT {pull}44455[#44455] (issue: {issue}43439[#43439]) * [ML][Data Frame] add support for geo_bounds aggregation {pull}44441[#44441] * [ML-DataFrame] Add a frequency option to transform config, default 1m {pull}44120[#44120] Distributed:: * Do not create engine under IndexShard#mutex {pull}45263[#45263] (issue: {issue}43699[#43699]) Docs Infrastructure:: * add clarification around TESTSETUP docu and error message {pull}43306[#43306] Engine:: * Flush engine after big merge {pull}46066[#46066] * Do sync before closeIntoReader when rolling generation to improve index performance {pull}45765[#45765] (issue: {issue}45371[#45371]) * Refactor index engines to manage readers instead of searchers {pull}43860[#43860] * Async IO Processor release before notify {pull}43682[#43682] * Enable indexing optimization using sequence numbers on replicas {pull}43616[#43616] (issue: {issue}34099[#34099]) Features/Features:: * Disable String interning on field names for JSON parsing {pull}41039[#41039] (issue: {issue}39890[#39890]) Features/ILM:: * Add node setting for disabling SLM {pull}46794[#46794] (issue: {issue}38461[#38461]) * Include in-progress snapshot for a policy with get SLM policy API {pull}45245[#45245] * Add option to filter ILM explain response {pull}44777[#44777] (issue: {issue}44189[#44189]) * Expose index age in ILM explain output {pull}44457[#44457] (issue: {issue}38988[#38988]) Features/Indices APIs:: * Add Clone Index API {pull}44267[#44267] (issue: {issue}44128[#44128]) * Add description to force-merge tasks {pull}41365[#41365] (issue: {issue}15975[#15975]) Features/Ingest:: * Fix IngestService to respect original document content type {pull}45799[#45799] * Ingest Attachment: Upgrade tika to v1.22 {pull}45575[#45575] * Retrieve processors instead of checking existence {pull}45354[#45354] * Add ingest processor existence helper method {pull}45156[#45156] * Change the ingest simulate api to not include dropped documents {pull}44161[#44161] (issue: {issue}36150[#36150]) Features/Java High Level REST Client:: * Add XContentType as parameter to HLRC ART#createServerTestInstance {pull}46036[#46036] (issue: {issue}45970[#45970]) * Add CloseIndexResponse to HLRC {pull}44349[#44349] (issue: {issue}39687[#39687]) * Add mapper-extras and the RankFeatureQuery in the hlrc {pull}43713[#43713] (issue: {issue}43634[#43634]) * removing background state update of Request object by RequestConverte… {pull}40156[#40156] (issue: {issue}39666[#39666]) * Add delete aliases API to the high-level REST client {pull}32909[#32909] (issue: {issue}27205[#27205]) Features/Watcher:: * Add SSL/TLS settings for watcher email {pull}45272[#45272] (issue: {issue}30307[#30307]) * Watcher reporting: add email warning if CSV attachment contains values that may be interperted as formulas {pull}44460[#44460] * Watcher add stopped listener {pull}43939[#43939] (issue: {issue}42409[#42409]) * Improve CryptoService error message on missing secure file {pull}43623[#43623] (issue: {issue}43619[#43619]) Geo:: * Support WKT point conversion to geo_point type {pull}44107[#44107] (issue: {issue}41821[#41821]) Infra/Circuit Breakers:: * Fix G1 GC default IHOP {pull}46169[#46169] Infra/Core:: * Add OCI annotations and adjust existing annotations {pull}45167[#45167] (issues: {issue}45162[#45162], {issue}45166[#45166]) * Use the full hash in build info {pull}45163[#45163] (issue: {issue}45162[#45162]) Infra/Packaging:: * Remove redundant Java check from Sys V init {pull}45793[#45793] (issue: {issue}45593[#45593]) * Notify systemd when Elasticsearch is ready {pull}44673[#44673] Infra/Plugins:: * Make plugin verification FIPS 140 compliant {pull}44224[#44224] (issue: {issue}41263[#41263]) Infra/Scripting:: * Whitelist randomUUID in Painless {pull}45148[#45148] (issue: {issue}39080[#39080]) * Add missing ZonedDateTime methods for joda compat layer {pull}44829[#44829] (issue: {issue}44411[#44411]) * Remove stale permissions from untrusted policy {pull}44783[#44783] Infra/Settings:: * Add more meaningful keystore version mismatch errors {pull}46291[#46291] (issue: {issue}44624[#44624]) * Lift the restrictions that uppercase is not allowed in Setting Name. {pull}45222[#45222] (issue: {issue}43835[#43835]) * Normalize environment paths {pull}45179[#45179] (issue: {issue}45176[#45176]) Machine Learning:: * [ML] Support boolean fields for DF analytics {pull}46037[#46037] * [ML] Add description to DF analytics {pull}45774[#45774] * [ML] Add regression analysis to DF analytics {pull}45292[#45292] * Introduce formal node ML role {pull}45174[#45174] (issues: {issue}29943[#29943], {issue}43175[#43175]) * [ML] Improve CSV header row detection in find_file_structure {pull}45099[#45099] (issue: {issue}45047[#45047]) * [ML] Outlier detection should only fetch docs that have the analyzed … {pull}44944[#44944] * Persist DatafeedTimingStats with RefreshPolicy.NONE by default {pull}44940[#44940] (issue: {issue}44792[#44792]) * Add result_type field to TimingStats and DatafeedTimingStats documents {pull}44812[#44812] * Implement exponential average search time per hour statistics. {pull}44683[#44683] (issue: {issue}29857[#29857]) * [ML] Add r_squared eval metric to regression {pull}44248[#44248] * [ML] Adds support for regression.mean_squared_error to eval API {pull}44140[#44140] * Add DatafeedTimingStats.average_search_time_per_bucket_ms and TimingStats.total_bucket_processing_time_ms stats {pull}44125[#44125] (issue: {issue}29857[#29857]) * [ML] Add DatafeedTimingStats to datafeed GetDatafeedStatsAction.Response {pull}43045[#43045] (issue: {issue}29857[#29857]) Network:: * Better logging for TLS message on non-secure transport channel {pull}45835[#45835] (issue: {issue}32688[#32688]) * Asynchronously connect to remote clusters {pull}44825[#44825] (issue: {issue}40150[#40150]) * Improve errors when TLS files cannot be read {pull}44787[#44787] (issue: {issue}43079[#43079]) * Add per-socket keepalive options {pull}44055[#44055] * Move ConnectionManager to async APIs {pull}42636[#42636] Ranking:: * Search enhancement: pinned queries {pull}44345[#44345] (issue: {issue}44074[#44074]) * Fix parameter value for calling data.advanceExact {pull}44205[#44205] * Add l1norm and l2norm distances for vectors {pull}44116[#44116] (issue: {issue}37947[#37947]) Recovery:: * Ignore translog retention policy if soft-deletes enabled {pull}45473[#45473] (issue: {issue}45136[#45136]) * Only retain reasonable history for peer recoveries {pull}45208[#45208] (issue: {issue}41536[#41536]) * Use index for peer recovery instead of translog {pull}45136[#45136] (issues: {issue}38904[#38904], {issue}41536[#41536], {issue}42211[#42211]) * Trim local translog in peer recovery {pull}44756[#44756] * Make peer recovery send file chunks async {pull}44468[#44468] (issues: {issue}36195[#36195], {issue}44040[#44040]) SQL:: * SQL: Support queries with HAVING over SELECT {pull}46709[#46709] (issue: {issue}37051[#37051]) * SQL: Break TextFormatter/Cursor dependency {pull}45613[#45613] (issue: {issue}45516[#45516]) * SQL: remove deprecated use of "interval" from date_histogram usage {pull}45501[#45501] (issue: {issue}43922[#43922]) * SQL: use hasValue() methods from Elasticsearch's InspectionHelper classes {pull}44745[#44745] (issue: {issue}35745[#35745]) * Switch from using docvalue_fields to extracting values from _source {pull}44062[#44062] (issue: {issue}41852[#41852]) Search:: * Adds usage stats for vectors: {pull}44512[#44512] * Associate sub-requests to their parent task in multi search API {pull}44492[#44492] * Cancel search task on connection close {pull}43332[#43332] Security:: * Set security index refresh interval to 1s {pull}45434[#45434] (issue: {issue}44934[#44934]) * Handle pwd protected keystores in all CLI tools {pull}45289[#45289] * Reload secure settings with password {pull}43197[#43197] * Add passphrase support to elasticsearch-keystore {pull}38498[#38498] (issues: {issue}32691[#32691], {issue}37472[#37472]) Snapshot/Restore:: * add disable_chunked_encoding configuration {pull}44052[#44052] * Repository Cleanup Endpoint {pull}43900[#43900] Task Management:: * Remove task null check in TransportAction {pull}45014[#45014] * TaskListener#onFailure to accept Exception instead of Throwable {pull}44946[#44946] * Move child task cancellation to TaskManager {pull}44573[#44573] (issue: {issue}44494[#44494]) [[bug-7.4.0]] [float] === Bug fixes Aggregations:: * Fix early termination of aggregators that run with breadth-first mode {pull}44963[#44963] (issue: {issue}44909[#44909]) * Support BucketScript paths of type string and array. {pull}44694[#44694] (issue: {issue}44385[#44385]) Allocation:: * Avoid overshooting watermarks during relocation {pull}46079[#46079] (issue: {issue}45177[#45177]) * Cluster health should await events plus other things {pull}44348[#44348] * Do not copy initial recovery filter during split {pull}44053[#44053] (issue: {issue}43955[#43955]) Analysis:: * Enable reloading of synonym_graph filters {pull}45135[#45135] (issue: {issue}45127[#45127]) * Fix AnalyzeAction response serialization {pull}44284[#44284] (issue: {issue}44078[#44078]) Authentication:: * Fallback to realm authc if ApiKey fails {pull}46538[#46538] * Enforce realm name uniqueness {pull}46253[#46253] * Allow empty token endpoint for implicit flow {pull}45038[#45038] * Do not use scroll when finding duplicate API key {pull}45026[#45026] * Fix broken short-circuit in getUnlicensedRealms {pull}44399[#44399] * Fix X509AuthenticationToken principal {pull}43932[#43932] (issues: {issue}34396[#34396], {issue}43796[#43796]) Authorization:: * Do not rewrite aliases on remove-index from aliases requests {pull}46989[#46989] * Give kibana user privileges to create APM agent config index {pull}46765[#46765] (issue: {issue}45610[#45610]) * Add `manage_own_api_key` cluster privilege {pull}45696[#45696] (issue: {issue}40031[#40031]) * Sparse role queries can throw an NPE {pull}45053[#45053] CCR:: * Clean up ShardFollowTasks for deleted indices {pull}44702[#44702] (issue: {issue}34404[#34404]) * Skip update if leader and follower settings identical {pull}44535[#44535] (issue: {issue}44521[#44521]) * Avoid stack overflow in auto-follow coordinator {pull}44421[#44421] (issue: {issue}43251[#43251]) * Avoid NPE when checking for CCR index privileges {pull}44397[#44397] (issue: {issue}44172[#44172]) CRUD:: * Ignore replication for noop updates {pull}46458[#46458] (issues: {issue}41065[#41065], {issue}44603[#44603], {issue}46366[#46366]) * Allow _update on write alias {pull}45318[#45318] (issue: {issue}31520[#31520]) * Do not allow version in Rest Update API {pull}43516[#43516] (issue: {issue}42497[#42497]) Cluster Coordination:: * Assert no exceptions during state application {pull}47090[#47090] (issue: {issue}47038[#47038]) * Avoid counting votes from master-ineligible nodes {pull}43688[#43688] Data Frame:: * [ML][Transform] Use field_caps API for mapping deduction {pull}46703[#46703] (issue: {issue}46694[#46694]) * [ML-DataFrame] Fix off-by-one error in checkpoint operations_behind {pull}46235[#46235] * [ML][Data Frame] moves failure state transition for MT safety {pull}45676[#45676] (issue: {issue}45664[#45664]) * [ML][Data Frame] fixing _start?force=true bug {pull}45660[#45660] * [ML][Data frame] fixing failure state transitions and race condition {pull}45627[#45627] (issues: {issue}45562[#45562], {issue}45609[#45609]) * [ML-DataFrame] fix starting a batch data frame after stopping at runtime {pull}45340[#45340] (issues: {issue}44219[#44219], {issue}45339[#45339]) * [ML][Data Frames] Fix null aggregation handling in indexer {pull}45061[#45061] (issue: {issue}44906[#44906]) * [ML][Data Frames] unify validation exceptions between PUT/_preview {pull}44983[#44983] (issue: {issue}44953[#44953]) * [ML][Data Frame] treat bulk index failures as an indexing failure {pull}44351[#44351] (issue: {issue}44101[#44101]) * [ML][Data Frame] prevent task from attempting to run when failed {pull}44239[#44239] (issue: {issue}44121[#44121]) * [ML][Data Frame] responding with 409 status code when failing _stop {pull}44231[#44231] (issue: {issue}44103[#44103]) * [ML][Data Frame] adds index validations to _start data frame transform {pull}44191[#44191] (issue: {issue}44104[#44104]) * [ML] Data frame task failure do not make a 500 response {pull}44058[#44058] (issue: {issue}44011[#44011]) * [ML-DataFrame] audit message missing for autostop {pull}43984[#43984] (issue: {issue}43977[#43977]) Distributed:: * Fix false positive out of sync warning in synced-flush {pull}46576[#46576] (issues: {issue}28464[#28464], {issue}30244[#30244]) * Suppress warning logs from background sync on relocated primary {pull}46247[#46247] (issues: {issue}40800[#40800], {issue}42241[#42241]) * Ensure AsyncTask#isScheduled remain false after close {pull}45687[#45687] (issue: {issue}45576[#45576]) * Update translog checkpoint after marking operations as persisted {pull}45634[#45634] (issue: {issue}29161[#29161]) * Fix clock used in update requests {pull}45262[#45262] (issue: {issue}45254[#45254]) * Restore DefaultShardOperationFailedException's reason during deserialization {pull}45203[#45203] * Use IndicesModule named writables in elasticsearch-shard tool {pull}45036[#45036] (issue: {issue}44628[#44628]) Engine:: * Handle delete document level failures {pull}46100[#46100] (issue: {issue}46083[#46083]) * Handle no-op document level failures {pull}46083[#46083] * Remove leniency during replay translog in peer recovery {pull}44989[#44989] * Throw TranslogCorruptedException in more cases {pull}44217[#44217] * Fail engine if hit document failure on replicas {pull}43523[#43523] (issues: {issue}40435[#40435], {issue}43228[#43228]) Features/ILM:: * Handle partial failure retrieving segments in SegmentCountStep {pull}46556[#46556] * Fixes for API specification {pull}46522[#46522] * Fix SnapshotLifecycleMetadata xcontent serialization {pull}46500[#46500] (issue: {issue}46499[#46499]) * Updated slm API spec parameters and URL {pull}44797[#44797] * Fix swapped variables in error message {pull}44300[#44300] Features/Indices APIs:: * Fix a bug where mappings are dropped from rollover requests. {pull}45411[#45411] (issue: {issue}45399[#45399]) * Create index with typeless mapping {pull}45120[#45120] * Check shard limit after applying index templates {pull}44619[#44619] (issues: {issue}34021[#34021], {issue}44567[#44567], {issue}44619[#44619]) * Validate index settings after applying templates {pull}44612[#44612] (issues: {issue}34021[#34021], {issue}44567[#44567]) Features/Ingest:: * Allow dropping documents with auto-generated ID {pull}46773[#46773] (issue: {issue}46678[#46678]) Features/Java High Level REST Client:: * [HLRC] Send min_score as query string parameter to the count API {pull}46829[#46829] (issue: {issue}46474[#46474]) * HLRC multisearchTemplate forgot params {pull}46492[#46492] (issue: {issue}46488[#46488]) * terminateAfter added to the RequestConverter {pull}46474[#46474] (issue: {issue}46446[#46446]) * [Closes #44045] Added 'slices' parameter when submitting reindex request via Java high level REST client {pull}45690[#45690] (issue: {issue}44045[#44045]) * HLRC: Fix '+' Not Correctly Encoded in GET Req. {pull}33164[#33164] (issue: {issue}33077[#33077]) Features/Watcher:: * Fix class used to initialize logger in Watcher {pull}46467[#46467] * Fix wrong URL encoding in watcher HTTP client {pull}45894[#45894] (issue: {issue}44970[#44970]) * Fix watcher HttpClient URL creation {pull}45207[#45207] (issue: {issue}44970[#44970]) * Log write failures for watcher history document. {pull}44129[#44129] Geo:: * Geo: fix geo query decomposition {pull}44924[#44924] (issue: {issue}44891[#44891]) * Geo: add validator that only checks altitude {pull}43893[#43893] Highlighting:: * Fix highlighting for script_score query {pull}46507[#46507] (issue: {issue}46471[#46471]) Infra/Core:: * Always check that cgroup data is present {pull}45606[#45606] (issue: {issue}45396[#45396]) * Safe publication of DelayedAllocationService and SnapshotShardsService {pull}45517[#45517] (issue: {issue}38560[#38560]) * Add default CLI JVM options {pull}44545[#44545] (issues: {issue}219[#219], {issue}42021[#42021]) * Fix decimal point parsing for date_optional_time {pull}43859[#43859] (issue: {issue}43730[#43730]) Infra/Logging:: * Fix types field in JSON Search Slow Logs {pull}44641[#44641] * Add types field to JSON slow logs in 7.x {pull}44592[#44592] (issues: {issue}41354[#41354], {issue}44178[#44178]) Infra/Packaging:: * Add destructiveDistroTest meta task {pull}45762[#45762] * Use bundled JDK in Sys V init {pull}45593[#45593] (issue: {issue}45542[#45542]) * Restore setting up temp dir for windows service {pull}44541[#44541] Infra/Plugins:: * Do not checksum all bytes at once in plugin install {pull}44649[#44649] (issue: {issue}44545[#44545]) Infra/REST API:: * Improve error message when index settings are not a map {pull}45588[#45588] (issue: {issue}45126[#45126]) * Add is_write_index column to cat.aliases {pull}44772[#44772] * Fix URL documentation in API specs {pull}44487[#44487] Infra/Scripting:: * Fix bugs in Painless SCatch node {pull}45880[#45880] * Fix JodaCompatibleZonedDateTime casts in Painless {pull}44874[#44874] Infra/Settings:: * bug fix about elasticsearch.common.settings.Settings.processSetting {pull}44047[#44047] (issue: {issue}43791[#43791]) Machine Learning:: * [ML] fix two datafeed flush lockup bugs {pull}46982[#46982] * [ML] Throw an error when a datafeed needs CCS but it is not enabled for the node {pull}46044[#46044] (issue: {issue}46025[#46025]) * Handle "null" value of Estimate memory usage API response gracefully. {pull}45726[#45726] (issue: {issue}44699[#44699]) * [ML] Remove timeout on waiting for DF analytics result processor to complete {pull}45724[#45724] (issue: {issue}45723[#45723]) * [ML] Check dest index is empty when starting DF analytics {pull}45094[#45094] * [ML] Catch any error thrown while closing data frame analytics process {pull}44958[#44958] * Treat PostDataActionResponse.DataCounts.bucketCount as incremental rather than absolute (total). {pull}44803[#44803] (issue: {issue}44792[#44792]) * Treat big changes in searchCount as significant and persist the document after such changes {pull}44413[#44413] (issues: {issue}44196[#44196], {issue}44335[#44335]) * Update .ml-config mappings before indexing job, datafeed or df analytics config {pull}44216[#44216] (issue: {issue}44263[#44263]) * [ML] Wait for .ml-config primary before assigning persistent tasks {pull}44170[#44170] (issue: {issue}44156[#44156]) * [ML] Fix ML memory tracker lockup when inner step fails {pull}44158[#44158] (issue: {issue}44156[#44156]) * [ML] Fix datafeed checks when a concrete remote index is present {pull}43923[#43923] (issue: {issue}42113[#42113]) Mapping:: * Make sure to validate the type before attempting to merge a new mapping. {pull}45157[#45157] (issues: {issue}29316[#29316], {issue}43012[#43012]) * Ensure field caps doesn't error on rank feature fields. {pull}44370[#44370] (issue: {issue}44330[#44330]) * Prevent types deprecation warning for indices.exists requests {pull}43963[#43963] (issue: {issue}43905[#43905]) * Add include_type_name in indices.exitst REST API spec {pull}43910[#43910] (issue: {issue}43905[#43905]) Network:: * Fix Broken HTTP Request Breaking Channel Closing {pull}45958[#45958] (issues: {issue}43362[#43362], {issue}43850[#43850]) * Fix plaintext on TLS port logging {pull}45852[#45852] (issue: {issue}32688[#32688]) * transport.publish_address should contain CNAME {pull}45626[#45626] (issues: {issue}32806[#32806], {issue}39970[#39970]) * Fix bug in copying bytes for socket write {pull}45463[#45463] (issue: {issue}45444[#45444]) Recovery:: * Never release store using CancellableThreads {pull}45409[#45409] (issues: {issue}45136[#45136], {issue}45237[#45237]) * Remove leniency in reset engine from translog {pull}44711[#44711] Rollup:: * Fix Rollup job creation to work with templates {pull}43943[#43943] SQL:: * SQL: Properly handle indices with no/empty mapping {pull}46775[#46775] (issue: {issue}46757[#46757]) * SQL: improve ResultSet behavior when no rows are available {pull}46753[#46753] (issue: {issue}46750[#46750]) * SQL: use the correct data type for types conversion {pull}46574[#46574] (issue: {issue}46090[#46090]) * SQL: Fix issue with common type resolution {pull}46565[#46565] (issue: {issue}46551[#46551]) * SQL: fix scripting for grouped by datetime functions {pull}46421[#46421] (issue: {issue}40241[#40241]) * SQL: Use null schema response {pull}46386[#46386] (issue: {issue}46381[#46381]) * SQL: Fix issue with IIF function when condition folds {pull}46290[#46290] (issue: {issue}46268[#46268]) * SQL: Fix issue with DataType for CASE with NULL {pull}46173[#46173] (issue: {issue}46032[#46032]) * SQL: adds format parameter to range queries for constant date comparisons {pull}45326[#45326] (issue: {issue}45139[#45139]) * SQL: uniquely named inner_hits sections for each nested field condition {pull}45039[#45039] (issues: {issue}33080[#33080], {issue}44544[#44544]) * SQL: fix URI path being lost in case of hosted ES scenario {pull}44776[#44776] (issue: {issue}44721[#44721]) * SQL: change the size of the list of concrete indices when resolving multiple indices {pull}43878[#43878] (issue: {issue}43876[#43876]) * SQL: handle double quotes escaping {pull}43829[#43829] (issue: {issue}43810[#43810]) * SQL: add pretty printing to JSON format {pull}43756[#43756] * SQL: handle SQL not being available in a more graceful way {pull}43665[#43665] (issue: {issue}41279[#41279]) Search:: * Multi-get requests should wait for search active {pull}46283[#46283] (issue: {issue}27500[#27500]) * Ensure top docs optimization is fully disabled for queries with unbounded max scores. {pull}46105[#46105] (issue: {issue}45933[#45933]) * Disallow partial results when shard unavailable {pull}45739[#45739] (issue: {issue}42612[#42612]) * Prevent Leaking Search Tasks on Exceptions in FetchSearchPhase and DfsQueryPhase {pull}45500[#45500] * Fix an NPE when requesting inner hits and _source is disabled. {pull}44836[#44836] (issue: {issue}43517[#43517]) * Don't use index_phrases on graph queries {pull}44340[#44340] (issue: {issue}43976[#43976]) Security:: * Initialize document subset bit set cache used for DLS {pull}46211[#46211] (issue: {issue}45147[#45147]) * Fix .security-* indices auto-create {pull}44918[#44918] * SecurityIndexManager handle RuntimeException while reading mapping {pull}44409[#44409] * Do not swallow I/O exception getting authentication {pull}44398[#44398] (issues: {issue}44172[#44172], {issue}44397[#44397]) * Use system context for looking up connected nodes {pull}43991[#43991] (issue: {issue}43974[#43974]) Snapshot/Restore:: * Fix Bug in Snapshot Status Response Timestamps {pull}46919[#46919] (issue: {issue}46913[#46913]) * GCS deleteBlobsIgnoringIfNotExists should catch StorageException {pull}46832[#46832] (issue: {issue}46772[#46772]) * Fix TransportSnapshotsStatusAction ThreadPool Use {pull}45824[#45824] * Stop Executing SLM Policy Transport Action on Snapshot Pool {pull}45727[#45727] (issue: {issue}45594[#45594]) * Check again on-going snapshots/restores of indices before closing {pull}43873[#43873] * Make Timestamps Returned by Snapshot APIs Consistent {pull}43148[#43148] (issue: {issue}43074[#43074]) * Recursively Delete Unreferenced Index Directories {pull}42189[#42189] (issue: {issue}13159[#13159]) Task Management:: * Catch AllocatedTask registration failures {pull}45300[#45300] [[regression-7.4.0]] [float] === Regressions Aggregations:: * Implement rounding optimization for fixed offset timezones {pull}46670[#46670] (issue: {issue}45702[#45702]) [[upgrade-7.4.0]] [float] === Upgrades Infra/Core:: * Update joda to 2.10.3 {pull}45495[#45495] Infra/Packaging:: * Upgrade to JDK 12.0.2 {pull}45172[#45172] Network:: * Upgrade to Netty 4.1.38 {pull}45132[#45132] Search:: * Upgrade to lucene snapshot 8.3.0-snapshot-8dd116a6158 {pull}45604[#45604] (issue: {issue}43976[#43976])