Resolve index names to Index instances early
Today index names are often resolved lazily, only when they are really
needed. This can be problematic especially when it gets to mapping updates
etc. when a node sends a mapping update to the master but while the request
is in-flight the index changes for whatever reason we would still apply the update
since we use the name of the index to identify the index in the clusterstate.
The problem is that index names can be reused which happens in practice and sometimes
even in a automated way rendering this problem as realistic.
In this change we resolve the index including it's UUID as early as possible in places
where changes to the clusterstate are possible. For instance mapping updates on a node use a
concrete index rather than it's name and the master will fail the mapping update iff
the index can't be found by it's <name, uuid> tuple.
Closes#17048
Today index names are often resolved lazily, only when they are really
needed. This can be problematic especially when it gets to mapping updates
etc. when a node sends a mapping update to the master but while the request
is in-flight the index changes for whatever reason we would still apply the update
since we use the name of the index to identify the index in the clusterstate.
The problem is that index names can be reused which happens in practice and sometimes
even in a automated way rendering this problem as realistic.
In this change we resolve the index including it's UUID as early as possible in places
where changes to the clusterstate are possible. For instance mapping updates on a node use a
concrete index rather than it's name and the master will fail the mapping update iff
the index can't be found by it's <name, uuid> tuple.
Closes#17048
Changes:
- no more option to configure eager/lazy loading of the norms (useless now
that orms are disk-based)
- only the `string`, `text` and `keyword` fields support the `norms` setting
- the `norms` setting takes a boolean that decides whether norms should be
stored in the index but old options are still supported to give users time
to upgrade
- setting a `boost` no longer implicitely enables norms (for new indices only,
this is still needed for old indices)
To make API's output more easy to read we are suppressing stack traces (#12991) unless explicitly requested by setting `error_trace=true` on the request. To compensate we are logging the stacktrace into the logs so people can look it up even the error_trace wasn't enabled. Currently we do so using the `INFO` level which can be verbose if an api is called repeatedly by some automation. For example, if someone tries to read from an index that doesn't exist we will respond with a 404 exception and log under info every time. We should reduce the level to `DEBUG` as we do with other API driven errors. Internal errors (rest codes >=500) are logged as WARN.
Closes#16627
The CONTRIBUTING.md file can be in the root directory or
in the .github directory and will still be used for
the contributing guidelines on Github.
Moved back to the root directory so that it is more
visible outside Github
When unzipping a plugin zip, the zip entries are resolved relative to
the directory being unzipped into. However, there are currently no
checks that the entry name was not absolute, or relatively points
outside of the plugin dir. This change adds a check for those two cases.
Today we use hardcoded ports to form a cluster in the mulit-node case.
The hardcoded URIs are passed to the unicast host list which is error prone and
might cause problems if those ports are exhausted etc. This commit moves to a
less error prone way of forming the cluster where all nodes are started with port `0`
and all but the first node wait for the first node to write it's ports file to form a
cluster. This seed node is enough to form a cluster.
This commit adds fields bytes_recovered and files_recovered to the cat
recovery API. These fields, respectively, indicate the total number of
bytes and files recovered. Additionally, for consistency, some totals
fields and translog recovery fields have been renamed.
Closes#17064
This tries to remove friction to upgrade to 5.0 that would be caused by mapping
changes:
- old ways to specify mapping settings (eg. store: yes instead of store:true)
will still work but a deprecation warning will be logged
- string mappings that only use the most common options will be upgraded
automatically to text/keyword
The change to move dynamic mapping handling to the end of document
parsing has an edge case which can cause dynamic mappings to fail
document parsing. If field a.b is added as an as part of the root update,
followed by a.c.d, then we need to expand the mappers on the stack,
since a is hidden inside the root update which exists on the stack.
This change adds a test for this case, as well as tries to better
document how the logic works for building up the stack before adding a
dynamic mapper.
The refresh tests were failing rarely due to refreshes happening
automatically on indexes with -1 refresh intervals. This commit moves
the refresh test into a unit test where we can check if it was attempted
so we never get false failures from background refreshes.
It also stopped refresh from being run if the reindex request was canceled.
Simon Willnauer
Clinton Gormley
Simon Willnauer
Adrien Grand
Jason Tedor
Boaz Leskes
David Pilato
Yannick Welsch
Ryan Ernst
Daniel Mitterdorfer
Ed Winn
Nik Everett
Zachary Tong