When need to check the whole hierarchy of objects to know if the newly inserted "newField" object is part of an arbitrary holding object or not.
Reproduced with `gradle :modules:percolator:test -Dtests.seed=736B0B67DA7A3632 -Dtests.class=org.elasticsearch.percolator.PercolateQueryBuilderTests -Dtests.method="testUnknownObjectException" -Dtests.security.manager=true -Dtests.locale=es-ES -Dtests.timezone=ART`
Previous to this change the nesting of aggregation profiling results
would be incorrect when the request contains a terms aggregation and the
collect mode is (implicitly or explicitly) set to `breadth_first`. This
was because the aggregation profiling has to make the assumption that
the `preCollection()` method of children aggregations is always called in
the `preCollection()` method of their parent aggregation. When the collect
mode is `breadth_first` the `preCollection` of the children aggregations
was delayed until the documents were replayed.
This change moves the `preCollection()` of deferred aggregations to run
during the `preCollection()` of the parent aggregation. This should have
no adverse impact on the breadth_first mode as there is no allocation of
memory in any of the aggregations.
We also apply the same logic to the diversified sampler aggregation as
we did to the terms aggregation to move the `preCollection()` of the
child aggregations method to be called during the `preCollection()` of
the parent aggregation.
This commit also includes a fix so that the `ProfilingLeafBucketCollector`
propagates the scorer to its delegate so the diversified sampler agg works
when profiling is enabled.
If they are specified by a mapping update, these properties are currently
ignored. This commit also fixes the handling of `dynamic_templates` so that it
is possible to remove templates (and so that it works more similarly to all
other mapping properties).
Closes#20111
This method fails when a randomized string value contains a double-quote. This commit changes the method so that it is not based on string concatenation anymore. It now use XContentGenerator & XContentParser to mutate the valid queries.
Related #19864
This is a house cleaning commit that refactors GeoPointFieldMapperLegacy to LegacyGeoPointFieldMapper for consistency with Legacy Numerics and IP field mappers.
IndexedGeoBoundingBoxQuery and InMemoryGeoBoundingBoxQuery are also deprecated and refactored as Legacy classes.
This change adds a special field named _none_ that allows to disable the retrieval of the stored fields in a search request or in a TopHitsAggregation.
To completely disable stored fields retrieval (including disabling metadata fields retrieval such as _id or _type) use _none_ like this:
````
POST _search
{
"stored_fields": "_none_"
}
````
Today we do a lot of accounting inside the engine to maintain locations
of documents inside the transaction log. This is only needed to ensure
we can return the documents source from the engine if it hasn't been refreshed.
Aside of the added complexity to be able to read from the currently writing translog,
maintainance of pointers into the translog this also caused inconsistencies like different values
of the `_ttl` field if it was read from the tlog or not. TermVectors are totally different if
the document is fetched from the tranlog since copy fields are ignored etc.
This chance will simply call `refresh` if the documents latest version is not in the index. This
streamlines the semantics of the `_get` API and allows for more optimizations inside the engine
and on the transaction log. Note: `_refresh` is only called iff the requested document is not refreshed
yet but has recently been updated or added.
#Relates to #19787
I was writing tests for RAM usage estimation of LiveVersionMap and found a
couple issues:
- The BytesRef objects used as uids were oversized since they were created
via `new BytesRef(CharSequence)` which creates a `byte[]` whose size is 3x
the length of the provided char sequence. Given that our uids are most of
times ASCII sequences, this is a waste of memory.
- `VersionValue` was using `translogLocation.size` instead of
`translogLocation.ramBytesUsed()` for RAM estimation, which is completely
unrelated to the memory footprint of the `Translog.Location` object.
In particular, the latter issue could cause RAM usage estimation to be
significantly overestimated, especially on large documents.
I also added tests for ram accounting.
This was previously broken because run and integTest used the same
configuration name. This change makes the configuration name prefixed by
the task the cluster is created for.
Now that Elasticsearch defaults to 2gb heaps those boxes need more ram,
especially opensuse-13 for some reason. It has failed to run the
packaging tests ever since the change.....
Deprecates the optimize_bbox parameter on geodistance queries. This has no longer been needed since version 2.2 because lucene geo distance queries (postings and LatLonPoint) already optimize by bounding box.
In 1e91f3b we disabled annotation processors globally. However, some
project like JMH need annotation processing, so we add an ability to
selectively enabled annotation processing for certain projects by
setting an external property in the corresponding Gradle build script.
Note that `javac` would allow to set a specific annotation processor
with the command line option `-processor`. However, due to a bug in
Gradle we we cannot use this option and need to enable all annotation
processors.
Fix field examples to make documents actually visible
This commit adds refresh calls to field examples an removes not working
`_routing` and `_field_names` script access.
Closes#20118
Catching and suppressing AlreadyClosedException from Lucene is dangerous because it can mean there is a bug in ES since ES should normally guard against invoking Lucene classes after they were closed.
I reviewed the cases where we catch AlreadyClosedException from Lucene and removed the ones that I believe are not needed, or improved comments explaining why ACE is OK in that case.
I think (@s1monw can you confirm?) that holding the engine's readLock means IW will not be closed, except if disaster strikes (failEngine) at which point I think it's fine to see the original ACE in the logs?
Closes#19861
When a SearchContext is closed it's reader / searcher reference is closed too.
If this happens while a search is accessing it's reader reference it can lead
to an unexpected `AlreadyClosedException` or worst case, an already closed MMapDirectory
is access causing a `SIGSEV` like in #20008 (even though the window for this is very small).
SearchContext can be closed concurrently if:
* an index is deleted / removed from the node
* a search context is idle for too long and is cleaned by the reaper
* an explicit freeContext message is received
This change adds reference counting to the SearchContext base class and it's used
inside SearchService each time the context is accessed.
Closes#20008
the setting in elasticsearch.yml, so that when a user uncomments
out a setting by just removing the #, the setting actually
takes effect. Before, it was very easy to uncomment out a
setting by just removing the #, leaving a single whitespace
character before the setting name, which would cause the
setting to not get picked up by Elasticsearch.
Closes#20090
Jack Conradson
Tanguy Leroux
Colin Goodheart-Smithe
Adrien Grand
Dominik Stadler
Michael McCandless
Mike McCandless
Igor Motov
Jim Ferenczi
Nicholas Knize
Ryan Ernst
Clinton Gormley
Simon Willnauer
Jason Tedor
Nik Everett
ddddn
Lee Hinman
Daniel Mitterdorfer
Masaru Hasegawa
Thiago Souza
Munish Goyal
Ali Beyad