Currently, when a user tries to install an old plugin (pre 2.x) on a 2.x
node, the error message is cryptic (just printing the file path that was
missing, when looking for the descriptor). This improves the message to
be more explicit that the descriptor is missing, and suggests the
problem might be the plugin was built before 2.0.
closes#15197
This commit addresses some issues that arose during the review of #14899
but were lost during squash while integrating into master.
- the number of test threads is dropped to at most eight
- a local variable is renamed for clarity
- task priorities are randomized
This commit fixes a test bug in
ClusterService#testClusterStateBatchedUpdates. In particular, in the
case that an executor did not receive a task assignment from the random
assignments, it would not have an entry in the map of executors to
counts of assigned tasks. The fix is to just check if each executor has
an entry in the counts map.
This commit modifies IndexingMemoryController to be stateless. Rather
than statefully tracking the indexing status of shards,
IndexingMemoryController can grab all available shards, check their idle
state, and then resize the buffers based on the number of and which
shards are not idle.
The driver for this change is a performance regression that can arise in
some scenarios after #13918. One scenario under which this performance
regression can arise is if an index is deleted and then created
again. Because IndexingMemoryController was previously statefully
tracking the state of shards via a map of ShardIds, the new shards with
the same ShardIds as previously existing shards would not be detected
and therefore their version maps would never be resized from the
defaults. This led to an explosion in the number of merges causing a
degradation in performance.
Closes#15225
Today we only check mapping compatibility when adding mappers to the
lookup structure. However, at this stage, the mapping has already been merged
partially, so we can leave mappings in a bad state. This commit removes the
compatibility check from Mapper.merge entirely and performs it _before_ we
call Mapper.merge.
One minor regression is that the exception messages don't group together errors
that come from MappedFieldType.checkCompatibility and Mapper.merge. Since we
run the former before the latter, Mapper.merge won't even have a chance to let
the user know about conflicts if conflicts were discovered by
MappedFieldType.checkCompatibility.
Close#15049
The `translated` flag makes LineStringBuilder stateful and gets set
to true under certain conditions when building a Shape or Geometry
from the ShapeBuilder. This makes building operations not be idempotent,
so calling build() more than once on a LineStringBuilder might change the
builder itself. This PR fixes this by replacing the instance variable by
a local `translated` flag that is only updated internally during the
building process and created again on any subsequent calls to build()
or buildGeometry().
Failures to merge a mapping can either come as a MergeMappingException if they
come from Mapper.merge or as an IllegalArgumentException if they come from
FieldTypeLookup.checkCompatibility. I think we should settle on one: this pull
request replaces all usage of MergeMappingException with
IllegalArgumentException.
The ttl could be specified as a time value only via the REST layer. That is now possible via java api too, either as a string or as a proper TimeValue. The internal format in IndexRequest becomes now TimeValue, which will then still converted to a long before storing the document.
Closes#15047
- Supports ImmutableOpenIntMap besides java.util.Map and ImmutableOpenMap
- Map keys can be any value (not only String)
- Map values do not have to implement Diffable interface. In that case custom value serializer needs to be provided.
Several settings have been deprecated or are replaced with new settings after refactorings
in version 1.x. This commit removes the support for these settings.
The settings are:
* `index.shard.recovery.translog_size`
* `index.shard.recovery.translog_ops`
* `index.shard.recovery.file_chunk_size`
* `index.shard.recovery.concurrent_streams`
* `index.shard.recovery.concurrent_small_file_streams`
* `indices.recovery.max_size_per_sec`
When not in debug mode, we currently only print the message of an
exception. However, this is not usually useful without knowing what the
exception type was. This change makes cli tools use toString() on the
exception so we get the type + message.
This commit wraps the trace logging statements in
TransportBroadcastByNodeAction in trace enabled checks to avoid
unnecessarily allocating objects.
The most egregious offenders were the two trace logging statements in
BroadcastByNodeTransportRequestHandler#onShardOperation. Aside from the
usual object allocations that occur when invoking ESLogger#trace (the
allocated object array for the varargs Object... parameter), these two
logging statements were invoking ShardRouting#shortSummary generating a
bunch of char arrays and Strings (from the StringBuilder, and so a bunch
of array copies as well). In a scenario where there are a lot of shards
and this method is being invoked frequently (e.g., constantly hitting
the _stats endpoint), these two unprotected trace logging statements
were generating a lot of unnecessary allocations.
This commit modifies the handling of shard started cluster state updates
to use the general cluster state batching mechanism. An advantage of
this approach is we now get correct per-listener notification on
failures.
This commit removes a simple early-out check in
MetaDataMappingService#executeRefresh. The early-out is unnecessary
because the cluster state task execution framework will not invoke
ClusterStateTaskExecutor#execute if the list of tasks is empty.
This commit updates a stale Javadoc on
MetaDataMappingService#executeRefresh. Previously this method handled
refresh and update tasks. Update tasks have been removed and the method
was renamed, but the Javadoc was not updated to reflect this.
This is due to the fact that the query cache will still call the
onDocIdSetEviction callback in this case but with a number of entries equal to
zero.
Close#15043
This commit removes and now forbids all uses of the type-unsafe empty
Collections fields Collections#EMPTY_LIST, Collections#EMPTY_MAP, and
Collections#EMPTY_SET. The type-safe methods Collections#emptyList,
Collections#emptyMap, and Collections#emptySet should be used instead.
Today we try to have type-level granularity when dealing with mappings. This
does not play well with the cross-type validations that we are adding. For
instance we prevent the `_parent` field to point to an existing type. This
validation would be skipped today in the case of dedicated master nodes, since
those master nodes would only create the type that is being updated when
updating a mapping.
In 1.x it is possible via index templates to create an index with an alias with the same name as the index. The index name must match the index template and have an alias with the same name as the index being created.
This change attempts to simplify the gradle tasks for precommit. One
major part of that is using a "less groovy style", as well as being more
consistent about how tasks are created and where they are configured. It
also allows the things creating the tasks to set up inter task
dependencies, instead of assuming them (ie decoupling from tasks
eleswhere in the build).
This adds safety that you can't index into the `_default_` type (it was possible
before), and can't add default mappers to the field type lookups (was not
happening in tests but I think this is still a good check).
Also MapperService.types() now excludes `_default` so that eg. the `ids` query
does not try to search on this type anymore.
This commit modifies the handling of shard failure cluster state updates
to use the general cluster state batching mechanism. An advantage of
this approach is we now get correct per-listener notification on
failures.
This change pulls out the common fields that HighlighterBuilder shares with
its nested Field class into a new abstract CommonHighlighterOptions superclass
which also gets equals() and hashCode() method and methods to serialize the
common fields to a StreamOutput and read them from a stream.
Relates to #15044
Validation is not done as part of the distance setter method and tested in GeoDistanceQueryBuilderTests. Fixed GeoDistanceTests to adapt to the new validation.
Closes#15135
When creating an index on master for the purpose of updating mappings, a
mapping being updated could needlessly be merged multiple times. This
commit ensures that each mapping is merged at most once while preparing
to update mappings.
When creating an index on master for the purpose of updating mappings,
the default mapping could needlessly be added multiple times. This
commit ensures that the default mapping is added at most once while
preparing to update mappings.
This commit addresses an issues introduced in #14899 to apply mapping
updates in batches. The issue is that an existing mapping for a type
could be lost if that type came in a batch that already contained a
mapping update for another type on the same index. The underlying issue
was that the existing mapping would not be merged in because the merging
logic was only tripped once per index, rather than for all types seeing
updates for each index. Resolving this issue is simply a matter of
ensuring that all existing types seeing updates are merged in.
Closes#15129
The REST bulk API rejects use of `refresh` at the item level. But the Java API lets the user setting it.
We need to have the same behavior and don't let think the user he can define `refresh` per bulk item.
Note that the user can still define `refresh` on the bulk itself.
Also a user can create with Java API an IndexRequest without any source which is causing a NPE when evaluating the bulk item size.
Closes#7361.
Closes#15120.
I will followup with ITs and other modules. By fixing this, these tests become more reliable (will never sporatically
fail due to other stuff on your machine: ports are assigned by the OS), and it allows us to move forward with
gradle parallel builds, in my tests this is a nice speedup, but we can't do it until tests are cleaned up
This commit splits cluster state update tasks into roles. Those roles
are:
- task info
- task configuration
- task executor
- task listener
All tasks that have the same executor will be executed in batches. This
removes the need for local batching as was previously in
MetaDataMappingService.
Additionally, this commit reintroduces batching on mapping update calls.
Relates #13627
Do not to load fields from _source when using the `fields` option.
Non stored (non existing) fields are ignored by the fields visitor when using the `fields` option.
Fixes#10783
Support * wildcard to retrieve stored fields when using the `fields` option.
Supported pattern styles are "xxx*", "*xxx", "*xxx*" and "xxx*yyy".
Ryan Ernst
Robert Muir
Jason Tedor
Robert Muir
Lee Hinman
Adrien Grand
Yannick Welsch
Christoph Büscher
javanna
Simon Willnauer
Martijn van Groningen
Boaz Leskes
Areek Zillur
David Pilato
Britta Weber
Jim Ferenczi
Jim Ferenczi
Michael McCandless