In several places in the code we need to notify a node it needs to do something (typically the master). When that node is the local node, we have an optimization in serveral places that runs the execution code immediately instead of sending the request through the wire to itself. This is a shame as we need to implement the same pattern again and again. On top of that we may forget (see note bellow) to do so and we might have to write some craft if the code need to run under another thread pool.
This commit folds the optimization in the TrasnportService, shortcutting wire serliazition if the target node is local.
Note: this was discovered by #10247 which tries to import a dangling index quickly after the cluster forms. When sending an import dangling request to master, the code didn't take into account that fact that the local node may master. If this happens quickly enough, one would get a NodeNotConnected exception causing the dangling indices not to be imported. This will succeed after 10s where InternalClusterService.ReconnectToNodes runs and actively connects the local node to itself (which is not needed), potentially after another cluster state update.
Closes#10350
The exceptionCaught method had default access, which imposes a requirement
for subclasses that need to override this method to be in a specific package. This
change simply makes the method protected, which removes the package requirement.
We still have a lot of APIs that use setNextReader in order to change the
current segment that should be considered. This commit moves such APIs to
getLeafXXX() instead to be more in-line with Lucene 5's collector API.
I also renamed setDocId to setDocument to be more in-line with the doc values
APIs.
Close#10389
Relates to #10154 and #10150
Adds link to additional information on how document frequencies are treated across shards to the cutoff_frequency parameter documentation.
Closes#10451
These tests create artificial hash collisions in order to make sure that they
can be resolved correctly. But this also makes the tests very slow if there
are too many collisions because insertions/deletions become linear in such
cases. The tests have been modified to not do too many iterations when
collisions are likely.
Close#10442
Closes#10435.
Squashed commit of the following:
commit aa1935c790b2731fc2bbc7de6142b09e3fe8bd4a
Author: Ryan Ernst <ryan@iernst.net>
Date: Mon Apr 6 13:44:40 2015 -0700
fix index lookup
commit bb6373595ff62ffc56fdf0cba3ac9c0ebe679946
Merge: 916962b eb3a170
Author: Robert Muir <rmuir@apache.org>
Date: Mon Apr 6 14:24:38 2015 -0400
Merge branch 'lucene_r1671277' of github.com:elasticsearch/elasticsearch into lucene_r1671277
commit 916962b82d192a53add471b4cc4a1396bc30eb0e
Merge: 197b3a2 21f72fe
Author: Robert Muir <rmuir@apache.org>
Date: Mon Apr 6 07:09:41 2015 -0400
Merge branch 'master' into lucene_r1671277
commit eb3a1703f7932ddd0cf3e83bec0e86131d255407
Author: Ryan Ernst <ryan@iernst.net>
Date: Sat Apr 4 11:06:03 2015 -0700
re-enable index lookup tests
commit 80d65d5eab39062dd8364687da74ddbb87ebcb76
Author: Ryan Ernst <ryan@iernst.net>
Date: Sat Apr 4 10:39:52 2015 -0700
update pom to point to new snapshot repo
commit 197b3a21ac2c2d70c9f740fe53e58632a22d1aad
Author: Robert Muir <rmuir@apache.org>
Date: Sat Apr 4 12:51:22 2015 -0400
fix postingsenum usage
commit 0e2b7a00cd07d068f755c51185ac521aa1eb0326
Author: Robert Muir <rmuir@apache.org>
Date: Sat Apr 4 12:21:23 2015 -0400
upgrade to lucene r1671277 (have not yet run tests or looked at postings changes)
The current implementation of AbstractBlobContainer.deleteByPrefix() calls AbstractBlobContainer.deleteBlobsByFilter() which calls BlobContainer.listBlobs() for deleting files, resulting in loading all files in order to delete few of them. This can be improved by calling BlobContainer.listBlobsByPrefix() directly.
This problem happened in #10344 when the repository verification process tries to delete a blob prefixed by "tests-" to ensure that the repository is accessible for the node. When doing so we have the following calling graph: BlobStoreRepository.endVerification() -> BlobContainer.deleteByPrefix() -> AbstractBlobContainer.deleteByPrefix() -> AbstractBlobContainer.deleteBlobsByFilter() -> BlobContainer.listBlobs()... and boom.
Also, AbstractBlobContainer.listBlobsByPrefix() and BlobContainer.deleteBlobsByFilter() can be removed because it has the same drawbacks as AbstractBlobContainer.deleteByPrefix() and also lists all blobs. Listing blobs by prefix can be done at the FsBlobContainer level.
Related to #10344
I do not know normally whether this section is the right place or not, please let me know.
Here is the pes plugin's GitHub site :https://github.com/kodcu/pesCloses#10398
The static old index tests currently take a long time to run because
each index version essentially recreates the cluster, and spins up
new nodes. This PR instead loads each old version into the existing
cluster as a dangling index. It also removes the intermediate
"StaticIndexBackwardCompatibilityTest" which was an extra layer
with no purpose, and moves a shared version of a commonly found
function to get an http client.
The test now takes between 40 and 60 seconds for me. I also ran it
"under stress" by running all ES tests in one shell, while
simultaneously running 10 iterations of the old index tests. Each
iteration took on average about 90 seconds, which is much better
than the 20+ minutes we see in master on jenkins.
closes#10247
When doc values are explicitly set to the default value serialization
is skipped. This means the alternate way of specifying doc values,
through `fielddata.format: doc_values`, will take precedense if
present.
This change fixes doc values to always be serialized when an explicit value
was passed, so that it continues to take precedence over
`fielddata.format`.
closes#10297closes#10302
The current version is normally a snapshot while in development.
However, when the release process changes the snapshot flag to false,
this causes the static bwc tests to fail because they cannot
find an index for the current version. Instead, this change
skips the current version, because there is no need to test
a verion's bwc against itself.
closes#10292closes#10293
We had an undocumented parameter called `numeric_resolution` which allows to
configure how to deal with dates when provided as a number. The default is to
handle them as milliseconds, but you can also opt-on for eg. seconds.
Close#10072
We recently increased the size of bw indexes and backward compatibility tests
are now taking more time so it makes sense to ask them to do a bit less. This
commit changes the number of replicas we try to copy primaries to from (2 or 3)
to (1 or 2).
Separate repository registration to make sure that failure in registering one repository doesn't cause failures to register other repositories.
Closes#10351
1.1.0 is affected by #5817 which prevents merges from keeping up with the
indexing rate. As a consequence it generates lots of segments and makes bw
compat tests slow. So I added a special case for this version to index fewer
documents.
Boaz Leskes
Adrien Grand
jaymode
Stefan
Ryan Ernst
Shay Banon
Simon Willnauer
Isabel Drost-Fromm
joelbourbon
Robert Muir
Tanguy Leroux
Isabel Drost-Fromm
Lee Hinman
Masaru Hasegawa
Christian Rohling
Hakan Özler
Alexander Reelsen
Dustin Shiver
Maurizio De Magnis
Clinton Gormley
B1nj0y
Martijn van Groningen
Guillaume Dievart
Igor Motov
Eike Dehling