Commit Graph

11247 Commits

Author SHA1 Message Date
Michael McCandless 9981d69e06 Core: add trace logging for the commitTranslog=false case in InternalEngine.flush 2015-04-09 05:00:40 -04:00
Simon Willnauer a243b3f924 [LOGGING] Use static logger name in Engine.java
To ensure subclasses like MockInternalEngine which is in a different
package (test.engine) are logging under the same logger name this commit
moves to a static logger class to determin the logger name. This way
all subclasses of engine will log under `index.engine` which also plays
nicely with `@TestLogging` where log messages sometimes disappeared since
they were enabled for the `index.engine` package but not for `test.engine`
2015-04-09 10:07:50 +02:00
Tanguy Leroux e60f61d0a9 Fix maven-resources-plugin warnings
Commit 168238dab6 declared multiple maven-resources-plugin usages instead of declaring multiple executions for the same plugin... resulting to Maven warnings.

Closes #10433
2015-04-09 10:05:10 +02:00
Ryan Ernst a5bfe332e5 Tests: increase logging for lucene commit during flush in InternalEngine 2015-04-09 00:25:05 -07:00
Ryan Ernst c821b8d3b4 Tests: remove static bwc delete index timeout, so that the slow delete
failure can repro with additional logging
2015-04-08 23:29:37 -07:00
Simon Willnauer b52d24a031 [TEST] increase logging for pulling snapshot from InternalEngine 2015-04-09 06:41:11 +02:00
Ryan Ernst b343d688c5 Tests: Increase delete timeout for static bwc tests to 1 minute 2015-04-08 21:24:36 -07:00
Spencer Alger 0cd1848e88 Indexed scripts/templates: fix yaml test indentation 2015-04-08 15:36:23 -07:00
Ryan Ernst ab395c1267 Tests: allow up to 30s to delete indexes in old static tests
When deleting an index, the tests run checkindex, which requires a
flush.  This can be very slow on virtualized hardware..
2015-04-08 13:57:22 -07:00
Ryan Ernst 178f650552 Tests: Fix static bwc replicas at 1 2015-04-08 09:02:44 -07:00
Simon Willnauer f687377e2f [ENGINE] sync translog before closing engine
If the translog is buffered we must make sure everything is synced to disk
before we rollback the writer otherwise we open a window for potential dataloss due
to stupid errors preventing the translog from being closed.
2015-04-08 17:57:33 +02:00
Ryan Ernst 6df978e76e Test: increasing replicas timeout to avoid slowness on virtualized
hardware (aka jenkins)
2015-04-08 08:44:39 -07:00
javanna acabf2d55a Cluster state REST api: print routing_nodes out only when requested through specific flag
For bacwards compatibility reasons routing_nodes were previously printed out when routing_table was requested, together with the actual routing_table. Now they are printed out only when requests through `routing_nodes` flag.

Relates to #10412
Closes #10486
2015-04-08 16:10:36 +02:00
Boaz Leskes 174d141512 Test: increase recovery concurrency in OldIndexBackwardsCompatibilityTests.testOldIndices
Also added some logging to help pin point slowness
2015-04-08 15:44:45 +02:00
Leonardo Menezes 5fd9aee16e Cluster state REST api: routing_nodes as an independent metric option
Cluster state api returns both routing_table and routing_nodes sections whenever routing_table is requested. That is pretty much the same info, just grouped differently. This commit allows to differentiate between the two. Yet, routing_table still returns both for bw comp reasons.

Closes #10352
Closes #10412
2015-04-08 15:35:11 +02:00
marko asplund 5585175173 Docs: fix typos in example JSON data
Closes #10479
2015-04-08 13:40:35 +02:00
javanna d9aebf4906 Scripting: remove deprecated methods from ScriptService
Removed the following methods from `ScriptService`, which don't require the `ScriptContext` argument:

```
public CompiledScript compile(String lang,  String script, ScriptType scriptType)

public ExecutableScript executable(String lang, String script, ScriptType scriptType, Map<String, Object> vars)

public SearchScript search(SearchLookup lookup, String lang, String script, ScriptType scriptType, @Nullable Map<String, Object> vars)
```

Also removed the ScriptContext.Standard.GENERIC_PLUGIN enum value, as it was used only for backwards compatibility.

 Plugins that make use of scripts should declare their own script contexts through `ScriptModule#registerScriptContext` and use them when compiling/executing scripts.

Closes #10476
2015-04-08 12:20:03 +02:00
javanna 7bd7ea8f13 Scripting: allow plugins to define custom operations that they use scripts for
Plugins can now define multiple operations/contexts that they use scripts for. Fine-grained settings can then be used to enable/disable scripts based on each single registered context.

Also added a new generic category called `plugin`, which will be used as a default when the context is not specified. This allows us to restore backwards compatibility for plugins on `ScriptService` by restoring the old methods that don't require the script context and making them internally use the `plugin` context, as they can only be called from plugins.

Closes #10347
Closes #10419
2015-04-08 11:57:00 +02:00
blackorzar 5d4a4fc3ff Docs: Include groovy community
This change includes the a new community: groovy to point to the official client.

Closes #10470
2015-04-08 11:46:05 +02:00
javanna df875707ec Indexed scripts/templates: return response body when script is not found
Align get indexed scripts and get search template apis to our get api, which returns a response body when the document is not found, with a found boolean flag. Also, return metadata info all the time too.

Closes #7325
Closes #10396
2015-04-08 10:44:11 +02:00
Adrien Grand 85e4c21b0a Tests: Disable the query cache for parent/child unit tests. 2015-04-08 09:58:59 +02:00
Adrien Grand 717dbb0599 Tests: Fix bug in SimpleSortTests.
This tests adds a mappings with {"fielddata": {"format": "doc_values"}} but the
default mapping has {"doc_values": false} so when the document mapper parsing
logic merges both we have {"doc_values": false,"fielddata": {"format": "doc_values"}}
and {"doc_values": false} wins, so the test is not using doc values while it
thought it would.
2015-04-08 09:45:50 +02:00
Boaz Leskes 80e86e5719 Transport: shortcut local execution
In several places in the code we need to notify a node it needs to do something (typically the master). When that node is the local node, we have an optimization in serveral places that runs the execution code immediately instead of sending the request through the wire to itself. This is a shame as we need to implement the same pattern again and again. On top of that we may forget (see note bellow) to do so and we might have to write some craft if the code need to run under another thread pool.

This commit folds the optimization in the TrasnportService, shortcutting wire serliazition if the target node is local.

Note: this was discovered by #10247 which tries to import a dangling index quickly after the cluster forms. When sending an import dangling request to master, the code didn't take into account that fact that the local node may master. If this happens quickly enough, one would get a NodeNotConnected exception causing the dangling indices not to be imported. This will succeed after 10s where InternalClusterService.ReconnectToNodes runs and actively connects the local node to itself (which is not needed), potentially after another cluster state update.

Closes #10350
2015-04-08 09:10:47 +02:00
Adrien Grand a24d898b3a Merge pull request #10468 from jpountz/upgrade/5.1.0-snapshot-1671894
Upgrade to lucene-5.1.0-snapshot-1671894.
2015-04-07 20:07:18 +02:00
Adrien Grand 89b97c1c8d Upgrade to lucene-5.1.0-snapshot-1671894.
This includes a fix for LUCENE-6406.
2015-04-07 20:04:54 +02:00
jaymode 1edf883bae Netty: make exceptionCaught method protected
The exceptionCaught method had default access, which imposes a requirement
for subclasses that need to override this method to be in a specific package. This
change simply makes the method protected, which removes the package requirement.
2015-04-07 13:09:47 -04:00
Adrien Grand 22472b7b57 Merge pull request #10389 from jpountz/enhancement/per_segment_apis
Internal: Make APIs work per-segment like Lucene's Collector.

Close #10389
2015-04-07 18:16:58 +02:00
Stefan f7e6d79569 [DOCS] update versions in java api module.xml
The filenames are updated to fit to the current elasticsearch version
2015-04-07 18:10:53 +02:00
Adrien Grand 967a4e2a86 Internal: Make APIs work per-segment like Lucene's Collector.
We still have a lot of APIs that use setNextReader in order to change the
current segment that should be considered. This commit moves such APIs to
getLeafXXX() instead to be more in-line with Lucene 5's collector API.

I also renamed setDocId to setDocument to be more in-line with the doc values
APIs.

Close #10389
2015-04-07 17:56:01 +02:00
Adrien Grand 884c40abac Merge pull request #9165 from jpountz/enhancement/reduce_search_pool_size
Internal: reduce the size of the search thread pool.
2015-04-07 17:49:50 +02:00
Ryan Ernst e575c4ce53 Tests: Add --all flag to create-bwc script to regenerate all indexes
It is currently a pain to regenerate all the bwc indexes when making
a change to the generation script.  This makes it a single command.
2015-04-07 08:32:34 -07:00
Adrien Grand eb52febf0c Merge pull request #10393 from jpountz/fix/112_snapshot
Internal: Fix snapshot status of version 1.1.2.
2015-04-07 17:28:40 +02:00
Shay Banon af19ec3006 Remove content thread safe from REST layer
there is no need for this anymore, for some time, since in netty now we rely on copying over the buffer and reusing it
closes #10429
2015-04-07 18:11:48 +03:00
Adrien Grand ca6f6d4283 Tests: work-around for LUCENE-6406. 2015-04-07 16:57:20 +02:00
Simon Willnauer c2f9987a15 [TEST] expect simpletext CFS format in test 2015-04-07 15:36:45 +02:00
Simon Willnauer f1abf0aa10 Update Lucene version constants for 1.4.5 and above to 4.10.4 2015-04-07 15:26:58 +02:00
Isabel Drost-Fromm 60bb65c4d9 Docs: Note on shard vs. index level doc frequencies.
Relates to #10154 and #10150

Adds link to additional information on how document frequencies are treated across shards to the cutoff_frequency parameter documentation.

Closes #10451
2015-04-07 14:28:01 +02:00
joelbourbon 3c52bc1098 Docs: Missing 1 escape character in example
Closes #10446
2015-04-07 14:10:17 +02:00
Adrien Grand 37f3351b58 Tests: Speed up CopyOnWriteHash(Map|Set)Tests
These tests create artificial hash collisions in order to make sure that they
can be resolved correctly. But this also makes the tests very slow if there
are too many collisions because insertions/deletions become linear in such
cases. The tests have been modified to not do too many iterations when
collisions are likely.

Close #10442
2015-04-07 13:34:08 +02:00
Robert Muir 1d56502e52 Upgrade to Lucene 5.1 r1671277.
Closes #10435.

Squashed commit of the following:

commit aa1935c790b2731fc2bbc7de6142b09e3fe8bd4a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Apr 6 13:44:40 2015 -0700

    fix index lookup

commit bb6373595ff62ffc56fdf0cba3ac9c0ebe679946
Merge: 916962b eb3a170
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 14:24:38 2015 -0400

    Merge branch 'lucene_r1671277' of github.com:elasticsearch/elasticsearch into lucene_r1671277

commit 916962b82d192a53add471b4cc4a1396bc30eb0e
Merge: 197b3a2 21f72fe
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 07:09:41 2015 -0400

    Merge branch 'master' into lucene_r1671277

commit eb3a1703f7932ddd0cf3e83bec0e86131d255407
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 11:06:03 2015 -0700

    re-enable index lookup tests

commit 80d65d5eab39062dd8364687da74ddbb87ebcb76
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 10:39:52 2015 -0700

    update pom to point to new snapshot repo

commit 197b3a21ac2c2d70c9f740fe53e58632a22d1aad
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:51:22 2015 -0400

    fix postingsenum usage

commit 0e2b7a00cd07d068f755c51185ac521aa1eb0326
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:21:23 2015 -0400

    upgrade to lucene r1671277 (have not yet run tests or looked at postings changes)
2015-04-07 06:09:35 -04:00
Tanguy Leroux a598123838 AbstractBlobContainer.deleteByPrefix() should not list all blobs
The current implementation of AbstractBlobContainer.deleteByPrefix() calls AbstractBlobContainer.deleteBlobsByFilter() which calls BlobContainer.listBlobs() for deleting files, resulting in loading all files in order to delete few of them. This can be improved by calling BlobContainer.listBlobsByPrefix() directly.

This problem happened in #10344 when the repository verification process tries to delete a blob prefixed by "tests-" to ensure that the repository is accessible for the node. When doing so we have the following calling graph: BlobStoreRepository.endVerification() -> BlobContainer.deleteByPrefix() -> AbstractBlobContainer.deleteByPrefix() -> AbstractBlobContainer.deleteBlobsByFilter() -> BlobContainer.listBlobs()... and boom.

Also, AbstractBlobContainer.listBlobsByPrefix() and BlobContainer.deleteBlobsByFilter() can be removed because it has the same drawbacks as AbstractBlobContainer.deleteByPrefix() and also lists all blobs. Listing blobs by prefix can be done at the FsBlobContainer level.

Related to #10344
2015-04-07 11:04:41 +02:00
Isabel Drost-Fromm 110b4c9d81 Update README.md
Updating link to documentation docs from old elasticsearch to new elastic repository.
2015-04-07 10:31:48 +02:00
Boaz Leskes 9e18746ea0 Test: increase timeout in UnicastZenPingTests#testSimplePings 2015-04-07 10:07:58 +02:00
Boaz Leskes 59d6bd8881 Test: mute testOldIndexes
Ensure greens times out often. See https://github.com/elastic/elasticsearch/issues/10434
2015-04-07 09:19:01 +02:00
Ryan Ernst 9c209e46be Test: make httpClient() helper not rely on internalCluster (which may be read only) 2015-04-06 23:25:53 -07:00
Lee Hinman eed7c8af6d [DOCS] Document `indices.recovery.concurrent_small_file_streams` 2015-04-06 11:16:50 -06:00
Masaru Hasegawa 21f72fed0a Queries: Avoid NPE during query parsing
Workaround until Jackson is upgraded to 2.6.x.
Fixes #8629.
2015-04-06 12:26:14 +09:00
Christian Rohling 795b889da2 Warning in documentation for deprecation of rivers
Adding a warning in the documentation on the wiki to inform users that rivers are being deprecated.

Closes #10423
2015-04-05 20:54:26 +02:00
Hakan Özler a97af113f4 Add Pes Plugin to the plugin page
I do not know normally whether this section is the right place or not, please let me know.

Here is the pes plugin's GitHub site :https://github.com/kodcu/pes

Closes #10398
2015-04-05 20:14:24 +02:00
Alexander Reelsen 6170c274ba Documentation: Add note about not having sources in repositories
Closes #10390
2015-04-05 18:08:57 +02:00