Commit Graph

8889 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe 0e5f9898d1 Aggregations: DateHistogramBuilder accepts String preOffset and postOffset
This is what DateHistogramParser expects so will enable the builder to build valid requests using these variables.
Also added tests for preOffset and postOffset since these tests did not exist

Closes #5586
2014-07-10 16:38:09 +01:00
Simon Willnauer 0a988ad8f7 [STORE] Treat reading past EOF as an corrupted index when we fail to read segment infos 2014-07-10 17:25:47 +02:00
Simon Willnauer 81e86eba6e [TEST] Wait for longer on slow nodes until replicating has kicked in 2014-07-10 16:52:51 +02:00
javanna eddb378bae [TEST] added ability to provide settings for external nodes in backwards compatibility tests
Closes #6809
2014-07-10 16:45:16 +02:00
Simon Willnauer e7c67bf03b [TEST] Do RollingUpgrade in BWC tests 2014-07-10 16:24:01 +02:00
Simon Willnauer 62002e8192 [TEST] Close TransportClient after it's used in BulkProcessorTests otherwise it will leave threads behind 2014-07-10 16:06:38 +02:00
Simon Willnauer da148ca8b8 [TEST] Subclass ElasticsearchTestCase in LoggingConfigurationTests 2014-07-10 15:28:38 +02:00
Simon Willnauer 72e6150bc1 [STORE]: Make use of Lucene build-in checksums
Since Lucene version 4.8 each file has a checksum written as it's
footer. We used to calculate the checksums for all files transparently
on the filesystem layer (Directory / Store) which is now not necessary
anymore. This commit makes use of the new checksums in a backwards
compatible way such that files written with the old checksum mechanism
are still compared against the corresponding Alder32 checksum while
newer files are compared against the Lucene build in CRC32 checksum.

Since now every written file is checksummed by default this commit
also verifies the checksum for files during recovery and restore if
applicable.

Closes #5924

This commit also has a fix for #6808 since the added tests in
`CorruptedFileTest.java` exposed the issue.

Closes #6808
2014-07-10 15:04:00 +02:00
Shay Banon 9ca5e6e3e1 Add local node to cluster state
Today, the tribe node needs the local node so it adds it when it starts, but other APIs would benefit from adding the local node, also, adding the local node should be done in a cleaner manner, where it belongs, which is right after the discovery service starts in the cluster service
closes #6811
2014-07-10 14:49:52 +02:00
Iulia Pasov eed3513c37 Docs: Update plugins.asciidoc to fix typo
Changed the name of the European Environment Agency (from European Environmental Agency)

Closes #6807
2014-07-10 14:04:26 +02:00
Simon Willnauer c9266e8b6b [TEST] Wait for primary allocations before restart
This commit ensures that all primaries are allocated before we
restart the node. If one primary is in post recovery when we
restart it will not be allocated otherwise.
2014-07-10 11:54:24 +02:00
Karel Minarik 4ddec99703 [DOC] Added comprehensive documentation for the Ruby and Rails integrations 2014-07-10 11:21:27 +02:00
Simon Willnauer 154bd0309c [DOCS] Fix typo in reference 2014-07-10 08:47:18 +02:00
Simon Willnauer fcadab869d [TEST] SuppressSysoutChecks on ElasticsearchTokenStreamTestCase 2014-07-10 07:48:12 +02:00
uboness c324103cbb added a fix to the PluginManagerTests to create config & bin dirs if they don't exist 2014-07-10 00:50:11 +02:00
Guillaume Nodet 263819c674 [ENV] Release node env if initialization fails
If the node initialisation fails, make sure the
node environment is closed correctly and thus
all locks (on data directories) being properly released.

Closes #6715
2014-07-10 00:14:52 +02:00
Simon Willnauer d82a434d10 [STORE] Make a hybrid directory default using `mmapfs` and `niofs`
`mmapfs` is really good for random access but can have sideeffects if
memory maps are large depending on the operating system etc. A hybrid
solution where only selected files are actually memory mapped but others
mostly consumed sequentially brings the best of both worlds and
minimizes the memory map impact.
This commit mmaps only the `dvd` and `tim` file for fast random access
on docvalues and term dictionaries.

Closes #6636
2014-07-10 00:01:43 +02:00
Simon Willnauer b69fa52588 [TEST] Mute PluginManagerTests#testLocalPluginInstallWithBinAndConfig 2014-07-10 00:00:41 +02:00
Simon Willnauer 9e4d738d7e [TEST] SuppressSysoutChecks on ElasticsearchPostingsFormatTest 2014-07-09 23:24:31 +02:00
uboness 6dae32b09a Added a check on moving bin & config plugin dirs
Plugins can contain bin & config sub-dirs that are copied to es's bin & config directories. If moving these directories fails we now throw an error.
2014-07-09 23:05:12 +02:00
Shay Banon 808c52706a [TEST] relax size test, to not run into OOM 2014-07-09 23:03:06 +02:00
Shay Banon 94539979c7 [TEST] make completion tests more resilient to delayed mapping
especially the tests that check for update of mapping, we need to make sure that the cluster is green so mappings won't get override, also, put mapping during index creation when possible
2014-07-09 22:19:06 +02:00
uboness 610900b781 Enables plugins to define default logging configuration for their needs.
- adds support for multiple logging configurations under the config dir (will pick up any logging.xxx in the config folder tree)
- plugins can now define a top level config directory that will be copied under es config dir and will be renamed after the plugin name (same as the support we have the plugin "bin" dirs)

 Closes #6802
2014-07-09 21:46:21 +02:00
Shay Banon 3e202b4343 Improve large bytes request handling by detecting content composite buffer
There is a special type of request that tries to not allocate another buffer when sending bytes request (used by the public cluster state action). With the new pages bytes reference support, the content can already be a composite channel buffer, take that into account when building the actual composite buffer that will be sent over the network
closes #6756
2014-07-09 21:40:15 +02:00
Shay Banon 8910e09beb Disable JSONP by default
By default, disable the option to use JSONP in our REST layer
closes #6795
2014-07-09 21:17:17 +02:00
Shay Banon b301132d7b [TEST] move more tests to assertBusy from awaitBusy 2014-07-09 21:16:23 +02:00
David Pilato 0a58781d2d Modify command window title (windows)
When launching multiple nodes in MS Windows env, batch command window does not come by default with any title.

This patch add `Elasticsearch VERSION` as a title, where VERSION is the current elasticsearch version.

Same for `plugin.bat` and `service.bat`.

Closes #6336
Closes #6752
2014-07-09 18:15:13 +02:00
Shay Banon 386a14370a Upgrade to jackson core 2.4.1.1
Note, we had to disable the symbol overflow, since the many mapping case was tripping it
closes #6789
2014-07-09 17:49:51 +02:00
Areek Zillur 95a5eedd3c Merge pull request #6792 from areek/fix/6717
Remove Lucene's deprecated PatternAnalyzer
2014-07-09 11:18:51 -04:00
Valdis Rigdon 747fd38411 [Test] BackgroundIndexer uses the wrong logger
Closes #6781
2014-07-09 16:49:05 +02:00
Roman Sanchez 1cca0071eb Docs: New entry/front-end
Adding, Calaca, a simple search client for Elasticsearch.

Closes #6754
2014-07-09 16:36:23 +02:00
Iulia Pasov a79d0744d3 Docs: Update plugins.asciidoc
Closes #6683
2014-07-09 16:15:59 +02:00
Clinton Gormley d0441857cd Fix typo in Hunspell logging 2014-07-09 15:56:07 +02:00
Simon Willnauer 73b9d37064 Add v1.2.3 to Version.java 2014-07-09 15:02:18 +02:00
Shay Banon ecde4e0c8e Improve Settings#get lookup for camel case support
Today, if we miss on a get on setting, we try its camel case version. The assumption is that in our code, we never use camel case to lookup a setting, but we want to support camel case if the user provided one.
This can be expensive (#toCamelCase) when the get on the setting is done in a tight call, which is evident when running the allocation deciders as part of the reroute logic.
Instead of doing the camel case check on get, prepare an additional map that includes all the settings that are provided as came case, and try and lookup from it if needed.
closes #6765
2014-07-09 14:23:37 +02:00
Martijn van Groningen 9abb7c45b4 Store: Before removing shard physically from disk verify that another node in the cluster actually holds an active shard copy.
Closes #6692
2014-07-09 12:57:04 +02:00
Boaz Leskes 684e698627 [Code] Control whether MapperService docMapper iterator should contain DEFAULT_MAPPING
At the moment one can iterate the MapperService to go through all document mappers. This includes the document mapper of DEFAULT_MAPPING, which may be surprising and lead to unintended results. This commit removes the Iterable implementation and add a docMappers method that asks the caller to make an explicit choice

Closes #6793
2014-07-09 11:31:52 +02:00
Clinton Gormley b6baa4be4a Update preference.asciidoc
Clarify that `preference` is a query string parameter only
and provide an example.
2014-07-09 11:13:17 +02:00
Boaz Leskes af119dfdd9 [Engine] checkVersionMapRefresh shouldn't use indexWriter.getConfig(). We run it out of lock, the indexWriter may be closed..
Relates to #6443, #6786

Closes #6794
2014-07-09 10:43:28 +02:00
Areek Zillur 14af0cb0f3 Remove Lucene's deprecated PatternTokenizer
Instead of using the PatternTokenizer, the functionality was replicated by using Lucene's StopFilter, PatterTokenizer and LowerCaseFilter

Closes #6717
2014-07-08 21:39:39 -04:00
Simon Willnauer 57cd8f765f [ENGINE] Prevent NPE if engine is closed while version map is checked
We check if the version map needs to be refreshed after we released
the readlock which can cause the the engine being closed before we
read the value from the volatile `indexWriter` field which can cause an
NPE on the indexing thread. This commit also fixes a potential uncaught
exception if the refresh failed due to the engine being already closed.

Relates to #6443
Closes #6786
2014-07-09 00:22:31 +02:00
Shay Banon b97b670011 add more options to many mappings test 2014-07-08 23:46:33 +02:00
Spencer Alger c268e2d0ef [API SPEC] Update indicies.stats group param 2014-07-08 11:25:13 -07:00
Shay Banon c9a46978e1 Remove intern calls on FieldMapper#Names
remove internal callas on FieldMappers#Names, we properly reuse FieldMapper, so there is no need to try and call intern in order to reuse the names. This can be heavy with many fields and continuous mapping parsing.
closes #6747
2014-07-08 20:17:48 +02:00
Shay Banon 5643563730 Add missing pre built analysis components
closes #6770
2014-07-08 19:54:05 +02:00
Shay Banon bb92d46163 [TEST] move tribe tests to use assertBusy 2014-07-08 19:06:59 +02:00
Shay Banon 10030a63cc Add assertBusy helper test method
We use awaitBusy in our tests, the problem is that we have to check if it awaited or not, and then try and keep around somehow more info around why the predicate failed and a timeout happened.
The idea of assertBusy is to allow to simply write "regular" test code, and if the test code trips, it will busy wait till a timeout. This allows us to keep around the assertion information and properly throw it for information that is inherently kept in the failure itself.
2014-07-08 18:58:10 +02:00
mikemccand 7335b5db22 Translog: change flush_threshold_ops to unlimited by default, so we only flush by byte size by default
If you are indexing tiny documents then the previous default (5000)
was too low, causing excessive fsyncs with high indexing rates.  With
this change we now only flush by byte size (200 MB) by default for
better indexing performance for tiny documents.

Closes #6783
2014-07-08 12:42:17 -04:00
mikemccand a8417a7de3 Core: trigger refresh when the live version map is using too much RAM
When refresh_interval is long or disabled, and indexing rate is high,
it's possible for live version map to use non-trivial amounts of RAM.
With this change we now trigger a refresh in such cases to clear the
version map so we don't use unbounded RAM.

Closes #6443
2014-07-08 12:07:48 -04:00
Clinton Gormley 5ebc238a25 Docs: Changed the path separator for Windows in the README
Closes #1854
2014-07-08 16:44:58 +02:00