Commit Graph

11710 Commits

Author SHA1 Message Date
Adrien Grand 884c40abac Merge pull request #9165 from jpountz/enhancement/reduce_search_pool_size
Internal: reduce the size of the search thread pool.
2015-04-07 17:49:50 +02:00
Ryan Ernst e575c4ce53 Tests: Add --all flag to create-bwc script to regenerate all indexes
It is currently a pain to regenerate all the bwc indexes when making
a change to the generation script.  This makes it a single command.
2015-04-07 08:32:34 -07:00
Adrien Grand eb52febf0c Merge pull request #10393 from jpountz/fix/112_snapshot
Internal: Fix snapshot status of version 1.1.2.
2015-04-07 17:28:40 +02:00
Shay Banon af19ec3006 Remove content thread safe from REST layer
there is no need for this anymore, for some time, since in netty now we rely on copying over the buffer and reusing it
closes #10429
2015-04-07 18:11:48 +03:00
Adrien Grand ca6f6d4283 Tests: work-around for LUCENE-6406. 2015-04-07 16:57:20 +02:00
Simon Willnauer c2f9987a15 [TEST] expect simpletext CFS format in test 2015-04-07 15:36:45 +02:00
Simon Willnauer f1abf0aa10 Update Lucene version constants for 1.4.5 and above to 4.10.4 2015-04-07 15:26:58 +02:00
Isabel Drost-Fromm 60bb65c4d9 Docs: Note on shard vs. index level doc frequencies.
Relates to #10154 and #10150

Adds link to additional information on how document frequencies are treated across shards to the cutoff_frequency parameter documentation.

Closes #10451
2015-04-07 14:28:01 +02:00
joelbourbon 3c52bc1098 Docs: Missing 1 escape character in example
Closes #10446
2015-04-07 14:10:17 +02:00
Adrien Grand 37f3351b58 Tests: Speed up CopyOnWriteHash(Map|Set)Tests
These tests create artificial hash collisions in order to make sure that they
can be resolved correctly. But this also makes the tests very slow if there
are too many collisions because insertions/deletions become linear in such
cases. The tests have been modified to not do too many iterations when
collisions are likely.

Close #10442
2015-04-07 13:34:08 +02:00
Robert Muir 1d56502e52 Upgrade to Lucene 5.1 r1671277.
Closes #10435.

Squashed commit of the following:

commit aa1935c790b2731fc2bbc7de6142b09e3fe8bd4a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Apr 6 13:44:40 2015 -0700

    fix index lookup

commit bb6373595ff62ffc56fdf0cba3ac9c0ebe679946
Merge: 916962b eb3a170
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 14:24:38 2015 -0400

    Merge branch 'lucene_r1671277' of github.com:elasticsearch/elasticsearch into lucene_r1671277

commit 916962b82d192a53add471b4cc4a1396bc30eb0e
Merge: 197b3a2 21f72fe
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 07:09:41 2015 -0400

    Merge branch 'master' into lucene_r1671277

commit eb3a1703f7932ddd0cf3e83bec0e86131d255407
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 11:06:03 2015 -0700

    re-enable index lookup tests

commit 80d65d5eab39062dd8364687da74ddbb87ebcb76
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 10:39:52 2015 -0700

    update pom to point to new snapshot repo

commit 197b3a21ac2c2d70c9f740fe53e58632a22d1aad
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:51:22 2015 -0400

    fix postingsenum usage

commit 0e2b7a00cd07d068f755c51185ac521aa1eb0326
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:21:23 2015 -0400

    upgrade to lucene r1671277 (have not yet run tests or looked at postings changes)
2015-04-07 06:09:35 -04:00
Tanguy Leroux a598123838 AbstractBlobContainer.deleteByPrefix() should not list all blobs
The current implementation of AbstractBlobContainer.deleteByPrefix() calls AbstractBlobContainer.deleteBlobsByFilter() which calls BlobContainer.listBlobs() for deleting files, resulting in loading all files in order to delete few of them. This can be improved by calling BlobContainer.listBlobsByPrefix() directly.

This problem happened in #10344 when the repository verification process tries to delete a blob prefixed by "tests-" to ensure that the repository is accessible for the node. When doing so we have the following calling graph: BlobStoreRepository.endVerification() -> BlobContainer.deleteByPrefix() -> AbstractBlobContainer.deleteByPrefix() -> AbstractBlobContainer.deleteBlobsByFilter() -> BlobContainer.listBlobs()... and boom.

Also, AbstractBlobContainer.listBlobsByPrefix() and BlobContainer.deleteBlobsByFilter() can be removed because it has the same drawbacks as AbstractBlobContainer.deleteByPrefix() and also lists all blobs. Listing blobs by prefix can be done at the FsBlobContainer level.

Related to #10344
2015-04-07 11:04:41 +02:00
Isabel Drost-Fromm 110b4c9d81 Update README.md
Updating link to documentation docs from old elasticsearch to new elastic repository.
2015-04-07 10:31:48 +02:00
Boaz Leskes 9e18746ea0 Test: increase timeout in UnicastZenPingTests#testSimplePings 2015-04-07 10:07:58 +02:00
Boaz Leskes 59d6bd8881 Test: mute testOldIndexes
Ensure greens times out often. See https://github.com/elastic/elasticsearch/issues/10434
2015-04-07 09:19:01 +02:00
Ryan Ernst 9c209e46be Test: make httpClient() helper not rely on internalCluster (which may be read only) 2015-04-06 23:25:53 -07:00
Lee Hinman eed7c8af6d [DOCS] Document `indices.recovery.concurrent_small_file_streams` 2015-04-06 11:16:50 -06:00
Masaru Hasegawa 21f72fed0a Queries: Avoid NPE during query parsing
Workaround until Jackson is upgraded to 2.6.x.
Fixes #8629.
2015-04-06 12:26:14 +09:00
Christian Rohling 795b889da2 Warning in documentation for deprecation of rivers
Adding a warning in the documentation on the wiki to inform users that rivers are being deprecated.

Closes #10423
2015-04-05 20:54:26 +02:00
Hakan Özler a97af113f4 Add Pes Plugin to the plugin page
I do not know normally whether this section is the right place or not, please let me know.

Here is the pes plugin's GitHub site :https://github.com/kodcu/pes

Closes #10398
2015-04-05 20:14:24 +02:00
Alexander Reelsen 6170c274ba Documentation: Add note about not having sources in repositories
Closes #10390
2015-04-05 18:08:57 +02:00
Dustin Shiver ae60144123 Update for clarification
Make it clear which nodes in the cluster should have `http.enabled` set to `false`.

Closes #10305
2015-04-05 18:05:14 +02:00
Maurizio De Magnis 589f8d4468 typo in ruby docs
Closes #10331
2015-04-05 15:14:40 +02:00
Clinton Gormley 276dbc2925 Update repositories.asciidoc
Added a warning explaining how `add-apt-repository` adds a `deb-src` entry, which can result in errors.

Closes #10223
2015-04-05 11:43:49 +02:00
Clinton Gormley 9607f4c22d Docs: Elasticsearch will refuse to start with a known bad JVM 2015-04-04 20:21:37 +02:00
Ryan Ernst 9191da9d9f Test: temporarily avoiding replica problems with 0.90.13, see #10434 2015-04-04 09:27:12 -07:00
Clinton Gormley a95b11ca61 Document `doc_values` for field type `ip`
Closes #9809
2015-04-04 17:51:28 +02:00
Clinton Gormley c046398093 Update indices.asciidoc
Fixed typo in cat indices

Relates to #7936
2015-04-04 16:50:04 +02:00
B1nj0y bb7cbea71e Docs: Update client.asciidoc
change a typo.

Closes #10039
2015-04-04 15:21:11 +02:00
Ryan Ernst c3011cead4 Tests: Revamp static bwc test framework to use dangling indexes
The static old index tests currently take a long time to run because
each index version essentially recreates the cluster, and spins up
new nodes.  This PR instead loads each old version into the existing
cluster as a dangling index. It also removes the intermediate
"StaticIndexBackwardCompatibilityTest" which was an extra layer
with no purpose, and moves a shared version of a commonly found
function to get an http client.

The test now takes between 40 and 60 seconds for me. I also ran it
"under stress" by running all ES tests in one shell, while
simultaneously running 10 iterations of the old index tests. Each
iteration took on average about 90 seconds, which is much better
than the 20+ minutes we see in master on jenkins.

closes #10247
2015-04-03 23:21:55 -07:00
Ryan Ernst d379b3618e Mappings: Fix doc values representation to always serliaze if explicitly set
When doc values are explicitly set to the default value serialization
is skipped. This means the alternate way of specifying doc values,
through `fielddata.format: doc_values`, will take precedense if
present.

This change fixes doc values to always be serialized when an explicit value
was passed, so that it continues to take precedence over
`fielddata.format`.

closes #10297
closes #10302
2015-04-03 19:35:52 -07:00
Ryan Ernst 96fa466414 Tests: Skip Version.CURRENT for static bwc indexes
The current version is normally a snapshot while in development.
However, when the release process changes the snapshot flag to false,
this causes the static bwc tests to fail because they cannot
find an index for the current version.  Instead, this change
skips the current version, because there is no need to test
a verion's bwc against itself.

closes #10292
closes #10293
2015-04-03 19:25:43 -07:00
Adrien Grand c7115f8364 Mappings: Bring back numeric_resolution.
We had an undocumented parameter called `numeric_resolution` which allows to
configure how to deal with dates when provided as a number. The default is to
handle them as milliseconds, but you can also opt-on for eg. seconds.

Close #10072
2015-04-03 19:54:14 +02:00
Martijn van Groningen 3f8908acfb Test: Not all shards maybe in a started state after the initial scroll search, but are after we do the stats call, so add ensureGreen to ensure we don't up with this discrepancy 2015-04-03 18:30:59 +02:00
Adrien Grand 24ca482918 Tests: Make OldIndexBackwardsCompatibilityTests copy to fewer replicas.
We recently increased the size of bw indexes and backward compatibility tests
are now taking more time so it makes sense to ask them to do a bit less. This
commit changes the number of replicas we try to copy primaries to from (2 or 3)
to (1 or 2).
2015-04-03 16:46:30 +02:00
Adrien Grand d3b1567f79 Tests: Add more assertions to SearchStatsTests. 2015-04-03 16:09:57 +02:00
Guillaume Dievart adcb782423 Update core-types.asciidoc 2015-04-03 14:12:29 +02:00
Adrien Grand 523033bce6 Test: Disable TRACE logging on aggregations tests. 2015-04-03 11:33:30 +02:00
Martijn van Groningen ef1f834cf9 inner hits: Make sure inner hits also work for nested fields defined in object field
Closes #10334
2015-04-03 08:57:10 +02:00
Igor Motov 5379fa04de Snapshot/Restore: separate repository registration
Separate repository registration to make sure that failure in registering one repository doesn't cause failures to register other repositories.

Closes #10351
2015-04-02 22:23:43 -04:00
Lee Hinman 8f13c5ad4d [TEST] Turn on more logging for testDataPathCanBeChanged 2015-04-02 20:11:33 -06:00
Eike Dehling 2fc2c8284a Mapping: add consistent serialization order for field data settings
Closes #10370
Closes #10318
2015-04-02 21:41:52 +02:00
Adrien Grand 1401075070 Tests: Speed up backward-compatibility tests for 1.1.0
1.1.0 is affected by #5817 which prevents merges from keeping up with the
indexing rate. As a consequence it generates lots of segments and makes bw
compat tests slow. So I added a special case for this version to index fewer
documents.
2015-04-02 19:11:12 +02:00
Tanguy Leroux 168238dab6 Add properties files to configure startup and installation scripts
Many scripts are used to start/stop and install/uninstall elasticsearch. These scripts share a lot of configuration properties like directory paths, max value for a setting, default user etc. Most of the values are identical but some of them are different depending of the platform (Debian-based or Redhat-based OS), depending of the way elasticsearch is started (shell script, systemd, sysv-init...) or the way it is installed (zip, rpm, deb...). Today the values are duplicated in multiple places, making it difficult to maintain the scripts or to update a value.

This pull request make this more uniform: values used in scripts must be defined in a common packaging.properties file. Each value can be overridden in another specific packaging.properties file for Debian or Redhat. All startup and installation scripts are filtered with the common then the custom packaging.properties files before being packaged as a zip/tar.gz/rpm/dpkf archive.
2015-04-02 18:09:43 +02:00
Adrien Grand 08388e0389 Internal: reduce the size of the search thread pool.
Follow-up of #9135. We initially decreased the stack size because it would end
up a lot of memory when there are many threads. But we also have some thread
pools that may be oversized, in particular the search thread pool.

This commit proposes to decrease the default search thread pool size from
`3 * num_procs` to `3 * num_procs / 2 + 1`. This is large enough to be sure
that we can use all the machine resources even with a search-only work load
but not too large in order to not consume too much memory because of the stack
size and thread locals.
2015-04-02 16:32:20 +02:00
Adrien Grand e8a42c6954 Merge pull request #7961 from jpountz/enhancement/boolean_doc_values
Fielddata: Add doc values support to boolean fields.

Close #7961
2015-04-02 16:06:52 +02:00
Adrien Grand 08f93cf33f Add doc values support to boolean fields.
This pull request makes boolean handled like dates and ipv4 addresses: things
are stored as as numerics under the hood and aggregations add some special
formatting logic in order to return true/false in addition to 1/0.

For example, here is an output of a terms aggregation on a boolean field:
```
   "aggregations": {
      "top_f": {
         "doc_count_error_upper_bound": 0,
         "buckets": [
            {
               "key": 0,
               "key_as_string": "false",
               "doc_count": 2
            },
            {
               "key": 1,
               "key_as_string": "true",
               "doc_count": 1
            }
         ]
      }
   }
```

Sorted numeric doc values are used under the hood.

Close #4678
Close #7851
2015-04-02 15:40:46 +02:00
Tanguy Leroux 877b8fa27a Merge pull request #9532 from nickcanz/master
service.bat file should explicitly use the Windows find command.
2015-04-02 15:34:44 +02:00
Adrien Grand 3d09292a9f Internal: Fix snapshot status of version 1.1.2.
org.elasticsearch.Version mistakenly tags 1.1.2 as a snapshot version while it
has been released.
2015-04-02 15:34:38 +02:00
Martijn van Groningen e390ef5bc1 inner hits: Make sure `size` of `0` works on the `inner_hits` level.
Closes #10358
2015-04-02 15:08:39 +02:00