Commit Graph

1565 Commits

Author SHA1 Message Date
Shay Banon 5fa66cd592 Node Stats: Allow to explicitly get specific indices level node stats element
closes #2871
2013-04-07 20:22:48 -07:00
Shay Banon 15d7ae5983 FieldData Stats: Add field data stats to indices stats API
closes #2870
2013-04-07 18:30:24 -07:00
Martijn van Groningen 86c1714bf3 Also test the `fields` option. 2013-04-07 21:52:19 +02:00
Simon Willnauer 7ad03ed789 Use IndexOption.DOCS_ONLY for boolean fields
Closes #2866
2013-04-06 22:41:22 +02:00
Shay Banon 9f6c8c88f3 improve on shard level filter/id cache stats
use just the removal listener and back to the IndexReader#coreCacheKey as the actual field as part of the cache key
2013-04-06 00:02:42 +02:00
Shay Banon 815917fbf8 confusing code..., but we can't release the searcher in a get result case
we need that searcher later on..., need to think of how to simplify that..., added a comment for now
2013-04-05 23:27:03 +02:00
Simon Willnauer 36ffd6d582 release searcher in finally block rather than relying on an exception that is thrown 2013-04-05 22:45:52 +02:00
Shay Banon 84670212a6 Filter / Id Cache Stats: Add to Indices Stats API, revise node stats API
closes #2862
2013-04-05 20:02:32 +02:00
Simon Willnauer 5e7ad9832c Added more evil tests for different field data implementations 2013-04-05 18:12:50 +02:00
Martijn van Groningen 224faffead Added an extended test for terms facet with a decent number of documents / field values and randomly tests various options. Also fixed an issue where `regex` and `excludes` were ignored when `all_terms` was used. 2013-04-05 17:38:46 +02:00
David Pilato 4b1ec037f8 Fix test for #2668. 2013-04-05 15:00:28 +02:00
Martijn van Groningen 9b5c74d43e Made sure `all_terms` works consistently. In some cases the `all_terms` option was ignored: * Faceting on number based fields. * The `execution_type` was set to `map`. * In the case the `fields` option was used.
Closes #2861
2013-04-05 14:27:19 +02:00
Shay Banon 831ea789aa rename getByOrd to getValueByOrd (to match BytesValues.WithOrdinals)
also make it public so it can be used when iterating over ords
2013-04-05 13:56:33 +02:00
Shay Banon bcc14cde9f make numeric namings consistent with bytes ones
also add the ability to get the ordinals from DoubleValues.WithOrdinals and LongValues.WithOrdinals
2013-04-05 13:33:56 +02:00
David Pilato 36b92be212 List of existing plugins with Node Info API
We want to display information about loaded plugins in Node Info API using plugin option:

```sh
curl http://localhost:9200/_nodes?plugin=true
```

For example, on a 4 nodes cluster, it could provide the following output:

```javascript
{
  "ok" : true,
  "cluster_name" : "test-cluster-MacBook-Air-de-David.local",
  "nodes" : {
    "lodYfbFTRnmwE6rjWGGyQQ" : {
      "name" : "node1",
      "transport_address" : "inet[/172.18.58.139:9300]",
      "hostname" : "MacBook-Air-de-David.local",
      "version" : "0.90.0.Beta2-SNAPSHOT",
      "http_address" : "inet[/172.18.58.139:9200]",
      "plugins" : [ ]
    },
    "hJLXmY_NTrCytiIMbX4_1g" : {
      "name" : "node4",
      "transport_address" : "inet[/172.18.58.139:9303]",
      "hostname" : "MacBook-Air-de-David.local",
      "version" : "0.90.0.Beta2-SNAPSHOT",
      "http_address" : "inet[/172.18.58.139:9203]",
      "plugins" : [ {
        "name" : "test-plugin",
        "description" : "test-plugin description",
        "site" : true,
        "jvm" : false
      }, {
        "name" : "test-no-version-plugin",
        "description" : "test-no-version-plugin description",
        "site" : true,
        "jvm" : false
      }, {
        "name" : "dummy",
        "description" : "No description found for dummy.",
        "url" : "/_plugin/dummy/",
        "site" : false,
        "jvm" : true
      } ]
    },
    "bnoySsBfTrSzbDRZ0BFHvg" : {
      "name" : "node2",
      "transport_address" : "inet[/172.18.58.139:9301]",
      "hostname" : "MacBook-Air-de-David.local",
      "version" : "0.90.0.Beta2-SNAPSHOT",
      "http_address" : "inet[/172.18.58.139:9201]",
      "plugins" : [ {
        "name" : "dummy",
        "description" : "This is a description for a dummy test site plugin.",
        "url" : "/_plugin/dummy/",
        "site" : false,
        "jvm" : true
      } ]
    },
    "0Vwil01LSfK9YgRrMce3Ug" : {
      "name" : "node3",
      "transport_address" : "inet[/172.18.58.139:9302]",
      "hostname" : "MacBook-Air-de-David.local",
      "version" : "0.90.0.Beta2-SNAPSHOT",
      "http_address" : "inet[/172.18.58.139:9202]",
      "plugins" : [ {
        "name" : "test-plugin",
        "description" : "test-plugin description",
        "site" : true,
        "jvm" : false
      } ]
    }
  }
}
```

Information are cached for 10 seconds by default. Modify `plugins.info_refresh_interval` property if needed.
Setting `plugins.info_refresh_interval` to `-1` will cause infinite caching.
Setting `plugins.info_refresh_interval` to `0` will disable caching.

Closes #2668.
2013-04-05 11:36:56 +02:00
Simon Willnauer f3e6fe094a beef up term facet tests 2013-04-05 11:05:24 +02:00
Simon Willnauer 9fbe075aec Added test that compares concurrent facet execution results with a serial execution result 2013-04-05 10:36:53 +02:00
Shay Banon 5af6343697 allow to disable the optimization of removal of ords on single value numerics/geo field data
field data settings in the mappings can have ordinals=always option
2013-04-05 00:44:07 +02:00
Shay Banon 54f685674b Thread Pool: Update default settings (move from default cached to fixed)
closes #2858
2013-04-04 23:24:49 +02:00
Simon Willnauer f1dd867c4f Catch Throwable when listener is called rather then Exception to prevent possible hangs if fatal exceptions or errors are thrown 2013-04-04 22:58:38 +02:00
Shay Banon a206aa4548 Settings / Config: Allow to explicitly specify external environment variable syntax, in which case its optional
fixes #2855
2013-04-04 16:30:24 +02:00
Simon Willnauer d758401add Cleanup ScriptDocValues. This commit adds a getValues method to all ScriptDocValues for easy access
in scripts via doc['field'].values / value.
2013-04-04 16:07:54 +02:00
Alexander Reelsen 4f96b36376 Returning configuration of root field mappers toXContent method only if they are enabled 2013-04-04 15:55:12 +02:00
Alexander Reelsen fbdf89c636 Fix for ttl fieldmapper to support disabling correctly. Also returning only booleans, not enums in toXContent 2013-04-04 12:27:23 +02:00
Alexander Reelsen 955788e9a5 Allowing to disable size field mapper after enabling 2013-04-04 09:41:41 +02:00
Alexander Reelsen e662e4d55d Allowing to disable index field mapper after enabling 2013-04-04 09:41:41 +02:00
Alexander Reelsen 9cc2563d5e Allowing to disable timestamp field mapper after enabling 2013-04-04 09:41:41 +02:00
Simon Willnauer 223ec2c42d Beef up FieldData tests by running one on one duells 2013-04-03 18:38:25 +02:00
Igor Motov 356329df00 Improve stability of ClusterHealthTests 2013-04-03 12:07:42 -04:00
Igor Motov d2f6349dcf Improve stability of MinimumMasterNodesTests 2013-04-03 11:51:28 -04:00
Martijn van Groningen 0a89c80554 Fixed issue where a doc is omitted from the hits if it has no geo point and sorting is based on geo distance.
Closes #2851
2013-04-03 17:25:16 +02:00
Simon Willnauer bbe619a416 Call onFailure for every exception case even in the case of an error / runtime exception
Closes #2848
2013-04-03 12:25:58 +02:00
Simon Willnauer eb8b38d027 Upgrade to Lucene 4.2.1 2013-04-03 12:22:39 +02:00
Martijn van Groningen af2f31c33e Fixed typo 2013-04-02 22:31:06 +02:00
Martijn van Groningen cf00acf5b0 If no specified index or alias exists and `ignore_indices` is set to `missing` an index missing error is returned instead of resolving to all open indices (e.g. when searching). This breaks backwards comp. with 0.20.x and before.
Closes #2837
2013-04-02 19:06:17 +02:00
Alexander Reelsen 0a466352cd Add support for creating a fedora RPM package with maven
Note: This has been disabled by default and is therefore not included in a
standard build. The main reason for this is, that you need to have a RPM
binary and the rpm development packages installed, which is not the case
on many systems.

The package contains an init.d-script as well as systemd configurations.

You can build your own RPM package simply by running 'maven rpm:rpm'
2013-04-02 16:19:45 +02:00
Shay Banon 10a76ad5d8 fix seen readers counter
since clear can be called on percolator as well, we need to make sure we inc the counter even for non segment readers
2013-04-02 13:25:56 +02:00
Shay Banon 31d1e6cfe7 Field Data: Simplify field data cache settings
closes #2843
2013-04-02 12:44:39 +02:00
Alexander Reelsen d866321c55 Merge pull request #2811 from spinscale/document-mapper-merge
Allow to update ttl field mapping after initial creation. Fixes #2136
2013-04-01 23:37:29 -07:00
Simon Willnauer 7efa92636a Cut over to IntsRef in favor of IntsArrayRef 2013-03-31 10:46:21 +02:00
Simon Willnauer b3356d9f8d remove dead code 2013-03-31 10:19:17 +02:00
Simon Willnauer 2a09342405 remove Bytes.java in favor of BytesRef / ArrayUtils 2013-03-31 08:54:39 +02:00
Simon Willnauer e864d5785e optimize matcher reset to not create unnecessary string objects 2013-03-30 17:35:23 +01:00
Simon Willnauer fefa8da2ea remove StringValues in favor of BytesValues 2013-03-30 17:35:23 +01:00
Simon Willnauer dff2a9279c clean-up double values 2013-03-30 17:35:23 +01:00
Simon Willnauer d5c271acf5 clean-up long values 2013-03-30 17:35:22 +01:00
Simon Willnauer 5aedf74fb0 Remove getValues from numeric and string field data & clean up geo field data 2013-03-30 17:35:22 +01:00
Simon Willnauer 7f81469137 Refactor BytesValues to be reused as the interface for HashedBytesValues and remove HashBytesValues. 2013-03-30 17:35:22 +01:00
Simon Willnauer 129f02623b Added FST based FieldData implementation holding all data in a per segment FST.
This commit factors our a common API for BytesValues based impl to shared code and reduce code duplication.
2013-03-30 17:35:22 +01:00
Shay Banon 72c76c2799 fail on malformed sort 2013-03-30 13:58:39 +01:00
Shay Banon 6a1cb8f61b {sort: "field"} throws misleading errors
fixes #2835
2013-03-30 13:46:53 +01:00
Martijn van Groningen 2e93329e23 If match then go to next doc 2013-03-29 16:57:42 +01:00
Martijn van Groningen a89dde8bac Fixed `bool` filter bugs:
* In the case only should clauses were specified with specific type of filters, the first clause determined which documents matched.
 * In some cases the minimum at least 1 should clause should match behaviour was broken.
2013-03-29 16:48:36 +01:00
Igor Motov b657bdfa1a Optimize aliases processing
Closes# 2832
2013-03-29 10:44:45 -04:00
Alexander Reelsen a880a6c85e Allow to update ttl field mapping after initial creation. Fixes #2136
Adding possibility to change TTL field mapper data without specifying enabled flag in mapping update
2013-03-28 17:25:28 +01:00
Martijn van Groningen 941aa17a43 Added sort mode to geo distance sorting. Closes #1846 2013-03-28 17:04:42 +01:00
Igor Motov 9bc50ea609 Fix LeastUsedDistributor and ensure random distribution for multiple non-fs directories
If we cannot determine available space the fallback scenario is now to use random distribution instead of always using the last directory.

Fixes #2820
2013-03-28 11:08:54 -04:00
Shay Banon 1fc37e5954 Segments API: Add version & compound for each segment
closes #2823
2013-03-28 15:34:38 +01:00
Shay Banon 473473e867 remove the field settings for query parser cache, not really relevant 2013-03-27 20:39:36 +01:00
Shay Banon c18c609af1 Date math in query_string caches now()
fixes #2808
2013-03-27 20:32:38 +01:00
Igor Motov 5bb75f9da3 Move applying alias filter to ContextSearch#preProcess() 2013-03-27 09:23:54 -04:00
Simon Willnauer 17f83f33bb Terminate early when no terms left in the suggest string.
Closes #2817
2013-03-26 17:44:34 +01:00
Igor Motov 9ae421a8b2 Fix filtering aliases with non-empty sort options
Fixes #2816
2013-03-26 07:23:44 -04:00
Shay Banon d35a3b03c8 Warmers: Have an explicit warmer thread pool
add 1 in case there is 1 core...
closes #2815
2013-03-25 23:34:52 +01:00
Simon Willnauer aa97c031f2 Don't reset tokenstream before passing to the MemoryIndex, otherwise some tokenizer might swallow tokens.
Closes #2814
2013-03-25 22:46:11 +01:00
Shay Banon b7106622d8 Warmers: Have an explicit warmer thread pool
Have an explicit threadpool warmer that is dedicated to execute warmers. Currently, it uses the search threadpool, which does not work well since the number of concurrent searches should be separate from the number of concurrent warmers allows, also the characteristics of the search pool (for example, bounded queue_size) might not fit well with how warmers should be executed (they should not be "rejected").

closes #2815
2013-03-25 16:46:37 +01:00
Shay Banon 0e815ce11c add 0.20.7 2013-03-25 12:33:55 +01:00
adavis 6a93fbcf07 Adding parsing for zero terms query for multi match
Tests for multi-match zero_terms_query and making references to the ZeroTermsQuery enum consistent to others used in MultiMatchQueryBuilder
2013-03-23 08:59:39 +01:00
adavis 3f83904680 Fixes java6_u31 compile error w.r.t. type inference 2013-03-22 16:46:42 -07:00
Simon Willnauer 560d2c094e Fix issue where entire shards are lost due to too many open files exceptions and a but in Lucene's
IndexWriter / DirectoryReader where an existsing index was not detected and then deleted due to a wrong
creation mode. See LUCENE-4870

Closes #2812
2013-03-22 17:18:55 +01:00
Florian Schilling 1a67793a4b Added Script test for geo distance tests and modified GeoUtils.normalizePoint() 2013-03-22 13:34:18 +01:00
Simon Willnauer 075779a397 Call onMissing if doc has no value in the field.
Closes #2807
2013-03-21 22:45:17 +01:00
Simon Willnauer 064d272916 Respect offset and length when iterating over BytesRef in Uid. The length is starting at offset
Closes #2806
2013-03-21 19:29:05 +01:00
Simon Willnauer 5f05c2106f Use more efficient StemmerOverrideFilter from Lucene trunk
Closes #2800
2013-03-21 07:58:51 +01:00
Shay Banon ea698add72 move to 0.90.0.RC2 snap 2013-03-20 19:06:30 +01:00
Shay Banon a2f14b68e8 release 0.90.0.RC1 2013-03-20 19:05:08 +01:00
Florian Schilling f08d458545 # GeoShape Precision
The `geo_shape` precision could be only set via `tree_levels` so far. A new option `precision` now allows to set the levels of the underlying tree structures to be set by distances like `50m`. The def

## Example
```json
curl -XPUT 'http://127.0.0.1:9200/myindex/' -d '{
  "mappings" : {
      "type1": {
          "dynamic": "false",
          "properties": {
              "location" : {
                  "type" : "geo_shape",
                  "geohash" : "true",
                  "store" : "yes",
                  "precision":"50m"
              }
          }
      }
  }
}'
```

## Changes
- GeoUtils defines the [WGS84](http://en.wikipedia.org/wiki/WGS84) reference ellipsoid of earth
- DistanceUnits refer to a more precise definition of earth circumference
- DistanceUnits for inch, yard and meter have been defined
- Set default levels in GeoShapeFieldMapper to 50m precision

Closes #2803
2013-03-20 14:52:47 +01:00
Simon Willnauer 4705eb2959 Lazily initialize the delegate in BloomFilteredPostingsFormat to prevent unnecessary loading if bloomfilter terminates early 2013-03-20 12:43:17 +01:00
Simon Willnauer 747ce36915 Specialise the default codec to reuse Lucene41 files in the common case.
Closes #2799
2013-03-20 12:43:17 +01:00
Shay Banon 54e7e309a5 better comment... 2013-03-19 14:36:13 +01:00
Shay Banon d5beea4bba if multicast socket closes, try and restart it
also, throttle on socket failures, so it won't spin out of control...
relates to #2783
2013-03-19 11:20:47 +01:00
Shay Banon f4a212420b multicastSocket should be volatile as well... 2013-03-19 10:23:39 +01:00
Shay Banon c92207f483 broadcast API to by default ignore missing / illegal shard state
this happens for example because we list assigned shards, and they might not have been allocated on the relevant node yet, no need to list those as actual failures in some APIs
2013-03-19 10:22:43 +01:00
Shay Banon aca713d68e tar.gz distro by mistake include a windows lib 2013-03-18 22:46:04 +01:00
Shay Banon 566d1d13f7 fix javadoc 2013-03-18 22:04:31 +01:00
Clinton Gormley 2123ab591c Correct filter strategy opt: random_access_random to random_access_always 2013-03-18 20:17:26 +01:00
Shay Banon 7d9cef904b Field Data: optimize long type to use narrowest possible type automatically
closes #2795
2013-03-18 12:37:15 +01:00
Shay Banon 82072fc47f make ES compile with java 8
- that isAnnotationPresent bug is known, and probably will be fixed in later versions, but it costs us nothing to not use it now
- some tests fail, mainly due to consistent ordering expected from Map (within versions) which does not seem to be preserved, need to fix those tests to be agnostic to it
2013-03-18 01:33:09 +01:00
Shay Banon e347a626da use ImmutableList.Builder instead of ArrayList 2013-03-17 21:55:07 +01:00
Shay Banon 2ed6ea25cc fix logging message to include the index
also add the list of current indices
2013-03-16 22:58:45 +01:00
Shay Banon 111a13222e Mapping: dynamic flag is explicitly returned even when not set
fixes #2789
2013-03-16 01:29:22 +01:00
Simon Willnauer c25eb7defe Fix bug in RateLimiter.SimpleRateLimiter causing numeric overflow in StoreStats
Closes #2785
2013-03-15 23:36:31 +01:00
Shay Banon d5da8f22ff improve TODO comment 2013-03-15 21:46:02 +01:00
Simon Willnauer 0e3b88be35 add CamelCase support to Suggester where missing 2013-03-15 15:07:15 +01:00
Simon Willnauer e0eff7d9d3 Remove `sort_order` and `sort_mode` in favor of `order` and `mode`
Closes #2781
2013-03-15 13:57:39 +01:00
Simon Willnauer 33608c333f Add `sort_oder` and `sortOrder` as valid field names for defining the sort order in a Sort object.
Closes #2767
2013-03-15 08:42:19 +01:00
Simon Willnauer 5f20d81199 Make StupidBackoff the default smoothing model for phrase suggester
Closes #2780
2013-03-14 23:03:15 +01:00
Shay Banon 91c51ef05c minor cleanup suggest api
- make sure we close the parser
- fail when no content is provided in the rest request
- reuse the suggest parse element
2013-03-13 12:18:14 -07:00
Florian Schilling 25bd9cecd0 # REST Suggester API
The REST Suggester API binds the 'Suggest API' to the REST Layer directly. Hence there is no need to touch the query layer for requesting suggestions.
This API extracts the Phrase Suggester API and makes 'suggestion request' top-level objects in suggestion requests. The complete API can be found in the
underlying ["Suggest Feature API"](http://www.elasticsearch.org/guide/reference/api/search/suggest.html).

# API Example
The following examples show how Suggest Actions work on the REST layer. According to this a simple request and its response will be shown.

## Suggestion Request
```json
curl -XPOST 'localhost:9200/_suggest?pretty=true' -d '{
    "text" : "Xor the Got-Jewel",
    "simple_phrase" : {
        "phrase" : {
            "analyzer" : "bigram",
            "field" : "bigram",
            "size" : 1,
            "real_word_error_likelihood" : 0.95,
            "max_errors" : 0.5,
            "gram_size" : 2
        }
    }
}'
```
This example shows how to query a suggestion for the global text 'Xor the Got-Jewel'. A 'simple phrase' suggestion is requested and
a 'direct generator' is configured to generate the candidates.

## Suggestion Response
On success the request above will reply with a response like the following:
```json
{
    "simple_phrase" : [ {
        "text" : "Xor the Got-Jewel",
        "offset" : 0,
        "length" : 17,
        "options" : [ {
            "text" : "xorr the the got got jewel",
            "score" : 3.5283546E-4
        } ]
    } ]
}
```
The 'suggest'-response contains a single 'simple phrase' which contains an 'option' in turn. This option represents a suggestion of the
queried text. It contains the corrected text and a score indicating the probability of this option to be meant.

Closes #2774
2013-03-13 19:36:29 +01:00
Jörg Prante a127f2d2e8 avoiding NPE in Sigar FS 2013-03-13 10:05:59 -07:00