Commit Graph

1385 Commits

Author SHA1 Message Date
Shay Banon 1e937fd5d1 Allow index: "no" for _type
fixes #2696
2013-02-26 22:06:52 +01:00
Martijn van Groningen 7c53d22ce9 Moved resolveClosestNestedObjectMapper to MapperService 2013-02-26 17:48:02 +01:00
Igor Motov de243493c9 Changing dynamic index and cluster settings should work on master-only nodes
Fixes #2675
2013-02-26 08:54:46 -05:00
Shay Banon bd75b731c6 move to 0.90.0.Beta2 snap 2013-02-26 10:33:57 +01:00
Shay Banon ab3a59e0bf release 0.90.0.Beta1 2013-02-26 10:32:50 +01:00
Martijn van Groningen 2b5e3f5586 Fixed resolving closest nested object when sorting on a field inside nested object 2013-02-25 16:21:22 +01:00
Martijn van Groningen c751df5ee5 Removed unused nested children collector. 2013-02-25 14:13:59 +01:00
Shay Banon c7a05b1dda add helper method to know if ObjectMappers have a nested mapping 2013-02-25 13:40:05 +01:00
Shay Banon 6e3300efd3 better error message on nested sorting 2013-02-25 13:32:00 +01:00
Shay Banon 4bb4e49155 Empty list in ids query should not fail, but match no docs
relates to #2687
2013-02-25 12:51:34 +01:00
Shay Banon bde36647fb Terms/Ids filter: Support empty list of values, resulting in no match for it
closes #2687
also closes #2686
2013-02-25 12:26:49 +01:00
Shay Banon 4145d154bb add a test for empty lookup terms filter 2013-02-25 11:58:58 +01:00
Shay Banon 10ca4d5305 move internal stream facet type lookup to work with bytes 2013-02-25 10:57:18 +01:00
Lukas Vlcek a42f9491b5 fix typo in exception 2013-02-24 07:47:25 +01:00
Shay Banon 595e0e254e [Code refactoring] IndicesStats -> IndicesStatsResponse
fixes #1782
2013-02-23 14:23:36 +01:00
David Pilato 4c493ac71d Revert changes on *Request classes from issue
Relative to #2657
2013-02-23 10:37:56 +01:00
David Pilato a646e126e9 Display list of all available site plugins on /_plugins/ end point fix #2664 2013-02-23 09:34:06 +01:00
Shay Banon eea3a01765 only return 404 on actual index settings missing, on "_all", return 200
relates to #2676
2013-02-22 23:08:38 +01:00
Shay Banon 915019587d Get settings on empty node fails with ArrayIndexOutOfBoundsException[0]
fixes #2676
2013-02-22 23:08:33 +01:00
Igor Motov b8cc8e56c4 Improve stability of SimpleRobinEngineTests 2013-02-22 14:59:49 -05:00
Shay Banon ad70105c39 keep the rescorer builder consistent with other builders, without the use of setters 2013-02-22 14:06:39 +01:00
Shay Banon 03fdc6aa80 Query DSL: Terms filter to allow for terms lookup from another document
closes #2674
2013-02-22 14:04:10 +01:00
Shay Banon 6978aa2189 mark source as "safe" when copying it over 2013-02-22 12:59:41 +01:00
Shay Banon a234e45b59 fix boolean to is from get
relates to #2657
2013-02-22 12:45:56 +01:00
Igor Motov ec3492c67c Improve stability of the testReusePeerRecovery test 2013-02-21 16:06:33 -05:00
Shay Banon b7f5295674 update jsr166y adn jst166e to latest versions 2013-02-21 21:11:14 +01:00
Shay Banon 4753ffdf1e allow to set which queue implementation to use
expert setting, but still would be great to be able to control it
2013-02-21 20:07:40 +01:00
Ilya Nazarov da3d682f0e Check for java-6-openjdk-i386 in init.d
There is check for /usr/lib/jvm/java-6-openjdk-amd64, but no for 32-bit systems (/usr/lib/jvm/java-6-openjdk-i386).
2013-02-21 21:13:51 +07:00
Igor Motov 4ea4de6f8d Add logging information for releasing node lock 2013-02-20 17:53:27 -05:00
Shay Banon 7bb092440a facet refactoring, default collector base post implementation
automatically implement post based on collector
2013-02-20 15:36:11 +01:00
Igor Motov ce6f0e27bf Make file distribution among several disks configurable
Fixes #2650
2013-02-19 21:43:43 -05:00
David Pilato b7afa0f44e Fix test for Support trailing slashes on plugin _site URLs #2654 2013-02-19 21:16:47 +01:00
Martijn van Groningen 3b31c1216e Made the `term_vector` json field the leading way of configuring term vectors. Supported options: `no`, `yes`, `with_offsets`, `with_positions`, `with_positions_offsets` and`with_positions_offsets_payloads`. 2013-02-19 20:55:43 +01:00
Igor Motov 5b9e9a004a Make sure that in SitePluginTests http client connects to the correct node and closes the node after the test 2013-02-19 14:42:24 -05:00
Igor Motov f96c1f1e10 When a node is leaving LocalDiscovery cluster, rerouting should be performed on the master node 2013-02-19 13:14:33 -05:00
Igor Motov d126558dec Add check for health timeout to shardCleanup test 2013-02-19 13:12:26 -05:00
David Pilato 8ab9d2dd1f Support trailing slashes on plugin _site URLs fix #2654 2013-02-19 09:21:45 +01:00
Igor Motov cfaa859bb2 Improve stability of UpdateNumberOfReplicasTests 2013-02-18 20:12:39 -05:00
Igor Motov 4222478b18 Make it simpler to determine which version of state was used to calculate health 2013-02-18 20:02:29 -05:00
Igor Motov 5746c50ef9 Improve stability of shardsCleanup test 2013-02-18 19:35:12 -05:00
Igor Motov 183a74c866 Improve stability of testSimpleAwareness test 2013-02-18 19:31:07 -05:00
Martijn van Groningen 303e87fb69 Added support for sorting by fields inside one or more nested objects.
The sorting by nested field support has the following parameters on top of the already existing sort options:

nested_path - Defines the on what nested object to sort. The actual sort field must be a direct field inside this nested object. The default is to use the most immediate inherited nested object from the sort field.
nested_filter - A filter the inner objects inside the nested path should match with in order for its field values to be taken into account by sorting. Common case is to repeat the query / filter inside the nested filter or query. By default no nested_filter is active.
Either the highest (max) or lowest (min) inner object is picked for during sorting depending on the sort_mode being used. The sort_mode options avg and sum can still be used for number based fields inside nested objects. All the values for the sort field are taken into account for each nested object.

Closes #2662
2013-02-18 22:10:41 +01:00
Simon Willnauer 8db436f107 Remove backported Lucene 4 spatial code in favor of the released version in Lucene 4.1 2013-02-18 18:43:55 +01:00
Jeffrey Gerard 0dfc2169d7 Added Testcase and BugFix fixing #2626 where GeoShape intersects filter omitted matching docs.
SpatialPrefixTree#recursiveGetNodes uses an optimization that prevents
recursion into the deepest tree level if a parent node in the penultimate
level covers all its children.  This produces a bug if the optimization
happens both at indexing and at query/filter time.

This patch fixes the bug by disabling the optimization at indexing time
(to avoid adding overhead for query-heavy workloads).

See LUCENE-4770 for reference
2013-02-18 18:43:47 +01:00
David Pilato cc83c2f848 refactoring getter/setters
Fixes #2657
2013-02-18 11:09:32 -05:00
Martijn van Groningen ac2e6a3a4d Fixed nested facets with filters. 2013-02-18 11:01:18 -05:00
Simon Willnauer 24291d40f4 Expose CJKWidthTokenFilter and CJKBigramTokenFilter
Closes #2660
2013-02-18 11:01:17 -05:00
Shay Banon 547bd7abf2 add our own bloom filter implementation
uses more hash iterations, yet require less memory for the same fpp
relates to #2411
2013-02-18 11:01:17 -05:00
Igor Motov 512585da82 Fix race condition in adding TimeoutClusterStateListener
Fixes #2658
2013-02-18 11:01:17 -05:00
Shay Banon 435eabd4a0 allow to access the global node settings in a static manner 2013-02-18 11:01:17 -05:00
Shay Banon e365ecce10 fix check on which settings to change on 2013-02-18 11:01:17 -05:00
Shay Banon 73a447da86 initial facet refactoring
the main goal of the facet refactoring is to allow for two modes of facet execution, collector based, that get callbacks as hist match, and post based, which iterates over all the relevant hits
it also includes a some simplification of the facet implementation
2013-02-16 02:25:04 +01:00
Shay Banon 06b82a45d4 Simplified range syntax when using a query string
closes #2655
2013-02-15 01:30:55 +01:00
Shay Banon 4714a6acc9 Clear cache: allow to invalidate specific filter cache keys
closes #2653
2013-02-14 21:13:19 +01:00
Shay Banon c12c456192 add note on not using totalSize in merge 2013-02-14 14:30:46 +01:00
Shay Banon e8e3dd1c9d add 0.20.6 ver 2013-02-14 14:29:30 +01:00
Igor Motov 37f16127c5 Fix ScriptFilter cache key calculation
Fixes #2651
2013-02-14 06:13:26 -05:00
Igor Motov 6b49457d9d Optimize conversion to a cacheable DocIdSet 2013-02-13 21:04:54 -05:00
Shay Banon 883c593d7e delay reroute only after we publish that a shard has started 2013-02-14 00:10:52 +01:00
Shay Banon 681239b413 Warmers do not load field data cache for sorting on new segments
fixes #2649
2013-02-13 17:51:34 +01:00
Shay Banon f41eccc7a5 updating non dynamic settings throws an error now 2013-02-13 14:28:16 +01:00
Martijn van Groningen 2193a8e401 Let the update index settings action fail if non dynamic settings are changed for open indices.
Closes #2647
2013-02-13 13:10:56 +01:00
Shay Banon 5ad540a1aa possibly incorrect use of Lucene OneMerge.totalBytesSize
fixes #2643
2013-02-12 22:09:55 +01:00
Martijn van Groningen 3a2d40acd9 Added more trace logging related to finding master. 2013-02-12 21:40:12 +01:00
Shay Banon 5519f80abb add increased timeout waiting for relocation when running on small boxes 2013-02-12 21:23:18 +01:00
Martijn van Groningen fc13499ff5 Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued.
The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value.

Relates to #2634
2013-02-12 20:38:24 +01:00
Shay Banon 7d13545e33 delete indices before running the tests 2013-02-12 19:28:48 +01:00
Shay Banon 668bcd0eb7 Bulk execution while a shard is replication might send erroneous version conflict failures for certain items
fixes #2642
2013-02-12 17:38:06 +01:00
Simon Willnauer a7bbab7e87 # Rescore Feature
The rescore feature allows te rescore a document returned by a query based
on a secondary algorithm. Rescoring is commonly used if a scoring algorithm
is too costly to be executed across the entire document set but efficient enough
to be executed on the Top-K documents scored by a faster retrieval method. Rescoring
can help to improve precision by reordering a larger Top-K window than actually
returned to the user. Typically is it executed on a window between 100 and 500 documents
while the actual result window requested by the user remains the same.

# Query Rescorer

The `query` rescorer executes a secondary query only on the Top-K results of the actual
user query and rescores the documents based on a linear combination of the user query's score
and the score of the `rescore_query`. This allows to execute any exposed query as a
`rescore_query` and supports a `query_weight` as well as a `rescore_query_weight` to weight the
factors of the linear combination.

# Rescore API

The `rescore` request is defined along side the query part in the json request:

```json
curl -s -XPOST 'localhost:9200/_search' -d {
  "query" : {
    "match" : {
      "field1" : {
        "query" : "the quick brown",
        "type" : "boolean",
        "operator" : "OR"
      }
    }
  },
  "rescore" : {
    "window_size" : 50,
    "query" : {
      "rescore_query" : {
        "match" : {
          "field1" : {
            "query" : "the quick brown",
            "type" : "phrase",
            "slop" : 2
          }
        }
      },
      "query_weight" : 0.7,
      "rescore_query_weight" : 1.2
    }
  }
}
```

Each `rescore` request is executed on a per-shard basis within the same roundtrip. Currently the rescore API
has only one implementation (the `query` rescorer) which modifies the result set in-place. Future developments
could include dedicated rescore results if needed by the implemenation ie. a pair-wise reranker.
*Note:* Only regualr queries are rescored, if the search type is set to `scan` or `count` rescorers are not executed.

Closes #2640
2013-02-12 17:10:00 +01:00
Shay Banon c65aff7775 Index with no replicas might loose on going documents while relocating a shard
fixes #26421
2013-02-12 17:03:28 +01:00
Martijn van Groningen e54f010a4d Also support camel case notation for minimal norwegian. 2013-02-12 16:39:11 +01:00
morsegel ca7920a398 added norwegian minimal stemmer 2013-02-12 16:32:38 +01:00
Igor Motov f98bd654a8 Fix filter cache stats calculation
Fixes #2609
2013-02-11 10:28:53 -05:00
uboness a2b87e28f6 fixed a bug in PrioritizedThreadPoolExecutor:
now execute(Runnable) satisfies the priority and fifo nature of same-priority runnables
2013-02-09 04:20:16 +01:00
uboness eef3610e12 fixed a bug in PrioritizedThreadPoolExecutor:
now execute(Runnable) verifies the command is added as Comparable
2013-02-09 03:33:12 +01:00
uboness 678a8664f6 fixed a bug in PrioritizedThreadPoolExecutor:
now execute(Runnable) verifies the command is added as PrioritizedRunnable
2013-02-09 03:26:52 +01:00
uboness 6d9048f8cc added priority support for cluster state updates:
* URGENT:
    * cluster_reroute (api)
    * refresh-mapping
    * cluster_update_settings
    * reroute_after_cluster_update_settings
    * create-index
    * delete-index
    * index-aliases
    * remove-index-template
    * create-index-template
    * update-mapping
    * remove-mapping
    * put-mapping
    * open-index
    * close-index
    * update-settings

* HIGH
    * routing-table-updater
    * zen-disco-node_left
    * zen-disco-master_failed
    * shard-failed
    * shard-started

* NORMAL
    * all other actions
2013-02-09 01:14:57 +01:00
Simon Willnauer f5331c9535 Cleanup NumericFieldData. FieldData interfaces are reduced to long and double while internal
represenations still operate on the actual datatypes.
2013-02-08 20:58:36 +01:00
Martijn van Groningen 1189a2c2c2 Extended mv sorting integration test 2013-02-08 15:24:56 +01:00
Martijn van Groningen 8c7779057c Added sort by field that have multiple values per document.
Closes #2634
2013-02-08 13:28:40 +01:00
Simon Willnauer 033d6e4306 don't use substraction for comparison if datatypes can overflow 2013-02-08 10:07:31 +01:00
Martijn van Groningen f97021b165 Fixes size assertion failure. 2013-02-07 16:50:54 +01:00
Martijn van Groningen e2cb7edb08 Added more info to assert 2013-02-07 13:52:25 +01:00
Martijn van Groningen e72e323c8a Attempt to fix "No active shards" failure 2013-02-07 10:14:10 +01:00
Lee Hinman ed43ad07d7 Throw a more meaningful message when no document is specified for indexing 2013-02-06 22:33:02 +01:00
Florian Schilling a52e01f3e5 Remove XTermsFilter and UidFilter in favour of Lucene 4.1 TermsFilter 2013-02-06 18:45:05 +01:00
Igor Motov 6890c9fa62 Move action.wait_on_mapping_change setting to pom 2013-02-06 11:48:58 -05:00
Igor Motov ed09ba0a18 Improve stability of RecoveryPercolatorTests
Without "action.wait_on_mapping_change" setting set to true, the test node might get shutdown before updated mapping is saved.
2013-02-05 14:53:46 -05:00
Igor Motov 8277833f8d Fix settings processing in WordDelimiterTokenFilterFactory 2013-02-05 10:03:00 -05:00
Martijn van Groningen 19295280d9 Made sure that wrapped child query / parent query gets rewritten only once. 2013-02-05 10:27:31 +01:00
Igor Motov 9e89323ad2 Add proper cleanup to InternalSettingsPerparerTests 2013-02-04 19:58:40 -05:00
Martijn van Groningen bc667c378e Made SoftWrapper fields final. 2013-02-04 14:47:36 +01:00
Martijn van Groningen 8109d13733 Use CacheRecycler when resolving parent docs in TopChildrenQuery. 2013-02-04 12:46:30 +01:00
Martijn van Groningen 9c3a86875b Removed `execution_type` for has_child and has_parent. 2013-02-04 11:37:40 +01:00
Igor Motov 20ce01bd53 Add additional query validation to the terms query parser
Fixes #2608
2013-02-03 09:44:16 -05:00
Shay Banon ebc0c8cc6d when we fix maxMergeAtOnce, make sure to not set it to 1 as its an illegal value 2013-02-01 19:00:01 +01:00
Shay Banon a8c9e580ed add getMaxOrd, and properly document the difference between it and numOrds 2013-02-01 16:13:13 +01:00
Shay Banon 6f1932ab67 support yaml detection on char sequence 2013-02-01 12:46:19 +01:00
Simon Willnauer 6468c15446 check for == 0 rather than > 0 2013-02-01 11:11:47 +01:00
Simon Willnauer c18ae4a194 fix getMemorySizeInBytes in SparseMultiArrayOrdinals 2013-02-01 11:09:09 +01:00