Commit Graph

1170 Commits

Author SHA1 Message Date
Simon Willnauer c1a9c802f1 lucene 4: XContentParser now has bytesOrNull and returns bytesref directly 2012-11-12 13:44:33 +01:00
Simon Willnauer 479f1784e8 lucene 4: converted queryparser to lucene classic query parser 2012-11-12 13:44:32 +01:00
Simon Willnauer 5d47ad4648 lucene 4: upgraded FuzzyQueryParser + Builder to use integer edit distance rather
than floats (bw compatible)
2012-11-12 13:44:32 +01:00
Igor Motov b1eaec6c6a lucene 4: change Unicode utils to use BytesRef instead of UTF8Result 2012-11-12 13:44:32 +01:00
uboness c3633ab99f lucene 4: changed InternalIndexShard#checkIndex to use the new fixIndex and indexExists apis 2012-11-12 13:44:32 +01:00
Igor Motov 8009b80481 lucene 4: fix access to segment name due to SegmentInfo refactoring 2012-11-12 13:44:32 +01:00
Shay Banon 4b84078f91 lucene 4: text comparator should always work on bytes 2012-11-12 13:44:32 +01:00
Martijn van Groningen 65ce3aea57 lucene 4: Upgraded the function/sort classes. 2012-11-12 13:44:32 +01:00
Martijn van Groningen 48b8d0544f lucene 4: Moved SearchScript from IndexReader to AtomicReader. This also touches the seach/lookup classes 2012-11-12 13:44:32 +01:00
Martijn van Groningen d820bfe11b lucene 4: Changed from BytesReference to Text as internal term representation for facet keys. Text now also implements comparable. 2012-11-12 13:44:32 +01:00
Igor Motov b128b7a750 lucene 4: use CharArraySet for stem exclusions, stop words and articles and fix analyzer namespaces 2012-11-12 13:44:32 +01:00
Igor Motov 1cc5ee7ad9 lucene 4: implement createComponents in Analyzers 2012-11-12 13:44:32 +01:00
Igor Motov 6fad75df82 lucene 4: remove Pattern tokenizer and filter 2012-11-12 13:44:32 +01:00
Igor Motov 097cb2dac7 lucene 4: migrate char filter from CharStream to Reader 2012-11-12 13:44:31 +01:00
Shay Banon f572a7bcf7 lucene 4: no close on searcher anymore 2012-11-12 13:44:31 +01:00
Shay Banon ed03741353 lucene 4: hashCode and equals for Text and BytesReference
now that we are going to use those more in places like facets, they need to implement equals and hasCode to be used in hashes
2012-11-12 13:44:31 +01:00
Martijn van Groningen 15c9cd5142 lucene 4: Field name no longed interned when loading field data cache and return empty field data cache for fields that don't exist. 2012-11-12 13:44:31 +01:00
Martijn van Groningen 454954e7be lucene 4: Fix field data, facets and field comparators 2012-11-12 13:44:31 +01:00
Shay Banon 81d148b4e4 lucene 4: fix warmup process
also removed ExtendedIndexSearcher, we should do whats needed with the new context and leaves methods
2012-11-12 13:44:31 +01:00
Shay Banon 0c24928ef4 lucene 4: fix similarity packaging 2012-11-12 13:44:31 +01:00
Shay Banon f4418fb181 lucene 4: fix segments info usage 2012-11-12 13:44:31 +01:00
Shay Banon 7972f6f959 lucene 4: fix call to expungeDeletes 2012-11-12 13:44:31 +01:00
Shay Banon 386c2ebdb9 lucene 4: remove bloom cache
we can add bloom cache, if we need it, as a codec on the uid field
we still need to rewrite the UidFilter to not use bloom, but that will be the regular one
2012-11-12 13:44:31 +01:00
Igor Motov 05138bb2fb lucene 4: upgrade analyzers 2012-11-12 13:44:30 +01:00
Shay Banon 7aacc8d448 lucene 4: upgrade store/dir 2012-11-12 13:44:30 +01:00
Shay Banon 3d4ca81c29 remove XIndexWriter
removing the buffered deletes bloom filter no longer requires setting the bloom filter on it
2012-11-12 13:44:30 +01:00
Shay Banon f9b0fcb3a3 remove BufferedDeletesStream
by default, we will put a bloom filter code on the _uid field, so no need for the optimization of using bloom filters when trying to delete a doc by _uid term per segment
2012-11-12 13:44:30 +01:00
Shay Banon edaa65dba2 Multi Match: Wrongly defaults to dis_max instead of bool
fixes #2397
2012-11-10 14:59:45 +01:00
Njal Karevoll f33e353259 The index of the next RestFilter must be incremented before the current filter starts processing.
Otherwise, synchronous filters will not work. For example, the following filter would cause a StackOverflowError:

public class SimpleRestFilter extends RestFilter {
    @Override
    public void process(RestRequest request, RestChannel channel, RestFilterChain filterChain) {
        filterChain.continueProcessing(request, channel);
    }
}
2012-11-09 22:04:09 +01:00
Shay Banon a8e43578a2 Adding a type with _source or _all enabled fails, when these are disabled in index
fixes #2394
2012-11-09 17:21:25 +01:00
Martijn van Groningen 4a9faac470 Added must/should/mustNot method variants that accepts vararg FilterBuilder instances. 2012-11-06 14:23:03 +01:00
Shay Banon 31a8e92b8e Node Stats: add max content length to http info 2012-11-06 11:37:23 +01:00
Shay Banon 0d5530e55f Node Stats: add available processors to OS info 2012-11-06 11:31:23 +01:00
Shay Banon 33900476f4 Node Stats: Add largest thread pool count per thread pool stats
closes #2382
2012-11-06 11:25:38 +01:00
Igor Motov af1e8c0eb1 Add auto index creation on update request
Fixes #2375
2012-11-02 10:18:51 -04:00
Martijn van Groningen ef25ac2414 Fixed that the `ignore_indices` option isn't set in multi search api. Closes #2380 2012-11-02 10:58:02 +01:00
Aaron Dixon bd9a5bfa0c fixed issue2371 (incorrect behavior of path_match) 2012-11-01 22:11:33 +01:00
Chris Male 768b8b4d2b Changed SpatialRelation contains to within 2012-11-01 22:03:47 +01:00
Igor Motov 23f7b0002a Deleting a non-existent warmer shouldn't cause request to hang
Fixes #2363
2012-11-01 21:49:54 +01:00
Igor Motov 7aa3f51b6e Simplify plugin installation troubleshooting
Add check for write access to plugins directory before trying to install plugin. Support verbose mode that prints out intermediate exceptions. Don't install plugin is plugin directory already exists. Fixes #2357
2012-11-01 21:47:15 +01:00
Igor Motov a2628b5eb2 Upsert should return fields Fixes #2362 2012-11-01 21:43:23 +01:00
Shay Banon d8013aecc3 add 0.19.11 and 0.19.12 versions 2012-10-24 01:41:46 +02:00
Igor Motov 29928a9e15 Add test for percolation with the _size field enabled 2012-10-23 23:08:52 +02:00
Igor Motov a760f906e5 Percolation shouldn't fail when the _size field is enabled 2012-10-23 23:08:52 +02:00
Shay Banon 227250aaa0 add fvh and plain shortcuts for highlighter types 2012-10-23 02:28:17 +02:00
Igor Motov c551f93cae Add highlighter type switch 2012-10-23 02:25:55 +02:00
Shay Banon 4573abc737 nested facet fails with filter caching turned off
fixed #2340
2012-10-22 20:04:57 +02:00
Shay Banon 04eabbd38a Mapping: string mapping to automatically set omit_norms to true and index_options to docs when setting index to not_analyzed
closes #2349
2012-10-22 19:01:01 +02:00
Shay Banon e40f4d2b1a add cluster name back to cluster state API 2012-10-22 15:35:04 +02:00
Shay Banon a3ffc65909 add 0.20.0.RC2 to versions on master 2012-10-22 01:56:23 +02:00
Shay Banon c1489be816 forgot to mark 0.21.0 as snap 2012-10-22 01:55:33 +02:00
Shay Banon 6ec0071fa4 move master to be 0.21.0 Beta1 snap 2012-10-22 01:51:20 +02:00
Shay Banon fe980343be release 0.20.0.RC1 2012-10-22 01:48:09 +02:00
Shay Banon c0e2eb5f5f better workaround for snappy error output
instead of setting the err stream to null, set it to a null output print stream
2012-10-22 00:15:12 +02:00
Shay Banon faa7501715 Aliases API times out when actions end up with no actual change to aliases
closes #2345
2012-10-21 02:45:56 +02:00
Shay Banon a3e7daf415 Auto create index to support white/black lists of index patterns
closes #2344
2012-10-21 02:27:30 +02:00
Martijn van Groningen 6a936a9efc Added method that accepts a map as script parameters 2012-10-21 00:46:19 +02:00
Martijn van Groningen ee5df74a6b Fixed delete by query issue with index aliases and nested mappings.
The issue was that under these circumstances the delete by query operation would run forever.
What also is fixed is that during shard recovery when delete by query is replayed nested docs
are also deleted. Closes #2302
2012-10-20 23:39:15 +02:00
Shay Banon 246dc1d992 formatting 2012-10-19 09:48:48 +02:00
Simon Willnauer b6a83fd8b2 #2332 support CustomScoreQuery highlighting and expose QueryBuilder on CustomScoreQuery via Java API 2012-10-19 09:23:16 +02:00
Shay Banon 2c7850584d move all allocations to be Decision base
this again will allow to provide better output as to why a specific decision has been made (like a shard needing to move from a node)
2012-10-16 10:10:06 -04:00
Shay Banon a963bc1dc9 Setting index_analyzer incorrectly sets analyzer
fixes #2329
2012-10-16 09:19:47 -04:00
Shay Banon c2073c343d improve allocation decision to allow to explain why a decision has been made
building the infra to support explaining why an allocation decision has been made, for example, why a shard is not allocated on a specific node
2012-10-16 09:03:54 -04:00
Shay Banon 384286d463 Mapping: Using _default_ mapping type with _source excludes (or array based config) can cause the array to grow indefinitely
closes #2317
2012-10-11 19:02:06 -07:00
Shay Banon 27562270ca simplify cleaning up non allocation shard code
its pretty complex today, simplify it by doing a single iteration, and handling both dangling shard cleanup on existing indices and shards that have been reallocated to a differetn node
2012-10-11 18:46:24 -07:00
Shay Banon 320c9b7681 close parser when parsing update body 2012-10-06 19:47:11 +02:00
Shay Banon 2fa017db11 Terms filter: Add `or` and `or_nocache` execution modes
closes #2306
2012-10-05 18:21:58 +02:00
Shay Banon 9122befdfb add logging on the size of message received 2012-10-04 20:33:25 +02:00
Shay Banon 8ea019aa82 formatting 2012-10-04 20:07:57 +02:00
Henrik Lindström 4d969add80 If types and index mappings are empty consider the mapping as an "empty" mapping and return status 200. 2012-10-04 20:07:10 +02:00
Shay Banon 2c150419ce add additional (internal) flush option
this flush option allows to just commit and index writer, without actually clearing the translog
2012-10-03 16:36:13 -04:00
Shay Banon 2532761c51 increase the clean interval of filter cache 2012-10-01 15:30:14 -04:00
Shay Banon 9096baf7d9 1.7 compiler failure
fixes #2294.
2012-10-01 08:11:28 -04:00
Shay Banon 9ad8c08505 fix segments per tier to the proper 10 default 2012-09-30 10:12:03 -04:00
Shay Banon 433b189eb4 provide the timer for netty
this helps to control the thread name of the thread it created
2012-09-30 09:11:03 -04:00
Shay Banon 05653cbdaa move cleaning filter cache on closed readers to separate thread
improve cleaning the global weighted cache when a reader closes, move it to a separate thread, so iterating over the cache entries will nto happen on each segment closed, but instead be "bulked"
2012-09-29 14:06:28 +02:00
Shay Banon 9dd8e1ea05 refresh thread pool to use scaling with default size of 10 2012-09-29 13:21:18 +02:00
Shay Banon 5bf566660b fix detection when JTS / spatial4j are not available 2012-09-28 22:29:08 +02:00
Shay Banon 7d6c567e6f return null instead of empty doc set
null can more easily be optimized in other filters
2012-09-28 17:18:07 +02:00
Martijn van Groningen 51e69e1a9e Fixed NPE when using `has_parent` or `has_child` filter/query.
The NPE occurred when for an arbitrary segment no parent documents exist for a has_parent filter/query and no child documents exist for a has_child filter/query.

Closes #2297
2012-09-28 17:08:30 +02:00
Shay Banon 2df6b4b0ea more guice to use Objects from guava 2012-09-28 14:03:41 +02:00
Shay Banon f9fc9bdb20 query_string on multiple fields with "*" fails in 0.19.9
fixed #2296
2012-09-28 14:00:43 +02:00
Shay Banon d8a3096a1a make sure to pass the updated settings from the plugins to the modules with transport client 2012-09-28 13:47:34 +02:00
Shay Banon 613c70c289 introduce TransportResponse
a class that needs to be used when sending a response over the transport layer, with an option to have headers
2012-09-27 18:05:16 +02:00
Shay Banon 15453272f7 fetching a shape should not occur on separate thread 2012-09-26 23:47:27 +02:00
Shay Banon cfe7504d1c introduce TransportRequest (with optional headers)
introduce a new class, TransportRequest, which includes headers. This class can be used when sending requests over the transport layer, and ActionRequest also extends it now.
This is the first phase of the refactoring part in the transport layer and action layer to allow for simpler implementations of those as well as simpler "filtering" capabilities in the future
2012-09-26 23:46:28 +02:00
Shay Banon 15fbbd43ce Bulk Delete item when broadcast to all shard (lack of routing) might not be applied correctly
closes #2285.
2012-09-24 14:04:10 +02:00
Martijn van Groningen 81a6940ad3 Fixed score explain is for `custom_filters_score` query.
Only the explain of the filter was included. This fix adds an explain for the inner query and wraps it in a top-level explanation.
2012-09-24 14:00:34 +02:00
Shay Banon 6e66f45f58 minor geo shape fetch improvements 2012-09-23 21:31:13 +02:00
Chris Male 05e0b4d4e0 Added ShapeFetchService with support in GeoShapeQueryParser/FilterParser 2012-09-23 21:24:13 +02:00
Shay Banon df0ff176d4 add simpler way to set the epoll bug workaround flag for netty 2012-09-22 19:50:43 +02:00
Shay Banon f73d684c76 use the version to deserialize published cluster state 2012-09-22 19:47:50 +02:00
Shay Banon d5e11fdcd9 disconnect from unicast nodes before raising event 2012-09-22 19:38:34 +02:00
Shay Banon aa435a288b add node version to each request/response transport message
a version associated with each request, and having it associated with the stream when (de)serializing, can allow us in the future to better support cross version communication
2012-09-22 02:35:29 +02:00
Shay Banon cc7eb263be Transport: Add header token
This allows to more easily identify when illegal content is being sent to the transport port
closes #2280
2012-09-21 23:56:27 +02:00
Shay Banon a3af3d2f47 Norms field is ignored when doing a match_all query
we shouldn't optimize a match all query if it has norms associated with it, fixes #2277
2012-09-21 18:57:21 +02:00
Chris Male 4f5e62e988 Added MultiPolygon parsing and serialization support 2012-09-21 14:03:21 +02:00
Martijn van Groningen 8080fdc509 Added types exists api
The types exists api checks whether one or more types exists in one or more indices.

## Example usage
curl -XHEAD 'localhost:9200/twitter/tweet'

## Options
* `index` - One or more indices. Either specified as query string parameter or in the uri path.
* `type` - One or more types. Either specified as query string parameter or in the uri path.
* `ignore_missing` -  Determines what type of indices to exclude from a request. The option can have the following values: `none` or `missing`.

Closes #2273
2012-09-21 10:21:32 +02:00
Chris Male 6fc0b83e07 Upgraded to Spatial4j 0.3 2012-09-20 23:55:51 +02:00
Shay Banon 0fadbf2177 more easily add a field with boost to multi match builder 2012-09-20 12:38:13 +02:00
Shay Banon 83a39bd509 improve test by not waiting for green state, no need 2012-09-20 12:00:58 +02:00
Shay Banon f8e1291243 global node indices level queries to be created by guice 2012-09-20 11:54:46 +02:00
Shay Banon 4e19de1a08 don't serialize ignore_unmapped to mapping on default value
fix ignore_unmapped being serialized as part of the mappings even though it has a default value and not explicitly set
2012-09-19 20:54:13 +02:00
Shay Banon 7f6f001d15 better reason logged when not master 2012-09-19 18:28:29 +02:00
Martijn van Groningen afc99ac42c Added id cache size to nodes stats api
Also added bloom cache size the node *rest* api
Closes #2264
2012-09-19 13:16:24 +02:00
Shay Banon 86e88a86b8 expose the highlighter builder Field constructor 2012-09-19 11:47:48 +02:00
Martijn van Groningen d5aa35e0ea Added better error handling for has_child, has_parent and top_children.
If has_parent, has_child or top_children are executed incorrectly then a better exception is thrown. This gives a better error description when one of these queries or filters is being used in count api.

Closes #2261
2012-09-18 13:26:23 +02:00
Shay Banon 7924115b90 Disable allocation: New indices allocation not to be disabled by default
When setting cluster.routing.allocation.disable_allocation, it causes new indices primary shards to not be allocated. By default, new indices created should allow to, at the very least, allocate primary shards so they become operations. A new setting, cluster.routing.allocation.disable_new_allocation, allows to also disable "new" allocations.
closes #2258.
2012-09-17 16:00:55 +02:00
Shay Banon 90e0a70e0e cancel allocation command to allow_primary to be cancelled 2012-09-17 12:31:33 +02:00
Martijn van Groningen 42864f7481 Fixed wrong jdoc 2012-09-17 12:00:44 +02:00
Shay Banon afca5ef15f The reroute command allows to explcitiyly execute a cluster reroute allocation command including specific commands. For example, a shard can be moved from one node to another explicitly, an allocation can be canceled, or an unassigned shard can be explicitly allocated on a specific node.
Here is a short example of how a simple reroute API call:

    curl -XPOST 'localhost:9200/_cluster/reroute' -d '{
        "commands" : [
            {"move" : {"index" : "test", "shard" : 0, "from_node" : "node1", "to_node" : "node2"}},
            {"allocate" : {"index" : "test", "shard" : 1, "node" : "node3"}}
        ]
    }'

An importnat aspect to remember is the fact that once when an allocation occurs, the cluster will aim at rebalancing its state back to an even state. For example, if the allocation includes moving a shard from `node1` to `node2`, in an "even" state, then another shard will be moved from `node2` to `node1` to even things out.

The cluster can be set to disable allocations, which means that only the explicitl allocations will be performed. Obviously, only once all commands has been applied, the cluster will aim to be rebalance its state.

Anohter option is to run the commands in "dry_run" (as a URI flag, or in the request body). This will cause the commands to apply to the current cluster state, and reutrn the resulting cluster after the comamnds (and rebalancing) has been applied.

The commands supporterd are:

* `move`: Move a started shard from one node to anotehr node. Accepts `index` and `shard` for index name and shard number, `from_node` for the node to move the shard "from", and `to_node` for the node to move the shard to.
* `cancel`: Cancel allocation of a shard (or recovery). Accepts `index` and `shard` for index name and shar number, and `node` for the node to cancel the shard allocation on.
* `allocate`: Allocate an unassigned shard to a node. Accepts the `index` and `shard` for index name and shard number, and `node` to allocate the shard to. It also accepts `allow_primary` flag to explciitly specify that it is allowed to explciitly allocate a primary shard (might result in data loss).

closes #2256
2012-09-15 22:56:14 +02:00
Martijn van Groningen 9c04e6a4f4 Clean up has_parent query code Renamed some exception messages and removed method from builder 2012-09-14 19:06:12 +02:00
Martijn van Groningen cfe76546f2 Added has_parent query The `has_parent` query works the same as the `has_parent` filter, by automatically wrapping the filter with a constant_score. It has the same syntax as the `has_parent` filter. Closes #2254 2012-09-14 17:34:01 +02:00
Shay Banon 7bf233a174 fix cleanup of of has child/parent
check if the collected uids were processed before trying to push them back to the cache
2012-09-14 15:01:21 +02:00
Martijn van Groningen 3cd54fc4ee Improve `has_child` filter / query performance (#2251) Added a new has_child filter implementation, that works _uid based instead of bitset based. This implementation is about ~2 till ~6 times (depending on the query) faster than the already existing bitset based implementation. 2012-09-14 14:37:29 +02:00
Shay Banon 4dddeeb30c When configuring mmapfs it is not used since 0.19.7, closes #2253. 2012-09-14 14:30:44 +02:00
Shay Banon ef9974ce2c add serialization options for allocation commands 2012-09-14 14:26:39 +02:00
Martijn van Groningen 2bd9b3aed0 Added `has_parent` filter (#2243)
The `has_parent` filter accepts a query and a parent type. The query is executed in the parent document space, which is specified by the parent type. This filter return child documents which associated parents have matched. For the rest `has_parent` filter has the same options and works in the same manner as the `has_child` filter.

This is an experimental filter.

Filter example
###################
```
{
    "has_parent" : {
        "parent_type" : "blog"
        "query" : {
            "term" : {
                "tag" : "something"
            }
        }
    }
}
```
The `parent_type` field name can also be abbreviated to `type`.

Memory considerations
###############
With the current implementation, all _id values are loaded to memory (heap) in order to support fast lookups, so make sure there is enough mem for it.

This issue originates from issue #792
2012-09-13 13:35:45 +02:00
Shay Banon e530f03b94 internal custom allocation commands
add support for internal custom allocation commands, including allocation, move, and cancel (shard).
also, fix #2242, which causes the cluster state to be in inconsistent state when a shard being the source of relocation is failed
2012-09-12 15:13:27 +02:00
Martijn van Groningen b6a9bd9a31 - Fixed boosting per field with multi_match query. 2012-09-12 11:43:33 +02:00
Shay Banon fbd352b448 sync with latest guava version of abstract future 2012-09-07 11:13:43 +02:00
Shay Banon 8038616fbc add field with text/bytes and builder string 2012-09-06 20:47:13 +02:00
Shay Banon b26fd600f0 allow to configure cors on http
Allow to configure cors on http, if its enabled or not, and what it should output
2012-09-06 20:35:36 +02:00
Karel Minarik f659cad8d6 Added proper headers for cross-origin resource sharing (CORS) with Ajax
Previously, when responding to Ajax requests, elasticsearch did not send proper headers for
cross-origin resource sharing (CORS) -- see issues #828, #2186.

With this commit, Ajax requests should be working. Example:

    jQuery.ajax({
      url: "http://localhost:9200/_search",
      type: "POST",
      contentType: 'application/json; charset=UTF-8',
      success: function(data) { console.log(data) }
    });

See:

* http://www.nczonline.net/blog/2010/05/25/cross-domain-ajax-with-cross-origin-resource-sharing/
* http://www.w3.org/TR/cors/#access-control-allow-headers-response-header

Closes #2186, fixes #828
2012-09-06 17:14:55 +02:00
Shay Banon f2db8eaca2 IndexMetaData.Builder.Index(String name) isn't renaming this.index
fixed #2236.
2012-09-06 16:30:05 +02:00
Martijn van Groningen 33e7d769ca Fixed no valid missing index type error. 2012-09-05 19:20:53 +02:00
Shay Banon 24ce2ef537 handle EOF when handling arrays as well 2012-09-05 11:40:39 +02:00
Henrik Lindström 26feea8a58 Throw MapperParserException if trying to parse value as object. 2012-09-04 16:12:02 +02:00
Shay Banon 92dfb2b809 Nodes level API: Allow to specify `data:true` to only execute the APIs on data nodes, closes #2232 2012-09-04 00:03:06 +02:00
Shay Banon 0de8c06900 fix logging if millisecond value in load field data 2012-09-03 23:53:50 +02:00
Shay Banon a42159f8d5 Shard Allocation: `index.routing.allocation....` settings do not "remove" the setting on empty string, closes #2229. 2012-09-03 16:44:23 +02:00
Shay Banon edfbfc1c87 Update Settings API: Allow body request to be wrapped with `settings` element to conform with other APIs, closes #2227. 2012-09-02 17:40:08 +02:00
Shay Banon 6c3847b0a9 move spatial4j and jts to be optional dependencies
allowing data and client nodes to work without them, disabling shapes if needed
2012-09-01 00:05:49 +02:00
Kenn Knowles e66bd4359a Fix reversed ShingleFilter constructor arguments 2012-08-31 22:25:07 +02:00
Martijn van Groningen 9b29950997 Added fields option to explain api. #2203 2012-08-31 22:19:09 +02:00
Martijn van Groningen cd0e1226e1 Added a global ignore_malformed index setting. #2220 Also extended the ignore_malformed support to TTL, Ip and timestamp field types. 2012-08-31 22:10:46 +02:00
Shay Banon 888b7cc48f only print failure to reconnect to node from other nodes after several retries 2012-08-31 17:14:04 +02:00
Martijn van Groningen dea2de3304 Add ignore_indices option for search, multisearch, count and other Broadcast request. #2209 2012-08-27 15:36:14 +03:00
Martijn van Groningen 1d4aee6086 - Explain api opens 2 engine searchers, but closes only 1 engine searcher. Closes #2206 2012-08-27 12:20:02 +03:00
Shay Banon da61ddee3c 0.19.9 no longer supports /_all/_status, closes #2205. 2012-08-25 14:18:48 +03:00
Shay Banon dfa67bf071 allow to register custom state associated with cluster state 2012-08-25 00:40:44 +03:00
Martijn van Groningen bbe735f2cc Fixed issue #2197 2012-08-25 00:38:26 +03:00
uboness b4b33bb205 Local node master listener
* Fixed an issue where dynamic update to minimum_master_nodes settings would not take immediate effect
* Added LocalNodeMasterListener support to the ClusterService. Enables listening to when the local node becomes/stopped being a master
2012-08-24 02:25:13 +02:00
Shay Banon 36ff6c9b8b add a lock not allowing connecting to nodes while shutting down 2012-08-24 02:23:43 +02:00
Shay Banon 333293babd ignore cancelled key exception as well 2012-08-23 23:35:48 +02:00
Shay Banon 072fcaa760 Bulk UDP, closes #2201
A Bulk UDP service is a service listening over UDP for bulk format requests. The idea is to provide a low latency UDP service that allows to easily index data that is not of critical nature.

The Bulk UDP service is disabled by default, but can be enabled by setting `bulk.udp.enabled` to `true`.

The bulk UDP service performs intenral bulk aggregation of the data and then flushes it based on several parametres:

* `bulk.udp.bulk_actions`: The number of actions to flush a bulk after, defaults to `1000`.
* `bulk.udp.bulk_size`: The size of the current bulk request to flush the request once exceeded, defaults to `5mb`.
* `bulk.udp.flush_interval`: An interval after which the current request is flushed, regarldess of the above limits. Defaults to `5s`.
* `bulk.udp.concurrent_requests`: The number on max in flight bulk requests allowed. Defaults to `4`.

The network settings allowed are:

* `bulk.udp.host`: The host to bind to, defualts to `network.host` which defaults to any.
* `bulk.udp.port`: The port to use, defaults to `9700-9800`.

Here is an example of how it can be used:

    > cat bulk.txt
    { "index" : { "_index" : "test", "_type" : "type1" } }
    { "field1" : "value1" }
    { "index" : { "_index" : "test", "_type" : "type1" } }
    { "field1" : "value1" }

    > cat bulk.txt | nc -w 0 -u localhost 9700
2012-08-23 22:17:37 +02:00
Shay Banon ba98b04081 fix wrong check on close 2012-08-23 20:58:40 +02:00
Shay Banon bb609f696f add 0.19.10 2012-08-23 13:52:57 +02:00
Shay Banon 1f3b0d4144 Reduce index.shard.recovery.concurrent_streams from 5 to 3 to reduce the load when doing recovery, closes #2199.
the index.shard.recovery.concurrent_streams controls how many streams are opened from a recovery source to a recovery target to transfer index files. Reduce it from 5 to 3 to reduce the load when doing recovery (for example, due to relocation).

Note, recent changes in network buffering will mean that recovery will progress considerably faster, so this change will not affect recovery times.
2012-08-23 13:20:54 +02:00
Shay Banon 37b33cf77d protect a bit more from bogus (not relevant) exceptions when closing a node 2012-08-23 01:31:11 +02:00
Shay Banon 14c11a94fb MinimumNumberShouldMatch inconcistency, closes #2194
Streamline the use of minimum should match to all relevant queries to accept `minimum_should_match`, and allow all relevant queries to accept the advanced "string" based config
2012-08-22 14:15:08 +02:00
Shay Banon 177568a61f no need to optimize on the bool query parser, we already do that in the optimizeQuery method 2012-08-22 13:15:22 +02:00
uboness 3fdb9f0a27 Enabled the option of configuring plugin types in the settings. This will also help in tests when testing plugin related functionality 2012-08-21 23:00:24 +02:00
Shay Banon 2dd82675fb when using wildcard notation delete all indices might kick in even though its disabled 2012-08-21 21:49:16 +02:00
Shay Banon c33ca2921d small optimization for global facet with facet filter, by removing the double check on the facet filter 2012-08-21 19:32:55 +02:00
Shay Banon f03ff2976a reformat 2012-08-21 14:22:12 +02:00
Jörg Prante ef6ad6c966 mapping BigDecimal to double or string in XContentBuilder 2012-08-21 14:21:30 +02:00
Shay Banon b5d9d3dbee small renames here and there 2012-08-21 14:16:51 +02:00
Martijn van Groningen 8365e7ba0b - Added explain api. #2184 2012-08-21 13:26:17 +02:00
Shay Banon a8656c8a6f Incorrect application of boost in SpanNotQueryBuilder, closes #2192. 2012-08-21 13:08:23 +02:00
Shay Banon dbe2f53a00 Support YAML as content type 2012-08-20 17:40:19 +02:00
Shay Banon 9aae62b4a6 All Field: Automatically detect when field level boosting is used, and optimize when its not, closes #2189. 2012-08-20 15:07:32 +02:00
Shay Banon e1fe89389c change mappersAdded flag to mappingsModified to better reflect any possible change in mappings, not just additions 2012-08-20 13:36:28 +02:00
Shay Banon 79cb0eafc4 better close logic for concurrent recovery streams 2012-08-20 02:05:54 +02:00
Shay Banon 08ecd9d772 refactor all Queue and BlockingQueue creations into a single factory method 2012-08-19 18:15:48 +02:00
Shay Banon e3a9271000 unify more count and search implementation 2012-08-19 16:54:53 +02:00
Shay Banon bc2887344e convert to optimized match all query also when executing "*" query 2012-08-18 18:15:54 +02:00
Simon Willnauer b0b5775c98 use term query instead of a specialized SpanTermQuery on _all field if positions are omitted 2012-08-16 10:42:14 -07:00
Shay Banon a4acb9a95b move debug level on failure to clean log files from warn to debug 2012-08-16 10:34:31 -07:00
Shay Banon f317399b46 allow to flush bulk processor based on time, and better listener api 2012-08-16 09:14:19 -07:00
Shay Banon ab49a8c2fc improve update test to wait for green cluster state 2012-08-14 01:47:18 +02:00
Shay Banon 70f212a4dc more improvements for handling of large / many mappings by batching the removal of mappings 2012-08-14 00:18:11 +02:00
Shay Banon 5715588b96 fix geo shape to use the new index options 2012-08-13 17:30:38 +02:00
Simon Willnauer 53f65d8ff2 Remove / deprecated omit_term_freq_and_positions in favor of IndexOptions exposed via mapping API 2012-08-13 17:19:08 +02:00
Shay Banon eda3da2aea fix geo shape tests 2012-08-13 14:40:36 +02:00
Chris Male bea4346f3a Added GeoShape indexing and querying support 2012-08-13 13:44:29 +02:00
Shay Banon ad0e916fb7 don't allow an empty type in the mapper service 2012-08-13 12:42:11 +02:00
Shay Banon bdea0e2edd add a simple bulk processor to simplify using the bulk API 2012-08-13 11:24:48 +02:00
Martijn van Groningen b979dfa0be Add lenient option to match & multi_match queries. #2156 2012-08-09 21:56:50 +02:00
Shay Banon fedd1965ea Update API: Update through an alias with routing configured on it fail to use the routing, closes #2155. 2012-08-09 15:14:52 +02:00
Shay Banon a1ab48c804 handle null query returning from query parsers properly
now that we support "null" queries, for example, when parsing a query and its lenient, make sure to handle it where applicable
2012-08-09 13:50:10 +02:00
Martijn van Groningen e43dd4687e - Added support for multi match query. 2012-08-09 11:36:59 +02:00
Shay Banon 9d558580a0 simplify the parsing part of match query 2012-08-08 15:15:51 +02:00
Shay Banon 640c0c2933 move the fieldName and text ot the parse element so the query can be reused 2012-08-08 15:06:06 +02:00
Shay Banon 055f0809c2 Rename text query to match query (text query still works, with variants), closes #2150 2012-08-08 14:44:39 +02:00
Jacob Evans 3925c0b4e1 Allow custom type_table to be passed to Lucene's WordDelimiterFilter() constructor, instead of just the default 2012-08-08 00:18:24 +02:00
Martijn van Groningen 195e586fd8 - Fixed timezone parsing when input starts with '+'sign. Fixes issue #2141 2012-08-07 22:53:00 +02:00
Martijn van Groningen 37e7a54b0e Fixed top children query bug reported in issue #2140
Fixed type.
2012-08-06 22:02:03 +02:00
Shay Banon 346fb9ed83 create a unique recovery id when recovering from a peer shard instead of using shard id
this allows to handle better cases where we need to cancel an existing recovery
2012-08-06 17:09:00 +02:00
Shay Banon d2ac219f01 remove rouge sout... 2012-08-06 10:35:05 +02:00
Shay Banon ed2196f5a5 better percent calc for hot threads, including more details on top threads (time out of interval) 2012-08-04 22:20:47 +03:00
Shay Banon 3afeac5b12 better debug output when failing to load global state file 2012-08-03 20:28:33 +03:00
Shay Banon ad392bb11e Improve recovery time when processing large mappings, closes #2138. 2012-08-03 18:40:49 +03:00
Martijn van Groningen 0e3c825501 Added ignore_malformed mapping parameter for all number like types. Issue #2120 2012-08-03 10:41:07 +03:00
Martijn van Groningen 08ba05e713 Overwrite ignore_above parameter when mapping is updated. Issue #2121 2012-08-03 10:38:08 +03:00
Shay Banon de3fb50c3b expose snapshots parameter 2012-08-02 15:24:25 +03:00
Shay Banon 8be5c72200 Cluster Nodes hot_threads API, closes #2134. 2012-08-02 14:52:31 +03:00
Shay Banon e26a56e025 add to the keep alive thread name the version number, so it will be simpler to know from stack traces the es version 2012-08-02 10:50:28 +03:00
Shay Banon 1a33687101 better javadoc 2012-08-02 09:45:32 +03:00
Shay Banon 7a0d7f531d fix test 2012-08-02 09:40:54 +03:00
Shay Banon e88dbafe51 rename Test to Tests, so it will be executed as part of the mvn tests as well, reformat a bit 2012-08-01 16:20:37 +03:00
Simon Willnauer d13a7809d1 #2116 Expose all ShingleFilter settings via ShingleTokenFilterFactory 2012-08-01 16:18:58 +03:00
Shay Banon 0492d9b8cb fix test failure message... 2012-07-31 21:02:34 +02:00
Shay Banon 82cfe0e8b2 upgrade to latest testng, improve console output when running test, add more options as env vars when using maven 2012-07-31 20:24:39 +02:00
Shay Banon 8dcee09868 expose the max direct memory allowed in jvm info, and guess better then receive buffer size predictor size based on it 2012-07-31 15:06:21 +02:00
Shay Banon bbc45fefe5 rename limit to ignore_above, and create a dedicated test 2012-07-31 13:00:10 +02:00
Martijn van Groningen 41b3a454cf Issue #2121 Added limit parameter for string type. 2012-07-31 13:00:03 +02:00
Shay Banon 68c76e6738 try harder to load the default mapping from the class load, and if we fail, just use the default value (which people really never change). 2012-07-31 00:10:25 +02:00
Igor Motov 0a5a068269 Support files with no extension in config mappings 2012-07-30 22:28:48 +02:00
Shay Banon a63aa6cc18 Increase default recovery chunk size, closes #2125. 2012-07-30 21:48:49 +02:00
Shay Banon 4eb85bbbd6 Transport/Http: Remove explicit setting of send/receive buffer, and improve netty receive buffer predictor, closes #2124. 2012-07-30 21:37:38 +02:00
Shay Banon 408a74206f bytes reference to know how to convert to channel buffer, so it can be used directly when sending it over with netty 2012-07-29 16:50:43 +02:00
Shay Banon 7edafcf9a0 Node Stats: Add jvm buffer pools stats (when available, for java 7 and above), closes #2122. 2012-07-29 00:49:18 +02:00
Shay Banon 3338736c87 improve cached buffer allocation sizes based on jvm heap configured 2012-07-28 22:01:49 +02:00
Shay Banon a41477e0fa improve writing text as utf8 when having a string if we can write directly the utf8 bytes 2012-07-28 20:21:20 +02:00
Shay Banon 90e94ebab9 Upgrade to Lucene 3.6.1, closes #2110. 2012-07-22 13:28:39 -07:00
Drew Raines 2253d03709 Fix NodeStats comment. 2012-07-21 14:04:00 -07:00
Shay Banon 443f740795 *:* query string query should resolve to match_all query, since the wildcard field support, it no longer does 2012-07-21 13:55:50 -07:00
Shay Banon 153c0de636 make sure we use utf8 charset 2012-07-19 11:29:27 -07:00
Shay Banon cc2a6babda move to use murmur3 for bloom filter 2012-07-19 10:53:54 -07:00
Shay Banon bd219f3b34 better failure when no filter is provided to bool filter 2012-07-14 13:17:21 +02:00
Andrius Juozapaitis c3ccfe001a If all filters are null OR is empty... 2012-07-14 13:04:18 +02:00
Andrius Juozapaitis 83a61cae36 Avoid NPE in case must, should or not filters are omitted. 2012-07-14 13:04:18 +02:00
Shay Banon 7a23b4f0b6 add query types also to field query 2012-07-12 11:30:35 +02:00
Shay Banon 0cebcc56a4 move to use linked blocking queue 2012-07-12 11:28:55 +02:00
Igor Motov 51ff83942c Query Parser caching does not take types into account 2012-07-12 11:28:26 +02:00
Shay Banon 5517df6353 don't (re) create a string from a numeric timestamp, just return the string we get 2012-07-11 21:28:14 +02:00
Shay Banon 29d6b0e30a add debug logging when updating the shard indexing buffer 2012-07-11 18:06:28 +02:00
Shay Banon ee3253d558 OutOfMemory Error -> corrupted translog entry in shards, closes #2098. 2012-07-11 16:05:20 +02:00
Shay Banon de5068388e add rejected metric to thread pool stats 2012-07-11 10:59:00 +02:00
Shay Banon 83323f2c88 improve the logic of expanding byte arrays 2012-07-11 10:58:55 +02:00
Shay Banon 7ff4371f95 add more trace logging for recovery 2012-07-11 01:19:11 +02:00
Shay Banon 877f2ffcd7 propagate cluster event to shard and meta states in local gateway even when state persistence is disabled, so they can act on it if needed 2012-07-11 00:02:18 +02:00
Shay Banon 57e966e9d7 upgrade to jackson 2.0.4 2012-07-10 23:44:02 +02:00
Shay Banon 99d2f27c84 Introduce Text abstraction, allowing for improved representation of strings, apply to HighlightedField (breaks backward for Java API from String to Text), closes #2093.
By introducing the Text abstraction, we can keep (long) text fields in their UTF8 bytes format, and no need to convert them to a string when serializing it back to Json for example.

The first place we can apply this is to highlighted text, which can be long.. . This does breaks backward comp. for people using the Java API where the HighlightField now has a Text as its content, and not String.
2012-07-10 00:47:37 +02:00
Shay Banon 0bb7496dfe yuck!, hack around snappy printing to System.err the failure to load its library if its not found 2012-07-09 20:14:03 +02:00
Shay Banon 1a085d9bfa Stored binary fields to change internal representation (break Java API if used), closes #2092. 2012-07-09 16:53:20 +02:00
Nicolas Lalevée e91a0d85a7 Better ExplainableSearchScript interface 2012-07-09 16:34:54 +02:00
Shay Banon 1dd30bd388 deprecate readUTF, use readString instead (better named) 2012-07-08 21:21:17 +02:00
Shay Banon d3e9506a1d ad more info when rejecting execution 2012-07-08 12:06:57 +02:00
Shay Banon b27cc1318f streams to handle BytesReference as a generic type as well 2012-07-08 01:59:24 +02:00
Shay Banon 12beff9176 use netty frame decoder again
with the new bytes reference abstraction, and the new composite buffer handling in frame decoder, it makes more sense to move back and use netty frame decoder again, since we can easily slice refernces acorss composite buffers, and reduce copies when using compression
2012-07-08 00:56:56 +02:00
Shay Banon f7b538e17f move rest request to provide content as BytesReference
this allows for better buffer usage, specifically when forwarding requests to other nodes
2012-07-07 22:49:31 +02:00
Shay Banon 5f5458fd56 add byte buffer backed bytes reference 2012-07-07 21:33:01 +02:00
Shay Banon 35233564fd buffer management refactoring
First phase at improving buffer management and reducing even further buffer copies. Introduce a BytesReference abstraction, allowing to more easily slice and "read/write references" from streams. This is the foundation for later using it to create smarter buffers on top of composite netty channels for example (which http now produces) as well as reducing buffer copies when sending transport/rest responses.
2012-07-07 01:26:41 +02:00
Shay Banon 5d93b2bfe9 Thread Pool: Allow to configure the query_type for fixed thread pool (linked/array), defaults to array, closes #2086. 2012-07-06 11:48:45 +02:00
Shay Banon 8d1e04a973 have the quick rolling restart stress test also wait for 0 relocating shards 2012-07-06 01:01:18 +02:00
Shay Banon dcf9194820 fix naming, also make explainable search script extend search script 2012-07-06 01:00:44 +02:00
Nicolas Lalevée a77c824476 Add an interface to be implemented by native scripts to be able to explain the score 2012-07-06 00:55:12 +02:00
Shay Banon 29826863dd Local Gateway: old global state files are not properly deleted, closes #2085. 2012-07-06 00:19:34 +02:00
Shay Banon d844093953 only set the decoder type if set in the settings 2012-07-05 23:47:03 +02:00
Shay Banon 2328776ec5 allow to configure the decoder type for LZF, between optimal and safe 2012-07-05 23:46:02 +02:00
Shay Banon 0a615ca35a when needing to copy over the bytes from the netty content channel, zero it out so ti can be GC'ed 2012-07-05 23:36:27 +02:00
Shay Banon 5f1b1c6f69 Upgrade to Netty 3.5.2, closes #2084. 2012-07-05 23:09:26 +02:00
Shay Banon 57023c8ba9 Compression: Support snappy as a compression option, closes #2081. 2012-07-04 17:14:12 +02:00
Shay Banon 2436f645aa add 0.19.9 2012-07-02 22:10:28 +02:00
Shay Banon 1c3c737524 don't double compress already compressed files when doing recovery 2012-07-02 21:41:40 +02:00
Shay Banon 1668533556 improve dangling index support to not detect explicit deleted index as dangling, harden when we delete the _state of an index 2012-07-02 17:17:16 +02:00
Shay Banon b2c4876626 (full) recovery memory based indices with local gateway, don't fail them, closes #2077. 2012-07-02 00:31:10 +02:00
Shay Banon bdf146ded6 Exists API can send response body, closes #2072. 2012-07-02 00:04:49 +02:00
Shay Banon 3ab85bacdd don't delete shards state in the state handle, but let the indices store eventually delete it based on its logic of when to delete a shard 2012-07-01 21:51:19 +02:00
Shay Banon 6cd9b92f95 ElasticSearchIllegalStateException: stream marked as compressed, but no compressor found on transport layer, closes #2076. 2012-07-01 20:26:33 +02:00
Shay Banon 743d3f7e4b better failure message when failing to detect compressor on compressed transport stream 2012-07-01 18:34:58 +02:00
Shay Banon e5c89def42 Support wildcard and +/- notation for multi index APIs, closes #2074. 2012-07-01 18:16:04 +02:00
Shay Banon 22bdd161ad return proper content type for mp3 2012-06-30 18:26:21 +02:00
Shay Banon 565db26e13 Store Compression: integer overflow causes failed reads (index is safe), closes #2071. 2012-06-30 01:37:46 +02:00
Shay Banon 8bab859822 simplify tests, doc file length 2012-06-29 16:01:17 +02:00
Shay Banon f2e39e4ee2 Auto import dangling indices, closes #2067. 2012-06-29 01:01:26 +02:00
Shay Banon 07454243e3 getSourceAsString() for doc inserted as SMILE fails, auto convert to JSON, closes #2064. 2012-06-28 14:17:58 +02:00
Shay Banon a872c88f03 dangling index handling might still remove the state files for the dangling index, closes #2065. 2012-06-28 13:32:44 +02:00
Shay Banon 7454c7c192 freeing the search context should execute on the same thread, its cheap 2012-06-28 09:56:49 +02:00
Shay Banon 016e2e7288 Have rest request also allow to get the content as a streamable
(note, this will break plugins that implement a rest handler..., like wares, thrift, and memcached, until they are upgraded)
2012-06-28 02:00:47 +02:00
Shay Banon 1ffd68f2de Upgrade to netty 3.5.1 2012-06-28 00:51:37 +02:00
Shay Banon 0aa0b9ef22 cleanups 2012-06-27 21:50:36 +02:00
Matt Weber d6bc17fee5 Partial update without script
Allow the use of "doc" as the update source when a script is not
specified.  New fields are added, existing fields are overwritten, and
maps are merged recursively.
2012-06-27 21:29:22 +02:00
Igor Motov a4ad84b5e4 Enable validation of queries with has_child and script filters 2012-06-27 21:23:02 +02:00
Igor Motov dbeda1ab2b Add missing serialization for error and explanation in validate query request 2012-06-27 21:23:01 +02:00
Shay Banon 644dcb99cf Setting index to no ends up using by default the "keyword" analyzer on it, which means one can't highlight on it when searching on _all, closes #2062. 2012-06-27 21:16:03 +02:00
Petri Lehtinen 834c79597e deb: Allow configuring max open files and max locked memory limits 2012-06-27 13:09:46 +02:00
Shay Banon d2bf87c89c print the memory pools on GC logging 2012-06-27 12:05:53 +02:00
Shay Banon acc9783386 add 0.19.8 2012-06-26 19:59:40 +02:00
Shay Banon 2b893fe1e5 Use bloom filter when flushing (applying deletes), closes #2058. 2012-06-26 16:45:29 +02:00
Shay Banon 2eee92726c support also queue for setting queue size in fixed thread pool 2012-06-26 13:02:43 +02:00
Igor Motov 7f77cfada0 Add support for zero queue size in the search thread pool 2012-06-26 13:01:18 +02:00
Shay Banon eb954a6157 better failure message when failing to load _uid field 2012-06-26 02:40:33 +02:00
Shay Banon 12a644c89b Stored Compression: failure to fetch document in certain cases (read failure, index compression works), closes #2055. 2012-06-26 01:54:19 +02:00
Shay Banon 6c74f99ed3 better generics in plugin module instance providers 2012-06-25 22:50:15 +02:00
Shay Banon c39a27a328 "index_analyzer" and "search_analyzer" should override "analyzer", closes #2052. 2012-06-25 22:05:34 +02:00
Shay Banon 11ca3ea0ec deb package fails to start elasticsearch
The ability to use es.default prefix broke the deb package that uses it now.
2012-06-25 21:59:12 +02:00
Shay Banon 5533fe2b88 don't double set settings with es.default. or elasticseearch.default.
though harmless, since we don't have any "default" component, and effectively has not effect on the settings used, it still puts settings that we don't really need
2012-06-25 21:31:18 +02:00
Shay Banon 2e610ea653 add 0.19.7 2012-06-25 17:34:31 +02:00
Shay Banon d01048c93d return 404 when trying to delete a mapping that does not exists 2012-06-25 17:25:19 +02:00
Shay Banon c0f9e337ce elasticsearch fails to start due to verification error, closes #2051. 2012-06-25 17:00:26 +02:00
Shay Banon e523c71a88 add 0.19.6 2012-06-25 15:37:35 +02:00
Shay Banon 376779df18 wait for remove mapping to happen on the cluster before returning control back 2012-06-25 14:30:18 +02:00
Shay Banon 6e7764a083 reduce objects created with bloom filter operations 2012-06-24 20:58:44 +02:00
Shay Banon 2fb867b467 Store Compression: Term Vector Vector, closes #2049. 2012-06-23 23:11:00 +02:00
Shay Banon ded5b773da River name re-use not possible between node shutdowns, closes #1921.
This effectively happens because we don't flush before deleting a mapping, which causes it to still reply the changes happening for it on the transaction log, meaning the mapping are still around on the index level.
2012-06-23 20:26:27 +02:00
Shay Banon 6fb836c25e better thread naming 2012-06-23 18:35:42 +02:00
Shay Banon 3163499aef Return 503 when threadpool limit is reached, closes #2048. 2012-06-23 17:26:50 +02:00
Shay Banon efe85f322a better type based inference when using scripts 2012-06-23 17:25:02 +02:00
Shay Banon df62e39d20 change index.store.compress.stored_fields setting to index.store.compress.stored 2012-06-22 20:08:23 +02:00
Shay Banon ae77013004 Rest API: Add `HEAD` support for `/{index}/{type}/{id}` to quickly check if doc exists or not, closes #2047. 2012-06-22 19:32:38 +02:00
Shay Banon ed71662275 add debug log if using compressed stored fields 2012-06-22 18:23:15 +02:00
Shay Banon 1780a2a067 Failure to recover properly on node(s) restart
When a node restarts, it might be canceling one recovery of a shard id only to get another one in the next cycle. We should detect this case and handle it properly.

This is a fix to the annoying message seen by users: suspect illegal state: trying to move shard from primary mode to replica mode.
2012-06-22 17:46:57 +02:00
Shay Banon cc3fab45ff Improve cluster resiliency to disconnected sub clusters + fix a shard allocation bug with quick rolling restarts
Two main changes:

Improve cluster resiliency to disconnected sub clusters. If a node pings a master and that node is no longer registered with the master, improve the rejoin process of that node to the cluster. Also, if a master receives a message from another master, pick one to force to rejoin the cluster (based on cluster state versioning).
On quick rolling restart, without waiting for shard allocation, the shard allocation logic can mess up its counts, causing for strange logic in allocating shards, or validation failures on routing table allocation.
2012-06-22 03:36:54 +02:00
Shay Banon 90371beedc Store Throttling (node level and/or index level) with options on merge or all, closes #2041.
Allow to configure store throttling (only applied on file system based storage), which allows to control the maximum bytes per sec written to the file system. It can be configured to only apply while merging, or on all output operations. The setting can eb set on the node level (in which case the throttling is done across all shards allocated on the node), or index level, in which case it only applied to that index.

The node level settings are indices.store.throttle.type to set the type, with values of none, merge and all (defaults to none). And, also, indices.store.throttle.max_bytes_per_sec (defaults to 0), which can be set to something like 1mb.

The index level settings is index.store.throttle.type for the type, with values of node, none, merge, and all. Defaults to node which will use the "shared" throttling on the node level. And, index.store.throttle.max_bytes_per_sec (defaults to 0).
2012-06-21 21:21:49 +02:00
Shay Banon 9e6cfa77a5 better compressed input offset data structure 2012-06-20 16:01:09 +02:00
Shay Banon b009c9c652 Stored Fields Compression, closes #2037.
Compressing the stored fields file (the .fdt file) directly allows to have better compression on the size of the index, specifically when indexing (and storing) small documents. The compression will be considerably more effective compared to compressing each doc on its own (when setting compress on the _source mapper). The downside is that more data needs to be uncompressed when loading documents.

The settings to control it is `index.store.compress.stored_fields` set to `true` (it defaults to `false`), and can be enabled dynamically using the update settings API. This allows to enabled compression at a later stage (i.e. old time based indices), and then optimize the index to make sure it gets compressed.
2012-06-20 05:31:34 +02:00
Shay Banon fbf4c70af9 add simple compression bench 2012-06-19 13:15:44 +02:00
Martijn van Groningen d66f401ce6 Better fix for mv field highlighting issue #1994 2012-06-19 04:13:47 +02:00
Shay Banon aebd27afbd abstract compression
abstract the LZF compression into a compress package allowing for different implementation in the future
2012-06-19 04:07:11 +02:00
Shay Banon 1a98a9184e fix test to shutdown threadpool 2012-06-19 03:37:08 +02:00
Shay Banon 7b3b130a62 fix tests to shutdown threadpool 2012-06-19 03:33:44 +02:00
Shay Banon 0a459f7ceb using node settings to get the node name for thread names 2012-06-19 03:03:11 +02:00
Shay Banon 0f47518752 add getTerms for uid filter 2012-06-18 15:04:49 +02:00
Shay Banon 0c7690d105 disable lastGc
disable  using lastGc, it gives strange results...
2012-06-16 15:40:44 +02:00
Chris Male 040fa2581a Added GeoDistance test which verifies the difference in behaviour between ARC and PLANE, causing elliptical results 2012-06-15 22:55:45 +02:00
Shay Banon 14fc3910ff better log message 2012-06-14 22:41:11 +02:00
Shay Banon 2280915d3c upgrade to joda 2.1
with the hack of duplicating BaseDateTime to remove the volatile
2012-06-14 21:57:01 +02:00
Shay Banon 982c8b4664 fix test to work with new noramalization 2012-06-14 15:55:33 +02:00
Chris Male 2315e6d239 Incorporated changes to normalization of latitude and longitudes so latitude normalization is correct and longitude is normalized at the same time 2012-06-14 15:43:36 +02:00
Shay Banon 133bd72f8d Multi Search API: Allow to set search_type on REST endpoint URI to apply to all search requests, closes #2023. 2012-06-13 20:47:24 +02:00
Shay Banon cf1fd26346 fix imports 2012-06-13 16:24:06 +02:00
Ozgur Orhan b80eee305e Enum -> toString -> toLowerCase fails for TR locale (for opType especially) 2012-06-13 16:20:55 +02:00
Shay Banon cecb762ede use an array to represent the keys in the uid filter
fix equals and hashcode as well
2012-06-13 16:08:03 +02:00
Shay Banon dfe6e58e37 use an array to represent the keys in the uid filter 2012-06-13 16:03:45 +02:00
Shay Banon 6eb419649a better/faster parsing of update request (with upsert) 2012-06-13 13:12:37 +02:00
Shay Banon 0b4fe4add3 rename doc to upsert in update API
a better descriptive name for it, and won't clash with future features on the update api
2012-06-13 12:42:10 +02:00
Benjamin Devèze e4b11e0b15 fix bug index.ttl.disable_purge should be in IndexMetaData 2012-06-12 23:51:32 +02:00
Shay Banon feb81dd365 improve filter cache cached value to store docset only 2012-06-12 23:26:15 +02:00
Shay Banon 4f59e5d283 combine cacheable method of bit sets into one method 2012-06-12 19:49:59 +02:00
Shay Banon 83df0587ad move remaining CHM construction to the factory method 2012-06-12 17:26:54 +02:00
Shay Banon ab35e7ce7a move remaining CHM construction to the factory method 2012-06-12 17:26:36 +02:00
Shay Banon 95fc7a39a3 move remaining CHM construction to the factory method 2012-06-12 17:23:01 +02:00
Shay Banon de3c74ab96 add a flag to restore old multi value behavior with fast vector highlighter 2012-06-12 00:19:56 +02:00
Martijn van Groningen 1319ed9322 Fixes highlight issue for multivalues fields described in issue #1994 2012-06-11 23:44:45 +02:00
Shay Banon 9798d3e8fb Discovery: Join process to better validate join request, closes #2019.
The join proces received on the master from a node joining the cluster now tries to validate by connecting to the relevant node. It should also send a message to it to double check that all is in order.
2012-06-11 23:44:27 +02:00
Shay Banon 53be815e22 Init script in Debian package ignores some options in elasticsearch.yml, closes #1765. 2012-06-11 13:49:59 +02:00
Shay Banon 8072af9078 Allow to pass es.default. settings to the process, using it as default value unless specified in the config file, closes #2016. 2012-06-11 13:30:55 +02:00
Aurelien Requiem 6d867126c9 lintian rule : missing-dependency-on-libc on lib/sigar/libsigar-amd64-linux.so
* added "linc6" as a package dependency as it's required for file
	  lib/sigar/libsigar-amd64-linux.so and lib/sigar/libsigar-x86-linux.so
	  Refer to Debian Policy Manual section 8.6 (Dependencies between the
	  library and other packages - the shlibs system) for details.
2012-06-11 12:29:09 +02:00
Aurelien Requiem 6d477aee0d lintian rule : possible-unindented-list-in-extended-description in src/deb/control/control
* replaced asterisk (*) in the package description to match lintian
	  rule. Refer to Debian Policy Manual section 5.6.13 (Description)
	  for details
2012-06-11 12:29:09 +02:00
Aurelien Requiem 8ad6e7a372 lintian rule : maintainer-script-needs-depends-on-adduser postinst
* added dependency in control for "adduser" package as the command is
	  used in the postinst script
2012-06-11 12:29:08 +02:00
Aurelien Requiem 17a28b0fe0 lintian rule : extended-description-line-too-long in src/deb/control/control
* reduced description lines to 80 chars max to comply with lintian
2012-06-11 12:29:08 +02:00
Shay Banon fd3c4d14af use djb hash to choose doc lock, and lock pool 2012-06-11 02:02:01 +02:00
Shay Banon e53db1a478 Quoted query_string gives NullPointerException with not_analyzed field (0.19.4), closes #2006. 2012-06-11 00:24:17 +02:00
Shay Banon 8f0bc799c6 Upgrade to latest jst166y and jsr166e
Embed the code now in our source, since jsr166e jar generation with 1.6 instead of 1.7 is complicated when doing it on its own as it relies on ThreadLocalRandom, and we have it in jsr166y
2012-06-10 00:42:54 +02:00
Shay Banon 726ec8b9ff Allow to filter client and "just" data nodes from controlling elected master (default client to true), closes #2014. 2012-06-09 01:28:02 +02:00
Shay Banon 395a09db03 convert to json wasn't taking reformat flag properly into account 2012-06-08 13:54:07 +02:00
Nicolas Lalevée 4df978e479 Fix an incorrect error message when query parse fail, closes #1996 2012-06-08 13:46:05 +02:00