Commit Graph

999 Commits

Author SHA1 Message Date
kimchy ee33ee457a allow to explicitly disable compression in peer recovery (defaults to true) 2010-08-15 15:13:45 +03:00
kimchy f6c58ff35d change default calibrate_size_by_deletes for merge policy to true from false (this is what upcoming lucene version will use, and makes much more sense) 2010-08-15 09:01:09 +03:00
kimchy 6d509a5e4e explicitly clean stream handles 2010-08-15 03:08:01 +03:00
kimchy c18904eb96 Recovery: Using transport compression explicitly when recovering an index from a peer shard, closes #322. 2010-08-15 03:01:35 +03:00
kimchy 1ee2f80e68 Transport: add global compression support compressing all internal transport communication (using lzf), closes #321. 2010-08-15 02:57:22 +03:00
kimchy c10544479f refactor cached streams into a separate class 2010-08-14 13:47:34 +03:00
kimchy 01b3d42cee separet recovery tests to one with small buffer size and on with bigger one 2010-08-14 13:44:42 +03:00
kimchy e1226626bc improve thread cached bytes allocation 2010-08-14 02:26:12 +03:00
kimchy 7c5f0fe405 minor transport refactoring, add options to send response, though no options to set yet 2010-08-14 02:25:58 +03:00
kimchy b3afca2589 remove compressor package, move lzf to a different package 2010-08-14 02:25:30 +03:00
kimchy 0adb2edba2 clean code 2010-08-14 01:23:04 +03:00
kimchy 75ee01a6b4 clean builder code when dealing with maps 2010-08-14 00:37:33 +03:00
kimchy 4a9c7d672e clean code 2010-08-13 23:50:16 +03:00
kimchy d12c757824 Gateway: Chunk based storage broken, fails to recover from gateway, closes #318. 2010-08-13 14:03:52 +03:00
kimchy 12ef12f7aa better handling of closing a shard / index while retrying recovery 2010-08-12 21:44:27 +03:00
kimchy 2bd9a63467 Query DSL: custom score script, allow to use `_score` as well as `score` as the underlying query score, closes #316. 2010-08-12 18:41:29 +03:00
kimchy 98bc8285ea Analysis: Add `char_filter` on top of `tokenizer`, `filter`, and `analyzer`. Add an `html_strip` char filter, closes #315. 2010-08-12 18:16:30 +03:00
kimchy e29925684a Term Facets: Add `reverse_cout` and `reverse_term` to `order` options, closes #314. 2010-08-12 15:26:26 +03:00
kimchy b1d1f1ff94 improve peer recovery shard state handling. when throttling a recovery, don't restore the non recovering state and then move to recovery again with each retry, keep the recovering state while retrying 2010-08-12 12:31:27 +03:00
kimchy c0a7dc327c range facet: from is used in to_str, should be to 2010-08-11 23:40:42 +03:00
kimchy a6d84f5942 no need to spawn a thread when response to a ping response 2010-08-11 13:20:13 +03:00
kimchy fbea92e688 REST API hang on a bogus call, closes #297. 2010-08-11 13:18:33 +03:00
kimchy ee26d55296 Remove memory monitor and move translog operations threshold to shard level setting, closes #312. 2010-08-11 12:54:00 +03:00
kimchy df4ece8cef remove no longer required commented code ;) 2010-08-11 11:43:52 +03:00
kimchy 9f8e5cb2c0 add a simple integration level test for scroll 2010-08-10 20:48:05 +03:00
kimchy 7977edd5db a filter based on the type of the document is built in several places in the code, which is wasteful. Instead, create a filter based on the type on the document mapper level, and reuse it where applicable. 2010-08-10 18:34:49 +03:00
kimchy 5f986ef422 build a match all filter internal implementation and use it where appropiate (instead of using query filter wrapper around a match all query) 2010-08-10 18:18:51 +03:00
kimchy f2018e2f86 Facets: Automatically filter by type for facets that use explicit field names that are prefixed by the type, closes #310. 2010-08-10 17:03:14 +03:00
kimchy 61bb9d0ff9 Mappers: also use `name` as a smart lookup for field/property on top of indexName and fullName, closes #309. 2010-08-10 16:14:15 +03:00
kimchy 879191c435 don't add the same object twice under rare cases where multiple threads are introducing the same new dynamic object 2010-08-10 09:18:53 +03:00
kimchy 4bb1baa9d4 remove nb structures 2010-08-10 00:05:38 +03:00
kimchy 05d07b036a remove ning async http client, no need for it 2010-08-10 00:00:24 +03:00
kimchy cacb5a54f1 upgrade to joda 1.6.1 2010-08-09 21:12:44 +03:00
kimchy f3cc8d1249 when serializing mappers into json, also serialize index, type and id mappings (index for the enabled flag, and id/type for the store) 2010-08-09 18:25:45 +03:00
kimchy f10699d5b8 javadoc 2010-08-09 09:06:14 +03:00
kimchy 6611e7db44 refactor custom sort parsing to be "code" pluggable 2010-08-09 08:57:56 +03:00
kimchy ff8d743337 add geo distance sort option, refactor java level sort API to have sort builders (similar to facets) 2010-08-08 21:46:11 +03:00
kimchy 8b19d353c1 add sort values as part of the response per search hit 2010-08-08 18:21:34 +03:00
kimchy 37af7f5aef add sort values as part of the response per search hit 2010-08-08 18:21:17 +03:00
kimchy 7cb2010dda add script support for terms facets, allowing to control if a certain term will be included in the facets result (boolean result), or muging of the term 2010-08-08 10:37:14 +03:00
kimchy ee1d50f8d8 add script support for terms facets, allowing to control if a certain term will be included in the facets result (boolean result), or muging of the term 2010-08-08 10:27:06 +03:00
kimchy 566ae94478 add the ability to control the "order" by of terms facet, can either be `term` or `count`. defaults to `count`. 2010-08-08 09:31:40 +03:00
kimchy d4547c629f New API, update settings, allowing to change specific runtime settings of the cluster (indices). `number_of_replicas` is the only settings supported, allowing to change the number of repliacs of one or more indices. 2010-08-08 09:17:09 +03:00
kimchy 6d3d9fd807 start work on being able to change the number of replicas at runtime 2010-08-05 11:00:02 +03:00
kimchy 860d8058b6 rename jmx settings to conform with casing (old casing also works) 2010-08-04 22:27:26 +03:00
kimchy 8c5fdf4aee Ability to return all stored fields with a *, closes #296. 2010-08-04 13:22:55 +03:00
kimchy d72de60b6f Add the ability to store the index name in the doc, closes #292. 2010-08-04 11:41:56 +03:00
kimchy a44d30bb61 After gateway recovery, mappings keep being applied on each cluster change, closes #295. 2010-08-04 09:02:10 +03:00
kimchy 959eb0e703 Geo: Polygon based filter, closes #294. 2010-08-03 21:52:13 +03:00
kimchy 39f344c11e change the format of bounding box filter to conform more to the "field_name" : { ... } structure 2010-08-03 19:18:16 +03:00
kimchy 91c256361e support for custom script based sorting 2010-08-03 18:14:02 +03:00
kimchy 3d31c38f11 refactoring transport service to allow for send options 2010-08-03 10:06:30 +03:00
kimchy 04781e038a when timeouting, and closed, raise onClose and not onTimeout 2010-08-03 10:05:19 +03:00
kimchy de8ffaf5fb Geo: bounding box filter, closes #290. 2010-08-03 01:51:46 +03:00
kimchy d1e404c1b3 change default geo point resolution from 32 to 64 2010-08-03 00:08:56 +03:00
kimchy 07ab5dcf9b reuse constants for facet type 2010-08-02 22:27:03 +03:00
kimchy dc6ef326d9 Facets: Filter based facet, closes #289. 2010-08-02 19:17:57 +03:00
kimchy 4e6087ee2d improve concurrency when loading field data 2010-08-02 18:20:49 +03:00
kimchy 7719dcd790 add an option to provide the array of ranges "on" the field name itself 2010-08-02 18:09:26 +03:00
kimchy 99af6df0ea add an option to provide the array of ranges "on" the field name itself 2010-08-02 16:39:28 +03:00
kimchy 6ed89ad406 fix comment 2010-08-02 16:35:08 +03:00
kimchy 32e26a6bd0 fix valueField name 2010-08-02 16:32:52 +03:00
kimchy 76cfbe6a44 support dates in range facet 2010-08-02 10:35:21 +03:00
kimchy 4bbbf186b6 fix some javadocs 2010-08-02 09:53:08 +03:00
kimchy 38fe6860d8 update docs 2010-08-02 09:39:17 +03:00
kimchy fdb0c14a17 upgrade to guava r06 2010-08-01 23:01:33 +03:00
kimchy 423ee159a5 upgrade to jackson 1.5.5 2010-08-01 22:51:15 +03:00
kimchy ef861a6b7b Search Facets: Range Facet, closes #287. 2010-08-01 20:08:27 +03:00
kimchy 9f441d8b9f add filter and global support 2010-08-01 17:09:40 +03:00
kimchy e77d1d0147 add filter and global support 2010-08-01 17:08:44 +03:00
kimchy c5811eb060 add a check for at least one range 2010-08-01 17:05:06 +03:00
kimchy c08791f816 fix type on xcontent result 2010-08-01 16:34:00 +03:00
kimchy 168363106b fix doc 2010-08-01 16:31:54 +03:00
kimchy b8b21a3363 Geo: `geo_distance` facet, closes #286. 2010-08-01 13:31:03 +03:00
kimchy dfb68c6310 Search fails when sorting on a field that has no values in the hits returned, closes #285. 2010-07-31 16:41:15 +03:00
kimchy a26b4f31e1 print the exception when failing to add a specific mapping during cluster state update 2010-07-31 15:41:11 +03:00
kimchy 484665af80 handle cases when there is no value 2010-07-31 02:30:52 +03:00
kimchy 9af288a42f fix test for existance of directory based mappings when creating an index 2010-07-29 19:22:21 +03:00
kimchy 0186a03684 add another script function test 2010-07-29 16:07:57 +03:00
kimchy 4f407e18aa Put Mapping: When using a single node and updating a mapping, it is not marked as `acknowledged`, closes #280. 2010-07-29 15:21:11 +03:00
kimchy e955e41a91 improve handling of failure when trying to read md5, should be ignored on any exception, not just io exception 2010-07-29 15:10:37 +03:00
kimchy 194e6cbff6 Geo support: geo_point type and geo_distance filter, closes #278, #279 2010-07-28 21:06:18 +03:00
kimchy af055f75ce should be static 2010-07-26 13:46:01 +03:00
kimchy f251e2fd32 support lt(N), le(N), gt(N), ge(N) in wait for nodes (thanks Lukas) 2010-07-26 13:44:29 +03:00
kimchy 65284ba2ba Terms Factes: Allow to provide regex controlling which terms should be included, closes #277. 2010-07-26 12:23:03 +03:00
kimchy 03cd2f3772 fix <= 2010-07-26 00:22:21 +03:00
kimchy ac7c8cb650 Analysis: Add pattern analyzer, closes #276. 2010-07-25 22:40:50 +03:00
kimchy 477a24efc6 Mapping: Revise dynamic mapping (into default), merge default to new mappings, closes #275. 2010-07-25 21:31:16 +03:00
kimchy 1884c4219a Mapping: Dynamic mapping definitions are ignored, closes #274. 2010-07-25 12:03:04 +03:00
kimchy 457b56937e small improvements to groovy client, allows for closure in the search request 2010-07-25 10:32:22 +03:00
kimchy 77b6d1d8b8 remove cloud plugin, reimplement an AWS specific cloud plugin with S3 gateway and EC2 discovery 2010-07-24 23:39:43 +03:00
kimchy a6a993ff08 fix ignored exception 2010-07-24 18:48:28 +03:00
kimchy 0072dd816b nicer logging, and use sync streams 2010-07-23 23:10:59 +03:00
kimchy 23ddf8bc53 do a reroute when a node becomes master 2010-07-23 20:55:26 +03:00
kimchy f6255f8ad9 change logging level to debug 2010-07-23 19:29:14 +03:00
kimchy 72682a6730 always retry on connect exception with master operations 2010-07-23 01:59:56 +03:00
kimchy 9283e2a7ad local discovery should update the local metadata state once it started 2010-07-23 01:39:37 +03:00
kimchy 5706e5f6b9 rename node close to node closed exception 2010-07-23 01:12:36 +03:00
kimchy 34b84b0e61 when retrying on primary shard when node disconnected, add it again the the cluster service as a listener 2010-07-23 01:07:09 +03:00
kimchy 8875b9641b cleanup test 2010-07-23 00:50:08 +03:00
kimchy 2a3130c649 REST Search API: Change `score` to `_score` to denote sorting by hit score, closes #271. 2010-07-22 23:39:55 +03:00
kimchy 42c2b6b743 throw a proper exception when no type is found 2010-07-22 21:17:11 +03:00
kimchy c1ab9f290f remove compile warnings 2010-07-22 00:25:25 +03:00
kimchy b55fd0c0fc verify that data that is written to the blob (in fs) is the same as expected in the blob write api call 2010-07-21 23:46:11 +03:00
kimchy 4dcf219086 after raising a failure, return, don't try and write the blob 2010-07-21 23:45:15 +03:00
kimchy 2432a7cb5b fixed highlighted styled built in schema tags to properly increate from hlt1 to hlt10 2010-07-21 20:38:13 +03:00
kimchy 8ec7ee66a9 Search: Sending a request that fails to parse can cause file leaks, closes #270. 2010-07-21 16:59:58 +03:00
kimchy 0cb97e4044 cancel the timeout in any case 2010-07-21 16:38:37 +03:00
kimchy 8d45901a8a change logging level, nodes FD will detect it as well 2010-07-21 16:30:40 +03:00
kimchy 48d33ec70a Cluster Health API: Add `wait_for_nodes` (accepts "N", "<N", ">N", "<=N", and ">=N"), closes #269. 2010-07-21 16:29:44 +03:00
kimchy 2a0e8d4ec9 better state when closing shard, and handling its state 2010-07-21 09:20:28 +03:00
kimchy 3629540953 when trying to rejoin the cluster, clean the routing/metadata so the node will get fresh shards 2010-07-21 08:55:35 +03:00
kimchy 15cf7a2361 initialize the nodes before a possible retry, so we have the local node id for it 2010-07-21 01:53:00 +03:00
kimchy 39e546b64d add a proper master not discovered exception for master based operations (cluster state) and the ability to control the timeout till master is found 2010-07-21 01:51:58 +03:00
kimchy a2ceaa91cc improve retry when closing the node performing an operation 2010-07-20 22:34:09 +03:00
kimchy 1ff8b43aa7 clean timeout listener from on going listeners once its removed 2010-07-20 21:11:56 +03:00
kimchy f4064038e4 improve load test 2010-07-20 20:53:14 +03:00
kimchy b3b5c9b0cc don't execute it on a thread pool 2010-07-20 18:34:01 +03:00
kimchy 553e8134ee create a send transaction exception on the calling thread and fill its stack, so we know where its coming from 2010-07-20 18:33:44 +03:00
kimchy 07480846c3 fix logging to actually pass the actual logging parameters 2010-07-20 18:33:12 +03:00
kimchy 6915b39dc3 clean timeout listener task when removing the listener 2010-07-20 18:32:49 +03:00
kimchy 45e54c1705 add retry if master is not connected till master is connected or a timeout has passed to master based operations (like cluster health) 2010-07-20 18:32:21 +03:00
kimchy 45c821316b no need for volatile on writer, its always accessed under a lock 2010-07-20 13:58:01 +03:00
kimchy 3c8cf68a17 improve timeout cluster service listener 2010-07-20 13:50:19 +03:00
kimchy 4ac1409afa don't list connect exceptions 2010-07-20 11:01:08 +03:00
kimchy 679f3758b7 ignore a more genric exception 2010-07-20 09:29:10 +03:00
kimchy 3867591ffa clean up the java api a bit 2010-07-20 09:18:16 +03:00
kimchy 2ee870e01e nicer logging, append space before index + shard id logging 2010-07-20 08:53:57 +03:00
kimchy 6f2c0f4e55 spelling 2010-07-19 17:51:42 +03:00
kimchy 5f0470d68b Support Cross-Origin resource in http/rest module, closes #218. 2010-07-19 15:06:13 +03:00
kimchy e48b1d98db more recovery under load tests, now with nodes shutdown, ignore node connected exception or retry when performing replicated operations 2010-07-19 11:33:21 +03:00
kimchy 9def68a733 add full rolling restart recvoery test 2010-07-19 10:08:12 +03:00
kimchy df602054fb fix relocation of primaries and perform the operation on the relocated primary replica as well 2010-07-19 09:54:52 +03:00
kimchy 0d20790ffe when shutting down a node, close shard (and indices) in parallel 2010-07-19 02:17:59 +03:00
kimchy d657d4447b more shard relocation tests, create a cached executor that can be used to allocate threads that are not affected by the actual thread pool impl (can always fork, no blocking) 2010-07-19 01:45:09 +03:00
kimchy f9cd7cb932 add a recovery test while under indexing load 2010-07-18 23:57:57 +03:00
kimchy 9c9dfe0b39 add a recovery test while under indexing load 2010-07-18 23:40:13 +03:00
kimchy fefcfb5b5c refactor recovery to be handled on the node level (and not per shard), with better retry mechanism when doing peer shard recovery 2010-07-18 22:54:21 +03:00
kimchy 8ac8dd818d now that we have a file based transaction log, increate the memory clean threshold 2010-07-18 21:24:52 +03:00
kimchy 49f8501a5b nicer serilization 2010-07-17 22:58:16 +03:00
kimchy 910ad2a408 fix optimization to not do sorting when searching with score "desc" sorting 2010-07-17 19:38:04 +03:00
kimchy 08ca383fd5 more work on cloud gateway 2010-07-17 15:20:13 +03:00
kimchy 6194b3ab0f add NaN handling for specific hit score as well 2010-07-17 13:52:31 +03:00
kimchy 1fccaf06e9 start work on reimplementing cloud gateway 2010-07-17 13:12:53 +03:00
kimchy e61dc78c21 max_score == NaN, closes #263. 2010-07-17 01:39:12 +03:00
kimchy 3006183191 fix a problem where max_score was NaN if there were no hits from a shard (though there are hits from other shards), it will still be NaN where there are no hits 2010-07-15 22:58:18 +03:00
kimchy cf998e2f89 protected when not of the same type mapper when merging 2010-07-15 22:35:18 +03:00
kimchy 9f6fde9adb better handling of failed merge because of different types 2010-07-15 20:40:40 +03:00
kimchy cfa56cb3d6 Put Mapping: When updating existing mappings, the request returns with acknowledged `false`, closes #262. 2010-07-15 20:03:44 +03:00
kimchy c987e8a534 Querying mapping on a non-master throws an error, closes #261. 2010-07-15 19:03:45 +03:00