Commit Graph

1006 Commits

Author SHA1 Message Date
kimchy 6d3d9fd807 start work on being able to change the number of replicas at runtime 2010-08-05 11:00:02 +03:00
kimchy 860d8058b6 rename jmx settings to conform with casing (old casing also works) 2010-08-04 22:27:26 +03:00
kimchy 8c5fdf4aee Ability to return all stored fields with a *, closes #296. 2010-08-04 13:22:55 +03:00
kimchy d72de60b6f Add the ability to store the index name in the doc, closes #292. 2010-08-04 11:41:56 +03:00
kimchy a44d30bb61 After gateway recovery, mappings keep being applied on each cluster change, closes #295. 2010-08-04 09:02:10 +03:00
kimchy 959eb0e703 Geo: Polygon based filter, closes #294. 2010-08-03 21:52:13 +03:00
kimchy 39f344c11e change the format of bounding box filter to conform more to the "field_name" : { ... } structure 2010-08-03 19:18:16 +03:00
kimchy 91c256361e support for custom script based sorting 2010-08-03 18:14:02 +03:00
kimchy 3d31c38f11 refactoring transport service to allow for send options 2010-08-03 10:06:30 +03:00
kimchy 04781e038a when timeouting, and closed, raise onClose and not onTimeout 2010-08-03 10:05:19 +03:00
kimchy de8ffaf5fb Geo: bounding box filter, closes #290. 2010-08-03 01:51:46 +03:00
kimchy d1e404c1b3 change default geo point resolution from 32 to 64 2010-08-03 00:08:56 +03:00
kimchy 07ab5dcf9b reuse constants for facet type 2010-08-02 22:27:03 +03:00
kimchy dc6ef326d9 Facets: Filter based facet, closes #289. 2010-08-02 19:17:57 +03:00
kimchy 4e6087ee2d improve concurrency when loading field data 2010-08-02 18:20:49 +03:00
kimchy 7719dcd790 add an option to provide the array of ranges "on" the field name itself 2010-08-02 18:09:26 +03:00
kimchy 99af6df0ea add an option to provide the array of ranges "on" the field name itself 2010-08-02 16:39:28 +03:00
kimchy 6ed89ad406 fix comment 2010-08-02 16:35:08 +03:00
kimchy 32e26a6bd0 fix valueField name 2010-08-02 16:32:52 +03:00
kimchy 76cfbe6a44 support dates in range facet 2010-08-02 10:35:21 +03:00
kimchy 4bbbf186b6 fix some javadocs 2010-08-02 09:53:08 +03:00
kimchy 38fe6860d8 update docs 2010-08-02 09:39:17 +03:00
kimchy fdb0c14a17 upgrade to guava r06 2010-08-01 23:01:33 +03:00
kimchy 423ee159a5 upgrade to jackson 1.5.5 2010-08-01 22:51:15 +03:00
kimchy ef861a6b7b Search Facets: Range Facet, closes #287. 2010-08-01 20:08:27 +03:00
kimchy 9f441d8b9f add filter and global support 2010-08-01 17:09:40 +03:00
kimchy e77d1d0147 add filter and global support 2010-08-01 17:08:44 +03:00
kimchy c5811eb060 add a check for at least one range 2010-08-01 17:05:06 +03:00
kimchy c08791f816 fix type on xcontent result 2010-08-01 16:34:00 +03:00
kimchy 168363106b fix doc 2010-08-01 16:31:54 +03:00
kimchy b8b21a3363 Geo: `geo_distance` facet, closes #286. 2010-08-01 13:31:03 +03:00
kimchy dfb68c6310 Search fails when sorting on a field that has no values in the hits returned, closes #285. 2010-07-31 16:41:15 +03:00
kimchy a26b4f31e1 print the exception when failing to add a specific mapping during cluster state update 2010-07-31 15:41:11 +03:00
kimchy 484665af80 handle cases when there is no value 2010-07-31 02:30:52 +03:00
kimchy 9af288a42f fix test for existance of directory based mappings when creating an index 2010-07-29 19:22:21 +03:00
kimchy 0186a03684 add another script function test 2010-07-29 16:07:57 +03:00
kimchy 4f407e18aa Put Mapping: When using a single node and updating a mapping, it is not marked as `acknowledged`, closes #280. 2010-07-29 15:21:11 +03:00
kimchy e955e41a91 improve handling of failure when trying to read md5, should be ignored on any exception, not just io exception 2010-07-29 15:10:37 +03:00
kimchy 194e6cbff6 Geo support: geo_point type and geo_distance filter, closes #278, #279 2010-07-28 21:06:18 +03:00
kimchy af055f75ce should be static 2010-07-26 13:46:01 +03:00
kimchy f251e2fd32 support lt(N), le(N), gt(N), ge(N) in wait for nodes (thanks Lukas) 2010-07-26 13:44:29 +03:00
kimchy 65284ba2ba Terms Factes: Allow to provide regex controlling which terms should be included, closes #277. 2010-07-26 12:23:03 +03:00
kimchy 03cd2f3772 fix <= 2010-07-26 00:22:21 +03:00
kimchy ac7c8cb650 Analysis: Add pattern analyzer, closes #276. 2010-07-25 22:40:50 +03:00
kimchy 477a24efc6 Mapping: Revise dynamic mapping (into default), merge default to new mappings, closes #275. 2010-07-25 21:31:16 +03:00
kimchy 1884c4219a Mapping: Dynamic mapping definitions are ignored, closes #274. 2010-07-25 12:03:04 +03:00
kimchy 457b56937e small improvements to groovy client, allows for closure in the search request 2010-07-25 10:32:22 +03:00
kimchy 77b6d1d8b8 remove cloud plugin, reimplement an AWS specific cloud plugin with S3 gateway and EC2 discovery 2010-07-24 23:39:43 +03:00
kimchy a6a993ff08 fix ignored exception 2010-07-24 18:48:28 +03:00
kimchy 0072dd816b nicer logging, and use sync streams 2010-07-23 23:10:59 +03:00
kimchy 23ddf8bc53 do a reroute when a node becomes master 2010-07-23 20:55:26 +03:00
kimchy f6255f8ad9 change logging level to debug 2010-07-23 19:29:14 +03:00
kimchy 72682a6730 always retry on connect exception with master operations 2010-07-23 01:59:56 +03:00
kimchy 9283e2a7ad local discovery should update the local metadata state once it started 2010-07-23 01:39:37 +03:00
kimchy 5706e5f6b9 rename node close to node closed exception 2010-07-23 01:12:36 +03:00
kimchy 34b84b0e61 when retrying on primary shard when node disconnected, add it again the the cluster service as a listener 2010-07-23 01:07:09 +03:00
kimchy 8875b9641b cleanup test 2010-07-23 00:50:08 +03:00
kimchy 2a3130c649 REST Search API: Change `score` to `_score` to denote sorting by hit score, closes #271. 2010-07-22 23:39:55 +03:00
kimchy 42c2b6b743 throw a proper exception when no type is found 2010-07-22 21:17:11 +03:00
kimchy c1ab9f290f remove compile warnings 2010-07-22 00:25:25 +03:00
kimchy b55fd0c0fc verify that data that is written to the blob (in fs) is the same as expected in the blob write api call 2010-07-21 23:46:11 +03:00
kimchy 4dcf219086 after raising a failure, return, don't try and write the blob 2010-07-21 23:45:15 +03:00
kimchy 2432a7cb5b fixed highlighted styled built in schema tags to properly increate from hlt1 to hlt10 2010-07-21 20:38:13 +03:00
kimchy 8ec7ee66a9 Search: Sending a request that fails to parse can cause file leaks, closes #270. 2010-07-21 16:59:58 +03:00
kimchy 0cb97e4044 cancel the timeout in any case 2010-07-21 16:38:37 +03:00
kimchy 8d45901a8a change logging level, nodes FD will detect it as well 2010-07-21 16:30:40 +03:00
kimchy 48d33ec70a Cluster Health API: Add `wait_for_nodes` (accepts "N", "<N", ">N", "<=N", and ">=N"), closes #269. 2010-07-21 16:29:44 +03:00
kimchy 2a0e8d4ec9 better state when closing shard, and handling its state 2010-07-21 09:20:28 +03:00
kimchy 3629540953 when trying to rejoin the cluster, clean the routing/metadata so the node will get fresh shards 2010-07-21 08:55:35 +03:00
kimchy 15cf7a2361 initialize the nodes before a possible retry, so we have the local node id for it 2010-07-21 01:53:00 +03:00
kimchy 39e546b64d add a proper master not discovered exception for master based operations (cluster state) and the ability to control the timeout till master is found 2010-07-21 01:51:58 +03:00
kimchy a2ceaa91cc improve retry when closing the node performing an operation 2010-07-20 22:34:09 +03:00
kimchy 1ff8b43aa7 clean timeout listener from on going listeners once its removed 2010-07-20 21:11:56 +03:00
kimchy f4064038e4 improve load test 2010-07-20 20:53:14 +03:00
kimchy b3b5c9b0cc don't execute it on a thread pool 2010-07-20 18:34:01 +03:00
kimchy 553e8134ee create a send transaction exception on the calling thread and fill its stack, so we know where its coming from 2010-07-20 18:33:44 +03:00
kimchy 07480846c3 fix logging to actually pass the actual logging parameters 2010-07-20 18:33:12 +03:00
kimchy 6915b39dc3 clean timeout listener task when removing the listener 2010-07-20 18:32:49 +03:00
kimchy 45e54c1705 add retry if master is not connected till master is connected or a timeout has passed to master based operations (like cluster health) 2010-07-20 18:32:21 +03:00
kimchy 45c821316b no need for volatile on writer, its always accessed under a lock 2010-07-20 13:58:01 +03:00
kimchy 3c8cf68a17 improve timeout cluster service listener 2010-07-20 13:50:19 +03:00
kimchy 4ac1409afa don't list connect exceptions 2010-07-20 11:01:08 +03:00
kimchy 679f3758b7 ignore a more genric exception 2010-07-20 09:29:10 +03:00
kimchy 3867591ffa clean up the java api a bit 2010-07-20 09:18:16 +03:00
kimchy 2ee870e01e nicer logging, append space before index + shard id logging 2010-07-20 08:53:57 +03:00
kimchy 6f2c0f4e55 spelling 2010-07-19 17:51:42 +03:00
kimchy 5f0470d68b Support Cross-Origin resource in http/rest module, closes #218. 2010-07-19 15:06:13 +03:00
kimchy e48b1d98db more recovery under load tests, now with nodes shutdown, ignore node connected exception or retry when performing replicated operations 2010-07-19 11:33:21 +03:00
kimchy 9def68a733 add full rolling restart recvoery test 2010-07-19 10:08:12 +03:00
kimchy df602054fb fix relocation of primaries and perform the operation on the relocated primary replica as well 2010-07-19 09:54:52 +03:00
kimchy 0d20790ffe when shutting down a node, close shard (and indices) in parallel 2010-07-19 02:17:59 +03:00
kimchy d657d4447b more shard relocation tests, create a cached executor that can be used to allocate threads that are not affected by the actual thread pool impl (can always fork, no blocking) 2010-07-19 01:45:09 +03:00
kimchy f9cd7cb932 add a recovery test while under indexing load 2010-07-18 23:57:57 +03:00
kimchy 9c9dfe0b39 add a recovery test while under indexing load 2010-07-18 23:40:13 +03:00
kimchy fefcfb5b5c refactor recovery to be handled on the node level (and not per shard), with better retry mechanism when doing peer shard recovery 2010-07-18 22:54:21 +03:00
kimchy 8ac8dd818d now that we have a file based transaction log, increate the memory clean threshold 2010-07-18 21:24:52 +03:00
kimchy 49f8501a5b nicer serilization 2010-07-17 22:58:16 +03:00
kimchy 910ad2a408 fix optimization to not do sorting when searching with score "desc" sorting 2010-07-17 19:38:04 +03:00
kimchy 08ca383fd5 more work on cloud gateway 2010-07-17 15:20:13 +03:00
kimchy 6194b3ab0f add NaN handling for specific hit score as well 2010-07-17 13:52:31 +03:00
kimchy 1fccaf06e9 start work on reimplementing cloud gateway 2010-07-17 13:12:53 +03:00
kimchy e61dc78c21 max_score == NaN, closes #263. 2010-07-17 01:39:12 +03:00
kimchy 3006183191 fix a problem where max_score was NaN if there were no hits from a shard (though there are hits from other shards), it will still be NaN where there are no hits 2010-07-15 22:58:18 +03:00
kimchy cf998e2f89 protected when not of the same type mapper when merging 2010-07-15 22:35:18 +03:00
kimchy 9f6fde9adb better handling of failed merge because of different types 2010-07-15 20:40:40 +03:00
kimchy cfa56cb3d6 Put Mapping: When updating existing mappings, the request returns with acknowledged `false`, closes #262. 2010-07-15 20:03:44 +03:00
kimchy c987e8a534 Querying mapping on a non-master throws an error, closes #261. 2010-07-15 19:03:45 +03:00
kimchy ee60d7dedc better exception message when failing to parse a specific field 2010-07-15 18:46:12 +03:00
kimchy ca8ad83092 only count data_nodes with recover_after_nodes parameter 2010-07-15 01:12:00 +03:00
kimchy 61fadb4dc0 use safe method (to get proper exception) to get shard injector 2010-07-14 23:39:19 +03:00
kimchy f6fa056f1c fix REST api for cluster state and the ability to filter parts of the data returned 2010-07-14 16:05:47 +03:00
kimchy 95ba62f83d Translog: Implement a file system based translog and make it the default, closes #260. 2010-07-14 14:53:58 +03:00
kimchy e7d80b8244 remove max_shards_per_node setting (actually, you could not set it even...), will be revisted when proper SLA based shard allocation will be implemented 2010-07-14 14:29:56 +03:00
kimchy a6bd64f30d Facet results vary depending on size, closes #259. 2010-07-14 11:30:39 +03:00
kimchy 2704ab3d69 improve join process in cluster, fetch the cluster meta-data on join and handle new meta data 2010-07-14 10:34:22 +03:00
kimchy 141506afc5 improve join process in cluster, fetch the cluster meta-data on join and handle new meta data 2010-07-14 09:28:56 +03:00
kimchy 2e2f22fbd6 nullify the updater on stop 2010-07-14 08:55:57 +03:00
kimchy c4d6ccd07b nicer logging 2010-07-14 08:41:42 +03:00
kimchy 234455530a Make merging mappings smarter, closes #253. 2010-07-14 07:32:17 +03:00
kimchy 892dadca59 refactor meta data opertions into discrete services that do all operations within the cluster update process 2010-07-13 18:01:57 +03:00
kimchy ed04721f08 doc 2010-07-13 10:19:57 +03:00
kimchy f0cf552bc5 sorting breaks when sorting on a field that has no value in some documents 2010-07-13 09:38:55 +03:00
kimchy e6bd3f2693 Put Mapping: Fail when an analyzer is specified that was not configured, closes #252. 2010-07-12 23:18:32 +03:00
kimchy 65dffdf9cf throw a descriptive exception when trying to sort on multi token or multi values field per doc 2010-07-12 23:13:37 +03:00
kimchy ff17183798 add more sort tests 2010-07-12 22:25:49 +03:00
kimchy fe50a6f64e improve allocation of shards based on existing work location, wait with index creation on other nodes, so listing their respective existing location will work 2010-07-12 01:33:38 +03:00
kimchy 6722e05418 only list unallocated shards on data nodes 2010-07-11 22:11:06 +03:00
kimchy 52ac24fa23 more debug statr on recovery from gateway 2010-07-11 21:51:45 +03:00
kimchy e7a8da8236 do flush within the finalization of index shard recovery 2010-07-11 21:20:36 +03:00
kimchy 30b34b975c fix spelling 2010-07-11 20:53:03 +03:00
kimchy 294f09a1d7 Shutdown API: Improve behavior when shutting down the whole cluster, closes #250. 2010-07-11 20:41:58 +03:00
kimchy 09493691a7 nicer serializtion 2010-07-11 15:41:12 +03:00
kimchy 671a14ce87 freq is always positive int 2010-07-11 15:35:53 +03:00
kimchy a6ef805aa8 add doc 2010-07-11 00:40:29 +03:00
kimchy ef369fc40f javadoc 2010-07-10 23:21:34 +03:00
kimchy d0cdbeffba only send intiial state recived after sending join request, just when we actually get a new cluster state that includes us. Also, handle no nodes to send to with generic transport nodes action. 2010-07-10 15:43:49 +03:00
kimchy a0ead02299 Don't create / use the work directory if not needed (for example, on client / non data) nodes, closes #249. 2010-07-10 14:29:39 +03:00
kimchy 7ba3b3a9eb Zen Discovery: Control which nodes are allowed to become masters, closes #248. 2010-07-09 04:27:13 +03:00
kimchy b657ffc5e7 Zen Discovery: Control which nodes are allowed to become masters, closes #248. 2010-07-09 04:22:03 +03:00
kimchy d1f5577c40 add channel based read from fs snapshot, reusing the same file descriptor with direct position based reads 2010-07-09 03:51:36 +03:00
kimchy 6cdd6a015c reuse file buffer 2010-07-09 03:24:31 +03:00
kimchy d531d82cfb Zen Discovery: A node might get into an infinite state of trying to find a master (when client / non_master) nodes exists, closes #247. 2010-07-09 00:43:06 +03:00
kimchy 87eb6bed8f nicer logging 2010-07-08 22:53:41 +03:00
kimchy 720e6a6d5b nicer logging 2010-07-08 22:52:03 +03:00
kimchy be6aaa157f fixed a bug in fs translog where it wasn't seeking correctly, and not counting the number of operations 2010-07-08 22:18:26 +03:00
kimchy ef148077fc better logging 2010-07-08 21:37:10 +03:00
kimchy 84af9f5681 improve handling of translog snapshotting 2010-07-08 20:51:37 +03:00
kimchy 4429a61528 better cleanup on shard startup failure 2010-07-08 19:20:03 +03:00
kimchy b69fc265c5 index writer ram buffer size can only be 2G, make sure not to set an illegal value 2010-07-08 19:03:55 +03:00
kimchy 400ec7b8d1 index writer ram buffer size can only be 2G, make sure not to set an illegal value 2010-07-08 18:58:04 +03:00
kimchy 63deef42cf add a help appendable blob store based on immutable one 2010-07-08 18:53:40 +03:00
kimchy 8d669ff54e move sorting to be done using the new fielddata cache, and not using lucene built in FieldCache 2010-07-08 14:42:35 +03:00
kimchy 97b5b41522 enable eager cache eviction based on readers, requires a nasty hack to Lucene, basically overriding two classes to hook into when a reader actually gets closed (and only enabled when elasticsearch is before lucene in the classpath, which we take for in the startup scripts) 2010-07-07 17:15:26 +03:00
kimchy a36b06c682 ignore failed snapshot on a non started index 2010-07-07 16:59:33 +03:00
kimchy a8c317e355 fix terms facets test to take into account distributed terms request 2010-07-07 16:33:39 +03:00
kimchy 5ff46911b6 ignore recovery when no node to recover from (it has disconnected, for example) 2010-07-07 16:30:28 +03:00
kimchy 3fb68c52da ignore snapshot on a shard thas has just been created 2010-07-07 16:30:17 +03:00
kimchy 11a8d637c0 ignore snapshot on a shard thas has just been created 2010-07-07 16:26:19 +03:00
kimchy e40aa540fe ignore recovery when no node to recover from (it has disconnected, for example) 2010-07-07 16:25:55 +03:00
kimchy 9578ad3ef1 Terms Facets: Allow to specify a set of terms to exclude in the request, closes #246. 2010-07-07 14:41:34 +03:00
kimchy 2b5458daf6 implement a simple file system based translog, not the defeault yet, should provide better memory management 2010-07-07 09:30:07 +03:00
kimchy b2bdb149f9 log shard close failure when closing an index 2010-07-06 22:50:40 +03:00
kimchy 6aa9be238c use the atomic integer counter to represent the size 2010-07-06 18:45:19 +03:00
kimchy 267016758d improve handling of memory caching with file system, only force compound file when really needed (when an extension that exists within the compound file is part of the memory cached extensions) 2010-07-06 18:21:36 +03:00
kimchy ac01bb86e8 clean lock only before opening an index writer 2010-07-06 18:13:05 +03:00
kimchy 2908418d37 checn the default byte buffer cache to 200mb, now that its global per node 2010-07-06 17:58:30 +03:00
kimchy 070cb5b295 allocate replica shards (backups) only after the primaries have been allocated, this results in better theoretical shard allocation, and the ability to reuse local storage index files (comared against the backup) 2010-07-06 17:58:05 +03:00
kimchy 1d39bb4d51 cache md5 computation for unallocated shard and reused it 2010-07-06 09:00:24 +03:00
kimchy d4f86899e3 more work on reuse work dir, improved transaction log to move to stream based 2010-07-06 00:12:40 +03:00
kimchy b078c9206a more work on reuse work dir, added md5 logic to identify what need to be recovered or not, note, requires reindexing 2010-07-05 14:13:55 +03:00
kimchy 79b69764e0 doc 2010-07-01 15:03:11 +03:00
kimchy e8b261c165 better shutdown handling 2010-07-01 10:28:57 +03:00
kimchy ad5525f575 fix unwrapping of exceptions 2010-07-01 10:27:53 +03:00
kimchy f6c9c49645 better logging when failing to read full content of a transport message 2010-07-01 03:00:57 +03:00
kimchy c75a56ca17 better shutdown handling 2010-07-01 02:41:00 +03:00
kimchy c2d02e4e3a wrong serialization of shardId 2010-07-01 02:18:35 +03:00
kimchy d848ab3df3 rename executors to esexecutors 2010-07-01 00:06:34 +03:00
kimchy b0d736b489 make the shard routing an actual component 2010-06-30 23:39:27 +03:00
kimchy 811559f393 add another test if the pool size is greater than the core pool size before a worker can exit 2010-06-30 23:08:05 +03:00
kimchy 6709518c4e add the ability to list shard index blobs without actually having one created 2010-06-30 22:54:59 +03:00
kimchy 66096e97f2 Remove the terms API (replaced by facets in search), closes #242. 2010-06-30 19:49:41 +03:00
kimchy df7e3f55a4 improve cluster blocks data structures for simpler usage 2010-06-30 16:06:31 +03:00
kimchy 4049da97fb time value to support parsing days as well 2010-06-30 13:00:25 +03:00
kimchy 088e0b5a64 Automatic management of indexing buffer size, closes #241. 2010-06-30 01:46:57 +03:00
kimchy fdb2eff998 Automatic management of indexing buffer size, closes #241. 2010-06-29 22:54:54 +03:00
kimchy 03ac38fdba better benchmark on memory monitor 2010-06-29 22:12:43 +03:00
kimchy 3f6cd46736 allow to configure tcp network modules (transport, memcached and http) to use blocking io (just enabling it to do some performance tests) 2010-06-29 21:07:38 +03:00
kimchy 01ff81fa89 fix log message 2010-06-29 19:49:28 +03:00
kimchy bc803b4fcf close the thread pool on test 2010-06-29 18:35:03 +03:00
kimchy 703657699a refactor new thread pools in, remove dynamic ones 2010-06-29 18:20:31 +03:00
kimchy de4b21e986 backups don't become primaries on node failures, primaries are first allocated and then start initializing from gateway 2010-06-29 18:20:13 +03:00
kimchy 514df4ee3f refactor the shard routing abstraction, no need for it 2010-06-29 18:02:10 +03:00
kimchy fe5e5a073b work on scaling and blocking thread pool 2010-06-29 18:01:54 +03:00
kimchy 9051ae3a65 work on scaling and blocking thread pool 2010-06-29 17:50:05 +03:00
kimchy 297001de53 update benchmark code 2010-06-29 08:59:28 +03:00
kimchy aa5791a14f move block metadata checks to the transport level 2010-06-29 08:51:36 +03:00
kimchy a82bf74c1c expose blocks in the cluster state rest api 2010-06-29 01:00:07 +03:00
kimchy 5f538b1ba3 Block operation performed on cluster until it recovered from the gateway, closes #239. 2010-06-28 22:53:13 +03:00
kimchy b970f0dcc2 fix toString of bytes value 2010-06-28 17:53:07 +03:00
kimchy 63523f4525 Query DSL: field query does not take into account `allow_leading_wildcards`, closes #236. 2010-06-28 12:33:36 +03:00