Commit Graph

312 Commits

Author SHA1 Message Date
kimchy 02981f6101 Routing: Allow to specify on the `_routing` mapping that its required, and fail index operations that do not provide one, closes #520. 2010-11-16 15:10:41 +02:00
kimchy 5c6c4bfb5a Histogram Facet: Allow to define a key field and value script, closes #517. 2010-11-15 15:02:35 +02:00
kimchy 37335bc6a9 also retry when there are no routing shards, might happen in case of recovery from gateway before shard routing initialization 2010-11-14 20:09:00 +02:00
kimchy beb77c9b0a fix a bug in mapping serialization (introduced on 0.13), where properties gets underscore cased by mistake, and not retain their original naming 2010-11-11 15:38:20 +02:00
kimchy 17cba798e6 add double restart test for local gateway 2010-11-11 14:08:58 +02:00
kimchy 4c6af6afa5 Cluster health API should return RED status (on applicable levels) when an index has not recovered from the gateway, closes #507. 2010-11-09 21:25:22 +02:00
kimchy d0e0b85178 add many indices stress test 2010-11-09 16:22:50 +02:00
kimchy 7340d6973d slim down the mapping serialization to xcontent (json), don't write default values 2010-11-07 12:33:44 +02:00
kimchy 6804c02e97 Create specific `data` location for indices (move from work), closes #473. 2010-11-03 14:41:29 +02:00
kimchy 92b3ae3f73 Change single operation shard hashing to only use id, and not id and type, closes #472. 2010-11-03 12:47:34 +02:00
kimchy a62f1f3e0d API: Allow to control document shard routing, and search shard routing, closes #470. 2010-11-02 19:56:18 +02:00
kimchy 4ff1b429f1 Possible (rare) shard index corruption / different doc count on recovery (gateway / shard), closes #466. 2010-11-01 23:00:16 +02:00
kimchy 4579c04a9e Mapper: Ip Type Support (ipv4), auto detection with dynamic mapping, closes #461. 2010-10-28 12:02:20 +02:00
kimchy 184276154c add search stress test 2010-10-26 17:17:12 +02:00
kimchy 5966d2005d refacto package name from facets to facet (make more sense when facet will be easily pluggable) 2010-10-25 18:49:30 +02:00
kimchy ca7a7467dc Indices API: Add open and close index, closes #447. 2010-10-23 21:52:09 +02:00
kimchy a76824e395 make index not recovered a bock used in both gateways (shared/blob and local) 2010-10-23 17:03:38 +02:00
kimchy f0f62ce00c improve test to wait for 2 nodes 2010-10-23 01:24:53 +02:00
kimchy e11589f3c7 add doc 2010-10-23 01:07:37 +02:00
kimchy 9237dafef9 Query DSL: Query String _missing_ and _exists_ syntax, closes #446. 2010-10-22 17:51:19 +02:00
kimchy f63ee3158a Query DSL: Add `exists` and `missing` filters to filter documents where a field either has a value or not in them, closes #445. 2010-10-22 15:14:05 +02:00
kimchy 5d1d927e09 Write Consistency Level for index/delete/delete_by_query/bulk with one/quorum/all. Defaults to quorum, closes #444. 2010-10-22 02:51:00 +02:00
kimchy 0b593bd6d9 upgrade to guava r07 2010-10-20 12:09:19 +02:00
kimchy ae5bc20959 Reduce multiple indices overhead, don't create Indices data on nodes that don't hold any shard for an index, closes #441. 2010-10-20 02:57:49 +02:00
kimchy a4eea0aeaa change index creation / deletion logic not to wait for it to be created on other nodes since now, they might not be created... 2010-10-19 19:33:27 +02:00
kimchy 53a3df5d8e move list of local files to be indices level, so no need for index created on a node in order to list it 2010-10-19 17:36:21 +02:00
kimchy 4c8978237f move operation routing to be cluster level and not index level 2010-10-19 16:23:39 +02:00
kimchy 5d54e36964 Statistical Facet: Allow to compute statistical facets on more than one field, closes #436. 2010-10-17 16:22:20 +02:00
kimchy 1587a3815e more options on stress rolling restart test, variable number of fields, variable length of text 2010-10-16 03:26:11 +02:00
kimchy b09675dcfa add a stress test for rolling restart nodes 2010-10-16 01:05:25 +02:00
kimchy 4f9d9c1395 add test for merging two stats field facet using same facet name 2010-10-11 00:40:37 +02:00
kimchy 6314c2460c Geo Overhaul (work with multiple locations), closes #414. 2010-10-08 22:31:21 +02:00
kimchy d0bf743ab4 Query DSL: query_string / field to use the optimized match_all query when using * (or *:*), closes #413. 2010-10-07 16:48:24 +02:00
kimchy cf5ed1d177 Term Facet: Use a script that provides the facets to index, closes #410. 2010-10-07 01:49:12 +02:00
kimchy 22ed71e2e6 allow to execute term facet on more than one field 2010-10-06 13:39:02 +02:00
kimchy c9a47a126f Scripts: Allow to access `_fields` providing access to stored fields, closes #402. 2010-10-03 12:51:41 +02:00
kimchy 5b8bc333bd JavaScript Plugin: Allow to use javascript for scripts, closes #401. 2010-10-03 02:20:37 +02:00
kimchy ce28882660 add refresh option to index/create/delete opereation, REST allows for refresh parameter (defaults to false) 2010-09-26 09:07:37 +02:00
kimchy eb4f4f99b3 add recover_after gateway tests 2010-09-20 09:44:57 +02:00
kimchy 770bac252a fix local discovery to handle master nodes specifically better 2010-09-19 23:49:15 +02:00
kimchy 95bb02b330 simple specific master nodes test 2010-09-19 21:11:53 +02:00
kimchy 8def6f59b3 simple specific master nodes test 2010-09-19 18:48:48 +02:00
kimchy 20b6688b0a make ping tests run faster using cluster state api 2010-09-19 18:13:03 +02:00
kimchy f82ceb1e1e do snapshots on more lower level interval 2010-09-18 22:18:21 +02:00
kimchy 2372f481aa Shared Storage Gateway (fs/s3): Wrong snapshotting of transaction log, closes #375. 2010-09-18 21:33:38 +02:00
kimchy ff081240eb imrpove caching and fix delete mapping logic 2010-09-17 01:25:37 +02:00
kimchy 4b87f63bed Terms Facet: Allow to provide `_index` as the field, resulting in facets on indices, closes #374. 2010-09-16 15:11:30 +02:00
kimchy f49f3e169a Mapping API: Delete Mapping (with data), closes #373. 2010-09-16 14:35:07 +02:00
kimchy f6fa6ea44e more work on bulk, client API 2010-09-14 22:08:43 +02:00
kimchy 38aeba438d XContent refactoring, closes #367. 2010-09-12 23:20:15 +02:00
kimchy eccc7d5ef2 Search API: Allow to name filters and return per hit the filters it matched on, closes #364. 2010-09-11 12:38:19 +03:00
kimchy 4e661c165a Geo search across anti-meridian doesn't work, closes #363. 2010-09-10 16:04:10 +03:00
kimchy f270fc00d2 earch API: Automatically identify "script" fields on the field elements in search, close #357. 2010-09-06 11:51:31 +03:00
kimchy b40f0bc5a4 Local Gateway: Expose `gateway.local.initial_shards` to control when to recover and index, closes #352. 2010-09-03 18:38:24 +03:00
kimchy cd28afe950 Search API: Allow for script fields to extract parts of the stored _`source`, closes #349. 2010-09-02 21:29:41 +03:00
kimchy 0250896a09 reduce shard search response size by not serilaizing the node id, index, and shard id 2010-09-02 11:19:25 +03:00
kimchy 042af200e8 Gateway: Default to `local` gateway (replace `none` gateway), closes #347. 2010-09-01 23:21:26 +03:00
kimchy ba5cd19989 gateway test to make sure it recovers from the latest version 2010-08-31 21:13:42 +03:00
kimchy 289cfdf25a improve replica allocation, fix test to wait for recovery on node2 2010-08-30 23:37:09 +03:00
kimchy 61764c5b69 add another test for local gateway, fix race when loading the current state of a node 2010-08-30 19:29:00 +03:00
kimchy d9979f8dfe more work on local gateway, start integration test it 2010-08-30 01:25:45 +03:00
kimchy 5b5d42445d move the shared fs gateway base class to fs package 2010-08-29 23:22:55 +03:00
kimchy 19ddee5ec9 add a test for time interval 2010-08-26 18:12:51 +03:00
kimchy b52d854711 smarter handling of throttling when preferring allocating to an existing work location 2010-08-23 21:39:54 +03:00
kimchy 1461da5b49 move prefetch allocation to be a NodeAllocation 2010-08-23 17:22:44 +03:00
kimchy edbd586ee5 add load test for recovery from gateway 2010-08-23 16:33:33 +03:00
kimchy 5bd37f6f47 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 14:48:49 +03:00
kimchy 1517fa3d28 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 01:45:03 +03:00
kimchy 14237317fc support compression of the source field 2010-08-15 17:57:14 +03:00
kimchy 01b3d42cee separet recovery tests to one with small buffer size and on with bigger one 2010-08-14 13:44:42 +03:00
kimchy d12c757824 Gateway: Chunk based storage broken, fails to recover from gateway, closes #318. 2010-08-13 14:03:52 +03:00
kimchy e29925684a Term Facets: Add `reverse_cout` and `reverse_term` to `order` options, closes #314. 2010-08-12 15:26:26 +03:00
kimchy c0a7dc327c range facet: from is used in to_str, should be to 2010-08-11 23:40:42 +03:00
kimchy 9f8e5cb2c0 add a simple integration level test for scroll 2010-08-10 20:48:05 +03:00
kimchy f2018e2f86 Facets: Automatically filter by type for facets that use explicit field names that are prefixed by the type, closes #310. 2010-08-10 17:03:14 +03:00
kimchy 05d07b036a remove ning async http client, no need for it 2010-08-10 00:00:24 +03:00
kimchy ff8d743337 add geo distance sort option, refactor java level sort API to have sort builders (similar to facets) 2010-08-08 21:46:11 +03:00
kimchy 37af7f5aef add sort values as part of the response per search hit 2010-08-08 18:21:17 +03:00
kimchy 7cb2010dda add script support for terms facets, allowing to control if a certain term will be included in the facets result (boolean result), or muging of the term 2010-08-08 10:37:14 +03:00
kimchy ee1d50f8d8 add script support for terms facets, allowing to control if a certain term will be included in the facets result (boolean result), or muging of the term 2010-08-08 10:27:06 +03:00
kimchy 566ae94478 add the ability to control the "order" by of terms facet, can either be `term` or `count`. defaults to `count`. 2010-08-08 09:31:40 +03:00
kimchy d4547c629f New API, update settings, allowing to change specific runtime settings of the cluster (indices). `number_of_replicas` is the only settings supported, allowing to change the number of repliacs of one or more indices. 2010-08-08 09:17:09 +03:00
kimchy 8c5fdf4aee Ability to return all stored fields with a *, closes #296. 2010-08-04 13:22:55 +03:00
kimchy 91c256361e support for custom script based sorting 2010-08-03 18:14:02 +03:00
kimchy de8ffaf5fb Geo: bounding box filter, closes #290. 2010-08-03 01:51:46 +03:00
kimchy dc6ef326d9 Facets: Filter based facet, closes #289. 2010-08-02 19:17:57 +03:00
kimchy 76cfbe6a44 support dates in range facet 2010-08-02 10:35:21 +03:00
kimchy ef861a6b7b Search Facets: Range Facet, closes #287. 2010-08-01 20:08:27 +03:00
kimchy b8b21a3363 Geo: `geo_distance` facet, closes #286. 2010-08-01 13:31:03 +03:00
kimchy dfb68c6310 Search fails when sorting on a field that has no values in the hits returned, closes #285. 2010-07-31 16:41:15 +03:00
kimchy 0186a03684 add another script function test 2010-07-29 16:07:57 +03:00
kimchy 194e6cbff6 Geo support: geo_point type and geo_distance filter, closes #278, #279 2010-07-28 21:06:18 +03:00
kimchy 65284ba2ba Terms Factes: Allow to provide regex controlling which terms should be included, closes #277. 2010-07-26 12:23:03 +03:00
kimchy 72682a6730 always retry on connect exception with master operations 2010-07-23 01:59:56 +03:00
kimchy 9283e2a7ad local discovery should update the local metadata state once it started 2010-07-23 01:39:37 +03:00
kimchy 8875b9641b cleanup test 2010-07-23 00:50:08 +03:00
kimchy 48d33ec70a Cluster Health API: Add `wait_for_nodes` (accepts "N", "<N", ">N", "<=N", and ">=N"), closes #269. 2010-07-21 16:29:44 +03:00
kimchy a2ceaa91cc improve retry when closing the node performing an operation 2010-07-20 22:34:09 +03:00
kimchy f4064038e4 improve load test 2010-07-20 20:53:14 +03:00
kimchy 45e54c1705 add retry if master is not connected till master is connected or a timeout has passed to master based operations (like cluster health) 2010-07-20 18:32:21 +03:00
kimchy 3c8cf68a17 improve timeout cluster service listener 2010-07-20 13:50:19 +03:00
kimchy e48b1d98db more recovery under load tests, now with nodes shutdown, ignore node connected exception or retry when performing replicated operations 2010-07-19 11:33:21 +03:00
kimchy 9def68a733 add full rolling restart recvoery test 2010-07-19 10:08:12 +03:00
kimchy d657d4447b more shard relocation tests, create a cached executor that can be used to allocate threads that are not affected by the actual thread pool impl (can always fork, no blocking) 2010-07-19 01:45:09 +03:00
kimchy f9cd7cb932 add a recovery test while under indexing load 2010-07-18 23:57:57 +03:00
kimchy 9c9dfe0b39 add a recovery test while under indexing load 2010-07-18 23:40:13 +03:00
kimchy a6bd64f30d Facet results vary depending on size, closes #259. 2010-07-14 11:30:39 +03:00
kimchy f0cf552bc5 sorting breaks when sorting on a field that has no value in some documents 2010-07-13 09:38:55 +03:00
kimchy ff17183798 add more sort tests 2010-07-12 22:25:49 +03:00
kimchy 8d669ff54e move sorting to be done using the new fielddata cache, and not using lucene built in FieldCache 2010-07-08 14:42:35 +03:00
kimchy a8c317e355 fix terms facets test to take into account distributed terms request 2010-07-07 16:33:39 +03:00
kimchy 9578ad3ef1 Terms Facets: Allow to specify a set of terms to exclude in the request, closes #246. 2010-07-07 14:41:34 +03:00
kimchy d4f86899e3 more work on reuse work dir, improved transaction log to move to stream based 2010-07-06 00:12:40 +03:00
kimchy b078c9206a more work on reuse work dir, added md5 logic to identify what need to be recovered or not, note, requires reindexing 2010-07-05 14:13:55 +03:00
kimchy 66096e97f2 Remove the terms API (replaced by facets in search), closes #242. 2010-06-30 19:49:41 +03:00
kimchy 33d357dbb4 Memory Store / FS Memory: Create a node level memory store cache and allocator, closes #235. 2010-06-26 22:34:30 +03:00
kimchy 00d2abef40 refactor facets builder API to make it more manageable 2010-06-26 01:36:42 +03:00
kimchy 5df2257319 add test for facets with search size 0 2010-06-25 22:46:07 +03:00
kimchy 7ed7c6db4e Gateway: Internal refactoring, requires manual upgrade when using fs gateway, closes #232. 2010-06-21 22:40:12 +03:00
kimchy 384f8a4f42 Return the maxScore per search and score for each search hit, closes #205. 2010-06-20 00:23:27 +03:00
kimchy 626989c55e don't return the source when asking for scripted fields 2010-06-16 04:48:06 +03:00
kimchy 2d6f61b3c1 Facets: Script Histogram facet, closes #228. 2010-06-16 04:22:00 +03:00
kimchy bb24b56c66 Facets: Script statistical facets, closes #227. 2010-06-16 02:18:24 +03:00
kimchy c2786038e2 Script Filter: Support providing a custom script as a filter, closes #226. 2010-06-16 01:32:04 +03:00
kimchy f75abbaa00 big refactoring thanks to proper jarjar built from source with asm 3.3 allowing to jarjar guice and others, includes package relocations 2010-06-15 17:59:04 +03:00
kimchy d5bd77cc38 big refactoring thanks to proper jarjar built from source with asm 3.3 allowing to jarjar guice and others, includes package relocations 2010-06-15 17:28:05 +03:00
kimchy 636bfe8466 big refactoring thanks to proper jarjar built from source with asm 3.3 allowing to jarjar guice and others, includes package relocations 2010-06-15 16:51:38 +03:00
kimchy 0a1bc874c3 Search: Allow to specify script fields to be loaded, closes #221. 2010-06-14 12:50:33 +03:00
kimchy 953779ccea allow to pass parameters to custom score script 2010-06-14 04:12:57 +03:00
kimchy 751c56f31e Query DSL: custom score (scripted) query, closes #220. 2010-06-14 03:15:23 +03:00
kimchy 64479a11c3 Search Facets: Histogram facets to allow to specify different key and value fields 2010-06-09 00:50:07 +03:00
kimchy 5ca050ffcb Search Facets: Facets to allow to define filters on them, closes #217. 2010-06-08 22:30:21 +03:00
kimchy 47b3a81bec Search Facets: Histogram Facets, closes #215 2010-06-08 15:45:58 +03:00
kimchy 874993557c add sum of squares, variance, and std deviation for statistical facet 2010-06-08 10:39:45 +03:00
kimchy 8aeb589a42 numeric statistical facets support 2010-06-05 23:44:34 +03:00
kimchy 88ba980f79 refactoring of facets to be more open for different facet types 2010-06-05 21:49:10 +03:00
kimchy bcbc0dd741 Search: Search requests hangs when no indices exists, closes #209. 2010-06-05 01:36:24 +03:00
kimchy 0901322337 rename field facet to term facet 2010-06-04 07:03:30 +03:00
kimchy 66c9f2f834 Search Facets: Field Facets, closes #207 2010-06-04 02:31:49 +03:00
kimchy 4b85407f5e work on a nicer java api 2010-05-29 22:23:39 +03:00
kimchy e59b413980 work on a nicer java api 2010-05-29 05:50:17 +03:00
kimchy 84a5c1eac8 Replication Actions: Allow to control replication type - `async` or `sync`, closes #196. 2010-05-28 03:47:35 +03:00
kimchy 32e4c405de add async http client 2010-05-24 23:27:07 +03:00
kimchy 5c6864e8b3 inital work on hadoop plugin - gateway hdfs support 2010-05-23 08:17:52 +03:00
kimchy 216dda3f9c Throttling of recovery (both gateway recovery and peer node recovery), closes #176. 2010-05-16 23:37:56 +03:00
kimchy 836461e6de improve search serialization and deserialization 2010-05-13 18:39:24 +03:00
kimchy 040030dac8 reafctor inject to util from util.guice 2010-05-10 01:34:22 +03:00
kimchy 8053237643 move from gcommon package 2010-05-04 16:02:26 +03:00
kimchy 34d99c39a5 XContent - An abstraction on top of content (JSON inspired), closes #152. 2010-04-29 23:10:47 +03:00
kimchy f8f65c991a finalize abstracting json into xcontent 2010-04-29 02:11:53 +03:00