Commit Graph

394 Commits

Author SHA1 Message Date
kimchy a0fd47159e improve internal scan handling, by using count type for the first scan 2011-02-24 01:04:49 +02:00
kimchy ee9beda398 Search: Add search_type for `count` to return count but still support facets, closes #718. 2011-02-24 00:29:42 +02:00
kimchy f7100c0698 Improve from hits pagination (duplicates), closes #717. 2011-02-23 22:57:44 +02:00
kimchy 6b7192e744 add from/to test without sorting on single shard 2011-02-23 21:41:31 +02:00
kimchy 8371920a89 add min/max to terms stats facet 2011-02-23 20:36:12 +02:00
kimchy 58dbf6d5ca fix terms facet TOTAL comparator to also do secondary sort when totals are equal 2011-02-23 00:55:40 +02:00
kimchy 06850f8673 add another test to scan 2011-02-22 21:01:11 +02:00
kimchy 953fcbc58c Get API: Allow to provide a script as a field, closes #708. 2011-02-22 02:28:19 +02:00
kimchy 818f3b4d75 Search: Add search type `scan` allowing to efficiently scan large result set, closes #707. 2011-02-22 00:11:31 +02:00
kimchy 352cb74f96 Search: Terms Stats Facet, closes #705. 2011-02-19 23:37:04 +02:00
kimchy 585c310aa4 add edge histogram test 2011-02-18 07:35:33 +02:00
kimchy fb90392040 add status based test 2011-02-17 01:28:54 +02:00
kimchy 6ffd60dfb5 change var name to better reflect the fact that its data and not work 2011-02-17 01:07:05 +02:00
kimchy 9ae9ab9553 add flush to search 1 stress test 2011-02-10 16:47:43 +02:00
kimchy 4bcedde011 don't require to provide the type of the doc in the doc element in percolator request, also, change the REST endpoint for percolator to include the type, basically: {index}/{type}/_percolate 2011-02-08 22:24:44 +02:00
kimchy 3062d5b86c nice usage of versioning parse element 2011-02-08 17:50:28 +02:00
kimchy ac8646deb7 Search: By default, don't return the version per hit, unless `version` is set to `true`, closes #676. 2011-02-08 17:48:13 +02:00
kimchy b8c644cab5 ip mapping no longer enabled 2011-02-08 11:24:14 +02:00
kimchy 4319e3cb05 add a test to fetch just an array element (which has no mapping) 2011-02-07 20:24:15 +02:00
kimchy 904bee12bc Search: Failure when sorting on `short` type, closes #665. 2011-02-02 17:55:21 +02:00
kimchy 4ed82023ce another embedded test for query and fetch on a single shard 2011-02-02 14:04:24 +02:00
kimchy 0bce34cb5a optimize single shard search (or search result) by not doing another round of sorting 2011-02-02 13:52:34 +02:00
kimchy c86551d9b5 move master node tests to cluster package, and force it to use zen disco always 2011-02-02 11:15:12 +02:00
kimchy d2a3f5142f improve the order of shutdown of top level components in node, close indices first and applying cluster changes, also, improve atomicity of closing of indices and shards 2011-02-01 23:10:15 +02:00
kimchy 8927e1dee4 Search Scripts: Allow to access score in facet related scripts using `doc.score`, closes #663. 2011-02-01 13:41:58 +02:00
kimchy cc6f65f8b8 Search: When sorting, allow to pass `track_scores` and set it to `true` to get scores/max_score back, closes #662. 2011-02-01 12:38:46 +02:00
kimchy f8b5628db9 have rolling restart stress test create incremental ids 2011-01-31 21:41:46 +02:00
kimchy cebdae5707 refactor script execution handling to improve performance, also introduce specific search script executor 2011-01-31 16:08:06 +02:00
kimchy 5d39cc93dd refresh for test 2011-01-31 15:51:39 +02:00
kimchy e516051ea5 Search: When fetching _parent, only the _id should be returned, and not type#id, closes #658. 2011-01-30 21:16:45 +02:00
kimchy 5b4846b0b6 Translog Flushing: Improve logic, flush not just by operations in the translog, closes #656. 2011-01-30 17:41:22 +02:00
kimchy 94c632b79b Java Client - Allow to set facets in binary format, closes #442. 2011-01-30 15:53:10 +02:00
kimchy 0b09fd0806 Facets: Remove redundant data returned as part of facet response (for example, the field name being faceted), closes #655. 2011-01-29 20:44:58 +02:00
kimchy 332c4cfa53 use bloom filter to reduce seeks when needing to load version 2011-01-29 03:55:00 +02:00
kimchy 00fa7c95fb improve test time 2011-01-26 23:04:10 +02:00
kimchy 5ecb847a3a improve test time 2011-01-26 22:59:35 +02:00
kimchy 6c21c30f31 Index Merge: Improve internal (segment) merging by not initiating it on doc operations, closes #653. 2011-01-26 22:39:40 +02:00
kimchy 5a4686aee5 Search: Allow to pass a search filter, applying only on the query (and not on facets for example), closes #650. 2011-01-26 12:55:30 +02:00
kimchy 00aad4c123 rename tests 2011-01-25 14:40:04 +02:00
kimchy 95dc08fa3b refactor search tests into separate packages 2011-01-24 16:06:28 +02:00
kimchy 95b6184135 Terms Facets: Allow to get `all_terms` back (possibly with count 0), closes #647. 2011-01-23 16:09:18 +02:00
kimchy ce4f09c2b1 Strict dynamic setting: Refuse to index a document with fields not present in the mapping definition, closes #643. 2011-01-22 01:08:41 +02:00
kimchy b1d13febbf Percolate on Index and Bulk, closes #636. 2011-01-19 16:45:45 +02:00
kimchy 574c455203 Terms Facet: Add option include counts where term is missing, closes #632. 2011-01-18 21:51:16 +02:00
kimchy a4c2087511 initial not recovered block on node init, and remove it as usual (once node is initialized / recovered) 2011-01-18 14:08:45 +02:00
kimchy b4e5a542f3 internal refactoring/simplification in percolator 2011-01-17 13:57:32 +02:00
kimchy 38d10d19bc Add `prefer_local` flag to analyze and percolate request, closes #625. 2011-01-14 01:01:37 +02:00
kimchy 180d225016 Percolator, closes #624. 2011-01-13 16:20:31 +02:00
kimchy 85b6a982d4 A setting to auto expand the number of replicas of an index (based on data nodes), closes #623. 2011-01-12 16:27:36 +02:00
kimchy ba9a12e201 Mapper: Add `byte` type (8bit signed), closes #620. 2011-01-11 16:01:10 +02:00
kimchy d4246da7b3 /_open is not recovering indices correctly, closes #615. 2011-01-09 23:31:10 +02:00
kimchy fe0f9ebc9d make lucene document creation allowed to be indexed twice 2011-01-07 14:09:39 +02:00
kimchy b9e808f755 improve version loading by going directly to the relevant reader 2011-01-06 11:50:16 +02:00
kimchy 31231531e1 Highlighting: The result of highlighting for a hit can contain data from another document, closes #600. 2011-01-05 15:45:46 +02:00
kimchy 0faa05b3f2 add more integration versioning tests 2011-01-04 18:09:46 +02:00
kimchy 9423378f27 add heavy concurrent updates to same doc, and make sure it has the same data on all replicas 2011-01-04 17:44:33 +02:00
kimchy 07d361816e Search: Date Histogram Facet, closes #591. 2011-01-02 15:07:45 +02:00
kimchy aec720218d Histogram Facet: Improve perf / memory by defaulting to just providing counts with no totals, closes #587. 2011-01-01 17:27:55 +02:00
kimchy e6b4834768 Highlighting: Allow to highlight on fields without term vectors, closes #585. 2010-12-31 18:57:26 +02:00
kimchy 82298d890c Terms Facet: Performance and memory improvements when faceting numeric fields, closes #583. 2010-12-30 19:45:36 +02:00
kimchy 8e8a3a894f upgrade testing framework jars, improve build scripts to share the testing jars 2010-12-30 16:08:38 +02:00
kimchy 76d042f3c5 explicitly pass null values to methods where applicable 2010-12-30 13:09:11 +02:00
kimchy ed996c3e85 Non-data master nodes and non-master data nodes fail to store data, closes #579. 2010-12-30 12:10:40 +02:00
kimchy f993c4b72b start refactoring of facets to make them more pluggable 2010-12-29 12:49:02 +02:00
kimchy efc95dc00a Java Search API: Passing a the query as json fails, closes #568. 2010-12-20 11:31:09 +02:00
kimchy 415bb5d7f3 Java TransportClient: By default, don't sniff other nodes and use addresses provided as is, closes #565. 2010-12-17 19:31:28 +02:00
kimchy 1c5477d4ed Improve local node recovery process, removing using a block as indication for an index not being recovered. Moving to shard replication group level allocation (with the ability to distinguish between api based creation or recovery based creation). This also allows for throttling based control on initial local gateway based shard allocation. 2010-12-12 22:46:27 +02:00
kimchy bc04243a2b Search Fields: If a field is not stored, automatically extract it from _source (without the need for _source prefix), closes #562. 2010-12-12 05:00:24 +02:00
kimchy 216b2ab912 Highlighting: Automatically use the field values extracted from _source if not stored explicitly in the mapping, closes #561. 2010-12-12 04:40:25 +02:00
kimchy a914865c45 Close API might cause index data to be wiped, closes #560. 2010-12-10 22:19:45 +02:00
kimchy 2a7f964ca2 Geo search fails to find some docs when wrapping over IDL, closes #559. 2010-12-10 08:39:10 +02:00
kimchy e1495ff8be add scrolling update test 2010-12-08 20:56:09 +02:00
kimchy 54437c1bd3 Parent / Child Support, closes #553. 2010-12-08 00:16:05 +02:00
kimchy d3cae103d5 wait for green status in test 2010-12-04 23:57:44 +02:00
kimchy 84f97e96a5 Scrolling issues, fix releasing search context eagerly, they should not be released when scrolling, closes #136. 2010-11-29 02:05:45 +02:00
kimchy 577f06fd43 add indices API to groovy client 2010-11-26 22:19:57 +02:00
kimchy 73e5eb9e14 clean compile warnings 2010-11-26 16:03:25 +02:00
kimchy ebd6316db9 Index templates, closes #540. 2010-11-26 15:45:18 +02:00
kimchy b4113d57d3 Analyze API: An API to analyzer custom text based on an optional analyzer, closes #529. 2010-11-21 17:33:35 +02:00
kimchy 7db5e63ab7 rename shardsIt to shardIt 2010-11-21 13:54:44 +02:00
kimchy fdb98b1dcb Highlighting bug where fragments are broken when number_of_fragments is 0, closes #479. 2010-11-18 20:52:51 +02:00
kimchy 44775c2aa8 Routing: Allow to define `path` on the `_routing` mapping, to automatically extract the routing from it, closes #524. 2010-11-17 17:05:51 +02:00
kimchy 0172eaea89 add broadcasting when no routing and its marked as required to bulk delete as well 2010-11-16 18:53:32 +02:00
kimchy a04d8ec5ad Routing: When specify in the mapping _routing required, a delete without explicit routing value should automatically be broadcasted to all shards, closes #522. 2010-11-16 18:31:08 +02:00
kimchy 02981f6101 Routing: Allow to specify on the `_routing` mapping that its required, and fail index operations that do not provide one, closes #520. 2010-11-16 15:10:41 +02:00
kimchy 5c6c4bfb5a Histogram Facet: Allow to define a key field and value script, closes #517. 2010-11-15 15:02:35 +02:00
kimchy 37335bc6a9 also retry when there are no routing shards, might happen in case of recovery from gateway before shard routing initialization 2010-11-14 20:09:00 +02:00
kimchy beb77c9b0a fix a bug in mapping serialization (introduced on 0.13), where properties gets underscore cased by mistake, and not retain their original naming 2010-11-11 15:38:20 +02:00
kimchy 17cba798e6 add double restart test for local gateway 2010-11-11 14:08:58 +02:00
kimchy 4c6af6afa5 Cluster health API should return RED status (on applicable levels) when an index has not recovered from the gateway, closes #507. 2010-11-09 21:25:22 +02:00
kimchy d0e0b85178 add many indices stress test 2010-11-09 16:22:50 +02:00
kimchy 7340d6973d slim down the mapping serialization to xcontent (json), don't write default values 2010-11-07 12:33:44 +02:00
kimchy 6804c02e97 Create specific `data` location for indices (move from work), closes #473. 2010-11-03 14:41:29 +02:00
kimchy 92b3ae3f73 Change single operation shard hashing to only use id, and not id and type, closes #472. 2010-11-03 12:47:34 +02:00
kimchy a62f1f3e0d API: Allow to control document shard routing, and search shard routing, closes #470. 2010-11-02 19:56:18 +02:00
kimchy 4ff1b429f1 Possible (rare) shard index corruption / different doc count on recovery (gateway / shard), closes #466. 2010-11-01 23:00:16 +02:00
kimchy 4579c04a9e Mapper: Ip Type Support (ipv4), auto detection with dynamic mapping, closes #461. 2010-10-28 12:02:20 +02:00
kimchy 184276154c add search stress test 2010-10-26 17:17:12 +02:00
kimchy 5966d2005d refacto package name from facets to facet (make more sense when facet will be easily pluggable) 2010-10-25 18:49:30 +02:00
kimchy ca7a7467dc Indices API: Add open and close index, closes #447. 2010-10-23 21:52:09 +02:00
kimchy a76824e395 make index not recovered a bock used in both gateways (shared/blob and local) 2010-10-23 17:03:38 +02:00
kimchy f0f62ce00c improve test to wait for 2 nodes 2010-10-23 01:24:53 +02:00
kimchy e11589f3c7 add doc 2010-10-23 01:07:37 +02:00
kimchy 9237dafef9 Query DSL: Query String _missing_ and _exists_ syntax, closes #446. 2010-10-22 17:51:19 +02:00
kimchy f63ee3158a Query DSL: Add `exists` and `missing` filters to filter documents where a field either has a value or not in them, closes #445. 2010-10-22 15:14:05 +02:00
kimchy 5d1d927e09 Write Consistency Level for index/delete/delete_by_query/bulk with one/quorum/all. Defaults to quorum, closes #444. 2010-10-22 02:51:00 +02:00
kimchy 0b593bd6d9 upgrade to guava r07 2010-10-20 12:09:19 +02:00
kimchy ae5bc20959 Reduce multiple indices overhead, don't create Indices data on nodes that don't hold any shard for an index, closes #441. 2010-10-20 02:57:49 +02:00
kimchy a4eea0aeaa change index creation / deletion logic not to wait for it to be created on other nodes since now, they might not be created... 2010-10-19 19:33:27 +02:00
kimchy 53a3df5d8e move list of local files to be indices level, so no need for index created on a node in order to list it 2010-10-19 17:36:21 +02:00
kimchy 4c8978237f move operation routing to be cluster level and not index level 2010-10-19 16:23:39 +02:00
kimchy 5d54e36964 Statistical Facet: Allow to compute statistical facets on more than one field, closes #436. 2010-10-17 16:22:20 +02:00
kimchy 1587a3815e more options on stress rolling restart test, variable number of fields, variable length of text 2010-10-16 03:26:11 +02:00
kimchy b09675dcfa add a stress test for rolling restart nodes 2010-10-16 01:05:25 +02:00
kimchy 4f9d9c1395 add test for merging two stats field facet using same facet name 2010-10-11 00:40:37 +02:00
kimchy 6314c2460c Geo Overhaul (work with multiple locations), closes #414. 2010-10-08 22:31:21 +02:00
kimchy d0bf743ab4 Query DSL: query_string / field to use the optimized match_all query when using * (or *:*), closes #413. 2010-10-07 16:48:24 +02:00
kimchy cf5ed1d177 Term Facet: Use a script that provides the facets to index, closes #410. 2010-10-07 01:49:12 +02:00
kimchy 22ed71e2e6 allow to execute term facet on more than one field 2010-10-06 13:39:02 +02:00
kimchy c9a47a126f Scripts: Allow to access `_fields` providing access to stored fields, closes #402. 2010-10-03 12:51:41 +02:00
kimchy 5b8bc333bd JavaScript Plugin: Allow to use javascript for scripts, closes #401. 2010-10-03 02:20:37 +02:00
kimchy ce28882660 add refresh option to index/create/delete opereation, REST allows for refresh parameter (defaults to false) 2010-09-26 09:07:37 +02:00
kimchy eb4f4f99b3 add recover_after gateway tests 2010-09-20 09:44:57 +02:00
kimchy 770bac252a fix local discovery to handle master nodes specifically better 2010-09-19 23:49:15 +02:00
kimchy 95bb02b330 simple specific master nodes test 2010-09-19 21:11:53 +02:00
kimchy 8def6f59b3 simple specific master nodes test 2010-09-19 18:48:48 +02:00
kimchy 20b6688b0a make ping tests run faster using cluster state api 2010-09-19 18:13:03 +02:00
kimchy f82ceb1e1e do snapshots on more lower level interval 2010-09-18 22:18:21 +02:00
kimchy 2372f481aa Shared Storage Gateway (fs/s3): Wrong snapshotting of transaction log, closes #375. 2010-09-18 21:33:38 +02:00
kimchy ff081240eb imrpove caching and fix delete mapping logic 2010-09-17 01:25:37 +02:00
kimchy 4b87f63bed Terms Facet: Allow to provide `_index` as the field, resulting in facets on indices, closes #374. 2010-09-16 15:11:30 +02:00
kimchy f49f3e169a Mapping API: Delete Mapping (with data), closes #373. 2010-09-16 14:35:07 +02:00
kimchy f6fa6ea44e more work on bulk, client API 2010-09-14 22:08:43 +02:00
kimchy 38aeba438d XContent refactoring, closes #367. 2010-09-12 23:20:15 +02:00
kimchy eccc7d5ef2 Search API: Allow to name filters and return per hit the filters it matched on, closes #364. 2010-09-11 12:38:19 +03:00
kimchy 4e661c165a Geo search across anti-meridian doesn't work, closes #363. 2010-09-10 16:04:10 +03:00
kimchy f270fc00d2 earch API: Automatically identify "script" fields on the field elements in search, close #357. 2010-09-06 11:51:31 +03:00
kimchy b40f0bc5a4 Local Gateway: Expose `gateway.local.initial_shards` to control when to recover and index, closes #352. 2010-09-03 18:38:24 +03:00
kimchy cd28afe950 Search API: Allow for script fields to extract parts of the stored _`source`, closes #349. 2010-09-02 21:29:41 +03:00
kimchy 0250896a09 reduce shard search response size by not serilaizing the node id, index, and shard id 2010-09-02 11:19:25 +03:00
kimchy 042af200e8 Gateway: Default to `local` gateway (replace `none` gateway), closes #347. 2010-09-01 23:21:26 +03:00
kimchy ba5cd19989 gateway test to make sure it recovers from the latest version 2010-08-31 21:13:42 +03:00
kimchy 289cfdf25a improve replica allocation, fix test to wait for recovery on node2 2010-08-30 23:37:09 +03:00
kimchy 61764c5b69 add another test for local gateway, fix race when loading the current state of a node 2010-08-30 19:29:00 +03:00
kimchy d9979f8dfe more work on local gateway, start integration test it 2010-08-30 01:25:45 +03:00
kimchy 5b5d42445d move the shared fs gateway base class to fs package 2010-08-29 23:22:55 +03:00
kimchy 19ddee5ec9 add a test for time interval 2010-08-26 18:12:51 +03:00
kimchy b52d854711 smarter handling of throttling when preferring allocating to an existing work location 2010-08-23 21:39:54 +03:00
kimchy 1461da5b49 move prefetch allocation to be a NodeAllocation 2010-08-23 17:22:44 +03:00
kimchy edbd586ee5 add load test for recovery from gateway 2010-08-23 16:33:33 +03:00