Commit Graph

428 Commits

Author SHA1 Message Date
kimchy 6317483fe3 add total count to terms stats and compute mean properly, also better use caching 2011-04-06 15:50:09 +03:00
kimchy 35be46df71 Bug: Sort on a column of type 'short' throws an exception, closes #835. 2011-04-06 02:25:56 +03:00
kimchy 4721f9aa69 fix cache recycling logic to make sure we don't override our own data in facets 2011-04-05 23:25:39 +03:00
kimchy 6f827731d6 normalize to/from properly in bounded histogram 2011-04-05 15:10:55 +03:00
kimchy 9b8eceb229 Histogram Facet: Add ability to define bounds (from/to) to both improve performance and provide additional bound filtering, closes #834. 2011-04-05 13:53:19 +03:00
kimchy 79939222e4 Range Facet: Fix wrong total computation with multi valued fields by introducing total_count, add min/max stats, closes #832. 2011-04-05 00:42:46 +03:00
kimchy 8d1e9dbd3c Histogram Facet: Add min/max stats when providing value field / script, closes #830. 2011-04-04 23:48:54 +03:00
kimchy 46088b9f8a Histogram Facet: Improve value field case performance, fix wrong total computation with multi valued fields by introducing total_count, closes #829. 2011-04-04 21:30:46 +03:00
kimchy 105d60ac9c Geo Distance / Range Facets might count documents several times for a range entry if the field is multi valued, closes #824. 2011-04-04 17:44:38 +03:00
kimchy 4e4495ff1d Upgrade to Lucene 3.1, closes #825. 2011-04-04 12:54:15 +03:00
kimchy e4cbdfa05b Terms Facet: Performance improvements, closes #822. 2011-04-01 16:30:45 +03:00
kimchy 2a032f8718 add remote many indices creation test 2011-03-29 17:17:54 +02:00
kimchy 508d1d40fb Indices Status API: Remove settings/aliases section, and add `recovery`/`snapshot` flags, closes #809. 2011-03-29 13:05:36 +02:00
kimchy 49f79f54d2 log when search failure happen when scanning 2011-03-28 18:17:36 +02:00
kimchy f00664621f more scan tests to try and recreate the context missing failure, no luck yet... 2011-03-28 17:21:59 +02:00
kimchy 64a89eb0e4 add the ability to control the number of shards and replicas in the restart stress test, also add a step to verify version numbers 2011-03-28 14:21:49 +02:00
kimchy d2e61af9ee maintain total hits across scan scroll requests 2011-03-20 00:19:26 +02:00
kimchy 70fc8d9af0 Percolator: When deleting an index, make sure to delete all its percolated queries from the _percolator index, closes #793. 2011-03-19 00:24:20 +02:00
kimchy 09fbc919b8 "exclude" functionality missing for terms facet on numeric fields, closes #789. 2011-03-18 11:08:11 +02:00
kimchy 923fcf239c Percolator failure when deleting and creating an index, also causes problem with percolated queries isolation between different indices, closes #790. 2011-03-18 10:49:28 +02:00
kimchy 25124b084b Query: Provide an option to analyze wildcard/prefix in query_string / field queries, closes #787. 2011-03-17 22:01:22 +02:00
kimchy 96d06d6dc8 Cluster Health API: Asking for health on an index that does not exists should return RED status, closes #781. 2011-03-15 21:22:46 +02:00
kimchy 1bcd3b67ee Search / Get: Allow to specify a preference on which shards (or order) it will be executed, closes #769. 2011-03-14 11:30:01 +02:00
kimchy 700a2a9577 Sort: Support "missing" specific handling, include _last, _first, and custom value (for numeric values), closes #772. 2011-03-13 02:29:05 +02:00
kimchy bdb0fd23e9 Versioning: Add a new version_type parameter, with a new type - `external`, closes #776. 2011-03-12 20:41:55 +02:00
kimchy faefc772a4 Searching while an index is being allocated and no active shards exists within a "shard replication group" can cause search "misses", closes #736. 2011-03-02 05:11:02 +02:00
kimchy bb1668c2b7 count date histo using the wrong counts aggregator to return results when using time based setting interval 2011-03-02 03:13:25 +02:00
kimchy 6097365738 eer Recovery: Batch translog based operation recovery based on size and not just number of operations, closes #729. 2011-03-01 00:49:41 +02:00
kimchy 3bac33e69e don't use AfterTest / BeforeTest in any tests 2011-02-28 22:28:55 +02:00
kimchy 24199a3394 add simple unicast disco test on hte node level 2011-02-28 21:43:31 +02:00
kimchy 3b967040da Percolate: Failure to percolate with specific query (when creating the _percolate index before the actual index), closes #725. 2011-02-27 01:06:50 +02:00
kimchy fdef88debb Bulk API: Improve memory usage when executing large bulk requests, closes #724. 2011-02-26 23:46:22 +02:00
kimchy a0fd47159e improve internal scan handling, by using count type for the first scan 2011-02-24 01:04:49 +02:00
kimchy ee9beda398 Search: Add search_type for `count` to return count but still support facets, closes #718. 2011-02-24 00:29:42 +02:00
kimchy f7100c0698 Improve from hits pagination (duplicates), closes #717. 2011-02-23 22:57:44 +02:00
kimchy 6b7192e744 add from/to test without sorting on single shard 2011-02-23 21:41:31 +02:00
kimchy 8371920a89 add min/max to terms stats facet 2011-02-23 20:36:12 +02:00
kimchy 58dbf6d5ca fix terms facet TOTAL comparator to also do secondary sort when totals are equal 2011-02-23 00:55:40 +02:00
kimchy 06850f8673 add another test to scan 2011-02-22 21:01:11 +02:00
kimchy 953fcbc58c Get API: Allow to provide a script as a field, closes #708. 2011-02-22 02:28:19 +02:00
kimchy 818f3b4d75 Search: Add search type `scan` allowing to efficiently scan large result set, closes #707. 2011-02-22 00:11:31 +02:00
kimchy 352cb74f96 Search: Terms Stats Facet, closes #705. 2011-02-19 23:37:04 +02:00
kimchy 585c310aa4 add edge histogram test 2011-02-18 07:35:33 +02:00
kimchy fb90392040 add status based test 2011-02-17 01:28:54 +02:00
kimchy 6ffd60dfb5 change var name to better reflect the fact that its data and not work 2011-02-17 01:07:05 +02:00
kimchy 9ae9ab9553 add flush to search 1 stress test 2011-02-10 16:47:43 +02:00
kimchy 4bcedde011 don't require to provide the type of the doc in the doc element in percolator request, also, change the REST endpoint for percolator to include the type, basically: {index}/{type}/_percolate 2011-02-08 22:24:44 +02:00
kimchy 3062d5b86c nice usage of versioning parse element 2011-02-08 17:50:28 +02:00
kimchy ac8646deb7 Search: By default, don't return the version per hit, unless `version` is set to `true`, closes #676. 2011-02-08 17:48:13 +02:00
kimchy b8c644cab5 ip mapping no longer enabled 2011-02-08 11:24:14 +02:00
kimchy 4319e3cb05 add a test to fetch just an array element (which has no mapping) 2011-02-07 20:24:15 +02:00
kimchy 904bee12bc Search: Failure when sorting on `short` type, closes #665. 2011-02-02 17:55:21 +02:00
kimchy 4ed82023ce another embedded test for query and fetch on a single shard 2011-02-02 14:04:24 +02:00
kimchy 0bce34cb5a optimize single shard search (or search result) by not doing another round of sorting 2011-02-02 13:52:34 +02:00
kimchy c86551d9b5 move master node tests to cluster package, and force it to use zen disco always 2011-02-02 11:15:12 +02:00
kimchy d2a3f5142f improve the order of shutdown of top level components in node, close indices first and applying cluster changes, also, improve atomicity of closing of indices and shards 2011-02-01 23:10:15 +02:00
kimchy 8927e1dee4 Search Scripts: Allow to access score in facet related scripts using `doc.score`, closes #663. 2011-02-01 13:41:58 +02:00
kimchy cc6f65f8b8 Search: When sorting, allow to pass `track_scores` and set it to `true` to get scores/max_score back, closes #662. 2011-02-01 12:38:46 +02:00
kimchy f8b5628db9 have rolling restart stress test create incremental ids 2011-01-31 21:41:46 +02:00
kimchy cebdae5707 refactor script execution handling to improve performance, also introduce specific search script executor 2011-01-31 16:08:06 +02:00
kimchy 5d39cc93dd refresh for test 2011-01-31 15:51:39 +02:00
kimchy e516051ea5 Search: When fetching _parent, only the _id should be returned, and not type#id, closes #658. 2011-01-30 21:16:45 +02:00
kimchy 5b4846b0b6 Translog Flushing: Improve logic, flush not just by operations in the translog, closes #656. 2011-01-30 17:41:22 +02:00
kimchy 94c632b79b Java Client - Allow to set facets in binary format, closes #442. 2011-01-30 15:53:10 +02:00
kimchy 0b09fd0806 Facets: Remove redundant data returned as part of facet response (for example, the field name being faceted), closes #655. 2011-01-29 20:44:58 +02:00
kimchy 332c4cfa53 use bloom filter to reduce seeks when needing to load version 2011-01-29 03:55:00 +02:00
kimchy 00fa7c95fb improve test time 2011-01-26 23:04:10 +02:00
kimchy 5ecb847a3a improve test time 2011-01-26 22:59:35 +02:00
kimchy 6c21c30f31 Index Merge: Improve internal (segment) merging by not initiating it on doc operations, closes #653. 2011-01-26 22:39:40 +02:00
kimchy 5a4686aee5 Search: Allow to pass a search filter, applying only on the query (and not on facets for example), closes #650. 2011-01-26 12:55:30 +02:00
kimchy 00aad4c123 rename tests 2011-01-25 14:40:04 +02:00
kimchy 95dc08fa3b refactor search tests into separate packages 2011-01-24 16:06:28 +02:00
kimchy 95b6184135 Terms Facets: Allow to get `all_terms` back (possibly with count 0), closes #647. 2011-01-23 16:09:18 +02:00
kimchy ce4f09c2b1 Strict dynamic setting: Refuse to index a document with fields not present in the mapping definition, closes #643. 2011-01-22 01:08:41 +02:00
kimchy b1d13febbf Percolate on Index and Bulk, closes #636. 2011-01-19 16:45:45 +02:00
kimchy 574c455203 Terms Facet: Add option include counts where term is missing, closes #632. 2011-01-18 21:51:16 +02:00
kimchy a4c2087511 initial not recovered block on node init, and remove it as usual (once node is initialized / recovered) 2011-01-18 14:08:45 +02:00
kimchy b4e5a542f3 internal refactoring/simplification in percolator 2011-01-17 13:57:32 +02:00
kimchy 38d10d19bc Add `prefer_local` flag to analyze and percolate request, closes #625. 2011-01-14 01:01:37 +02:00
kimchy 180d225016 Percolator, closes #624. 2011-01-13 16:20:31 +02:00
kimchy 85b6a982d4 A setting to auto expand the number of replicas of an index (based on data nodes), closes #623. 2011-01-12 16:27:36 +02:00
kimchy ba9a12e201 Mapper: Add `byte` type (8bit signed), closes #620. 2011-01-11 16:01:10 +02:00
kimchy d4246da7b3 /_open is not recovering indices correctly, closes #615. 2011-01-09 23:31:10 +02:00
kimchy fe0f9ebc9d make lucene document creation allowed to be indexed twice 2011-01-07 14:09:39 +02:00
kimchy b9e808f755 improve version loading by going directly to the relevant reader 2011-01-06 11:50:16 +02:00
kimchy 31231531e1 Highlighting: The result of highlighting for a hit can contain data from another document, closes #600. 2011-01-05 15:45:46 +02:00
kimchy 0faa05b3f2 add more integration versioning tests 2011-01-04 18:09:46 +02:00
kimchy 9423378f27 add heavy concurrent updates to same doc, and make sure it has the same data on all replicas 2011-01-04 17:44:33 +02:00
kimchy 07d361816e Search: Date Histogram Facet, closes #591. 2011-01-02 15:07:45 +02:00
kimchy aec720218d Histogram Facet: Improve perf / memory by defaulting to just providing counts with no totals, closes #587. 2011-01-01 17:27:55 +02:00
kimchy e6b4834768 Highlighting: Allow to highlight on fields without term vectors, closes #585. 2010-12-31 18:57:26 +02:00
kimchy 82298d890c Terms Facet: Performance and memory improvements when faceting numeric fields, closes #583. 2010-12-30 19:45:36 +02:00
kimchy 8e8a3a894f upgrade testing framework jars, improve build scripts to share the testing jars 2010-12-30 16:08:38 +02:00
kimchy 76d042f3c5 explicitly pass null values to methods where applicable 2010-12-30 13:09:11 +02:00
kimchy ed996c3e85 Non-data master nodes and non-master data nodes fail to store data, closes #579. 2010-12-30 12:10:40 +02:00
kimchy f993c4b72b start refactoring of facets to make them more pluggable 2010-12-29 12:49:02 +02:00
kimchy efc95dc00a Java Search API: Passing a the query as json fails, closes #568. 2010-12-20 11:31:09 +02:00
kimchy 415bb5d7f3 Java TransportClient: By default, don't sniff other nodes and use addresses provided as is, closes #565. 2010-12-17 19:31:28 +02:00
kimchy 1c5477d4ed Improve local node recovery process, removing using a block as indication for an index not being recovered. Moving to shard replication group level allocation (with the ability to distinguish between api based creation or recovery based creation). This also allows for throttling based control on initial local gateway based shard allocation. 2010-12-12 22:46:27 +02:00
kimchy bc04243a2b Search Fields: If a field is not stored, automatically extract it from _source (without the need for _source prefix), closes #562. 2010-12-12 05:00:24 +02:00
kimchy 216b2ab912 Highlighting: Automatically use the field values extracted from _source if not stored explicitly in the mapping, closes #561. 2010-12-12 04:40:25 +02:00
kimchy a914865c45 Close API might cause index data to be wiped, closes #560. 2010-12-10 22:19:45 +02:00
kimchy 2a7f964ca2 Geo search fails to find some docs when wrapping over IDL, closes #559. 2010-12-10 08:39:10 +02:00
kimchy e1495ff8be add scrolling update test 2010-12-08 20:56:09 +02:00
kimchy 54437c1bd3 Parent / Child Support, closes #553. 2010-12-08 00:16:05 +02:00
kimchy d3cae103d5 wait for green status in test 2010-12-04 23:57:44 +02:00
kimchy 84f97e96a5 Scrolling issues, fix releasing search context eagerly, they should not be released when scrolling, closes #136. 2010-11-29 02:05:45 +02:00
kimchy 577f06fd43 add indices API to groovy client 2010-11-26 22:19:57 +02:00
kimchy 73e5eb9e14 clean compile warnings 2010-11-26 16:03:25 +02:00
kimchy ebd6316db9 Index templates, closes #540. 2010-11-26 15:45:18 +02:00
kimchy b4113d57d3 Analyze API: An API to analyzer custom text based on an optional analyzer, closes #529. 2010-11-21 17:33:35 +02:00
kimchy 7db5e63ab7 rename shardsIt to shardIt 2010-11-21 13:54:44 +02:00
kimchy fdb98b1dcb Highlighting bug where fragments are broken when number_of_fragments is 0, closes #479. 2010-11-18 20:52:51 +02:00
kimchy 44775c2aa8 Routing: Allow to define `path` on the `_routing` mapping, to automatically extract the routing from it, closes #524. 2010-11-17 17:05:51 +02:00
kimchy 0172eaea89 add broadcasting when no routing and its marked as required to bulk delete as well 2010-11-16 18:53:32 +02:00
kimchy a04d8ec5ad Routing: When specify in the mapping _routing required, a delete without explicit routing value should automatically be broadcasted to all shards, closes #522. 2010-11-16 18:31:08 +02:00
kimchy 02981f6101 Routing: Allow to specify on the `_routing` mapping that its required, and fail index operations that do not provide one, closes #520. 2010-11-16 15:10:41 +02:00
kimchy 5c6c4bfb5a Histogram Facet: Allow to define a key field and value script, closes #517. 2010-11-15 15:02:35 +02:00
kimchy 37335bc6a9 also retry when there are no routing shards, might happen in case of recovery from gateway before shard routing initialization 2010-11-14 20:09:00 +02:00
kimchy beb77c9b0a fix a bug in mapping serialization (introduced on 0.13), where properties gets underscore cased by mistake, and not retain their original naming 2010-11-11 15:38:20 +02:00
kimchy 17cba798e6 add double restart test for local gateway 2010-11-11 14:08:58 +02:00
kimchy 4c6af6afa5 Cluster health API should return RED status (on applicable levels) when an index has not recovered from the gateway, closes #507. 2010-11-09 21:25:22 +02:00
kimchy d0e0b85178 add many indices stress test 2010-11-09 16:22:50 +02:00
kimchy 7340d6973d slim down the mapping serialization to xcontent (json), don't write default values 2010-11-07 12:33:44 +02:00
kimchy 6804c02e97 Create specific `data` location for indices (move from work), closes #473. 2010-11-03 14:41:29 +02:00
kimchy 92b3ae3f73 Change single operation shard hashing to only use id, and not id and type, closes #472. 2010-11-03 12:47:34 +02:00
kimchy a62f1f3e0d API: Allow to control document shard routing, and search shard routing, closes #470. 2010-11-02 19:56:18 +02:00
kimchy 4ff1b429f1 Possible (rare) shard index corruption / different doc count on recovery (gateway / shard), closes #466. 2010-11-01 23:00:16 +02:00
kimchy 4579c04a9e Mapper: Ip Type Support (ipv4), auto detection with dynamic mapping, closes #461. 2010-10-28 12:02:20 +02:00
kimchy 184276154c add search stress test 2010-10-26 17:17:12 +02:00
kimchy 5966d2005d refacto package name from facets to facet (make more sense when facet will be easily pluggable) 2010-10-25 18:49:30 +02:00
kimchy ca7a7467dc Indices API: Add open and close index, closes #447. 2010-10-23 21:52:09 +02:00
kimchy a76824e395 make index not recovered a bock used in both gateways (shared/blob and local) 2010-10-23 17:03:38 +02:00
kimchy f0f62ce00c improve test to wait for 2 nodes 2010-10-23 01:24:53 +02:00
kimchy e11589f3c7 add doc 2010-10-23 01:07:37 +02:00
kimchy 9237dafef9 Query DSL: Query String _missing_ and _exists_ syntax, closes #446. 2010-10-22 17:51:19 +02:00
kimchy f63ee3158a Query DSL: Add `exists` and `missing` filters to filter documents where a field either has a value or not in them, closes #445. 2010-10-22 15:14:05 +02:00
kimchy 5d1d927e09 Write Consistency Level for index/delete/delete_by_query/bulk with one/quorum/all. Defaults to quorum, closes #444. 2010-10-22 02:51:00 +02:00
kimchy 0b593bd6d9 upgrade to guava r07 2010-10-20 12:09:19 +02:00
kimchy ae5bc20959 Reduce multiple indices overhead, don't create Indices data on nodes that don't hold any shard for an index, closes #441. 2010-10-20 02:57:49 +02:00
kimchy a4eea0aeaa change index creation / deletion logic not to wait for it to be created on other nodes since now, they might not be created... 2010-10-19 19:33:27 +02:00
kimchy 53a3df5d8e move list of local files to be indices level, so no need for index created on a node in order to list it 2010-10-19 17:36:21 +02:00
kimchy 4c8978237f move operation routing to be cluster level and not index level 2010-10-19 16:23:39 +02:00
kimchy 5d54e36964 Statistical Facet: Allow to compute statistical facets on more than one field, closes #436. 2010-10-17 16:22:20 +02:00
kimchy 1587a3815e more options on stress rolling restart test, variable number of fields, variable length of text 2010-10-16 03:26:11 +02:00
kimchy b09675dcfa add a stress test for rolling restart nodes 2010-10-16 01:05:25 +02:00
kimchy 4f9d9c1395 add test for merging two stats field facet using same facet name 2010-10-11 00:40:37 +02:00
kimchy 6314c2460c Geo Overhaul (work with multiple locations), closes #414. 2010-10-08 22:31:21 +02:00
kimchy d0bf743ab4 Query DSL: query_string / field to use the optimized match_all query when using * (or *:*), closes #413. 2010-10-07 16:48:24 +02:00
kimchy cf5ed1d177 Term Facet: Use a script that provides the facets to index, closes #410. 2010-10-07 01:49:12 +02:00