Commit Graph

1714 Commits

Author SHA1 Message Date
kimchy 8587f16119 make sure to call releaseExternalResources on another thread to make sure its not called from a netty IO handler 2011-02-01 22:16:09 +02:00
kimchy 237e936884 if action is performing the operation, return true to indicate it 2011-02-01 17:00:59 +02:00
kimchy 8927e1dee4 Search Scripts: Allow to access score in facet related scripts using `doc.score`, closes #663. 2011-02-01 13:41:58 +02:00
kimchy cc6f65f8b8 Search: When sorting, allow to pass `track_scores` and set it to `true` to get scores/max_score back, closes #662. 2011-02-01 12:38:46 +02:00
kimchy 5da14a7ed1 Geo: All geo components that accept an array of [lat, lon] to change to do [lon, lat] to conform with GeoJSON, closes #661. 2011-02-01 01:39:01 +02:00
kimchy f8b5628db9 have rolling restart stress test create incremental ids 2011-01-31 21:41:46 +02:00
kimchy c7add5f377 upgrade to mvel 2.1 snapshot to gain performance improvements 2011-01-31 21:41:23 +02:00
kimchy 54c770bcc9 improve hash wheel timer 2011-01-31 16:26:17 +02:00
kimchy cebdae5707 refactor script execution handling to improve performance, also introduce specific search script executor 2011-01-31 16:08:06 +02:00
kimchy 5d39cc93dd refresh for test 2011-01-31 15:51:39 +02:00
kimchy 04c2a2a543 move ngram to the default processor 2011-01-31 11:52:02 +02:00
kimchy e85f9c276f only allow _score to be used, and remove score from custom_score scripts 2011-01-31 02:19:41 +02:00
kimchy bdb87c7a62 refactor a bit search script execution 2011-01-31 01:29:30 +02:00
kimchy 919143f6b6 don't break if no document mapper exists 2011-01-31 00:00:24 +02:00
kimchy e516051ea5 Search: When fetching _parent, only the _id should be returned, and not type#id, closes #658. 2011-01-30 21:16:45 +02:00
kimchy 8b9ec890c3 Thread Pool: Increase the keep alive time of threads from 60 seconds to 60 minutes, closes #657. 2011-01-30 20:34:34 +02:00
kimchy 5b4846b0b6 Translog Flushing: Improve logic, flush not just by operations in the translog, closes #656. 2011-01-30 17:41:22 +02:00
kimchy 8884b575c1 more doc 2011-01-30 16:30:17 +02:00
kimchy 94c632b79b Java Client - Allow to set facets in binary format, closes #442. 2011-01-30 15:53:10 +02:00
kimchy e5db3175fb add explicit internal methods for getting doc fields 2011-01-30 00:34:50 +02:00
kimchy ad4bb464e8 upgraded to new trove, major upgrade, better perf, but damn, 2mb larger in size... 2011-01-30 00:05:56 +02:00
kimchy 0b09fd0806 Facets: Remove redundant data returned as part of facet response (for example, the field name being faceted), closes #655. 2011-01-29 20:44:58 +02:00
kimchy 7fa5b0cb57 only add highlighted fragments that have a score > 0 (when ending up using the default highlighter) 2011-01-29 15:29:25 +02:00
kimchy 2d94087f90 explicitly set optimize parameters when doing async merge 2011-01-29 04:11:56 +02:00
kimchy 4a27671d0c moe internal stats on translog 2011-01-29 04:04:58 +02:00
kimchy 332c4cfa53 use bloom filter to reduce seeks when needing to load version 2011-01-29 03:55:00 +02:00
kimchy ead4d600be add a test for term docs version deletes 2011-01-27 00:00:29 +02:00
kimchy 30f54fe23e change default async interval to 1 second, to do merges (if needed) more often 2011-01-26 23:29:24 +02:00
kimchy 00fa7c95fb improve test time 2011-01-26 23:04:10 +02:00
kimchy 5ecb847a3a improve test time 2011-01-26 22:59:35 +02:00
kimchy 24d5a758f0 fix thread name 2011-01-26 22:43:02 +02:00
kimchy c7e73e4558 tookInMillis is not serialized over network for bulk response 2011-01-26 22:41:41 +02:00
kimchy 6c21c30f31 Index Merge: Improve internal (segment) merging by not initiating it on doc operations, closes #653. 2011-01-26 22:39:40 +02:00
kimchy e1d56ea467 call maybeMerge outside of writeLock to reduce work done under writeLock 2011-01-26 17:16:32 +02:00
kimchy 5a4686aee5 Search: Allow to pass a search filter, applying only on the query (and not on facets for example), closes #650. 2011-01-26 12:55:30 +02:00
kimchy 10ff150bb8 don't log index missing exception when trying to optimize single shard search request 2011-01-25 18:29:28 +02:00
kimchy 00aad4c123 rename tests 2011-01-25 14:40:04 +02:00
kimchy a5e02d559f optimize single index conversion from alias to concrete index 2011-01-24 21:25:20 +02:00
kimchy 95dc08fa3b refactor search tests into separate packages 2011-01-24 16:06:28 +02:00
kimchy 256f1d3611 add a test for collector ordering 2011-01-24 13:58:09 +02:00
kimchy 9735d3e6d7 don't establish a connection between two client nodes 2011-01-24 13:34:43 +02:00
kimchy 9d75849362 better shutdown of sockets 2011-01-24 13:15:57 +02:00
kimchy 3537de4530 don't have the constructor private 2011-01-23 21:07:04 +02:00
kimchy 95b6184135 Terms Facets: Allow to get `all_terms` back (possibly with count 0), closes #647. 2011-01-23 16:09:18 +02:00
kimchy 85ee6eaf10 add logging on which state file failed to load 2011-01-23 02:59:42 +02:00
kimchy ce4f09c2b1 Strict dynamic setting: Refuse to index a document with fields not present in the mapping definition, closes #643. 2011-01-22 01:08:41 +02:00
kimchy 3907c8c680 add a test for custom type class name in token filter factory 2011-01-21 02:07:02 +02:00
kimchy 9801ddeb0d Analysis: Add phonetic token filter (metaphone, soundex, ...), closes #644. 2011-01-21 01:57:16 +02:00
kimchy 5316cd1931 should be debug logging, not warn 2011-01-20 17:38:24 +02:00
kimchy 545d8f35df No cluster name filtering when sniffing is not enabled, closes #637. 2011-01-20 17:28:45 +02:00
kimchy 9ff8006d37 have compilation and javadoc use UTF8 encoding 2011-01-20 14:24:12 +02:00
kimchy 034a66263a Analysis: Add stemming to czech analyzer, closes #642. 2011-01-20 13:30:48 +02:00
Lukáš Vlček e0fa15a365 added Czech Stemmer, closes #635 2011-01-20 12:34:51 +02:00
kimchy 04f8b55686 ES hangs connection when parent is not specified in bulk index, closes #641. 2011-01-20 12:28:55 +02:00
kimchy b4467c7d1f use caching scorer when there are multiple collectors, so if they call score as well, it will not be computed again 2011-01-19 22:59:28 +02:00
kimchy 7359f7983f handle rest HEAD on / path 2011-01-19 21:56:53 +02:00
kimchy b1d13febbf Percolate on Index and Bulk, closes #636. 2011-01-19 16:45:45 +02:00
kimchy 2eb8278a80 better logging when not allocating a shard due to required number of existing shards 2011-01-19 00:30:56 +02:00
kimchy 574c455203 Terms Facet: Add option include counts where term is missing, closes #632. 2011-01-18 21:51:16 +02:00
kimchy e4a6e99f69 improve handling when not to persist current state to gateway based on blocks and have a block indicate if it also blocks state persistence 2011-01-18 15:28:55 +02:00
kimchy 87d5a92edb move handling of recovered metadata to GatewayService, reducing code duplication in different gateway implementations 2011-01-18 14:38:10 +02:00
kimchy a4c2087511 initial not recovered block on node init, and remove it as usual (once node is initialized / recovered) 2011-01-18 14:08:45 +02:00
kimchy 4750ccdc1c move nullable annotation to the common package, and move javax inject (from guice 3) to internal common as well 2011-01-17 19:45:37 +02:00
kimchy ba725f8986 fix a (rare) case where search will fail due to failed serialization 2011-01-17 14:37:36 +02:00
kimchy b4e5a542f3 internal refactoring/simplification in percolator 2011-01-17 13:57:32 +02:00
kimchy f2bd2c7bbd protect against a case when string field data value is null 2011-01-17 12:39:11 +02:00
kimchy 89d31bbfde properly print debug statement when nodes based operations fail 2011-01-17 12:38:08 +02:00
kimchy f55fa4223e improve loading queries by adding all of them in batch to the executor 2011-01-17 11:11:13 +02:00
kimchy 20c77a2ac5 clear the percolator reader from the cache explicitly 2011-01-17 10:11:14 +02:00
kimchy 1f289df128 more append benchmark 2011-01-16 12:47:54 +02:00
kimchy 285afe8053 add refresh interval to os/process/network info api which represents the refresh interval of their respective stats 2011-01-16 12:33:23 +02:00
kimchy c0e2e14c36 fix javadoc 2011-01-15 02:15:36 +02:00
kimchy 998f5f4188 add lt(Object) to range filter builder 2011-01-15 02:15:29 +02:00
kimchy 771e028e54 simple fs bench test 2011-01-15 02:14:10 +02:00
kimchy 2381f668f8 wait for mappings to be created on other nodes before returning from the put mapping request 2011-01-15 02:13:57 +02:00
kimchy f8b1d1eebd add explicit type missing exception when trying to search on a type that is not registered 2011-01-14 17:53:45 +02:00
kimchy a8be99b7e0 Node Stats: Process / Network / Os should only refresh every 5 seconds (refresh_interval to set it), closes #626. 2011-01-14 16:00:55 +02:00
kimchy 38d10d19bc Add `prefer_local` flag to analyze and percolate request, closes #625. 2011-01-14 01:01:37 +02:00
kimchy ff347858c5 properly report teh files and size recovered when doing primary local gateway recovery 2011-01-13 23:59:35 +02:00
kimchy 7fc04a4ab6 make sure to close the term docs when loading version 2011-01-13 22:49:56 +02:00
kimchy ad236d19ee add logging if failing to execute on a node 2011-01-13 16:51:06 +02:00
kimchy 180d225016 Percolator, closes #624. 2011-01-13 16:20:31 +02:00
kimchy 2d180eb28a upgrade to sigar 1.6.4 2011-01-13 16:02:35 +02:00
kimchy 577135db97 upgrade jackson to 1.7.1 2011-01-13 15:50:58 +02:00
kimchy 7213539d72 factor out exists collector 2011-01-12 20:51:19 +02:00
kimchy f8b1a8e38b add internal event when an index shard is started 2011-01-12 20:47:35 +02:00
kimchy 5e27e3fca4 support reader as well 2011-01-12 17:11:23 +02:00
kimchy 85b6a982d4 A setting to auto expand the number of replicas of an index (based on data nodes), closes #623. 2011-01-12 16:27:36 +02:00
kimchy f5a9f2d948 If the cluster does not contain any indices then index templates do not survive cluster restart, closes #617. 2011-01-11 23:38:46 +02:00
kimchy a0a714e6a5 Shared Gateway: Allow to set the number of concurrent streams doing snapshot operations, closes #621. 2011-01-11 22:03:44 +02:00
kimchy 93dec72e43 optimize a filtered query with match_all query into a constant score that just wraps the filter 2011-01-11 19:17:17 +02:00
kimchy 6130ac2ee7 optimize query facet in case the query is a filtered query with match_all query to use just the filter 2011-01-11 18:49:05 +02:00
kimchy ba9a12e201 Mapper: Add `byte` type (8bit signed), closes #620. 2011-01-11 16:01:10 +02:00
kimchy 38ea07cfbe fix field data cache setting: used under index.cache.field.type, available values: resident, soft, weak 2011-01-11 06:33:24 +02:00
kimchy a5432ba7ce limit the number of addresses to try and connect to when doing unicast discovery with just the host and not the port to 5 2011-01-11 00:53:53 +02:00
kimchy 90d005a330 add trace logging to unicast discovery 2011-01-10 23:30:46 +02:00
kimchy 50cdf2920f Make ToXContent return a builder 2011-01-10 22:11:12 +02:00
kimchy 89314f29a7 improve multi value field cache handling both in terms of memory usage and GC behavior 2011-01-10 20:58:31 +02:00
kimchy 233ed1f8c6 improve multi value field cache handling both in terms of memory usage and GC behavior 2011-01-10 20:18:12 +02:00
kimchy cb8ceb1a39 upgrade to guice 3.0 rc2 2011-01-10 17:49:39 +02:00
kimchy d4246da7b3 /_open is not recovering indices correctly, closes #615. 2011-01-09 23:31:10 +02:00
kimchy d650705a03 refresh param to bulk not working, closes #614. 2011-01-09 22:06:51 +02:00
kimchy 373af6b1e0 use fs channel to write to file to improve speed of adding to translog 2011-01-09 20:53:40 +02:00
kimchy 5c08271e74 improve numeric stream analysis performance 2011-01-09 19:35:24 +02:00
kimchy 3c3d01347e improve all stream analysis performance 2011-01-09 02:51:41 +02:00
kimchy 8de7beadc8 Highlighting: Sometimes highlighting returns empty fragments even thought there should be, closes #613. 2011-01-08 03:19:49 +02:00
kimchy 7364159d69 NPE for when passing null date to content builder, closes #610. 2011-01-07 23:28:57 +02:00
kimchy 286a6fac76 upgrade to jackson 1.7, also fix (really unused case) of getting compressed smile stored through REST with smile context type 2011-01-07 16:35:52 +02:00
kimchy b9be6d9ea7 Geo-distance in scripts (like custom_score), closes #607. 2011-01-07 15:22:11 +02:00
kimchy 8b8bad7572 build on aferreira improvements to handle empty parameters with no =, and improve component decoding logic 2011-01-07 14:34:34 +02:00
Adriano Ferreira 2d81acbbf2 Adding test for edge cases of RestUtils.decodeQueryString()
When writing tests for the fix of decodeQueryString() to
handle gracefully edge cases like:

    &a
    &a&b

it arises the question of what behavior is desirable
for these pathological cases (vs the regular p=v pairs).
This change just skips them which is consonant
to the preexisting code.

To be thorough, we add tests for a bunch of edge cases, like:

    QUERY           PARSED PARAMS

    ?               {}
    ?&              {}
    ?=              { "": "" }
    ?a              {}
    ?p=v&a          { "p": "v" }
    ?p=v&a&p1=v1    { "p": "v", "p1": "v1" }
    ?a&b            {}
2011-01-07 14:13:26 +02:00
Adriano Ferreira 154dcf007e Make RestUtils.decodeQueryString() more robust in edge cases
The code of decodeQueryString() had some trouble with weird URLs:

(1) an input like "uri?param&p=v" causes an exception to be thrown
(2) an input like "uri?param1&param2" causes an infinite loop

This could be verified against an ES server with requests like

    curl -XGET localhost:9200/test/_analyze?t&text=this+is+a+test
    # the exception stack trace shows up in logs

    curl -XGET localhost:9200/test/_analyze?t1&t2&text=this+is+a+test
    # never returns, never ends

This change fixes these issues.
2011-01-07 14:13:25 +02:00
Adriano Ferreira edb075b611 Minor changes to RestUtils.decodeQueryString() tests
Testing

    RestUtils.decodeQueryString("something", "something".indexOf('?') + 1, params);

is not really checking decoding of an empty query.
Instead, it is testing decoding of "something"
as a query (because "something.index('?')+1" evaluates to 0).
The parameter map is left empty because
"malformed" pairs like "something" are currently skipped.

Instead, this change modify this test to check the edge cases:

+ fromIndex >= queryString.length()
+ fromIndex < 0
2011-01-07 14:13:25 +02:00
kimchy fe0f9ebc9d make lucene document creation allowed to be indexed twice 2011-01-07 14:09:39 +02:00
kimchy c7cb353956 fix serialization of count request 2011-01-06 20:44:36 +02:00
kimchy 6a0a9ff374 parse query with offset 2011-01-06 16:13:15 +02:00
kimchy b9e808f755 improve version loading by going directly to the relevant reader 2011-01-06 11:50:16 +02:00
kimchy 26c5f6c482 Allow ./bin/plugin to install plugins from the local filesystem / custom url, closes #595. 2011-01-06 10:46:42 +02:00
kimchy cc5978393b with the new stemmer analyzer, don't break when there is no language stopwords, but just create it without stopwords 2011-01-06 09:53:56 +02:00
harryf 4a45df88c6 Add Snowball stemmer as analyzer and filter 2011-01-06 09:40:49 +02:00
kimchy 66d63055df Add to node stats the number of times field cache was evicted due to memory constraints, closes #603. 2011-01-05 22:37:15 +02:00
kimchy 754b0d7a0f Add `resident` field cache type, closes #602. 2011-01-05 21:23:47 +02:00
kimchy 0954df0623 change the default interval for gc checks to 1 second 2011-01-05 20:40:52 +02:00
kimchy 31231531e1 Highlighting: The result of highlighting for a hit can contain data from another document, closes #600. 2011-01-05 15:45:46 +02:00
kimchy 6258915205 Bulk API: Add how long the bulk API took (in milliseconds) to the response, closes #599. 2011-01-05 14:12:45 +02:00
kimchy 29b308005a try to be more predictable when doing pagination (still can return duplicates when going to different shards) by sorting on the shards results based on the shard id first 2011-01-04 23:42:44 +02:00
kimchy 0faa05b3f2 add more integration versioning tests 2011-01-04 18:09:46 +02:00
kimchy 9423378f27 add heavy concurrent updates to same doc, and make sure it has the same data on all replicas 2011-01-04 17:44:33 +02:00
kimchy 45c1ab06b3 Versioning, closes #594. 2011-01-04 04:04:30 +02:00
kimchy 9335b3a9e6 Search: Add a `timed_out` element indicating if the search request timed out, closes #592. 2011-01-03 15:25:24 +02:00
kimchy ff6d7254fe optimize query parsing to use single field parser when executing on single field 2011-01-03 15:14:28 +02:00
kimchy 2c8c6aa22a add a uid scan benchmark to check perf 2011-01-03 14:34:38 +02:00
kimchy dc17d10b81 small refactor 2011-01-03 13:32:09 +02:00
kimchy 5e029865a8 Des.config=/path/to/config/file doesn't replace $ES_HOME/elasticsearch.conf, just appends to it, closes #588. 2011-01-03 12:53:50 +02:00
kimchy ccdbbef276 don't send leave request when the master node is null 2011-01-03 12:01:33 +02:00
kimchy d2a70b21cd key (which is time) can be negative as well 2011-01-02 22:03:52 +02:00
kimchy 07d361816e Search: Date Histogram Facet, closes #591. 2011-01-02 15:07:45 +02:00
kimchy d523104b91 add some sanity tests with joda for date time manipulation 2011-01-01 23:48:25 +02:00
kimchy b7b775228e add some sanity tests with joda for date time manipulation 2011-01-01 23:40:46 +02:00
kimchy aec720218d Histogram Facet: Improve perf / memory by defaulting to just providing counts with no totals, closes #587. 2011-01-01 17:27:55 +02:00
kimchy 45b93e2781 add histogram facet benchmark 2011-01-01 16:33:18 +02:00
kimchy e6b4834768 Highlighting: Allow to highlight on fields without term vectors, closes #585. 2010-12-31 18:57:26 +02:00
kimchy 7c959e7ec3 Master based operations (create index, delete index) to automatically retry on retryable cluster blocks (like recovery from gateway) 2010-12-31 14:09:45 +02:00
kimchy a92dbc537a those two should not be final... 2010-12-31 00:23:04 +02:00
Tim Dysinger fd593acafe make every static field final (immutable) 2010-12-31 00:22:31 +02:00
Tim Dysinger 6b672e29f5 Remove infinite loop typo 2010-12-31 00:01:45 +02:00
kimchy 82298d890c Terms Facet: Performance and memory improvements when faceting numeric fields, closes #583. 2010-12-30 19:45:36 +02:00
kimchy 019359a896 add more scanning options on the field data cache 2010-12-30 17:08:32 +02:00
kimchy 6c8b06c967 clean unused method 2010-12-30 16:45:12 +02:00
kimchy 8e8a3a894f upgrade testing framework jars, improve build scripts to share the testing jars 2010-12-30 16:08:38 +02:00