Commit Graph

1597 Commits

Author SHA1 Message Date
kimchy 16ed69daa3 add routing to delete by query rest endpoint 2010-11-18 13:46:45 +02:00
kimchy cf4f6decc4 add a test for overloaded method of builder and List 2010-11-18 02:25:22 +02:00
kimchy 282a7a5162 Add non inspection checks 2010-11-18 02:05:32 +02:00
kimchy 44775c2aa8 Routing: Allow to define `path` on the `_routing` mapping, to automatically extract the routing from it, closes #524. 2010-11-17 17:05:51 +02:00
kimchy 4e75f3de4a republish the cluster state if a node exists in the cluster 2010-11-17 11:33:33 +02:00
kimchy 0172eaea89 add broadcasting when no routing and its marked as required to bulk delete as well 2010-11-16 18:53:32 +02:00
kimchy 619efea940 Routing: When specify in the mapping _routing required, a delete without explicit routing value should automatically be broadcasted to all shards, closes #522. 2010-11-16 18:31:43 +02:00
kimchy a04d8ec5ad Routing: When specify in the mapping _routing required, a delete without explicit routing value should automatically be broadcasted to all shards, closes #522. 2010-11-16 18:31:08 +02:00
kimchy 2483a37db5 fix comparator implementation in histogram facet 2010-11-16 15:30:27 +02:00
kimchy e7fdea8858 fix underscore casing to transform Name to name, and not _name 2010-11-16 15:16:54 +02:00
kimchy 02981f6101 Routing: Allow to specify on the `_routing` mapping that its required, and fail index operations that do not provide one, closes #520. 2010-11-16 15:10:41 +02:00
kimchy 8a8a6d5547 Mapping: Rename _attributes to _meta, closes #518. 2010-11-16 01:34:01 +02:00
kimchy b8b4cbbb46 add mapping metadata to cluster state metadata, will allow in the future to add specific mappings hints (since we don't have the parsed mapping on each node anymore) 2010-11-15 21:21:18 +02:00
kimchy 51c18cd9c1 refactor ip field mapper into its own package 2010-11-15 19:21:23 +02:00
kimchy 36183af22c automatically use match_all query when executing a _all:* query string 2010-11-15 17:15:24 +02:00
kimchy effdd52586 sync changes done on the indices cluster service with changes happening in an async manner during recovery 2010-11-15 16:54:29 +02:00
kimchy 4b06eeb75a better desc 2010-11-15 16:39:16 +02:00
kimchy a342029d36 Histogram Facet: Allow to define a key field and value script, closes #517. 2010-11-15 15:05:50 +02:00
kimchy 5c6c4bfb5a Histogram Facet: Allow to define a key field and value script, closes #517. 2010-11-15 15:02:35 +02:00
kimchy b1db5c43d6 Mapper: `date` type to also allow providing values in numeric (milliseconds since epoch), closes #516. 2010-11-15 14:12:54 +02:00
kimchy 6d214d69b9 Mapper: Store the routing (if provided) under a `_routing` field, closes #515. 2010-11-15 11:34:56 +02:00
kimchy 8ee038574d add _routing to documetn indexed when providing routing value 2010-11-14 21:23:52 +02:00
kimchy 9505cb14f1 bring back the retry on shard missing 2010-11-14 21:02:46 +02:00
kimchy fa1d8c7bcd only remove the shard (not clean it) when ignoring recovery 2010-11-14 20:09:27 +02:00
kimchy 37335bc6a9 also retry when there are no routing shards, might happen in case of recovery from gateway before shard routing initialization 2010-11-14 20:09:00 +02:00
kimchy d0ac9087b6 add bootstrap.mlockall setting to control if mlockall should be called or not 2010-11-14 15:54:40 +02:00
kimchy 30929c3d01 better logging, include index and shard id in log statement 2010-11-14 00:24:51 +02:00
kimchy 31986c4486 add a check to not allow to create an alias with a name of an existing index 2010-11-13 18:53:12 +02:00
kimchy 73bf371912 fix exception logging on failed index creation 2010-11-13 18:08:11 +02:00
kimchy 38d77f8cf3 Query DSL: Allow to provide pattern field names when using query_string query, closes #511. 2010-11-13 02:26:14 +02:00
kimchy 6839cc6965 clear memory after parsing a doc, and not on parsing of a new doc 2010-11-12 15:36:23 +02:00
kimchy d994a862dd add support for analyzer mapping when the mapping is not indexd nor stored 2010-11-12 14:54:28 +02:00
kimchy bba98cf1b7 cleanup traversing unneeded mappings 2010-11-12 14:50:05 +02:00
kimchy beb77c9b0a fix a bug in mapping serialization (introduced on 0.13), where properties gets underscore cased by mistake, and not retain their original naming 2010-11-11 15:38:20 +02:00
kimchy 70a0e110d4 apping: add threshold to _source field compression, closes #506. 2010-11-10 21:03:54 +02:00
kimchy a2e674df49 add filteredQuery to QueryBuilders to conform with xxxQuery notation 2010-11-10 00:03:33 +02:00
kimchy 4c6af6afa5 Cluster health API should return RED status (on applicable levels) when an index has not recovered from the gateway, closes #507. 2010-11-09 21:25:22 +02:00
kimchy 4695f93276 add index routing table changed 2010-11-09 16:31:21 +02:00
kimchy 8ceba0dca1 don't overflow with cluster events if the index level block was removed already 2010-11-09 15:42:27 +02:00
kimchy 36bf5ee9fe add toString to cluster block 2010-11-09 15:41:47 +02:00
kimchy a0f4359ffa handle any exception when failing to snapshot 2010-11-09 15:34:43 +02:00
kimchy 8266315f51 handle any exception when failing to snapshot 2010-11-09 15:34:17 +02:00
kimchy f9fb67616d don't write checksum file for the segments file, we anyhow recover it and that way the index can be read by pure Lucene code 2010-11-09 14:27:28 +02:00
kimchy b5f1a1b49c Parsing a deeply nested json might fail 2010-11-09 10:26:15 +02:00
kimchy 33ff95af05 add more trove helpers 2010-11-09 09:35:35 +02:00
kimchy 8e2db2c68f rename field to path in _analyzer mapping 2010-11-09 09:16:47 +02:00
kimchy 6b5e884b93 have the default path value for analyzer mapper to be _analyzer. 2010-11-09 08:59:30 +02:00
kimchy 6567fcaf35 remove an index level block if its empty (make no sense to keep an empty set for it, also, shouldn't shot it back in APIs) 2010-11-09 00:08:30 +02:00
kimchy c69b94d769 rename field to path in _analyzer mapping 2010-11-08 11:07:34 +02:00
kimchy c9228ed26d Get Action: Allow to pass refresh forcing getting latest, closes #488. 2010-11-07 23:51:18 +02:00
kimchy c095d72439 Get Action: Allow to pass refresh forcing getting latest, closes #488. 2010-11-07 23:50:48 +02:00
kimchy e51523385d Analysis: Allow to alias analyzers, closes #486. 2010-11-07 20:10:53 +02:00
kimchy 171fa4a7e8 Mapper: An analyzer mapper allowing to control the index analyzer of a document based on a document field, closes #485. 2010-11-07 18:18:24 +02:00
kimchy b45ade467e send transport exception is a wrapper exception around the actual exception 2010-11-07 17:56:50 +02:00
kimchy c9e8a2ac68 no need to pass the id mapprt to serialization twice... 2010-11-07 16:00:27 +02:00
kimchy d77a0c41d0 make sure attributes are also serialized across restarts 2010-11-07 15:32:37 +02:00
kimchy 598225f833 document level analyzer (analyzer, index_analyzer, search_analyzer) are not serialized (and not maintained across restarts), closes #484. 2010-11-07 13:13:29 +02:00
kimchy 7340d6973d slim down the mapping serialization to xcontent (json), don't write default values 2010-11-07 12:33:44 +02:00
kimchy e44604b441 better transport exception handling, refactor out an action based transport exception, and don't fill stack trace only for remote ones 2010-11-07 04:08:05 +02:00
kimchy ef3e7fe62b renamve method 2010-11-07 00:02:58 +02:00
kimchy 6b952f6719 Fs Gateway: Add (native) file lock to ensure two nodes in a split brain are not updating same gateway, closes #481. 2010-11-06 23:27:46 +02:00
kimchy 998bde0820 Add an option to display max_open_files, by setting -Des.max-open-files to `true`, closes #483. 2010-11-06 22:47:09 +02:00
kimchy b568174bee add a comment on why we don't mark as relocated when relcoating primaries 2010-11-06 22:03:11 +02:00
kimchy 87efccf677 expose boolean and Object as values for TermQueryBuilder and FieldQueryBuilder 2010-11-06 01:04:14 +02:00
kimchy 129b9a3938 Query DSL: Allow to control (globally) the max clause count for `bool` query (defaults to 1024), closes #482. 2010-11-06 00:59:37 +02:00
kimchy 9385476340 fix expected setting, had an additional _ by mistake 2010-11-05 20:54:01 +02:00
kimchy 53a98275cf remove unused variable 2010-11-05 18:39:10 +02:00
kimchy 99a3e615ab Improve performance when searching across multiple types, closes #478. 2010-11-04 20:34:49 +02:00
kimchy bbd63f0ffe constant_score query might apply deletes wrongly, closes #477. 2010-11-04 20:18:05 +02:00
kimchy 770ccf421b don't write the local gateway state if there is no master, since we don't want to override the fact that they do exists on that node under the respective version 2010-11-04 12:17:17 +02:00
kimchy e2d6f82cd3 Mapper: Add `path_match` for full object navigation path matching, closes #476. 2010-11-04 10:55:19 +02:00
kimchy 805d43c77a properly clean index shard and index level data when deleting them 2010-11-03 15:34:30 +02:00
kimchy 6e0180db6a refactor node data location of index and shard into common code 2010-11-03 15:07:23 +02:00
kimchy 6804c02e97 Create specific `data` location for indices (move from work), closes #473. 2010-11-03 14:41:29 +02:00
kimchy 92b3ae3f73 Change single operation shard hashing to only use id, and not id and type, closes #472. 2010-11-03 12:47:34 +02:00
kimchy 8d454ba293 Query DSL: Add `field_masking_span` query, closes #471. 2010-11-02 23:50:28 +02:00
kimchy a62f1f3e0d API: Allow to control document shard routing, and search shard routing, closes #470. 2010-11-02 19:56:18 +02:00
kimchy 8e2e85f460 enhance bulk stress test 2010-11-02 14:01:22 +02:00
kimchy 3fe2851dae Index Merge: Change default `index.merge.scheduler.max_thread_count` from ` to be adaptive based on processor count, closes #469. 2010-11-02 14:00:58 +02:00
kimchy 2cdaf6357b Rivers: Add a _status doc for each river, closes #468. 2010-11-02 12:09:45 +02:00
kimchy ef4c44577a add toString to match_all docs 2010-11-02 10:23:05 +02:00
kimchy 4ff1b429f1 Possible (rare) shard index corruption / different doc count on recovery (gateway / shard), closes #466. 2010-11-01 23:00:16 +02:00
kimchy 49439a09e2 fix logging 2010-11-01 20:42:10 +02:00
kimchy f79719beaa fix parsing of bytes value 2010-10-31 15:47:19 +02:00
kimchy feb854b742 On full cluster restart, replicas transaction logs are not getting cleaned, closes #465. 2010-10-31 14:36:10 +02:00
kimchy b8fa68f97a check if the commit point has a size at all, if not, not flushed yet, its ok 2010-10-31 12:23:56 +02:00
kimchy e6b4a8b55c remove inclusion of config and logging file in jar 2010-10-31 10:50:35 +02:00
kimchy 96a1ad6335 When on linux, force it not to swap elasticsearch process (jvm), closes #464. 2010-10-30 19:40:24 +02:00
kimchy 1feb43a0af Index: Move compound format setting to `index.compound_format` (old setting still supported), closes #463. 2010-10-30 18:24:28 +02:00
kimchy db24148413 fix listAll when using switch dir to return unique list of files 2010-10-30 01:26:00 +02:00
kimchy b315c5f848 add getOffsetGap to field analyzer 2010-10-29 19:05:11 +02:00
kimchy 8f7a8dabc1 only flush if there is something to flush 2010-10-29 17:50:51 +02:00
kimchy 11c55185e8 add another test 2010-10-29 00:22:49 +02:00
kimchy c194fa67e9 default logger (when embedded in java) to log with org.elasticsearch prefix for simpler logging configruation 2010-10-28 23:39:37 +02:00
kimchy 7082593166 add term filter toString 2010-10-28 23:18:24 +02:00
kimchy e31cbb5267 fix NPE when enabling trace logging 2010-10-28 21:47:55 +02:00
kimchy 4579c04a9e Mapper: Ip Type Support (ipv4), auto detection with dynamic mapping, closes #461. 2010-10-28 12:02:20 +02:00
kimchy 6f8b859d90 better inet address serialization, also better handle ipv6 2010-10-28 01:48:54 +02:00
kimchy e7fdf16402 add number of data nodes to cluster health 2010-10-28 00:39:53 +02:00
kimchy d487d809ea Query DSL: Revert back and have range, terms, and prefix filter _cache set to true, closes #456. 2010-10-27 20:34:24 +02:00
kimchy 3a9dc8f452 change back to cached thread pool 2010-10-27 20:13:42 +02:00
kimchy 185f5a9e18 Search: Change default operation threading to `thread_per_shard` from `single_thread`, closes #455. 2010-10-27 19:51:49 +02:00
kimchy ca67c12de5 add and/not/or docid sets, not just docsets, and improve caching behavior 2010-10-27 18:16:15 +02:00
kimchy 8fef3df16f improve threading usage on discovery and fd 2010-10-26 23:49:09 +02:00
kimchy 73e6aa72cd for some reason the test only runs with @Test on the main class... 2010-10-26 20:20:58 +02:00
Adriano Ferreira c08ad0cad6 Fix parsing of "geo_distance" filter wrt "distance" / "unit" parameters
The problem was that when "unit" was given,
the conversion to miles was happening too early,
which caused wrong computations. This change
postpones this computation when one really knows
which unit should be used.
2010-10-26 20:13:29 +02:00
Adriano Ferreira 9b206146e3 Fix MILES.toKilometers() and KILOMETERS.toMiles()
This mistake should have been caught by DistanceUnitTests.
But the problem is that the tests in this file does not
run during the execution of the test suite, and I don't have
a clue why this is so.
2010-10-26 20:13:29 +02:00
Adriano Ferreira 2d15cd8009 Another test for parsing "geo_distance" filter: 12mi and km
If an explicit unit is provided with "distance",
the "unit" can be safely ignored, as it works
as a fallback unit.
2010-10-26 20:13:29 +02:00
Adriano Ferreira 8c8b7bee3a Three tests for parsing "geo_distance" filter: 12 miles in km
The added test files should be equivalent. Actually they
hit the same bug as change

    Two tests for parsing "geo_distance" filter: distance/unit parameters
2010-10-26 20:13:29 +02:00
Adriano Ferreira 244a566e49 Two tests for parsing "geo_distance" filter: default unit is km 2010-10-26 20:13:29 +02:00
Adriano Ferreira 0660a2912e Two tests for parsing "geo_distance" filter: distance/unit parameters
Those two are supposed to be equivalent:

    distance: 12, unit: "mi"

    vs

    distance: "12", unit: "mi"

but they are not because of an underlying bug in the query parsing
code, providing non-equivalent behavior whether a number or a string
comes via JSON.
2010-10-26 20:13:29 +02:00
kimchy 14703aa1be doc 2010-10-26 20:05:57 +02:00
kimchy 5804e9132a Query DSL: `term` filter to have `_cache` set to `true` by default, closes #453, 2010-10-26 13:40:14 +02:00
kimchy 442f1d7677 add the request id to the timeout message 2010-10-26 11:22:02 +02:00
kimchy c59bfea43c add the request id to the timeout message 2010-10-26 11:19:31 +02:00
kimchy 529303c533 Query: match_all query (also in query_string: *:*) is very slow 2010-10-26 11:04:55 +02:00
kimchy 5966d2005d refacto package name from facets to facet (make more sense when facet will be easily pluggable) 2010-10-25 18:49:30 +02:00
kimchy 0a0ac35792 bring back query parsing cache (weak) 2010-10-25 18:09:40 +02:00
kimchy 2c65be2aad better exception when a field is missing 2010-10-25 16:33:55 +02:00
kimchy bc4121c06b Query DSL: `term`, `term`, `prefix`, and `range` filter are now weakly cached, for more strong caching, set `_cache` to true, closes #450. 2010-10-25 15:59:15 +02:00
kimchy 5c5b882b5d clean code 2010-10-25 14:44:47 +02:00
kimchy 517f3eb0fd chance default query parser cache to none from weak 2010-10-25 14:43:06 +02:00
kimchy deada942e5 Query DSL: `constant_score` and `filtered` queries cache filters by default, remove it, closes #449. 2010-10-25 13:52:15 +02:00
kimchy 835a81c16c always use deletion aware constant score 2010-10-25 13:35:43 +02:00
kimchy b7eacc3618 add comment / todo 2010-10-24 12:38:24 +02:00
kimchy 9a80fedf52 add simple map stream capability 2010-10-23 23:19:08 +02:00
kimchy ca7a7467dc Indices API: Add open and close index, closes #447. 2010-10-23 21:52:09 +02:00
kimchy a76824e395 make index not recovered a bock used in both gateways (shared/blob and local) 2010-10-23 17:03:38 +02:00
kimchy 3f8c03db95 add test in node allocation is there are no shards, no need to do local gateway allocation 2010-10-23 01:25:23 +02:00
kimchy e11589f3c7 add doc 2010-10-23 01:07:37 +02:00
kimchy 62526444fd better caching of missing filter / query 2010-10-22 19:44:25 +02:00
kimchy b0a0604473 Query DSL: Query String _missing_ and _exists_ syntax, closes #446. 2010-10-22 18:14:07 +02:00
kimchy 8b0daf48f4 Query DSL: Query String _missing_ and _exists_ syntax, closes #446. 2010-10-22 17:54:02 +02:00
kimchy 9237dafef9 Query DSL: Query String _missing_ and _exists_ syntax, closes #446. 2010-10-22 17:51:19 +02:00
kimchy f63ee3158a Query DSL: Add `exists` and `missing` filters to filter documents where a field either has a value or not in them, closes #445. 2010-10-22 15:14:05 +02:00
kimchy 9a8e033424 improve fixing negative query with adding the optimized match_all query 2010-10-22 14:50:48 +02:00
kimchy 5d1d927e09 Write Consistency Level for index/delete/delete_by_query/bulk with one/quorum/all. Defaults to quorum, closes #444. 2010-10-22 02:51:00 +02:00
kimchy 5649df572a update search expiration to work in scheduled reaper mode 2010-10-21 16:38:44 +02:00
kimchy c37a0afbf0 improve update of mappings, and delete index process with the new optimization of when index is created 2010-10-20 12:49:21 +02:00
kimchy ae5bc20959 Reduce multiple indices overhead, don't create Indices data on nodes that don't hold any shard for an index, closes #441. 2010-10-20 02:57:49 +02:00
kimchy 1235358848 remove unused indices service 2010-10-19 19:50:12 +02:00
kimchy a4eea0aeaa change index creation / deletion logic not to wait for it to be created on other nodes since now, they might not be created... 2010-10-19 19:33:27 +02:00
kimchy 69b8b0f437 don't use the index serivce to get the stored blobs in shared gateway, move it to upper node level 2010-10-19 18:03:48 +02:00
kimchy 53a3df5d8e move list of local files to be indices level, so no need for index created on a node in order to list it 2010-10-19 17:36:21 +02:00
kimchy 5635260922 remove check on if indices has an index, cluster state is enough 2010-10-19 16:42:39 +02:00
kimchy 4c8978237f move operation routing to be cluster level and not index level 2010-10-19 16:23:39 +02:00
kimchy 1eee7f381a dont case node names and index names 2010-10-18 19:03:48 +02:00
kimchy 5c760d1e8c don't case the index name in cluster health, use string constants in xcontent generation 2010-10-18 18:55:38 +02:00
kimchy 3a52c2f7f4 Improve automatic publish network address logic, closes #439. 2010-10-18 15:19:24 +02:00
kimchy 019c844dd1 Plugins: plugins should now be in extracted format under `ES_HOME/plugins`, closes #438. 2010-10-18 11:14:16 +02:00
kimchy 0a3d187e6a Mapper: Allow to configure `date_formats` only on the root object mapper, closes #437. 2010-10-17 19:56:58 +02:00
kimchy 69ecf8b66b rollback commit, allocation should only happen when index is not recovered 2010-10-17 19:43:30 +02:00
kimchy 9a09261383 remove checking for client node on rivers, its automatically done with master and non river node anyhow 2010-10-17 19:20:37 +02:00
kimchy 648da2e6ff add cluster name to cluster health response 2010-10-17 18:58:59 +02:00
kimchy 5d54e36964 Statistical Facet: Allow to compute statistical facets on more than one field, closes #436. 2010-10-17 16:22:20 +02:00
kimchy 2fc0022477 clean code 2010-10-17 02:33:12 +02:00
kimchy caa6495e2b doc 2010-10-17 01:52:23 +02:00
kimchy 4708d31040 add the option to control filter caching on each filter in the java filter builders 2010-10-16 16:12:38 +02:00
kimchy d9f966d83c Query DSL: Numeric Range Filter - A filter that uses the field data cache to perform numeric tests, closes #435. 2010-10-16 15:38:43 +02:00
kimchy cc3a1dbe90 move geo handling (filter) into the index package from common.lucene since it uses field data and not really only dependant on lucene constucts 2010-10-16 12:30:20 +02:00
kimchy b09675dcfa add a stress test for rolling restart nodes 2010-10-16 01:05:25 +02:00
kimchy bbb05a57e0 add the ability to run check index before starting a shard, used for internal testing (as it has a large overhead) 2010-10-15 20:56:30 +02:00
kimchy 6c9120a51b Local Gateway: Don't block meta operations (delete index) on an index that is not recovered due to not all shards being available, closes #433. 2010-10-15 11:04:54 +02:00
kimchy ae05ce0551 Scripting: Allow to define scripts within `config/scripts`, automatically compiled and can be referenced by name, closes #429. 2010-10-14 16:14:23 +02:00
kimchy e0488fa87e Plugins: Allow to place "extracted" plugins under the `plugins` directory, closes #428. 2010-10-14 14:04:00 +02:00
kimchy 8d533e8a99 dynamic tempaltes are now named (change to the mapping definition) to allow for simpler merging 2010-10-14 12:56:50 +02:00
kimchy ad01f19db8 move dynamic tempaltes to only allow to be defined on root object mapping (so easily reusable and simpler merging) #397. 2010-10-14 12:25:32 +02:00
kimchy 1578da404c improve geo internal cache of lat/lon, stored two double arrays instead of an array of GeoPoints. 2010-10-13 22:03:48 +02:00
kimchy 9077bb6528 Geo Overhaul (work with multiple locations), closes #414. Change (again) how geo location is stored, not using geohash but explicit double conversion 2010-10-13 03:35:35 +02:00
kimchy 8a8a4d648a Query DSL: Fine grained filter caching control with sensible defaults, closes #424. 2010-10-13 01:37:43 +02:00
kimchy 3fa214653e increase precision to 16 from 12, omit norms / tf on geohash 2010-10-12 22:30:56 +02:00
kimchy c3cb5a3e34 Use Base64 encoding for UUID generation (auto generated doc ids, and node ids), closes #422. 2010-10-12 00:17:01 +02:00
kimchy 18d8e9dcdd add more script tests 2010-10-10 20:05:37 +02:00
kimchy a14b73bc13 Bulk API: Rename `index` to `_index`, `type` to `_type` and `id` to `_id`, closes #421. 2010-10-10 19:31:56 +02:00
kimchy c15a612b8a optimize get/delete/index response xcontent generation 2010-10-09 04:06:03 +02:00
kimchy 087b4e6f23 optimize search response xcontent generation 2010-10-09 04:01:17 +02:00
kimchy f64e636415 add xcontent string that will allow to speed up json/smile serialization 2010-10-09 03:06:01 +02:00
kimchy 6314c2460c Geo Overhaul (work with multiple locations), closes #414. 2010-10-08 22:31:21 +02:00
kimchy 523a8b4c3e rename xonctent mappers to just mappers 2010-10-08 15:05:07 +02:00
kimchy a61f3a939b add option to reuse double[] when decoding geohash 2010-10-07 23:58:49 +02:00
kimchy 0f6beeb263 reuse search lookup data for the two remaining cases (custom score and script filter) by having a "current" search context to access 2010-10-07 23:18:26 +02:00
kimchy 34ed85a40f internal refactoring of field data type 2010-10-07 18:41:17 +02:00
kimchy d0bf743ab4 Query DSL: query_string / field to use the optimized match_all query when using * (or *:*), closes #413. 2010-10-07 16:48:24 +02:00
kimchy cf5ed1d177 Term Facet: Use a script that provides the facets to index, closes #410. 2010-10-07 01:49:12 +02:00
kimchy 4c34840627 fix dynamic tempaltes persistency and merging (so it will be there across restarts, and when a new mapping is merged) 2010-10-07 00:44:53 +02:00
kimchy 4422a4c48a doc 2010-10-06 14:07:34 +02:00
kimchy 22ed71e2e6 allow to execute term facet on more than one field 2010-10-06 13:39:02 +02:00
kimchy 31d94b19a1 Local Gateway: Possible failure to allocate shards to nodes when more than one index exists in the cluster (on full cluster restart), closes #409. 2010-10-06 00:35:10 +02:00
kimchy 1f49eb0b9d Gateway: Add `gateway.expected_nodes` for state recovery, closes #404. 2010-10-04 16:20:32 +02:00
kimchy c737b36df7 better handling of dynamic types in template mappings 2010-10-04 12:37:56 +02:00
kimchy 425744e0db add bzip2 compress lib 2010-10-03 16:24:10 +02:00
kimchy 7180636df6 add close to script services 2010-10-03 14:31:36 +02:00
kimchy c9a47a126f Scripts: Allow to access `_fields` providing access to stored fields, closes #402. 2010-10-03 12:51:41 +02:00
kimchy 5b8bc333bd JavaScript Plugin: Allow to use javascript for scripts, closes #401. 2010-10-03 02:20:37 +02:00
kimchy 5a7c8fe2cb allow script engines to have several types 2010-10-02 21:18:01 +02:00
kimchy a127499e3c doc 2010-10-02 21:07:59 +02:00
kimchy 31ebed11fd script handling refactoring, better support for languages that perform better with some caching on a per search aspect (and execution per doc) 2010-10-02 17:32:29 +02:00
kimchy 9e8ebd46e8 Groovy Plugin: Allow to run use groovy for scripts (where applicable), closes #398. 2010-10-02 01:22:05 +02:00
kimchy aa116e5d40 add lang parameter to all script constructs, allowing for (later) custom script languages 2010-10-01 22:14:20 +02:00
kimchy 99fcfde307 Mapper: Dynamic Template Support, closes #397. 2010-10-01 16:56:04 +02:00
kimchy f66f0218ca `/_cluster/nodes/stats` is broken in 0.11, closes #391. 2010-09-30 08:35:11 +02:00
kimchy 3b02bd3952 rename package 2010-09-30 08:17:33 +02:00
kimchy d3978383a5 when using keyword based analayzer on _all, an NPE is thrown since there is no current entry, ignore it (it does not make sense to have keywork analyzer on _all field...) 2010-09-28 14:53:37 +02:00
kimchy 55ae6b2001 fix possible problem when trying to convert a binary format to a string 2010-09-28 12:32:40 +02:00
kimchy 86e17eae2c fix auto conversion from source content type to rest content type 2010-09-28 00:01:35 +02:00
kimchy 1f3adc213f add nodes attributes to cluster state 2010-09-27 10:46:11 +02:00
kimchy d9c2cc9d37 allow to sepcify stopwords as comma delimiated list of words 2010-09-26 22:50:25 +02:00
kimchy ce28882660 add refresh option to index/create/delete opereation, REST allows for refresh parameter (defaults to false) 2010-09-26 09:07:37 +02:00
kimchy 2288c5d670 setHighlighterPostTags sets pre tags instead of post, closes #386. 2010-09-25 00:01:14 +02:00
kimchy ab9aa15bb4 add auto sync of translog when using local gateway 2010-09-24 21:25:10 +02:00
kimchy 1c2733ebee add flush to gateway 2010-09-24 12:29:52 +02:00
kimchy 485f904140 Default to not using compound file format for the index (increases number of open files, but faster indexing and recovery), closes #384. 2010-09-24 11:46:41 +02:00
kimchy 2f43af0bd7 use directly the directory and not the metaData API 2010-09-24 11:36:13 +02:00
kimchy 9bb86ea865 imrpvoe boolean filter performance 2010-09-24 00:59:26 +02:00
kimchy 31fd196687 doc the use of mapbuilder 2010-09-23 20:06:42 +02:00
kimchy 097071a9d2 add smart name filter wrapping for geo filters to automatically filter by type if as well if provided 2010-09-23 19:34:41 +02:00
kimchy 832914bf6d create a mapbuilder when listing files and not ImmutableMap builder since it seems like listFiles might return duplicate files. 2010-09-23 19:13:48 +02:00
kimchy 761639150d doc 2010-09-23 19:03:48 +02:00
kimchy c96014bb90 default the _river to 1/1 for better OOB experiance (single node recovery with local gateway support) 2010-09-23 02:46:13 +02:00
kimchy ecaaeb5250 River CouchDB Plugin, closes #382. 2010-09-23 02:26:12 +02:00
kimchy a59912db02 disable restart API 2010-09-23 00:02:12 +02:00
kimchy fd9266dbc6 fix wrong REST response generation of cluster state 2010-09-22 16:03:42 +02:00
kimchy 91344170ee doc 2010-09-22 15:39:30 +02:00
kimchy f301138e6f Change default thread pool to scaling from cached, closes #381. 2010-09-22 15:20:32 +02:00
kimchy 6d0829f331 imrpove shard target streaming, only do it when needed 2010-09-22 14:36:29 +02:00
kimchy cfdf6b0103 add support for op type in also in bulk 2010-09-22 00:07:45 +02:00
kimchy 3479f2a981 better exception message 2010-09-21 23:49:55 +02:00
kimchy 2cbcc8dd2e better handling of search context timeout 2010-09-21 23:17:46 +02:00
kimchy b9b91db5c8 building search response should handle cases where the sort is null valued 2010-09-21 22:13:27 +02:00
kimchy aa68667d63 rabbitmq river 2010-09-21 17:32:34 +02:00
kimchy 32acd88b2e default river index to have 1 shard with 2 replicas by default 2010-09-21 15:07:51 +02:00
kimchy 8dfac512ba move parsing of framed bulk binary data into the bulk request 2010-09-21 14:29:19 +02:00
kimchy 4fe7cbccca rename indexer to river 2010-09-21 13:35:09 +02:00
kimchy 69fad80b01 add rest response for allocations 2010-09-21 13:05:10 +02:00
kimchy a1299a3d4f refactoring in routing logic to allow adding allocation explanation that later can be shown 2010-09-21 11:39:22 +02:00
kimchy b5b36215ae refactoring in routing logic to allow adding allocation explanation that later can be shown 2010-09-21 11:37:36 +02:00
kimchy a68c00cd0d add special handling for primitive arrays as fields in xcontent builder 2010-09-21 01:41:35 +02:00
kimchy 121e548d76 add to nodes stats the total store size of the indices shards allocated to that node 2010-09-21 01:11:43 +02:00
kimchy 94a77b69d6 add a twitter indexer 2010-09-20 18:17:02 +02:00
kimchy eb4f4f99b3 add recover_after gateway tests 2010-09-20 09:44:57 +02:00
kimchy 3ec95f4e84 javadoc 2010-09-20 08:58:33 +02:00
kimchy 770bac252a fix local discovery to handle master nodes specifically better 2010-09-19 23:49:15 +02:00
kimchy 25246902cc Gateway: Add `recover_after_data_nodes` and `recover_after_master_nodes`, closes #376. 2010-09-19 22:46:13 +02:00
kimchy 8def6f59b3 simple specific master nodes test 2010-09-19 18:48:48 +02:00
kimchy 589dbce89b move persistenting state and sharsd state in local gateway to another thread 2010-09-19 17:54:14 +02:00
kimchy e3948115f4 doc 2010-09-18 22:00:18 +02:00
kimchy 2372f481aa Shared Storage Gateway (fs/s3): Wrong snapshotting of transaction log, closes #375. 2010-09-18 21:33:38 +02:00
kimchy a2011e0151 no need to cache match all filter 2010-09-17 16:47:35 +02:00
kimchy cd7b84e808 improve match_all query to use a filter when possible 2010-09-17 16:38:27 +02:00
kimchy 3c95d6a215 more work on inexers 2010-09-17 16:36:12 +02:00
kimchy ff081240eb imrpove caching and fix delete mapping logic 2010-09-17 01:25:37 +02:00
kimchy 08d7125cd5 more work on indexer 2010-09-17 00:22:42 +02:00
kimchy 4b87f63bed Terms Facet: Allow to provide `_index` as the field, resulting in facets on indices, closes #374. 2010-09-16 15:11:30 +02:00
kimchy f49f3e169a Mapping API: Delete Mapping (with data), closes #373. 2010-09-16 14:35:07 +02:00
kimchy aef4704fed more indexer work 2010-09-16 13:22:09 +02:00
kimchy 09cdc1c379 don't wait for indices if there are none when using local gateway 2010-09-15 18:22:13 +02:00
kimchy e52daa9670 automatic index creation when using the bulk api 2010-09-15 18:18:48 +02:00
kimchy e93eb16deb docs and wrap the rest bulk request in an array 2010-09-15 17:59:10 +02:00
kimchy 5a5a892cc7 return the correct content type 2010-09-15 12:55:36 +02:00
kimchy 8db2a217c0 Expose internal search source builder 2010-09-15 12:43:21 +02:00
kimchy 3afe4da550 Bulk API, closes #371. 2010-09-15 12:22:07 +02:00
kimchy f6fa6ea44e more work on bulk, client API 2010-09-14 22:08:43 +02:00
kimchy d96ffe9153 internal bulk operaiton on the index shard 2010-09-14 01:46:44 +02:00
kimchy 85160ae341 Mapper: Upgrading a simple mapping to multi_field mapping fails on merge conflicts, closes #369. 2010-09-14 00:10:01 +02:00
kimchy 686b59c33f more work on indexer 2010-09-13 23:00:34 +02:00
kimchy 0c8dfd967c better raw generation of json 2010-09-13 02:43:39 +02:00
kimchy 2d5883ccb5 improve normalization performance from one content type to the other 2010-09-13 02:09:15 +02:00
kimchy 38aeba438d XContent refactoring, closes #367. 2010-09-12 23:20:15 +02:00
kimchy 4822517d5d add query parser cache to cache common query strings (+settings) with the parsed queries 2010-09-12 17:08:26 +02:00
kimchy 77a7938891 move to use guice to register also the default query parsers and filters 2010-09-12 16:46:57 +02:00
kimchy 2442e1fb16 improve query string parsing by reusing the same parser (bounded to the query parser context) 2010-09-12 15:39:07 +02:00
kimchy ed5ffd6591 Setting `path.work` does not affect the logs location, closes #366. 2010-09-11 22:22:49 +03:00
kimchy bc42352f7e change the local gateway metadata file from state- to metadata- 2010-09-11 21:45:45 +03:00
kimchy 0718757612 Query DSL: `match_all` filter, closes #365. 2010-09-11 16:01:30 +03:00
kimchy eccc7d5ef2 Search API: Allow to name filters and return per hit the filters it matched on, closes #364. 2010-09-11 12:38:19 +03:00
kimchy ab2a655a59 parsing a query should result in ParsedQuery object, so we can piggyback on it more parsing state / data 2010-09-11 00:34:22 +03:00
kimchy a47638dd3c imrove filter caching handling by not holding an implicit ref to the parent cache 2010-09-11 00:11:47 +03:00
kimchy de05a10ae5 use fastGet in docset 2010-09-10 16:16:45 +03:00
kimchy 4e661c165a Geo search across anti-meridian doesn't work, closes #363. 2010-09-10 16:04:10 +03:00
kimchy b31b0e979c Search failure when no results are found and sorting by score, closes #362. 2010-09-09 15:47:55 +03:00
kimchy 31362e4c60 upgrade to joda 1.6 2010-09-09 00:53:20 +03:00
kimchy a768016779 Allow to configure a common logger prefix using `es.logger.prefix` system prop, closes #360. 2010-09-08 12:16:41 +03:00
kimchy e151b83372 add simple dummy indexer 2010-09-07 18:19:02 +03:00
kimchy ad5945f141 Mapper: `geo_point` to support passing array of [lat, lon], closes #361. 2010-09-07 17:23:06 +03:00
kimchy c045b4d0ef Mapper: `geo_point` type to allow to set `store` on the mapping, closes #359. 2010-09-07 14:10:29 +03:00
kimchy 7c48f27e60 throw an exception when no lat/lon or geohash are found 2010-09-07 11:36:12 +03:00
kimchy 2cb45d4fe7 hack to work around complete text highlighting 2010-09-07 01:28:30 +03:00
kimchy bbb2033d3b start working on indexers 2010-09-07 00:41:30 +03:00
kimchy f270fc00d2 earch API: Automatically identify "script" fields on the field elements in search, close #357. 2010-09-06 11:51:31 +03:00
kimchy 243b7455e8 fix javadoc 2010-09-06 01:10:46 +03:00
kimchy cc1eac147a Allow to specify highlighter parameters on a per field level basis, closes #356. 2010-09-05 20:47:38 +03:00
Lukáš Vlček 8efe5dc9ae Highlighter enhancements. 2010-09-05 14:52:48 +03:00
kimchy c050bb58da allow to set a query using a string as well 2010-09-04 21:30:02 +03:00
kimchy 42b2e60af2 Create Index / Update Settings: Automatically prefix all settings with `index.`, closes #355. 2010-09-04 14:04:51 +03:00
kimchy 4f08801bb9 Thrift Plugin: Expose REST interface using thrift, closes #354. 2010-09-04 02:45:37 +03:00
kimchy 3d30b9c034 fix not called processModule on plugin for index and shard level modules 2010-09-03 19:06:11 +03:00
kimchy 1079941397 javadoc 2010-09-03 18:40:30 +03:00
kimchy b40f0bc5a4 Local Gateway: Expose `gateway.local.initial_shards` to control when to recover and index, closes #352. 2010-09-03 18:38:24 +03:00
kimchy 575f5b406b abstract away script execution 2010-09-03 01:49:06 +03:00
kimchy cd28afe950 Search API: Allow for script fields to extract parts of the stored _`source`, closes #349. 2010-09-02 21:29:41 +03:00
kimchy 1e75638b31 refactor field funtion to use internal docmap to be used in scripts 2010-09-02 17:19:50 +03:00
kimchy 807c485a38 make the cached compile scripts a soft map 2010-09-02 16:29:41 +03:00
kimchy 7bd08d638d JMX: Disable creating by default an RMI JMX connector, closes #348. 2010-09-02 14:17:53 +03:00
kimchy 0250896a09 reduce shard search response size by not serilaizing the node id, index, and shard id 2010-09-02 11:19:25 +03:00
kimchy 042af200e8 Gateway: Default to `local` gateway (replace `none` gateway), closes #347. 2010-09-01 23:21:26 +03:00
kimchy 0de24218ff improve local gateway, store started shards and metadata in separated files, metadata stored on master nodes, and started shards on data nodes. 2010-09-01 21:32:07 +03:00
kimchy 7709cd1bc2 improve GC logging 2010-09-01 15:02:56 +03:00
kimchy 7d6aa6c027 local gateway: don't store the local state when not a data/master node 2010-09-01 00:07:10 +03:00
kimchy a3efa21d71 Mapper: Failure to handle null valued objects ("obj1" : null), closes #341. 2010-08-31 13:33:21 +03:00
kimchy 289cfdf25a improve replica allocation, fix test to wait for recovery on node2 2010-08-30 23:37:09 +03:00
kimchy 9503fca2ae fix logging 2010-08-30 22:53:47 +03:00
kimchy 61764c5b69 add another test for local gateway, fix race when loading the current state of a node 2010-08-30 19:29:00 +03:00
kimchy 908fba44e7 fsync translog when closing and not deleting, allocate failed shard to another started shard 2010-08-30 16:55:57 +03:00
kimchy d9979f8dfe more work on local gateway, start integration test it 2010-08-30 01:25:45 +03:00
kimchy 8ed54c24bd change nodeFile to nodeLocation 2010-08-29 23:25:23 +03:00
kimchy 269616f35e more work on getting local gateway to work, apply transaction log operations when primary "recovering" 2010-08-29 16:56:04 +03:00
kimchy 4f4471483d initial work on local gateway 2010-08-29 01:24:23 +03:00
kimchy 163b7be639 refactor gateway service to allow for more custom gateway implemenations 2010-08-28 00:34:15 +03:00
kimchy a10ab7fe08 better exception thrown when failing to handle responses 2010-08-27 14:14:28 +03:00
kimchy e85c47db74 improve stack trace when failing to send a transaction request 2010-08-27 13:52:58 +03:00
kimchy e5b041c8ef simplify gateway snapshot operation 2010-08-27 00:41:55 +03:00
kimchy a49af6072b add objects option to range filter builder 2010-08-26 16:39:39 +03:00
kimchy d56b4d266e move query hint only to broadcast requests that use it 2010-08-26 14:31:54 +03:00
kimchy dfa24f6d03 improve caching of stored metadata fetched from nodes during allocation in order to reuse existing shards 2010-08-25 23:12:07 +03:00
kimchy 2910b6ab7f add callback for shards_started and shards_failed to better handle caching on node allocations level 2010-08-25 22:20:27 +03:00
kimchy 3ebd03f69a doc 2010-08-25 22:03:11 +03:00
kimchy 9d615a4f0b Query DSL: Improve `or` and `and` filters to accept just array of filters, closes #336. 2010-08-25 18:06:28 +03:00
kimchy c7a36ed431 add an exception when on filtesr are provided to "and" and "not" filters 2010-08-25 17:56:50 +03:00
kimchy fcdc5f872a improve (perf) getting last committed version of the index to create a new translog with 2010-08-25 15:12:26 +03:00
kimchy 9d08e56d8a cache nodes store metadata for on a shard basis when iteratring over the unassigned nodes 2010-08-25 15:03:25 +03:00
kimchy f36d80f66d don't go and find local storage for a shard on nodes if its not going to be allocated 2010-08-25 15:01:09 +03:00
kimchy 4bfd0a8c26 cache commit points during smart allocation based on local storage when recovering from gateway 2010-08-25 01:12:16 +03:00
kimchy 244cd42298 allow to use just hosts when using unicast discover, will use the transport level ports 2010-08-25 00:44:04 +03:00
kimchy fc812a306b change logging 2010-08-24 23:33:03 +03:00
kimchy ba5814a72f change log level 2010-08-24 23:25:20 +03:00
kimchy 31aed01220 increase timeout to 5 seconds 2010-08-24 23:18:58 +03:00
kimchy fdc82dd5a3 ignore parsing of wrong file format in the gateway (log warn), use index vesrion as the translog id 2010-08-24 23:12:18 +03:00
kimchy d0f5bc3403 add russian letter tokenizer 2010-08-24 23:11:22 +03:00
kimchy 49b4659322 Indexing Buffer Size: Refine default setting to 10% (from 40%) and add more settings, closes #334. 2010-08-24 18:12:45 +03:00
kimchy 7ae8d4c669 thread pool type setting is wrong, fix blocking max setting to have a minimum of 10, use cached TP where needed 2010-08-24 15:28:54 +03:00
kimchy b8ab50828c only list and delete files we created 2010-08-24 14:48:49 +03:00
kimchy df23d8f7ec don't snapshot a translog that has no content 2010-08-24 03:58:21 +03:00
kimchy 6c29142b91 throw recovery failure if failing to parse all commit points 2010-08-24 01:28:39 +03:00
kimchy b52d854711 smarter handling of throttling when preferring allocating to an existing work location 2010-08-23 21:39:54 +03:00
kimchy 78b6879ecd move rebalance only when active logic into node allocation, add canRebalance hook point 2010-08-23 18:28:35 +03:00
kimchy 149962691e move rebalance only when active logic into node allocation, add canRebalance hook point 2010-08-23 18:27:01 +03:00
kimchy 4300a6ca18 move rebalance only when active logic into node allocation, add canRebalance hook point 2010-08-23 18:26:17 +03:00
kimchy 0e47898902 add /_shutdown to the REST endpoints 2010-08-23 18:01:47 +03:00
kimchy 5900e01a0d list remote nodest storage information with a timeout 2010-08-23 18:00:37 +03:00
kimchy 1461da5b49 move prefetch allocation to be a NodeAllocation 2010-08-23 17:22:44 +03:00