honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,934 Commits 7 Branches 6 Tags 546 MiB
Commit Graph

1717 Commits

Author SHA1 Message Date
kimchy 5c5b882b5d clean code 2010-10-25 14:44:47 +02:00
kimchy 517f3eb0fd chance default query parser cache to none from weak 2010-10-25 14:43:06 +02:00
kimchy deada942e5 Query DSL: `constant_score` and `filtered` queries cache filters by default, remove it, closes #449. 2010-10-25 13:52:15 +02:00
kimchy 835a81c16c always use deletion aware constant score 2010-10-25 13:35:43 +02:00
kimchy b7eacc3618 add comment / todo 2010-10-24 12:38:24 +02:00
kimchy 9a80fedf52 add simple map stream capability 2010-10-23 23:19:08 +02:00
kimchy ca7a7467dc Indices API: Add open and close index, closes #447. 2010-10-23 21:52:09 +02:00
kimchy a76824e395 make index not recovered a bock used in both gateways (shared/blob and local) 2010-10-23 17:03:38 +02:00
kimchy 3f8c03db95 add test in node allocation is there are no shards, no need to do local gateway allocation 2010-10-23 01:25:23 +02:00
kimchy f0f62ce00c improve test to wait for 2 nodes 2010-10-23 01:24:53 +02:00
kimchy e11589f3c7 add doc 2010-10-23 01:07:37 +02:00
kimchy 62526444fd better caching of missing filter / query 2010-10-22 19:44:25 +02:00
kimchy b0a0604473 Query DSL: Query String _missing_ and _exists_ syntax, closes #446. 2010-10-22 18:14:07 +02:00
kimchy 8b0daf48f4 Query DSL: Query String _missing_ and _exists_ syntax, closes #446. 2010-10-22 17:54:02 +02:00
kimchy 9237dafef9 Query DSL: Query String _missing_ and _exists_ syntax, closes #446. 2010-10-22 17:51:19 +02:00
kimchy f63ee3158a Query DSL: Add `exists` and `missing` filters to filter documents where a field either has a value or not in them, closes #445. 2010-10-22 15:14:05 +02:00
kimchy 9a8e033424 improve fixing negative query with adding the optimized match_all query 2010-10-22 14:50:48 +02:00
kimchy 5d1d927e09 Write Consistency Level for index/delete/delete_by_query/bulk with one/quorum/all. Defaults to quorum, closes #444. 2010-10-22 02:51:00 +02:00
kimchy 5649df572a update search expiration to work in scheduled reaper mode 2010-10-21 16:38:44 +02:00
kimchy c37a0afbf0 improve update of mappings, and delete index process with the new optimization of when index is created 2010-10-20 12:49:21 +02:00
kimchy 0b593bd6d9 upgrade to guava r07 2010-10-20 12:09:19 +02:00
kimchy ae5bc20959 Reduce multiple indices overhead, don't create Indices data on nodes that don't hold any shard for an index, closes #441. 2010-10-20 02:57:49 +02:00
kimchy 1235358848 remove unused indices service 2010-10-19 19:50:12 +02:00
kimchy a4eea0aeaa change index creation / deletion logic not to wait for it to be created on other nodes since now, they might not be created... 2010-10-19 19:33:27 +02:00
kimchy 69b8b0f437 don't use the index serivce to get the stored blobs in shared gateway, move it to upper node level 2010-10-19 18:03:48 +02:00
kimchy 53a3df5d8e move list of local files to be indices level, so no need for index created on a node in order to list it 2010-10-19 17:36:21 +02:00
kimchy 5635260922 remove check on if indices has an index, cluster state is enough 2010-10-19 16:42:39 +02:00
kimchy 4c8978237f move operation routing to be cluster level and not index level 2010-10-19 16:23:39 +02:00
kimchy 1eee7f381a dont case node names and index names 2010-10-18 19:03:48 +02:00
kimchy 5c760d1e8c don't case the index name in cluster health, use string constants in xcontent generation 2010-10-18 18:55:38 +02:00
kimchy 3a52c2f7f4 Improve automatic publish network address logic, closes #439. 2010-10-18 15:19:24 +02:00
kimchy 019c844dd1 Plugins: plugins should now be in extracted format under `ES_HOME/plugins`, closes #438. 2010-10-18 11:14:16 +02:00
kimchy 0a3d187e6a Mapper: Allow to configure `date_formats` only on the root object mapper, closes #437. 2010-10-17 19:56:58 +02:00
kimchy 69ecf8b66b rollback commit, allocation should only happen when index is not recovered 2010-10-17 19:43:30 +02:00
kimchy 9a09261383 remove checking for client node on rivers, its automatically done with master and non river node anyhow 2010-10-17 19:20:37 +02:00
kimchy 648da2e6ff add cluster name to cluster health response 2010-10-17 18:58:59 +02:00
kimchy 5d54e36964 Statistical Facet: Allow to compute statistical facets on more than one field, closes #436. 2010-10-17 16:22:20 +02:00
kimchy 2fc0022477 clean code 2010-10-17 02:33:12 +02:00
kimchy caa6495e2b doc 2010-10-17 01:52:23 +02:00
kimchy 4708d31040 add the option to control filter caching on each filter in the java filter builders 2010-10-16 16:12:38 +02:00
kimchy d9f966d83c Query DSL: Numeric Range Filter - A filter that uses the field data cache to perform numeric tests, closes #435. 2010-10-16 15:38:43 +02:00
kimchy cc3a1dbe90 move geo handling (filter) into the index package from common.lucene since it uses field data and not really only dependant on lucene constucts 2010-10-16 12:30:20 +02:00
kimchy 1587a3815e more options on stress rolling restart test, variable number of fields, variable length of text 2010-10-16 03:26:11 +02:00
kimchy b09675dcfa add a stress test for rolling restart nodes 2010-10-16 01:05:25 +02:00
kimchy bbb05a57e0 add the ability to run check index before starting a shard, used for internal testing (as it has a large overhead) 2010-10-15 20:56:30 +02:00
kimchy 6c9120a51b Local Gateway: Don't block meta operations (delete index) on an index that is not recovered due to not all shards being available, closes #433. 2010-10-15 11:04:54 +02:00
kimchy ae05ce0551 Scripting: Allow to define scripts within `config/scripts`, automatically compiled and can be referenced by name, closes #429. 2010-10-14 16:14:23 +02:00
kimchy e0488fa87e Plugins: Allow to place "extracted" plugins under the `plugins` directory, closes #428. 2010-10-14 14:04:00 +02:00
kimchy 8d533e8a99 dynamic tempaltes are now named (change to the mapping definition) to allow for simpler merging 2010-10-14 12:56:50 +02:00
kimchy ad01f19db8 move dynamic tempaltes to only allow to be defined on root object mapping (so easily reusable and simpler merging) #397. 2010-10-14 12:25:32 +02:00
kimchy 1578da404c improve geo internal cache of lat/lon, stored two double arrays instead of an array of GeoPoints. 2010-10-13 22:03:48 +02:00
kimchy 9077bb6528 Geo Overhaul (work with multiple locations), closes #414. Change (again) how geo location is stored, not using geohash but explicit double conversion 2010-10-13 03:35:35 +02:00
kimchy 8a8a4d648a Query DSL: Fine grained filter caching control with sensible defaults, closes #424. 2010-10-13 01:37:43 +02:00
kimchy 3fa214653e increase precision to 16 from 12, omit norms / tf on geohash 2010-10-12 22:30:56 +02:00
kimchy c3cb5a3e34 Use Base64 encoding for UUID generation (auto generated doc ids, and node ids), closes #422. 2010-10-12 00:17:01 +02:00
kimchy 4f9d9c1395 add test for merging two stats field facet using same facet name 2010-10-11 00:40:37 +02:00
kimchy 18d8e9dcdd add more script tests 2010-10-10 20:05:37 +02:00
kimchy a14b73bc13 Bulk API: Rename `index` to `_index`, `type` to `_type` and `id` to `_id`, closes #421. 2010-10-10 19:31:56 +02:00
kimchy c15a612b8a optimize get/delete/index response xcontent generation 2010-10-09 04:06:03 +02:00
kimchy 087b4e6f23 optimize search response xcontent generation 2010-10-09 04:01:17 +02:00
kimchy f64e636415 add xcontent string that will allow to speed up json/smile serialization 2010-10-09 03:06:01 +02:00
kimchy bd962ba693 upgrade to jackson 1.6.1 2010-10-09 01:51:40 +02:00
kimchy 6314c2460c Geo Overhaul (work with multiple locations), closes #414. 2010-10-08 22:31:21 +02:00
kimchy 523a8b4c3e rename xonctent mappers to just mappers 2010-10-08 15:05:07 +02:00
kimchy a61f3a939b add option to reuse double[] when decoding geohash 2010-10-07 23:58:49 +02:00
kimchy 0f6beeb263 reuse search lookup data for the two remaining cases (custom score and script filter) by having a "current" search context to access 2010-10-07 23:18:26 +02:00
kimchy 34ed85a40f internal refactoring of field data type 2010-10-07 18:41:17 +02:00
kimchy d0bf743ab4 Query DSL: query_string / field to use the optimized match_all query when using * (or *:*), closes #413. 2010-10-07 16:48:24 +02:00
kimchy cf5ed1d177 Term Facet: Use a script that provides the facets to index, closes #410. 2010-10-07 01:49:12 +02:00
kimchy 4c34840627 fix dynamic tempaltes persistency and merging (so it will be there across restarts, and when a new mapping is merged) 2010-10-07 00:44:53 +02:00
kimchy 4422a4c48a doc 2010-10-06 14:07:34 +02:00
kimchy 22ed71e2e6 allow to execute term facet on more than one field 2010-10-06 13:39:02 +02:00
kimchy 31d94b19a1 Local Gateway: Possible failure to allocate shards to nodes when more than one index exists in the cluster (on full cluster restart), closes #409. 2010-10-06 00:35:10 +02:00
kimchy 1f49eb0b9d Gateway: Add `gateway.expected_nodes` for state recovery, closes #404. 2010-10-04 16:20:32 +02:00
kimchy c737b36df7 better handling of dynamic types in template mappings 2010-10-04 12:37:56 +02:00
kimchy 425744e0db add bzip2 compress lib 2010-10-03 16:24:10 +02:00
kimchy 7180636df6 add close to script services 2010-10-03 14:31:36 +02:00
kimchy c9a47a126f Scripts: Allow to access `_fields` providing access to stored fields, closes #402. 2010-10-03 12:51:41 +02:00
kimchy 5b8bc333bd JavaScript Plugin: Allow to use javascript for scripts, closes #401. 2010-10-03 02:20:37 +02:00
kimchy 5a7c8fe2cb allow script engines to have several types 2010-10-02 21:18:01 +02:00
kimchy a127499e3c doc 2010-10-02 21:07:59 +02:00
kimchy 31ebed11fd script handling refactoring, better support for languages that perform better with some caching on a per search aspect (and execution per doc) 2010-10-02 17:32:29 +02:00
kimchy 9e8ebd46e8 Groovy Plugin: Allow to run use groovy for scripts (where applicable), closes #398. 2010-10-02 01:22:05 +02:00
kimchy aa116e5d40 add lang parameter to all script constructs, allowing for (later) custom script languages 2010-10-01 22:14:20 +02:00
kimchy 99fcfde307 Mapper: Dynamic Template Support, closes #397. 2010-10-01 16:56:04 +02:00
kimchy f66f0218ca `/_cluster/nodes/stats` is broken in 0.11, closes #391. 2010-09-30 08:35:11 +02:00
kimchy 3b02bd3952 rename package 2010-09-30 08:17:33 +02:00
kimchy d3978383a5 when using keyword based analayzer on _all, an NPE is thrown since there is no current entry, ignore it (it does not make sense to have keywork analyzer on _all field...) 2010-09-28 14:53:37 +02:00
kimchy 55ae6b2001 fix possible problem when trying to convert a binary format to a string 2010-09-28 12:32:40 +02:00
kimchy 86e17eae2c fix auto conversion from source content type to rest content type 2010-09-28 00:01:35 +02:00
kimchy 1f3adc213f add nodes attributes to cluster state 2010-09-27 10:46:11 +02:00
kimchy d9c2cc9d37 allow to sepcify stopwords as comma delimiated list of words 2010-09-26 22:50:25 +02:00
kimchy ce28882660 add refresh option to index/create/delete opereation, REST allows for refresh parameter (defaults to false) 2010-09-26 09:07:37 +02:00
kimchy 2288c5d670 setHighlighterPostTags sets pre tags instead of post, closes #386. 2010-09-25 00:01:14 +02:00
kimchy ab9aa15bb4 add auto sync of translog when using local gateway 2010-09-24 21:25:10 +02:00
kimchy 1c2733ebee add flush to gateway 2010-09-24 12:29:52 +02:00
kimchy 485f904140 Default to not using compound file format for the index (increases number of open files, but faster indexing and recovery), closes #384. 2010-09-24 11:46:41 +02:00
kimchy 2f43af0bd7 use directly the directory and not the metaData API 2010-09-24 11:36:13 +02:00
kimchy 9bb86ea865 imrpvoe boolean filter performance 2010-09-24 00:59:26 +02:00
kimchy 31fd196687 doc the use of mapbuilder 2010-09-23 20:06:42 +02:00
kimchy 097071a9d2 add smart name filter wrapping for geo filters to automatically filter by type if as well if provided 2010-09-23 19:34:41 +02:00
kimchy 832914bf6d create a mapbuilder when listing files and not ImmutableMap builder since it seems like listFiles might return duplicate files. 2010-09-23 19:13:48 +02:00
kimchy 761639150d doc 2010-09-23 19:03:48 +02:00
kimchy c96014bb90 default the _river to 1/1 for better OOB experiance (single node recovery with local gateway support) 2010-09-23 02:46:13 +02:00
kimchy ecaaeb5250 River CouchDB Plugin, closes #382. 2010-09-23 02:26:12 +02:00
kimchy a59912db02 disable restart API 2010-09-23 00:02:12 +02:00
kimchy fd9266dbc6 fix wrong REST response generation of cluster state 2010-09-22 16:03:42 +02:00
kimchy 91344170ee doc 2010-09-22 15:39:30 +02:00
kimchy f301138e6f Change default thread pool to scaling from cached, closes #381. 2010-09-22 15:20:32 +02:00
kimchy 6d0829f331 imrpove shard target streaming, only do it when needed 2010-09-22 14:36:29 +02:00
kimchy cfdf6b0103 add support for op type in also in bulk 2010-09-22 00:07:45 +02:00
kimchy 3479f2a981 better exception message 2010-09-21 23:49:55 +02:00
kimchy 2cbcc8dd2e better handling of search context timeout 2010-09-21 23:17:46 +02:00
kimchy b9b91db5c8 building search response should handle cases where the sort is null valued 2010-09-21 22:13:27 +02:00
kimchy aa68667d63 rabbitmq river 2010-09-21 17:32:34 +02:00
kimchy 32acd88b2e default river index to have 1 shard with 2 replicas by default 2010-09-21 15:07:51 +02:00
kimchy 8dfac512ba move parsing of framed bulk binary data into the bulk request 2010-09-21 14:29:19 +02:00
kimchy 4fe7cbccca rename indexer to river 2010-09-21 13:35:09 +02:00
kimchy 69fad80b01 add rest response for allocations 2010-09-21 13:05:10 +02:00
kimchy a1299a3d4f refactoring in routing logic to allow adding allocation explanation that later can be shown 2010-09-21 11:39:22 +02:00
kimchy b5b36215ae refactoring in routing logic to allow adding allocation explanation that later can be shown 2010-09-21 11:37:36 +02:00
kimchy dbb2ae69b9 upgrade to latest mvel (2.0.18) 2010-09-21 01:57:41 +02:00
kimchy a68c00cd0d add special handling for primitive arrays as fields in xcontent builder 2010-09-21 01:41:35 +02:00
kimchy 121e548d76 add to nodes stats the total store size of the indices shards allocated to that node 2010-09-21 01:11:43 +02:00
kimchy 94a77b69d6 add a twitter indexer 2010-09-20 18:17:02 +02:00
kimchy eb4f4f99b3 add recover_after gateway tests 2010-09-20 09:44:57 +02:00
kimchy 3ec95f4e84 javadoc 2010-09-20 08:58:33 +02:00
kimchy 770bac252a fix local discovery to handle master nodes specifically better 2010-09-19 23:49:15 +02:00
kimchy 25246902cc Gateway: Add `recover_after_data_nodes` and `recover_after_master_nodes`, closes #376. 2010-09-19 22:46:13 +02:00
kimchy 95bb02b330 simple specific master nodes test 2010-09-19 21:11:53 +02:00
kimchy 8def6f59b3 simple specific master nodes test 2010-09-19 18:48:48 +02:00
kimchy 20b6688b0a make ping tests run faster using cluster state api 2010-09-19 18:13:03 +02:00
kimchy 589dbce89b move persistenting state and sharsd state in local gateway to another thread 2010-09-19 17:54:14 +02:00
kimchy f82ceb1e1e do snapshots on more lower level interval 2010-09-18 22:18:21 +02:00
kimchy e3948115f4 doc 2010-09-18 22:00:18 +02:00
kimchy 2372f481aa Shared Storage Gateway (fs/s3): Wrong snapshotting of transaction log, closes #375. 2010-09-18 21:33:38 +02:00
kimchy a2011e0151 no need to cache match all filter 2010-09-17 16:47:35 +02:00
kimchy cd7b84e808 improve match_all query to use a filter when possible 2010-09-17 16:38:27 +02:00
kimchy 3c95d6a215 more work on inexers 2010-09-17 16:36:12 +02:00
kimchy ff081240eb imrpove caching and fix delete mapping logic 2010-09-17 01:25:37 +02:00
kimchy 08d7125cd5 more work on indexer 2010-09-17 00:22:42 +02:00
kimchy 4b87f63bed Terms Facet: Allow to provide `_index` as the field, resulting in facets on indices, closes #374. 2010-09-16 15:11:30 +02:00
kimchy f49f3e169a Mapping API: Delete Mapping (with data), closes #373. 2010-09-16 14:35:07 +02:00
kimchy aef4704fed more indexer work 2010-09-16 13:22:09 +02:00
kimchy 0a6600818a add count at the end of the stress test 2010-09-15 23:51:25 +02:00
kimchy 09cdc1c379 don't wait for indices if there are none when using local gateway 2010-09-15 18:22:13 +02:00
kimchy e52daa9670 automatic index creation when using the bulk api 2010-09-15 18:18:48 +02:00
kimchy e93eb16deb docs and wrap the rest bulk request in an array 2010-09-15 17:59:10 +02:00
kimchy 5a5a892cc7 return the correct content type 2010-09-15 12:55:36 +02:00
kimchy 8db2a217c0 Expose internal search source builder 2010-09-15 12:43:21 +02:00
kimchy 3afe4da550 Bulk API, closes #371. 2010-09-15 12:22:07 +02:00
kimchy f6fa6ea44e more work on bulk, client API 2010-09-14 22:08:43 +02:00
kimchy d96ffe9153 internal bulk operaiton on the index shard 2010-09-14 01:46:44 +02:00
kimchy 85160ae341 Mapper: Upgrading a simple mapping to multi_field mapping fails on merge conflicts, closes #369. 2010-09-14 00:10:01 +02:00
kimchy 686b59c33f more work on indexer 2010-09-13 23:00:34 +02:00
kimchy 0c8dfd967c better raw generation of json 2010-09-13 02:43:39 +02:00
kimchy 2d5883ccb5 improve normalization performance from one content type to the other 2010-09-13 02:09:15 +02:00
kimchy 38aeba438d XContent refactoring, closes #367. 2010-09-12 23:20:15 +02:00
kimchy 4822517d5d add query parser cache to cache common query strings (+settings) with the parsed queries 2010-09-12 17:08:26 +02:00
kimchy 77a7938891 move to use guice to register also the default query parsers and filters 2010-09-12 16:46:57 +02:00
kimchy 2442e1fb16 improve query string parsing by reusing the same parser (bounded to the query parser context) 2010-09-12 15:39:07 +02:00
kimchy ed5ffd6591 Setting `path.work` does not affect the logs location, closes #366. 2010-09-11 22:22:49 +03:00
kimchy bc42352f7e change the local gateway metadata file from state- to metadata- 2010-09-11 21:45:45 +03:00
kimchy 0718757612 Query DSL: `match_all` filter, closes #365. 2010-09-11 16:01:30 +03:00
kimchy eccc7d5ef2 Search API: Allow to name filters and return per hit the filters it matched on, closes #364. 2010-09-11 12:38:19 +03:00
kimchy ab2a655a59 parsing a query should result in ParsedQuery object, so we can piggyback on it more parsing state / data 2010-09-11 00:34:22 +03:00
kimchy a47638dd3c imrove filter caching handling by not holding an implicit ref to the parent cache 2010-09-11 00:11:47 +03:00
kimchy de05a10ae5 use fastGet in docset 2010-09-10 16:16:45 +03:00
kimchy 4e661c165a Geo search across anti-meridian doesn't work, closes #363. 2010-09-10 16:04:10 +03:00
kimchy b31b0e979c Search failure when no results are found and sorting by score, closes #362. 2010-09-09 15:47:55 +03:00
kimchy 31362e4c60 upgrade to joda 1.6 2010-09-09 00:53:20 +03:00
kimchy a768016779 Allow to configure a common logger prefix using `es.logger.prefix` system prop, closes #360. 2010-09-08 12:16:41 +03:00
kimchy e151b83372 add simple dummy indexer 2010-09-07 18:19:02 +03:00
kimchy ad5945f141 Mapper: `geo_point` to support passing array of [lat, lon], closes #361. 2010-09-07 17:23:06 +03:00
kimchy c045b4d0ef Mapper: `geo_point` type to allow to set `store` on the mapping, closes #359. 2010-09-07 14:10:29 +03:00
kimchy 7c48f27e60 throw an exception when no lat/lon or geohash are found 2010-09-07 11:36:12 +03:00
kimchy 2cb45d4fe7 hack to work around complete text highlighting 2010-09-07 01:28:30 +03:00
kimchy bbb2033d3b start working on indexers 2010-09-07 00:41:30 +03:00
kimchy f270fc00d2 earch API: Automatically identify "script" fields on the field elements in search, close #357. 2010-09-06 11:51:31 +03:00
kimchy 243b7455e8 fix javadoc 2010-09-06 01:10:46 +03:00
kimchy cc1eac147a Allow to specify highlighter parameters on a per field level basis, closes #356. 2010-09-05 20:47:38 +03:00
Lukáš Vlček 8efe5dc9ae Highlighter enhancements. 2010-09-05 14:52:48 +03:00
kimchy c050bb58da allow to set a query using a string as well 2010-09-04 21:30:02 +03:00
kimchy 42b2e60af2 Create Index / Update Settings: Automatically prefix all settings with `index.`, closes #355. 2010-09-04 14:04:51 +03:00
kimchy 4f08801bb9 Thrift Plugin: Expose REST interface using thrift, closes #354. 2010-09-04 02:45:37 +03:00
kimchy 3d30b9c034 fix not called processModule on plugin for index and shard level modules 2010-09-03 19:06:11 +03:00
kimchy 1079941397 javadoc 2010-09-03 18:40:30 +03:00
kimchy b40f0bc5a4 Local Gateway: Expose `gateway.local.initial_shards` to control when to recover and index, closes #352. 2010-09-03 18:38:24 +03:00
kimchy 575f5b406b abstract away script execution 2010-09-03 01:49:06 +03:00
kimchy cd28afe950 Search API: Allow for script fields to extract parts of the stored _`source`, closes #349. 2010-09-02 21:29:41 +03:00
kimchy 1e75638b31 refactor field funtion to use internal docmap to be used in scripts 2010-09-02 17:19:50 +03:00
kimchy 807c485a38 make the cached compile scripts a soft map 2010-09-02 16:29:41 +03:00
kimchy 7bd08d638d JMX: Disable creating by default an RMI JMX connector, closes #348. 2010-09-02 14:17:53 +03:00
kimchy 0250896a09 reduce shard search response size by not serilaizing the node id, index, and shard id 2010-09-02 11:19:25 +03:00
kimchy 042af200e8 Gateway: Default to `local` gateway (replace `none` gateway), closes #347. 2010-09-01 23:21:26 +03:00
kimchy 0de24218ff improve local gateway, store started shards and metadata in separated files, metadata stored on master nodes, and started shards on data nodes. 2010-09-01 21:32:07 +03:00
kimchy 7709cd1bc2 improve GC logging 2010-09-01 15:02:56 +03:00
kimchy 7d6aa6c027 local gateway: don't store the local state when not a data/master node 2010-09-01 00:07:10 +03:00
kimchy ba5cd19989 gateway test to make sure it recovers from the latest version 2010-08-31 21:13:42 +03:00
kimchy a3efa21d71 Mapper: Failure to handle null valued objects ("obj1" : null), closes #341. 2010-08-31 13:33:21 +03:00
kimchy f3292f6778 upgrade netty version to 3.2.2 2010-08-30 23:57:37 +03:00
kimchy 289cfdf25a improve replica allocation, fix test to wait for recovery on node2 2010-08-30 23:37:09 +03:00
kimchy 9503fca2ae fix logging 2010-08-30 22:53:47 +03:00
kimchy 61764c5b69 add another test for local gateway, fix race when loading the current state of a node 2010-08-30 19:29:00 +03:00
kimchy 908fba44e7 fsync translog when closing and not deleting, allocate failed shard to another started shard 2010-08-30 16:55:57 +03:00
kimchy d9979f8dfe more work on local gateway, start integration test it 2010-08-30 01:25:45 +03:00
kimchy 8ed54c24bd change nodeFile to nodeLocation 2010-08-29 23:25:23 +03:00
kimchy 5b5d42445d move the shared fs gateway base class to fs package 2010-08-29 23:22:55 +03:00
kimchy 269616f35e more work on getting local gateway to work, apply transaction log operations when primary "recovering" 2010-08-29 16:56:04 +03:00
kimchy 4f4471483d initial work on local gateway 2010-08-29 01:24:23 +03:00
kimchy 163b7be639 refactor gateway service to allow for more custom gateway implemenations 2010-08-28 00:34:15 +03:00
kimchy a10ab7fe08 better exception thrown when failing to handle responses 2010-08-27 14:14:28 +03:00
kimchy e85c47db74 improve stack trace when failing to send a transaction request 2010-08-27 13:52:58 +03:00
kimchy e5b041c8ef simplify gateway snapshot operation 2010-08-27 00:41:55 +03:00
kimchy 19ddee5ec9 add a test for time interval 2010-08-26 18:12:51 +03:00
kimchy a49af6072b add objects option to range filter builder 2010-08-26 16:39:39 +03:00
kimchy d56b4d266e move query hint only to broadcast requests that use it 2010-08-26 14:31:54 +03:00
kimchy dfa24f6d03 improve caching of stored metadata fetched from nodes during allocation in order to reuse existing shards 2010-08-25 23:12:07 +03:00
kimchy 2910b6ab7f add callback for shards_started and shards_failed to better handle caching on node allocations level 2010-08-25 22:20:27 +03:00
kimchy 3ebd03f69a doc 2010-08-25 22:03:11 +03:00
kimchy 9d615a4f0b Query DSL: Improve `or` and `and` filters to accept just array of filters, closes #336. 2010-08-25 18:06:28 +03:00
kimchy c7a36ed431 add an exception when on filtesr are provided to "and" and "not" filters 2010-08-25 17:56:50 +03:00
kimchy fcdc5f872a improve (perf) getting last committed version of the index to create a new translog with 2010-08-25 15:12:26 +03:00
kimchy 9d08e56d8a cache nodes store metadata for on a shard basis when iteratring over the unassigned nodes 2010-08-25 15:03:25 +03:00
kimchy f36d80f66d don't go and find local storage for a shard on nodes if its not going to be allocated 2010-08-25 15:01:09 +03:00
kimchy 4bfd0a8c26 cache commit points during smart allocation based on local storage when recovering from gateway 2010-08-25 01:12:16 +03:00
kimchy 244cd42298 allow to use just hosts when using unicast discover, will use the transport level ports 2010-08-25 00:44:04 +03:00
kimchy fc812a306b change logging 2010-08-24 23:33:03 +03:00
kimchy ba5814a72f change log level 2010-08-24 23:25:20 +03:00
kimchy 31aed01220 increase timeout to 5 seconds 2010-08-24 23:18:58 +03:00
kimchy fdc82dd5a3 ignore parsing of wrong file format in the gateway (log warn), use index vesrion as the translog id 2010-08-24 23:12:18 +03:00
kimchy d0f5bc3403 add russian letter tokenizer 2010-08-24 23:11:22 +03:00
kimchy 49b4659322 Indexing Buffer Size: Refine default setting to 10% (from 40%) and add more settings, closes #334. 2010-08-24 18:12:45 +03:00
kimchy 7ae8d4c669 thread pool type setting is wrong, fix blocking max setting to have a minimum of 10, use cached TP where needed 2010-08-24 15:28:54 +03:00
kimchy b8ab50828c only list and delete files we created 2010-08-24 14:48:49 +03:00
kimchy df23d8f7ec don't snapshot a translog that has no content 2010-08-24 03:58:21 +03:00
kimchy 6c29142b91 throw recovery failure if failing to parse all commit points 2010-08-24 01:28:39 +03:00
kimchy b52d854711 smarter handling of throttling when preferring allocating to an existing work location 2010-08-23 21:39:54 +03:00
kimchy 78b6879ecd move rebalance only when active logic into node allocation, add canRebalance hook point 2010-08-23 18:28:35 +03:00
kimchy 149962691e move rebalance only when active logic into node allocation, add canRebalance hook point 2010-08-23 18:27:01 +03:00
kimchy 4300a6ca18 move rebalance only when active logic into node allocation, add canRebalance hook point 2010-08-23 18:26:17 +03:00
kimchy 0e47898902 add /_shutdown to the REST endpoints 2010-08-23 18:01:47 +03:00
kimchy 5900e01a0d list remote nodest storage information with a timeout 2010-08-23 18:00:37 +03:00
kimchy 1461da5b49 move prefetch allocation to be a NodeAllocation 2010-08-23 17:22:44 +03:00
kimchy edbd586ee5 add load test for recovery from gateway 2010-08-23 16:33:33 +03:00
kimchy 939d583550 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 15:42:03 +03:00
kimchy b49d331c86 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 15:28:12 +03:00
kimchy 62cc4d554a better logging message for reused vs. recovered from gateway 2010-08-23 15:25:06 +03:00
kimchy 5bd37f6f47 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 14:48:49 +03:00
kimchy 1517fa3d28 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 01:45:03 +03:00
kimchy 36ad3b246a addlow to add node allocation ot the module 2010-08-22 12:35:59 +03:00
kimchy 664f28492d improvements to the module system, allow for a module to spawn other modules, and pre process modules before injector creation 2010-08-22 10:42:25 +03:00
kimchy aa28b93610 refactor how throttling is done, instead of doing it after a shard is allocated to a node, and then wait till its allowed to recover, do it on the allocation level, and don't allocate a shard to a node that has N number of recoveries going on it 2010-08-22 02:47:34 +03:00
kimchy 7592862646 add a callback for allocation as well 2010-08-22 01:00:37 +03:00
kimchy a3852766ff only check for prefered allocation on data nodes 2010-08-22 00:32:43 +03:00
kimchy 3f701365b0 move the allocate replica after primary active logic to a node allocation 2010-08-22 00:14:56 +03:00
kimchy 5ded04c5cb refactor shard allocation to be more pluggable for node allocation "deciders" 2010-08-21 21:59:31 +03:00
kimchy 2865ceef85 more on sorted nodes when allocating shards above average number of shards per node 2010-08-21 19:55:38 +03:00
kimchy 76e1a6b1bf check also for negative values 2010-08-21 19:51:14 +03:00
kimchy 5d2c005622 pass to canAllocate (though nothing is done there now) the actual routing nodes 2010-08-21 17:52:24 +03:00
kimchy 1ddd969fa7 pass to canAllocate (though nothing is done there now) the actual routing nodes 2010-08-21 17:52:02 +03:00
kimchy 11dbbebb7a renamve var to better reflect its state (started, not active) 2010-08-21 17:50:08 +03:00
kimchy 3117341f44 Shards Allocation: Only rebalance a shard if all its instances are already active, closes #331. 2010-08-21 17:49:19 +03:00
kimchy 38e6649a7e Cluster Health API: Add `initializing_shards` and `unassigned_shards` to the response, closes #330. 2010-08-21 16:03:29 +03:00
kimchy 113ea1bb1b don't throttle gateway recovery based on peer recoveries (gateway recoveries are important to do as fast as possible). still throttle peer recoveries based on both. 2010-08-21 15:37:12 +03:00
kimchy e12bdd9faf rename RETRY stage in recovery to THROTTLE 2010-08-21 15:13:34 +03:00
kimchy 0c72bb2125 better writing of local md5 caches 2010-08-21 02:16:36 +03:00
kimchy 1ef1e9b25b don't spawn on received cluster state 2010-08-20 19:36:39 +03:00
kimchy 800e450e3a handle local md5 cases, don't include them in the list of files the directory has 2010-08-20 19:30:26 +03:00
kimchy 9acae60dbf also have a minimum value for concurrent recoveries, at 3 2010-08-20 17:28:31 +03:00
kimchy f62f7b8ffe check the progress in case the index size has not been set yet... 2010-08-20 17:17:32 +03:00
kimchy 9f0d5f8ef1 better default concurrent recoveries, based on number of processors, tap it at 10 2010-08-20 17:06:56 +03:00
kimchy 8c25be6dee better exception message with primary not started exception on index/delete/delete_by_query apis 2010-08-20 16:36:25 +03:00
kimchy 07285d81d1 add stream done when there is no file (should not happen) 2010-08-20 15:59:46 +03:00
kimchy b609162be3 don't ignore recovery on throttling unless the shard is closed 2010-08-20 15:58:24 +03:00
kimchy 4e74001bde move marking recovering before creating a status 2010-08-20 15:24:49 +03:00
kimchy 4c00e89f5e move memory monitor benchamrk to single indexing stress test 2010-08-20 15:05:21 +03:00
kimchy ef64f5d970 move memory monitor benchamrk to single indexing stress test 2010-08-20 13:41:33 +03:00
kimchy c0552bdc70 Can't set "no stopwords" on analyzer, closes #329. 2010-08-20 03:22:43 +03:00
kimchy 6f1a9fbfb9 create md5 files on the local index storage as well, so they won't have to be recomputed when performing initial recovery 2010-08-20 02:28:14 +03:00
kimchy acae0650c8 bail out of recovery when close and still waiting on throttling 2010-08-20 01:24:02 +03:00
kimchy a05c367c5e don't snapshot gateway when cleaning shard 2010-08-20 00:59:37 +03:00
kimchy 4a2e076c6a do a reuse work prefetching not on the cluster state thread, so it won't block while fetching md5s from other nodes 2010-08-20 00:45:17 +03:00
kimchy a5c5f65da5 fix recovery status when all index got reused 2010-08-19 22:46:58 +03:00
kimchy 7437acfcea add progress to indices status API 2010-08-19 22:29:31 +03:00
kimchy 8d0e5b239c recator source rest building into common code 2010-08-19 20:24:12 +03:00
kimchy 7b1093fb16 anoter compressed string test 2010-08-19 18:45:15 +03:00
kimchy 19abe7a2a5 when using compressed source, extract the compressed source directly into the response without any buffering 2010-08-19 18:29:36 +03:00
kimchy 46ccee8f89 when deleting an index, remove any blocks associated with it 2010-08-19 17:21:14 +03:00
kimchy c40935ae14 store the mapping definition in compressed form internally to save memory 2010-08-19 17:06:36 +03:00
kimchy a2d10d490d allocate list when deserializing, NPE galore... 2010-08-19 16:40:51 +03:00
kimchy 04ae1581a0 fix NPE introduced on optimization of index status on docs status creation 2010-08-19 15:37:06 +03:00
kimchy 70c344b079 add compressed string 2010-08-19 15:33:57 +03:00
kimchy 1a18387fab add `standard_html_strip` analyzer that combines the standard analyzer with html_strip char filter. 2010-08-19 15:10:31 +03:00
kimchy 22ea5e6608 fix primary election on replica relocation 2010-08-19 14:59:50 +03:00
kimchy 9c0bbe9bb5 log failure to notify master on shard started/failed 2010-08-19 13:49:48 +03:00
kimchy fd569a9523 renme tests to reflect replica and not backup 2010-08-19 13:11:53 +03:00
kimchy 27e92cd505 fix a case where node might not exists on relocating, don't fail the replication action (index / delete / ...) because of it 2010-08-19 13:08:20 +03:00
kimchy 51656552a5 better mehtod name 2010-08-19 11:20:06 +03:00
kimchy 965d7303cf index status action - some docs and extract common docs status 2010-08-19 09:34:02 +03:00
kimchy a3107bc5b1 add snapshot status (either current one or the latest one executed) to indices status api 2010-08-18 17:11:29 +03:00
kimchy 91aada2706 boolean values cannot be searched with expected queries, closes #327. 2010-08-18 15:11:23 +03:00
kimchy de71a1ce99 filtered more_like_this doesn't parse correctly, closes #326. 2010-08-18 14:36:34 +03:00
kimchy 2259ef671b Gateway: Failure to read full translog from the gateway, closes #328. 2010-08-18 14:29:46 +03:00
kimchy 3f9034b41c add gateway recovery status to the indices status API exposing both on going and summary when recovering from a gateway 2010-08-17 22:34:56 +03:00
kimchy 311520d146 add peer recovery status to the indices status API exposing both on going and summary when recovering from a peer shard 2010-08-17 21:33:11 +03:00
kimchy 5fb80c391b add peer recovery status to the indices status API exposing both on going and summary when recovering from a peer shard 2010-08-17 21:23:05 +03:00
kimchy 96fc16dddb add ability to iterate over assigned shards, and add an extension point to control shard routing iteration in the broadcast based action support 2010-08-17 14:04:12 +03:00
kimchy 7833cb1c76 Document IDs need to be URL decoded when indexed, closes #324. 2010-08-17 10:47:02 +03:00
kimchy c989d3a928 fix limit on UTF writing (the new mechanism) to be higher than short 2010-08-17 10:21:26 +03:00
kimchy e1f3fee4e4 internally store on going state of snapshot to gateway 2010-08-17 10:03:19 +03:00
kimchy 29e981d28d internally store on going state of recovery from gateway 2010-08-17 08:17:29 +03:00
kimchy 92fd9af2b9 more info stored on shard recovery process, start time and recovery time 2010-08-16 18:22:19 +03:00
kimchy 92aa91d7b2 update the current recvoerd files size in peer recovery 2010-08-16 09:11:51 +03:00
kimchy a012e0f1bd allow to configure translog batch size (internal) and compress translog operations when doing peer recvoery 2010-08-16 09:03:48 +03:00
kimchy 1bdce4c7ef capture current state of peer level recovery 2010-08-16 09:00:34 +03:00
kimchy 57ee1bdc55 improve utf based stream input / output handling 2010-08-15 21:19:36 +03:00
kimchy 14237317fc support compression of the source field 2010-08-15 17:57:14 +03:00
kimchy ee33ee457a allow to explicitly disable compression in peer recovery (defaults to true) 2010-08-15 15:13:45 +03:00
kimchy f6c58ff35d change default calibrate_size_by_deletes for merge policy to true from false (this is what upcoming lucene version will use, and makes much more sense) 2010-08-15 09:01:09 +03:00
kimchy 6d509a5e4e explicitly clean stream handles 2010-08-15 03:08:01 +03:00
kimchy c18904eb96 Recovery: Using transport compression explicitly when recovering an index from a peer shard, closes #322. 2010-08-15 03:01:35 +03:00
kimchy 1ee2f80e68 Transport: add global compression support compressing all internal transport communication (using lzf), closes #321. 2010-08-15 02:57:22 +03:00
kimchy c10544479f refactor cached streams into a separate class 2010-08-14 13:47:34 +03:00
kimchy 01b3d42cee separet recovery tests to one with small buffer size and on with bigger one 2010-08-14 13:44:42 +03:00
kimchy e1226626bc improve thread cached bytes allocation 2010-08-14 02:26:12 +03:00
kimchy 7c5f0fe405 minor transport refactoring, add options to send response, though no options to set yet 2010-08-14 02:25:58 +03:00
kimchy b3afca2589 remove compressor package, move lzf to a different package 2010-08-14 02:25:30 +03:00
kimchy 0adb2edba2 clean code 2010-08-14 01:23:04 +03:00
kimchy 75ee01a6b4 clean builder code when dealing with maps 2010-08-14 00:37:33 +03:00
kimchy 4a9c7d672e clean code 2010-08-13 23:50:16 +03:00
kimchy d12c757824 Gateway: Chunk based storage broken, fails to recover from gateway, closes #318. 2010-08-13 14:03:52 +03:00
kimchy 12ef12f7aa better handling of closing a shard / index while retrying recovery 2010-08-12 21:44:27 +03:00
kimchy 2bd9a63467 Query DSL: custom score script, allow to use `_score` as well as `score` as the underlying query score, closes #316. 2010-08-12 18:41:29 +03:00
kimchy 98bc8285ea Analysis: Add `char_filter` on top of `tokenizer`, `filter`, and `analyzer`. Add an `html_strip` char filter, closes #315. 2010-08-12 18:16:30 +03:00
kimchy e29925684a Term Facets: Add `reverse_cout` and `reverse_term` to `order` options, closes #314. 2010-08-12 15:26:26 +03:00
kimchy b1d1f1ff94 improve peer recovery shard state handling. when throttling a recovery, don't restore the non recovering state and then move to recovery again with each retry, keep the recovering state while retrying 2010-08-12 12:31:27 +03:00
kimchy c0a7dc327c range facet: from is used in to_str, should be to 2010-08-11 23:40:42 +03:00
kimchy a6d84f5942 no need to spawn a thread when response to a ping response 2010-08-11 13:20:13 +03:00
kimchy fbea92e688 REST API hang on a bogus call, closes #297. 2010-08-11 13:18:33 +03:00
kimchy ee26d55296 Remove memory monitor and move translog operations threshold to shard level setting, closes #312. 2010-08-11 12:54:00 +03:00
kimchy df4ece8cef remove no longer required commented code ;) 2010-08-11 11:43:52 +03:00
kimchy 9f8e5cb2c0 add a simple integration level test for scroll 2010-08-10 20:48:05 +03:00
kimchy 7977edd5db a filter based on the type of the document is built in several places in the code, which is wasteful. Instead, create a filter based on the type on the document mapper level, and reuse it where applicable. 2010-08-10 18:34:49 +03:00
kimchy 5f986ef422 build a match all filter internal implementation and use it where appropiate (instead of using query filter wrapper around a match all query) 2010-08-10 18:18:51 +03:00
kimchy f2018e2f86 Facets: Automatically filter by type for facets that use explicit field names that are prefixed by the type, closes #310. 2010-08-10 17:03:14 +03:00
kimchy 61bb9d0ff9 Mappers: also use `name` as a smart lookup for field/property on top of indexName and fullName, closes #309. 2010-08-10 16:14:15 +03:00
kimchy 879191c435 don't add the same object twice under rare cases where multiple threads are introducing the same new dynamic object 2010-08-10 09:18:53 +03:00
kimchy 4bb1baa9d4 remove nb structures 2010-08-10 00:05:38 +03:00
kimchy 05d07b036a remove ning async http client, no need for it 2010-08-10 00:00:24 +03:00
kimchy cacb5a54f1 upgrade to joda 1.6.1 2010-08-09 21:12:44 +03:00
kimchy f3cc8d1249 when serializing mappers into json, also serialize index, type and id mappings (index for the enabled flag, and id/type for the store) 2010-08-09 18:25:45 +03:00
kimchy f10699d5b8 javadoc 2010-08-09 09:06:14 +03:00
kimchy 6611e7db44 refactor custom sort parsing to be "code" pluggable 2010-08-09 08:57:56 +03:00
kimchy ff8d743337 add geo distance sort option, refactor java level sort API to have sort builders (similar to facets) 2010-08-08 21:46:11 +03:00
kimchy 8b19d353c1 add sort values as part of the response per search hit 2010-08-08 18:21:34 +03:00
kimchy 37af7f5aef add sort values as part of the response per search hit 2010-08-08 18:21:17 +03:00
kimchy 7cb2010dda add script support for terms facets, allowing to control if a certain term will be included in the facets result (boolean result), or muging of the term 2010-08-08 10:37:14 +03:00
kimchy ee1d50f8d8 add script support for terms facets, allowing to control if a certain term will be included in the facets result (boolean result), or muging of the term 2010-08-08 10:27:06 +03:00
kimchy 566ae94478 add the ability to control the "order" by of terms facet, can either be `term` or `count`. defaults to `count`. 2010-08-08 09:31:40 +03:00
kimchy d4547c629f New API, update settings, allowing to change specific runtime settings of the cluster (indices). `number_of_replicas` is the only settings supported, allowing to change the number of repliacs of one or more indices. 2010-08-08 09:17:09 +03:00
kimchy 6d3d9fd807 start work on being able to change the number of replicas at runtime 2010-08-05 11:00:02 +03:00
kimchy 860d8058b6 rename jmx settings to conform with casing (old casing also works) 2010-08-04 22:27:26 +03:00
kimchy 8c5fdf4aee Ability to return all stored fields with a *, closes #296. 2010-08-04 13:22:55 +03:00
kimchy d72de60b6f Add the ability to store the index name in the doc, closes #292. 2010-08-04 11:41:56 +03:00
kimchy a44d30bb61 After gateway recovery, mappings keep being applied on each cluster change, closes #295. 2010-08-04 09:02:10 +03:00
kimchy 959eb0e703 Geo: Polygon based filter, closes #294. 2010-08-03 21:52:13 +03:00
kimchy 39f344c11e change the format of bounding box filter to conform more to the "field_name" : { ... } structure 2010-08-03 19:18:16 +03:00
kimchy 91c256361e support for custom script based sorting 2010-08-03 18:14:02 +03:00
kimchy 3d31c38f11 refactoring transport service to allow for send options 2010-08-03 10:06:30 +03:00
kimchy 04781e038a when timeouting, and closed, raise onClose and not onTimeout 2010-08-03 10:05:19 +03:00
kimchy de8ffaf5fb Geo: bounding box filter, closes #290. 2010-08-03 01:51:46 +03:00
kimchy d1e404c1b3 change default geo point resolution from 32 to 64 2010-08-03 00:08:56 +03:00
kimchy 07ab5dcf9b reuse constants for facet type 2010-08-02 22:27:03 +03:00
kimchy dc6ef326d9 Facets: Filter based facet, closes #289. 2010-08-02 19:17:57 +03:00
kimchy 4e6087ee2d improve concurrency when loading field data 2010-08-02 18:20:49 +03:00
kimchy 7719dcd790 add an option to provide the array of ranges "on" the field name itself 2010-08-02 18:09:26 +03:00
kimchy 99af6df0ea add an option to provide the array of ranges "on" the field name itself 2010-08-02 16:39:28 +03:00
kimchy 6ed89ad406 fix comment 2010-08-02 16:35:08 +03:00
kimchy 32e26a6bd0 fix valueField name 2010-08-02 16:32:52 +03:00
kimchy 76cfbe6a44 support dates in range facet 2010-08-02 10:35:21 +03:00
kimchy 4bbbf186b6 fix some javadocs 2010-08-02 09:53:08 +03:00
kimchy 38fe6860d8 update docs 2010-08-02 09:39:17 +03:00
kimchy fdb0c14a17 upgrade to guava r06 2010-08-01 23:01:33 +03:00
kimchy 423ee159a5 upgrade to jackson 1.5.5 2010-08-01 22:51:15 +03:00
kimchy ef861a6b7b Search Facets: Range Facet, closes #287. 2010-08-01 20:08:27 +03:00
kimchy 9f441d8b9f add filter and global support 2010-08-01 17:09:40 +03:00
kimchy e77d1d0147 add filter and global support 2010-08-01 17:08:44 +03:00
kimchy c5811eb060 add a check for at least one range 2010-08-01 17:05:06 +03:00
kimchy c08791f816 fix type on xcontent result 2010-08-01 16:34:00 +03:00
kimchy 168363106b fix doc 2010-08-01 16:31:54 +03:00
kimchy b8b21a3363 Geo: `geo_distance` facet, closes #286. 2010-08-01 13:31:03 +03:00
kimchy dfb68c6310 Search fails when sorting on a field that has no values in the hits returned, closes #285. 2010-07-31 16:41:15 +03:00
kimchy a26b4f31e1 print the exception when failing to add a specific mapping during cluster state update 2010-07-31 15:41:11 +03:00
kimchy 484665af80 handle cases when there is no value 2010-07-31 02:30:52 +03:00
kimchy 9af288a42f fix test for existance of directory based mappings when creating an index 2010-07-29 19:22:21 +03:00
kimchy 0186a03684 add another script function test 2010-07-29 16:07:57 +03:00
kimchy 4f407e18aa Put Mapping: When using a single node and updating a mapping, it is not marked as `acknowledged`, closes #280. 2010-07-29 15:21:11 +03:00
kimchy e955e41a91 improve handling of failure when trying to read md5, should be ignored on any exception, not just io exception 2010-07-29 15:10:37 +03:00
kimchy 194e6cbff6 Geo support: geo_point type and geo_distance filter, closes #278, #279 2010-07-28 21:06:18 +03:00
kimchy af055f75ce should be static 2010-07-26 13:46:01 +03:00
kimchy f251e2fd32 support lt(N), le(N), gt(N), ge(N) in wait for nodes (thanks Lukas) 2010-07-26 13:44:29 +03:00
kimchy 65284ba2ba Terms Factes: Allow to provide regex controlling which terms should be included, closes #277. 2010-07-26 12:23:03 +03:00
kimchy 03cd2f3772 fix <= 2010-07-26 00:22:21 +03:00
kimchy ac7c8cb650 Analysis: Add pattern analyzer, closes #276. 2010-07-25 22:40:50 +03:00
kimchy 477a24efc6 Mapping: Revise dynamic mapping (into default), merge default to new mappings, closes #275. 2010-07-25 21:31:16 +03:00
kimchy 1884c4219a Mapping: Dynamic mapping definitions are ignored, closes #274. 2010-07-25 12:03:04 +03:00
kimchy 457b56937e small improvements to groovy client, allows for closure in the search request 2010-07-25 10:32:22 +03:00
kimchy 77b6d1d8b8 remove cloud plugin, reimplement an AWS specific cloud plugin with S3 gateway and EC2 discovery 2010-07-24 23:39:43 +03:00
kimchy a6a993ff08 fix ignored exception 2010-07-24 18:48:28 +03:00
kimchy 0072dd816b nicer logging, and use sync streams 2010-07-23 23:10:59 +03:00
kimchy 23ddf8bc53 do a reroute when a node becomes master 2010-07-23 20:55:26 +03:00
kimchy f6255f8ad9 change logging level to debug 2010-07-23 19:29:14 +03:00
kimchy 72682a6730 always retry on connect exception with master operations 2010-07-23 01:59:56 +03:00
kimchy 9283e2a7ad local discovery should update the local metadata state once it started 2010-07-23 01:39:37 +03:00
kimchy 5706e5f6b9 rename node close to node closed exception 2010-07-23 01:12:36 +03:00
kimchy 34b84b0e61 when retrying on primary shard when node disconnected, add it again the the cluster service as a listener 2010-07-23 01:07:09 +03:00
kimchy 8875b9641b cleanup test 2010-07-23 00:50:08 +03:00
kimchy 2a3130c649 REST Search API: Change `score` to `_score` to denote sorting by hit score, closes #271. 2010-07-22 23:39:55 +03:00
kimchy 42c2b6b743 throw a proper exception when no type is found 2010-07-22 21:17:11 +03:00
kimchy c1ab9f290f remove compile warnings 2010-07-22 00:25:25 +03:00
kimchy b55fd0c0fc verify that data that is written to the blob (in fs) is the same as expected in the blob write api call 2010-07-21 23:46:11 +03:00
kimchy 4dcf219086 after raising a failure, return, don't try and write the blob 2010-07-21 23:45:15 +03:00
kimchy 2432a7cb5b fixed highlighted styled built in schema tags to properly increate from hlt1 to hlt10 2010-07-21 20:38:13 +03:00
kimchy 8ec7ee66a9 Search: Sending a request that fails to parse can cause file leaks, closes #270. 2010-07-21 16:59:58 +03:00
kimchy 0cb97e4044 cancel the timeout in any case 2010-07-21 16:38:37 +03:00
kimchy 8d45901a8a change logging level, nodes FD will detect it as well 2010-07-21 16:30:40 +03:00
kimchy 48d33ec70a Cluster Health API: Add `wait_for_nodes` (accepts "N", "<N", ">N", "<=N", and ">=N"), closes #269. 2010-07-21 16:29:44 +03:00
kimchy 2a0e8d4ec9 better state when closing shard, and handling its state 2010-07-21 09:20:28 +03:00
kimchy 3629540953 when trying to rejoin the cluster, clean the routing/metadata so the node will get fresh shards 2010-07-21 08:55:35 +03:00
kimchy 15cf7a2361 initialize the nodes before a possible retry, so we have the local node id for it 2010-07-21 01:53:00 +03:00
kimchy 39e546b64d add a proper master not discovered exception for master based operations (cluster state) and the ability to control the timeout till master is found 2010-07-21 01:51:58 +03:00
kimchy a2ceaa91cc improve retry when closing the node performing an operation 2010-07-20 22:34:09 +03:00
kimchy 1ff8b43aa7 clean timeout listener from on going listeners once its removed 2010-07-20 21:11:56 +03:00
kimchy f4064038e4 improve load test 2010-07-20 20:53:14 +03:00
kimchy b3b5c9b0cc don't execute it on a thread pool 2010-07-20 18:34:01 +03:00
kimchy 553e8134ee create a send transaction exception on the calling thread and fill its stack, so we know where its coming from 2010-07-20 18:33:44 +03:00
kimchy 07480846c3 fix logging to actually pass the actual logging parameters 2010-07-20 18:33:12 +03:00
kimchy 6915b39dc3 clean timeout listener task when removing the listener 2010-07-20 18:32:49 +03:00
kimchy 45e54c1705 add retry if master is not connected till master is connected or a timeout has passed to master based operations (like cluster health) 2010-07-20 18:32:21 +03:00
kimchy 45c821316b no need for volatile on writer, its always accessed under a lock 2010-07-20 13:58:01 +03:00
kimchy 3c8cf68a17 improve timeout cluster service listener 2010-07-20 13:50:19 +03:00
kimchy 4ac1409afa don't list connect exceptions 2010-07-20 11:01:08 +03:00
kimchy 679f3758b7 ignore a more genric exception 2010-07-20 09:29:10 +03:00
kimchy 3867591ffa clean up the java api a bit 2010-07-20 09:18:16 +03:00
kimchy 2ee870e01e nicer logging, append space before index + shard id logging 2010-07-20 08:53:57 +03:00
kimchy 6f2c0f4e55 spelling 2010-07-19 17:51:42 +03:00
kimchy 5f0470d68b Support Cross-Origin resource in http/rest module, closes #218. 2010-07-19 15:06:13 +03:00
kimchy e48b1d98db more recovery under load tests, now with nodes shutdown, ignore node connected exception or retry when performing replicated operations 2010-07-19 11:33:21 +03:00
kimchy 9def68a733 add full rolling restart recvoery test 2010-07-19 10:08:12 +03:00
kimchy df602054fb fix relocation of primaries and perform the operation on the relocated primary replica as well 2010-07-19 09:54:52 +03:00
kimchy 0d20790ffe when shutting down a node, close shard (and indices) in parallel 2010-07-19 02:17:59 +03:00
kimchy d657d4447b more shard relocation tests, create a cached executor that can be used to allocate threads that are not affected by the actual thread pool impl (can always fork, no blocking) 2010-07-19 01:45:09 +03:00
kimchy f9cd7cb932 add a recovery test while under indexing load 2010-07-18 23:57:57 +03:00
kimchy 9c9dfe0b39 add a recovery test while under indexing load 2010-07-18 23:40:13 +03:00
kimchy fefcfb5b5c refactor recovery to be handled on the node level (and not per shard), with better retry mechanism when doing peer shard recovery 2010-07-18 22:54:21 +03:00
kimchy 8ac8dd818d now that we have a file based transaction log, increate the memory clean threshold 2010-07-18 21:24:52 +03:00
kimchy 49f8501a5b nicer serilization 2010-07-17 22:58:16 +03:00
kimchy 910ad2a408 fix optimization to not do sorting when searching with score "desc" sorting 2010-07-17 19:38:04 +03:00
kimchy 08ca383fd5 more work on cloud gateway 2010-07-17 15:20:13 +03:00
kimchy 6194b3ab0f add NaN handling for specific hit score as well 2010-07-17 13:52:31 +03:00
kimchy 1fccaf06e9 start work on reimplementing cloud gateway 2010-07-17 13:12:53 +03:00
kimchy e61dc78c21 max_score == NaN, closes #263. 2010-07-17 01:39:12 +03:00
kimchy 3006183191 fix a problem where max_score was NaN if there were no hits from a shard (though there are hits from other shards), it will still be NaN where there are no hits 2010-07-15 22:58:18 +03:00
kimchy cf998e2f89 protected when not of the same type mapper when merging 2010-07-15 22:35:18 +03:00
kimchy 9f6fde9adb better handling of failed merge because of different types 2010-07-15 20:40:40 +03:00
kimchy cfa56cb3d6 Put Mapping: When updating existing mappings, the request returns with acknowledged `false`, closes #262. 2010-07-15 20:03:44 +03:00
kimchy c987e8a534 Querying mapping on a non-master throws an error, closes #261. 2010-07-15 19:03:45 +03:00
kimchy ee60d7dedc better exception message when failing to parse a specific field 2010-07-15 18:46:12 +03:00
kimchy ca8ad83092 only count data_nodes with recover_after_nodes parameter 2010-07-15 01:12:00 +03:00
kimchy 61fadb4dc0 use safe method (to get proper exception) to get shard injector 2010-07-14 23:39:19 +03:00
kimchy f6fa056f1c fix REST api for cluster state and the ability to filter parts of the data returned 2010-07-14 16:05:47 +03:00
kimchy 95ba62f83d Translog: Implement a file system based translog and make it the default, closes #260. 2010-07-14 14:53:58 +03:00
kimchy e7d80b8244 remove max_shards_per_node setting (actually, you could not set it even...), will be revisted when proper SLA based shard allocation will be implemented 2010-07-14 14:29:56 +03:00
kimchy a6bd64f30d Facet results vary depending on size, closes #259. 2010-07-14 11:30:39 +03:00
kimchy 2704ab3d69 improve join process in cluster, fetch the cluster meta-data on join and handle new meta data 2010-07-14 10:34:22 +03:00
kimchy 141506afc5 improve join process in cluster, fetch the cluster meta-data on join and handle new meta data 2010-07-14 09:28:56 +03:00
kimchy 2e2f22fbd6 nullify the updater on stop 2010-07-14 08:55:57 +03:00
kimchy c4d6ccd07b nicer logging 2010-07-14 08:41:42 +03:00
kimchy 234455530a Make merging mappings smarter, closes #253. 2010-07-14 07:32:17 +03:00
kimchy 892dadca59 refactor meta data opertions into discrete services that do all operations within the cluster update process 2010-07-13 18:01:57 +03:00
kimchy ed04721f08 doc 2010-07-13 10:19:57 +03:00
kimchy f0cf552bc5 sorting breaks when sorting on a field that has no value in some documents 2010-07-13 09:38:55 +03:00
kimchy e6bd3f2693 Put Mapping: Fail when an analyzer is specified that was not configured, closes #252. 2010-07-12 23:18:32 +03:00
kimchy 65dffdf9cf throw a descriptive exception when trying to sort on multi token or multi values field per doc 2010-07-12 23:13:37 +03:00
kimchy ff17183798 add more sort tests 2010-07-12 22:25:49 +03:00
kimchy fe50a6f64e improve allocation of shards based on existing work location, wait with index creation on other nodes, so listing their respective existing location will work 2010-07-12 01:33:38 +03:00
kimchy 6722e05418 only list unallocated shards on data nodes 2010-07-11 22:11:06 +03:00
kimchy 52ac24fa23 more debug statr on recovery from gateway 2010-07-11 21:51:45 +03:00
kimchy e7a8da8236 do flush within the finalization of index shard recovery 2010-07-11 21:20:36 +03:00
kimchy 30b34b975c fix spelling 2010-07-11 20:53:03 +03:00
kimchy 294f09a1d7 Shutdown API: Improve behavior when shutting down the whole cluster, closes #250. 2010-07-11 20:41:58 +03:00
kimchy 09493691a7 nicer serializtion 2010-07-11 15:41:12 +03:00
kimchy 671a14ce87 freq is always positive int 2010-07-11 15:35:53 +03:00
kimchy a6ef805aa8 add doc 2010-07-11 00:40:29 +03:00
kimchy ef369fc40f javadoc 2010-07-10 23:21:34 +03:00
kimchy d0cdbeffba only send intiial state recived after sending join request, just when we actually get a new cluster state that includes us. Also, handle no nodes to send to with generic transport nodes action. 2010-07-10 15:43:49 +03:00
kimchy a0ead02299 Don't create / use the work directory if not needed (for example, on client / non data) nodes, closes #249. 2010-07-10 14:29:39 +03:00
kimchy 7ba3b3a9eb Zen Discovery: Control which nodes are allowed to become masters, closes #248. 2010-07-09 04:27:13 +03:00
kimchy b657ffc5e7 Zen Discovery: Control which nodes are allowed to become masters, closes #248. 2010-07-09 04:22:03 +03:00
kimchy d1f5577c40 add channel based read from fs snapshot, reusing the same file descriptor with direct position based reads 2010-07-09 03:51:36 +03:00
kimchy 6cdd6a015c reuse file buffer 2010-07-09 03:24:31 +03:00
kimchy d531d82cfb Zen Discovery: A node might get into an infinite state of trying to find a master (when client / non_master) nodes exists, closes #247. 2010-07-09 00:43:06 +03:00
kimchy 87eb6bed8f nicer logging 2010-07-08 22:53:41 +03:00
kimchy 720e6a6d5b nicer logging 2010-07-08 22:52:03 +03:00
kimchy be6aaa157f fixed a bug in fs translog where it wasn't seeking correctly, and not counting the number of operations 2010-07-08 22:18:26 +03:00
kimchy ef148077fc better logging 2010-07-08 21:37:10 +03:00
kimchy 84af9f5681 improve handling of translog snapshotting 2010-07-08 20:51:37 +03:00
kimchy 4429a61528 better cleanup on shard startup failure 2010-07-08 19:20:03 +03:00
kimchy b69fc265c5 index writer ram buffer size can only be 2G, make sure not to set an illegal value 2010-07-08 19:03:55 +03:00
kimchy 400ec7b8d1 index writer ram buffer size can only be 2G, make sure not to set an illegal value 2010-07-08 18:58:04 +03:00
kimchy 63deef42cf add a help appendable blob store based on immutable one 2010-07-08 18:53:40 +03:00
kimchy 8d669ff54e move sorting to be done using the new fielddata cache, and not using lucene built in FieldCache 2010-07-08 14:42:35 +03:00
kimchy 97b5b41522 enable eager cache eviction based on readers, requires a nasty hack to Lucene, basically overriding two classes to hook into when a reader actually gets closed (and only enabled when elasticsearch is before lucene in the classpath, which we take for in the startup scripts) 2010-07-07 17:15:26 +03:00
kimchy a36b06c682 ignore failed snapshot on a non started index 2010-07-07 16:59:33 +03:00
kimchy a8c317e355 fix terms facets test to take into account distributed terms request 2010-07-07 16:33:39 +03:00
kimchy 5ff46911b6 ignore recovery when no node to recover from (it has disconnected, for example) 2010-07-07 16:30:28 +03:00
kimchy 3fb68c52da ignore snapshot on a shard thas has just been created 2010-07-07 16:30:17 +03:00
kimchy 11a8d637c0 ignore snapshot on a shard thas has just been created 2010-07-07 16:26:19 +03:00
kimchy e40aa540fe ignore recovery when no node to recover from (it has disconnected, for example) 2010-07-07 16:25:55 +03:00
kimchy 9578ad3ef1 Terms Facets: Allow to specify a set of terms to exclude in the request, closes #246. 2010-07-07 14:41:34 +03:00
kimchy 2b5458daf6 implement a simple file system based translog, not the defeault yet, should provide better memory management 2010-07-07 09:30:07 +03:00
kimchy b2bdb149f9 log shard close failure when closing an index 2010-07-06 22:50:40 +03:00
kimchy 6aa9be238c use the atomic integer counter to represent the size 2010-07-06 18:45:19 +03:00
kimchy 267016758d improve handling of memory caching with file system, only force compound file when really needed (when an extension that exists within the compound file is part of the memory cached extensions) 2010-07-06 18:21:36 +03:00
kimchy ac01bb86e8 clean lock only before opening an index writer 2010-07-06 18:13:05 +03:00
kimchy 2908418d37 checn the default byte buffer cache to 200mb, now that its global per node 2010-07-06 17:58:30 +03:00
kimchy 070cb5b295 allocate replica shards (backups) only after the primaries have been allocated, this results in better theoretical shard allocation, and the ability to reuse local storage index files (comared against the backup) 2010-07-06 17:58:05 +03:00
kimchy 1d39bb4d51 cache md5 computation for unallocated shard and reused it 2010-07-06 09:00:24 +03:00
kimchy d4f86899e3 more work on reuse work dir, improved transaction log to move to stream based 2010-07-06 00:12:40 +03:00
kimchy b078c9206a more work on reuse work dir, added md5 logic to identify what need to be recovered or not, note, requires reindexing 2010-07-05 14:13:55 +03:00
kimchy 79b69764e0 doc 2010-07-01 15:03:11 +03:00
kimchy e8b261c165 better shutdown handling 2010-07-01 10:28:57 +03:00
kimchy ad5525f575 fix unwrapping of exceptions 2010-07-01 10:27:53 +03:00
kimchy f6c9c49645 better logging when failing to read full content of a transport message 2010-07-01 03:00:57 +03:00
kimchy c75a56ca17 better shutdown handling 2010-07-01 02:41:00 +03:00
kimchy c2d02e4e3a wrong serialization of shardId 2010-07-01 02:18:35 +03:00
kimchy d848ab3df3 rename executors to esexecutors 2010-07-01 00:06:34 +03:00
kimchy b0d736b489 make the shard routing an actual component 2010-06-30 23:39:27 +03:00
kimchy 811559f393 add another test if the pool size is greater than the core pool size before a worker can exit 2010-06-30 23:08:05 +03:00
kimchy 6709518c4e add the ability to list shard index blobs without actually having one created 2010-06-30 22:54:59 +03:00
kimchy 66096e97f2 Remove the terms API (replaced by facets in search), closes #242. 2010-06-30 19:49:41 +03:00
kimchy df7e3f55a4 improve cluster blocks data structures for simpler usage 2010-06-30 16:06:31 +03:00
kimchy 4049da97fb time value to support parsing days as well 2010-06-30 13:00:25 +03:00
kimchy 088e0b5a64 Automatic management of indexing buffer size, closes #241. 2010-06-30 01:46:57 +03:00
kimchy fdb2eff998 Automatic management of indexing buffer size, closes #241. 2010-06-29 22:54:54 +03:00
kimchy 03ac38fdba better benchmark on memory monitor 2010-06-29 22:12:43 +03:00
kimchy 3f6cd46736 allow to configure tcp network modules (transport, memcached and http) to use blocking io (just enabling it to do some performance tests) 2010-06-29 21:07:38 +03:00
kimchy 01ff81fa89 fix log message 2010-06-29 19:49:28 +03:00
kimchy bc803b4fcf close the thread pool on test 2010-06-29 18:35:03 +03:00