honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,688 Commits 7 Branches 6 Tags 546 MiB
Commit Graph

1491 Commits

Author SHA1 Message Date
kimchy 5635260922 remove check on if indices has an index, cluster state is enough 2010-10-19 16:42:39 +02:00
kimchy 4c8978237f move operation routing to be cluster level and not index level 2010-10-19 16:23:39 +02:00
kimchy 1eee7f381a dont case node names and index names 2010-10-18 19:03:48 +02:00
kimchy 5c760d1e8c don't case the index name in cluster health, use string constants in xcontent generation 2010-10-18 18:55:38 +02:00
kimchy 3a52c2f7f4 Improve automatic publish network address logic, closes #439. 2010-10-18 15:19:24 +02:00
kimchy 019c844dd1 Plugins: plugins should now be in extracted format under `ES_HOME/plugins`, closes #438. 2010-10-18 11:14:16 +02:00
kimchy 0a3d187e6a Mapper: Allow to configure `date_formats` only on the root object mapper, closes #437. 2010-10-17 19:56:58 +02:00
kimchy 69ecf8b66b rollback commit, allocation should only happen when index is not recovered 2010-10-17 19:43:30 +02:00
kimchy 9a09261383 remove checking for client node on rivers, its automatically done with master and non river node anyhow 2010-10-17 19:20:37 +02:00
kimchy 648da2e6ff add cluster name to cluster health response 2010-10-17 18:58:59 +02:00
kimchy 5d54e36964 Statistical Facet: Allow to compute statistical facets on more than one field, closes #436. 2010-10-17 16:22:20 +02:00
kimchy 2fc0022477 clean code 2010-10-17 02:33:12 +02:00
kimchy caa6495e2b doc 2010-10-17 01:52:23 +02:00
kimchy 4708d31040 add the option to control filter caching on each filter in the java filter builders 2010-10-16 16:12:38 +02:00
kimchy d9f966d83c Query DSL: Numeric Range Filter - A filter that uses the field data cache to perform numeric tests, closes #435. 2010-10-16 15:38:43 +02:00
kimchy cc3a1dbe90 move geo handling (filter) into the index package from common.lucene since it uses field data and not really only dependant on lucene constucts 2010-10-16 12:30:20 +02:00
kimchy 1587a3815e more options on stress rolling restart test, variable number of fields, variable length of text 2010-10-16 03:26:11 +02:00
kimchy b09675dcfa add a stress test for rolling restart nodes 2010-10-16 01:05:25 +02:00
kimchy bbb05a57e0 add the ability to run check index before starting a shard, used for internal testing (as it has a large overhead) 2010-10-15 20:56:30 +02:00
kimchy 6c9120a51b Local Gateway: Don't block meta operations (delete index) on an index that is not recovered due to not all shards being available, closes #433. 2010-10-15 11:04:54 +02:00
kimchy ae05ce0551 Scripting: Allow to define scripts within `config/scripts`, automatically compiled and can be referenced by name, closes #429. 2010-10-14 16:14:23 +02:00
kimchy e0488fa87e Plugins: Allow to place "extracted" plugins under the `plugins` directory, closes #428. 2010-10-14 14:04:00 +02:00
kimchy 8d533e8a99 dynamic tempaltes are now named (change to the mapping definition) to allow for simpler merging 2010-10-14 12:56:50 +02:00
kimchy ad01f19db8 move dynamic tempaltes to only allow to be defined on root object mapping (so easily reusable and simpler merging) #397. 2010-10-14 12:25:32 +02:00
kimchy 1578da404c improve geo internal cache of lat/lon, stored two double arrays instead of an array of GeoPoints. 2010-10-13 22:03:48 +02:00
kimchy 9077bb6528 Geo Overhaul (work with multiple locations), closes #414. Change (again) how geo location is stored, not using geohash but explicit double conversion 2010-10-13 03:35:35 +02:00
kimchy 8a8a4d648a Query DSL: Fine grained filter caching control with sensible defaults, closes #424. 2010-10-13 01:37:43 +02:00
kimchy 3fa214653e increase precision to 16 from 12, omit norms / tf on geohash 2010-10-12 22:30:56 +02:00
kimchy c3cb5a3e34 Use Base64 encoding for UUID generation (auto generated doc ids, and node ids), closes #422. 2010-10-12 00:17:01 +02:00
kimchy 4f9d9c1395 add test for merging two stats field facet using same facet name 2010-10-11 00:40:37 +02:00
kimchy 18d8e9dcdd add more script tests 2010-10-10 20:05:37 +02:00
kimchy a14b73bc13 Bulk API: Rename `index` to `_index`, `type` to `_type` and `id` to `_id`, closes #421. 2010-10-10 19:31:56 +02:00
kimchy c15a612b8a optimize get/delete/index response xcontent generation 2010-10-09 04:06:03 +02:00
kimchy 087b4e6f23 optimize search response xcontent generation 2010-10-09 04:01:17 +02:00
kimchy f64e636415 add xcontent string that will allow to speed up json/smile serialization 2010-10-09 03:06:01 +02:00
kimchy bd962ba693 upgrade to jackson 1.6.1 2010-10-09 01:51:40 +02:00
kimchy 6314c2460c Geo Overhaul (work with multiple locations), closes #414. 2010-10-08 22:31:21 +02:00
kimchy 523a8b4c3e rename xonctent mappers to just mappers 2010-10-08 15:05:07 +02:00
kimchy a61f3a939b add option to reuse double[] when decoding geohash 2010-10-07 23:58:49 +02:00
kimchy 0f6beeb263 reuse search lookup data for the two remaining cases (custom score and script filter) by having a "current" search context to access 2010-10-07 23:18:26 +02:00
kimchy 34ed85a40f internal refactoring of field data type 2010-10-07 18:41:17 +02:00
kimchy d0bf743ab4 Query DSL: query_string / field to use the optimized match_all query when using * (or *:*), closes #413. 2010-10-07 16:48:24 +02:00
kimchy cf5ed1d177 Term Facet: Use a script that provides the facets to index, closes #410. 2010-10-07 01:49:12 +02:00
kimchy 4c34840627 fix dynamic tempaltes persistency and merging (so it will be there across restarts, and when a new mapping is merged) 2010-10-07 00:44:53 +02:00
kimchy 4422a4c48a doc 2010-10-06 14:07:34 +02:00
kimchy 22ed71e2e6 allow to execute term facet on more than one field 2010-10-06 13:39:02 +02:00
kimchy 31d94b19a1 Local Gateway: Possible failure to allocate shards to nodes when more than one index exists in the cluster (on full cluster restart), closes #409. 2010-10-06 00:35:10 +02:00
kimchy 1f49eb0b9d Gateway: Add `gateway.expected_nodes` for state recovery, closes #404. 2010-10-04 16:20:32 +02:00
kimchy c737b36df7 better handling of dynamic types in template mappings 2010-10-04 12:37:56 +02:00
kimchy 425744e0db add bzip2 compress lib 2010-10-03 16:24:10 +02:00
kimchy 7180636df6 add close to script services 2010-10-03 14:31:36 +02:00
kimchy c9a47a126f Scripts: Allow to access `_fields` providing access to stored fields, closes #402. 2010-10-03 12:51:41 +02:00
kimchy 5b8bc333bd JavaScript Plugin: Allow to use javascript for scripts, closes #401. 2010-10-03 02:20:37 +02:00
kimchy 5a7c8fe2cb allow script engines to have several types 2010-10-02 21:18:01 +02:00
kimchy a127499e3c doc 2010-10-02 21:07:59 +02:00
kimchy 31ebed11fd script handling refactoring, better support for languages that perform better with some caching on a per search aspect (and execution per doc) 2010-10-02 17:32:29 +02:00
kimchy 9e8ebd46e8 Groovy Plugin: Allow to run use groovy for scripts (where applicable), closes #398. 2010-10-02 01:22:05 +02:00
kimchy aa116e5d40 add lang parameter to all script constructs, allowing for (later) custom script languages 2010-10-01 22:14:20 +02:00
kimchy 99fcfde307 Mapper: Dynamic Template Support, closes #397. 2010-10-01 16:56:04 +02:00
kimchy f66f0218ca `/_cluster/nodes/stats` is broken in 0.11, closes #391. 2010-09-30 08:35:11 +02:00
kimchy 3b02bd3952 rename package 2010-09-30 08:17:33 +02:00
kimchy d3978383a5 when using keyword based analayzer on _all, an NPE is thrown since there is no current entry, ignore it (it does not make sense to have keywork analyzer on _all field...) 2010-09-28 14:53:37 +02:00
kimchy 55ae6b2001 fix possible problem when trying to convert a binary format to a string 2010-09-28 12:32:40 +02:00
kimchy 86e17eae2c fix auto conversion from source content type to rest content type 2010-09-28 00:01:35 +02:00
kimchy 1f3adc213f add nodes attributes to cluster state 2010-09-27 10:46:11 +02:00
kimchy d9c2cc9d37 allow to sepcify stopwords as comma delimiated list of words 2010-09-26 22:50:25 +02:00
kimchy ce28882660 add refresh option to index/create/delete opereation, REST allows for refresh parameter (defaults to false) 2010-09-26 09:07:37 +02:00
kimchy 2288c5d670 setHighlighterPostTags sets pre tags instead of post, closes #386. 2010-09-25 00:01:14 +02:00
kimchy ab9aa15bb4 add auto sync of translog when using local gateway 2010-09-24 21:25:10 +02:00
kimchy 1c2733ebee add flush to gateway 2010-09-24 12:29:52 +02:00
kimchy 485f904140 Default to not using compound file format for the index (increases number of open files, but faster indexing and recovery), closes #384. 2010-09-24 11:46:41 +02:00
kimchy 2f43af0bd7 use directly the directory and not the metaData API 2010-09-24 11:36:13 +02:00
kimchy 9bb86ea865 imrpvoe boolean filter performance 2010-09-24 00:59:26 +02:00
kimchy 31fd196687 doc the use of mapbuilder 2010-09-23 20:06:42 +02:00
kimchy 097071a9d2 add smart name filter wrapping for geo filters to automatically filter by type if as well if provided 2010-09-23 19:34:41 +02:00
kimchy 832914bf6d create a mapbuilder when listing files and not ImmutableMap builder since it seems like listFiles might return duplicate files. 2010-09-23 19:13:48 +02:00
kimchy 761639150d doc 2010-09-23 19:03:48 +02:00
kimchy c96014bb90 default the _river to 1/1 for better OOB experiance (single node recovery with local gateway support) 2010-09-23 02:46:13 +02:00
kimchy ecaaeb5250 River CouchDB Plugin, closes #382. 2010-09-23 02:26:12 +02:00
kimchy a59912db02 disable restart API 2010-09-23 00:02:12 +02:00
kimchy fd9266dbc6 fix wrong REST response generation of cluster state 2010-09-22 16:03:42 +02:00
kimchy 91344170ee doc 2010-09-22 15:39:30 +02:00
kimchy f301138e6f Change default thread pool to scaling from cached, closes #381. 2010-09-22 15:20:32 +02:00
kimchy 6d0829f331 imrpove shard target streaming, only do it when needed 2010-09-22 14:36:29 +02:00
kimchy cfdf6b0103 add support for op type in also in bulk 2010-09-22 00:07:45 +02:00
kimchy 3479f2a981 better exception message 2010-09-21 23:49:55 +02:00
kimchy 2cbcc8dd2e better handling of search context timeout 2010-09-21 23:17:46 +02:00
kimchy b9b91db5c8 building search response should handle cases where the sort is null valued 2010-09-21 22:13:27 +02:00
kimchy aa68667d63 rabbitmq river 2010-09-21 17:32:34 +02:00
kimchy 32acd88b2e default river index to have 1 shard with 2 replicas by default 2010-09-21 15:07:51 +02:00
kimchy 8dfac512ba move parsing of framed bulk binary data into the bulk request 2010-09-21 14:29:19 +02:00
kimchy 4fe7cbccca rename indexer to river 2010-09-21 13:35:09 +02:00
kimchy 69fad80b01 add rest response for allocations 2010-09-21 13:05:10 +02:00
kimchy a1299a3d4f refactoring in routing logic to allow adding allocation explanation that later can be shown 2010-09-21 11:39:22 +02:00
kimchy b5b36215ae refactoring in routing logic to allow adding allocation explanation that later can be shown 2010-09-21 11:37:36 +02:00
kimchy dbb2ae69b9 upgrade to latest mvel (2.0.18) 2010-09-21 01:57:41 +02:00
kimchy a68c00cd0d add special handling for primitive arrays as fields in xcontent builder 2010-09-21 01:41:35 +02:00
kimchy 121e548d76 add to nodes stats the total store size of the indices shards allocated to that node 2010-09-21 01:11:43 +02:00
kimchy 94a77b69d6 add a twitter indexer 2010-09-20 18:17:02 +02:00
kimchy eb4f4f99b3 add recover_after gateway tests 2010-09-20 09:44:57 +02:00
kimchy 3ec95f4e84 javadoc 2010-09-20 08:58:33 +02:00
kimchy 770bac252a fix local discovery to handle master nodes specifically better 2010-09-19 23:49:15 +02:00
kimchy 25246902cc Gateway: Add `recover_after_data_nodes` and `recover_after_master_nodes`, closes #376. 2010-09-19 22:46:13 +02:00
kimchy 95bb02b330 simple specific master nodes test 2010-09-19 21:11:53 +02:00
kimchy 8def6f59b3 simple specific master nodes test 2010-09-19 18:48:48 +02:00
kimchy 20b6688b0a make ping tests run faster using cluster state api 2010-09-19 18:13:03 +02:00
kimchy 589dbce89b move persistenting state and sharsd state in local gateway to another thread 2010-09-19 17:54:14 +02:00
kimchy f82ceb1e1e do snapshots on more lower level interval 2010-09-18 22:18:21 +02:00
kimchy e3948115f4 doc 2010-09-18 22:00:18 +02:00
kimchy 2372f481aa Shared Storage Gateway (fs/s3): Wrong snapshotting of transaction log, closes #375. 2010-09-18 21:33:38 +02:00
kimchy a2011e0151 no need to cache match all filter 2010-09-17 16:47:35 +02:00
kimchy cd7b84e808 improve match_all query to use a filter when possible 2010-09-17 16:38:27 +02:00
kimchy 3c95d6a215 more work on inexers 2010-09-17 16:36:12 +02:00
kimchy ff081240eb imrpove caching and fix delete mapping logic 2010-09-17 01:25:37 +02:00
kimchy 08d7125cd5 more work on indexer 2010-09-17 00:22:42 +02:00
kimchy 4b87f63bed Terms Facet: Allow to provide `_index` as the field, resulting in facets on indices, closes #374. 2010-09-16 15:11:30 +02:00
kimchy f49f3e169a Mapping API: Delete Mapping (with data), closes #373. 2010-09-16 14:35:07 +02:00
kimchy aef4704fed more indexer work 2010-09-16 13:22:09 +02:00
kimchy 0a6600818a add count at the end of the stress test 2010-09-15 23:51:25 +02:00
kimchy 09cdc1c379 don't wait for indices if there are none when using local gateway 2010-09-15 18:22:13 +02:00
kimchy e52daa9670 automatic index creation when using the bulk api 2010-09-15 18:18:48 +02:00
kimchy e93eb16deb docs and wrap the rest bulk request in an array 2010-09-15 17:59:10 +02:00
kimchy 5a5a892cc7 return the correct content type 2010-09-15 12:55:36 +02:00
kimchy 8db2a217c0 Expose internal search source builder 2010-09-15 12:43:21 +02:00
kimchy 3afe4da550 Bulk API, closes #371. 2010-09-15 12:22:07 +02:00
kimchy f6fa6ea44e more work on bulk, client API 2010-09-14 22:08:43 +02:00
kimchy d96ffe9153 internal bulk operaiton on the index shard 2010-09-14 01:46:44 +02:00
kimchy 85160ae341 Mapper: Upgrading a simple mapping to multi_field mapping fails on merge conflicts, closes #369. 2010-09-14 00:10:01 +02:00
kimchy 686b59c33f more work on indexer 2010-09-13 23:00:34 +02:00
kimchy 0c8dfd967c better raw generation of json 2010-09-13 02:43:39 +02:00
kimchy 2d5883ccb5 improve normalization performance from one content type to the other 2010-09-13 02:09:15 +02:00
kimchy 38aeba438d XContent refactoring, closes #367. 2010-09-12 23:20:15 +02:00
kimchy 4822517d5d add query parser cache to cache common query strings (+settings) with the parsed queries 2010-09-12 17:08:26 +02:00
kimchy 77a7938891 move to use guice to register also the default query parsers and filters 2010-09-12 16:46:57 +02:00
kimchy 2442e1fb16 improve query string parsing by reusing the same parser (bounded to the query parser context) 2010-09-12 15:39:07 +02:00
kimchy ed5ffd6591 Setting `path.work` does not affect the logs location, closes #366. 2010-09-11 22:22:49 +03:00
kimchy bc42352f7e change the local gateway metadata file from state- to metadata- 2010-09-11 21:45:45 +03:00
kimchy 0718757612 Query DSL: `match_all` filter, closes #365. 2010-09-11 16:01:30 +03:00
kimchy eccc7d5ef2 Search API: Allow to name filters and return per hit the filters it matched on, closes #364. 2010-09-11 12:38:19 +03:00
kimchy ab2a655a59 parsing a query should result in ParsedQuery object, so we can piggyback on it more parsing state / data 2010-09-11 00:34:22 +03:00
kimchy a47638dd3c imrove filter caching handling by not holding an implicit ref to the parent cache 2010-09-11 00:11:47 +03:00
kimchy de05a10ae5 use fastGet in docset 2010-09-10 16:16:45 +03:00
kimchy 4e661c165a Geo search across anti-meridian doesn't work, closes #363. 2010-09-10 16:04:10 +03:00
kimchy b31b0e979c Search failure when no results are found and sorting by score, closes #362. 2010-09-09 15:47:55 +03:00
kimchy 31362e4c60 upgrade to joda 1.6 2010-09-09 00:53:20 +03:00
kimchy a768016779 Allow to configure a common logger prefix using `es.logger.prefix` system prop, closes #360. 2010-09-08 12:16:41 +03:00
kimchy e151b83372 add simple dummy indexer 2010-09-07 18:19:02 +03:00
kimchy ad5945f141 Mapper: `geo_point` to support passing array of [lat, lon], closes #361. 2010-09-07 17:23:06 +03:00
kimchy c045b4d0ef Mapper: `geo_point` type to allow to set `store` on the mapping, closes #359. 2010-09-07 14:10:29 +03:00
kimchy 7c48f27e60 throw an exception when no lat/lon or geohash are found 2010-09-07 11:36:12 +03:00
kimchy 2cb45d4fe7 hack to work around complete text highlighting 2010-09-07 01:28:30 +03:00
kimchy bbb2033d3b start working on indexers 2010-09-07 00:41:30 +03:00
kimchy f270fc00d2 earch API: Automatically identify "script" fields on the field elements in search, close #357. 2010-09-06 11:51:31 +03:00
kimchy 243b7455e8 fix javadoc 2010-09-06 01:10:46 +03:00
kimchy cc1eac147a Allow to specify highlighter parameters on a per field level basis, closes #356. 2010-09-05 20:47:38 +03:00
Lukáš Vlček 8efe5dc9ae Highlighter enhancements. 2010-09-05 14:52:48 +03:00
kimchy c050bb58da allow to set a query using a string as well 2010-09-04 21:30:02 +03:00
kimchy 42b2e60af2 Create Index / Update Settings: Automatically prefix all settings with `index.`, closes #355. 2010-09-04 14:04:51 +03:00
kimchy 4f08801bb9 Thrift Plugin: Expose REST interface using thrift, closes #354. 2010-09-04 02:45:37 +03:00
kimchy 3d30b9c034 fix not called processModule on plugin for index and shard level modules 2010-09-03 19:06:11 +03:00
kimchy 1079941397 javadoc 2010-09-03 18:40:30 +03:00
kimchy b40f0bc5a4 Local Gateway: Expose `gateway.local.initial_shards` to control when to recover and index, closes #352. 2010-09-03 18:38:24 +03:00
kimchy 575f5b406b abstract away script execution 2010-09-03 01:49:06 +03:00
kimchy cd28afe950 Search API: Allow for script fields to extract parts of the stored _`source`, closes #349. 2010-09-02 21:29:41 +03:00
kimchy 1e75638b31 refactor field funtion to use internal docmap to be used in scripts 2010-09-02 17:19:50 +03:00
kimchy 807c485a38 make the cached compile scripts a soft map 2010-09-02 16:29:41 +03:00
kimchy 7bd08d638d JMX: Disable creating by default an RMI JMX connector, closes #348. 2010-09-02 14:17:53 +03:00
kimchy 0250896a09 reduce shard search response size by not serilaizing the node id, index, and shard id 2010-09-02 11:19:25 +03:00
kimchy 042af200e8 Gateway: Default to `local` gateway (replace `none` gateway), closes #347. 2010-09-01 23:21:26 +03:00
kimchy 0de24218ff improve local gateway, store started shards and metadata in separated files, metadata stored on master nodes, and started shards on data nodes. 2010-09-01 21:32:07 +03:00
kimchy 7709cd1bc2 improve GC logging 2010-09-01 15:02:56 +03:00
kimchy 7d6aa6c027 local gateway: don't store the local state when not a data/master node 2010-09-01 00:07:10 +03:00
kimchy ba5cd19989 gateway test to make sure it recovers from the latest version 2010-08-31 21:13:42 +03:00
kimchy a3efa21d71 Mapper: Failure to handle null valued objects ("obj1" : null), closes #341. 2010-08-31 13:33:21 +03:00
kimchy f3292f6778 upgrade netty version to 3.2.2 2010-08-30 23:57:37 +03:00
kimchy 289cfdf25a improve replica allocation, fix test to wait for recovery on node2 2010-08-30 23:37:09 +03:00
kimchy 9503fca2ae fix logging 2010-08-30 22:53:47 +03:00
kimchy 61764c5b69 add another test for local gateway, fix race when loading the current state of a node 2010-08-30 19:29:00 +03:00
kimchy 908fba44e7 fsync translog when closing and not deleting, allocate failed shard to another started shard 2010-08-30 16:55:57 +03:00
kimchy d9979f8dfe more work on local gateway, start integration test it 2010-08-30 01:25:45 +03:00
kimchy 8ed54c24bd change nodeFile to nodeLocation 2010-08-29 23:25:23 +03:00
kimchy 5b5d42445d move the shared fs gateway base class to fs package 2010-08-29 23:22:55 +03:00
kimchy 269616f35e more work on getting local gateway to work, apply transaction log operations when primary "recovering" 2010-08-29 16:56:04 +03:00
kimchy 4f4471483d initial work on local gateway 2010-08-29 01:24:23 +03:00
kimchy 163b7be639 refactor gateway service to allow for more custom gateway implemenations 2010-08-28 00:34:15 +03:00
kimchy a10ab7fe08 better exception thrown when failing to handle responses 2010-08-27 14:14:28 +03:00
kimchy e85c47db74 improve stack trace when failing to send a transaction request 2010-08-27 13:52:58 +03:00
kimchy e5b041c8ef simplify gateway snapshot operation 2010-08-27 00:41:55 +03:00
kimchy 19ddee5ec9 add a test for time interval 2010-08-26 18:12:51 +03:00
kimchy a49af6072b add objects option to range filter builder 2010-08-26 16:39:39 +03:00
kimchy d56b4d266e move query hint only to broadcast requests that use it 2010-08-26 14:31:54 +03:00
kimchy dfa24f6d03 improve caching of stored metadata fetched from nodes during allocation in order to reuse existing shards 2010-08-25 23:12:07 +03:00
kimchy 2910b6ab7f add callback for shards_started and shards_failed to better handle caching on node allocations level 2010-08-25 22:20:27 +03:00
kimchy 3ebd03f69a doc 2010-08-25 22:03:11 +03:00
kimchy 9d615a4f0b Query DSL: Improve `or` and `and` filters to accept just array of filters, closes #336. 2010-08-25 18:06:28 +03:00
kimchy c7a36ed431 add an exception when on filtesr are provided to "and" and "not" filters 2010-08-25 17:56:50 +03:00
kimchy fcdc5f872a improve (perf) getting last committed version of the index to create a new translog with 2010-08-25 15:12:26 +03:00
kimchy 9d08e56d8a cache nodes store metadata for on a shard basis when iteratring over the unassigned nodes 2010-08-25 15:03:25 +03:00
kimchy f36d80f66d don't go and find local storage for a shard on nodes if its not going to be allocated 2010-08-25 15:01:09 +03:00
kimchy 4bfd0a8c26 cache commit points during smart allocation based on local storage when recovering from gateway 2010-08-25 01:12:16 +03:00
kimchy 244cd42298 allow to use just hosts when using unicast discover, will use the transport level ports 2010-08-25 00:44:04 +03:00
kimchy fc812a306b change logging 2010-08-24 23:33:03 +03:00
kimchy ba5814a72f change log level 2010-08-24 23:25:20 +03:00
kimchy 31aed01220 increase timeout to 5 seconds 2010-08-24 23:18:58 +03:00
kimchy fdc82dd5a3 ignore parsing of wrong file format in the gateway (log warn), use index vesrion as the translog id 2010-08-24 23:12:18 +03:00
kimchy d0f5bc3403 add russian letter tokenizer 2010-08-24 23:11:22 +03:00
kimchy 49b4659322 Indexing Buffer Size: Refine default setting to 10% (from 40%) and add more settings, closes #334. 2010-08-24 18:12:45 +03:00
kimchy 7ae8d4c669 thread pool type setting is wrong, fix blocking max setting to have a minimum of 10, use cached TP where needed 2010-08-24 15:28:54 +03:00
kimchy b8ab50828c only list and delete files we created 2010-08-24 14:48:49 +03:00
kimchy df23d8f7ec don't snapshot a translog that has no content 2010-08-24 03:58:21 +03:00
kimchy 6c29142b91 throw recovery failure if failing to parse all commit points 2010-08-24 01:28:39 +03:00
kimchy b52d854711 smarter handling of throttling when preferring allocating to an existing work location 2010-08-23 21:39:54 +03:00
kimchy 78b6879ecd move rebalance only when active logic into node allocation, add canRebalance hook point 2010-08-23 18:28:35 +03:00
kimchy 149962691e move rebalance only when active logic into node allocation, add canRebalance hook point 2010-08-23 18:27:01 +03:00
kimchy 4300a6ca18 move rebalance only when active logic into node allocation, add canRebalance hook point 2010-08-23 18:26:17 +03:00
kimchy 0e47898902 add /_shutdown to the REST endpoints 2010-08-23 18:01:47 +03:00
kimchy 5900e01a0d list remote nodest storage information with a timeout 2010-08-23 18:00:37 +03:00
kimchy 1461da5b49 move prefetch allocation to be a NodeAllocation 2010-08-23 17:22:44 +03:00
kimchy edbd586ee5 add load test for recovery from gateway 2010-08-23 16:33:33 +03:00
kimchy 939d583550 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 15:42:03 +03:00
kimchy b49d331c86 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 15:28:12 +03:00
kimchy 62cc4d554a better logging message for reused vs. recovered from gateway 2010-08-23 15:25:06 +03:00
kimchy 5bd37f6f47 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 14:48:49 +03:00
kimchy 1517fa3d28 initial chunk of work on refactoring the gateway, syntatic gateway files names, commit points that include translog information 2010-08-23 01:45:03 +03:00
kimchy 36ad3b246a addlow to add node allocation ot the module 2010-08-22 12:35:59 +03:00
kimchy 664f28492d improvements to the module system, allow for a module to spawn other modules, and pre process modules before injector creation 2010-08-22 10:42:25 +03:00
kimchy aa28b93610 refactor how throttling is done, instead of doing it after a shard is allocated to a node, and then wait till its allowed to recover, do it on the allocation level, and don't allocate a shard to a node that has N number of recoveries going on it 2010-08-22 02:47:34 +03:00
kimchy 7592862646 add a callback for allocation as well 2010-08-22 01:00:37 +03:00
kimchy a3852766ff only check for prefered allocation on data nodes 2010-08-22 00:32:43 +03:00
kimchy 3f701365b0 move the allocate replica after primary active logic to a node allocation 2010-08-22 00:14:56 +03:00
kimchy 5ded04c5cb refactor shard allocation to be more pluggable for node allocation "deciders" 2010-08-21 21:59:31 +03:00
kimchy 2865ceef85 more on sorted nodes when allocating shards above average number of shards per node 2010-08-21 19:55:38 +03:00
kimchy 76e1a6b1bf check also for negative values 2010-08-21 19:51:14 +03:00
kimchy 5d2c005622 pass to canAllocate (though nothing is done there now) the actual routing nodes 2010-08-21 17:52:24 +03:00
kimchy 1ddd969fa7 pass to canAllocate (though nothing is done there now) the actual routing nodes 2010-08-21 17:52:02 +03:00
kimchy 11dbbebb7a renamve var to better reflect its state (started, not active) 2010-08-21 17:50:08 +03:00
kimchy 3117341f44 Shards Allocation: Only rebalance a shard if all its instances are already active, closes #331. 2010-08-21 17:49:19 +03:00
kimchy 38e6649a7e Cluster Health API: Add `initializing_shards` and `unassigned_shards` to the response, closes #330. 2010-08-21 16:03:29 +03:00
kimchy 113ea1bb1b don't throttle gateway recovery based on peer recoveries (gateway recoveries are important to do as fast as possible). still throttle peer recoveries based on both. 2010-08-21 15:37:12 +03:00
kimchy e12bdd9faf rename RETRY stage in recovery to THROTTLE 2010-08-21 15:13:34 +03:00
kimchy 0c72bb2125 better writing of local md5 caches 2010-08-21 02:16:36 +03:00
kimchy 1ef1e9b25b don't spawn on received cluster state 2010-08-20 19:36:39 +03:00
kimchy 800e450e3a handle local md5 cases, don't include them in the list of files the directory has 2010-08-20 19:30:26 +03:00
kimchy 9acae60dbf also have a minimum value for concurrent recoveries, at 3 2010-08-20 17:28:31 +03:00
kimchy f62f7b8ffe check the progress in case the index size has not been set yet... 2010-08-20 17:17:32 +03:00
kimchy 9f0d5f8ef1 better default concurrent recoveries, based on number of processors, tap it at 10 2010-08-20 17:06:56 +03:00
kimchy 8c25be6dee better exception message with primary not started exception on index/delete/delete_by_query apis 2010-08-20 16:36:25 +03:00
kimchy 07285d81d1 add stream done when there is no file (should not happen) 2010-08-20 15:59:46 +03:00
kimchy b609162be3 don't ignore recovery on throttling unless the shard is closed 2010-08-20 15:58:24 +03:00
kimchy 4e74001bde move marking recovering before creating a status 2010-08-20 15:24:49 +03:00
kimchy 4c00e89f5e move memory monitor benchamrk to single indexing stress test 2010-08-20 15:05:21 +03:00
kimchy ef64f5d970 move memory monitor benchamrk to single indexing stress test 2010-08-20 13:41:33 +03:00
kimchy c0552bdc70 Can't set "no stopwords" on analyzer, closes #329. 2010-08-20 03:22:43 +03:00
kimchy 6f1a9fbfb9 create md5 files on the local index storage as well, so they won't have to be recomputed when performing initial recovery 2010-08-20 02:28:14 +03:00
kimchy acae0650c8 bail out of recovery when close and still waiting on throttling 2010-08-20 01:24:02 +03:00
kimchy a05c367c5e don't snapshot gateway when cleaning shard 2010-08-20 00:59:37 +03:00
kimchy 4a2e076c6a do a reuse work prefetching not on the cluster state thread, so it won't block while fetching md5s from other nodes 2010-08-20 00:45:17 +03:00
kimchy a5c5f65da5 fix recovery status when all index got reused 2010-08-19 22:46:58 +03:00
kimchy 7437acfcea add progress to indices status API 2010-08-19 22:29:31 +03:00
kimchy 8d0e5b239c recator source rest building into common code 2010-08-19 20:24:12 +03:00
kimchy 7b1093fb16 anoter compressed string test 2010-08-19 18:45:15 +03:00
kimchy 19abe7a2a5 when using compressed source, extract the compressed source directly into the response without any buffering 2010-08-19 18:29:36 +03:00
kimchy 46ccee8f89 when deleting an index, remove any blocks associated with it 2010-08-19 17:21:14 +03:00
kimchy c40935ae14 store the mapping definition in compressed form internally to save memory 2010-08-19 17:06:36 +03:00
kimchy a2d10d490d allocate list when deserializing, NPE galore... 2010-08-19 16:40:51 +03:00
kimchy 04ae1581a0 fix NPE introduced on optimization of index status on docs status creation 2010-08-19 15:37:06 +03:00
kimchy 70c344b079 add compressed string 2010-08-19 15:33:57 +03:00
kimchy 1a18387fab add `standard_html_strip` analyzer that combines the standard analyzer with html_strip char filter. 2010-08-19 15:10:31 +03:00
kimchy 22ea5e6608 fix primary election on replica relocation 2010-08-19 14:59:50 +03:00
kimchy 9c0bbe9bb5 log failure to notify master on shard started/failed 2010-08-19 13:49:48 +03:00
kimchy fd569a9523 renme tests to reflect replica and not backup 2010-08-19 13:11:53 +03:00
kimchy 27e92cd505 fix a case where node might not exists on relocating, don't fail the replication action (index / delete / ...) because of it 2010-08-19 13:08:20 +03:00
kimchy 51656552a5 better mehtod name 2010-08-19 11:20:06 +03:00
kimchy 965d7303cf index status action - some docs and extract common docs status 2010-08-19 09:34:02 +03:00
kimchy a3107bc5b1 add snapshot status (either current one or the latest one executed) to indices status api 2010-08-18 17:11:29 +03:00
kimchy 91aada2706 boolean values cannot be searched with expected queries, closes #327. 2010-08-18 15:11:23 +03:00
kimchy de71a1ce99 filtered more_like_this doesn't parse correctly, closes #326. 2010-08-18 14:36:34 +03:00
kimchy 2259ef671b Gateway: Failure to read full translog from the gateway, closes #328. 2010-08-18 14:29:46 +03:00
kimchy 3f9034b41c add gateway recovery status to the indices status API exposing both on going and summary when recovering from a gateway 2010-08-17 22:34:56 +03:00
kimchy 311520d146 add peer recovery status to the indices status API exposing both on going and summary when recovering from a peer shard 2010-08-17 21:33:11 +03:00
kimchy 5fb80c391b add peer recovery status to the indices status API exposing both on going and summary when recovering from a peer shard 2010-08-17 21:23:05 +03:00
kimchy 96fc16dddb add ability to iterate over assigned shards, and add an extension point to control shard routing iteration in the broadcast based action support 2010-08-17 14:04:12 +03:00
kimchy 7833cb1c76 Document IDs need to be URL decoded when indexed, closes #324. 2010-08-17 10:47:02 +03:00
kimchy c989d3a928 fix limit on UTF writing (the new mechanism) to be higher than short 2010-08-17 10:21:26 +03:00
kimchy e1f3fee4e4 internally store on going state of snapshot to gateway 2010-08-17 10:03:19 +03:00
kimchy 29e981d28d internally store on going state of recovery from gateway 2010-08-17 08:17:29 +03:00
kimchy 92fd9af2b9 more info stored on shard recovery process, start time and recovery time 2010-08-16 18:22:19 +03:00
kimchy 92aa91d7b2 update the current recvoerd files size in peer recovery 2010-08-16 09:11:51 +03:00
kimchy a012e0f1bd allow to configure translog batch size (internal) and compress translog operations when doing peer recvoery 2010-08-16 09:03:48 +03:00
kimchy 1bdce4c7ef capture current state of peer level recovery 2010-08-16 09:00:34 +03:00
kimchy 57ee1bdc55 improve utf based stream input / output handling 2010-08-15 21:19:36 +03:00
kimchy 14237317fc support compression of the source field 2010-08-15 17:57:14 +03:00
kimchy ee33ee457a allow to explicitly disable compression in peer recovery (defaults to true) 2010-08-15 15:13:45 +03:00
kimchy f6c58ff35d change default calibrate_size_by_deletes for merge policy to true from false (this is what upcoming lucene version will use, and makes much more sense) 2010-08-15 09:01:09 +03:00
kimchy 6d509a5e4e explicitly clean stream handles 2010-08-15 03:08:01 +03:00
kimchy c18904eb96 Recovery: Using transport compression explicitly when recovering an index from a peer shard, closes #322. 2010-08-15 03:01:35 +03:00
kimchy 1ee2f80e68 Transport: add global compression support compressing all internal transport communication (using lzf), closes #321. 2010-08-15 02:57:22 +03:00
kimchy c10544479f refactor cached streams into a separate class 2010-08-14 13:47:34 +03:00
kimchy 01b3d42cee separet recovery tests to one with small buffer size and on with bigger one 2010-08-14 13:44:42 +03:00
kimchy e1226626bc improve thread cached bytes allocation 2010-08-14 02:26:12 +03:00
kimchy 7c5f0fe405 minor transport refactoring, add options to send response, though no options to set yet 2010-08-14 02:25:58 +03:00
kimchy b3afca2589 remove compressor package, move lzf to a different package 2010-08-14 02:25:30 +03:00
kimchy 0adb2edba2 clean code 2010-08-14 01:23:04 +03:00
kimchy 75ee01a6b4 clean builder code when dealing with maps 2010-08-14 00:37:33 +03:00
kimchy 4a9c7d672e clean code 2010-08-13 23:50:16 +03:00
kimchy d12c757824 Gateway: Chunk based storage broken, fails to recover from gateway, closes #318. 2010-08-13 14:03:52 +03:00
kimchy 12ef12f7aa better handling of closing a shard / index while retrying recovery 2010-08-12 21:44:27 +03:00
kimchy 2bd9a63467 Query DSL: custom score script, allow to use `_score` as well as `score` as the underlying query score, closes #316. 2010-08-12 18:41:29 +03:00
kimchy 98bc8285ea Analysis: Add `char_filter` on top of `tokenizer`, `filter`, and `analyzer`. Add an `html_strip` char filter, closes #315. 2010-08-12 18:16:30 +03:00
kimchy e29925684a Term Facets: Add `reverse_cout` and `reverse_term` to `order` options, closes #314. 2010-08-12 15:26:26 +03:00
kimchy b1d1f1ff94 improve peer recovery shard state handling. when throttling a recovery, don't restore the non recovering state and then move to recovery again with each retry, keep the recovering state while retrying 2010-08-12 12:31:27 +03:00
kimchy c0a7dc327c range facet: from is used in to_str, should be to 2010-08-11 23:40:42 +03:00
kimchy a6d84f5942 no need to spawn a thread when response to a ping response 2010-08-11 13:20:13 +03:00
kimchy fbea92e688 REST API hang on a bogus call, closes #297. 2010-08-11 13:18:33 +03:00
kimchy ee26d55296 Remove memory monitor and move translog operations threshold to shard level setting, closes #312. 2010-08-11 12:54:00 +03:00
kimchy df4ece8cef remove no longer required commented code ;) 2010-08-11 11:43:52 +03:00
kimchy 9f8e5cb2c0 add a simple integration level test for scroll 2010-08-10 20:48:05 +03:00
kimchy 7977edd5db a filter based on the type of the document is built in several places in the code, which is wasteful. Instead, create a filter based on the type on the document mapper level, and reuse it where applicable. 2010-08-10 18:34:49 +03:00
kimchy 5f986ef422 build a match all filter internal implementation and use it where appropiate (instead of using query filter wrapper around a match all query) 2010-08-10 18:18:51 +03:00
kimchy f2018e2f86 Facets: Automatically filter by type for facets that use explicit field names that are prefixed by the type, closes #310. 2010-08-10 17:03:14 +03:00
kimchy 61bb9d0ff9 Mappers: also use `name` as a smart lookup for field/property on top of indexName and fullName, closes #309. 2010-08-10 16:14:15 +03:00
kimchy 879191c435 don't add the same object twice under rare cases where multiple threads are introducing the same new dynamic object 2010-08-10 09:18:53 +03:00
kimchy 4bb1baa9d4 remove nb structures 2010-08-10 00:05:38 +03:00
kimchy 05d07b036a remove ning async http client, no need for it 2010-08-10 00:00:24 +03:00
kimchy cacb5a54f1 upgrade to joda 1.6.1 2010-08-09 21:12:44 +03:00
kimchy f3cc8d1249 when serializing mappers into json, also serialize index, type and id mappings (index for the enabled flag, and id/type for the store) 2010-08-09 18:25:45 +03:00
kimchy f10699d5b8 javadoc 2010-08-09 09:06:14 +03:00
kimchy 6611e7db44 refactor custom sort parsing to be "code" pluggable 2010-08-09 08:57:56 +03:00
kimchy ff8d743337 add geo distance sort option, refactor java level sort API to have sort builders (similar to facets) 2010-08-08 21:46:11 +03:00
kimchy 8b19d353c1 add sort values as part of the response per search hit 2010-08-08 18:21:34 +03:00
kimchy 37af7f5aef add sort values as part of the response per search hit 2010-08-08 18:21:17 +03:00
kimchy 7cb2010dda add script support for terms facets, allowing to control if a certain term will be included in the facets result (boolean result), or muging of the term 2010-08-08 10:37:14 +03:00
kimchy ee1d50f8d8 add script support for terms facets, allowing to control if a certain term will be included in the facets result (boolean result), or muging of the term 2010-08-08 10:27:06 +03:00
kimchy 566ae94478 add the ability to control the "order" by of terms facet, can either be `term` or `count`. defaults to `count`. 2010-08-08 09:31:40 +03:00
kimchy d4547c629f New API, update settings, allowing to change specific runtime settings of the cluster (indices). `number_of_replicas` is the only settings supported, allowing to change the number of repliacs of one or more indices. 2010-08-08 09:17:09 +03:00
kimchy 6d3d9fd807 start work on being able to change the number of replicas at runtime 2010-08-05 11:00:02 +03:00
kimchy 860d8058b6 rename jmx settings to conform with casing (old casing also works) 2010-08-04 22:27:26 +03:00
kimchy 8c5fdf4aee Ability to return all stored fields with a *, closes #296. 2010-08-04 13:22:55 +03:00
kimchy d72de60b6f Add the ability to store the index name in the doc, closes #292. 2010-08-04 11:41:56 +03:00
kimchy a44d30bb61 After gateway recovery, mappings keep being applied on each cluster change, closes #295. 2010-08-04 09:02:10 +03:00
kimchy 959eb0e703 Geo: Polygon based filter, closes #294. 2010-08-03 21:52:13 +03:00
kimchy 39f344c11e change the format of bounding box filter to conform more to the "field_name" : { ... } structure 2010-08-03 19:18:16 +03:00
kimchy 91c256361e support for custom script based sorting 2010-08-03 18:14:02 +03:00
kimchy 3d31c38f11 refactoring transport service to allow for send options 2010-08-03 10:06:30 +03:00
kimchy 04781e038a when timeouting, and closed, raise onClose and not onTimeout 2010-08-03 10:05:19 +03:00
kimchy de8ffaf5fb Geo: bounding box filter, closes #290. 2010-08-03 01:51:46 +03:00
kimchy d1e404c1b3 change default geo point resolution from 32 to 64 2010-08-03 00:08:56 +03:00
kimchy 07ab5dcf9b reuse constants for facet type 2010-08-02 22:27:03 +03:00
kimchy dc6ef326d9 Facets: Filter based facet, closes #289. 2010-08-02 19:17:57 +03:00
kimchy 4e6087ee2d improve concurrency when loading field data 2010-08-02 18:20:49 +03:00