kimchy
|
d1f5577c40
|
add channel based read from fs snapshot, reusing the same file descriptor with direct position based reads
|
2010-07-09 03:51:36 +03:00 |
|
kimchy
|
6cdd6a015c
|
reuse file buffer
|
2010-07-09 03:24:31 +03:00 |
|
kimchy
|
d531d82cfb
|
Zen Discovery: A node might get into an infinite state of trying to find a master (when client / non_master) nodes exists, closes #247.
|
2010-07-09 00:43:06 +03:00 |
|
kimchy
|
87eb6bed8f
|
nicer logging
|
2010-07-08 22:53:41 +03:00 |
|
kimchy
|
720e6a6d5b
|
nicer logging
|
2010-07-08 22:52:03 +03:00 |
|
kimchy
|
be6aaa157f
|
fixed a bug in fs translog where it wasn't seeking correctly, and not counting the number of operations
|
2010-07-08 22:18:26 +03:00 |
|
kimchy
|
ef148077fc
|
better logging
|
2010-07-08 21:37:10 +03:00 |
|
kimchy
|
84af9f5681
|
improve handling of translog snapshotting
|
2010-07-08 20:51:37 +03:00 |
|
kimchy
|
4429a61528
|
better cleanup on shard startup failure
|
2010-07-08 19:20:03 +03:00 |
|
kimchy
|
b69fc265c5
|
index writer ram buffer size can only be 2G, make sure not to set an illegal value
|
2010-07-08 19:03:55 +03:00 |
|
kimchy
|
400ec7b8d1
|
index writer ram buffer size can only be 2G, make sure not to set an illegal value
|
2010-07-08 18:58:04 +03:00 |
|
kimchy
|
63deef42cf
|
add a help appendable blob store based on immutable one
|
2010-07-08 18:53:40 +03:00 |
|
kimchy
|
8d669ff54e
|
move sorting to be done using the new fielddata cache, and not using lucene built in FieldCache
|
2010-07-08 14:42:35 +03:00 |
|
kimchy
|
97b5b41522
|
enable eager cache eviction based on readers, requires a nasty hack to Lucene, basically overriding two classes to hook into when a reader actually gets closed (and only enabled when elasticsearch is before lucene in the classpath, which we take for in the startup scripts)
|
2010-07-07 17:15:26 +03:00 |
|
kimchy
|
a36b06c682
|
ignore failed snapshot on a non started index
|
2010-07-07 16:59:33 +03:00 |
|
kimchy
|
a8c317e355
|
fix terms facets test to take into account distributed terms request
|
2010-07-07 16:33:39 +03:00 |
|
kimchy
|
5ff46911b6
|
ignore recovery when no node to recover from (it has disconnected, for example)
|
2010-07-07 16:30:28 +03:00 |
|
kimchy
|
3fb68c52da
|
ignore snapshot on a shard thas has just been created
|
2010-07-07 16:30:17 +03:00 |
|
kimchy
|
11a8d637c0
|
ignore snapshot on a shard thas has just been created
|
2010-07-07 16:26:19 +03:00 |
|
kimchy
|
e40aa540fe
|
ignore recovery when no node to recover from (it has disconnected, for example)
|
2010-07-07 16:25:55 +03:00 |
|
kimchy
|
9578ad3ef1
|
Terms Facets: Allow to specify a set of terms to exclude in the request, closes #246.
|
2010-07-07 14:41:34 +03:00 |
|
kimchy
|
2b5458daf6
|
implement a simple file system based translog, not the defeault yet, should provide better memory management
|
2010-07-07 09:30:07 +03:00 |
|
kimchy
|
b2bdb149f9
|
log shard close failure when closing an index
|
2010-07-06 22:50:40 +03:00 |
|
kimchy
|
6aa9be238c
|
use the atomic integer counter to represent the size
|
2010-07-06 18:45:19 +03:00 |
|
kimchy
|
267016758d
|
improve handling of memory caching with file system, only force compound file when really needed (when an extension that exists within the compound file is part of the memory cached extensions)
|
2010-07-06 18:21:36 +03:00 |
|
kimchy
|
ac01bb86e8
|
clean lock only before opening an index writer
|
2010-07-06 18:13:05 +03:00 |
|
kimchy
|
2908418d37
|
checn the default byte buffer cache to 200mb, now that its global per node
|
2010-07-06 17:58:30 +03:00 |
|
kimchy
|
070cb5b295
|
allocate replica shards (backups) only after the primaries have been allocated, this results in better theoretical shard allocation, and the ability to reuse local storage index files (comared against the backup)
|
2010-07-06 17:58:05 +03:00 |
|
kimchy
|
1d39bb4d51
|
cache md5 computation for unallocated shard and reused it
|
2010-07-06 09:00:24 +03:00 |
|
kimchy
|
d4f86899e3
|
more work on reuse work dir, improved transaction log to move to stream based
|
2010-07-06 00:12:40 +03:00 |
|
kimchy
|
b078c9206a
|
more work on reuse work dir, added md5 logic to identify what need to be recovered or not, note, requires reindexing
|
2010-07-05 14:13:55 +03:00 |
|
kimchy
|
79b69764e0
|
doc
|
2010-07-01 15:03:11 +03:00 |
|
kimchy
|
e8b261c165
|
better shutdown handling
|
2010-07-01 10:28:57 +03:00 |
|
kimchy
|
ad5525f575
|
fix unwrapping of exceptions
|
2010-07-01 10:27:53 +03:00 |
|
kimchy
|
f6c9c49645
|
better logging when failing to read full content of a transport message
|
2010-07-01 03:00:57 +03:00 |
|
kimchy
|
c75a56ca17
|
better shutdown handling
|
2010-07-01 02:41:00 +03:00 |
|
kimchy
|
c2d02e4e3a
|
wrong serialization of shardId
|
2010-07-01 02:18:35 +03:00 |
|
kimchy
|
d848ab3df3
|
rename executors to esexecutors
|
2010-07-01 00:06:34 +03:00 |
|
kimchy
|
b0d736b489
|
make the shard routing an actual component
|
2010-06-30 23:39:27 +03:00 |
|
kimchy
|
811559f393
|
add another test if the pool size is greater than the core pool size before a worker can exit
|
2010-06-30 23:08:05 +03:00 |
|
kimchy
|
6709518c4e
|
add the ability to list shard index blobs without actually having one created
|
2010-06-30 22:54:59 +03:00 |
|
kimchy
|
66096e97f2
|
Remove the terms API (replaced by facets in search), closes #242.
|
2010-06-30 19:49:41 +03:00 |
|
kimchy
|
df7e3f55a4
|
improve cluster blocks data structures for simpler usage
|
2010-06-30 16:06:31 +03:00 |
|
kimchy
|
4049da97fb
|
time value to support parsing days as well
|
2010-06-30 13:00:25 +03:00 |
|
kimchy
|
088e0b5a64
|
Automatic management of indexing buffer size, closes #241.
|
2010-06-30 01:46:57 +03:00 |
|
kimchy
|
fdb2eff998
|
Automatic management of indexing buffer size, closes #241.
|
2010-06-29 22:54:54 +03:00 |
|
kimchy
|
03ac38fdba
|
better benchmark on memory monitor
|
2010-06-29 22:12:43 +03:00 |
|
kimchy
|
3f6cd46736
|
allow to configure tcp network modules (transport, memcached and http) to use blocking io (just enabling it to do some performance tests)
|
2010-06-29 21:07:38 +03:00 |
|
kimchy
|
01ff81fa89
|
fix log message
|
2010-06-29 19:49:28 +03:00 |
|
kimchy
|
bc803b4fcf
|
close the thread pool on test
|
2010-06-29 18:35:03 +03:00 |
|
kimchy
|
703657699a
|
refactor new thread pools in, remove dynamic ones
|
2010-06-29 18:20:31 +03:00 |
|
kimchy
|
de4b21e986
|
backups don't become primaries on node failures, primaries are first allocated and then start initializing from gateway
|
2010-06-29 18:20:13 +03:00 |
|
kimchy
|
514df4ee3f
|
refactor the shard routing abstraction, no need for it
|
2010-06-29 18:02:10 +03:00 |
|
kimchy
|
fe5e5a073b
|
work on scaling and blocking thread pool
|
2010-06-29 18:01:54 +03:00 |
|
kimchy
|
9051ae3a65
|
work on scaling and blocking thread pool
|
2010-06-29 17:50:05 +03:00 |
|
kimchy
|
297001de53
|
update benchmark code
|
2010-06-29 08:59:28 +03:00 |
|
kimchy
|
aa5791a14f
|
move block metadata checks to the transport level
|
2010-06-29 08:51:36 +03:00 |
|
kimchy
|
a82bf74c1c
|
expose blocks in the cluster state rest api
|
2010-06-29 01:00:07 +03:00 |
|
kimchy
|
5f538b1ba3
|
Block operation performed on cluster until it recovered from the gateway, closes #239.
|
2010-06-28 22:53:13 +03:00 |
|
kimchy
|
b970f0dcc2
|
fix toString of bytes value
|
2010-06-28 17:53:07 +03:00 |
|
kimchy
|
63523f4525
|
Query DSL: field query does not take into account `allow_leading_wildcards`, closes #236.
|
2010-06-28 12:33:36 +03:00 |
|
kimchy
|
cb9be9973b
|
add the ability to define meta _attributes for mapping, basically a place to store additional unstructured data on the mapping
|
2010-06-28 09:10:08 +03:00 |
|
kimchy
|
3770924300
|
allow to control the buffer size of the gateway blob container
|
2010-06-27 23:26:32 +03:00 |
|
kimchy
|
a7c13826da
|
more work on reusing work directory, clean unallocated shards when they are not needed
|
2010-06-27 18:19:12 +03:00 |
|
kimchy
|
b43f0f5965
|
add streamable to stores files metadata
|
2010-06-27 14:18:14 +03:00 |
|
kimchy
|
4ae45946a2
|
nicer logging
|
2010-06-27 13:50:27 +03:00 |
|
kimchy
|
fc7392a98b
|
fix logging
|
2010-06-27 13:07:21 +03:00 |
|
kimchy
|
90c995dda5
|
add todo
|
2010-06-27 12:28:30 +03:00 |
|
kimchy
|
1e7481d5ab
|
fix logging
|
2010-06-27 12:15:58 +03:00 |
|
kimchy
|
c56609a0ad
|
fix default merge size to not truncate on long
|
2010-06-27 12:14:38 +03:00 |
|
kimchy
|
bbb9c3883f
|
fix jdoc
|
2010-06-27 12:05:30 +03:00 |
|
kimchy
|
f1f695b4d9
|
fix logging
|
2010-06-27 09:39:07 +03:00 |
|
kimchy
|
b9590b5ed6
|
fix logging
|
2010-06-27 09:37:44 +03:00 |
|
kimchy
|
edd8786a81
|
close open index input on failure
|
2010-06-27 02:13:59 +03:00 |
|
kimchy
|
33d357dbb4
|
Memory Store / FS Memory: Create a node level memory store cache and allocator, closes #235.
|
2010-06-26 22:34:30 +03:00 |
|
kimchy
|
00d2abef40
|
refactor facets builder API to make it more manageable
|
2010-06-26 01:36:42 +03:00 |
|
kimchy
|
5df2257319
|
add test for facets with search size 0
|
2010-06-25 22:46:07 +03:00 |
|
kimchy
|
83010f7ee1
|
more apis on the index level store to help clean unused work shards when needed
|
2010-06-25 03:39:42 +03:00 |
|
kimchy
|
811856e1f8
|
better method name
|
2010-06-25 01:41:06 +03:00 |
|
kimchy
|
98e93322ae
|
use java user.dir as the default "path.home" location
|
2010-06-24 22:11:13 +03:00 |
|
kimchy
|
4709f00790
|
add to index store the ability to get the backing store free / total space
|
2010-06-24 18:05:21 +03:00 |
|
kimchy
|
edf0075025
|
more work on reusing work node level data, now when recovering from peers
|
2010-06-24 15:34:02 +03:00 |
|
kimchy
|
57169d4233
|
initial work on reusing work node level data when recovering from gateway
|
2010-06-24 00:32:05 +03:00 |
|
kimchy
|
0d7cebbaf0
|
upgrade to async http client 1.0.0
|
2010-06-23 21:51:29 +03:00 |
|
kimchy
|
296b6b2f57
|
use custom similarity in search (if there is one)
|
2010-06-23 14:30:14 +03:00 |
|
kimchy
|
a7fdd36f6e
|
clean files that are in the store but not in the gateway
|
2010-06-23 12:21:04 +03:00 |
|
kimchy
|
d5f35f3cee
|
upgrade to netty 3.2.1.Final
|
2010-06-23 10:52:16 +03:00 |
|
kimchy
|
2121aaad3b
|
part of the index store enhacement
|
2010-06-23 10:49:06 +03:00 |
|
kimchy
|
c4aff14e15
|
remove field options from field cache and freqs, simpler and they are not used in any case
|
2010-06-23 09:48:46 +03:00 |
|
kimchy
|
cccce2b114
|
fix work with cluster setting
|
2010-06-23 08:00:29 +03:00 |
|
kimchy
|
3b4584040b
|
add index level store constuct (on top of the shard level store)
|
2010-06-23 07:47:18 +03:00 |
|
kimchy
|
8e62bb6934
|
Cluster State API: Allow to filter the state on nodes, routing_table, metadata, and indices, closes #234.
|
2010-06-22 16:19:10 +03:00 |
|
kimchy
|
dcb0d18519
|
javadoc
|
2010-06-22 16:09:49 +03:00 |
|
kimchy
|
622d4041b8
|
handle cases where the node does not exists in the disco nodes
|
2010-06-22 10:59:24 +03:00 |
|
kimchy
|
7c931f34fa
|
ignore state exceptions when trying to flush
|
2010-06-22 10:26:44 +03:00 |
|
kimchy
|
0c6b82aeca
|
fix javadoc
|
2010-06-22 09:26:37 +03:00 |
|
kimchy
|
3d84af2a40
|
refactor hdfs gateway to use the new common blobstore
|
2010-06-22 09:10:00 +03:00 |
|
kimchy
|
21627bca10
|
return after notifiying listener on failure
|
2010-06-21 23:29:13 +03:00 |
|
kimchy
|
893de86e9e
|
fix missing break statemetns (via Lukas)
|
2010-06-21 22:45:02 +03:00 |
|
kimchy
|
7ed7c6db4e
|
Gateway: Internal refactoring, requires manual upgrade when using fs gateway, closes #232.
|
2010-06-21 22:40:12 +03:00 |
|
kimchy
|
733abdcac3
|
more unboxed math
|
2010-06-20 14:42:11 +03:00 |
|
kimchy
|
701920c6c1
|
remove unused API in field mapper (that also cause compilation to break on eclipse)
|
2010-06-20 12:12:01 +03:00 |
|
kimchy
|
e846ed6465
|
Lucene: Upgrade to latest 3.0.2 version, closes #231.
|
2010-06-20 01:04:39 +03:00 |
|
kimchy
|
384f8a4f42
|
Return the maxScore per search and score for each search hit, closes #205.
|
2010-06-20 00:23:27 +03:00 |
|
kimchy
|
0f2147aeec
|
add some prepare APIs to the groovy APIs
|
2010-06-19 21:09:03 +03:00 |
|
kimchy
|
891c69f8de
|
refactor builder requets into common base class
|
2010-06-18 14:26:33 +03:00 |
|
kimchy
|
adef9d69ff
|
Gateway: Allow to configure a `recovery_after_time` and `recover_after_nodes`, closes #223.
|
2010-06-18 05:32:51 +03:00 |
|
kimchy
|
22926f0026
|
Analysis: When specifying empty array for stopwords, use an empty list for stopwords, closes #230.
|
2010-06-18 00:34:18 +03:00 |
|
kimchy
|
8e4c139c8b
|
Analysis: fields that are `not_analyzed` should automatically default to keyword analyzer, closes #229.
|
2010-06-17 23:23:53 +03:00 |
|
kimchy
|
60bd8f34d2
|
add debug logging of the ping responses received
|
2010-06-17 22:37:47 +03:00 |
|
kimchy
|
f64b96e3c8
|
add an exception if there is no master node set
|
2010-06-17 22:19:43 +03:00 |
|
kimchy
|
7b145fd4e1
|
Expose the mappings of a given index or index/type, closes #222. Added the ability to filter returned data from the cluster state API, and use it to expose REST GET endpoint for "/_mapping", "/{index}/_mapping", "/{index}/{type}/_mapping" ({index} can also be "_all").
|
2010-06-17 17:56:07 +03:00 |
|
kimchy
|
3bc8c307f1
|
simplify netty transport to use single channel
|
2010-06-17 12:50:01 +03:00 |
|
kimchy
|
597f020156
|
fork node connected and disconnected on a different thread pool
|
2010-06-17 12:47:42 +03:00 |
|
kimchy
|
662ded3801
|
connect to a node when it joins the cluster on the disco level, so if it fails, it will be propagated back and the node will not be added to the cluster state
|
2010-06-17 12:46:53 +03:00 |
|
kimchy
|
86134d32a8
|
nicer logging
|
2010-06-17 12:46:05 +03:00 |
|
kimchy
|
d1c7bde07e
|
make the rest cluster state return mappings as object keyed type
|
2010-06-17 02:45:17 +03:00 |
|
kimchy
|
38005826e7
|
detect write errors early and notify transport handler
|
2010-06-17 02:17:54 +03:00 |
|
kimchy
|
8af925a0b1
|
add internal flags to help simulate forcefull disconnections
|
2010-06-17 01:05:23 +03:00 |
|
kimchy
|
a60e542718
|
cancel the timeout if we fail to send the request
|
2010-06-16 23:07:41 +03:00 |
|
kimchy
|
b0c8255cd9
|
clean the cluster state when starting (handles restarts better)
|
2010-06-16 21:59:26 +03:00 |
|
kimchy
|
a59679b0c6
|
better failure messages
|
2010-06-16 21:47:58 +03:00 |
|
kimchy
|
4d26acd6da
|
add aliases to cluster state rest response
|
2010-06-16 06:08:36 +03:00 |
|
kimchy
|
a547c1ccc0
|
cluster state to return native json (if possible) as part of the mappings
|
2010-06-16 05:48:30 +03:00 |
|
kimchy
|
626989c55e
|
don't return the source when asking for scripted fields
|
2010-06-16 04:48:06 +03:00 |
|
kimchy
|
2d6f61b3c1
|
Facets: Script Histogram facet, closes #228.
|
2010-06-16 04:22:00 +03:00 |
|
kimchy
|
bb24b56c66
|
Facets: Script statistical facets, closes #227.
|
2010-06-16 02:18:24 +03:00 |
|
kimchy
|
c2786038e2
|
Script Filter: Support providing a custom script as a filter, closes #226.
|
2010-06-16 01:32:04 +03:00 |
|
kimchy
|
be3b779caa
|
Tcp Transport: Reduce `transport.tcp.connection_per_node` from 5 to 1, closes #225.
|
2010-06-15 23:36:31 +03:00 |
|
kimchy
|
8e589b5a04
|
nicer code
|
2010-06-15 23:14:04 +03:00 |
|
kimchy
|
eaf124123d
|
reduce long name
|
2010-06-15 23:09:09 +03:00 |
|
kimchy
|
65039ff21d
|
Zen Discovery: When a master node is forcefully killed, other nodes might not monitor the other elected master, closes #224
|
2010-06-15 23:08:52 +03:00 |
|
kimchy
|
eb68891ae5
|
move size value/unit to bytes size value/unit, and create a ten based size value/unit
|
2010-06-15 18:26:44 +03:00 |
|
kimchy
|
a18e58b394
|
move size value/unit to bytes size value/unit, and create a ten based size value/unit
|
2010-06-15 18:16:10 +03:00 |
|
kimchy
|
f75abbaa00
|
big refactoring thanks to proper jarjar built from source with asm 3.3 allowing to jarjar guice and others, includes package relocations
|
2010-06-15 17:59:04 +03:00 |
|
kimchy
|
d5bd77cc38
|
big refactoring thanks to proper jarjar built from source with asm 3.3 allowing to jarjar guice and others, includes package relocations
|
2010-06-15 17:28:05 +03:00 |
|
kimchy
|
636bfe8466
|
big refactoring thanks to proper jarjar built from source with asm 3.3 allowing to jarjar guice and others, includes package relocations
|
2010-06-15 16:51:38 +03:00 |
|
kimchy
|
48979ab6c8
|
improve gateway recovery when using delay index creation, close the loophole when the master was shut down before the delay expired and now other node becoming master will do the recovery
|
2010-06-15 10:27:26 +03:00 |
|
kimchy
|
98df1b3433
|
Gateway: Allow to configure a delay till index creation from gateway will occur, closes #223.
|
2010-06-15 01:25:55 +03:00 |
|
kimchy
|
e0d20af743
|
ignore when flush can't be done
|
2010-06-14 13:25:18 +03:00 |
|
kimchy
|
0a1bc874c3
|
Search: Allow to specify script fields to be loaded, closes #221.
|
2010-06-14 12:50:33 +03:00 |
|
kimchy
|
c087bbe804
|
use List for fieldNames to reduce extra array construction
|
2010-06-14 04:36:03 +03:00 |
|
kimchy
|
953779ccea
|
allow to pass parameters to custom score script
|
2010-06-14 04:12:57 +03:00 |
|
kimchy
|
1b32c1ccf4
|
add cleanable thread local to single values
|
2010-06-14 03:42:43 +03:00 |
|
kimchy
|
751c56f31e
|
Query DSL: custom score (scripted) query, closes #220.
|
2010-06-14 03:15:23 +03:00 |
|
kimchy
|
2d0785078c
|
move to field.data package
|
2010-06-13 18:05:59 +03:00 |
|
kimchy
|
a9fc276a3e
|
refactor the custom boost factor query into a more general function boost query
|
2010-06-13 16:51:19 +03:00 |
|
kimchy
|
ec481159d6
|
pass score docs in order in custom boost factor query
|
2010-06-13 12:41:16 +03:00 |
|
kimchy
|
2daa54a2bb
|
don't clean thread locals explicitly, let GC clean them
|
2010-06-12 12:45:38 +03:00 |
|
kimchy
|
d515fe8644
|
add internal support for Rest/HTTP OPTIONS and HEAD
|
2010-06-11 00:13:08 +03:00 |