b39a64e49f
accept grok matches with no captures instead of NPE
2016-01-12 12:01:55 -08:00
f429cb08d7
Merge branch 'master' into feature/aggs-refactoring
2016-01-12 12:59:52 +00:00
4d38a47eb5
Review feedback and several cleanups
2016-01-12 12:06:14 +01:00
7bdd2583aa
Merge remote-tracking branch 'es/master' into feature/ingest
2016-01-12 01:01:30 +01:00
9d7d89d04d
Merge branch 'master' into feature/aggs-refactoring
2016-01-11 12:50:31 +00:00
1637fe9e0b
Moved the grok processor to its own module, so that it will available out-of-the-box, while its dependencies are isolated
2016-01-08 21:59:23 +01:00
81a7607256
Remove -Xlint:-deprecation from plugins
...
Instead we suppress warnings about using deprecated stuff near the usage
site with a comment about why its ok.
2016-01-07 20:44:46 -05:00
3b7d1b47f7
Merge branch 'master' into feature/aggs-refactoring
...
# Conflicts:
# core/src/main/java/org/elasticsearch/percolator/PercolatorService.java
# core/src/main/java/org/elasticsearch/percolator/QueryCollector.java
2016-01-07 12:42:02 +00:00
0786c506dc
Remove a few more Xlint skips
2016-01-06 23:28:13 -05:00
2d6adf6428
Percolator refactoring:
...
* Added percolator field mapper that extracts the query terms and indexes these terms with the percolator query.
* At percolate time these extracted terms are used to query percolator queries that are like to be evaluated. This can significantly cut down the time it takes to percolate. Whereas before all percolator queries were evaluated if they matches with the document being percolated.
* Changes made to percolator queries are no longer immediately visible, a refresh needs to happen before the changes are visible.
* By default the percolate api only returns upto 10 matches instead of returning all matching percolator queries.
* Made percolate more modular, so that it is easier to add unit tests.
* Added unit tests for the percolator.
Closes #12664
Closes #13646
2016-01-06 16:08:10 +01:00
2c33f78192
Merge branch 'master' into feature/aggs-refactoring
...
# Conflicts:
# core/src/main/java/org/elasticsearch/search/aggregations/bucket/children/ChildrenParser.java
# core/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceParser.java
# test/framework/src/main/java/org/elasticsearch/test/TestSearchContext.java
2016-01-06 09:35:53 +00:00
a89dba27c2
Task Management: Add framework for registering and communicating with tasks
...
Adds task manager class and enables all activities to register with the task manager. Currently, the immutable Transport*Activity class represents activity itself shared across all requests. This PR adds and an additional structure Task that keeps track of currently running requests and can be used to communicate with these requests using TransportTaskAction.
Related to #15117
2016-01-05 12:24:43 -05:00
ea2a8150d3
Merge pull request #15692 from s1monw/issues/15675
...
Fix pharse suggest collate example to use correct script context/syntax
2016-01-05 09:27:18 +01:00
1a47226d9a
Merge pull request #15663 from jpountz/remove/mapping_backcompat
...
Remove mapping backward compatibilit with pre-2.0.
2016-01-04 10:05:39 +01:00
6d3c9b074c
Remove support for the `multi_field` type.
...
It is officially unsupported since version 1.0.
2015-12-30 12:03:15 +01:00
3b06cd8d92
Fix pharse suggest collate example to use correct script context/syntax
...
Closes #15675
2015-12-29 09:22:36 +01:00
180ab2493e
Improve thirdPartyAudit check, round 3
2015-12-28 22:38:55 -05:00
af122f4151
Remove mapping backward compatibilit with pre-2.0.
...
This removes the backward compatibility layer with pre-2.0 indices, notably
the extraction of _id, _routing or _timestamp from the source document when a
path is defined.
2015-12-24 13:47:37 +01:00
d8d8666877
Remove `index_name` back compat.
...
Since 2.0 we enforce that fields have the same full and index names. So in 3.x
we can remove the ability to have different names on the same field.
2015-12-23 14:55:26 +01:00
cf52e96c42
Upgrade to lucene-5.5.0-snapshot-1721183.
...
Some files that implement or use the Scorer API had to be changed because of
https://issues.apache.org/jira/browse/LUCENE-6919 .
2015-12-21 17:02:08 +01:00
97c2f7b037
Aggregations Refactor: Refactor Histogram and Date Histogram Aggregation
2015-12-21 09:34:05 +00:00
4ea19995cf
Remove wildcard imports
2015-12-18 12:43:47 -08:00
15588a4991
Aggregations: Run pipeline aggregations for empty buckets added in the Range Aggregation
...
Closes #15471
2015-12-18 14:42:05 +00:00
6ea16671f4
Simplify the Text API.
...
We have the Text API, which is essentially a wrapper around a String and a
BytesReference and then we have 3 implementations depending on whether the
String view should be cached, the BytesReference view should be cached, or both
should be cached.
This commit merges everything into a single Text that is essentially the old
StringAndBytesText impl.
Long term we should look into whether this API has any performance benefit or
if we could just use plain strings. This would greatly simplify all our other
APIs that currently use Text.
2015-12-17 17:22:38 +01:00
a7cc91e868
Merge pull request #15501 from rmuir/sheisty_classes
...
thirdPartyAudit round 2
2015-12-17 03:44:27 -05:00
6692e42d9a
thirdPartyAudit round 2
...
This fixes the `lenient` parameter to be `missingClasses`. I will remove this boolean and we can handle them via the normal whitelist.
It also adds a check for sheisty classes (jar hell with the jdk).
This is inspired by the lucene "sheisty" classes check, but it has false positives. This check is more evil, it validates every class file against the extension classloader as a resource, to see if it exists there. If so: jar hell.
This jar hell is a problem for several reasons:
1. causes insanely-hard-to-debug problems (like bugs in forbidden-apis)
2. hides problems (like internal api access)
3. the code you think is executing, is not really executing
4. security permissions are not what you think they are
5. brings in unnecessary dependencies
6. its jar hell
The more difficult problems are stuff like jython, where these classes are simply 'uberjared' directly in, so you cant just fix them by removing a bogus dependency. And there is a legit reason for them to do that, they want to support java 1.4.
2015-12-17 02:35:00 -05:00
4523eaec88
Added plumbing for compile time script parameters.
...
Closes #15464
2015-12-16 18:29:21 -08:00
4f9d4103f2
Merge pull request #15491 from rmuir/forbidden_third_party
...
Add gradle thirdPartyAudit to precommit tasks
2015-12-16 18:56:50 -05:00
42138007db
add some more comments about internal api usage
2015-12-16 18:56:02 -05:00
ee79d46583
Add gradle thirdPartyAudit to precommit tasks
2015-12-16 16:38:16 -05:00
a2b8f4b90a
Merge pull request #15434 from rjernst/http_type
...
Expose http.type setting, and collapse al(most all) modules relating to transport/http
2015-12-16 11:54:30 -08:00
5d5c6591aa
Validate that fields are defined only once.
...
There are two ways that a field can be defined twice:
- by reusing the name of a meta mapper in the root object (`_id`, `_routing`,
etc.)
- by defining a sub-field both explicitly in the mapping and through the code
in a field mapper (like ExternalMapper does)
This commit adds new checks in order to make sure this never happens.
Close #15057
2015-12-15 10:37:18 +01:00
60d35c81af
Plugins: Expose http.type setting, and collapse al(most all) modules relating to transport/http
...
This change adds back the http.type setting. It also cleans up all the
transport related guice code to be consolidated within the
NetworkModule (as transport and http related stuff is what and how ES
exposes over the network). The setter methods previously used by some
plugins to override eg the TransportService or HttpServerTransport are
removed, and those plugins should now register a custom implementation
of the class with a name and set that using the appropriate config
setting. Note that I think ActionModule should also be moved into here,
to sit along side the rest actions, but I left that for a followup.
closes #14148
2015-12-14 22:01:04 -08:00
11cbc08e45
Fix compilation in o.e.m.t.SimpleSortTests
2015-12-11 11:46:50 -05:00
3383c24be0
Remove and forbid use of Collections#shuffle(List) and Random#<init>()
...
This commit removes and now forbids all uses of
Collections#shuffle(List) and Random#<init>() across the codebase. The
rationale for removing and forbidding these methods is to increase test
reproducibility. As these methods use non-reproducible seeds, production
code and tests that rely on these methods contribute to
non-reproducbility of tests.
Instead of Collections#shuffle(List) the method
Collections#shuffle(List, Random) can be used. All that is required then
is a reproducible source of randomness. Consequently, the utility class
Randomness has been added to assist in creating reproducible sources of
randomness.
Instead of Random#<init>(), Random#<init>(long) with a reproducible seed
or the aforementioned Randomess class can be used.
Closes #15287
2015-12-11 11:16:38 -05:00
1e8f9558a0
Remove now-dead code in expressions (fixed in https://issues.apache.org/jira/browse/LUCENE-6920 )
2015-12-10 14:50:32 -05:00
2741888498
Remove RuntimePermission("accessDeclaredMembers")
...
Upgrades lucene to 5.5.0-1719088, randomizedtesting to 2.3.2, and securemock to 1.2
2015-12-10 14:26:55 -05:00
e454fadc22
Merge branch 'master' into shave_mustache
2015-12-10 07:58:24 -05:00
025e9818e7
Refactors TransportReplicationAction to decouple request routing and shard operation logic
2015-12-10 01:32:07 -05:00
61266dee19
mark messy rest test as messy and add to messy tests lists
2015-12-09 19:21:35 -05:00
74dc5bf20a
Fix template query parser tests to register mustache script engine
2015-12-08 21:33:48 -08:00
b7fb0824be
Fix render search template tests
2015-12-08 21:33:48 -08:00
a6e1655fe9
fix integ tests
2015-12-09 00:30:32 -05:00
17436d4332
get gradle check passing for lang-mustache
2015-12-09 00:10:37 -05:00
3632ddf497
split out mustache methods so groovy works again
2015-12-09 00:00:40 -05:00
907d3cf878
move template query builder tests back to core (using mockscript)
2015-12-08 20:26:45 -08:00
2e299860e5
refactor mustache to lang-mustache plugin. No rest tests yet.
2015-12-08 22:45:21 -05:00
3c419c2186
do expressions consistently with other engines
2015-12-05 22:08:40 -05:00
2169a123a5
Filter classes loaded by scripts
...
Since 2.2 we run all scripts with minimal privileges, similar to applets in your browser.
The problem is, they have unrestricted access to other things they can muck with (ES, JDK, whatever).
So they can still easily do tons of bad things
This PR restricts what classes scripts can load via the classloader mechanism, to make life more difficult.
The "standard" list was populated from the old list used for the groovy sandbox: though
a few more were needed for tests to pass (java.lang.String, java.util.Iterator, nothing scary there).
Additionally, each scripting engine typically needs permissions to some runtime stuff.
That is the downside of this "good old classloader" approach, but I like the transparency and simplicity,
and I don't want to waste my time with any feature provided by the engine itself for this, I don't trust them.
This is not perfect and the engines are not perfect but you gotta start somewhere. For expert users that
need to tweak the permissions, we already support that via the standard java security configuration files, the
specification is simple, supports wildcards, etc (though we do not use them ourselves).
2015-12-05 21:46:52 -05:00
46377778a9
Merge branch 'master' into getClassLoader
2015-12-04 15:58:36 -05:00
7160c5ec15
list modules separately in pluginservice
2015-12-04 01:13:17 -05:00
a8e9403204
added gradle checks for modules configuration, and ability to add
...
modules to integ test cluster
2015-12-03 20:53:06 -08:00
0a4a81afaf
Added modules, distributions now include them (just plugins installed in
...
a diff dir)
2015-12-03 14:18:26 -08:00
bf0caafa04
remove old modules and gradle
2011-12-06 01:01:04 +02:00
a8fd2d48b8
first cleanup phase, move to single src
2011-12-06 00:59:23 +02:00
9995d27841
Analysis: Add arabic, brazilian, czech to stemmer token filter language options, closes #1519 .
2011-12-04 16:36:42 +02:00
720954d8a6
Version missing a space, closes #1517 .
2011-12-04 12:48:29 +02:00
032e215f25
search missing from stats when using clear and search params, closes #1516 .
2011-12-02 12:32:51 +02:00
be282cc4c8
Improve highlighting perf (a bit) by reusing some constructs across hits, closes #1513 .
2011-11-30 19:15:31 +02:00
c93ddd9b61
check against hte fieldName...
2011-11-30 13:34:21 +02:00
6c552b4187
have DocSet implement Bits interface
2011-11-29 23:53:38 +02:00
a21c0829c7
Query DSL: Bool filter does not take should clauses properly into account, closes #1511 .
2011-11-29 22:39:07 +02:00
76307a5e73
cleanup
2011-11-29 21:28:07 +02:00
e9d2f44d49
Query DSL: Bool filter does not take should clauses properly into account, closes #1511 .
2011-11-29 21:27:36 +02:00
de49a313c5
Query DSL: Bool filter does not take should clauses properly into account, closes #1511 .
2011-11-29 21:23:41 +02:00
9bb9ce4e84
add 0.18.6
2011-11-29 13:55:11 +02:00
ae4ae598e3
make node closed exception serializable
2011-11-29 10:18:13 +02:00
b2fa6b7a94
When _source is disabled, don't return it in realtime get fetching the document from the transaction log, closes #1509 .
2011-11-29 09:39:46 +02:00
e9fe8ec469
sync with latest Lucene block join, init parentDoc to -1, which makes the assert commented out valid again
2011-11-28 23:47:31 +02:00
6b894d9f53
add the option to get mapping as a parsed map, also do some internal refactoring to share the code that parses into a map
2011-11-28 20:19:02 +02:00
b1707d219f
set "http" on NodeInfo instances if it is passed in the constructor
2011-11-28 17:08:28 +02:00
d5aa7574f3
Upgrade to netty 3.2.7, closes #1506 .
2011-11-28 14:48:49 +02:00
4be7866bd9
Registering a percolate query with additional "object" level metadata can fail, closes #1505 .
2011-11-27 18:06:18 +02:00
32f1edf6f7
Analysis: Add language setting to lowercase filter, supporting greek and turkish, closes #1503 .
2011-11-27 13:39:05 +02:00
a1a856e4b9
Support Greek stemming
...
While the GreekAnalyzer supports stemming, custom analyzers for the greek language cannot be build because there is no GreekStemmer (although lucene has one).
2011-11-27 13:22:58 +02:00
f18ad903a9
Upgrade to Lucene 3.5, closes #1502 .
2011-11-27 12:57:26 +02:00
fd5d754fe6
better failure messages when dynamic mapping is disabled
2011-11-25 09:40:44 +02:00
a7803855a1
add another path trie test
2011-11-25 00:21:01 +02:00
3f835eb780
even though it can't happen (offset is 0 when loading from the index), use properly the field to get binary values with offset and length
2011-11-24 21:44:18 +02:00
21988a0ca7
support compressed percolator index
2011-11-24 21:32:56 +02:00
ac2c2fb48d
enable unsafe optimization in lzf for 0.19
2011-11-24 20:12:51 +02:00
03c2e5ea52
improve how decoding is done on the transport layer, embedding FrameDecoder into the message handler, and reducing allocation of buffers and better guess into allocating cumalation buffers
2011-11-24 20:03:25 +02:00
f0efb8cdea
Using _parent:123 in a query string query fails to fetch docs, closes #1497 .
2011-11-24 13:59:10 +02:00
81965d0ea9
Support using _id:1234, or using term query/filter on _id even when _id is not indexed, closes #1496 .
2011-11-24 12:31:50 +02:00
daa4444e22
DocumentMapper.java wrong order on build rootMappers, closes #1493 .
2011-11-24 09:58:05 +02:00
2c0662e18e
Query DSL: indices query to allow to set a `no_match_query`, closes #1492 .
2011-11-23 19:01:14 +02:00
f47b77199d
Multi field mapper with more than one extra mapping can cause endless re-sync'ing of mapping between nodes, closes #1487 .
2011-11-23 09:04:28 +02:00
05f98634d8
IndicesQueryBuilder generates the wrong query name, closes #1485 .
2011-11-22 15:57:54 +02:00
f28c11b31c
By default, set http.compression to false, closes #1482 .
2011-11-21 19:50:24 +02:00
bff980c797
Allow empty Strings to be null for Number's and don't autodetect empty string fields as string types, closes #1473 .
2011-11-21 18:02:12 +02:00
fbb03c611a
add a sleep to fix test (need to think of a better fix)
2011-11-21 10:44:29 +02:00
b10094b109
support boost on block join query (nested)
2011-11-21 10:43:58 +02:00
90af54dad5
make sure bulk item failure on primary shards will not execute on a replica shard
2011-11-20 15:09:13 +02:00
bb8ff3814e
Binary field compression causes wrong _source decoding, closes #1475 .
2011-11-20 12:59:22 +02:00
6403a42e31
* Try to extract value from String using FieldMapper when reading from the transaction log.
2011-11-17 18:54:41 +02:00
4d607bdd4a
handle valueForString in binary case
2011-11-17 18:19:16 +02:00
c8641588ec
clean code, script service no longer used in get action
2011-11-17 15:48:05 +02:00
835dc33d45
make sure we use the default index analyzer for _all field, even though it works well without it now because the _analyzer field comes before it and sets the context.analyzer
2011-11-17 15:37:51 +02:00
b0880e114e
add another instanceof check on Date and not direct check
2011-11-17 14:27:40 +02:00
09fe743790
add another instanceof check on Date and not direct check
2011-11-17 14:27:16 +02:00
118acb3992
add 0.18.5 version
2011-11-16 22:00:13 +02:00
b0fc243138
add 0.18.4 version
2011-11-16 15:23:58 +02:00
72d2fd0da0
Inner queries not resolved correctly in has_child filter when searching directly against the parent type (in the URI for example), closes #1471 .
2011-11-16 14:15:29 +02:00
b5a0ae2fbe
better toString for custom score query
2011-11-15 22:20:15 +02:00
e52ec518f3
NPE when highlighting
2011-11-15 18:11:03 +02:00
afb690e233
refactor sub fetch phase to also allow for hits level execution
2011-11-15 18:09:45 +02:00
537d9c5db0
FVH should use the parsed query, and not a possible rewritten query
2011-11-15 16:45:05 +02:00
8f392af207
upgrade to mvel 2.1 Beta7
2011-11-15 16:23:13 +02:00
8d0ac1787b
upgrade to jackson 1.9.2
2011-11-15 15:53:18 +02:00
a6a2a264c3
Highlighting on term vector enabled field should not highlight filters by default, closes #1469 .
2011-11-15 13:42:15 +02:00
b8fe9999e0
XContentBuilder to handle extended classes of java.util.Date, closes #1465 .
2011-11-15 11:34:32 +02:00
abaf4b5196
Indices Stats API: Providing groups as part of the HTTP API does not return stats for those groups, closes #1468 .
2011-11-15 10:31:55 +02:00
0d714d580f
Add counts of currently executing get operations
2011-11-15 10:21:10 +02:00
9bf42fed1b
Nested facet execution can fail with ArrayIndexOutOfBounds, closes #1467 .
2011-11-15 09:23:56 +02:00
dc2824fe06
apply fix to unsafe decoder in lzf, still use vanilla for now
2011-11-15 08:46:05 +02:00
73ba30b18a
Date Range: Inclusive upper range does not round up properly, closes #1463 .
2011-11-14 17:16:42 +02:00
ed281fbfd5
Index shard search slow log (query and fetch), closes #1462 .
2011-11-14 14:10:39 +02:00
b43c8fe128
get should report time in millis, but return in nanoseconds
2011-11-14 10:40:05 +02:00
62d67e6c2d
Add counts of currently executing index, delete, query and fetch operations
2011-11-14 10:24:50 +02:00
eefbe52580
disable optimized decoder by default
2011-11-14 10:10:22 +02:00
eb1b0b150f
Using root object level mapping (_size, _source) can cause reparsing of the mapping on startup, close #1458 .
2011-11-13 20:56:16 +02:00
3da6e0a5b0
Percolate / Analyzer API can hang if it fails to execute, closes #1457 .
2011-11-13 19:54:13 +02:00
28c9595af3
* Add support for LZF compression on binary fields.
2011-11-13 14:32:53 +02:00
4bbf29834e
upgrade to lzf compress 0.9
2011-11-13 14:06:09 +02:00
27a7b0680c
fix top_terms_boost parsing for multi term rewrite
2011-11-13 12:51:10 +02:00
9cc0b7a28b
Fix processing of regex patterns in large terms facet requests
2011-11-13 12:49:12 +02:00
a3978402e5
add the search request to the search context
2011-11-13 12:48:17 +02:00
f4adff652c
Blank routing should be ignored, closes #1450 .
2011-11-13 12:04:02 +02:00
da98aab13f
also support _percolate for bulk item requests
2011-11-13 10:19:05 +02:00
21c4530aaf
Search: Search requests execute by mistake on the networking http IO thread, causing other http operations to hang, closes #1455 .
2011-11-11 22:22:21 +02:00
99d31cc8c8
better handlign of open channels
2011-11-11 22:22:15 +02:00
93f1d50c18
configure the scheduled TP to not wait for delayed execution, so we shutodown faster
2011-11-10 21:52:42 +02:00
68c8cac852
AbstractCompoundWordTokenFilterFactory should store the dictionary as a CharArraySet instance, not a Set, closes #1449 .
2011-11-10 20:12:35 +02:00
15c798fb4c
fix minimum_should_match in query_string builder
2011-11-10 12:13:31 +02:00
c4542c8349
increase search stats rounds to make sure the stats are higher than 0
2011-11-10 12:01:39 +02:00
62bc0836c1
improve boolean filter
2011-11-10 12:00:43 +02:00
ed8c289dec
Log warning when application of an updated cluster state fails with an exception
2011-11-10 10:12:58 +02:00
e7baf30bd2
Fix reporting of total indexing stats
2011-11-10 10:07:10 +02:00
eafa2b0c9a
Fix possible NPE in TransportNodesListGatewayStartedShards operation
2011-11-10 10:04:11 +02:00
c5ebe6e86f
Mapping: Improve applying guessed types on dynamic templates, closes #1446 .
2011-11-09 09:02:20 +02:00
75f522adaa
data.path locations are unevenly filled (0.18.2), closes #1436 .
2011-11-02 23:45:35 +02:00
3c2dd959cb
Aliases should be processed when index routing changes
2011-11-01 20:05:46 +02:00
ca7247e1b1
add more details logging on info of data path
2011-10-28 07:45:17 +02:00
65a525bb15
add 0.18.3
2011-10-27 20:42:43 +02:00
723d2e8b15
Search: Fetching fields that end up extracted from source might fail, closes #1431 .
2011-10-27 20:28:38 +02:00
8922d5b073
Merge Scheduler: Allow to set index.merge.scheduler.max_merge_count, closes #1430 .
2011-10-27 02:14:47 +02:00
efffcf2b26
add 0.18.2 option
2011-10-27 01:43:08 +02:00
99e2bb270c
External data location configuration fails (path.data setting), closes #1429 .
2011-10-27 01:30:33 +02:00
49a97f6fcf
move to 0.19.0 snap
2011-10-26 21:04:10 +02:00
3544b07330
release 0.18.0
2011-10-26 20:58:24 +02:00
Tal Levy
Colin Goodheart-Smithe
Simon Willnauer
Martijn van Groningen
Nik Everett
Igor Motov
Simon Willnauer
Adrien Grand
Robert Muir
Ryan Ernst
Robert Muir
Jack Conradson
Jason Tedor
Areek Zillur
Shay Banon
Njal Karevoll
George Chatzigeorgiou
Jeremie BORDIER