Commit Graph

10848 Commits

Author SHA1 Message Date
Simon Willnauer 1837d8a5fc fix test bug 2015-02-20 23:16:23 +01:00
Simon Willnauer d0a9c35c69 Prune all segment_N files not just newer ones 2015-02-20 22:30:02 +01:00
Simon Willnauer 136d36b724 [ENV] NodeEnv should lock all shards for an index
Today locking all shards only locks the shards that are present on
the node or that still have a shard directory. This can lead to odd
behavior if another shard that doesn't exist yet is allocated while
all shards are supposed to be locked.
2015-02-20 22:11:10 +01:00
Ryan Ernst 94348cb788 replace awaitsfix url with underlying lucene issue for index lookup failure 2015-02-20 11:18:53 -08:00
Nicholas Knize 9302126162 [GEO] Adds randomization to geo test suite
Adds RandomShapeGenerator for creating random shape types. This adds a level of randomized testing to the Geospatial logic. An initial randomized GeometryCollection test is added to the GeoShapeIntegrationTest suite for validating and verifying geo_shape filter behavior. The RandomShapeGenerator can/should be used in Unit and Integration testing to avoid biased testing.

closes #9588
2015-02-20 11:18:14 -06:00
Adrien Grand 7c20a8a913 Mute IndexLookupTests temporarily. 2015-02-20 15:01:04 +01:00
markharwood 29b1902cfb New aggregations feature - “PercentageScore” heuristic for significant_terms aggregation provides simple “per-capita” type measures.
Closes #9720
2015-02-20 13:22:08 +00:00
Simon Willnauer 50b9a8d6f2 Add missing segments file to the lucene file listing
Relates to #9761
2015-02-20 13:14:35 +01:00
Robert Muir f71beacbdb ban deprecated lucene postings apis 2015-02-20 07:11:37 -05:00
Robert Muir e8cfc11a6e add missing import 2015-02-20 07:02:25 -05:00
Robert Muir 30a4294a6a Upgrade to lucene r1660560
Squashed commit of the following:

commit 07391388715ed1f737e8acc391cea0bce5d79db9
Merge: a71cc45 b61b021
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Feb 20 06:58:11 2015 -0500

    Git really sucks

    Merge branch 'lucene_r1660560' of github.com:elasticsearch/elasticsearch into lucene_r1660560

commit b61b02163f62ad8ddd9906cedb3d57fed75eb52d
Author: Adrien Grand <jpountz@gmail.com>
Date:   Wed Feb 18 19:03:49 2015 +0100

    Try to improve TopDocs.merge usage.

commit bf8e4ac46d7fdaf9ae128606d96328a59784f126
Author: Ryan Ernst <ryan@iernst.net>
Date:   Wed Feb 18 07:43:37 2015 -0800

    reenable scripting test for accessing postings pieces.  commented out
    parts that fail because of bad assumptions

commit 6d4d635b1a23b33c437a6bae70beea70ad52d91c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 09:41:46 2015 -0500

    add some protection against broken asserts, but, also disable crappy test

commit c735bbb11f38782dfea9c4200fcf732564126bf5
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 02:21:30 2015 -0500

    cutover remaining stuff from old postings api

commit 11c9c2bea3db3ff1cd2807bd43e77b500b167aed
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 01:46:04 2015 -0500

    cut over most DocsEnum usage

commit bc18017662f6abddf3f074078f74e582494c88e2
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 01:19:35 2015 -0500

    upgrade to lucene_r1660560, modulo one test fail
2015-02-20 06:58:43 -05:00
Simon Willnauer a71cc45023 [INDICES] Retry if shard deletes fail due to IOExceptions
Today if a shard deletion fails we simply ignore it and move on. On system like
windows where a virus scanner can hold on to files or any other process ie. the admins
explorer window we fail to delete shards leaving large amout of data behind. We should try
best effort to clean those shards up before we ack the delete.
2015-02-20 12:50:40 +01:00
Simon Willnauer 8e09070246 [RESTORE] Refactor how restore cleans up files after snapshot was restored
Today we restore files by running through the directory removeing all files
not in the snapshot. Some files in that direcotry might belong there even though
we remove them. This commit moves the responsiblity of cleaning up pending files
to lucene by utilizing IndexWriter#IndexFileDeleter
2015-02-20 09:33:40 +01:00
Ryan Ernst dae0ed168a Test: Remove use of globalTempDir() and forbid it from future use.
Using this temp dir circumvents test isolation and isn't necessary.

closes #9777
2015-02-19 14:05:10 -08:00
Martijn van Groningen 16446440e4 Don't use the fixed bitset filter cache for child nested level filters, but the regular filter cache instead.
Random access based bitsets are not required for the child level nested level filters.

Closes #8810
2015-02-19 21:34:03 +01:00
Martijn van Groningen b2b7506ace inner hits: Don't fail if an object is specified as a nested value instead of an array.
Closes #9723
2015-02-19 21:18:38 +01:00
Michael McCandless 4859ce5d79 Core: include timestamp and params in hot threads
Include the timestamp and params (interval, busiestThreads,
ignoreIdleThreads), when hot threads were collected.

Closes #9773
2015-02-19 15:10:17 -05:00
Robert Muir 7dad162377 tests: minimize tests.policy a bit.
Closes #9765
2015-02-19 13:27:04 -05:00
Alexander Reelsen 5de564a19f Testing: Remove plus sign from YAML test due to encoding issue
The plus sign is not treated correctly in encoding and can lead
to problems, if the search request is encoded as HTTP parameter
instead of the HTTP body.

Relates #9769
2015-02-19 18:35:39 +01:00
Simon Willnauer 50a56b6c9d [RECOVERY] Only iterate the files that we recovered from the commit
Today we use Directory#listAll() to find all the files we recovered. Yet,
this is not accurate since there might be leftovers etc. It's better to
only iterate over the known files from the segments info that we recovered.
2015-02-19 17:37:37 +01:00
Adrien Grand 4708227ecf Codecs: Remove the ability to have custom per-field postings and doc values formats.
This commit makes the `postings_format` and `doc_values_format` options of
mappings illegal on 2.0 and ignored on 1.x (meaning that the default postings
and doc values formats from the codec will be used in such a case).

This removes a fair amount of code.

Close #8746 #9741
2015-02-19 15:47:25 +01:00
Ryan Ernst 2a217c2d20 add static bwc indexes for 1.3.9 and 1.4.4 2015-02-19 06:26:15 -08:00
Alexander Reelsen 1b54c22e94 Versioning: Adding 1.3.9 and 1.4.5 versions 2015-02-19 14:56:23 +01:00
Simon Willnauer 2064ba33f2 [TEST] Never flush in the test where we rely on recovering from translog 2015-02-19 14:53:01 +01:00
Simon Willnauer 795dd8111f Use index name rather than Index.java to lookup IndexService 2015-02-19 13:52:48 +01:00
Alexander Reelsen fe2b30230f Build: Check that all packages contain the same jar files
This should ensure, that RPM and DEB packages do not lack dependencies,
that have been included in the tar.gz and zip files
2015-02-19 10:58:15 +01:00
Simon Willnauer 3fe2d7cda8 [TEST] Relax path clearing assertion on Windows 2015-02-19 09:34:02 +01:00
Alexander Reelsen 2dc70597c2 Testing: Added REST tests to check if expression scripting works 2015-02-19 09:15:28 +01:00
Lee Hinman eb666f7f50 Add shadow replicas for shared filesystems
Squashed commit of the following:

commit 20835037c98e7d2fac4206c372717a05a27c4790
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 15:27:17 2015 -0700

    Use Enum for "_primary" preference

commit 325acbe4585179190a959ba3101ee63b99f1931a
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 14:32:41 2015 -0700

    Use ?preference=_primary automatically for realtime GET operations

commit edd49434af5de7e55928f27a1c9ed0fddb1fb133
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 14:32:06 2015 -0700

    Move engine creation into protected createNewEngine method

commit 67a797a9235d4aa376ff4af16f3944d907df4577
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 13:14:01 2015 -0700

    Factor out AssertingSearcher so it can be used by mock Engines

commit 62b0c28df8c23cc0b8205b33f7595c68ff940e2b
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 11:43:17 2015 -0700

    Use IndexMetaData.isIndexUsingShadowReplicas helper

commit 1a0d45629457578a60ae5bccbeba05acf5d79ddd
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:59:31 2015 -0700

    Rename usesSharedFilesystem -> isOnSharedFilesystem

commit 73c62df4fc7da8a5ed557620a83910d89b313aa1
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:58:02 2015 -0700

    Add MockShadowEngine and hook it up to be used

commit c8e8db473830fce1bdca3c4df80a685e782383bc
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:45:50 2015 -0700

    Clarify comment about pre-defined mappings

commit 60a4d5374af5262bd415f4ef40f635278ed12a03
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:18:22 2015 -0700

    Add a test for shadow replicas that uses field data

commit 7346f9f382f83a21cd2445b3386fe67472bc3184
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:37:14 2015 -0700

    Revert changes to RecoveryTarget.java

commit d90d6980c9b737bd8c0f4339613a5373b1645e95
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:35:44 2015 -0700

    Rename `ownsShard` to `canDeleteShardContent`

commit 23001af834d66278ac84d9a72c37b5d1f3a10a7b
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:35:25 2015 -0700

    Remove ShadowEngineFactory, add .newReadOnlyEngine method in EngineFactory

commit b64fef1d2c5e167713e869b22d388ff479252173
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:25:19 2015 -0700

    Add warning that predefined mappings should be used

commit a1b8b8cf0db49d1bd1aeb84e51491f7f0de43b59
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 14:31:50 2015 -0700

    Remove unused import and fix index creation example in docs

commit 0b1b852365ceafc0df86866ac3a4ffb6988b08e4
Merge: b9d1fed a22bd49
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 10:56:02 2015 -0700

    Merge remote-tracking branch 'refs/remotes/origin/master' into shadow-replicas

commit b9d1fed25ae472a9dce1904eb806702fba4d9786
Merge: 4473e63 41fd4d8
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 09:02:27 2015 -0700

    Merge remote-tracking branch 'refs/remotes/origin/master' into shadow-replicas

commit 4473e630460e2f0ca2a2e2478f3712f39a64c919
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 09:00:39 2015 -0700

    Add asciidoc documentation for shadow replicas

commit eb699c19f04965952ae45e2caf107124837c4654
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 16:15:39 2015 +0100

    remove last nocommit

commit c5ece6d16d423fbdd36f5d789bd8daa5724d77b0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 16:13:12 2015 +0100

    simplify shadow engine

commit 45cd34a12a442080477da3ef14ab2fe7947ea97e
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:32:57 2015 +0100

    fix tests

commit 744f228c192602a6737051571e040731d413ba8b
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:28:12 2015 +0100

    revert changes to IndexShardGateway - these are leftovers from previous iterations

commit 11886b7653dabc23655ec76d112f291301f98f4a
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:26:48 2015 +0100

    Back out non-shared FS code. this will go in in a second iteration

commit 77fba571f150a0ca7fb340603669522c3ed65363
Merge: e8ad614 2e3c6a9
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:16:46 2015 +0100

    Merge branch 'master' into shadow-replicas

    Conflicts:
    	src/main/java/org/elasticsearch/index/engine/Engine.java

commit e8ad61467304e6d175257e389b8406d2a6cf8dba
Merge: 48a700d 1b8d8da
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 10:54:20 2015 +0100

    Merge branch 'master' into shadow-replicas

commit 48a700d23cff117b8e4851d4008364f92b8272a0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 10:50:59 2015 +0100

    add test for failing shadow engine / remove nocommit

commit d77414c5e7b2cde830a8e3f70fe463ccc904d4d0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 10:27:56 2015 +0100

    remove nocommits in IndexMetaData

commit abb696563a9e418d3f842a790fcb832f91150be2
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 17:05:02 2015 +0100

    remove nocommit and simplify delete logic

commit 82b9f0449108cd4741568d9b4495bf6c10a5b019
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 16:45:27 2015 +0100

    reduce the changes compared to master

commit 28f069b6d99a65e285ac8c821e6a332a1d8eb315
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 16:43:46 2015 +0100

    fix primary relocation

commit c4c999dd61a44a7a0db9798275a622f2b85b1039
Merge: 2ae80f9 455a85d
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 15:04:26 2015 +0100

    Merge branch 'master' into shadow-replicas

commit 2ae80f9689346f8fd346a0d3775a6341874d8bef
Author: Lee Hinman <lee@writequit.org>
Date:   Fri Feb 13 16:25:34 2015 -0700

    throw UnsupportedOperationException on write operations in ShadowEngine

commit 740c28dd9ef987bf56b670fa1a8bcc6de2845819
Merge: e5bc047 305ba33
Author: Lee Hinman <lee@writequit.org>
Date:   Fri Feb 13 15:38:39 2015 -0700

    Merge branch 'master' into shadow-replicas

commit e5bc047d7c872ae960d397b1ae7b4b78d6a1ea10
Author: Lee Hinman <lee@writequit.org>
Date:   Fri Feb 13 11:38:09 2015 -0700

    Don't replicate document request when using shadow replicas

commit 213292e0679d8ae1492ea11861178236f4abd8ea
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Feb 13 13:58:05 2015 +0100

    add one more nocommit

commit 83d171cf632f9b77cca9de58505f7db8fcda5599
Merge: aea9692 09eb8d1
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Feb 13 13:52:29 2015 +0100

    Merge branch 'master' into shadow-replicas

commit aea96920d995dacef294e48e719ba18f1ecf5860
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Feb 13 09:56:41 2015 +0100

    revert unneeded changes on Store

commit ea4e3e58dc6959a92c06d5990276268d586735f3
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 14:26:30 2015 -0700

    Add documentation to ShadowIndexShard, remove nocommit

commit 4f71c8d9f706a0c1c39aa3a370efb1604559d928
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 14:17:22 2015 -0700

    Add documentation to ShadowEngine

commit 28a9d1842722acba7ea69e0fa65200444532a30c
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 14:08:25 2015 -0700

    Remove nocommit, document canDeleteIndexContents

commit d8d59dbf6d0525cd823d97268d035820e5727ac9
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 10:34:32 2015 -0700

    Refactor more shared methods into the abstract Engine

commit a7eb53c1e8b8fbfd9281b43ae39eacbe3cd1a0a6
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 17:38:59 2015 +0100

    Simplify shared filesystem recovery by using a dedicated recovery handler that skip
    most phases and enforces shard closing on the soruce before the target opens it's engine

commit a62b9a70adad87d7492c526f4daf868cb05018d9
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 15:59:54 2015 +0100

    fix compile error after upstream changes

commit abda7807bc3328a89fd783ca7ad8c6deac35f16f
Merge: f229719 35f6496
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 15:57:28 2015 +0100

    Merge branch 'master' into shadow-replicas

    Conflicts:
    	src/main/java/org/elasticsearch/index/engine/Engine.java

commit f2297199b7dd5d3f9f1f109d0ddf3dd83390b0d1
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 12:41:32 2015 +0100

    first cut at catchup from primary
    make flush to a refresh
    factor our ShadowIndexShard to have IndexShard be idential to the master and least intrusive

    cleanup abstractions

commit 4a367c07505b84b452807a58890f1cbe21711f27
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 09:50:36 2015 +0100

    fix primary promotion

commit cf2fb807e7e243f1ad603a79bc9d5f31a499b769
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 16:45:41 2015 -0700

    Make assertPathHasBeenCleared recursive

commit 5689b7d2f84ca1c41e4459030af56cb9c0151eff
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:58:19 2015 -0700

    Add testShadowReplicaNaturalRelocation

commit fdbe4133537eaeb768747c2200cfc91878afeb97
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:28:57 2015 -0700

    Use check for shared filesystem in primary -> primary relocation

    Also adds a nocommit

commit 06e2eb4496762130af87ce68a47d360962091697
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:21:32 2015 -0700

    Add a test checking that indices with shadow replicas clean up after themselves

commit e4dbfb09a689b449f0edf6ee24222d7eaba2a215
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:08:18 2015 -0700

    Fix segment info for ShadowEngine, remove test nocommit

commit 80cf0e884c66eda7d59ac5d59235e1ce215af8f5
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 14:30:13 2015 -0700

    Remove nocommit in ShadowEngineTests#testFailStart()

commit 5e33eeaca971807b342f9be51a6a566eee005251
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 14:22:59 2015 -0700

    Remove overly-complex test

commit 2378fbb917b467e79c0262d7a41c23321bbeb147
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 13:45:44 2015 -0700

    Fix missing import

commit 52e9cd1b8334a5dd228d5d68bd03fd0040e9c8e9
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 13:45:05 2015 -0700

    Add a test for replica -> primary promotion

commit a95adbeded426d7f69f6ddc4cbd6712b6f6380b4
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 12:54:14 2015 -0700

    Remove tests that don't apply to ShadowEngine

commit 1896feda9de69e4f9cf774ef6748a5c50e953946
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 10:29:12 2015 -0700

    Add testShadowEngineIgnoresWriteOperations and testSearchResultRelease

commit 67d7df41eac5e10a1dd63ddb31de74e326e9d38b
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 10:06:05 2015 -0700

    Add start of ShadowEngine unit tests

commit ca9beb2d93d9b5af9aa6c75dbc0ead4ef57e220d
Merge: 2d42736 57a4646
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 18:03:53 2015 +0100

    Merge branch 'master' into shadow-replicas

commit 2d42736fed3ed8afda7e4aff10b65d292e1c6f92
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 17:51:22 2015 +0100

    shortcut recovery if we are on a shared FS - no need to compare files etc.

commit 24d36c92dd82adce650e7ac8e9f0b43c83b2dc53
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 17:08:08 2015 +0100

    utilize the new delete code

commit 2a2eed10f58825aae29ffe4cf01aefa5743a97c7
Merge: 343dc0b 173cfc1
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 16:07:41 2015 +0100

    Merge branch 'master' into shadow-replicas

    Conflicts:
    	src/main/java/org/elasticsearch/gateway/GatewayMetaState.java

commit 343dc0b527a7052acdc783ac5abcaad1ef78dbda
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 16:05:28 2015 +0100

    long adder is not available in java7

commit be02cabfeebaea74b51b212957a2a466cfbfb716
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 10 22:04:24 2015 -0700

    Add test that restarts nodes to ensure shadow replicas recover

commit 7fcb373f0617050ca1a5a577b8cf32e32dc612b0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 10 23:19:21 2015 +0100

    make test more evil

commit 38135af0c1991b88f168ece0efb72ffe9498ff59
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 10 22:25:11 2015 +0100

    make tests pass

commit 05975af69e6db63cb95f3e40d25bfa7174e006ea
Author: Lee Hinman <lee@writequit.org>
Date:   Mon Jan 12 18:44:29 2015 +0100

    Add ShadowEngine
2015-02-18 15:34:06 -07:00
Simon Willnauer cef1c97ad4 [TEST] Add percolate type to the mapping 2015-02-18 16:06:28 +01:00
Christoph Büscher 6d41601b4e Aggregations: Format bucket key_as_string in `date_histogram` according to `time_zone`
Change bucket key_as_string to reflect `time_zone` parameter. Currently `time_zone`
shifts bucket boundaries to other time zone, but keys are displayed in UTC, so e.g.
daily buckets in "+01:00" time zone have key_as_string like "2014-01-01T23:00:00Z". With this
change the default is to format this dates according to the local time zone, so the
above bucket key would be "2014-01-02T00:00:00+01:00".

Closes #9710
Closes #9744
2015-02-18 15:49:38 +01:00
Simon Willnauer 4125e406c7 [TEST] prevent stackoverflow in mockInternalEngine 2015-02-18 14:40:22 +01:00
Simon Willnauer 8c3cb6c613 [TEST] don't cache store stats in tests that ensure stats are correct 2015-02-18 14:02:23 +01:00
Simon Willnauer dd740c2763 [TEST] remove wrong assertion - stream must be closed to assert the size 2015-02-18 13:54:14 +01:00
Simon Willnauer ddd16deb1d [ENGINE] Flush IndexWriter to disk on close and shutdown
Today we trash everything that has been indexed but not flushed to disk
if the engine is closed. This might not be desired if we shutting down a
node for restart / upgrade or if we close / archive an index. In such a
case we would like to flush the transaction log and commit everything to
disk. This commit adds a flag to the close method that is set on close
and shutdown but not when we remove the shard due to relocations
2015-02-18 13:41:28 +01:00
Simon Willnauer 85c611a1b7 [STORE] Add simple cache for StoreStats
this commit tries to reduce the filesystem calls to fetch metadata
by using a simple cache on top of the stats call.

Relates to #9683
2015-02-18 12:29:39 +01:00
Michael McCandless a7e238dbb8 Test: include dir listing when check index fails 2015-02-18 04:31:53 -05:00
David Pilato 3c2414128c Rename Maven repository id
We are using repository ids with spaces in our `pom.xml`. Although it's not forbidden, a common practice is to avoid space in id.

This commit changes codehaus snapshots and lucene snapshots to a consistent naming (using a dash, all lowercase).
We also add a name which is used by Maven when displaying some information about the repository.

This naming is also consistent with [elasticsearch-parent project](https://github.com/elasticsearch/elasticsearch-parent) which will be used in the next future in 1.x and master branch.

**Important note**: If you have trouble to compile elasticsearch or a plugin using `mvn compile` and hit a `Access denied to: [URL_HERE], ReasonPhrase: Forbidden. -> [Help 1]`, you can remove related maven files:

```sh
find ~/.m2/repository -name _remote.repositories -exec rm -v {} \;
find ~/.m2/repository -name _maven.repositories -exec rm -v {} \;
```

Another option is to tell Maven not using those files with `--llr`:

```sh
mvn compile --llr
```
2015-02-17 20:39:04 +01:00
Lee Hinman a22bd493a4 [CORE] move InternalEngine.segmentStats() into abstract Engine 2015-02-17 10:17:12 -07:00
Martijn van Groningen add4339446 Added nested scope to query parse context that keeps track the current nested level during search request parsing.
The nested scope is set by any nested feature, so that sub nested queries and filters know about their context and these sub nested queries and filters can construct the right parent filter.
Removed the LateBindingParentFilter workaround in the nested query parser in favour of the nested scope maintained in the query parse context.
Due to this change nested queries and filters can now also be included in nested sorting and inner hits, because those features also now use the nested scope.

This change doesn't fix the usage of nested filters in nested and reverse_nested aggregations. The `nested` filter shouldn't be used inside these aggregations and instead the `nested` and `reverse_nested` aggs should be used to query on the right level. In a different change `nested` inside a `nested` and `reverse_nested` aggregation should result in a parse error.

Closes #9305
2015-02-17 17:53:46 +01:00
Simon Willnauer 41fd4d8722 [GATEWAY] Don't rename recovery translog in gateway
we used to reuse files / write them twice in translog. Today we don't
do this anymore so the rename logic is unneeded.
2015-02-17 12:55:48 +01:00
Simon Willnauer 2e3c6a9118 [ENGINE] Move more methods into abstract Engine 2015-02-17 11:04:46 +01:00
Simon Willnauer 1b8d8da648 [TEST] use util method to propperly shudtown threadpools 2015-02-17 09:32:50 +01:00
Christoph Büscher 30fd70f07b Aggregations: Simplify time zone option in `date_histogram`
Removed the existing `pre_zone` and `post_zone` option in `date_histogram` in favor of
the simpler `time_zone` option. Previously, specifying different values for these could
lead to confusing scenarios where ES would return bucket keys that are not UTC.
Now `time_zone` is the only option setting, the calculation of date buckets to take place in the
preferred time zone, but after rounding converting the bucket key values back to UTC.

Closes #9062
Closes #9637
2015-02-16 16:54:06 +01:00
Alex Ksikes 455a85dc3b Term Vectors: requests are now timed
When asking for term statistics, generating term vectors on the fly or with
`dfs` set to `true`, some requests may take a while, so it is useful to know
exactly how long.

Closes #9583
2015-02-16 14:08:15 +01:00
Adrien Grand 9f4c56a7b8 Internal: Remove XCollector.
We don't actually need this interface, we can just run the post-collection
operation once collection is finished on impls that need it.

Close #9677
2015-02-16 12:03:54 +01:00
Adam 460e8d34fc Removed 'Master' from names.txt
Fixes #9698
2015-02-16 10:59:50 +00:00
Simon Willnauer 0698af93c2 [BUILD] Add marvel and license to the release script 2015-02-16 09:46:12 +01:00
Alexander Reelsen 5092904885 Packaging: Add antlr and asm dependencies 2015-02-13 23:55:35 +01:00
Lee Hinman 305ba33710 [CORE] Move as much as possible into abstract Engine
This paves the way for more shared code between the `InternalEngine` and
`ShadowEngine` by way of the abstract `Engine` class. No actual
functionality has been changed.
2015-02-13 14:43:58 -07:00