Commit Graph

11337 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe be647a89d3 Documentation for the derivative reducer 2015-04-21 15:06:20 +01:00
Colin Goodheart-Smithe 0f4b7f3b5c Added section for reducer aggregations in the main aggregation docs page 2015-04-21 15:06:19 +01:00
Colin Goodheart-Smithe ea1470a080 More tests for max bucket reducer 2015-04-20 15:09:10 +01:00
Colin Goodheart-Smithe 7fdf32fb0d changed `bucketsPaths` to `buckets_paths` 2015-04-13 15:13:02 +01:00
Colin Goodheart-Smithe 392f9ce1f8 clean up 2015-04-13 14:34:53 +01:00
Colin Goodheart-Smithe 306d94adb9 Revert "Added normalisation to Derivative Reducer"
This reverts commit 48a94a41df.
2015-04-13 14:24:23 +01:00
Colin Goodheart-Smithe 48a94a41df Added normalisation to Derivative Reducer
This changes adds the ability to specify the units for the x-axis for derivative values and calculate the derivative based on those units rather than the original histograms x-axis units
2015-04-13 11:44:29 +01:00
Colin Goodheart-Smithe 7d38989cdb Merge branch 'master' into feature/aggs_2_0 2015-04-13 09:53:49 +01:00
Uwe Schindler df118214c4 Update forbiddenapis to version 1.8 2015-04-13 01:27:56 +02:00
Britta Weber 4934def035 [TEST] better test start and end messages
print when cleaning up also for SingleNodeTests
distinguish between after suite and after test cleanup
2015-04-11 15:02:40 +02:00
Michael McCandless 3cc3390f17 don't return negative percentage when max < 0 2015-04-11 06:02:59 -04:00
Nicholas Knize 90e1775a2b [GEO] Correct ShapeBuilder coordinate parser to ignore values in 3rd+ dimension
ShapeBuilder's coordinate parser expected 2 double values for every coordinate array. If > 2 doubles were provided the parser terminated parsing of the coordinate array. This resulted in an invalid Shape state leaving LineStrings, LinearRings, and Polygons with a single coordinate. An incorrect parse exception was thrown. This corrects the parser to ignore those values in the 3rd+ dimension, correctly parsing the rest of the coordinate array.

Unit tests have been updated to verify the fix.

closes #10510
2015-04-10 15:33:21 -05:00
Nicholas Knize 754856289e [GEO] Add merge conflicts to GeoShapeFieldMapper
Prevents the user from changing strategies, tree, tree_level or precision. distance_error_pct changes are allowed as they do not compromise the integrity of the index. A separate issue is open for allowing users to change tree_level or precision.
2015-04-10 14:33:17 -05:00
Nicholas Knize a8a35d7c29 [GEO] Fix hole intersection at tangential coordinate
OGC SFA 2.1.10 assertion 3 allows interior boundaries to touch exterior boundaries provided they intersect at a single point. Issue #9511 provides an example where a valid shape is incorrectly interpreted as invalid (a false violation of assertion 3).  When the intersecting point appears as the first and last coordinate of the interior boundary in a polygon, the ShapeBuilder incorrectly counted this as multiple intersecting vertices. The fix required a little more than just a logic check. Passing the duplicate vertices resulted in a connected component in the edge graph causing an invalid self crossing polygon. This required additional logic to the edge assignment in order to correctly segment the connected components. Finally, an additional hole validation has been added along with proper unit tests for testing valid and invalid conditions (including dateline crossing polys).

closes #9511
2015-04-10 11:44:40 -05:00
David Pursehouse 97a9b4ec2f Fix m2e Eclipse Integration
Update Eclipse core prefs

Eclipse Luna overwrites the prefs file, putting all the settings
in alphabetical order and removing comments. This causes the prefs
files to be modified in the git workspace.

Update the file with the version generated by Eclipse to prevent it
from being modified every time.

No settings values are modified by this change.

This also  adds another plugin to the lifecycle mapping in the pom.xml which was missed in https://github.com/elastic/elasticsearch/pull/10524.
2015-04-10 16:48:04 +01:00
Adrien Grand ab8926bc6a Docs: fix build. 2015-04-10 17:38:36 +02:00
Robert Muir b936ec9a25 allow reflection of MXBean for file descriptor stats 2015-04-10 11:28:30 -04:00
Adrien Grand 0b413bd670 Merge pull request #10532 from jpountz/deprecate/limit_filter
Search: deprecate the limit filter.

Close #10532
2015-04-10 17:19:43 +02:00
Adrien Grand 5b3cc2f07c Search: deprecate the limit filter.
This is really a Collector instead of a filter. This commit deprecates the
`limit` filter, makes it a no-op and recommends to use the `terminate_after`
parameter instead that we introduced in the meantime.
2015-04-10 17:18:50 +02:00
Adrien Grand bad6656f02 Merge pull request #10391 from jpountz/fix/remove_flt
Queries: Remove fuzzy-like-this support.

Close #10391
2015-04-10 17:17:00 +02:00
Adrien Grand 919589b908 Queries: Remove fuzzy-like-this support.
The fuzzy-like-this query builds very expensive queries and only serves esoteric
use-cases.
2015-04-10 17:16:02 +02:00
Simon Willnauer bdc4c7f84c [BUILD] Add sigar binaries when running unittests
the sigar binaries are not available when running tests today. This
commit adds the path to the test run.
2015-04-10 14:37:18 +02:00
Colin Goodheart-Smithe 7e353e0768 removed erroneous tab character 2015-04-10 10:37:26 +01:00
Colin Goodheart-Smithe 4a09a98561 Fix to pom.xml to allow eclipse maven integration using m2e 2015-04-10 08:34:01 +01:00
Ryan Ernst 224c43564b Tests: Add multi data path testing to static bwc tests
This randomly chooses to run bwc indexes on single or multi data paths.

closes #10519
2015-04-09 14:31:31 -07:00
Ryan Ernst 9d0764e5df Tests: increasing timeout for replicas in static bwc tests, to account
for reallySlowJenkinsWhenIndexHasLotsOfSegments
2015-04-09 13:31:08 -07:00
Michael McCandless 78612cf0a8 fix indent 2015-04-09 14:47:48 -04:00
Michael McCandless 54b702db07 Tests: don't fsync so often
Most tests don't "really" need to fsync, and this is costly (makes
tests slower, wears out our SSDs).

This change makes it uncommon to actually fsync when Lucene asks for
it.  It's just a workaround (in MockDirectoryHelper) until we can
cutover Elasticseach to use MockFileSystem like Lucene.

Closes #10516
2015-04-09 14:31:42 -04:00
Michael McCandless 2f5cbf5f9d Core: log path.data filesystem details to INFO level
This change logs total space, free space, usable free space, an
estimate of whether the IO system spins (e.g., SSD or not), the mount
point and filesystem type, on node startup.

It produces log output like this:

[2015-04-09 12:09:30,244][INFO ][env                      ] [node_t0] node data locations details:
  -> /l/es.logspins/target/J0/data/TEST-haswell-CHILD_VM=[0]-CLUSTER_SEED=[2926863498862121027]-HASH=[AFC194B1B384B]/nodes/0, free_space [260.6gb], usable_space [256.3gb], total_space [465gb], spins? [no], mount [/ (/dev/mapper/haswell--vg-root)], type [btrfs]

Closes #10502
2015-04-09 13:11:49 -04:00
Simon Willnauer d470bdbff6 Only flush for checkindex if we have uncommitted changes
Today we force a flush before check index to ensure we have an index
to check on. Yet if the index is large and the FS is slow this can have
significant impact on the index deletion performance. This commit introduces
a check if there are any uncommitted changes in order to skip the additional commit.

Closes #10505
2015-04-09 18:25:57 +02:00
javanna 5367e04fbc [TEST] add 1.5.1 index for bw comp tests 2015-04-09 17:03:35 +02:00
Colin Goodheart-Smithe 56a37620ed Merge branch 'master' into feature/aggs_2_0
Conflicts:
	src/main/java/org/elasticsearch/percolator/PercolatorService.java
	src/main/java/org/elasticsearch/search/aggregations/AggregationPhase.java
	src/main/java/org/elasticsearch/search/aggregations/InternalAggregation.java
	src/main/java/org/elasticsearch/search/aggregations/InternalAggregations.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/InternalSingleBucketAggregation.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/filters/InternalFilters.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/InternalGeoHashGrid.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/InternalHistogram.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/range/InternalRange.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/GlobalOrdinalsSignificantTermsAggregator.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/InternalSignificantTerms.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantStringTermsAggregator.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/UnmappedSignificantTerms.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/GlobalOrdinalsStringTermsAggregator.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/InternalTerms.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/StringTermsAggregator.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/UnmappedTerms.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/avg/InternalAvg.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/cardinality/InternalCardinality.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/geobounds/InternalGeoBounds.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/max/InternalMax.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/min/InternalMin.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/percentiles/AbstractInternalPercentiles.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetric.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/stats/InternalStats.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/stats/extended/InternalExtendedStats.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/sum/InternalSum.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/tophits/InternalTopHits.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/valuecount/InternalValueCount.java
2015-04-09 15:33:34 +01:00
javanna 4608f868bf add 1.5.2 snapshot version and and make 1.5.1 a released version 2015-04-09 16:15:05 +02:00
Colin Goodheart-Smithe fcc09f62b9 Aggregations: removed aggregations from ReduceContext
ReduceContext contains the list of aggregations to reduce but these aggregations are set as null half of the time. This change makes the reduce(ReduceContext) method changed to reduce(List<InternalAggregation>, ReduceContext) and ReduceContext now only holds the BigArrays and Script services.
2015-04-09 14:58:17 +01:00
Simon Willnauer 3b41299273 Add missing hashCode method to RecoveryState#File 2015-04-09 15:07:28 +02:00
Clinton Gormley abc7de96ae Docs: Updated version annotations in master 2015-04-09 14:50:11 +02:00
David Pilato 88ee7a5dca Deprecate rivers
* In code, we mark `River`, `AbstractRiverComponent`, `RiverComponent` and `RiverName` classes as deprecated
* We log that information when a cluster is still using it
* We add this information in the plugins list as well
2015-04-09 14:29:16 +02:00
Colin Goodheart-Smithe e19d20b407 max bucket reducer and sibling reducer framework 2015-04-09 12:16:35 +01:00
Adrien Grand fae124103a Merge pull request #10420 from jpountz/feature/numeric_resolution
Mappings: Bring back numeric_resolution.

Close #10420
2015-04-09 12:28:33 +02:00
Adrien Grand e25db222ee Merge pull request #10418 from jpountz/enhancement/speed_up_aggs_include_exclude
Aggregations: Speed up include/exclude in terms aggregations with regexps.

Close #10418
2015-04-09 12:16:37 +02:00
Adrien Grand aecd9ac515 Aggregations: Speed up include/exclude in terms aggregations with regexps.
Today we check every regular expression eagerly against every possible term.
This can be very slow if you have lots of unique terms, and even the bottleneck
if your query is selective.

This commit switches to Lucene regular expressions instead of Java (not exactly
the same syntax yet most existing regular expressions should keep working) and
uses the same logic as RegExpQuery to intersect the regular expression with the
terms dictionary. I wrote a quick benchmark (in the PR) to make sure it made
things faster and the same request that took 750ms on master now takes 74ms with
this change.

Close #7526
2015-04-09 12:12:56 +02:00
Adrien Grand 2a844fc457 Merge pull request #10411 from jpountz/fix/multi_level_breadth_first
Aggregations: Fix multi-level breadth-first aggregations.

Close #10411
2015-04-09 12:09:48 +02:00
Adrien Grand 6b16b32174 Aggregations: Fix multi-level breadth-first aggregations.
The refactoring in #9544 introduced a regression that broke multi-level
aggregations using breadth-first. This was due to sub-aggregators creating
deferred collectors before their parent aggregator and then the parent
aggregator trying to collect sub aggregators directly instead of going through
the deferred wrapper.

This commit fixes the issue but we should try to simplify all the pre/post
collection logic that we have.

Also `breadth_first` is now automatically ignored if the sub aggregators need
scores (just like we ignore `execution_mode` when the value does not make sense
like using ordinals on a script).

Close #9823
2015-04-09 12:06:08 +02:00
Simon Willnauer 17c06f06ba [TEST] issue additional flush 2015-04-09 11:56:36 +02:00
Michael McCandless c9afeccac4 merged 2015-04-09 05:01:53 -04:00
Michael McCandless 9981d69e06 Core: add trace logging for the commitTranslog=false case in InternalEngine.flush 2015-04-09 05:00:40 -04:00
Simon Willnauer 953ae63d2b [TEST] beast slow index to fail with trace logging 2015-04-09 10:47:00 +02:00
Simon Willnauer a243b3f924 [LOGGING] Use static logger name in Engine.java
To ensure subclasses like MockInternalEngine which is in a different
package (test.engine) are logging under the same logger name this commit
moves to a static logger class to determin the logger name. This way
all subclasses of engine will log under `index.engine` which also plays
nicely with `@TestLogging` where log messages sometimes disappeared since
they were enabled for the `index.engine` package but not for `test.engine`
2015-04-09 10:07:50 +02:00
Tanguy Leroux e60f61d0a9 Fix maven-resources-plugin warnings
Commit 168238dab6 declared multiple maven-resources-plugin usages instead of declaring multiple executions for the same plugin... resulting to Maven warnings.

Closes #10433
2015-04-09 10:05:10 +02:00
Ryan Ernst a5bfe332e5 Tests: increase logging for lucene commit during flush in InternalEngine 2015-04-09 00:25:05 -07:00