Commit Graph

11344 Commits

Author SHA1 Message Date
Zachary Tong a03cefcece [DOCS] Add documentation for moving average 2015-04-22 18:59:39 -04:00
Zachary Tong 30177887b1 Add prediction capability to MovAvgReducer
This commit adds the ability for moving average models to output a "prediction" based on the current
moving average model.  For simple, linear and single, this prediction is simply converges on the
moving average's mean at the last point, leading to a straight line.  For double, this will
predict in the direction of the linear trend (either globally or locally, depending on beta).

Also adds some more tests.

Closes #10545
2015-04-22 18:59:30 -04:00
Colin Goodheart-Smithe dcf91ff02f Temporarily disabled gap policy randomisation in MovAvgTests 2015-04-22 16:01:23 +01:00
Colin Goodheart-Smithe 77e2f644e3 Derivative tests for gaps in metrics 2015-04-22 14:50:49 +01:00
Colin Goodheart-Smithe f6934e0410 unit test for derivative of metric agg with gaps 2015-04-22 10:06:22 +01:00
Colin Goodheart-Smithe 89d424e074 Derivative can now access multi-value metric aggregations 2015-04-21 16:00:02 +01:00
Colin Goodheart-Smithe bd28c9c44e Documentation for the max_bucket reducer 2015-04-21 15:06:20 +01:00
Colin Goodheart-Smithe be647a89d3 Documentation for the derivative reducer 2015-04-21 15:06:20 +01:00
Colin Goodheart-Smithe 0f4b7f3b5c Added section for reducer aggregations in the main aggregation docs page 2015-04-21 15:06:19 +01:00
Colin Goodheart-Smithe ea1470a080 More tests for max bucket reducer 2015-04-20 15:09:10 +01:00
Colin Goodheart-Smithe 7fdf32fb0d changed `bucketsPaths` to `buckets_paths` 2015-04-13 15:13:02 +01:00
Colin Goodheart-Smithe 392f9ce1f8 clean up 2015-04-13 14:34:53 +01:00
Colin Goodheart-Smithe 306d94adb9 Revert "Added normalisation to Derivative Reducer"
This reverts commit 48a94a41df.
2015-04-13 14:24:23 +01:00
Colin Goodheart-Smithe 48a94a41df Added normalisation to Derivative Reducer
This changes adds the ability to specify the units for the x-axis for derivative values and calculate the derivative based on those units rather than the original histograms x-axis units
2015-04-13 11:44:29 +01:00
Colin Goodheart-Smithe 7d38989cdb Merge branch 'master' into feature/aggs_2_0 2015-04-13 09:53:49 +01:00
Uwe Schindler df118214c4 Update forbiddenapis to version 1.8 2015-04-13 01:27:56 +02:00
Britta Weber 4934def035 [TEST] better test start and end messages
print when cleaning up also for SingleNodeTests
distinguish between after suite and after test cleanup
2015-04-11 15:02:40 +02:00
Michael McCandless 3cc3390f17 don't return negative percentage when max < 0 2015-04-11 06:02:59 -04:00
Nicholas Knize 90e1775a2b [GEO] Correct ShapeBuilder coordinate parser to ignore values in 3rd+ dimension
ShapeBuilder's coordinate parser expected 2 double values for every coordinate array. If > 2 doubles were provided the parser terminated parsing of the coordinate array. This resulted in an invalid Shape state leaving LineStrings, LinearRings, and Polygons with a single coordinate. An incorrect parse exception was thrown. This corrects the parser to ignore those values in the 3rd+ dimension, correctly parsing the rest of the coordinate array.

Unit tests have been updated to verify the fix.

closes #10510
2015-04-10 15:33:21 -05:00
Nicholas Knize 754856289e [GEO] Add merge conflicts to GeoShapeFieldMapper
Prevents the user from changing strategies, tree, tree_level or precision. distance_error_pct changes are allowed as they do not compromise the integrity of the index. A separate issue is open for allowing users to change tree_level or precision.
2015-04-10 14:33:17 -05:00
Nicholas Knize a8a35d7c29 [GEO] Fix hole intersection at tangential coordinate
OGC SFA 2.1.10 assertion 3 allows interior boundaries to touch exterior boundaries provided they intersect at a single point. Issue #9511 provides an example where a valid shape is incorrectly interpreted as invalid (a false violation of assertion 3).  When the intersecting point appears as the first and last coordinate of the interior boundary in a polygon, the ShapeBuilder incorrectly counted this as multiple intersecting vertices. The fix required a little more than just a logic check. Passing the duplicate vertices resulted in a connected component in the edge graph causing an invalid self crossing polygon. This required additional logic to the edge assignment in order to correctly segment the connected components. Finally, an additional hole validation has been added along with proper unit tests for testing valid and invalid conditions (including dateline crossing polys).

closes #9511
2015-04-10 11:44:40 -05:00
David Pursehouse 97a9b4ec2f Fix m2e Eclipse Integration
Update Eclipse core prefs

Eclipse Luna overwrites the prefs file, putting all the settings
in alphabetical order and removing comments. This causes the prefs
files to be modified in the git workspace.

Update the file with the version generated by Eclipse to prevent it
from being modified every time.

No settings values are modified by this change.

This also  adds another plugin to the lifecycle mapping in the pom.xml which was missed in https://github.com/elastic/elasticsearch/pull/10524.
2015-04-10 16:48:04 +01:00
Adrien Grand ab8926bc6a Docs: fix build. 2015-04-10 17:38:36 +02:00
Robert Muir b936ec9a25 allow reflection of MXBean for file descriptor stats 2015-04-10 11:28:30 -04:00
Adrien Grand 0b413bd670 Merge pull request #10532 from jpountz/deprecate/limit_filter
Search: deprecate the limit filter.

Close #10532
2015-04-10 17:19:43 +02:00
Adrien Grand 5b3cc2f07c Search: deprecate the limit filter.
This is really a Collector instead of a filter. This commit deprecates the
`limit` filter, makes it a no-op and recommends to use the `terminate_after`
parameter instead that we introduced in the meantime.
2015-04-10 17:18:50 +02:00
Adrien Grand bad6656f02 Merge pull request #10391 from jpountz/fix/remove_flt
Queries: Remove fuzzy-like-this support.

Close #10391
2015-04-10 17:17:00 +02:00
Adrien Grand 919589b908 Queries: Remove fuzzy-like-this support.
The fuzzy-like-this query builds very expensive queries and only serves esoteric
use-cases.
2015-04-10 17:16:02 +02:00
Simon Willnauer bdc4c7f84c [BUILD] Add sigar binaries when running unittests
the sigar binaries are not available when running tests today. This
commit adds the path to the test run.
2015-04-10 14:37:18 +02:00
Colin Goodheart-Smithe 7e353e0768 removed erroneous tab character 2015-04-10 10:37:26 +01:00
Colin Goodheart-Smithe 4a09a98561 Fix to pom.xml to allow eclipse maven integration using m2e 2015-04-10 08:34:01 +01:00
Ryan Ernst 224c43564b Tests: Add multi data path testing to static bwc tests
This randomly chooses to run bwc indexes on single or multi data paths.

closes #10519
2015-04-09 14:31:31 -07:00
Ryan Ernst 9d0764e5df Tests: increasing timeout for replicas in static bwc tests, to account
for reallySlowJenkinsWhenIndexHasLotsOfSegments
2015-04-09 13:31:08 -07:00
Michael McCandless 78612cf0a8 fix indent 2015-04-09 14:47:48 -04:00
Michael McCandless 54b702db07 Tests: don't fsync so often
Most tests don't "really" need to fsync, and this is costly (makes
tests slower, wears out our SSDs).

This change makes it uncommon to actually fsync when Lucene asks for
it.  It's just a workaround (in MockDirectoryHelper) until we can
cutover Elasticseach to use MockFileSystem like Lucene.

Closes #10516
2015-04-09 14:31:42 -04:00
Michael McCandless 2f5cbf5f9d Core: log path.data filesystem details to INFO level
This change logs total space, free space, usable free space, an
estimate of whether the IO system spins (e.g., SSD or not), the mount
point and filesystem type, on node startup.

It produces log output like this:

[2015-04-09 12:09:30,244][INFO ][env                      ] [node_t0] node data locations details:
  -> /l/es.logspins/target/J0/data/TEST-haswell-CHILD_VM=[0]-CLUSTER_SEED=[2926863498862121027]-HASH=[AFC194B1B384B]/nodes/0, free_space [260.6gb], usable_space [256.3gb], total_space [465gb], spins? [no], mount [/ (/dev/mapper/haswell--vg-root)], type [btrfs]

Closes #10502
2015-04-09 13:11:49 -04:00
Simon Willnauer d470bdbff6 Only flush for checkindex if we have uncommitted changes
Today we force a flush before check index to ensure we have an index
to check on. Yet if the index is large and the FS is slow this can have
significant impact on the index deletion performance. This commit introduces
a check if there are any uncommitted changes in order to skip the additional commit.

Closes #10505
2015-04-09 18:25:57 +02:00
javanna 5367e04fbc [TEST] add 1.5.1 index for bw comp tests 2015-04-09 17:03:35 +02:00
Colin Goodheart-Smithe 56a37620ed Merge branch 'master' into feature/aggs_2_0
Conflicts:
	src/main/java/org/elasticsearch/percolator/PercolatorService.java
	src/main/java/org/elasticsearch/search/aggregations/AggregationPhase.java
	src/main/java/org/elasticsearch/search/aggregations/InternalAggregation.java
	src/main/java/org/elasticsearch/search/aggregations/InternalAggregations.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/InternalSingleBucketAggregation.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/filters/InternalFilters.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/InternalGeoHashGrid.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/InternalHistogram.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/range/InternalRange.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/GlobalOrdinalsSignificantTermsAggregator.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/InternalSignificantTerms.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantStringTermsAggregator.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/significant/UnmappedSignificantTerms.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/GlobalOrdinalsStringTermsAggregator.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/InternalTerms.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/StringTermsAggregator.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java
	src/main/java/org/elasticsearch/search/aggregations/bucket/terms/UnmappedTerms.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/avg/InternalAvg.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/cardinality/InternalCardinality.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/geobounds/InternalGeoBounds.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/max/InternalMax.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/min/InternalMin.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/percentiles/AbstractInternalPercentiles.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetric.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/stats/InternalStats.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/stats/extended/InternalExtendedStats.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/sum/InternalSum.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/tophits/InternalTopHits.java
	src/main/java/org/elasticsearch/search/aggregations/metrics/valuecount/InternalValueCount.java
2015-04-09 15:33:34 +01:00
javanna 4608f868bf add 1.5.2 snapshot version and and make 1.5.1 a released version 2015-04-09 16:15:05 +02:00
Colin Goodheart-Smithe fcc09f62b9 Aggregations: removed aggregations from ReduceContext
ReduceContext contains the list of aggregations to reduce but these aggregations are set as null half of the time. This change makes the reduce(ReduceContext) method changed to reduce(List<InternalAggregation>, ReduceContext) and ReduceContext now only holds the BigArrays and Script services.
2015-04-09 14:58:17 +01:00
Simon Willnauer 3b41299273 Add missing hashCode method to RecoveryState#File 2015-04-09 15:07:28 +02:00
Clinton Gormley abc7de96ae Docs: Updated version annotations in master 2015-04-09 14:50:11 +02:00
David Pilato 88ee7a5dca Deprecate rivers
* In code, we mark `River`, `AbstractRiverComponent`, `RiverComponent` and `RiverName` classes as deprecated
* We log that information when a cluster is still using it
* We add this information in the plugins list as well
2015-04-09 14:29:16 +02:00
Colin Goodheart-Smithe e19d20b407 max bucket reducer and sibling reducer framework 2015-04-09 12:16:35 +01:00
Adrien Grand fae124103a Merge pull request #10420 from jpountz/feature/numeric_resolution
Mappings: Bring back numeric_resolution.

Close #10420
2015-04-09 12:28:33 +02:00
Adrien Grand e25db222ee Merge pull request #10418 from jpountz/enhancement/speed_up_aggs_include_exclude
Aggregations: Speed up include/exclude in terms aggregations with regexps.

Close #10418
2015-04-09 12:16:37 +02:00
Adrien Grand aecd9ac515 Aggregations: Speed up include/exclude in terms aggregations with regexps.
Today we check every regular expression eagerly against every possible term.
This can be very slow if you have lots of unique terms, and even the bottleneck
if your query is selective.

This commit switches to Lucene regular expressions instead of Java (not exactly
the same syntax yet most existing regular expressions should keep working) and
uses the same logic as RegExpQuery to intersect the regular expression with the
terms dictionary. I wrote a quick benchmark (in the PR) to make sure it made
things faster and the same request that took 750ms on master now takes 74ms with
this change.

Close #7526
2015-04-09 12:12:56 +02:00
Adrien Grand 2a844fc457 Merge pull request #10411 from jpountz/fix/multi_level_breadth_first
Aggregations: Fix multi-level breadth-first aggregations.

Close #10411
2015-04-09 12:09:48 +02:00
Adrien Grand 6b16b32174 Aggregations: Fix multi-level breadth-first aggregations.
The refactoring in #9544 introduced a regression that broke multi-level
aggregations using breadth-first. This was due to sub-aggregators creating
deferred collectors before their parent aggregator and then the parent
aggregator trying to collect sub aggregators directly instead of going through
the deferred wrapper.

This commit fixes the issue but we should try to simplify all the pre/post
collection logic that we have.

Also `breadth_first` is now automatically ignored if the sub aggregators need
scores (just like we ignore `execution_mode` when the value does not make sense
like using ordinals on a script).

Close #9823
2015-04-09 12:06:08 +02:00