24 Commits

Author SHA1 Message Date
Hendrik Muhs
c2b08bb72f [Transform] add support for percentile aggs (#51808)
make transform ready for multi value aggregations and add support for percentile

fixes #51663
2020-02-04 12:02:20 +01:00
Hendrik Muhs
53e4d1ef07 [Transform] fix TransformRobustnessIT intermittent test failures part 2 (#51523)
add wait for completion in transform robustness test to avoid occasional test failures during cleanup

fixes #51347
2020-01-28 13:37:01 +01:00
Hendrik Muhs
2239ba8c6e
[Transform] avoid mapping problems with index templates (#51368) (#51519)
insert explict mappings for objects in nested output to avoid clashes with index templates

fixes #51321
2020-01-28 11:31:07 +01:00
Hendrik Muhs
61663b495e add an integration test using date_nanos as timestamp (#51477)
add a test for using date_nanos as timestamp field in a continuous transform
2020-01-28 10:10:23 +01:00
Hendrik Muhs
b233e93014
[Transform] refactor naming leftovers and apply code formating (#51465) (#51470)
refactor renaming leftovers: "data frame transform" to "transforms", touch only internals (variable
names, non-public API's, doc strings, ...) and apply code-formatting (spotless). No logical changes.
2020-01-27 14:04:57 +01:00
Hendrik Muhs
d177747f66 fix TransformRobustnessIT intermittent test failures
ensure the cluster is not in some intermediate state when cleaning up.

fixes #51347
2020-01-24 15:19:11 +01:00
Hendrik Muhs
ded7407b4d
[Transform] Adapt tests for error message to 7.x format
adapt messages to 7.x format (#51398)

fixes #51360
2020-01-24 12:17:32 +01:00
Zachary Tong
2e314a133f Mute TransformTaskFailedStateIT#testStartFailedTransform
Tracking issue: https://github.com/elastic/elasticsearch/issues/51360
2020-01-23 12:53:02 -05:00
Zachary Tong
feb2a25761 Mute TransformTaskFailedStateIT#testForceStopFailedTransform
Tracking issue: https://github.com/elastic/elasticsearch/issues/51360
2020-01-23 11:58:36 -05:00
Hendrik Muhs
3553f68f5a [Transform] Handle permanent bulk indexing errors (#51307)
check bulk indexing error for permanent problems and ensure the state goes into failed instead of
retry. Corrects the stats API to show the real error and avoids excessive audit logging.

fixes #50122
2020-01-23 16:17:26 +01:00
Hendrik Muhs
13343b15c9 [Transform] Improve force stop robustness in case of an error (#51072)
If a transform config got lost (e.g. because the internal index disappeared) tasks could not be
stopped using transform API. This change makes it possible to stop transforms without a config,
meaning to remove the background task. In order to do so force must be set to true.
2020-01-17 07:42:21 +01:00
Hendrik Muhs
c3e4405ddf
[7.x][Transform] Transform fix force stop race condition (#49249) (#49420)
fix force stopping transform if indexer state hasn't been written and/or is set to STOPPED. In certain situations the transform could not be stopped, which means the task could not be removed. Introduces improved abstraction in order to better test state handling in future.
2019-11-21 13:52:14 +01:00
Hendrik Muhs
ca912624ec [Transform] improve error handling of script errors (#48887)
improve error handling for script errors, treating it as irrecoverable errors which puts the task
immediately into failed state, also improves the error extraction to properly report the script 
error.

fixes #48467
2019-11-18 10:24:39 +01:00
Rory Hunter
c46a0e8708
Apply 2-space indent to all gradle scripts (#49071)
Backport of #48849. Update `.editorconfig` to make the Java settings the
default for all files, and then apply a 2-space indent to all `*.gradle`
files. Then reformat all the files.
2019-11-14 11:01:23 +00:00
Benjamin Trent
6ea59dd428
[ML][Transforms] add wait_for_checkpoint flag to stop (#47935) (#48591)
Adds `wait_for_checkpoint` for `_stop` API.
2019-10-28 13:02:57 -04:00
Hendrik Muhs
3da91d5f7a
[Transform] Rename internal indexes for transform plugin (#47788) (#47900)
rename internal indexes of transform plugin

- rename audit index and create an alias for accessing it, BWC: add an alias for old indexes to
  keep them working, kibana UI will switch to use the read alias
- rename config index and provide BWC to read from old and new ones
2019-10-11 14:16:17 +02:00
Hendrik Muhs
0e7869128a
[7.5][Transform] introduce new roles and deprecate old ones (#47780) (#47819)
deprecate data_frame_transforms_{user,admin} roles and introduce transform_{user,admin} roles as replacement
2019-10-10 10:31:24 +02:00
Hendrik Muhs
5e0e54f455
[Transform] move root endpoint to _transform with BWC layer (#47127) (#47682)
move the main endpoint to /_transform/ from /_data_frame/transforms/ with providing backwards compatibility and deprecation warnings
2019-10-08 08:59:01 +02:00
Hendrik Muhs
7377ac4637 [Transform] Replace transforms with transform, index constants (#47023)
- replace "transforms" with "transform" for consistency
 - use constants for internal index naming wherever possible and document required changes
2019-09-25 08:31:43 +02:00
Hendrik Muhs
e974f178b5 [Transform] rename data frame transform to transform for hlrc client (#46933)
rename data frame transform to transform for hlrc
2019-09-25 08:31:43 +02:00
Hendrik Muhs
abe889af75
[7.5][Transform] rename classes in transform plugin (#46867)
rename classes and settings in transform plugin, provide BWC for old settings
2019-09-20 10:43:00 +02:00
Benjamin Trent
9cf9c64ec2
[7.x] [ML][Transforms] remove force flag from _start (#46414) (#46748)
* [ML][Transforms] remove `force` flag from _start (#46414)

* [ML][Transforms] remove `force` flag from _start

* fixing expected error message

* adjusting bwc version
2019-09-18 10:06:05 -04:00
Hendrik Muhs
c8f52ec4ff
[Transform] Rename data frame plugin to transform: classes in xpack.core (#46644) (#46734)
rename classes in xpack.core of transform plugin from "data frame transform" to "transform"
2019-09-16 13:39:22 +02:00
Hendrik Muhs
efea581dcc
[7.x][Transform]Rename data frame plugin to transform: plugin and package names (#46583)
rename data frame transform plugin to transform:

 - rename plugin data-frame to transform
 - change all package names from o.e.*.dataframe.* to o.e.*.transform.*
 - necessary changes to fix loading/testing
2019-09-11 14:50:08 +02:00