OpenSearch/x-pack/plugin/core
David Roberts b61202b0a8 [ML] Add a limit on line merging in find_file_structure (#42501)
When analysing a semi-structured text file the
find_file_structure endpoint merges lines to form
multi-line messages using the assumption that the
first line in each message contains the timestamp.
However, if the timestamp is misdetected then this
can lead to excessive numbers of lines being merged
to form massive messages.

This commit adds a line_merge_size_limit setting
(default 10000 characters) that halts the analysis
if a message bigger than this is created.  This
prevents significant CPU time being spent subsequently
trying to determine the internal structure of the
huge bogus messages.
2019-06-03 13:45:51 +01:00
..
forbidden Ensure SHA256 is not used in tests (#42289) 2019-05-22 09:54:24 +03:00
licenses Remove Log4j 1.2 API as a dependency (#42702) 2019-05-30 16:08:07 -04:00
src [ML] Add a limit on line merging in find_file_structure (#42501) 2019-06-03 13:45:51 +01:00
build.gradle Remove Log4j 1.2 API as a dependency (#42702) 2019-05-30 16:08:07 -04:00
snapshot.key Make licensing FIPS-140 compliant (#30251) 2018-05-02 22:19:43 +03:00