honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[ML] fixing file structure finder multiline merge max for delimited formats (#56023) (#56035) 

This commit correctly sets the maxLinesPerRow in the CsvPreference for delimited files given the file structure finder settings.

Previously, it was silently ignored.
This commit is contained in:
Benjamin Trent 2020-04-30 10:51:32 -04:00 committed by GitHub
parent 68985bc1ca
commit c36bcb4dd0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/filestructurefinder/DelimitedFileStructureFinderFactory.java
View File

@ -71,7 +71,8 @@ public class DelimitedFileStructureFinderFactory implements FileStructureFinderF
public FileStructureFinder createFromSample(List<String> explanation, String sample, String charsetName, Boolean hasByteOrderMarker, public FileStructureFinder createFromSample(List<String> explanation, String sample, String charsetName, Boolean hasByteOrderMarker,
int lineMergeSizeLimit, FileStructureOverrides overrides, TimeoutChecker timeoutChecker) int lineMergeSizeLimit, FileStructureOverrides overrides, TimeoutChecker timeoutChecker)
throws IOException { throws IOException {
CsvPreference adjustedCsvPreference = new CsvPreference.Builder(csvPreference).maxLinesPerRow(lineMergeSizeLimit).build();
return DelimitedFileStructureFinder.makeDelimitedFileStructureFinder(explanation, sample, charsetName, hasByteOrderMarker, return DelimitedFileStructureFinder.makeDelimitedFileStructureFinder(explanation, sample, charsetName, hasByteOrderMarker,
csvPreference, trimFields, overrides, timeoutChecker); adjustedCsvPreference, trimFields, overrides, timeoutChecker);
} }
} }