Christoph Büscher 4ffa050735 Allow custom characters in token_chars of ngram tokenizers (#49250) ... Currently the `token_chars` setting in both `edgeNGram` and `ngram` tokenizers only allows for a list of predefined character classes, which might not fit every use case. For example, including underscore "_" in a token would currently require the `punctuation` class which comes with a lot of other characters. This change adds an additional "custom" option to the `token_chars` setting, which requires an additional `custom_token_chars` setting to be present and which will be interpreted as a set of characters to inlcude into a token. Closes #25894		2019-11-20 10:37:12 +01:00
..
src	Allow custom characters in token_chars of ngram tokenizers (#49250)	2019-11-20 10:37:12 +01:00
build.gradle	Apply 2-space indent to all gradle scripts (#49071)	2019-11-14 11:01:23 +00:00