Alan Guo Xiang Tan 930f51e175 FEATURE: Split up text segmentation for Chinese and Japanese.
* Chinese segmenetation will continue to rely on cppjieba
* Japanese segmentation will use our port of TinySegmenter
* Korean currently does not rely on segmentation which was dropped in c677877e4fe5381f613279901f36ae255c909573
* SiteSetting.search_tokenize_chinese_japanese_korean has been split
into SiteSetting.search_tokenize_chinese and
SiteSetting.search_tokenize_japanese respectively
2022-02-07 09:21:14 +08:00
..
2020-01-15 11:27:12 +01:00
2021-10-27 11:39:28 +03:00
2021-05-31 10:22:50 +08:00
2020-05-05 13:46:57 +10:00
2021-06-21 11:06:58 +08:00
2020-07-21 15:55:03 +08:00
2020-04-30 16:48:53 +10:00