Benjamin Trent 5ab9e75e28

[7.x] [ML][Inference] lang_ident model (#50292) (#50675) ...

* [ML][Inference] lang_ident model (#50292)

This PR contains a java port of Google's CLD3 compact NN model https://github.com/google/cld3

The ported model is formatted to fit within our inference model formatting and stored as a resource in the `:xpack:ml:` plugin and is under basic license.

The model is broken up into two major parts:
- Preprocessing through the custom embedding (based on CLD3's embedding layer)
- Pushing the embedded text through the two layers of fully connected shallow NN. 

Main differences between this port and CLD3:
- We take advantage of Java's internal Unicode handling where possible (i.e. codepoints, characters, decoders, etc.)
- We do not trim down input text by removing duplicated tokens
- We do not encode doubles/floats as longs/integers.

2020-01-06 16:24:03 -05:00

..

licenses

[ML] Rename log structure to file structure (#33421)

2018-09-06 09:13:08 +01:00

qa

[7.x] Synchronize processInStream.close() call (#50581)

2020-01-03 10:23:51 +01:00

src

[7.x] [ML][Inference] lang_ident model (#50292) (#50675)

2020-01-06 16:24:03 -05:00

build.gradle

Apply 2-space indent to all gradle scripts (#49071)

2019-11-14 11:01:23 +00:00