mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-03-28 02:48:38 +00:00
34ee4c2d66
The shaded version of elasticsearch was built at the very beginning to avoid dependency conflicts in a specific case where:
* People use elasticsearch from Java
* People needs to embed elasticsearch jar within their own application (as it's today the only way to get a `TransportClient`)
* People also embed in their application another (most of the time older) version of dependency we are using for elasticsearch, such as: Guava, Joda, Jackson...
This conflict issue can be solved within the projects themselves by either upgrade the dependency version and use the one provided by elasticsearch or by shading elasticsearch project and relocating some conflicting packages.
Example
-------
As an example, let's say you want to use within your project `Joda 2.1` but elasticsearch `2.0.0-beta1` provides `Joda 2.8`.
Let's say you also want to run all that with shield plugin.
Create a new maven project or module with:
```xml
<groupId>fr.pilato.elasticsearch.test</groupId>
<artifactId>es-shaded</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<elasticsearch.version>2.0.0-beta1</elasticsearch.version>
</properties>
<dependencies>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>${elasticsearch.version}</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.plugin</groupId>
<artifactId>shield</artifactId>
<version>${elasticsearch.version}</version>
</dependency>
</dependencies>
```
And now shade and relocate all packages which conflicts with your own application:
```xml
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>org.joda</pattern>
<shadedPattern>fr.pilato.thirdparty.joda</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
```
You can create now a shaded version of elasticsearch + shield by running `mvn clean install`.
In your project, you can now depend on:
```xml
<dependency>
<groupId>fr.pilato.elasticsearch.test</groupId>
<artifactId>es-shaded</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.1</version>
</dependency>
```
Build then your TransportClient as usual:
```java
TransportClient client = TransportClient.builder()
.settings(Settings.builder()
.put("path.home", ".")
.put("shield.user", "username:password")
.put("plugin.types", "org.elasticsearch.shield.ShieldPlugin")
)
.build();
client.addTransportAddress(new InetSocketTransportAddress(new InetSocketAddress("localhost", 9300)));
// Index some data
client.prepareIndex("test", "doc", "1").setSource("foo", "bar").setRefresh(true).get();
SearchResponse searchResponse = client.prepareSearch("test").get();
```
If you want to use your own version of Joda, then import for example `org.joda.time.DateTime`. If you want to access to the shaded version (not recommended though), import `fr.pilato.thirdparty.joda.time.DateTime`.
You can run a simple test to make sure that both classes can live together within the same JVM:
```java
CodeSource codeSource = new org.joda.time.DateTime().getClass().getProtectionDomain().getCodeSource();
System.out.println("unshaded = " + codeSource);
codeSource = new fr.pilato.thirdparty.joda.time.DateTime().getClass().getProtectionDomain().getCodeSource();
System.out.println("shaded = " + codeSource);
```
It will print:
```
unshaded = (file:/path/to/joda-time-2.1.jar <no signer certificates>)
shaded = (file:/path/to/es-shaded-1.0-SNAPSHOT.jar <no signer certificates>)
```
This PR also removes fully-loaded module.
By the way, the project can now build with Maven 3.3.3 so we can relax a bit our maven policy.
178 lines
5.5 KiB
Plaintext
178 lines
5.5 KiB
Plaintext
[[java-docs-index]]
|
|
=== Index API
|
|
|
|
The index API allows one to index a typed JSON document into a specific
|
|
index and make it searchable.
|
|
|
|
|
|
[[java-docs-index-generate]]
|
|
==== Generate JSON document
|
|
|
|
There are several different ways of generating a JSON document:
|
|
|
|
* Manually (aka do it yourself) using native `byte[]` or as a `String`
|
|
|
|
* Using a `Map` that will be automatically converted to its JSON
|
|
equivalent
|
|
|
|
* Using a third party library to serialize your beans such as
|
|
http://wiki.fasterxml.com/JacksonHome[Jackson]
|
|
|
|
* Using built-in helpers XContentFactory.jsonBuilder()
|
|
|
|
Internally, each type is converted to `byte[]` (so a String is converted
|
|
to a `byte[]`). Therefore, if the object is in this form already, then
|
|
use it. The `jsonBuilder` is highly optimized JSON generator that
|
|
directly constructs a `byte[]`.
|
|
|
|
|
|
[[java-docs-index-generate-diy]]
|
|
===== Do It Yourself
|
|
|
|
Nothing really difficult here but note that you will have to encode
|
|
dates according to the
|
|
{ref}/mapping-date-format.html[Date Format].
|
|
|
|
[source,java]
|
|
--------------------------------------------------
|
|
String json = "{" +
|
|
"\"user\":\"kimchy\"," +
|
|
"\"postDate\":\"2013-01-30\"," +
|
|
"\"message\":\"trying out Elasticsearch\"" +
|
|
"}";
|
|
--------------------------------------------------
|
|
|
|
|
|
[[java-docs-index-generate-using-map]]
|
|
===== Using Map
|
|
|
|
Map is a key:values pair collection. It represents a JSON structure:
|
|
|
|
[source,java]
|
|
--------------------------------------------------
|
|
Map<String, Object> json = new HashMap<String, Object>();
|
|
json.put("user","kimchy");
|
|
json.put("postDate",new Date());
|
|
json.put("message","trying out Elasticsearch");
|
|
--------------------------------------------------
|
|
|
|
|
|
[[java-docs-index-generate-beans]]
|
|
===== Serialize your beans
|
|
|
|
Elasticsearch already uses http://wiki.fasterxml.com/JacksonHome[Jackson].
|
|
So you can use it to serialize your beans to JSON:
|
|
|
|
[source,java]
|
|
--------------------------------------------------
|
|
import com.fasterxml.jackson.databind.*;
|
|
|
|
// instance a json mapper
|
|
ObjectMapper mapper = new ObjectMapper(); // create once, reuse
|
|
|
|
// generate json
|
|
byte[] json = mapper.writeValueAsBytes(yourbeaninstance);
|
|
--------------------------------------------------
|
|
|
|
|
|
[[java-docs-index-generate-helpers]]
|
|
===== Use Elasticsearch helpers
|
|
|
|
Elasticsearch provides built-in helpers to generate JSON content.
|
|
|
|
[source,java]
|
|
--------------------------------------------------
|
|
import static org.elasticsearch.common.xcontent.XContentFactory.*;
|
|
|
|
XContentBuilder builder = jsonBuilder()
|
|
.startObject()
|
|
.field("user", "kimchy")
|
|
.field("postDate", new Date())
|
|
.field("message", "trying out Elasticsearch")
|
|
.endObject()
|
|
--------------------------------------------------
|
|
|
|
Note that you can also add arrays with `startArray(String)` and
|
|
`endArray()` methods. By the way, the `field` method +
|
|
accepts many object types. You can directly pass numbers, dates and even
|
|
other XContentBuilder objects.
|
|
|
|
If you need to see the generated JSON content, you can use the
|
|
`string()` method.
|
|
|
|
[source,java]
|
|
--------------------------------------------------
|
|
String json = builder.string();
|
|
--------------------------------------------------
|
|
|
|
|
|
[[java-docs-index-doc]]
|
|
==== Index document
|
|
|
|
The following example indexes a JSON document into an index called
|
|
twitter, under a type called tweet, with id valued 1:
|
|
|
|
[source,java]
|
|
--------------------------------------------------
|
|
import static org.elasticsearch.common.xcontent.XContentFactory.*;
|
|
|
|
IndexResponse response = client.prepareIndex("twitter", "tweet", "1")
|
|
.setSource(jsonBuilder()
|
|
.startObject()
|
|
.field("user", "kimchy")
|
|
.field("postDate", new Date())
|
|
.field("message", "trying out Elasticsearch")
|
|
.endObject()
|
|
)
|
|
.get();
|
|
--------------------------------------------------
|
|
|
|
Note that you can also index your documents as JSON String and that you
|
|
don't have to give an ID:
|
|
|
|
[source,java]
|
|
--------------------------------------------------
|
|
String json = "{" +
|
|
"\"user\":\"kimchy\"," +
|
|
"\"postDate\":\"2013-01-30\"," +
|
|
"\"message\":\"trying out Elasticsearch\"" +
|
|
"}";
|
|
|
|
IndexResponse response = client.prepareIndex("twitter", "tweet")
|
|
.setSource(json)
|
|
.get();
|
|
--------------------------------------------------
|
|
|
|
`IndexResponse` object will give you a report:
|
|
|
|
[source,java]
|
|
--------------------------------------------------
|
|
// Index name
|
|
String _index = response.getIndex();
|
|
// Type name
|
|
String _type = response.getType();
|
|
// Document ID (generated or not)
|
|
String _id = response.getId();
|
|
// Version (if it's the first time you index this document, you will get: 1)
|
|
long _version = response.getVersion();
|
|
// isCreated() is true if the document is a new one, false if it has been updated
|
|
boolean created = response.isCreated();
|
|
--------------------------------------------------
|
|
|
|
For more information on the index operation, check out the REST
|
|
{ref}/docs-index_.html[index] docs.
|
|
|
|
|
|
[[java-docs-index-thread]]
|
|
==== Operation Threading
|
|
|
|
The index API allows one to set the threading model the operation will be
|
|
performed when the actual execution of the API is performed on the same
|
|
node (the API is executed on a shard that is allocated on the same
|
|
server).
|
|
|
|
The options are to execute the operation on a different thread, or to
|
|
execute it on the calling thread (note that the API is still asynchronous). By
|
|
default, `operationThreaded` is set to `true` which means the operation
|
|
is executed on a different thread.
|