To simplify plugins maintenance and provide more value in the future, we are starting to build an `elasticsearch-parent` project.
This commit is the first step for this plugin to depend on this new `pom` maven project.
Calling a multi-statement score using curl e.g. `a=22; _score*a` is not possible with the current lang-python, nor is it possible to write a script using semicolons or multiple lines. I can only get compound statements (e.g. list comprehensions with embedded if statements) to work, so I'm limited in the complexity of my scoring process.
I'm using ES 1.3.2 and latest lang-python:
```
$ ls -la /usr/share/elasticsearch/plugins/lang-python/
-rw-r--r-- 1 root root 10482 Aug 27 17:20 elasticsearch-lang-python-2.3.0.jar
-rw-r--r-- 1 root root 14340135 Aug 27 17:20 jython-standalone-2.5.3.jar
```
Here's a worked example:
```
# Delete existing data, add 2 simple records, fetch both results to stdout
curl -XDELETE "http://localhost:9200/test"
curl -XPUT "http://localhost:9200/test/doc/1" -d '{
"num": 1.0
}'
curl -XPUT "http://localhost:9200/test/doc/2?refresh" -d '{
"num": 2.0
}'
# show our records
curl -XGET 'http://localhost:9200/test/_search' -d '{
"query" : { "match_all": {}}
}'
```
We'll run a simple query that uses `num` as a score modifier:
```doc["num"].value * _score```
If I create `/etc/elasticsearch/scripts/py1.py`:
```
doc["num"].value * _score
```
and wait for the script to reload (by monitoring the logs), I can call:
```
$ curl -XGET "http://localhost:9200/test/_search?pretty" -d'
{
"query": {
"function_score": {
"script_score": {
"script": "py1",
"lang": "python"
}
}
}
}'
```
and this will calculate the results.
The same can be achieved using an in-line call (ignoring `py1.py`):
```
curl -XGET "http://localhost:9200/test/_search?pretty" -d'
{
"query": {
"function_score": {
"script_score": {
"script": "doc[\"num\"].value * _score",
"lang": "python"
}
}
}
}'
```
However using more than 1 statement will fail. This example uses `;` to split 2 statements (this is legal in jython 2.5):
```
curl -XGET "http://localhost:9200/test/_search?pretty" -d'
{
"query": {
"function_score": {
"script_score": {
"script": "a=42; doc[\"num\"].value * _score",
"lang": "python"
}
}
}
}'
"reason" : "QueryPhaseExecutionException[[test][3]: query[function score (ConstantScore(*:*),function=script[a=42; doc[\"num\"].value * _score], params [null])],from[0],size[10]: Query Failed [Failed to execute main query]]; nested: NullPointerException; "
and the log message:
org.elasticsearch.search.query.QueryPhaseExecutionException: [test][3]: query[function score (ConstantScore(*:*),function=script[a=42; doc["num"].value * _score], params [null])],from[0],size[10]: Query Failed [Failed to execute main query]
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:162)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:261)
at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:206)
at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:203)
at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
```
Creating a `py2.py` in the scripts directory containing:
```
a=42; doc["num"].value * _score
```
and calling
```
$ curl -XGET "http://localhost:9200/test/_search?pretty" -d'
{
"query": {
"function_score": {
"script_score": {
"script": "py2",
"lang": "python"
}
}
}
}'
has the same error:
"reason" : "QueryPhaseExecutionException[[test][3]: query[function score (ConstantScore(*:*),function=script[py2], params [null])],from[0],size[10]: Query Failed [Failed to execute main query]]; nested: PyException; "
```
If a `py3.py` script is made with the same two statements split over two lines:
```
a=42
doc["num"].value * _score
```
then the same errors are thrown.
I'll note that if I experiment with equivalent groovy scripts then both the semicolon is allowed and multi-line scripts (in /scripts) are allowed.
Closes#19.
(cherry picked from commit 9fca562)
It sounds like Jython 2.5.3 is leaking some threads.
Jython 2.5.4.rc1 has the same issue.
Jython 2.7-b3 fixes it.
Typical error when running tests:
```
ERROR 0.00s J2 | PythonScriptEngineTests (suite) <<<
> Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.elasticsearch.script.python.PythonScriptEngineTests:
> 1) Thread[id=12, name=org.python.google.common.base.internal.Finalizer, state=WAITING, group=TGRP-PythonScriptEngineTests]
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at org.python.google.common.base.internal.Finalizer.run(Finalizer.java:127)
> at __randomizedtesting.SeedInfo.seed([7A5ECFD8D0474383]:0)
> Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
> 1) Thread[id=12, name=org.python.google.common.base.internal.Finalizer, state=WAITING, group=TGRP-PythonScriptEngineTests]
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at org.python.google.common.base.internal.Finalizer.run(Finalizer.java:127)
> at __randomizedtesting.SeedInfo.seed([7A5ECFD8D0474383]:0)
```
Closes#22.
Due to a change in elasticsearch 1.4.0, we need to apply a similar patch here.
See elasticsearch/elasticsearch#6864
See elasticsearch/elasticsearch#7819
Closes#16.
Closes#21.
We create branches:
* es-0.90 for elasticsearch 0.90
* es-1.0 for elasticsearch 1.0
* es-1.1 for elasticsearch 1.1
* master for elasticsearch master
We also check that before releasing we don't have a dependency to an elasticsearch SNAPSHOT version.
Add links to each version in documentation
Njal Karevoll
David Pilato
Adrien Grand
Michael McCandless
Robert Muir
tlrx
Britta Weber
Shay Banon