Commit Graph

17907 Commits

Author SHA1 Message Date
Martijn van Groningen 9dd52ad7d3 Removed pollution from the Processor.Factory interface.
1) It no longer extends from Closeable.
2) Removed the config directory setter. Implementation that relied on it, now get the location to the config dir via their constructors.
2015-12-01 17:32:37 +01:00
Martijn van Groningen fa9fcb3b11 geo processor should add a list of doubles instead of an array to the ingest document 2015-12-01 17:12:34 +01:00
Martijn van Groningen 99a4295330 If a list or map value gets set on ingest document a deep copy needs to be made.
If this is not done this can lead to processor configuration being changed by an bulk or index request.
2015-12-01 16:02:04 +01:00
Martijn van Groningen 4402da1af0 also change the tests to deal with Exception instead of IOException 2015-11-30 15:45:40 +01:00
Martijn van Groningen dde274d944 Replaced IOException with Exception on factory implementations' `Processor.Factory#create(Map)` method. 2015-11-30 15:37:16 +01:00
Martijn van Groningen fdf4543b8e Renamed `add` processor to `set` processor.
This name makes more sense, because if a field already exists it overwrites it.
2015-11-30 15:03:20 +01:00
javanna 43b861b076 IngestDocument to support accessing and modifying list items
When reading, through #getFieldValue and #hasField, and a list is encountered, the next element in the path is treated as the index of the item that the path points to (e.g. `list.0.key`). If the index is not a number or out of bounds, an exception gets thrown.

Added #appendFieldValue method that has the same behaviour as setFieldValue, but when a list is the last element in the path, instead of replacing the whole list it will simply add a new element to the existing list. This method is currently unused, we have to decide whether the set processor or a new processor should use it.

A few other changes made:
- Renamed hasFieldValue to hasField, as this method is not really about values but only keys. It will return true if a key is there but its value is null, while it returns false only when a field is not there at all.
- Changed null semantic in getFieldValue. null gets returned only when it was an actual value in the source, an exception is thrown otherwise when trying to access a non existing field, so that null != field not present.
- Made remove stricter about non existing fields. Throws error when trying to remove a non existing field. This is more consistent with the other methods in IngestDocument which are strict about fields that are not present.

Relates to #14324
2015-11-30 13:58:03 +01:00
Martijn van Groningen 467a47670c Merge remote-tracking branch 'es/master' into feature/ingest 2015-11-30 10:24:27 +01:00
Clinton Gormley 72be42d742 Document that _index is a virtual field and only supports term queries
Closes #15070
Closes #15081
2015-11-30 08:43:23 +01:00
Clinton Gormley 1685126bb6 Merge pull request #15085 from kaneshin/docs/modify/post_filter
Remove a trailing comma from an example data of JSON
2015-11-30 08:05:10 +01:00
Clinton Gormley 27dac8dc2c REST spec: Added the verbose flag to indices.segments
Relates to #9111
2015-11-30 07:41:29 +01:00
Ryan Ernst ddf1252a07 Merge pull request #15098 from rjernst/integ_test_output
Improve output when integ test fails
2015-11-29 19:55:23 -08:00
Ryan Ernst d37930d133 Remove leftover debugging code 2015-11-29 19:51:50 -08:00
Ryan Ernst 0fa05edf03 Build: Improve output when integ test fails
This outputs a lot more info when integ tests fail to start, as well as
(should) fix windows (at least in my VM run) integ tests.
2015-11-29 19:44:43 -08:00
Robert Muir fe20677819 Merge pull request #15097 from rmuir/buglevel
Add workaround for JDK-6427854
2015-11-29 17:25:11 -05:00
Robert Muir b7523bb974 Add workaround for JDK-6427854
See e.g. http://build-us-00.elastic.co/job/es_feature_ingest/2831/consoleFull

The bug can still happen, so we should let netty do this workaround
2015-11-29 16:12:52 -05:00
Robert Muir 93758874fb Merge pull request #15093 from rmuir/gradle_windoze_integ_tests
Fix integration tests and 'gradle run' to work on windoze
2015-11-29 12:15:30 -05:00
Robert Muir 8464b92aa3 Merge pull request #15094 from rmuir/windoze_charsets
do not assert charset or exact lengths for mapper-attachments tests.
2015-11-29 12:12:18 -05:00
Robert Muir a2816ec574 don't assert exact expected lengths for documents.
This will change depending on newline of the operating system
2015-11-29 11:48:54 -05:00
Robert Muir 415c37340a do not assert charset for mapper-attachments tests.
Its enough to test the content type for what we are testing.

Currently tests are flaky if charset is detected as e.g. windows-1252 vs iso-8859-1 and so on.
In fact, they fail on windows 100% of the time.

We are not trying to test charset detection heuristics (which might be different even due to newlines in tests or other things).
If we want to do test that, we should test it separately.
2015-11-29 11:40:27 -05:00
Robert Muir a01f86aaac Fix integration tests and gradle run to work on windoze 2015-11-29 07:39:18 -08:00
Boaz Leskes f8a027e591 Merge pull request #14801 from bleskes/resiliency_update
Update the resiliency page to 2.0.0
2015-11-29 12:21:15 +01:00
Michael McCandless fc6a2f318a Merge pull request #15090 from mikemccand/improve_completion_stats
Pull Fields instance once from LeafReader in completion stats.

Closes #6593
2015-11-29 06:17:53 -05:00
Michael McCandless 94954ece3a improve javadocs 2015-11-29 06:13:47 -05:00
Michael McCandless 8933947b95 only pull Fields once from the reader 2015-11-29 06:00:11 -05:00
Ryan Ernst 8e25dc4f80 Merge pull request #15089 from rjernst/extra_config_file_delay
Fix delayed extra config file checks to be right before copy
2015-11-28 19:44:16 -08:00
Ryan Ernst 018c766bf7 Build: Fix delayed extra config file checks to be right before copy
The current delay waits until later after normal configuration, but
still just after resolution (ie when paths would be known for
dependencies but not actual execution). This delays the checks further
to be done right before we actually execute the copy task.

closes #15068
2015-11-28 18:06:36 -08:00
Ryan Ernst 1ee59e4977 Merge pull request #15088 from rjernst/integ_test_command_line
Move debug options setting to before command line is logged
2015-11-28 17:19:39 -08:00
Nik Everett 114065b3fa Merge pull request #15031 from nik9000/integ_for_packages
Fix deb and rpm tests
2015-11-28 20:10:41 -05:00
Ryan Ernst f15ee9fa54 Build: Move debug options setting to before command line is logged
Currently if running with --info, the command line for ES, along with
env vars, are logged before they may be ammended to add debug options.
This moves the adding JAVA_OPTS to before we print the command.
2015-11-28 17:05:11 -08:00
Nik Everett eba594fc08 Remove strange checks and clean dirs 2015-11-28 19:44:53 -05:00
Nik Everett 45e71329ee [build] Fix deb and rpm tests
Gradle is causing so much trouble here! Its too cute for its own good.
2015-11-28 19:07:06 -05:00
Ryan Ernst e4ad7b77c4 Merge pull request #15086 from rjernst/packaging_artifact_ids
Fix rpm and deb generated poms to have correct artifactId
2015-11-28 14:06:24 -08:00
Ryan Ernst 7788c4cd69 Build: Fix rpm and deb generated poms to have correct artifactId
For all our distributions, we use "elasticsearch" as the artifactId.
This fixes the id for the generated poms of rpm and deb distributions.
2015-11-28 11:30:49 -08:00
Johnny Lim a66be6cfb9 Polish doc
Closes #15011
2015-11-28 18:25:42 +01:00
Shintaro Kaneko d7baeb1e7b Remove a trailing comma from an example data of JSON 2015-11-28 16:50:28 +00:00
Clinton Gormley 174b4bacbe Merge pull request #14871 from jamiemccarthy/doc-fix
Fix doc of nested_path sort option
2015-11-28 17:46:47 +01:00
Jason Spafford c7112bd866 Changed some of the language in documentation
Closes #14920
2015-11-28 16:00:29 +01:00
Clinton Gormley 2ab14cb21c Merge pull request #14900 from shikhar/patch-1
link to es-restlog plugin
2015-11-28 15:09:22 +01:00
Clinton Gormley a02e997ad5 Merge pull request #15051 from apatrida/patch-1
body attribute was at wrong nesting level
2015-11-28 11:31:42 +01:00
javanna b4b698f653 Merge branch 'master' into feature/ingest 2015-11-28 11:15:25 +01:00
Robert Muir 29853744a3 Merge pull request #15055 from rmuir/fun_with_windows
set ActiveProcessLimit=1 on windows
2015-11-27 16:57:34 -05:00
Christoph Büscher b375084305 Tests: Correction in AbstractQueryTestCase
Removed check that two query builder that are different according
to equals() have different hashCode since that is not required
by the contract of hashCode.
2015-11-27 17:36:57 +01:00
Daniel Mitterdorfer 2f87e0a4d9 Declare NetworkService non-final to support mocking in tests 2015-11-27 15:51:19 +01:00
Clinton Gormley 73b3922531 Merge pull request #15033 from yokotaso/patch-2
Wrong properties is used in document.
2015-11-27 14:13:36 +01:00
Simon Willnauer 65b661b1f4 [TEST] Fix MapperUpgrade tests to use a dedicated master to ensure dangeling index import works predictably
When importing dangling indices on a single node that is data and master eligable the async dangling index
call can still be in-flight when the cluster is checked for green / yellow. Adding a dedicated master node
and a data only node that does the importing fixes this issus just like we do in OldIndexBackwardsCompatibilityIT
2015-11-27 10:32:21 +01:00
Robert Muir ee827198ff set ActiveProcessLimit=1 on windows
This blocks process creation, similar to the approach for BSD.
2015-11-27 01:50:09 -05:00
Robert Muir aa56bfa562 Merge pull request #15039 from rmuir/setrlimit
set RLIMIT_NPROC = 0 on bsd/os X systems.
2015-11-26 21:18:57 -05:00
Robert Muir 7d1903b01e Merge pull request #15038 from rmuir/java_ram
Give enough ram so javac won't OOM
2015-11-26 21:18:16 -05:00
Jason Tedor 5c5e02e0bf Remove outdated comment and cleanup RestVerifyRepositoryAction 2015-11-26 19:11:11 -05:00