Commit Graph

17565 Commits

Author SHA1 Message Date
Camilo Díaz Repka d795ce3b6e XContentFactory.xContentType: allow for possible UTF-8 BOM for JSON XContentType
Fixes #14442
2015-11-09 11:19:54 +01:00
Adrien Grand 617c0c6a6d Build: Fix dev-tools/update_lucene.sh to use the new gradle build. 2015-11-09 11:01:20 +01:00
Robert Muir 336ac33eed Merge pull request #14609 from rmuir/32bit
fixup issues with 32-bit jvm
2015-11-08 21:23:37 -05:00
Robert Muir 0de503b34b fixup issues with 32-bit jvm
If you run tests under a 32-bit jvm, you will get a test failure in IndexStoreTests,
the logic there is wrong in the case of 32-bit (its NIOFSDirectory on linux).

Also if mlockall fails, you'll see huge bogus values (because of use of `long` instead of `NativeLong`)

finally add seccomp support for 32 bit too, and clean up all its `long` usage as well.
2015-11-08 21:08:08 -05:00
Clinton Gormley 0b9b9d7983 Revert "Removed redundant docs/community-clients/index.asciidoc"
These docs weren't redundant after all.

This reverts commit 22d163692c.
2015-11-08 23:24:45 +01:00
Clinton Gormley 4684be3680 Merge pull request #14391 from jurgc11/patch-2
Add community plugin
2015-11-08 22:12:26 +01:00
Clinton Gormley f0fda74083 Moved a breaking change to the correct section. Closes #14510 2015-11-08 21:53:56 +01:00
Clinton Gormley 1849c645f3 Merge pull request #14509 from elastic/markw/typo-parentchild-releasenotes-14508
Fixed minor typo
2015-11-08 21:48:40 +01:00
Clinton Gormley 22d163692c Removed redundant docs/community-clients/index.asciidoc 2015-11-08 21:45:38 +01:00
Clinton Gormley f7a7b192e4 Merge pull request #14504 from sckott/add-elasticdsl
Added new community R client, edited old entry
2015-11-08 21:41:55 +01:00
Clinton Gormley df865a2628 Merge pull request #14431 from arinto/patch-1
[DOCS] Missing tribe nodes' short description
2015-11-08 18:36:31 +01:00
andrejserafim 2bd353d72d replacing run.bat and run.sh with gradle run
run.sh and run.bat were calling out to the old maven build system.
This is no longer in place, so we've created new gradle tasks to
start an elasticsearch node from the current codebase.

fixed #14423
2015-11-08 17:07:19 +00:00
Clinton Gormley 28015d9e03 Docs: Made the list of breaking changes in each version more obvious 2015-11-08 12:59:07 +01:00
Simon Willnauer 47769cc4b4 add more logging to test for better readability 2015-11-08 12:43:24 +01:00
Simon Willnauer fa6a2932f5 Instead of relying on the in-flight number use a boolean flag on the
allocation to ensure that we skip rebalance if we have not processed
all pending shard / store fetches.
2015-11-08 12:40:05 +01:00
Clinton Gormley 1220b89a60 Docs: Fixed bad link in completion suggester 2015-11-08 09:51:14 +01:00
Robert Muir e88896e40e apply a few post-merge cleanups 2015-11-08 02:31:26 -05:00
Robert Muir a164586796 Merge branch 'mapper_migrate' into migrate-mapper-attachments 2015-11-08 02:19:50 -05:00
Robert Muir 550f462c47 migrate mapper attachments 2015-11-08 02:09:36 -05:00
Robert Muir 9e99db01b0 move import under license header 2015-11-08 01:39:46 -05:00
Robert Muir c07a512376 Lockdown parsers with minimal privileges
There have been security issues with tika's parsers in the past...
let's take away the network, filesystem, everything we can.

In some way, parsing these docs is a lot like executing untrusted code.

I know its not pretty, but I think its worth it.
2015-11-08 01:37:15 -05:00
Robert Muir 39bc92bfe6 remove unused imports 2015-11-08 00:35:28 -05:00
Robert Muir ecc8912164 Merge pull request #187 from rmuir/tika_tests
Add test documents from tika test suite
2015-11-08 00:31:21 -05:00
Robert Muir 6b91e53ab5 Add test documents from tika test suite
This patch adds a zip of about 200 files from tika's test suite,
and we assert some content comes back from each. This is a good exercise
of the various formats.

I removed any huge files to try to keep size reasonable, but we want
a bit of a variety so we know stuff is working.

I fixed issues with the parser config by running this.
2015-11-08 00:22:36 -05:00
Robert Muir 997fac15f4 Merge pull request #186 from rmuir/reduce_surface_area
Reduce surface area of mapper attachments:
2015-11-07 22:37:41 -05:00
Robert Muir b793f389d0 remove unnecessary file 2015-11-07 22:09:47 -05:00
Robert Muir 1fc89feb63 Reduce surface area of mapper attachments:
this removes a lot of obscure parsers, and leaves us with the basics.

This includes at least all of the formats listed on
https://github.com/elastic/elasticsearch-mapper-attachments/issues/163

I will start adding tests for each one of these document formats,
and take it as it goes and see what trouble we run into.

Closes #163
2015-11-07 21:00:00 -05:00
Ryan Ernst bfc235f2ed Merge pull request #14604 from rjernst/enforce_plugin_name
Enforce name is set for all plugins
2015-11-07 17:19:20 -08:00
Ryan Ernst 8d78f970b0 Build: Enforce name is set for all plugins
The plugin name currently defaults to the gradle project name. But the
gradle project name for standalone repo (like an external plugin would
be) defaults to the directory name of the repo. This is trappy, since it
depends on how the repo was checked out.

This change enforces the plugin name is always set.

closes #14603
2015-11-07 16:57:11 -08:00
Areek Zillur 6444194dae fix conflicts from merge 2015-11-07 18:22:28 -05:00
Robert Muir f2ee1d79fa Merge pull request #185 from rmuir/fix_security_exceptions
Fix security problems in mapper-attachments.
2015-11-07 18:13:36 -05:00
Robert Muir 6b831e1e8b be explicit about name and link to bugzilla for the POI issue 2015-11-07 18:13:05 -05:00
Areek Zillur d5198b326c fix comments and throw ElasticsearchException when we fail to report fst index size 2015-11-07 17:50:56 -05:00
Areek Zillur d37e011526 add query context builders 2015-11-07 17:50:56 -05:00
Areek Zillur 40022e2168 remove nocommit 2015-11-07 17:50:55 -05:00
Areek Zillur e87b4e00eb add back awaitfix tests 2015-11-07 17:50:55 -05:00
Simon Willnauer 6f9a486071 more parsers 2015-11-07 17:50:55 -05:00
Robert Muir b0fcbb7081 remove unnecessary unboxing, we do that as impl detail in parseTika 2015-11-07 17:48:45 -05:00
Simon Willnauer f5560a3087 add parsers 2015-11-07 17:47:46 -05:00
Simon Willnauer 8911469e63 some basic cleanups 2015-11-07 17:46:28 -05:00
Areek Zillur 75d55e11ad cut over to using ObjectParser for context mappings 2015-11-07 17:46:27 -05:00
Areek Zillur 96c9849f5a added FuzzyOptionsBuilder javadocs and cleanups 2015-11-07 17:46:27 -05:00
Areek Zillur dd1c687ace Completion Suggester V2
The completion suggester provides auto-complete/search-as-you-type functionality.
This is a navigational feature to guide users to relevant results as they are typing, improving search precision.
It is not meant for spell correction or did-you-mean functionality like the term or phrase suggesters.

The completions are indexed as a weighted FST (finite state transducer) to provide fast Top N prefix-based
searches suitable for serving relevant results as a user types.

closes #10746
2015-11-07 17:46:27 -05:00
Robert Muir 3c40ed22b3 Fix security problems in mapper-attachments.
Many of the tests were not running, or did not check the exceptions.
I renamed all tests to meet *Tests* so they run, and assert exception messages.

Also because we must (currently) invoke tika with additional privileges, I added
the security logic, and fixed unit testing to call our static method directly.

This must be package private for security reasons, i simply put everything in
org.elasticsearch.mapper.attachments package.

I upgraded tika to the latest, so we are up to date, and removed logic around
tika == null and old locale issues.
2015-11-07 17:45:16 -05:00
Ryan Ernst cdab360992 Merge pull request #184 from rjernst/update_to_gradle
Build: Switch mapper attachments to gradle
2015-11-07 13:11:05 -08:00
Ryan Ernst d0f5950c80 Merge pull request #14598 from rjernst/integ_test_parity
Improve integ test to match ant behavior
2015-11-07 13:09:04 -08:00
Ryan Ernst 22bcf197b7 Add some comments to particularly nasty groovy code 2015-11-07 13:06:14 -08:00
Robert Muir 5164ec8c20 Merge pull request #14602 from rmuir/buildSrc
add groovy build code (buildSrc/) in 'gradle eclipse'
2015-11-07 15:41:05 -05:00
Robert Muir 866a514785 add groovy build code (buildSrc/) in 'gradle eclipse'
This makes it a groovy project that works in eclipse.
You will have to install a plugin for groovy language support
(I used a snapshot build from https://github.com/groovy/groovy-eclipse/wiki)
2015-11-07 15:14:00 -05:00
Robert Muir e06cae84f3 only allow code to bind to the user's configured port numbers/ranges
Random code shouldn't be listening on sockets elsewhere.

Today its the wild west, but we only need to grant access to what the user configured.

This means e.g. multicast plugin has to declare its intentions in its security.policy

Closes #14549
2015-11-07 12:22:12 -05:00