1285 Commits

Author SHA1 Message Date
Robert Muir
f81b12e327 minimize accessiblity, remove unused threadpool 2015-12-21 12:39:40 -05:00
Adrien Grand
cf52e96c42 Upgrade to lucene-5.5.0-snapshot-1721183.
Some files that implement or use the Scorer API had to be changed because of
https://issues.apache.org/jira/browse/LUCENE-6919.
2015-12-21 17:02:08 +01:00
Adrien Grand
ac393b7a31 Make mappings tests more realistic.
DocumentMapperParser has both parse and parseCompressed methods. Except that the
parse methods are ONLY used from the unit tests. This commit removes the parse
method and moves all tests to parseCompressed so that they test more
realistically how mappings are managed.

Then I renamed parseCompressed to parse given that this is the only alternative
anyway.
2015-12-21 10:44:00 +01:00
Robert Muir
f67390e0c8 in the plugin: guard against HADOOP_HOME in environment on any platform.
hdfs fixture: minihdfs works on windows now, if things are properly set
but our test fixture still cannot launch this on windows.
2015-12-21 02:21:53 -05:00
Robert Muir
53530f1243 remove hacks, test fixtures are clean before each execution 2015-12-20 22:23:30 -05:00
Robert Muir
935c2c75f6 Remove slf4j hack 2015-12-20 22:08:18 -05:00
Robert Muir
04966bcc3e contain and improve hack 2015-12-20 21:02:03 -05:00
Robert Muir
03a2b6b01b Disable HDFS fixture on windows, it requires native libraries. 2015-12-20 16:30:19 -08:00
Robert Muir
a37417085d blind stab at unit test issues on windows 2015-12-20 18:31:55 -05:00
Robert Muir
ee546ff655 try to get windows working 2015-12-20 17:10:01 -05:00
Robert Muir
2347e3c373 Get forbidden apis passing again, this needs to be investigated 2015-12-20 16:17:17 -05:00
Robert Muir
7ac49bb278 Merge branch 'hdfs2-only' of github.com:costin/elasticsearch into hdfs2-only 2015-12-20 16:12:23 -05:00
Robert Muir
12a8428dfb Add MiniHDFS test fixture, started before integTest and shut down after.
Currently uses a hardcoded port (9999), need to apply MavenFilteringHack after it starts.
2015-12-20 16:00:37 -05:00
Martijn van Groningen
66d8b5342f s/PipelineStoreBootstrapper/IngestBootstrapper 2015-12-20 20:18:46 +01:00
Costin Leau
3204e87220 Restrict usage to HDFS only 2015-12-20 15:53:18 +02:00
Robert Muir
ae89c6e51c Merge branch 'master' into hdfs2-only 2015-12-19 21:52:52 -05:00
Ryan Ernst
9cb4c82c58 Build: Add fixture capabilities to integ tests
This change adds a Fixture class for use by gradle. A Fixture is an
external process that integration tests will use. It can be added as a
dependsOn for integTest, and will automatically be shutdown upon success
or failure, as well as relevant information dumped on failure. There is
also an example fixture in this change.
2015-12-19 15:46:21 -08:00
Robert Muir
8c6f5a0c60 add failing test 2015-12-19 15:05:38 -08:00
Robert Muir
5dcccca848 add example fixture 2015-12-19 15:05:37 -08:00
Robert Muir
d171773bdb remove leniency in tests 2015-12-19 04:39:01 -05:00
Robert Muir
e2b2ee24fa Add licensing for dependencies 2015-12-19 03:06:40 -05:00
Robert Muir
9df447295c Fix unit tests (also works from IDE). 2015-12-19 02:43:27 -05:00
Robert Muir
3269beeb4d don't throw exceptions from ctor, guice is hell 2015-12-19 02:09:14 -05:00
Robert Muir
f174e96a14 explicitly initialize some hadoop classes elevated, so we don't rely on classloading order.
maybe this allows us to do less stuff in doPriv later, we will see. at least it makes things
like unit testing easier.
2015-12-19 00:21:01 -05:00
Robert Muir
2e8c68d09b Remove no-longer needed domaincombiner stuff 2015-12-18 23:51:41 -05:00
Robert Muir
02fbd55118 enable thirdPartyAudit so you can see the crazy shit hadoop does 2015-12-18 23:45:05 -05:00
Robert Muir
bc11962438 get full snapshot restore tests passing 2015-12-18 23:16:41 -05:00
Robert Muir
fbe3d64ea4 add passing test that takes snapshot 2015-12-18 22:55:15 -05:00
Robert Muir
75ef9da53f get up to connectexception 2015-12-18 22:11:58 -05:00
Ryan Ernst
c2c5081830 Remove uneeded class loading stuff from hdfs plugin 2015-12-18 17:01:38 -08:00
Ryan Ernst
91fe99a7f6 Make hdfs plugin not use transitive deps 2015-12-18 16:52:22 -08:00
Costin Leau
7584810ff4 * Make plugin hadoop2-only
Polish MiniDFS cluster to be Hadoop2 (instead of Hadoop1) based
2015-12-19 01:35:53 +02:00
Ryan Ernst
690fb2cd3f Rename InternalFilters.Bucket to InternalFilters.InternalBucket to avoid name collision 2015-12-18 13:22:20 -08:00
Ryan Ernst
4ea19995cf Remove wildcard imports 2015-12-18 12:43:47 -08:00
Robert Muir
447729f0e1 add missing license headers 2015-12-18 13:08:17 -05:00
Robert Muir
2e2e328879 add missing license header 2015-12-18 13:02:39 -05:00
Martijn van Groningen
6dfcee6937 Added an internal reload pipeline api that makes sure pipeline changes are visible on all ingest nodes after modifcations have been made.
* Pipeline store can now only start when there is no .ingest index or all primary shards of .ingest have been started
* IngestPlugin adds`node.ingest` setting to `true`. This is used to figure out to what nodes to send the refresh request too. This setting isn't yet configurable. This will be done in a follow up issue.
* Removed the background pipeline updater and added added logic to deal with specific scenarious to reload all pipelines.
* Ingest services are no longer be managed by Guice. Only the bootstrapper gets managed by guice and that contructs
all the services/components ingest will need.
2015-12-18 18:24:27 +01:00
Martijn van Groningen
e8a8e22e09 Add template infrastructure, removed meta processor and added template support to set and remove processor.
Added ingest wide template infrastructure to IngestDocument
Added a TemplateService interface that the ingest framework uses
Added a TemplateService implementation that the ingest plugin provides that delegates to the ES' script service
Cut SetProcessor over to use the template infrastructure for the `field` and `value` settings.
Removed the MetaDataProcessor
Removed dependency on mustache library
Added qa  ingest mustache rest test so that the ingest and mustache integration can be tested.
2015-12-18 17:35:53 +01:00
Martijn van Groningen
a56902567e don't register rest actions on transport clients 2015-12-18 15:49:00 +01:00
Martijn van Groningen
2c5bb84851 fix copyDefaultGeoIp2DatabaseFiles task to work again 2015-12-18 15:45:56 +01:00
javanna
3e155f7b54 adapt to upstream changes: thirdPartyAudit.missingClasses set to true
geoip depends on asm and google http client which we don't need
2015-12-18 11:14:39 +01:00
javanna
f349669071 adapt to upstream changes: enableMockModules => getMockPlugins 2015-12-18 11:13:34 +01:00
javanna
8bae93eee1 adapt to upstream changes: StringText => Text 2015-12-18 11:13:18 +01:00
javanna
885b01fb49 adapt to upstream changes: RestModule -> NetworkModule 2015-12-18 10:36:29 +01:00
javanna
5f2df6b95a Merge branch 'master' into feature/ingest 2015-12-18 10:34:07 +01:00
Simon Willnauer
eca2435838 Merge branch 'master' into settings_prototype 2015-12-18 09:15:58 +01:00
Adrien Grand
6ea16671f4 Simplify the Text API.
We have the Text API, which is essentially a wrapper around a String and a
BytesReference and then we have 3 implementations depending on whether the
String view should be cached, the BytesReference view should be cached, or both
should be cached.

This commit merges everything into a single Text that is essentially the old
StringAndBytesText impl.

Long term we should look into whether this API has any performance benefit or
if we could just use plain strings. This would greatly simplify all our other
APIs that currently use Text.
2015-12-17 17:22:38 +01:00
Simon Willnauer
eae3da3b54 Merge branch 'master' into settings_prototype 2015-12-17 15:13:41 +01:00
Adrien Grand
6ccc759691 Merge pull request #15480 from jpountz/fix/mapping_explicit_defaults
Make mapping serialization more robust.
2015-12-17 11:23:27 +01:00
Robert Muir
a7cc91e868 Merge pull request #15501 from rmuir/sheisty_classes
thirdPartyAudit round 2
2015-12-17 03:44:27 -05:00