Commit Graph

13 Commits

Author SHA1 Message Date
Martijn van Groningen dbbb296322 added a `node.ingest` setting that controls whether ingest is active or not. Defaults to `false`.
If `node.ingest` isn't active then ingest related API calls fail and if the `pipeline_id` parameter is set then index and bulk requests fail.
2015-12-22 22:38:49 +01:00
Martijn van Groningen 3fcb8ecd54 Upgraded geoip2 and its dependencies 2015-12-22 11:39:43 +01:00
Martijn van Groningen e8a8e22e09 Add template infrastructure, removed meta processor and added template support to set and remove processor.
Added ingest wide template infrastructure to IngestDocument
Added a TemplateService interface that the ingest framework uses
Added a TemplateService implementation that the ingest plugin provides that delegates to the ES' script service
Cut SetProcessor over to use the template infrastructure for the `field` and `value` settings.
Removed the MetaDataProcessor
Removed dependency on mustache library
Added qa  ingest mustache rest test so that the ingest and mustache integration can be tested.
2015-12-18 17:35:53 +01:00
Martijn van Groningen 2c5bb84851 fix copyDefaultGeoIp2DatabaseFiles task to work again 2015-12-18 15:45:56 +01:00
javanna 3e155f7b54 adapt to upstream changes: thirdPartyAudit.missingClasses set to true
geoip depends on asm and google http client which we don't need
2015-12-18 11:14:39 +01:00
Martijn van Groningen d38cccb8a1 Fix issues after merging in master 2015-12-11 14:51:45 +01:00
Martijn van Groningen 347b8e600e specify all the dependencies of dependencies, because transitive dependencies have been disabled 2015-11-11 14:39:48 +07:00
javanna f657a7dbf3 Merge branch 'feature/ingest' into ingest/date 2015-11-09 13:42:07 +01:00
Martijn van Groningen 302621f16b geoip: don't store geoinfo if nothing can be resolved 2015-11-09 18:24:05 +07:00
javanna 7798c6cd49 unified date parser tests in a single test class, added more tests for date processor 2015-11-06 12:05:30 +01:00
Martijn van Groningen 03387266ca ingest: Added new `geoip` processor, that adds geographical information to documents based on an ip address.
The information is fetched from the Maxmind geolite2 database, that is embedded in the ingest plugin.
2015-11-03 12:06:29 +07:00
Tal Levy 464b46437f Introduce the GrokProcessor
Also moved all processor classes into a subdirectory and introduced a
ConfigException class to be a catch-all for things that can go wrong
when constructing new processors with configurations that possibly throw
exceptions. The GrokProcessor loads patterns from the resources
directory.

fix resource path issue, and add rest-api-spec test for grok

fix rest-spec tests

changes: license, remove configexception, throw IOException

add more tests and fix iso8601-hour pattern

move grok patterns from resources to config

fix tests with pom changes, updated IngestClientIT with grok processor

update gradle build script for grok deps and test configuration

move config files to src/main/packaging

move Env out of Processor, fix test for src/main/packaging change

add docs

clean up test resources task

update Grok to be immutable

- Updated the Grok class to be immutable. This means that all the
  pattern bank loading is handled by an external utility class called
  PatternUtils.
- fixed tabs in the nagios patterns file's comments
2015-11-02 20:27:49 -08:00
Tal Levy c6f4cd68da add gradle build script to plugin 2015-10-29 13:40:23 -07:00