193 lines
6.4 KiB
Plaintext
193 lines
6.4 KiB
Plaintext
[[ingest-geoip]]
|
|
=== Ingest Geoip Processor Plugin
|
|
|
|
The GeoIP processor adds information about the geographical location of IP addresses, based on data from the Maxmind databases.
|
|
This processor adds this information by default under the `geoip` field. The `geoip` processor can resolve both IPv4 and
|
|
IPv6 addresses.
|
|
|
|
The ingest-geoip plugin ships by default with the GeoLite2 City, GeoLite2 Country and GeoLite2 ASN geoip2 databases from Maxmind made available
|
|
under the CCA-ShareAlike 4.0 license. For more details see, http://dev.maxmind.com/geoip/geoip2/geolite2/
|
|
|
|
The GeoIP processor can run with other geoip2 databases from Maxmind. The files must be copied into the geoip config directory,
|
|
and the `database_file` option should be used to specify the filename of the custom database. Custom database files must be compressed
|
|
with gzip. The geoip config directory is located at `$ES_HOME/config/ingest-geoip` and holds the shipped databases too.
|
|
|
|
:plugin_name: ingest-geoip
|
|
include::install_remove.asciidoc[]
|
|
|
|
[[using-ingest-geoip]]
|
|
==== Using the Geoip Processor in a Pipeline
|
|
|
|
[[ingest-geoip-options]]
|
|
.Geoip options
|
|
[options="header"]
|
|
|======
|
|
| Name | Required | Default | Description
|
|
| `field` | yes | - | The field to get the ip address from for the geographical lookup.
|
|
| `target_field` | no | geoip | The field that will hold the geographical information looked up from the Maxmind database.
|
|
| `database_file` | no | GeoLite2-City.mmdb | The database filename in the geoip config directory. The ingest-geoip plugin ships with the GeoLite2-City.mmdb.gz and GeoLite2-Country.mmdb.gz files.
|
|
| `properties` | no | [`continent_name`, `country_iso_code`, `region_name`, `city_name`, `location`] * | Controls what properties are added to the `target_field` based on the geoip lookup.
|
|
| `ignore_missing` | no | `false` | If `true` and `field` does not exist, the processor quietly exits without modifying the document
|
|
|======
|
|
|
|
*Depends on what is available in `database_field`:
|
|
|
|
* If the GeoLite2 City database is used, then the following fields may be added under the `target_field`: `ip`,
|
|
`country_iso_code`, `country_name`, `continent_name`, `region_name`, `city_name`, `timezone`, `latitude`, `longitude`
|
|
and `location`. The fields actually added depend on what has been found and which properties were configured in `properties`.
|
|
* If the GeoLite2 Country database is used, then the following fields may be added under the `target_field`: `ip`,
|
|
`country_iso_code`, `country_name` and `continent_name`. The fields actually added depend on what has been found and which properties
|
|
were configured in `properties`.
|
|
* If the GeoLite2 ASN database is used, then the following fields may be added under the `target_field`: `ip`,
|
|
`asn`, and `organization_name`. The fields actually added depend on what has been found and which properties were configured
|
|
in `properties`.
|
|
|
|
Here is an example that uses the default city database and adds the geographical information to the `geoip` field based on the `ip` field:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT _ingest/pipeline/geoip
|
|
{
|
|
"description" : "Add geoip info",
|
|
"processors" : [
|
|
{
|
|
"geoip" : {
|
|
"field" : "ip"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
PUT my_index/my_type/my_id?pipeline=geoip
|
|
{
|
|
"ip": "8.8.8.8"
|
|
}
|
|
GET my_index/my_type/my_id
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
|
|
Which returns:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"found": true,
|
|
"_index": "my_index",
|
|
"_type": "my_type",
|
|
"_id": "my_id",
|
|
"_version": 1,
|
|
"_source": {
|
|
"ip": "8.8.8.8",
|
|
"geoip": {
|
|
"continent_name": "North America",
|
|
"country_iso_code": "US",
|
|
"location": { "lat": 37.751, "lon": -97.822 }
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE
|
|
|
|
Here is an example that uses the default country database and adds the
|
|
geographical information to the `geo` field based on the `ip` field`. Note that
|
|
this database is included in the plugin download. So this:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT _ingest/pipeline/geoip
|
|
{
|
|
"description" : "Add geoip info",
|
|
"processors" : [
|
|
{
|
|
"geoip" : {
|
|
"field" : "ip",
|
|
"target_field" : "geo",
|
|
"database_file" : "GeoLite2-Country.mmdb.gz"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
PUT my_index/my_type/my_id?pipeline=geoip
|
|
{
|
|
"ip": "8.8.8.8"
|
|
}
|
|
GET my_index/my_type/my_id
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
|
|
returns this:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"found": true,
|
|
"_index": "my_index",
|
|
"_type": "my_type",
|
|
"_id": "my_id",
|
|
"_version": 1,
|
|
"_source": {
|
|
"ip": "8.8.8.8",
|
|
"geo": {
|
|
"continent_name": "North America",
|
|
"country_iso_code": "US",
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE
|
|
|
|
|
|
Not all IP addresses find geo information from the database, When this
|
|
occurs, no `target_field` is inserted into the document.
|
|
|
|
Here is an example of what documents will be indexed as when information for "80.231.5.0"
|
|
cannot be found:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT _ingest/pipeline/geoip
|
|
{
|
|
"description" : "Add geoip info",
|
|
"processors" : [
|
|
{
|
|
"geoip" : {
|
|
"field" : "ip"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
PUT my_index/my_type/my_id?pipeline=geoip
|
|
{
|
|
"ip": "80.231.5.0"
|
|
}
|
|
GET my_index/my_type/my_id
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
|
|
Which returns:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"found": true,
|
|
"_index": "my_index",
|
|
"_type": "my_type",
|
|
"_id": "my_id",
|
|
"_version": 1,
|
|
"_source": {
|
|
"ip": "80.231.5.0"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE
|
|
|
|
[[ingest-geoip-settings]]
|
|
===== Node Settings
|
|
|
|
The geoip processor supports the following setting:
|
|
|
|
`ingest.geoip.cache_size`::
|
|
|
|
The maximum number of results that should be cached. Defaults to `1000`.
|
|
|
|
Note that these settings are node settings and apply to all geoip processors, i.e. there is one cache for all defined geoip processors.
|