217 Commits

Author SHA1 Message Date
Martijn van Groningen
1cfb6a79f1 Parent/child: refactored _parent field mapper and parent/child queries
* Cut the `has_child` and `has_parent` queries over to use Lucene's query time global ordinal join. The main benefit of this change is that parent/child queries can now efficiently execute if parent/child queries are wrapped in a bigger boolean query. If the rest of the query only hit a few documents both has_child and has_parent queries don't need to evaluate all parent or child documents any more.
* Cut the `_parent` field over to use doc values. This significantly reduces the on heap memory footprint of parent/child, because the parent id values are never loaded into memory.

Breaking changes:
* The `type` option on the `_parent` field can only point to a parent type that doesn't exist yet, so this means that an existing type/mapping can't become a parent type any longer.
* The `has_child` and `has_parent` queries can no longer be use in alias filters.

All these changes, improvements and breaks in compatibility only apply for indices created with ES version 2.0 or higher. For indices creates with ES <= 2.0 the older implementation is used.

It is highly recommended to re-index all your indices with parent and child documents to benefit from all the improvements that come with this refactoring. The easiest way to achieve this is by using the scan and bulk apis using a simple script.

Closes #6107
Closes #8134
2015-05-29 21:44:17 +02:00
Colin Goodheart-Smithe
35a58d874e Scripting: Unify script and template requests across codebase
This change unifies the way scripts and templates are specified for all instances in the codebase. It builds on the Script class added previously and adds request building and parsing support as well as the ability to transfer script objects between nodes. It also adds a Template class which aims to provide the same functionality for template APIs

Closes #11091
2015-05-29 16:52:04 +01:00
Adrien Grand
461683ac58 Mappings: Remove the compress/compress_threshold options of the BinaryFieldMapper.
This option is broken currently since it potentially interprets an incoming
binary value as compressed while it just happens that the first bytes are the
same as the LZF header.
2015-05-22 14:20:42 +02:00
Ryan Ernst
e29492ce94 Docs: Cleanup meta field docs
Meta fields were locked down to not allow exotic options to the
underlying field types in #8143. This change fixes the docs
to no longer refer to the old settings.

closes #10879
2015-05-07 11:26:49 -07:00
Adrien Grand
a0af88e996 Query DSL: Remove filter parsers.
This commit makes queries and filters parsed the same way using the
QueryParser abstraction. This allowed to remove duplicate code that we had
for similar queries/filters such as `range`, `prefix` or `term`.
2015-05-07 20:14:34 +02:00
Ryan Ernst
7a7bd6086a Mappings: Remove ability to disable _source field
Current features (eg. update API) and future features (eg. reindex API)
depend on _source. This change locks down the field so that
it can no longer be disabled. It also removes legacy settings
compress/compress_threshold.

closes #8142
closes #10915
2015-05-05 22:04:18 -07:00
Ryan Ernst
d2b12e4fc2 Mappings: Remove docs for type level analyzer defaults
These settings were removed in #9430.
2015-04-30 13:57:55 -07:00
Ryan Ernst
4ef9f3ca63 Mappings: Remove file based default mappings
Using files that must be specified on each node is an anti-pattern
from the API based goal of ES. This change removes the ability
to specify the default mapping with a file on each node.

closes #10620
2015-04-30 13:50:35 -07:00
Adrien Grand
6e076efdb9 Docs: Add documentation for the doc_values setting on the boolean field type.
Close #10431
2015-04-29 15:59:24 +02:00
Clinton Gormley
7aa4c7e256 Docs: Removed a reference to index_name from the array mapping page 2015-04-29 15:12:31 +02:00
Ryan Ernst
bf09e58cb3 Mappings: Remove includes and excludes from _source
Regardless of the outcome of #8142, we should at least enforce that
when _source is enabled, it is sufficient to reindex. This change
removes the excludes and includes settings, since these modify
the source, causing us to lose the ability to reindex some fields.

closes #10814
2015-04-28 15:03:51 -07:00
Clinton Gormley
2579cc31b1 Docs: Note that include_in_parent/root does not apply to geo-shape fields
Closes #10653
2015-04-25 16:49:49 +02:00
Nicholas Knize
453217fd7a [GEO] Prioritize tree_level and precision parameters over default distance_error_pct
If a user explicitly defined the tree_level or precision parameter in a geo_shape mapping their specification was always overridden by the default_error_pct parameter (even though our docs say this parameter is a 'hint'). This lead to unexpected accuracy problems in the results of a geo_shape filter. (example provided in issue #9691)

This simple patch fixes the unexpected behavior by setting the default distance_error_pct parameter to zero when the tree_level or precision parameters are provided by the user. Under the covers the quadtree will now use the tree level defined by the user. The docs will be updated to alert the user to exercise caution with these parameters.  Specifying a precision of "1m" for an index using large complex shapes can quickly lead to OOM issues.

closes #9691
2015-04-21 14:42:10 -05:00
Clinton Gormley
abc7de96ae Docs: Updated version annotations in master 2015-04-09 14:50:11 +02:00
Adrien Grand
fae124103a Merge pull request #10420 from jpountz/feature/numeric_resolution
Mappings: Bring back numeric_resolution.

Close #10420
2015-04-09 12:28:33 +02:00
Clinton Gormley
a95b11ca61 Document doc_values for field type ip
Closes #9809
2015-04-04 17:51:28 +02:00
Adrien Grand
c7115f8364 Mappings: Bring back numeric_resolution.
We had an undocumented parameter called `numeric_resolution` which allows to
configure how to deal with dates when provided as a number. The default is to
handle them as milliseconds, but you can also opt-on for eg. seconds.

Close #10072
2015-04-03 19:54:14 +02:00
Guillaume Dievart
adcb782423 Update core-types.asciidoc 2015-04-03 14:12:29 +02:00
Nicholas Knize
c2ec463cdb [GEO] fix docs for geo_point "validate" option
Documentation states false as the default for "validate", "validate_lon", and "validate_lat" leading to confusion as described in issue #9539. This simple fix corrects the documentation and communicates that these fields will be deprecated and removed in upcoming versions.

closes #9539
2015-03-23 15:34:37 -05:00
David Pilato
0c8da6bb84 [doc] Link mapper-attachment type documentation to its repo
As explained in elasticsearch/elasticsearch-mapper-attachments#101, we should have consistent documentation.

The best option is to link the  documentation in elasticsearch guide to the most recent README in the plugin repo.

Closes #9756
2015-02-27 22:18:59 +01:00
Martijn van Groningen
daefb4c673 Docs: Document that the fielddata loading defaults to eager on the _parent field.
Closes #9804
2015-02-22 23:15:59 +01:00
Clinton Gormley
20ece4acb5 Update core-types.asciidoc
Provide an example of how to disable norms

Closes #9641
2015-02-12 12:10:11 +01:00
Ryan Ernst
b3474f6b25 Mappings: Remove ability to set path for _id and _routing on 2.0+ indexes
_id and _routing now no longer support the 'path' setting on indexes
created with 2.0.  Indexes created before 2.0 still support this
setting for backcompat.

closes #6730
2015-02-10 10:53:44 -08:00
Ryan Ernst
c6968883a7 Mappings: Remove support for new indexes using path setting in
object/nested fields or index_name in any field

Backcompat is still here for indexes created before 2.0.

closes #6677
2015-02-05 12:44:43 -08:00
David Pilato
878e46d7f9 [Docs] fix missing space 2015-01-29 19:17:41 +01:00
Ryan Ernst
afcedb94ed Mappings: Remove index_analyzer setting to simplify analyzer logic
The `analyzer` setting is now the base setting, and `search_analyzer`
is simply an override of the search time analyzer.  When setting
`search_analyzer`, `analyzer` must be set.

closes #9371
2015-01-28 13:43:15 -08:00
Ryan Ernst
385c43c141 Mappings: Remove _analyzer
closes #9279
2015-01-26 09:14:17 -08:00
David Pilato
fb10346953 [Mapper] Add ignore_missing option to timestamp
Related to #9049.

By default, the default value for `timestamp` is `now` which means the date the document was processed by the indexing chain.

You can now reject documents which not provide a `timestamp` value by setting `ignore_missing` to false (default to `true`):

```js
{
    "tweet" : {
        "_timestamp" : {
            "enabled" : true,
            "ignore_missing" : false
        }
    }
}
```

When you update the cluster to 1.5 or master, this index created with 1.4 we automatically migrate an index created with 1.4 to the 1.5 syntax.

Let say you have defined this in elasticsearch 1.4.x:

```js
DELETE test
PUT test
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  }
}
PUT test/type/_mapping
{
  "type" : {
      "_timestamp" : {
          "enabled" : true,
          "default" : null
      }
  }
}
```

After migration, the mapping become:

```js
{
   "test": {
      "mappings": {
         "type": {
            "_timestamp": {
               "enabled": true,
               "store": false,
               "ignore_missing": false
            },
            "properties": {}
         }
      }
   }
}
```

Closes #8882.
2015-01-20 13:20:05 +01:00
dtpeacock
582d5e8d3c Doc has store "false" not store "true"
Came from 3465e69e83a4f9f375fb9486741db51efb800837 due to changing "yes" to "false".

Closes #9075
2014-12-29 11:59:22 +01:00
Nicholas Knize
77a7ef28b3 [GEO] Add optional left/right parameter to GeoJSON
This feature adds an optional orientation parameter to the GeoJSON document and geo_shape mapping enabling users to explicitly define how they want Elasticsearch to interpret vertex ordering.  The default uses the right-hand rule (counterclockwise for outer ring, clockwise for inner ring) complying with OGC Simple Feature Access standards. The parameter can be explicitly specified for an entire index using the geo_shape mapping by adding "orientation":{"left"|"right"|"cw"|"ccw"|"clockwise"|"counterclockwise"} and/or overridden on each insert by adding the same parameter to the GeoJSON document.

closes #8764
2014-12-22 12:09:45 -06:00
Nicholas Knize
ac0e37449e Adding unit test for self intersecting polygons. Relevant to #7751 even/odd discussion
Updating documentation to describe polygon ambiguity and vertex ordering.
2014-12-16 10:54:39 -06:00
Ryan Ernst
37287284e6 Settings: Remove mapping.date.round_ceil setting for date math parsing
The setting `mapping.date.round_ceil` (and the undocumented setting
`index.mapping.date.parse_upper_inclusive`) affect how date ranges using
`lte` are parsed.  In #8556 the semantics of date rounding were
solidified, eliminating the need to have different parsing functions
whether the date is inclusive or exclusive.

This change removes these legacy settings and improves the tests
for the date math parser (now at 100% coverage!). It also removes the
unnecessary function `DateMathParser.parseTimeZone` for which
the existing `DateTimeZone.forID` handles all use cases.

Any user previously using these settings can refer to the changed
semantics and change their query accordingly. This is a breaking change
because even dates without datemath previously used the different
parsing functions depending on context.

closes #8598
closes #8889
2014-12-15 13:13:45 -08:00
dw
ad408eee85 Docs: Reword note regarding _source for accuracy
Previously it suggested _source was always present, when that is not the case.

Closes #8491
2014-11-24 12:19:44 +01:00
Alban Perillat-Merceroz
54466938da Fix error in documentation
Indexation does not fail if no timestamp provided when there is a default value defined in mapping.
2014-11-23 14:02:51 +01:00
Martijn van Groningen
94c1a7dabe Docs: Fix incorrect documentation for the index.query.parse.allow_unmapped_fields setting.
The `index.query.parse.allow_unmapped_fields` setting can't influence whether unmapped fields are allowed
in alias filters and percolator queries.
2014-11-11 15:13:55 +00:00
Clinton Gormley
8f02c451b8 Update source-field.asciidoc
very minor typofix

Closes #8066
2014-10-29 14:51:05 +01:00
Adrien Grand
f4ee3f25e4 Mappings: Store _timestamp by default.
Storing `_timestamp` by default means that under the default configuration, you
would have all the information you need in order to reindex into a different
index.

Close #8139
2014-10-20 12:17:26 +02:00
Ryan Grimm
74586e2867 Docs: Added 'd' to the list of supported units.
Day was missing from the list of supported units in the date math section.

Closes #8151
2014-10-19 21:24:28 +02:00
Sergii Golubev
0fb81d6643 Docs: Update id-field.asciidoc
It is strange to provide an example with `"store" : false` when talking about possibility of enabling the field to be stored.
Broke the line in the mapping in two lines for better readability.
More verbose sentence above the mapping.

Closes #7894
2014-10-17 15:17:24 +02:00
Son
6f3227db01 Docs: Fix order for PUT _mapping docs
Closes #8083
2014-10-16 18:49:36 +02:00
Colin Goodheart-Smithe
c7971fb0f9 Docs: adds note about using null_value with dynamic templates
Closes #7874
2014-10-16 14:00:00 +01:00
Clinton Gormley
66cd3ad17e Docs: Update object-type.asciidoc to make object type more explicit
Closes #7646
2014-10-14 13:16:37 +02:00
Clinton Gormley
cb00d4a542 Docs: Removed all the added/deprecated tags from 1.x 2014-09-26 21:04:42 +02:00
Martijn van Groningen
52f1ab6e16 Core: Added the index.query.parse.allow_unmapped_fields setting to fail queries if they refer to unmapped fields.
The percolator and filters in aliases by default enforce strict query parsing.

Closes #7335
2014-09-09 15:00:47 +02:00
Peter Johnson @insertcoffee
10197936da Docs: typo?
Closes #7458
2014-08-26 15:26:34 +02:00
Britta Weber
2a32cf338e Docs: _ttl ignores all parameters except for enabled and default.
Also, add a line about that it can never be disabled after it was enabled.
2014-08-21 16:16:02 +02:00
Adrien Grand
ea96359d82 Facets: Removal from master.
Close #7337
2014-08-21 10:34:39 +02:00
Colin Goodheart-Smithe
b2286915cd Geo: Adds support for GeoJSON GeometryCollection
Closes #2796
2014-08-18 13:55:39 +01:00
Elie A
e1a2d76626 Fixing typo in the first JSON example
Closes #7172
2014-08-07 14:06:09 +02:00
Adrien Grand
a242a63817 [DOCS] Remove the section about codecs.
This documentation was dangerous because it felt like it was possible to gain
substantial performance by just switching the codec of the index.

However, non-default codecs are dangerous to use since they are not supported
in terms of backward compatibility, and most improvements that they bring have
been folded into the default codec anyway (for example, the default codec
"pulses" postings lists that contain a single document).
2014-08-07 11:24:44 +02:00