Commit Graph

262 Commits

Author SHA1 Message Date
Lee Hinman 5463f7953f Expose `simple_query_string` flags in `flags` parameter 2014-01-03 16:14:19 -07:00
Alexander Reelsen 811b7d7d78 Do not start packages on installation
The reason to not start packages on installation is to allow to configure
them before starting up (setting heap, cluster.name etc)

Also the documentation was updated in order to show, which statements need
to be executed.
In addition, these statements are also printed out when the package is
installed, depending on whether chkconfig, system or update-rc.d is used.

Closes #3722
2014-01-03 17:40:27 +01:00
Martijn van Groningen f1bf585089 The `fields` option should always return an array for json document fields and single valued field for metadata fields.
Also the `fields` option can only be used to fetch leaf fields, trying to do fetch object fields will return in a client error.

Closes #4542
2014-01-03 17:29:12 +01:00
David Pilato 0c7b494bb8 plugin manager: new `timeout` option
When testing plugin manager with real downloads, it could happen that the test run forever. Fortunately, test suite will be interrupted after 20 minutes, but it could be useful not to fail the whole test suite but only warn in that case.

By default, plugin manager still wait indefinitely but it can be modified using new `--timeout` option:

```sh
bin/plugin --install elasticsearch/kibana --timeout 30s

bin/plugin --install elasticsearch/kibana --timeout 1h
```

Closes #4603.
Closes #4600.
2014-01-03 16:48:18 +01:00
Britta Weber 9f54e9782d rename _shard -> _index and also rename classes and variables
closes #4584
2014-01-03 14:00:23 +01:00
Lee Hinman a754224751 Add field data memory circuit breaker.
This adds the field data circuit breaker, which is used to estimate
the amount of memory required to load field data before loading it. It
then raises a CircuitBreakingException if the limit is exceeded.

It is configured with two parameters:

`indices.fielddata.cache.breaker.limit` - the maximum number of bytes
of field data to be loaded before circuit breaking. Defaults to
`indices.fielddata.cache.size` if set, unbounded otherwise.

`indices.fielddata.cache.breaker.overhead` - a contast for all field
data estimations to be multiplied with before aggregation. Defaults to
1.03.

Both settings can be configured dynamically using the cluster update
settings API.
2014-01-02 15:04:47 -07:00
Martijn van Groningen aa548f5148 Remove GET `_aliases` api in favour for GET `_alias` api
Currently there are two get aliases apis that both have the same functionality, but have a different response structure. The reason for having 2 apis is historic.

The GET _alias api was added in 0.90.x and is more efficient since it only sends the needed alias data from the cluster state between the master node and the node that received the request. In the GET _aliases api the complete cluster state is send to the node that received the request and then the right information is filtered out and send back to the client.

The GET _aliases api should be removed in favour for the alias api

Closes to #4539
2014-01-02 13:56:11 +01:00
Martijn van Groningen f4bf0d5112 Replaced `ignore_indices` with `ignore_unavailable`, `expand_wildcards` and `allow_no_indices`.
* `ignore_unavailable` - Controls whether to ignore if any specified indices are unavailable, this includes indices that don't exist or closed indices. Either `true` or `false` can be specified.
* `allow_no_indices` - Controls whether to fail if a wildcard indices expressions results into no concrete indices. Either `true` or `false` can be specified. For example if the wildcard expression `foo*` is specified and no indices are available that start with `foo` then depending on this setting the request will fail. This setting is also applicable when `_all`, `*` or no index has been specified.
* `expand_wildcards` - Controls to what kind of concrete indices wildcard indices expression expand to. If `open` is specified then the wildcard expression if expanded to only open indices and if `closed` is specified then the wildcard expression if expanded only to closed indices. Also both values (`open,closed`) can be specified to expand to all indices.

Closes to #4436
2014-01-02 12:19:45 +01:00
Britta Weber 1ede9a5730 make term statistics accessible in scripts
term statistics can be accessed via the _shard variable.

Below is a minimal example. See documentation on details.

```

DELETE paytest

PUT paytest
{
    "mappings": {
        "test": {
            "_all": {
                "auto_boost": true,
                "enabled": true
            },
            "properties": {
                "text": {
                    "index_analyzer": "fulltext_analyzer",
                    "store": "yes",
                    "type": "string"
                }
            }
        }
    },
    "settings": {
        "analysis": {
            "analyzer": {
                "fulltext_analyzer": {
                    "filter": [
                        "my_delimited_payload_filter"
                    ],
                    "tokenizer": "whitespace",
                    "type": "custom"
                }
            },
            "filter": {
                "my_delimited_payload_filter": {
                    "delimiter": "+",
                    "encoding": "float",
                    "type": "delimited_payload_filter"
                }
            }
        },
        "index": {
            "number_of_replicas": 0,
            "number_of_shards": 1
        }
    }
}

POST paytest/test/1
{
    "text": "the+1 quick+2 brown+3 fox+4 is quick+10"
}

POST paytest/test/2
{
    "text": "the+1 quick+2 red+3 fox+4"
}

POST paytest/_refresh

POST paytest/_search
{
    "script_fields": {
       "ttf": {
          "script": "_shard[\"text\"][\"quick\"].ttf()"
       }
    }
}

POST paytest/_search
{
    "script_fields": {
       "freq": {
          "script": "_shard[\"text\"][\"quick\"].freq()"
       }
    }
}
POST paytest/test/2/_termvector
POST paytest/_search
{
    "script_fields": {
       "payloads": {
          "script": "term = _shard[\"text\"].get(\"red\",_PAYLOADS);payloads = []; for(pos : term){payloads.add(pos.payloadAsFloat(-1));} return payloads;"
       }
    }
}

POST paytest/_search
{
   "script_fields": {
      "tv": {
         "script": "_shard[\"text\"][\"quick\"].freq()"
      }
   },
   "query": {
      "function_score": {
         "functions": [
            {
               "script_score": {
                  "script": "_shard[\"text\"][\"quick\"].freq()"
               }
            }
         ]
      }
   }
}

```

closes #3772
2014-01-02 11:17:33 +01:00
Adrien Grand 1654ae8937 Explicit doc_values setting.
Once doc values are enabled on a field, they can't be disabled.

Close #4560
2013-12-30 11:10:52 +01:00
Adrien Grand 05448b6276 Doc values for geo points.
This commits add doc values support to geo point using the exact same approach
as for numeric data: geo points for a given document are stored uncompressed
and sequentially in a single binary doc values field.

Close #4207
2013-12-27 12:45:18 +01:00
Florian Schilling bc452dff84 * setup accurate GeoDistance Function
* adapt tests
* introduced default GeoDistance function
* Updated docs

closes #4498
2013-12-27 19:15:19 +09:00
Andrew Raines 69d88a1edd [DOCS] Add headers and help parameters. 2013-12-23 22:26:28 -06:00
Martijn van Groningen eb86a3a6fe [DOCS] Changed `shape_field_name` to `path` in geo_shape filter documentation.
Relates to #4486
2013-12-23 11:27:06 +01:00
Clinton Gormley 998b7b3b86 [DOCS] Fixed community links to official clients 2013-12-20 12:16:58 +01:00
Clinton Gormley dea6b112ae [DOCS] Corrected bloom loading docs 2013-12-20 11:20:54 +01:00
Clinton Gormley 2b8c82c883 [DOCS] Documented index.codec.bloom.load for #4525 2013-12-20 10:51:17 +01:00
Clinton Gormley 51dc057244 [DOCS] Added the official PHP client to the community page. 2013-12-20 10:51:17 +01:00
Richard Pijnenburg df85fdf88f Add repository information to docs
This adds the apt and yum repo information to the setup docs.
2013-12-19 15:58:08 +01:00
Adrien Grand 52db8eb324 More documentation improvements for fielddata loading. 2013-12-18 16:05:35 +01:00
Adrien Grand 07443089ce Improve documentation of the new `disabled` field data format. 2013-12-18 15:44:57 +01:00
Boaz Leskes 3c5106ae98 Added cluster health status to the Cluster Stats API
Relates to #4460
2013-12-18 12:03:49 +01:00
Chris Simpson 4f8c916eed [Docs] Fix Typo
Fixes small typo in the geo_distance aggregation docs.
2013-12-18 11:21:21 +01:00
spenceralger 89e6b9cfc4 Merge pull request #4494 from spenceralger/add_js_docs
JavaScript client docs
2013-12-17 14:41:57 -08:00
Spencer Alger a8ca8497c5 added doc page for the JavaScipt client, and listed it in the clients list. 2013-12-17 15:26:29 -07:00
Boaz Leskes 2b6214cff7 Added Cluster Stats API
Closes #4460
2013-12-17 13:14:46 +01:00
Grégory Quatannens c64abaae7e Fixing typo and grammar 2013-12-17 11:39:02 +01:00
Adrien Grand 33599d9a34 Compressed geo-point field data.
This commit allows to trade precision for memory when storing geo points.
This new field data impl accepts a `precision` parameter that controls the
maximum expected error for storing coordinates. This option can be updated on
a live index with the PUT mapping API.

Default precision is 1cm, which requires 8 bytes per geo-point (50% memory
saving compared to using 2 doubles).

Close #4386
2013-12-17 11:29:48 +01:00
Clinton Gormley 684affa5c7 [DOCS] Removed unused file 2013-12-17 11:28:19 +01:00
Alexander Reelsen b713cf56ed Allow to provide parameters not only through -D but as long parameters
All getopt long style parameters are now set as es. properties,

elasticsearch --path.data=/some/path

results in -Des.path.data=/some/path

Closes #4393
2013-12-17 10:43:27 +01:00
Alexander Reelsen c30945a3d8 Start elasticsearch in the foreground by default
Instead of using the '-f' parameter to start elasticsearch in the
foreground, this is now the default modus.

In order to start elasticsearch in the background, the '-d' parameter
can be used.

Closes #4392
2013-12-17 10:39:22 +01:00
Clinton Gormley 34b9b16233 [DOCS] Fixed some bad link refs 2013-12-16 18:07:33 +01:00
Martijn van Groningen 23d2b1ea7b Renamed top level `filter` to `post_filter`.
Closes #4119
2013-12-16 17:10:14 +01:00
Lee Hinman db431b7cb3 Remove the `field` and `text` queries.
The `text` query was replaced by the `match` query and has been
deprecated for quite a while.

The `field` query should be replaced by a `query_string` query with
the `default_field` specified.

Fixes #4033
2013-12-16 08:59:36 -07:00
Adrien Grand 4e7ce4ee02 Make field data changes immediately taken into account and add the ability to disallow field data loading.
This commit changes field data configuration updates so that they are
immediately taken into account for loading new segments. The way it works
is that field data configuration is now cached separately from the field
data cache, meaning that it is now possible to clear the field data
configuration from IndexFieldDataService while the cache will stay around. On
the next time that Elasticsearch will reload field data configuration, it will
check if there is already a cache entry, and reuse it if it exists.

To disable field data loading, all that is required is to change the field
data format to "none" (supported by all field data types) using the update
mapping API. Elasticsearch will then refuse to load field data on any new
segment, but field data which has been loaded on the previous segments will
remain available. So you need to clear the field data cache in order to
reclaim memory (otherwise memory will be reclaimed slower, as segments get
merged).

Close #4430
Close #4431
2013-12-16 14:34:33 +01:00
Adrien Grand 36bd9cc432 Aggregations: Ordinals-based string bucketing support.
When the ValuesSource has ordinals, terms ordinals are used as a cache key to
bucket ordinals. This can make terms aggregations on String terms significantly
faster.

Close #4350
2013-12-13 15:34:02 +01:00
Martijn van Groningen 10e2528cce Added the `force_source` option to highlighting that enforces to use of the _source even if there are stored fields.
The percolator uses this option to deal with the fact that the MemoryIndex doesn't support stored fields,
this is possible b/c the _source of the document being percolated is always present.

Closes #4348
2013-12-13 13:39:53 +01:00
Lee Hinman 77fcf71338 Add new `simple_query_string` query type
This adds support for Lucene's SimpleQueryParser by adding a new type
of query called the `simple_query_string`. The `simple_query_string`
query is designed to be able to parse human-entered queries without
throwing any exceptions.

Resolves #4159
2013-12-12 12:09:32 -07:00
Alexander Reelsen 81e13a870b Packaging: Ensure setting of sysctl vm.max_map_count
In order to be sure that memory mapped lucene directories are working
one can configure the kernel about how many memory mapped areas
a process may have. This setting ensure for the debian and redhat initscripts
as well as the systemd startup, that this setting is set high enough.

Closes #4397
2013-12-11 09:19:22 +01:00
Boaz Leskes 99b421925f Add wildcard support to field resolving in the Get Field Mapping API
Closes #4367
2013-12-10 23:46:37 +01:00
Simon Willnauer 6c189310b9 Remove 'term_index_interval' and 'term_index_divisor'
These settings are no longer relevant since they are codec /
postingsformat level settings since Lucene 4.0

Closes #3912
2013-12-10 16:54:08 +01:00
Martijn van Groningen ebf6519965 Added aggs option to percolate api documentation. 2013-12-10 14:09:37 +01:00
Lee Hinman bc9698a347 Support 'yaml' as a format for the Analyze API
Fixes #4311
2013-12-08 15:08:00 -07:00
Martijn van Groningen 8c1de501e7 Update percolator highlighting docs. 2013-12-07 16:40:49 -05:00
Adrien Grand 32eb5ffa92 [Docs] Document which encoding should be used in order to make sense of the offsets returned by the term vectors API.
Close #4363
2013-12-06 22:39:08 +01:00
Lee Hinman a1d4731137 [DOCS] Fix outdated link to wonderdog in community integration 2013-12-06 12:05:43 -07:00
Shay Banon 28eff2ba29 remove help command, list all cat commands in /_cat?h endpoint 2013-12-05 14:36:27 +01:00
Markus Fischer 2da0611dfb [DOCS] Completion suggest: Clarify de-duplication, optimize/merge
This contribution is based on the feedback given in issue #4254 and
issue #4255, and should clear things up, when suggestions are being
removed and not displayed anymore after deletion of data.
2013-12-05 11:10:56 +01:00
Nik Everett 8e34057bc0 Add support for combining fields to the FVH
The Fast Vector Highlighter can combine matches on multiple fields to
highlight a single field using `matched_fields`.  This is most
intuitive for multifields that analyze the same string in different
ways.  Example:
{
    "query": {
        "query_string": {
            "query": "content.plain:running scissors",
            "fields": ["content"]
        }
    },
    "highlight": {
        "order": "score",
        "fields": {
            "content": {
                "matched_fields": ["content", "content.plain"],
                "type" : "fvh"
            }
        }
    }
}

Closes #3750
2013-12-03 11:10:01 +01:00
Yousef 302c762d5e Wrong link to Token Filter 2013-12-03 10:39:13 +01:00
Nik Everett 7690b40ec6 Allow string fields to store token counts
To use this one you send a string to a field of type 'token_count'.  This
makes the most sense with a multi-field.
2013-12-03 09:39:32 +01:00
Alexander Reelsen 6528df2764 [DOCS] Test framework documentation
The java test framework using randomized testing is explained with a couple of examples.
2013-12-02 18:01:45 +01:00
Clinton Gormley 7d993fd917 [DOCS] Another cat?v change 2013-12-02 15:30:49 +01:00
Clinton Gormley 5b15ed73fa [DOCS] Linked cat-pending to cluster-pending 2013-12-02 15:29:47 +01:00
Clinton Gormley 992b2d82b0 [DOCS] Changed the _cat docs to use ?v instead of ?v=true 2013-12-02 15:27:41 +01:00
Clinton Gormley d9a480c97a [DOCS] Typos in aggregations 2013-12-02 15:14:25 +01:00
Conrad Pankoff 87246af256 [DOCS] Fixed typos and corrected grammar 2013-12-02 10:08:26 +01:00
uboness cdc7dfbb2c Changed the "script_lang" parameter to "lang" in all value source based aggs - to be consistent with all other script based APIs. 2013-12-02 02:01:03 +01:00
Clinton Gormley bc393b6d79 Changed the minScore comparator from > to >=
Closes #4303
2013-11-29 20:29:20 +01:00
uboness 0d6a35b9a7 - Added support for term filtering based on include/exclude regex on the terms agg
- Added javadoc to the TermsBuilder

Closes #4267
2013-11-29 13:46:48 +01:00
uboness afb0d119e4 - Added docs for the value_count aggregation
- Fixed typos in the terms facets docs
- Fixed aggregation docs layout
- Added docs for shard_size in term aggregation
2013-11-29 12:35:42 +01:00
Clinton Gormley b48344f296 [DOCS] Doc'ed cluster pending tasks 2013-11-29 08:21:26 +01:00
Andrew Raines 91999e14ce Add _cat/pending_tasks.
Closes #4251.
2013-11-29 01:09:06 -06:00
Lee Hinman 9939e81d88 [DOCS] Fix porter stem filter name in other stemming docs 2013-11-28 22:14:47 -07:00
Lee Hinman fb4e903e35 [DOCS] Fix name of porter stemming token filter 2013-11-28 22:01:19 -07:00
Clinton Gormley 6ce3495029 [DOCS] Fixed a bad link 2013-11-27 17:54:25 +01:00
Clinton Gormley cdc1935b6e [DOCS] Documented rest.action.multi.allow_explicit_index 2013-11-27 17:33:09 +01:00
Boaz Leskes c63d8c4fb5 [Docs] Added _source filtering to documentation
Relates to #3301
2013-11-26 19:16:24 +01:00
Britta Weber dbef64009f [DOC] add doc for multi term vector api
closes #3998
2013-11-26 17:03:14 +01:00
Alexander Reelsen bf74f49fdd Updated Analyzing/Fuzzysuggester from lucene trunk
* Minor alignments (like setter to ctor)
* FuzzySuggester has a unicode aware flag, which is not exposed in the fuzzy completion request parameters
* Made XAnalyzingSuggester flags (PAYLOAD_SEP, END_BYTE, SEP_LABEL) to be written into the postings format, so we can retain backwards compatibility
* The above change also implies, that these flags can be set per instantiated XAnalyzingSuggester
* CompletionPostingsFormatTest now uses a randomProvider for writing data to check for bwc
2013-11-26 12:52:06 +01:00
Martijn van Groningen a03556daa0 Added execution option to `range` filter, with the `index` and `fielddata` as values.
Deprecated `numeric_range` filter in favor for the `range` filter with `fielddata` as execution.

Closes #4034
2013-11-25 23:43:40 +01:00
uboness c7f6c5266d initial commit of the aggregations module
Closes #3300
2013-11-24 03:13:08 -08:00
Jun Ohtani 7bbe453273 [DOCS] Added elasticsearch-extended-analyze plugin 2013-11-21 09:48:00 +01:00
Clinton Gormley 7c59ed4087 [DOCS] Fixed duplicate docs ID in delete 2013-11-21 17:38:51 +11:00
Shay Banon a9880dcbf1 add timeout doc to delete 2013-11-20 12:50:03 -08:00
Matt Weber a841a422f6 Add a field data based TermsFilter
Add FieldDataTermsFilter that compares terms out of
the fielddata cache. When filtering on a large
set of terms this filter can be considerably faster
than using a standard lucene terms filter.

Add the "fielddata" execution mode to the
terms filter parser to enable the use of
the new FieldDataTermsFilter.

Add supporting tests and documentation.

Closes #4209
2013-11-19 19:18:16 +01:00
Andrew Raines 8fabeb1c0b First pass at cat docs. 2013-11-14 21:37:02 -05:00
Andrew Raines 5c085c1204 Fix misspellings. 2013-11-14 20:10:36 -05:00
Luca Cavanna 0aaa39d00a Minor improvements to indices filter and query & updated docs
Slightly simplified indices filter and query parsers code
Trimmed down tests where possible
2013-11-14 17:25:34 +01:00
Olivier Favre fa80ca97b2 Indices query/filter skip parsing altogether for irrelevant indices when possible
Closes #2416
2013-11-14 17:24:49 +01:00
Igor Motov 510397aecd Initial implementation of Snapshot/Restore API
Closes #3826
2013-11-10 18:26:56 -05:00
Lee Hinman f7d5d1e5c9 [DOCS] Update store docs to indicate mmapfs is now the default on 64-bit Linux 2013-11-09 11:42:43 -07:00
Clinton Gormley 5af4e02d6c [DOCS] Fix link to statsd plugin
Fixes #4128
2013-11-08 20:29:51 +01:00
Clinton Gormley 7189310764 In ctor of GeoPointFieldMapper, geohash_prefix now implicitly enables geohash option
Also improved docs for geopoint type and geohash_cell filte

Closes #3951
2013-11-08 13:52:17 +01:00
Cory G Watson 6bbcc34061 Add wabisabi to Scala clients. 2013-11-08 10:34:14 +01:00
Clinton Gormley b27976fbed [DOCS] Fixed the fielddata regex example on core mapping 2013-11-07 17:09:18 +01:00
Clinton Gormley 3465e69e83 [DOCS] Changed all store:yes/no to store:true/false
which is how this setting is stored internally
2013-11-07 16:57:18 +01:00
Simon Willnauer 77bc5d5ecf release [1.0.0.Beta1] 2013-11-06 15:32:43 +01:00
Simon Willnauer 9654631186 Change 'standart' analyzer to use emtpy stopword list by default.
The 'default' / 'standard' analyzer can be a trappy default sicne it filters
english stopwords by default. Yet a default should not be dedicated to a certain language
since elasticsearch is used in many different scenarios where a standard analysis chain
with specialization to english full-text might be rather counter productive.

This commit changes the 'standard' analyzer to use an empty stopword list for indices
that are created from 1.0.0.Beta1 version onwards but will maintain backwards compatibiliy
for older indices.

Closes #3775
2013-11-05 21:07:21 +01:00
Shay Banon 7c32269f4f Dist. Percolation: Use .percolator instead of _percolator for type name
Use .percolator as the internal (hidden) type name for percolators within the index. Seems nicer name to represent "hidden" types within an index.
closes #4090
2013-11-05 20:02:59 +01:00
Boaz Leskes a9fdcadf01 [DOCS] Added documentation for the keep word token filter 2013-11-04 18:38:44 +01:00
Clinton Gormley 356de95840 Added simplified range syntax to query string docs 2013-11-04 18:18:36 +01:00
Karel Minarik b93dac678f [DOC] Added a link to the official Ruby client to the "Clients" page 2013-11-04 11:47:14 +01:00
Karel Minarik 7023ef2e3f [DOCS] Added a basic information about the official Ruby client to documentation 2013-11-04 11:46:36 +01:00
Ben McCann 46edfc484a [DOCS] Add some documentation about the performance of `_source` usage in scripts. 2013-11-04 11:05:55 +01:00
Igor Motov c724f0de5d Initial implementation of ResourceWatcherService
Closes #4062
2013-11-03 21:55:54 -05:00
Dan Everton 6df60b7271 [DOC] Improve documentation on search stats groups
Document the ability to return all search statistics groups and provide examples of returning search statistics for groups.
2013-11-01 13:53:39 +01:00
Martijn van Groningen 30ab6f841d [DOCS] Fixed percolate docs errors 2013-11-01 11:44:07 +01:00
Clinton Gormley 4206cc988e [DOCS] Typo on shingle tokenfilter 2013-10-31 20:18:00 +01:00
Opak Alex 6856cfc5e3 add reference for ember-data-elasticsearch-kit to integrations page 2013-10-31 11:40:01 +01:00