Commit Graph

1280 Commits

Author SHA1 Message Date
Clinton Gormley aa94ced0ae Remove references to the thrift and memcached transport plugins
as they are no longer supported

Closes #10166
2015-03-19 20:49:58 +01:00
Clinton Gormley 25369f0727 Remove async replication from the docs and REST spec
Relates to #10114
2015-03-19 15:34:12 +01:00
javanna f4691458d5 [DOCS] added note about dynamic scriptings and updated links in getting started page
Closes #10074
2015-03-19 15:22:39 +01:00
javanna ddcecc2bc2 [DOCS] added instructions on how to write parameterized tests
Closes #9423
2015-03-19 12:43:51 +01:00
jaymode 105bdd486a [HTTP] add option to only return simple exception messages
Adds a setting to disable detailed error messages and full exception stack traces
in HTTP responses. When set to false, the error_trace request parameter will result
in a HTTP 400 response. When the error_trace parameter is not present, the message
of the first ElasticsearchException will be output and no nested exception messages
will be output.
2015-03-18 17:49:05 -04:00
Joshua Rich db2caa54cd Small grammar fix. 2015-03-17 11:27:13 -07:00
Boaz Leskes b605184471 Recovery: add total operations to the `_recovery` API
This commit adds the current total number of translog operations to the recovery reporting API. We also expose the recovered / total percentage:

```
"translog": {
   "recovered": 536,
   "total": 986,
   "percent": "54.3%",
   "total_time": "2ms",
   "total_time_in_millis": 2
},
```

Closes #9368
Closes #10042
2015-03-17 07:31:29 -07:00
Martijn van Groningen 4393939f5e inner_hits: Nested parent field should be resolved based on the parent inner hit definition, instead of the nested parent field in the mapping.
The behaviour is better in the case someone has multiple levels of nested object fields defined in the mapping and like to define a single inner_hits definition that is two or more levels deep.

If someone wants inner hits on a nested field that is 2 levels deep the following would need to be defined:

```
{
  ...
  "inner_hits" : {
     "path" : {
        "level1" : {
            "inner_hits" : {
               "path" : {
                  "level2" : {
                     "query" : { .... }
                  }
               }
            }
        }
     }
  }
}
```

With this change the above can be defined as:

```
{
  ...
  "inner_hits" : {
     "path" : {
        "level1.level2" : {
            "query" : { .... }
        }
     }
  }
}
```

Closes #9251
2015-03-16 16:31:03 -07:00
Michael McCandless 0683a66277 Core: remove index.fail_on_merge_failure
Always fail the engine if an unexpected exception is hit during merge.

Closes #10088
2015-03-14 09:53:42 -04:00
Lee Hinman 6aec68cd29 Revert "[QUERY] Remove lowercase_expanded_terms and locale options"
This reverts commit d1f7bd97cb.

Ryan pointed out that this needs to work with the multi term query, so
additional analysis and tests should be added.
2015-03-13 13:51:44 -06:00
Lee Hinman d1f7bd97cb [QUERY] Remove lowercase_expanded_terms and locale options
The analysis chain should be used instead of relying on this, as it is
confusing when dealing with different per-field analysers.

The `locale` option was only used for `lowercase_expanded_terms`, which,
once removed, is no longer needed, so it was removed as well.

Fixes #9978
Relates to #9973
2015-03-13 13:17:27 -06:00
Petr Bela f27cb07eb9 [DOCS] fix typo in scripting docs 2015-03-12 15:28:50 -07:00
olivier bourgain bcb4decca9 [DOCS] add missing comma in percentile_rank aggregation example 2015-03-10 08:21:06 -07:00
olivier bourgain fb7cd2ea9a [DOCS] Adjusted geo_distance aggregation example
unit is not returned in the response, but we have key and an implicit from starting at 0 for the first bucket
2015-03-10 08:20:20 -07:00
olivier bourgain eaeddc6bd4 [DOCS] missing curly brace in ip_range aggregation example 2015-03-10 08:19:57 -07:00
David Pilato d9c19cd846 [doc] Cat API: show open and closed indices in _cat/indices
Related to #7936
2015-03-09 15:45:26 -07:00
Britta Weber 580728dfd6 significant terms: add scriptable significance heuristic
This commit adds scripting capability to significant_terms.
Custom heuristics can be implemented with a script that provides
parameters subset_freq, superset_freq,subset_size, superset_size.

closes #7850
2015-03-06 17:06:04 +01:00
Matias Tealdi cba6dff3ac fixing typo in expDecayFunction and adding offset to all dacay functions
closes #9887
2015-03-05 12:28:08 +01:00
Colin Goodheart-Smithe 5c44db50bc [DOCS] update JAVA API with aggregation changes
The Histogram and Range APIs for the aggregations changed so that there was a common interface between he types of Range/Histogram. This PR reflects that change in the Java API docs

Contributes to #9976
2015-03-05 11:09:49 +00:00
Clinton Gormley 3f9d4f9635 Update query-string-syntax.asciidoc
Closes #9965
2015-03-03 20:03:51 +01:00
Clinton Gormley 6a43ed8b28 Updated the resiliency status page for v1.4.0
Closes #9969
2015-03-03 19:50:13 +01:00
Martijn van Groningen b669e37c0b Docs: updated resilience page 2015-03-03 15:25:10 +01:00
Pius 430b091e7d Docs: Added default value
Added default value to `cluster.routing.allocation.node_initial_primaries_recoveries`

Closes #9955
2015-03-03 10:15:11 +01:00
Clinton Gormley c223ed0db4 Update search-type.asciidoc
Changed search_type docs to reflect that the `(dfs_)query_and_fetch` modes are an internal optimization and should not be specified explicitly by the user.

Relates to #9606
2015-03-02 10:55:22 +01:00
Pius a90e5c03b7 Update getting-started.asciidoc
Closes #9932
2015-03-01 21:07:09 +01:00
Pius a182c7428d Added note on max # of docs allowed in a shard 2015-03-01 21:07:07 +01:00
Geoff Bourne 0e09c02c56 Spelling out the sort order options
Closes #9768
2015-03-01 21:05:52 +01:00
Clinton Gormley 4bb7520e6d Docs: Minor test change to Perl docs 2015-02-28 17:19:55 +01:00
Clinton Gormley 8f73d95409 Update search.asciidoc
Removed operation threading from the docs. Not supported since v1.2.0

Closes #9917
2015-02-28 04:48:57 +01:00
Clinton Gormley 4d78026ef1 Updated copyright years to include 2015 2015-02-28 03:18:45 +01:00
cgp b1e6df3b6c Update span-multi-term-query.asciidoc
Added comma - there is no "term range" query

Closes #9855
2015-02-28 03:05:05 +01:00
Clinton Gormley e194fb3a07 Docs: Default distance unit in geo distance agg is metres, not km
Closes #9812
2015-02-28 01:45:29 +01:00
David Pilato 0c8da6bb84 [doc] Link mapper-attachment type documentation to its repo
As explained in elasticsearch/elasticsearch-mapper-attachments#101, we should have consistent documentation.

The best option is to link the  documentation in elasticsearch guide to the most recent README in the plugin repo.

Closes #9756
2015-02-27 22:18:59 +01:00
IsaacHaze ee163e570b Docs: Update snapshots.asciidoc
Adds more determiners.

Closes #9673
2015-02-27 20:42:12 +01:00
Ryan Ernst 9d708e20a0 Mappings: Lock down _size field
This also changes the stored setting for _size to true (for
indexes created in 2.x).

see #8143
closes #9913
2015-02-27 11:09:52 -08:00
Ryan Ernst 3b7928d568 Mappings: Lock down _field_names field
Now that we have an explicit `enabled` flag, we can lock down
the field type so it is not mungeable.

see #8143
closes #9912
2015-02-26 15:15:59 -08:00
Ryan Ernst 7181bbde26 Mappings: Remove _boost field
This has been deprecated since 1.0.0.RC1. It is finally removed here.

closes #8875
2015-02-26 15:07:07 -08:00
Ryan Ernst 78df69e6a0 Mappings: Lock down _routing field
`required` is now the only changeable settings (on indexes created after 1.x).

see #8143
closes #9895
2015-02-26 13:09:41 -08:00
Boaz Leskes e9dbfa9ee6 Transport: added a simple request tracer, logging incoming and outgoing requests
The request tracer logs in TRACE level under the `transport.tracer` log and is dynamically configurable with include and exclude arrays to filter out unneeded info. By default all requests are logged with the exception of fault detection pings (fired every second).

add the notion of tracers in the MockTransportService for testing purposes

Closes #9286
2015-02-25 21:33:57 +01:00
Ryan Ernst 32e042f1c4 Mappings: Lock down _index field
see #8143
closes #9870
2015-02-25 12:24:55 -08:00
Lee Hinman 2e9ea4abaf Add support for `minimum_should_match` to `simple_query_string`
This behaves similar to the way that `minimum_should_match` works for
the `match` query (in fact it is implemented in the exact same way)

Fixes #6449
2015-02-25 11:35:33 -07:00
Boaz Leskes 3e32dd985a Recovery: RecoveryState clean up
To support the `_recovery` API, the recovery process keeps track of current progress in a class called RecoveryState. This class currently have some issues, mostly around concurrency (see #6644 ). This PR cleans it up as well as other issues around it:

- Make the Index subsection API cleaner:
- remove redundant information - all calculation is done based on the underlying file map
- clearer definition of what is what: total files, vs reused files (local files that match the source) vs recovered files (copied over). % based progress is reported based on recovered files only.
- cleaned up json response to match other API (sadly this breaks the structure). We now properly report human values for dates and other units.
- Add more robust unit testing
- Detail flag was passed along as state (it's now a ToXContent param)
- State lookup during reporting is now always done via the IndexShard , no more fall backs to many other classes.
- Cleanup APIs around time and move the little computations to the state class as opposed to doing them out of the API

I also improved error messages out of the REST testing infra for things I run into.

Closes #6644
Closes #9811
2015-02-25 17:34:22 +01:00
Igor Motov c5ebdf11bb Snapshot/Restore: add ability to retrieve currently running snapshots
Together with #8782 it should help in the situations simliar to #8887 by adding an ability to get information about currently running snapshot without accessing the repository itself.

Closes #8887
2015-02-25 11:06:32 -05:00
Boaz Leskes 6953777c3a API: add pending tasks count to cluster health
The number of current pending tasks is useful to detect and overloaded master. This commit adds it to the cluster health API. The complete list can be retrieved from the dedicated pending tasks API.

It also adds rest tests for the cluster health variants.

Closes #9877
2015-02-25 14:58:44 +01:00
Clinton Gormley 5a53ff6f1b Update migrate_2_0.asciidoc
More code formatting in breaking changes
2015-02-25 14:13:25 +01:00
Clinton Gormley e805fe71cc Update migrate_2_0.asciidoc
Code formatting in breaking changes
2015-02-25 14:11:57 +01:00
Clinton Gormley 5146cf6256 Update migrate_2_0.asciidoc
Fixed bad heading levels in breaking changes
2015-02-25 14:10:17 +01:00
Clinton Gormley 0c61ea803d Update migrate_2_0.asciidoc
Fixed bad asciidoc in breaking changes
2015-02-25 14:07:19 +01:00
Colin Goodheart-Smithe 2520dc78ec [DOCS] added a note for the default shard_size value 2015-02-25 11:00:55 +00:00
Ryan Ernst be0cef0c43 Mappings: Lock down _type field
see #8143
closes #9869
2015-02-24 22:37:41 -08:00
Ryan Ernst b96bd201c1 Mappings: Lock down _id field
There are two implications to this change.
First, percolator now uses _uid internally, extracting the id portion
when needed. Second, sorting on _id is no longer possible, since you
can no longer index _id. However, _uid can still be used to sort, and
is better anyways as indexing _id just to make it available to
fielddata for sorting is wasteful.

see #8143
closes #9842
2015-02-24 14:26:22 -08:00
Michael Sander fd6c6058ce Remove Triple Negative!
Double negatives are confusing, but a triple negative (1 no, 2 non, 3 null)? It takes five minutes to understand this little sentence.  Cleaned that up a bit.

Closes #9789
2015-02-23 20:09:05 +01:00
Colin Goodheart-Smithe 2753db4685 Scripting: Removed deprecated script parameter names
This change removes the deprecated script parameter names ('file', 'id', and 'scriptField').
It also removes the ability to load file scripts using the 'script' parameter. File scripts should be loaded using the 'script_file' parameter only.
2015-02-23 13:49:21 +00:00
Colin Goodheart-Smithe 7d3856c9d3 [DOCS] update script docs to use preferred script parameter names 2015-02-23 11:16:28 +00:00
Robert Muir 1e015e6e33 Tests: Remove global shared cluster
This was previously attempted in #8854. I revived that branch and did
some performance testing as was suggested in the comments there.

I fixed all the errors, mostly just the rest tests, which
needed to have http enabled on the node settings (the global cluster
previously had this always enabled). I also addressed the comments from
that issue.

My performance tests involved running the entire test suite on my
desktop which has 6 cores, 16GB of ram, and nothing else was being
run on the box at the time. I ran each set of settings 3 times and
took the average time.

| mode    | master | patch | diff |
| ------- | ------ | ----- | ---- |
| local   | 409s   | 417s  | +2%  |
| network | 368s   | 380s  | +3%  |

This increase in average time is clearly worthwhile to pay to achieve
isolation of tests. One caveat is the way I fixed the rest tests
is still to have one cluster for the entire suite, so all the rest
tests can still potentially affect each other, but this is an
issue for another day.

There were some oddities that I noticed while running these tests
that I would like to point out, as they probably deserve some
investigation (but orthogonal to this PR):
* The total test run times are highly variable (more than a minute between the min and max)
* Running in network mode is on average actually *faster* than local mode. How is this possible!?
2015-02-22 22:04:22 -08:00
Martijn van Groningen daefb4c673 Docs: Document that the fielddata loading defaults to eager on the _parent field.
Closes #9804
2015-02-22 23:15:59 +01:00
markharwood 29b1902cfb New aggregations feature - “PercentageScore” heuristic for significant_terms aggregation provides simple “per-capita” type measures.
Closes #9720
2015-02-20 13:22:08 +00:00
Adrien Grand 4708227ecf Codecs: Remove the ability to have custom per-field postings and doc values formats.
This commit makes the `postings_format` and `doc_values_format` options of
mappings illegal on 2.0 and ignored on 1.x (meaning that the default postings
and doc values formats from the codec will be used in such a case).

This removes a fair amount of code.

Close #8746 #9741
2015-02-19 15:47:25 +01:00
Lee Hinman eb666f7f50 Add shadow replicas for shared filesystems
Squashed commit of the following:

commit 20835037c98e7d2fac4206c372717a05a27c4790
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 15:27:17 2015 -0700

    Use Enum for "_primary" preference

commit 325acbe4585179190a959ba3101ee63b99f1931a
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 14:32:41 2015 -0700

    Use ?preference=_primary automatically for realtime GET operations

commit edd49434af5de7e55928f27a1c9ed0fddb1fb133
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 14:32:06 2015 -0700

    Move engine creation into protected createNewEngine method

commit 67a797a9235d4aa376ff4af16f3944d907df4577
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 13:14:01 2015 -0700

    Factor out AssertingSearcher so it can be used by mock Engines

commit 62b0c28df8c23cc0b8205b33f7595c68ff940e2b
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 11:43:17 2015 -0700

    Use IndexMetaData.isIndexUsingShadowReplicas helper

commit 1a0d45629457578a60ae5bccbeba05acf5d79ddd
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:59:31 2015 -0700

    Rename usesSharedFilesystem -> isOnSharedFilesystem

commit 73c62df4fc7da8a5ed557620a83910d89b313aa1
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:58:02 2015 -0700

    Add MockShadowEngine and hook it up to be used

commit c8e8db473830fce1bdca3c4df80a685e782383bc
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:45:50 2015 -0700

    Clarify comment about pre-defined mappings

commit 60a4d5374af5262bd415f4ef40f635278ed12a03
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 09:18:22 2015 -0700

    Add a test for shadow replicas that uses field data

commit 7346f9f382f83a21cd2445b3386fe67472bc3184
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:37:14 2015 -0700

    Revert changes to RecoveryTarget.java

commit d90d6980c9b737bd8c0f4339613a5373b1645e95
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:35:44 2015 -0700

    Rename `ownsShard` to `canDeleteShardContent`

commit 23001af834d66278ac84d9a72c37b5d1f3a10a7b
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:35:25 2015 -0700

    Remove ShadowEngineFactory, add .newReadOnlyEngine method in EngineFactory

commit b64fef1d2c5e167713e869b22d388ff479252173
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 18 08:25:19 2015 -0700

    Add warning that predefined mappings should be used

commit a1b8b8cf0db49d1bd1aeb84e51491f7f0de43b59
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 14:31:50 2015 -0700

    Remove unused import and fix index creation example in docs

commit 0b1b852365ceafc0df86866ac3a4ffb6988b08e4
Merge: b9d1fed a22bd49
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 10:56:02 2015 -0700

    Merge remote-tracking branch 'refs/remotes/origin/master' into shadow-replicas

commit b9d1fed25ae472a9dce1904eb806702fba4d9786
Merge: 4473e63 41fd4d8
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 09:02:27 2015 -0700

    Merge remote-tracking branch 'refs/remotes/origin/master' into shadow-replicas

commit 4473e630460e2f0ca2a2e2478f3712f39a64c919
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 17 09:00:39 2015 -0700

    Add asciidoc documentation for shadow replicas

commit eb699c19f04965952ae45e2caf107124837c4654
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 16:15:39 2015 +0100

    remove last nocommit

commit c5ece6d16d423fbdd36f5d789bd8daa5724d77b0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 16:13:12 2015 +0100

    simplify shadow engine

commit 45cd34a12a442080477da3ef14ab2fe7947ea97e
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:32:57 2015 +0100

    fix tests

commit 744f228c192602a6737051571e040731d413ba8b
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:28:12 2015 +0100

    revert changes to IndexShardGateway - these are leftovers from previous iterations

commit 11886b7653dabc23655ec76d112f291301f98f4a
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:26:48 2015 +0100

    Back out non-shared FS code. this will go in in a second iteration

commit 77fba571f150a0ca7fb340603669522c3ed65363
Merge: e8ad614 2e3c6a9
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 11:16:46 2015 +0100

    Merge branch 'master' into shadow-replicas

    Conflicts:
    	src/main/java/org/elasticsearch/index/engine/Engine.java

commit e8ad61467304e6d175257e389b8406d2a6cf8dba
Merge: 48a700d 1b8d8da
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 10:54:20 2015 +0100

    Merge branch 'master' into shadow-replicas

commit 48a700d23cff117b8e4851d4008364f92b8272a0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 10:50:59 2015 +0100

    add test for failing shadow engine / remove nocommit

commit d77414c5e7b2cde830a8e3f70fe463ccc904d4d0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 17 10:27:56 2015 +0100

    remove nocommits in IndexMetaData

commit abb696563a9e418d3f842a790fcb832f91150be2
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 17:05:02 2015 +0100

    remove nocommit and simplify delete logic

commit 82b9f0449108cd4741568d9b4495bf6c10a5b019
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 16:45:27 2015 +0100

    reduce the changes compared to master

commit 28f069b6d99a65e285ac8c821e6a332a1d8eb315
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 16:43:46 2015 +0100

    fix primary relocation

commit c4c999dd61a44a7a0db9798275a622f2b85b1039
Merge: 2ae80f9 455a85d
Author: Simon Willnauer <simonw@apache.org>
Date:   Mon Feb 16 15:04:26 2015 +0100

    Merge branch 'master' into shadow-replicas

commit 2ae80f9689346f8fd346a0d3775a6341874d8bef
Author: Lee Hinman <lee@writequit.org>
Date:   Fri Feb 13 16:25:34 2015 -0700

    throw UnsupportedOperationException on write operations in ShadowEngine

commit 740c28dd9ef987bf56b670fa1a8bcc6de2845819
Merge: e5bc047 305ba33
Author: Lee Hinman <lee@writequit.org>
Date:   Fri Feb 13 15:38:39 2015 -0700

    Merge branch 'master' into shadow-replicas

commit e5bc047d7c872ae960d397b1ae7b4b78d6a1ea10
Author: Lee Hinman <lee@writequit.org>
Date:   Fri Feb 13 11:38:09 2015 -0700

    Don't replicate document request when using shadow replicas

commit 213292e0679d8ae1492ea11861178236f4abd8ea
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Feb 13 13:58:05 2015 +0100

    add one more nocommit

commit 83d171cf632f9b77cca9de58505f7db8fcda5599
Merge: aea9692 09eb8d1
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Feb 13 13:52:29 2015 +0100

    Merge branch 'master' into shadow-replicas

commit aea96920d995dacef294e48e719ba18f1ecf5860
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Feb 13 09:56:41 2015 +0100

    revert unneeded changes on Store

commit ea4e3e58dc6959a92c06d5990276268d586735f3
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 14:26:30 2015 -0700

    Add documentation to ShadowIndexShard, remove nocommit

commit 4f71c8d9f706a0c1c39aa3a370efb1604559d928
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 14:17:22 2015 -0700

    Add documentation to ShadowEngine

commit 28a9d1842722acba7ea69e0fa65200444532a30c
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 14:08:25 2015 -0700

    Remove nocommit, document canDeleteIndexContents

commit d8d59dbf6d0525cd823d97268d035820e5727ac9
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Feb 12 10:34:32 2015 -0700

    Refactor more shared methods into the abstract Engine

commit a7eb53c1e8b8fbfd9281b43ae39eacbe3cd1a0a6
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 17:38:59 2015 +0100

    Simplify shared filesystem recovery by using a dedicated recovery handler that skip
    most phases and enforces shard closing on the soruce before the target opens it's engine

commit a62b9a70adad87d7492c526f4daf868cb05018d9
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 15:59:54 2015 +0100

    fix compile error after upstream changes

commit abda7807bc3328a89fd783ca7ad8c6deac35f16f
Merge: f229719 35f6496
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 15:57:28 2015 +0100

    Merge branch 'master' into shadow-replicas

    Conflicts:
    	src/main/java/org/elasticsearch/index/engine/Engine.java

commit f2297199b7dd5d3f9f1f109d0ddf3dd83390b0d1
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 12:41:32 2015 +0100

    first cut at catchup from primary
    make flush to a refresh
    factor our ShadowIndexShard to have IndexShard be idential to the master and least intrusive

    cleanup abstractions

commit 4a367c07505b84b452807a58890f1cbe21711f27
Author: Simon Willnauer <simonw@apache.org>
Date:   Thu Feb 12 09:50:36 2015 +0100

    fix primary promotion

commit cf2fb807e7e243f1ad603a79bc9d5f31a499b769
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 16:45:41 2015 -0700

    Make assertPathHasBeenCleared recursive

commit 5689b7d2f84ca1c41e4459030af56cb9c0151eff
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:58:19 2015 -0700

    Add testShadowReplicaNaturalRelocation

commit fdbe4133537eaeb768747c2200cfc91878afeb97
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:28:57 2015 -0700

    Use check for shared filesystem in primary -> primary relocation

    Also adds a nocommit

commit 06e2eb4496762130af87ce68a47d360962091697
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:21:32 2015 -0700

    Add a test checking that indices with shadow replicas clean up after themselves

commit e4dbfb09a689b449f0edf6ee24222d7eaba2a215
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 15:08:18 2015 -0700

    Fix segment info for ShadowEngine, remove test nocommit

commit 80cf0e884c66eda7d59ac5d59235e1ce215af8f5
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 14:30:13 2015 -0700

    Remove nocommit in ShadowEngineTests#testFailStart()

commit 5e33eeaca971807b342f9be51a6a566eee005251
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 14:22:59 2015 -0700

    Remove overly-complex test

commit 2378fbb917b467e79c0262d7a41c23321bbeb147
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 13:45:44 2015 -0700

    Fix missing import

commit 52e9cd1b8334a5dd228d5d68bd03fd0040e9c8e9
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 13:45:05 2015 -0700

    Add a test for replica -> primary promotion

commit a95adbeded426d7f69f6ddc4cbd6712b6f6380b4
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 12:54:14 2015 -0700

    Remove tests that don't apply to ShadowEngine

commit 1896feda9de69e4f9cf774ef6748a5c50e953946
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 10:29:12 2015 -0700

    Add testShadowEngineIgnoresWriteOperations and testSearchResultRelease

commit 67d7df41eac5e10a1dd63ddb31de74e326e9d38b
Author: Lee Hinman <lee@writequit.org>
Date:   Wed Feb 11 10:06:05 2015 -0700

    Add start of ShadowEngine unit tests

commit ca9beb2d93d9b5af9aa6c75dbc0ead4ef57e220d
Merge: 2d42736 57a4646
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 18:03:53 2015 +0100

    Merge branch 'master' into shadow-replicas

commit 2d42736fed3ed8afda7e4aff10b65d292e1c6f92
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 17:51:22 2015 +0100

    shortcut recovery if we are on a shared FS - no need to compare files etc.

commit 24d36c92dd82adce650e7ac8e9f0b43c83b2dc53
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 17:08:08 2015 +0100

    utilize the new delete code

commit 2a2eed10f58825aae29ffe4cf01aefa5743a97c7
Merge: 343dc0b 173cfc1
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 16:07:41 2015 +0100

    Merge branch 'master' into shadow-replicas

    Conflicts:
    	src/main/java/org/elasticsearch/gateway/GatewayMetaState.java

commit 343dc0b527a7052acdc783ac5abcaad1ef78dbda
Author: Simon Willnauer <simonw@apache.org>
Date:   Wed Feb 11 16:05:28 2015 +0100

    long adder is not available in java7

commit be02cabfeebaea74b51b212957a2a466cfbfb716
Author: Lee Hinman <lee@writequit.org>
Date:   Tue Feb 10 22:04:24 2015 -0700

    Add test that restarts nodes to ensure shadow replicas recover

commit 7fcb373f0617050ca1a5a577b8cf32e32dc612b0
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 10 23:19:21 2015 +0100

    make test more evil

commit 38135af0c1991b88f168ece0efb72ffe9498ff59
Author: Simon Willnauer <simonw@apache.org>
Date:   Tue Feb 10 22:25:11 2015 +0100

    make tests pass

commit 05975af69e6db63cb95f3e40d25bfa7174e006ea
Author: Lee Hinman <lee@writequit.org>
Date:   Mon Jan 12 18:44:29 2015 +0100

    Add ShadowEngine
2015-02-18 15:34:06 -07:00
Christoph Büscher 30fd70f07b Aggregations: Simplify time zone option in `date_histogram`
Removed the existing `pre_zone` and `post_zone` option in `date_histogram` in favor of
the simpler `time_zone` option. Previously, specifying different values for these could
lead to confusing scenarios where ES would return bucket keys that are not UTC.
Now `time_zone` is the only option setting, the calculation of date buckets to take place in the
preferred time zone, but after rounding converting the bucket key values back to UTC.

Closes #9062
Closes #9637
2015-02-16 16:54:06 +01:00
Blake Niemyjski 8cba6c3abb Fixed an invalid query
Closes #9682
2015-02-13 21:11:42 +01:00
Ryan Ernst 533fdbdf75 Mappings: Remove support for field access by short name
When multiple fields under object fields share the same name, accessing
by short name is ambiguous.  This removes support for short names,
always requiring the full name when used in queries.

closes #8872
2015-02-12 09:58:37 -08:00
Andreas Kohn 01b8479179 Allow configuration of the GC log file via an environment variable
Enabling GC logging works now by setting the environment variable ES_GC_LOG_FILE
to the full path to the GC log file. Missing directories will be created as needed.

The ES_USE_GC_LOGGING environment variable is no longer used.

Closes #8471
Closes #8479
2015-02-12 17:07:57 +01:00
gseng d1deb6bd1e Update update-settings.asciidoc
Updating to the fields as mentioned on http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-fielddata.html

Closes #9657
2015-02-12 13:09:16 +01:00
Clinton Gormley 856b0fa1a0 Docs: Fixed explanation of how the query string query is rewritten 2015-02-12 12:46:44 +01:00
Clinton Gormley 20ece4acb5 Update core-types.asciidoc
Provide an example of how to disable norms

Closes #9641
2015-02-12 12:10:11 +01:00
Ryan Ernst f735baf306 Core: Remove ability to run optimize and upgrade async
This has been very trappy. Rather than continue to allow buggy behavior
of having upgrade/optimize requests sidestep the single shard per node
limits optimize is supposed to be subject to, this removes
the ability to run the upgrade/optimize async.

closes #9638
2015-02-11 11:30:27 -08:00
Clinton Gormley faae98c5d8 Updated latest version in docs 2015-02-11 19:25:10 +01:00
Clinton Gormley 57a4646776 Docs: Added note about groovy sandbox vulnerability to modules/scripting 2015-02-11 17:54:53 +01:00
Clinton Gormley 6fadeeca56 Updated doc annotations for 1.4.3 2015-02-11 17:54:53 +01:00
Ryan Ernst b3474f6b25 Mappings: Remove ability to set path for _id and _routing on 2.0+ indexes
_id and _routing now no longer support the 'path' setting on indexes
created with 2.0.  Indexes created before 2.0 still support this
setting for backcompat.

closes #6730
2015-02-10 10:53:44 -08:00
Alfredo Serafini e607e53591 Update span-multi-term-query.asciidoc
added wildcard to the list of possible nested queries

Closes #9586
2015-02-09 16:01:46 +01:00
Christoph Büscher d2f852a274 Aggregations: Add 'offset' option to date_histogram, replacing 'pre_offset' and 'post_offset'
Add offset option to 'date_histogram' replacing and simplifying the previous 'pre_offset' and 'post_offset' options.
This change is part of a larger clean up task for `date_histogram` from issue #9062.
2015-02-09 14:03:28 +01:00
Christoph Büscher dfc0496fc0 Add warning to settings documentation about setting number_of_replicas on a closed index
Issue #9566 raises the point that setting the number of shards on a closed index can lead to this index not beeing able to open again. This change in documentation is ment to warn the user about this issue.
2015-02-06 12:09:24 +01:00
Ryan Ernst c6968883a7 Mappings: Remove support for new indexes using path setting in
object/nested fields or index_name in any field

Backcompat is still here for indexes created before 2.0.

closes #6677
2015-02-05 12:44:43 -08:00
Adrien Grand 95f46f1212 Docs: Use the new experimental annotation.
We now have a very useful annotation to mark features or parameters as
experimental. Let's use it! This commit replaces some custom text warnings with
this annotation and adds this annotation to some existing features/parameters:
 - inner_hits (unreleased yet)
 - terminate_after (released in 1.4)
 - per-bucket doc count errors in the terms agg (released in 1.4)

I also tagged with this annotation settings which should either be not needed
(like the ability to evict entries from the filter cache based on time) or that
are too deep into the way that Elasticsearch works like the Directory
implementation or merge settings.

Close #9563
2015-02-05 15:29:45 +01:00
Adrien Grand 3a486066fd Docs: Remove the experimental status of the cardinality and percentiles(-ranks) aggregations
These aggregations are not experimental anymore but some of their parameters
still are:
 - `precision_threshold` and `rehash` on `cardinality`
 - `compression` on percentiles(-ranks)

Close #9560
2015-02-05 15:18:40 +01:00
Masaru Hasegawa b4f7d26723 Fielddata: Change threshold value of fielddata.filter.frequency.max/min
Make it consider 1.0 as 100% instead of aboslute count 1.

Closes: #9327
2015-02-05 13:27:42 +09:00
Adam 928ea82188 Docs: Updated documentation for query-string-syntax to include '>' '<' and '=' as reserved characters
Closes #9518
2015-02-04 17:55:15 +01:00
Simon Willnauer 0c5599e1d1 [ENGINE] Remove full flush / FlushType.NEW_WRITER
The `full` option and `FlushType.NEW_WRITER` only exists to allow
realtime changes to two settings (`index.codec` and `index.concurrency`).
Those settings are very expert and don't really need to be updateable
in realtime.
2015-02-04 17:38:05 +01:00
Robert Muir 027730006b core: add 'checksum' option for index.shard.check_on_startup
The current "checkindex" on startup is very very expensive. This is
like running one of the old school hard drive diagnostic checkers and
usually not a good idea.

But we can do a CRC32 verification of files. We don't even need to
open an indexreader to do this, its much more lightweight.

This option (as well as the existing true/false) are randomized in
tests to find problems.

Also fix bug where use of the current option would always leak
an indexwriter lock.

Closes #9183
2015-02-03 00:10:08 -05:00
Ryan Ernst 6079d88d43 Mappings: Remove type prefix support from field names in queries
This is the first part of #8872.
2015-02-02 13:10:56 -08:00
Christoph Büscher 44193e7ba5 Aggregations: Add 'offset' option to histogram aggregation
Histogram aggregation supports an 'offset' option to move bucket boundaries.
In a histogram with buckets of size X these can be moved from 0, X, 2X, 3X,...
by an offset value of Y to Y, X+Y, 2X+Y, 3X+Y... by using the 'offset' option.
The previous 'pre_offset' and 'post_offset' options are removed in favour of
the simplified 'offset' option.

Closes #9417
Closes #9505
2015-02-02 18:23:01 +01:00
Clinton Gormley eea22d7731 Docs: Fixed asciidoc error in snapshots.asciidoc 2015-01-29 20:57:12 +01:00
J Charitopoulos be8d8d658c Docs: minor syntax
Closes #9481
2015-01-29 20:27:20 +01:00
Glen Smith 3d5fbfb997 Docs: Update pattern-replace-charfilter.asciidoc
Remove invalid trailing comma from json

Closes #9477
2015-01-29 20:24:08 +01:00
David Pilato 878e46d7f9 [Docs] fix missing space 2015-01-29 19:17:41 +01:00
Oliver e412dab63a Docs: Fix sample query
Closes #9472
2015-01-29 15:56:24 +01:00
Ryan Ernst afcedb94ed Mappings: Remove `index_analyzer` setting to simplify analyzer logic
The `analyzer` setting is now the base setting, and `search_analyzer`
is simply an override of the search time analyzer.  When setting
`search_analyzer`, `analyzer` must be set.

closes #9371
2015-01-28 13:43:15 -08:00
Zachary Tong a4eb1d5505 Aggregations: Add standard deviation bounds to extended_stats
Extended_stats now displays the upper and lower bounds on standard deviations (e.g. avg +/- std).
Default is to show 2 std above/below, but can be changed using the `sigma` parameter.
Accepts non-negative doubles

Closes #9356
2015-01-28 11:47:20 -05:00
J Charitopoulos b359520849 Docs: Update snapshots.asciidoc
minor syntax

Closes #9457
2015-01-28 15:54:13 +01:00
Clinton Gormley 8978aa5465 Docs: Improved the template query docs
Added the `file` and `id` parameters.

Closes #9458
2015-01-28 14:19:59 +01:00
Lee Hinman 2f6527f491 [DOCS] Update documentation for `max_token_length`
In 1.4 the behavior is different due to
https://issues.apache.org/jira/browse/LUCENE-5897
2015-01-27 13:52:14 -07:00
Colin Goodheart-Smithe 285ef0f06d Aggregations: Clean up response API for Aggregations
This change makes the response API object for Histogram Aggregations the same for all types of Histogram, and does the same for all types of Ranges.
The change removes getBucketByKey() from all aggregations except filters and terms. It also reduces the methods on the Bucket class to just getKey() and getKeyAsString().
The getKey() method returns Object and the actual Type is returns will be appropriate for the type of aggregation being run. e.g. date_histogram will return a DateTime for this method and Histogram will return a Number.
2015-01-27 10:53:44 +00:00
Christian Verkerk 5b31189498 Docs: Update cluster.asciidoc
Clarify the preferencing.

Closes #9434
2015-01-27 10:48:40 +01:00
Aske Hansen 084dc7a656 Docs: added searchkick
Closes #9416
2015-01-26 21:58:36 +01:00
Ryan Ernst 385c43c141 Mappings: Remove _analyzer
closes #9279
2015-01-26 09:14:17 -08:00
Thomas Eschemann d3e10f9111 Docs: Add R client to docs
Closes #9376
2015-01-21 14:40:49 +01:00
jhtimmins 4aba382358 Docs: Change "There are few concepts" to "There are a few concepts"
Closes #8888
2015-01-21 10:33:33 +01:00
Igor Motov c0da353ef5 Snapshot/Restore: add support for changing index settings during restore process
Closes #7887
2015-01-20 15:49:47 -05:00