Commit Graph

30146 Commits

Author SHA1 Message Date
Simon Willnauer fe83838ec3 LUCENE-8341: Record soft deletes in SegmentCommitInfo
This change add the number of documents that are soft deletes but
not hard deleted to the segment commit info. This is the last step
towards making soft deletes as powerful as hard deltes since now the
number of document can be read from commit points without opening a
full blown reader. This also allows merge posliies to make decisions
without requiring an NRT reader to get the relevant statistics. This
change doesn't enforce any field to be used as soft deletes and the statistic
is maintained per segment.
2018-06-04 15:05:12 +02:00
Andrzej Bialecki 2ef3d07561 SOLR-11911: Fix a number of synchronization issues in the simulator. Enable this test for now. 2018-06-04 14:20:19 +02:00
Noble Paul 727da63f4a SOLR-12354: Register the /admin/info/key end-point at the startup time to avoid 404 2018-06-04 22:14:37 +10:00
Simon Willnauer e7a0a12926 LUCENE-8335: Enforce soft-deletes field up-front
Soft deletes field must be marked as such once it's introduced
and can't be changed after the fact.

Co-authored-by: Nhat Nguyen <nhat.nguyen@elastic.co>
2018-06-04 08:28:38 +02:00
Cao Manh Dat 7720d73074 Remove file that get accident committed from previous commit 2018-06-04 11:37:47 +07:00
Cao Manh Dat ab316bbc91 SOLR-9922: Write buffering updates to another tlog 2018-06-04 11:32:31 +07:00
Simon Willnauer 3dc4fa199c Revert "LUCENE-8335: Enforce soft-deletes field up-front."
This reverts commit a2d9276674.
2018-06-02 13:47:24 +02:00
Simon Willnauer a2d9276674 LUCENE-8335: Enforce soft-deletes field up-front.
Soft deletes field must be marked as such once it's introduced
and can't be changed after the fact.
2018-06-02 13:14:53 +02:00
markrmiller 1ff24bbb28 SOLR-12290,SOLR-12391: Do not close any servlet streams and improve our servlet stream closing prevention code for users and devs. 2018-06-02 00:16:28 -05:00
David Smiley 4888adf08a SOLR-12337: Remove the obsolete QueryWrapperFilter 2018-06-01 17:12:21 -04:00
Cao Manh Dat 1d33130fcb SOLR-12433: Recovering flag of a replica is set equals to leader even it failed to receive update on recovering 2018-06-01 09:23:10 +07:00
Steve Rowe 252a8145d9 SOLR-12388: Add an expected exception message to SearchHandlerTest.testRequireZkConnectedDistrib() 2018-05-31 16:09:11 -04:00
David Smiley ce8735556d * SOLR-12366: A slow "live docs" implementation was being used instead of a bitset.
Affects classic faceting enum method, JSON Facets enum method, UnInvertedField faceting, GraphTermsQParser, JoinQParser.
  Renamed SolrIndexSearcher.getLiveDocs to getLiveDocSet.
2018-05-31 10:59:26 -04:00
David Smiley 1e63b32731 * SOLR-12366: A slow "live docs" implementation was being used instead of a bitset.
Affects classic faceting enum method, JSON Facets enum method, UnInvertedField faceting, GraphTermsQParser, JoinQParser.
  Renamed SolrIndexSearcher.getLiveDocs to getLiveDocSet.
2018-05-31 10:55:12 -04:00
Noble Paul 12269abe34 SOLR-12387: cluster-wide defaults for numShards, nrtReplicas, tlogReplicas, pullReplicas
SOLR-12389: support deeply nested json objects in clusterprops.json
2018-06-01 00:50:52 +10:00
Simon Willnauer 76263087b5 LUCENE-8338: Ensure number returned for PendingDeletes are well defined
Today a call to PendingDeletes#numPendingDeletes might return 0
if the deletes are written to disk. This doesn't mean these values are committed
or refreshed in the latest reader. Some places in IW use these numbers to make
decisions if there has been deletes added since last time checked (BufferedUpdateStream)
which can cause wrong (while not fatal) decision ie. to kick of new merges.

Now this API is made protected and not visible outside of PendingDeletes to prevent
any kind of confusion. The APIs now allow to get absolute numbers of getDelCount and numDocs
which have the same name and semantics as their relatives on IndexReader/Writer
and SegmentCommitInfo.
2018-05-30 20:48:51 +02:00
Houston Putman d243f35a54 SOLR-12271: Updating changes.txt 2018-05-30 11:45:24 -04:00
Houston Putman 0ef8e5aa80 SOLR-12271: Fix for analytics component reading negative values from double and float fields. 2018-05-30 11:45:01 -04:00
Andrzej Bialecki 0e4512c231 SOLR-11779: Use fixed Locale for graph labels. 2018-05-30 17:37:33 +02:00
Alan Woodward e3d4c7e9b7 LUCENE-8300: Allow unordered intervals to exclude overlaps 2018-05-30 16:15:39 +01:00
David Smiley 9aa16b64c7 SOLR-12374: Added SolrCore.withSearcher(lambda) convenience.
* and fixed SnapShooter.getIndexCommit bug forgetting to decref (rare?)
2018-05-30 11:00:00 -04:00
yonik 107fd24ec7 SOLR-12417: enforce valid function name for v.json 2018-05-30 10:15:09 -04:00
Andrzej Bialecki d27a2e8996 SOLR-12392: Fix waitForElapsed logic and state restoration. Enable the test. 2018-05-30 15:44:19 +02:00
Jan Høydahl 6ca0c5f98a SOLR-12401: Add refGuide docs and CHANGES 2018-05-30 12:51:00 +02:00
Steve Rowe bd14de24e9 SOLR-12388: print out exception when assert fails 2018-05-30 00:39:26 -04:00
Cao Manh Dat 6084da559c SOLR-12338: Replay buffering tlog in parallel 2018-05-30 11:05:48 +07:00
Varun Thacker 8821b80616 fix typo in autoscaling trigger page 2018-05-29 18:49:24 -07:00
Karl Wright 3764406352 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/lucene-solr 2018-05-29 21:03:43 -04:00
Karl Wright ceb4f768bf LUCENE-8337: Fix problems with how travel planes too close to edge of world are disallowed, and increase the size of the disallowed window by an order of magnitude. 2018-05-29 21:03:03 -04:00
David Smiley 6e0da7e2f8 SOLR-12374 Add SolrCore.withSearcher(lambda accepting SolrIndexSearcher) 2018-05-29 16:27:11 -04:00
Andrzej Bialecki 64e4dda64e Don't restart Overseer when picking a node randomly. Enable this test for now. 2018-05-29 21:52:31 +02:00
Karl Wright 7ce6dbda18 LUCENE-8337: Add disabled unit test for failure 2018-05-29 12:42:56 -04:00
Alan Woodward 4ea9d2ea8c LUCENE-8273: Adjust position increments when filtering stacked tokens 2018-05-29 15:59:36 +01:00
Simon Willnauer 34741a863a LUCENE-8330: Exclude MockRandomMP from basic tests 2018-05-29 16:58:03 +02:00
Andrzej Bialecki 6bbce38b77 SOLR-11779: Basic long-term collection of aggregated metrics. 2018-05-29 10:35:39 +02:00
Varun Thacker 44015e2acd SOLR-11453: Configuring slowQueryThresholdMillis logs slow requests to a separate file - solr_slow_requests.log 2018-05-28 12:43:05 -07:00
Varun Thacker 071df6e114 SOLR-12314: Use http timeout's defined in solr.xml for creating ConcurrentUpdateSolrClient during indexing requests between leader and replica 2018-05-28 11:21:49 -07:00
Joel 11cfb86489 SOLR-12401: Add getValue() and setValue() Stream Evaluators 2018-05-28 13:29:26 -04:00
Yuri Astrakhan d039730638 Fix incorrect BitUtil.deinterleave() description
Signed-off-by: Adrien Grand <jpountz@gmail.com>

Closes #367
2018-05-28 17:10:47 +02:00
Adrien Grand 1971ef3109 LUCENE-8186: LowerCaseTokenizerFactory now lowercases text in multi-term queries. 2018-05-28 16:20:49 +02:00
Adrien Grand 78ca82e63a Add 6.6.4 back compat test indexes. 2018-05-28 16:11:32 +02:00
Adrien Grand f8ae144054 LUCENE-8324: Make test pass with ExtraFS. 2018-05-28 15:41:17 +02:00
Adrien Grand b1613e4da6 TestLRUQueryCache#testBulkScorerLocking needs large IW buffers. 2018-05-28 15:22:21 +02:00
Simon Willnauer 0941cae532 LUCENE-8334: Ensure SR#getSementInfo() returns snapshot
The SegmentCommitInfo passed to the segment reader is mutated concurrently.
An instance obtained from SR#getSegmentInfo() might return wrong delete counts
or generation ids. This ensures that the SR will use a clone internally while stil
maintaining the original SI since it's needed inside IW for maintainance like
accessing pooled readers etc.
2018-05-28 14:28:36 +02:00
Noble Paul 4e12546b02 SOLR-12294: changed the package of the test file 2018-05-28 15:04:19 +10:00
Cao Manh Dat 2a5346dd29 SOLR-12399: SearchRateTriggerTest.testTrigger fail frequently 2018-05-28 09:48:22 +07:00
yonik f7500a6029 SOLR-12328: domain change using graph 2018-05-27 21:02:15 -04:00
yonik e5998fcb5d SOLR-9685: #tagging in JSON query syntax 2018-05-27 16:22:16 -04:00
Erick Erickson fd929c1d60 SOLR-12028: BadApple and AwaitsFix annotations usage 2018-05-25 10:39:58 -07:00
Noble Paul 18ad8d137a SOLR-12294: update processors loaded from runtime jars fail to load if they are specified in an update processor chain 2018-05-26 00:25:39 +10:00