Commit Graph

14416 Commits

Author SHA1 Message Date
Michael McCandless 91a4425f79 LUCENE-3892: use vInt to store the fixed value for entire block
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1372357 13f79535-47bb-0310-9956-ffa450edef68
2012-08-13 10:54:21 +00:00
Michael McCandless 423354db48 LUCENE-3892: make BlockPostingsWriter package private
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1371944 13f79535-47bb-0310-9956-ffa450edef68
2012-08-11 13:10:25 +00:00
Michael McCandless 0db218838d LUCENE-3892: add assert
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1371770 13f79535-47bb-0310-9956-ffa450edef68
2012-08-10 16:57:45 +00:00
Michael McCandless 0df8e43b22 LUCENE-3892: resolve some of the nocommits...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1371762 13f79535-47bb-0310-9956-ffa450edef68
2012-08-10 16:29:44 +00:00
Michael McCandless 4a5496ac97 LUCENE-3892: replace Block with BlockPacked
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1371519 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 22:29:36 +00:00
Adrien Grand 90918ad2d6 LUCENE-3892: fix constant names, cache encoded sizes instead of formats and move readVIntBlock to the postings reader.
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1371184 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 12:52:24 +00:00
Adrien Grand 0be8bfb4fc LUCENE-3892: backporting r1371010 and r1371011 to BlockPacked.
I changed the comment on the value of BLOCK_SIZE. Indeed, since PackedInts
encoding/decoding is long-aligned, all numbers of bits per value that are
co-prime with 64 cannot encode/decode less than 64 values per iteration, so the
block size should be at least 64. Switching it to a lower value (eg. 32) should
work but will perform many unnecessary operations in the encoding/decoding step.


git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1371114 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 09:16:09 +00:00
Michael McCandless 98ae806ba6 add comment; use BLOCK_SIZE static import not blockSize
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1371011 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 23:02:58 +00:00
Michael McCandless 1ff9383a84 LUCENE-4283: further optimize scan-after-advance
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1371010 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 23:02:05 +00:00
Adrien Grand 207c0c6d13 LUCENE-3892: Backport of Mike's last changes + removal of unnecessary casts.
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370850 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 17:28:16 +00:00
Adrien Grand c8fb5b38d5 Fix test.
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370847 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 17:26:49 +00:00
Adrien Grand c06b36c762 LUCENE-3892: Use int[] arrays to buffer data instead of long[] arrays.
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370819 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 16:12:29 +00:00
Adrien Grand 5e948be185 LUCENE-3892: Ability to decode to int[] arrays (in addition to long[] arrays).
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370817 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 15:46:28 +00:00
Adrien Grand 8eaed3c865 LUCENE-3892: Ability to select the right format based on an `acceptableOverheadRatio`.
The `acceptableOverheadRatio` is currenlty configurable through
BlockPackedPostingsWriter's constructor and defaults to PackedInts.DEFAULT.


git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370781 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 14:15:22 +00:00
Adrien Grand 482bd77c51 LUCENE-3892: add the all-values-same case back.
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370710 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 09:08:16 +00:00
Adrien Grand 0db24114ec LUCENE-3892: byte[] decoding for PACKED_SINGLE_BLOCK format.
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370582 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 22:54:28 +00:00
Michael McCandless eab777c3d3 LUCENE-4283: check next skip point to see if it's worth invoking skipper
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370534 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 21:26:14 +00:00
Michael McCandless 9d7cb7de69 LUCENE-3892: remove nocommit
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370533 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 21:25:45 +00:00
Michael McCandless ae8787358a remove 32 bit case (all our int values are >= 0)
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370330 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 16:01:27 +00:00
Michael McCandless 36e9b06bd6 carry back some code improvements from BlockPacked -> Block
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370328 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 15:57:58 +00:00
Adrien Grand 378cbd4523 LUCENE-3892: performance improvements to the 'BlockPacked' codec.
- backport of Mike's changes to the 'Block' codec,
 - revert of oal.util.packed.BulkOperation in order not to use java.nio.*Buffer,
 - add direct byte[] decoding and encoding to PackedInts.Decoder and PackedInts.Encoder.


git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370267 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 13:56:30 +00:00
Michael McCandless 844ffc433f LUCENE-3892: add nocommit
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370207 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 12:36:08 +00:00
Michael McCandless a858470de1 LUCENE-3892: only store numBits in the block header; compute numBytes @ read time
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370204 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 12:25:07 +00:00
Michael McCandless 7a0cdacb6e LUCENE-3892: fix incorrect numBits computation
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370194 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 11:45:54 +00:00
Michael McCandless 51987642c2 LUCENE-3892: up skipMultiplier to 8 (vs 4): it seems to help a bit
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370180 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 11:16:01 +00:00
Michael McCandless b18e428f41 remove fixed nocommit
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370179 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 11:15:11 +00:00
Michael McCandless 4de0089844 LUCENE-3892: decouple level 0 skipInterval from level 1+
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1370055 13f79535-47bb-0310-9956-ffa450edef68
2012-08-06 22:57:00 +00:00
Michael McCandless 7bf0d1f804 LUCENE-3892: inline nextDoc() for DocsAndPositionsEnum.advance
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1369885 13f79535-47bb-0310-9956-ffa450edef68
2012-08-06 17:25:00 +00:00
Michael McCandless d4c27acb1a LUCENE-3892: inline scanning after DocsEnum.advance
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1369703 13f79535-47bb-0310-9956-ffa450edef68
2012-08-05 23:33:00 +00:00
Michael McCandless de9586104c LUCENE-3892: some cleanup/refactoring
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1369697 13f79535-47bb-0310-9956-ffa450edef68
2012-08-05 22:39:32 +00:00
Michael McCandless aed1b5d760 LUCENE-3982: trunk upgrade
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/pforcodec_3892@1369470 13f79535-47bb-0310-9956-ffa450edef68
2012-08-04 21:26:10 +00:00
Mark Robert Miller de3d41432c rename CloudState to ClusterState
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369396 13f79535-47bb-0310-9956-ffa450edef68
2012-08-04 18:32:33 +00:00
Mark Robert Miller 96f7135cea retry on 503 along with 404 in this test
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369367 13f79535-47bb-0310-9956-ffa450edef68
2012-08-04 16:48:15 +00:00
Michael McCandless 100349a03e fixup more python3 issues
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369243 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 22:48:27 +00:00
Mark Robert Miller 854eabc534 LUCENE-3985: missed updating test file
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369241 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 22:47:24 +00:00
Robert Muir 120a816d0c expect licenses in the binary dist
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369240 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 22:45:52 +00:00
Mark Robert Miller 7add2f7c62 LUCENE-3985: add close methods to Overseer and ConnectionManager and use them
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369236 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 22:34:08 +00:00
Robert Muir 034207be25 clean up binary patterns for consolidated licenses/
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369232 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 22:11:07 +00:00
Robert Muir 4e43511bad exclude tools as it contains a lib jar and a clover README and sneaks its way into binary packaging
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369227 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 22:03:00 +00:00
Michael McCandless e76a86110b fixup some python3 issues
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369223 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 21:49:24 +00:00
Robert Muir bc0a8e69d8 initial cutover to python3 (this script works with neither 3 or nor 2 anymore, and still doesnt)
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369216 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 21:18:14 +00:00
Mark Robert Miller 917fe18679 put this optimize back in - we need to make sure the deletes are cleared
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369207 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 20:56:45 +00:00
Robert Muir 60deff559f remove dead code (same as superclass impl)
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369205 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 20:53:53 +00:00
Robert Muir 8f726e254b LUCENE-3616: throw exception on some illegal field configurations (applying boost to omitNorms or unindexed field), fix TextField(Reader) ctor as it cannot Store
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369196 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 20:26:13 +00:00
Mark Robert Miller 66dac680fe assume that sometimes the zk state can be stale vs our watcher based state when checking our state matches zk state
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369189 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 20:03:16 +00:00
Mark Robert Miller cd7c8221f7 logging tweaks
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369106 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 17:31:46 +00:00
Mark Robert Miller 92c1125331 add logging
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369103 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 17:18:00 +00:00
Mark Robert Miller 577b895dd4 add some logging to recovery process
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369097 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 17:11:29 +00:00
Mark Robert Miller e3383eba7d no need to do rm's here
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369073 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 16:13:57 +00:00
Mark Robert Miller b8366ce1dd some script update/fixes
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1369069 13f79535-47bb-0310-9956-ffa450edef68
2012-08-03 16:11:00 +00:00