lucene/modules/benchmark
Uwe Schindler bdaa02c3c0 LUCENE-2953: PriorityQueue's internal heap was made private final
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1079707 13f79535-47bb-0310-9956-ffa450edef68
2011-03-09 09:18:56 +00:00
..
conf LUCENE-2897: apply delete-by-term on flushed segment while we flush (still buffer delete-by-terms for past segments) 2011-01-31 23:35:02 +00:00
lib add xyz-LICENSE.txt for all third party jars 2011-03-06 16:50:22 +00:00
scripts LUCENE-2845: move contrib/benchmark to modules/benchmark 2011-01-04 12:28:10 +00:00
src LUCENE-2953: PriorityQueue's internal heap was made private final 2011-03-09 09:18:56 +00:00
.rsync-filter LUCENE-2845: move contrib/benchmark to modules/benchmark 2011-01-04 12:28:10 +00:00
CHANGES.txt LUCENE-1540: Improvements to contrib.benchmark for TREC collections - port/merge from 3x. 2011-02-05 00:35:09 +00:00
LICENSE.txt add missing LICENSE/NOTICE to benchmarks module 2011-03-06 20:48:10 +00:00
NOTICE.txt add missing LICENSE/NOTICE to benchmarks module 2011-03-06 20:48:10 +00:00
README.enwiki LUCENE-2845: move contrib/benchmark to modules/benchmark 2011-01-04 12:28:10 +00:00
build.xml LUCENE-2923: Restore benchmark jar production for generate-maven-artifacts target 2011-02-19 17:06:01 +00:00
sortBench.py LUCENE-2845: move contrib/benchmark to modules/benchmark 2011-01-04 12:28:10 +00:00

README.enwiki

Support exists for downloading, parsing, and loading the English
version of wikipedia (enwiki).

The build file can automatically try to download the most current
enwiki dataset (pages-articles.xml.bz2) from the "latest" directory,
http://download.wikimedia.org/enwiki/latest/. However, this file
doesn't always exist, depending on where wikipedia is in the dump
process and whether prior dumps have succeeded. If this file doesn't
exist, you can sometimes find an older or in progress version by
looking in the dated directories under
http://download.wikimedia.org/enwiki/. For example, as of this
writing, there is a page file in
http://download.wikimedia.org/enwiki/20070402/. You can download this
file manually and put it in temp. Note that the file you download will
probably have the date in the name, e.g.,
http://download.wikimedia.org/enwiki/20070402/enwiki-20070402-pages-articles.xml.bz2. When
you put it in temp, rename it to enwiki-latest-pages-articles.xml.bz2.

After that, ant enwiki should process the data set and run a load
test. Ant targets get-enwiki, expand-enwiki, and extract-enwiki can
also be used to download, decompress, and extract (to individual files
in work/enwiki) the dataset, respectively.