squash merge jira/solr-10290 into master

Squashed commit of the following:

commit babe763e9d0f9561622171a45ab78607955c5dab
Merge: 5a4d757 69783f6
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed May 10 13:26:09 2017 -0700

    Merge branch 'master' into jira/solr-10290

    Conflicts:
    	lucene/ivy-versions.properties
    	solr/CHANGES.txt
    	solr/core/src/java/org/apache/solr/handler/admin/CollectionHandlerApi.java
    	solr/licenses/android-json-LICENSE-ASL.txt
    	solr/licenses/asciidoctor-ant-LICENSE-ASL.txt
    	solr/solr-ref-guide/src/fonts/Noto_Sans/LICENSE.txt

commit 5a4d7579c7d7b84263fd96e2f0dbe0c9db2f0ca0
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed May 10 12:08:30 2017 -0700

    ref guide tools: replace org.json with (not quite) drop in clean room replacement from android, and add all missting licenes and checksums

commit 4570315c38a11cf18a336bb68f5ba2e9195e53e8
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed May 10 10:36:35 2017 -0700

    license headers for tools

commit bd3d1cfad6579fbcdca76d9a98ce6362174dcce7
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed May 10 10:32:08 2017 -0700

    fix solr 'resolve' target to also resolve ref guide

commit f132c9634ddc6423a2760f3df602b4add6052de8
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed May 10 10:22:44 2017 -0700

    Fix some ivy related precommit issues

commit 2210941839329422fa1e1de09f799678fc219cbc
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed May 10 10:09:38 2017 -0700

    remove old nocommit debugging code (was tired that night)

commit 4c80ac15abbae01b5adde2119dba450ae62f6698
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed May 10 10:08:28 2017 -0700

    tabs to spaces

commit da77243dd315c63242ac4f986c85f348ea302baa
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed May 10 09:47:02 2017 -0700

    fix broken anchors

commit 7856fdf7a1ffb8ce038f3a40e8057f9f62dc89ec
Author: Cassandra Targett <cassandra.targett@lucidworks.com>
Date:   Wed May 10 09:43:19 2017 -0500

    SOLR-10290: make page TOC appear on page load

commit 39fdd15700de7c3a23f332b11fd64b1393b37895
Author: Cassandra Targett <cassandra.targett@lucidworks.com>
Date:   Wed May 10 09:42:45 2017 -0500

    SOLR-10290: remove unused usermaps and tab formats

commit fb86e76d1a8caa6051a1a066496897abbe3e6830
Author: Cassandra Targett <cassandra.targett@lucidworks.com>
Date:   Wed May 10 08:38:57 2017 -0500

    Add back json and jsoup dependencies

commit 3d036dec3cd1e18e9048d1832f84207e625dd752
Author: Cassandra Targett <cassandra.targett@lucidworks.com>
Date:   Wed May 10 08:01:57 2017 -0500

    Update comment for asciidoctor-ant

commit ffdc0b4511b62642a01f9df16e1806199f17b818
Author: Alan Woodward <romseygeek@apache.org>
Date:   Wed May 10 08:29:55 2017 +0100

    LUCENE-7741: Add explain() to DoubleValuesSource

commit 370adaa6f1d033e94e051b50a9633a5e8737ea69
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Tue May 9 15:18:22 2017 +0100

    SOLR-10535: remove six unused test files
    (Jason Gerlowski via Christine Poerschke)

commit 017da204ed18dfbe4bd28b77d484a1a4b071b2d9
Author: Andrzej Bialecki <ab@apache.org>
Date:   Tue May 9 12:53:18 2017 +0200

    SOLR-10262: Add support for configurable metrics implementations.

commit dea9c334fc597953dcdff31ce6df73d584e744ec
Author: Jan Høydahl <janhoy@apache.org>
Date:   Tue May 9 13:03:18 2017 +0200

    SOLR-10644: solr.in.sh installed by install script should be writable by solr user

commit 788b696cdc6f89479e200189b29646d293f83094
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Tue May 9 12:42:41 2017 +0530

    SOLR-8440: Support for enabling basic authentication using bin/solr|bin/solr.cmd

      Usage:
        bin/solr auth -enable -prompt
        bin/solr auth -enable -credentials solr:SolrRocks
        bin/solr auth -disable

commit fba5c76fc2e830c44a1078ee492d3932e31b80dc
Author: Cao Manh Dat <datcm@apache.org>
Date:   Tue May 9 13:56:49 2017 +0700

    SOLR-10619: Optimize using cache for DistributedQueue in case of single-consumer

commit 1a8bdc53dd733d18391251029d99209f1e7269fe
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon May 8 22:21:10 2017 -0400

    SOLR-10638: Update CHANGES.txt

commit 33c2ffd91ed8bde2c1160e3642bcc7ffb227f442
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon May 8 22:04:51 2017 -0400

    SOLR-10638: Add normalize Stream Evaluator

commit b4646846e573bfeb53218dcb1944c82f3ea0abb0
Author: Scott Blum <dragonsinth@apache.org>
Date:   Mon May 8 20:32:38 2017 -0400

    SOLR-10524: Build fix for NPE

    Introduced by ZkStateWriter batching optimizations.

commit 5a253a428607ae1153852c9343e7871317d8789c
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Mon May 8 17:30:22 2017 -0700

    SOLR-10639: Removed entry from CHANGES.txt

commit 46e6133540bfb018da9f463bc970b041dc9f1b25
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Mon May 8 17:11:35 2017 -0700

    SOLR-10639: Fix NPE in LRU/LFU/FastLRU caches toString method

commit 4b116a96f3efd40b62b9e400e52df4d25972883f
Author: jdyer1 <jdyer@apache.org>
Date:   Mon May 8 13:28:55 2017 -0500

    SOLR-10522: Revert "SOLR-9972: SpellCheckComponent collations and suggestions returned as a JSON object rather than a list"

commit fd626bf0f17e1812bbded71e74446c1a7977fb02
Author: jdyer1 <jdyer@apache.org>
Date:   Mon May 8 13:26:42 2017 -0500

    SOLR-10522: Revert "SOLR-9972: SpellCheckComponent collations and suggestions returned as a JSON object rather than a list"

    This reverts commit 4cd3d15da8.

commit 42e4ea69c6d59bd7b36495149743af486129b42f
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon May 8 10:19:41 2017 -0400

    SOLR-10625: Update CHANGES.txt

commit 8675ced8e36eb2fc15edc0aa4dae34caec26d933
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon May 8 09:53:12 2017 -0400

    SOLR-10625: Add convolution Stream Evaluator

commit f830eff0c03e61b2608f8b9d12593b04ace8262f
Author: Shalin Shekhar Mangar <shalin@apache.org>
Date:   Mon May 8 16:11:01 2017 +0530

    Fixed nsToMs calculation in OverseerTest

commit 65d5ead9b0a443444529c5b0047b585353233e17
Author: Shalin Shekhar Mangar <shalin@apache.org>
Date:   Mon May 8 15:50:10 2017 +0530

    SOLR-10630: HttpSolrCall.getAuthCtx().new AuthorizationContext() {...}.getParams() sometimes throws java.lang.NullPointerException

commit 1df6d3b5e07edbe737473c72a1301b2859481671
Author: Cao Manh Dat <datcm@apache.org>
Date:   Mon May 8 16:21:19 2017 +0700

    SOLR-10524: Explore in-memory partitioning for processing Overseer queue messages

commit f627c9fbaf225684401fe2166c454f91999266b7
Author: Jan Høydahl <janhoy@apache.org>
Date:   Mon May 8 10:48:20 2017 +0200

    SOLR-8149: Admin UI - Plugins / Stats - active item is now highlighted

commit 09a7cba99c178636dc57c5ffc59c08c81b6504e0
Author: Joel Bernstein <jbernste@apache.org>
Date:   Sun May 7 21:02:41 2017 -0400

    SOLR-10626: Update CHANGES.txt

commit 0051bacaa35e05515c33b92c58828da3ea382d13
Author: Joel Bernstein <jbernste@apache.org>
Date:   Sun May 7 20:42:39 2017 -0400

    SOLR-10626: Add covariance Stream Evaluator

commit 07b707f4a2d153266a771dc9c5d3571db9f7628d
Author: Joel Bernstein <jbernste@apache.org>
Date:   Sun May 7 16:26:53 2017 -0400

    SOLR-10622: Update CHANGES.txt

commit c2a68d152534a8bef37f65cddc696e7983691821
Author: Joel Bernstein <jbernste@apache.org>
Date:   Sun May 7 15:23:56 2017 -0400

    SOLR-10622: Add regress and predict Stream Evaluators

commit a95438e851570c37ca5652e9ce41e57eb5aaa0fd
Author: Uwe Schindler <uschindler@apache.org>
Date:   Sun May 7 13:24:21 2017 +0200

    LUCENE-5365, LUCENE-7818: Fix incorrect condition in queryparser's QueryNodeOperation#logicalAnd()

commit 6de19d0f48fb1413a46379284928da8b35f0dd85
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Sat May 6 10:44:57 2017 +0300

    SOLR-10614: remove static backdoor fields from SimplePostTool.
    Enabling testTechproductsExample

commit 204b54b0ad15678a656c4544b52846ea69932153
Author: Joel Bernstein <jbernste@apache.org>
Date:   Sat May 6 12:53:29 2017 -0400

    SOLR-10601: Update CHANGES.txt

commit 264ec0e791c847717e41983fad2efa2f4a7851c3
Author: Joel Bernstein <jbernste@apache.org>
Date:   Sat May 6 12:45:53 2017 -0400

    SOLR-10536: Update CHANGES.txt

commit 1f6e30fef088c1629ad35ad149c0a655f105a414
Author: yonik <yonik@apache.org>
Date:   Sat May 6 05:49:19 2017 -0400

    SOLR-10547: consolidate MinAgg+MaxAgg, add min/max support for single valued string fields

commit 93962273a9fa79c9714edeb45df9cdb0f7e744e6
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Sat May 6 00:36:00 2017 +0300

    SOLR-10615: latching SDF.doFilter() on init(); respond 404 instead of 500 in case of init failures or corecontainer shutdown.

commit d22431b2bf782cbf11d1605e8f45b17f4b08643a
Author: Noble Paul <noble@apache.org>
Date:   Sat May 6 07:54:22 2017 +0930

    SOLR-9530: addressing test failures with seed 9F9128B8E3E8FAA7

commit 3da09c91c6729d5e9a349e831608e5d59c0de8e9
Author: Noble Paul <noble@apache.org>
Date:   Sat May 6 06:00:54 2017 +0930

    SOLR-9530: fixing test error

commit dd93e0f0ac07e3609e75e8fdef08cafb270dec38
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri May 5 14:45:11 2017 -0400

    SOLR-10582: Update CHANGES.txt

commit 45a7998d0d79a7e230b0c9fe4f8345b94529603c
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri May 5 14:36:32 2017 -0400

    SOLR-10559: Update CHANGES.txt

commit 717da83895dc23ef7c7c291404eb54463318241c
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri May 5 14:27:46 2017 -0400

    SOLR-10566: Update CHANGES.txt

commit 181d2821b1d2eed28172e748fc39415c679ada9d
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Thu May 4 18:20:52 2017 +0100

    SOLR-10572: Removed three "no longer supported in solrconfig.xml" asserts.

commit 338bdf27fb021a5b34b538d0ed65db9a9d774fb2
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri May 5 13:57:03 2017 -0400

    SOLR-10516: Update CHANGES.txt

commit 15f2014bfd2899f404579fb4c9d4738755399fe5
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri May 5 13:48:47 2017 -0400

    SOLR-10504: Update CHANGES.txt

commit 2cf6c76b58dca10d6f77035f2c74a429991fc493
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri May 5 13:40:02 2017 -0400

    SOLR-10274: Update CHANGES.txt

commit 4d6a8d92447d97c74171edf9f21df304bc4e6b3c
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri May 5 13:32:33 2017 -0400

    SOLR-10426: Add shuffle Streaming Expression

commit 9bf20444feb13c81047726c5f18cdb0145fd5beb
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri May 5 13:24:59 2017 -0400

    SOLR-10351: Update CHANGES.txt

commit ec687156c6ebe1c58654ff0f29061bf02adad79d
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri May 5 13:14:15 2017 -0400

    SOLR-10303: Update CHANGES.txt

commit 71f39e747b996bb734cbe4984ce651de59768d8d
Author: yonik <yonik@apache.org>
Date:   Thu May 4 23:50:50 2017 -0400

    tests: test reset for other bucket aggregations

commit 1ad2f3932b81ddacce428dc4f9edd17e57a19218
Author: Noble Paul <noble@apache.org>
Date:   Fri May 5 10:47:57 2017 +0930

    SOLR-9530: An Update Processor to convert normal update operation to an atomic operations such as add, set,inc, remove ,set, removeregex

commit d99c9e64270f3e8510a4e6fd86d25482b5d4b3c3
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Fri May 5 02:17:44 2017 +0300

    SOLR-9867: @Ignore TestSolrCLIRunExample.testTechproductsExample()

commit b2495028201a7cae9c4d0c1edf903ddeda5a2e2b
Author: Jan Høydahl <janhoy@apache.org>
Date:   Fri May 5 00:53:50 2017 +0200

    SOLR-7041: Cut over tests from <defaultSearchField> in schema to df on requests

commit f945c8608fe3d338d610fd573c0b1b6ad82d7610
Author: Mike McCandless <mikemccand@apache.org>
Date:   Thu May 4 15:16:05 2017 -0400

    LUCENE-7811: add concurrent SortedSet facets implementation

commit 60b27234bcf369b2c3e92137353b021119927a3c
Author: Erick Erickson <erick@apache.org>
Date:   Thu May 4 08:10:30 2017 -0700

    SOLR-10607: Improve RTimerTree documentation

commit 14e3451ebae14a50a419de37cbb42111ef43e798
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Thu May 4 17:18:24 2017 +0300

    SOLR-9867: fixing JvmMetricsTest broken earlier, bring back testTechproductsExample()
    and single SDF.cores assignment.

commit 76fc383911c83f5166ee5f2ae2dcace63efd2c10
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Thu May 4 16:19:14 2017 +0300

    SOLR-9867: rollback SDF.createCoreContainer(). disable testTechproductsExample

commit e6f7dd4ed90b089b963caa90437c7a431e4b170d
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Thu May 4 15:45:14 2017 +0300

    SOLR-9867: make sure cores are assigned in the end of SolrDispatchFilter.createCoreContainer() only

commit 487a804dcdf8b78f7f5a75eab8c94a0057c99c75
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Sat Apr 29 00:25:28 2017 +0300

    SOLR-9867: fixing TestSolrCLIRunExample.testTechproductsExample

    - SolrDispatchFilter.doFilter rejects invocation until init() is completed.
    - introducing isLoaded=false, isLoading=true core status
    - blocking shutdown until core loading stops
    - looping run example tool while core is loading 1 min max.

commit faa140d2ae46a04097b78fef8ab9cfaab77792ad
Author: Noble Paul <noble@apache.org>
Date:   Thu May 4 15:13:01 2017 +0930

    added extra check if it is a liveNode

commit d0fdfb1e6036b49a4c1de68e3f806a59f4d74154
Author: yonik <yonik@apache.org>
Date:   Wed May 3 23:04:33 2017 -0400

    SOLR-10596: fix unique/hll docvalue iterator reuse

commit 018801ad393e4c09ea9cfcd73873a9622f4ec702
Author: David Smiley <dsmiley@apache.org>
Date:   Wed May 3 15:50:34 2017 -0400

    LUCENE-7814: DateRangePrefixTree bug in years >= 292M

commit 3231c205722e0a7a49aa24c38100153d30dfdca4
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed May 3 10:30:02 2017 -0700

    SOLR-10583: JSON Faceting now supports a query time 'join' domain change option

commit 6a6063c6fbf4785043da0900321fbbce6f569787
Author: Joel Bernstein <jbernste@apache.org>
Date:   Wed May 3 10:25:30 2017 -0400

    SOLR-10601: StreamExpressionParser should handle white space around = in named parameters

commit 2b4835f24ff12f1b1b4bdcd9b5cab36fbcf75f3f
Author: David Smiley <dsmiley@apache.org>
Date:   Wed May 3 10:07:12 2017 -0400

    * SOLR-10549: (typo fix in CHANGES.txt)

commit f0b15e198146cc97df23420741564e10ce5924c5
Author: David Smiley <dsmiley@apache.org>
Date:   Wed May 3 09:30:34 2017 -0400

    * SOLR-10549: The new 'large' attribute had been forgotten in /schema/fields?showDefaults=true (David Smiley)

commit b3dc62dc4c1b2d849d9f29f73ccdb173f31d9ee0
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Sun Apr 30 23:37:51 2017 +0300

    SOLR-10588: Prevent redundant double core reload on config update.

commit fd874be6497d97b18311b823f1407174736dcd22
Author: Joel Bernstein <jbernste@apache.org>
Date:   Tue May 2 15:44:15 2017 -0400

    SOLR-10536: stats Streaming Expression should work in non-SolrCloud mode

commit f94865ed36f2b1062615c28bca47ca0be8d65ef0
Author: Erik Hatcher <ehatcher@apache.org>
Date:   Tue May 2 19:41:06 2017 -0400

    SOLR-1485: improve TokenStream API usage

commit 858a9bb70b8e0a27af11accb65528883e9752978
Author: Erick Erickson <erick@apache.org>
Date:   Tue May 2 13:58:47 2017 -0700

    SOLR-10519: SolrCLI.atPath cannot handle children that begin with a slash.

commit e9431219e402b9726ee7b7cc14a2e828554bdf9b
Author: Mark Miller <markrmiller@apache.org>
Date:   Tue May 2 15:31:57 2017 -0300

    SOLR-10316: Unloading a core can remove a ZK SolrCore registration entry for the wrong SolrCore.

commit 0418ea5b7a04e103d06abfeffd79bee4cf49ad24
Author: Mark Miller <markrmiller@apache.org>
Date:   Tue May 2 13:38:44 2017 -0300

    SOLR-10430: Add ls command to ZkCLI for listing only sub-directories.

commit 064172798acd2cc54bb9856a42118aade321786c
Author: Erik Hatcher <ehatcher@apache.org>
Date:   Tue May 2 10:13:24 2017 -0400

    SOLR-1485: remove unused import

commit c25560d6f36fd403aecf7614c90a0cd418f489bd
Author: Erik Hatcher <ehatcher@apache.org>
Date:   Tue May 2 07:50:12 2017 -0400

    SOLR-1485: fix tests, removing unnecessary tie to Similarity in PayloadDecoder

commit dd5da77c1ca89496daaafece9cb139932b957863
Author: Erik Hatcher <ehatcher@apache.org>
Date:   Mon May 1 21:35:29 2017 -0400

    SOLR-1485: Add payload support

commit 09fff4da5916f30a624d784bce55c0266716a0e3
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon May 1 12:32:37 2017 -0400

    SOLR-10559: Fix TupStream to respect field order

commit 235f303f9c33ef089dd2d08842012844afa995c2
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon May 1 12:06:00 2017 -0400

    SOLR-10566: Fix error handling

commit 1d20b518bf4a68405e909db66d49df7cfd9a40a5
Author: Steve Rowe <sarowe@apache.org>
Date:   Mon May 1 10:26:41 2017 -0400

    SOLR-9386: Move default clientPort specification to before calling QuorumPeerConfig.parseProperties(), which requires that clientPort be specified.

commit f890fbf106603310ca6c13c26d73906384ba2015
Author: Joel Bernstein <jbernste@apache.org>
Date:   Sun Apr 30 22:14:01 2017 -0400

    SOLR-10559: Remove debuggin

commit fdcde26d9d3a67a63a386466e68ec4ede6eb0eb7
Author: Jan Høydahl <janhoy@apache.org>
Date:   Fri Apr 28 22:12:17 2017 +0200

    SOLR-7041: Remove a lot of defaultOperator and defaultSearchField from test configs (still more work to do)

commit 97d80a1abfdf2014b1f6cebfab5d84181d01701d
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Sat Apr 29 13:35:24 2017 +0300

    LUCENE-7798: Add .equals and .hashCode to ToParentBlockJoinSortField

commit 177880ea0d6309c50ff7ab241549510ae12569d0
Author: Dennis Gove <dpgove@gmail.com>
Date:   Fri Apr 28 21:45:56 2017 -0400

    SOLR-10559: Updates TupStream and enhances evaluators to work over values in the SteamContext

commit 730dc2f01ce3d0ca45ee29c433923c6ae5dc456f
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri Apr 28 15:01:43 2017 -0400

    SOLR-10582: Add Correlation Evaluator

commit baa6cdaded4fd528bdd3801d64f28649416d1519
Author: Steve Rowe <sarowe@apache.org>
Date:   Fri Apr 28 15:36:50 2017 -0400

    SOLR-9596: Add Solr support for SimpleTextCodec, via <codecFactory class=solr.SimpleTextCodecFactory/> in solrconfig.xml (per-field specification in the schema is not possible).

commit eeeb947db5399abebcf361f20278fe17e5a61335
Author: Steve Rowe <sarowe@apache.org>
Date:   Fri Apr 28 11:24:53 2017 -0400

    SOLR-9386: Upgrade Zookeeper to 3.4.10

commit 21bab68f2fdb341723801fef7317cb8e43cbaba8
Author: Steve Rowe <sarowe@apache.org>
Date:   Fri Apr 28 10:14:38 2017 -0400

    LUCENE-7794: buildAndPushRelease.py should run validate and documentation-lint

commit 0c9cf102e2d330f405e18072cebc11b6fba99404
Author: Steve Rowe <sarowe@apache.org>
Date:   Fri Apr 28 09:58:02 2017 -0400

    LUCENE-7793: smokeTestRelease.py should run documentation-lint

commit 2b687866a96878f38ab46de22cceb9bd92d7a0dc
Author: Dawid Weiss <dweiss@apache.org>
Date:   Fri Apr 28 10:58:55 2017 +0200

    LUCENE-7796: Make IOUtils.reThrow idiom declare Error return type so
    callers may use it in a way that compiler knows subsequent code is
    unreachable. reThrow is now deprecated in favor of IOUtils.rethrowAlways.

commit 0b699d6a3338031d7812f1c0d07db86352ff1193
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Sat Apr 22 14:39:33 2017 +0300

    SOLR-10521: adding sort=childfield(field,$q) asc for {!parent} query.

commit 97c6ed53d19cda6a6c46209d858387949a34b088
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Apr 27 17:03:01 2017 -0400

    SOLR-10559: Fix precommit

commit d8e991cecdd0d829ae97ee92f628ffa2aebda87d
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Apr 27 16:40:24 2017 -0400

    SOLR-10559: Fixed compilation error

commit ff9fd224db521a1009be7ff5ac734c267b110088
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Apr 27 16:30:46 2017 -0400

    SOLR-10559: Cleaner syntax

commit 59c76a2e69701125a200f84a673129b24397d30b
Author: Erik Hatcher <ehatcher@apache.org>
Date:   Thu Apr 27 15:34:47 2017 -0400

    Remove unused imports

commit 44dff45ad5ac0f10d0838e70173dad73457e48eb
Author: Erik Hatcher <ehatcher@apache.org>
Date:   Thu Apr 27 15:16:43 2017 -0400

    Add CHANGES entry for LUCENE-7481

commit 2ff15d575046f6b5cc0291e33a0270e2940d67d1
Author: Erik Hatcher <ehatcher@apache.org>
Date:   Thu Apr 27 15:08:51 2017 -0400

    LUCENE-7481: Fix PayloadScoreQuery rewrite

commit 8f37fb29e94211869c3046ae048dc462dd187f9d
Author: Erik Hatcher <ehatcher@apache.org>
Date:   Thu Apr 27 08:55:05 2017 -0400

    LUCENE-7481: Fix SpanPayloadCheckQuery rewrite

commit 53c1c5fa205702425c7cfc45f480bc1ee1d2c92f
Author: Jim Ferenczi <jimczi@apache.org>
Date:   Thu Apr 27 13:53:52 2017 +0200

    Add 6.5.1 back compat test indexes

commit 733829de9d293af95ff1ef230f81a8ed537cb29a
Author: Jim Ferenczi <jimczi@apache.org>
Date:   Thu Apr 27 13:06:39 2017 +0200

    update doap files with Lucene / Solr 6.5.1 release

commit afc0a997d930c8a414850fa73a8616eb9dcc3d24
Author: Mike McCandless <mikemccand@apache.org>
Date:   Thu Apr 27 06:59:24 2017 -0400

    fix wrong version in logging message

commit 6082e9e1bd9eadc814132e39b096ea835830db9b
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Thu Apr 27 13:01:07 2017 +0300

    SOLR-10500: fix many parents with nested children per /update/json/docs request

commit 10c1379bc3cd2f0fb75b61e72e8d3bc3a6283ab0
Author: Noble Paul <noble@apache.org>
Date:   Thu Apr 27 12:38:34 2017 +0930

    SOLR-10507: Core Admin status command to emit collection details of each core

commit a8d2792b21f182aaaf9bc0da3aea571507b0f8ae
Author: Joel Bernstein <jbernste@apache.org>
Date:   Wed Apr 26 22:34:20 2017 -0400

    SEARCH-313: Handled unescaped plus sign in gap

commit df3e927ade3f6befdfde511abc4ef2c067df7268
Author: Erik Hatcher <ehatcher@apache.org>
Date:   Wed Apr 26 20:05:32 2017 -0400

    LUCENE-7808: Fix PayloadScoreQuery and SpanPayloadCheckQuery .equals and .hashCode methods.

commit 2e3ee9a40de51f4eb91bb80c870d95fb114d8e07
Author: David Smiley <dsmiley@apache.org>
Date:   Wed Apr 26 14:04:35 2017 -0400

    SOLR-10526: fix facet.heatmap facet exclusion when distributed/sharded

commit deb017464c5d63688848c722edd54df239eeedcd
Author: Andrzej Bialecki <ab@apache.org>
Date:   Wed Apr 26 19:50:31 2017 +0200

    SOLR-10569: "updateHandler" stats is null when queried via MBeans handler.

commit 3ead4b38d9f6ec28d2c44fd9e255432cca646ab8
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Wed Apr 26 17:01:29 2017 +0100

    SOLR-10046: move from 6.6.0 to 7.0.0 CHANGES.txt (backport yet to be completed)

commit 28e3134910655c0a41bd241652f1f514c4fe9c20
Author: Andrzej Bialecki <ab@apache.org>
Date:   Wed Apr 26 17:47:34 2017 +0200

    SOLR-10565: Make root names more unique.

commit 92901c859bf2029c49e5eb10dad89280860a928c
Author: Joel Bernstein <jbernste@apache.org>
Date:   Wed Apr 26 11:17:22 2017 -0400

    SOLR-10566: Fix precommit

commit 8a3cbb5c75df08858b04f3191d8b3cfbc748f28d
Author: Joel Bernstein <jbernste@apache.org>
Date:   Wed Apr 26 10:57:52 2017 -0400

    SOLR-10566: Add timeseries Streaming Expression

commit def07596935e3192d82edc6ce70b76ef8997c021
Author: Mike McCandless <mikemccand@apache.org>
Date:   Wed Apr 26 09:36:14 2017 -0400

    LUCENE-7792: add try/finally to make sure semaphore is released on exceptions

commit 004f086c1a7eff00362ebea350a27e648c798c96
Author: David Smiley <dsmiley@apache.org>
Date:   Wed Apr 26 08:38:52 2017 -0400

    SOLR-10537: Added SolrParams.toLocalParamsString() and moved QP.encodeLocalParamVal to ClientUtils

commit 6febaa31715fe57b2794aae0325b29a64aa6223e
Author: Mike McCandless <mikemccand@apache.org>
Date:   Wed Apr 26 06:38:28 2017 -0400

    LUCENE-7792: add optional concurrency to OfflineSorter

commit 0c4f83af1d5bcedafc6f2bf44ee5225ef9e7cd48
Author: Andrzej Bialecki <ab@apache.org>
Date:   Wed Apr 26 12:31:39 2017 +0200

    SOLR-10489: Fix an occasional NPE.

commit 0adfdba01e1790844375db4f319ca01a4060788c
Author: yonik <yonik@apache.org>
Date:   Tue Apr 25 22:56:44 2017 -0400

    SOLR-10480: fix offset param handling in JSON Facet API

commit 4e20b288a8d70043e1ae4e209c0e5e43574ff928
Author: Mike McCandless <mikemccand@apache.org>
Date:   Tue Apr 25 20:34:00 2017 -0400

    LUCENE-7801: SortedSetDocValuesReaderState now implements Accountable

commit 78ae8b98764e08c901031ab8674954678aa187cc
Author: Steve Rowe <sarowe@apache.org>
Date:   Tue Apr 25 17:07:42 2017 -0400

    SOLR-10310: Fix CopyFieldTest failure

commit 8c58414bf73a0949859d7f6eaea57519f9569e17
Author: Steve Rowe <sarowe@apache.org>
Date:   Tue Apr 25 12:02:25 2017 -0400

    SOLR-10310: By default, stop splitting on whitespace prior to analysis in edismax and standard/"lucene" query parsers

commit aff53529218c86f1e71dc1988fa277f72ab6eba8
Author: yonik <yonik@apache.org>
Date:   Tue Apr 25 11:00:30 2017 -0400

    SOLR-7452: JSON Facet API - refining for numBuckets

commit d19848665bb09fe0adc757ec0eb57b031db3d5fe
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Tue Apr 25 12:27:18 2017 +0300

    SOLR-10520: fix child.facet.field counts

commit 61c121cffdf4e1cd3005a6cea69a22761b4356d4
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon Apr 24 17:27:37 2017 -0400

    SOLR-10559: Add let and get Streaming Expressions

commit 2c20d1d5dfb69af3faee10c5cfc52ecddde5f527
Author: yonik <yonik@apache.org>
Date:   Mon Apr 24 18:17:17 2017 -0400

    SOLR-10548: SOLR-10552: numBuckets should use hll and ignore mincount>1 filtering

commit 5d57d866e31b8e3575b254dcbef0792b524e5c97
Author: Andrzej Bialecki <ab@apache.org>
Date:   Mon Apr 24 22:34:46 2017 +0200

    SOLR-10557: Make "compact" format default for /admin/metrics.

commit 26e717b22d74ea635a4cd510d8b0ccfd1a69e399
Author: Erick Erickson <erick@apache.org>
Date:   Mon Apr 24 12:17:46 2017 -0700

    SOLR-10493: Investigate SolrCloudExampleTest failures.

    (cherry picked from commit 0247acd)

commit 24bea6c588320decad5bcfc4c0e51843eae12b12
Author: Shalin Shekhar Mangar <shalin@apache.org>
Date:   Tue Apr 25 00:26:21 2017 +0530

    SOLR-10047: Move test into its own test class and force use of NoMergePolicy to fix test failures

    This closes #195

commit a11a49bbd644ffe157b75b295dbc981dbc8b0316
Author: Andrzej Bialecki <ab@apache.org>
Date:   Mon Apr 24 16:12:02 2017 +0200

    SOLR-10489 Tentative fix for a test failure (Mikhail Khludnev via ab)

commit b7e56dbbf41e914b78fb93afcbd32fe8dfc7bf1c
Author: Mike McCandless <mikemccand@apache.org>
Date:   Sat Apr 22 18:45:27 2017 -0400

    don't allow ExtrasFS for this test case

commit 044e0fb915124d05faeb736835af235015e4893f
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Sun Apr 23 00:28:20 2017 +0300

    SOLR-9217: delay JoinUtil call to createWeight for score join

commit 96f8100ad10d51e233a268a9afa07c376e1db047
Author: Joel Bernstein <jbernste@apache.org>
Date:   Sat Apr 22 17:26:19 2017 -0400

    SOLR-10551: Improve tests

commit 75a250d12d94caf2ff3077ea34bf6f0706642b33
Author: Joel Bernstein <jbernste@apache.org>
Date:   Sat Apr 22 16:38:38 2017 -0400

    SOLR-10551: Add list and cell Streaming Expressions

commit 744e5deaa4dbb7a97ab0aecf67c3f73e05032aba
Author: Mike McCandless <mikemccand@apache.org>
Date:   Sat Apr 22 08:49:34 2017 -0400

    LUCENE-7797: the static FSDirectory.listAll was always returning an empty array

commit 4f5ea24213321f63ccdaaa6d2699c6db9152e03a
Author: Jim Ferenczi <jimczi@apache.org>
Date:   Fri Apr 21 12:01:09 2017 +0200

    LUCENE-7791: add tests for index sorting with sparse text fields and norms

commit 3f59e233ee3f38fd4b543324fbe64a985833332a
Author: Jim Ferenczi <jimczi@apache.org>
Date:   Fri Apr 21 04:41:24 2017 +0200

    LUCENE-7791: add tests with index sorting and sparse docvalues fields

commit f933c812e4cadf92b1bd356780c1d8763e2fd1ed
Author: David Smiley <dsmiley@apache.org>
Date:   Thu Apr 20 17:46:28 2017 -0400

    SOLR-10499: facet.heatmap DocSet to Bits optimizations

commit 7270943e56bfff7d4be4bd5e2649b4078594a698
Author: Andrzej Bialecki <ab@apache.org>
Date:   Thu Apr 20 15:09:14 2017 +0200

    SOLR-10514 Upgrade Metrics library to 3.2.2.

commit a2b316e60fd3d10e8122ab9ac31b5127ec291a5e
Author: Shai Erera <shaie@apache.org>
Date:   Thu Apr 20 15:32:27 2017 +0300

    Shorten docFreq and totalTermFreq to df and ttf in TermsComponent

commit 20674464e04a261eda815e4a1a01e0111f882d60
Author: Shai Erera <shaie@apache.org>
Date:   Tue Apr 18 06:33:18 2017 +0300

    SOLR-10505: Add multi-field support to TermsComponent for terms stats

commit 4d7efe1f0c735788f75cb5b2d968ba8c77d6f76d
Author: Steve Rowe <sarowe@gmail.com>
Date:   Wed Apr 19 20:10:11 2017 -0400

    SOLR-10527: move CHANGES entry to 6.5.1 section

commit b07f6f4c4b0df53638eb1e672505fd57ba315335
Author: Steve Rowe <sarowe@gmail.com>
Date:   Wed Apr 19 19:02:32 2017 -0400

    SOLR-10527: edismax with sow=false fails to create dismax-per-term queries when any field is boosted

commit 7a9728df01cd7b5e7c42578290f427e7c44a64f9
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Wed Apr 19 12:05:00 2017 +0100

    (part 1 of several) SOLR-10415: use parameterized debug logging in SearchHandler and RealTimeGetComponent (Michael Braun via Christine Poerschke)

commit fd5f957bb0bc3c27cefe9ca26423ae6a440b0a74
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Wed Apr 19 11:50:24 2017 +0100

    SOLR-10394: a few more essentially non-public sortWithinGroup to withinGroupSort renames

commit 86fe1543b32db0c1316b49ef0b19cd36be03324f
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Wed Apr 19 11:45:59 2017 +0100

    SOLR-5127: Multiple highlight fields and wildcards are now supported e.g. hl.fl=title,text_*
    (Sven-S. Porst, Daniel Debray, Simon Endele, Christine Poerschke)

commit 972dc3164581950f1b33e74fa9841f86413c2cd3
Author: Dawid Weiss <dweiss@apache.org>
Date:   Wed Apr 19 12:21:18 2017 +0200

    LUCENE-7785: Move dictionary for Ukrainian analyzer to external dependency. (Andriy Rysin via Dawid Weiss)

commit 432aa350b8e7d7e436aea3556c7c27e99f09b4cb
Author: Joel Bernstein <jbernste@apache.org>
Date:   Tue Apr 18 20:56:21 2017 -0400

    SOLR-10516: Add eval() Streaming Expression

commit 036ce79066dbe1ff7b435bdf006ad069d6b73f8f
Author: David Smiley <dsmiley@apache.org>
Date:   Tue Apr 18 17:02:07 2017 -0400

    SOLR-10439: 'large' was forgotten in /schema/fields?showDefaults=true

commit c0f50ad282be823c192c9650b8d5a0442eedb6f0
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue Apr 18 11:58:35 2017 -0700

    SOLR-10472: Fixed uninversion (aka: FieldCache) bugs with the numeric PointField classes, and CurrencyField

commit 79ab781507d8d7728bbb89faae495d5174424676
Author: Scott Blum <dragonsinth@gmail.com>
Date:   Mon Apr 17 18:27:12 2017 -0400

    SOLR-10420: fix watcher leak in DistributedQueue

commit 2e9bea5c062da5e2efedd5158dcf84fab9978c50
Author: Scott Blum <dragonsinth@gmail.com>
Date:   Tue Apr 18 14:36:01 2017 -0400

    changes

commit f1ef3d1dbd95591460c7e00fcc31193ae70a1083
Author: David Smiley <dsmiley@apache.org>
Date:   Tue Apr 18 14:14:38 2017 -0400

    LUCENE-7769: UnifiedHighlighter wasn't seeing inside BoostQuery or SpanBoostQuery

commit 8a7401550a5f624286704736f9fdf957bdcce88e
Author: David Smiley <dsmiley@apache.org>
Date:   Tue Apr 18 12:32:12 2017 -0400

    LUCENE-7787: HeatmapFacetCounter Bits.MatchNoBits optimization

commit 43d02330c96128320cfa905ffc85f81d2bd8cb2c
Author: Joel Bernstein <jbernste@apache.org>
Date:   Tue Apr 18 11:21:15 2017 -0400

    SOLR-10504: Add echo Streaming Expression

commit d44f6698ed426f20833fb3f558afd6248c9c0456
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Tue Apr 18 15:53:13 2017 +0100

    Remove unused imports.

commit 44db665e9989550fe41261246d075889985fec91
Author: Andrzej Bialecki <ab@apache.org>
Date:   Tue Apr 18 12:46:39 2017 +0200

    SOLR-10418: Metrics API should return JVM system properties.

commit be6d90f5965b8408130dc14cace470a19f1a036b
Author: yonik <yonik@apache.org>
Date:   Mon Apr 17 22:30:29 2017 -0400

    SOLR-10082: JSON Facet API, add stddev and variance functions

commit 3e9d4c64ed0145aeeb3c3f5ec32c056e71541f26
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Tue Apr 18 03:29:45 2017 +0530

    SOLR-10447, SOLR-8589: Adding Yago Riveiro to changelog

commit edafee5838ac9ae469429a189aae2955ac955977
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Tue Apr 18 03:12:41 2017 +0530

    SOLR-10447, SOLR-4968, SOLR-8589: Adding contributors to CHANGES.txt

commit 4c25637aa8dc7c48399c583fe9f3aa17d0674971
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Tue Apr 18 03:09:46 2017 +0530

    SOLR-10446: Making HttpClusterStateProvider work with server that doesn't have LISTALIASES

commit 491d3231ac6221e6d7520013dcf28debea1be6f6
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon Apr 17 11:08:53 2017 -0400

    SOLR-10486: Fix precommit

commit a7a3cb69292d62824f54c2fc3dcff5a8c0b8c153
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon Apr 17 10:10:05 2017 -0400

    SOLR-10486: Add Length Conversion Evaluators

commit b903026ed4d72a423f39f3b28a3a55d2846b22c7
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Apr 13 11:57:45 2017 -0400

    SOLR-10485: Remove incorrect code comment

commit 249baee1850b0a2608ec029f8865fba54ecaff44
Author: Shalin Shekhar Mangar <shalin@apache.org>
Date:   Mon Apr 17 13:59:26 2017 +0530

    SOLR-10047: Mismatched Docvalues segments cause exception in Sorting/Faceting. Solr now uninverts per segment to avoid such exceptions

    Squashed commit of the following:

    commit c38f4cabc2828ee83b53b931dd829e29a3e1701c
    Author: Keith Laban <kelaban17@gmail.com>
    Date:   Tue Apr 11 17:17:05 2017 -0400

        reverted tests to using old wrap interface

    commit 806f33e092491cc6a2ee292d2934c76171e40dc7
    Author: Keith Laban <kelaban17@gmail.com>
    Date:   Tue Apr 11 17:13:34 2017 -0400

        updated UninvertingReader.wrap / tests

    commit b10bcab338b362b909491fea1cf13de66f5f17c0
    Author: Keith Laban <klaban1@bloomberg.net>
    Date:   Wed Apr 5 14:57:28 2017 -0400

        SOLR-10047 - Updated javadoc/renamed class/added getReaderCacheHelper

    commit 90ecf5a4ae4feaf3efc42a1ed8643ad21e1c73ce
    Author: Keith Laban <klaban1@bloomberg.net>
    Date:   Wed Jan 18 16:39:51 2017 -0500

        SOLR-10047 - SolrIndexSearcher, UninvertingReader, uninvert docvalues per segment

commit 3dbc429295ae4f565c2a998c4f3122e2ffa6502e
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Mon Apr 17 10:11:18 2017 +0530

    SOLR-10447, SOLR-10447: LISTALIASES Collections API command; CloudSolrClient can be initialized using Solr URL

       SOLR-10447: Collections API now supports a LISTALIASES command to return a list of all collection aliases.

       SOLR-10446: CloudSolrClient can now be initialized using the base URL of a Solr instance instead of
        ZooKeeper hosts. This is possible through the use of newly introduced HttpClusterStateProvider.
        To fetch a list of collection aliases, this depends on LISTALIASES command, and hence this way of
        initializing CloudSolrClient would not work with older versions of Solr that doesn't support LISTALIASES.

commit 6f691363fe914157a9c39e4de23e4d0eb49e4c34
Author: Mike McCandless <mikemccand@apache.org>
Date:   Fri Apr 14 08:15:32 2017 -0400

    LUCENE-7782: OfflineSorter now passes the number of items it will write to getWriter

commit 3a84e9a55a4547009ca8e115759d5886dcc2f207
Author: markrmiller <markrmiller@apache.org>
Date:   Fri Apr 14 01:33:19 2017 -0400

    SOLR-9936: Allow configuration for recoveryExecutor thread pool size.

commit 07c09b6f84defe5be00d135005ac56652ef72d75
Author: markrmiller <markrmiller@apache.org>
Date:   Fri Apr 14 01:17:03 2017 -0400

    SOLR-10151: Use monotonically incrementing counter for doc ids in TestRecovery.

commit 9ca384377c41e16721e1c84bb32fe20865cd0eec
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Thu Apr 13 17:31:22 2017 +0530

    SOLR-6736: Fix authorization permissions

commit 664a4236e4133359b6857d45149f019b9354bc03
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Apr 13 09:43:01 2017 +0200

    LUCENE-7783: Fix NPE when getting points on a non-existing field.

commit 3b41debf70f7f71df52b36a6de89f00545f454e6
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Apr 13 08:21:39 2017 +0200

    LUCENE-7780: Remove TermValComparator.isNull.

commit a216928eafc25e8af24c6a1384d5d6dfa22d9ff9
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Apr 13 08:20:56 2017 +0200

    LUCENE-7781: Call ensureOpen when registering closed listeners.

commit 58de889d2bc9de254c881121fa2bc6054261e600
Author: Erick Erickson <erick@apache.org>
Date:   Wed Apr 12 17:02:40 2017 -0700

    SOLR-10007: Clean up references to CoreContainer and CoreDescriptors

commit 27936c20edd5c828fe9c8ff69ab1a80cf011f8e0
Author: Joel Bernstein <jbernste@apache.org>
Date:   Wed Apr 12 17:26:30 2017 -0400

    SOLR-10485: Fix precommit

commit df9d1958d413499e657147459549a3ca8c821d71
Author: Joel Bernstein <jbernste@apache.org>
Date:   Wed Apr 12 17:09:36 2017 -0400

    SOLR-10485: Add CalculateStream to allow Solr to behave like a scientific calculator

commit 7da48bd9dfc0281e1fe2c7bc6feb316fb91d1aca
Author: Mike McCandless <mikemccand@apache.org>
Date:   Wed Apr 12 17:17:30 2017 -0400

    LUCENE-7779: don't call BytesSequencesReader.next again after it already returned null

commit 88e85dc0a5e5bf5ec4bff76747040bc5ca4bb111
Author: Joel Bernstein <jbernste@apache.org>
Date:   Wed Apr 12 14:46:31 2017 -0400

    SOLR-10303: Fix precommit

commit 7799fcf8c48bf103730d6741041b3a4edf6d04ab
Author: Joel Bernstein <jbernste@apache.org>
Date:   Wed Apr 12 13:18:19 2017 -0400

    SOLR-10303: Add the tuple context to avoid creating multiple LocalDateTime instances for the same Tuple

commit 8c67cbc345a01f198baaabec016fc33052d209cd
Author: Gethin James <gethin.james@alfresco.com>
Date:   Wed Apr 12 18:00:35 2017 +0200

    SOLR-10303:  Removing the unused class DatePartEvaluator from the test

commit 0f2259de6770163351f4e3f0ef4d2f85d34565bd
Author: Gethin James <gethin.james@alfresco.com>
Date:   Thu Apr 6 17:19:31 2017 +0200

    SOLR-10303:  Error message formatting for TemporalEvaluator

commit 3fc1c254efe6e87ce74a2afe49a7a581fb7f961a
Author: Gethin James <gethin.james@alfresco.com>
Date:   Thu Apr 6 17:19:02 2017 +0200

    SOLR-10303:  Removing the unused class, replaced by TemporalEvaluator

commit d138df5745ed2b1f05f132a480d2c63a4e9a0209
Author: Gethin James <gethin.james@alfresco.com>
Date:   Thu Apr 6 11:58:26 2017 +0200

    SOLR-10303:  Refactored to multiple TemporalEvaluator classes based on feedback

commit 3c65492947d844070efbe00d7029bd1a9886010f
Author: Gethin James <gethin.james@alfresco.com>
Date:   Mon Mar 20 17:08:15 2017 +0100

    SOLR-10303:  Switched to pascal casing

commit a325c359b0188ea8dd3ceb87bccad6a50e56c507
Author: Gethin James <gethin.james@alfresco.com>
Date:   Mon Mar 20 17:02:41 2017 +0100

    SOLR-10303:  Supporting more datatypes via a TemporalAccessor

commit 9f3f27d4eb403431878fb80fe08165341a6fed21
Author: Gethin James <gethin.james@alfresco.com>
Date:   Sat Mar 18 10:42:19 2017 +0100

    SOLR-10303:  Supporting epoch for LocalDateTime

commit 7d97c1692d32d31b94116ec4458fe571e873149f
Author: Gethin James <gethin.james@alfresco.com>
Date:   Fri Mar 17 18:11:00 2017 +0100

    SOLR-10303:  Switching from the fieldName param to subEvaluators

commit 748bff204b7037ad85b6d2c061a5aba4920304a1
Author: Gethin James <gethin.james@alfresco.com>
Date:   Fri Mar 17 15:30:12 2017 +0100

    SOLR-10303: Renamed to DatePartEvaluator and adding support for Instant, Date, LocalDateTime

commit e11a40426a36152ccf02f32d0d7ccc6c178b6053
Author: Gethin James <gethin.james@alfresco.com>
Date:   Fri Mar 17 14:07:50 2017 +0100

    SOLR-10303: Initial support for common date/time Stream Evaluators

commit 0f3a35220a1097e0f314b0a800ebbadae28e3cc8
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Tue Apr 11 11:16:23 2017 +0100

    LUCENE-7746: precommit to ignore less and (potentially) error more

commit e1e9a71f976aaaa5e43d2615cc6479e5a5259321
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Tue Apr 11 11:14:34 2017 +0100

    SOLR-10473: Correct LBHttpSolrClient's confusing SolrServerException message when timeAllowed is exceeded.

commit 4ed424dc8e6ffdc1deb9f5b479dc74adfe5eb7f5
Author: Andrzej Bialecki <ab@apache.org>
Date:   Wed Apr 12 10:40:58 2017 +0200

    SOLR-9959 Increase the timeout to allow searcher to register metrics.

commit 6df755782089c82e94e2c763aa1d6f5d39a17269
Author: Tommaso Teofili <tommaso@apache.org>
Date:   Wed Apr 12 09:55:03 2017 +0200

    LUCENE-7776 - adjusted failing tests in solr due to switching to bm25 in knn

commit 4485f66d4a91e3387cb88b4229eace1055dfa889
Author: Mike McCandless <mikemccand@apache.org>
Date:   Tue Apr 11 15:37:42 2017 -0400

    LUCENE-7760: improve setMaxTokenLength javadocs for StandardAnalyzer/Tokenizer and UAX29URLEmailAnalyzer/Tokenizer

commit 6e1418c9f2f6d60b3f6462ecc7da0ed2d1d3228b
Author: Joel Bernstein <jbernste@apache.org>
Date:   Tue Apr 11 15:36:03 2017 -0400

    SOLR-10274: fix precommit

commit 579b5f9199899225f40a82b95b4a213a2cfab7af
Author: Joel Bernstein <jbernste@apache.org>
Date:   Tue Apr 11 15:17:03 2017 -0400

    SOLR-10274: The search Streaming Expression should work in non-SolrCloud mode

commit dea126033ef363a9736b1fbb568838a9741432f8
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Tue Apr 11 20:01:05 2017 +0100

    Removed two unused imports.

commit ed9f434f37ee64ef0e2bfa5a479feac2f970cb45
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Tue Apr 11 19:36:44 2017 +0100

    LUCENE-7776: change javadocs default mention from Classic to BM25

    (Also kinda added missing javadoc for new method to fix 'ant precommit'.)

commit 72b4579b71e0a7958c8602eba0122f898d88a64b
Author: Andrzej Bialecki <ab@apache.org>
Date:   Tue Apr 11 19:22:23 2017 +0200

    SOLR-9959: SolrInfoMBean-s category and hierarchy cleanup.

commit c21e8664e3ccffee96addb46bd2426d3db80835c
Author: Mike McCandless <mikemccand@apache.org>
Date:   Tue Apr 11 11:47:31 2017 -0400

    LUCENE-7777: fix AIOOBE from ByteBlockPool.readBytes when byte block exceeds 32 KB

commit 6b3dcad5244406be51b03d33d1c1e988a3805120
Author: Tommaso Teofili <tommaso@apache.org>
Date:   Tue Apr 11 17:12:13 2017 +0200

    LUCENE-7776 - visualize diff btwn BytesRef values in ClassificationTestBase

commit 87d3b8cc5afd40512016407320b1d9356bfb543b
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Tue Apr 11 11:25:39 2017 +0100

    LUCENE-7776: remove unused import

commit 6586d7ce9022e913271e3cc4d83031c555bb7d85
Author: Tommaso Teofili <tommaso@apache.org>
Date:   Tue Apr 11 10:44:36 2017 +0200

    LUCENE-7776 - use bm25 for knn classifier

commit 58f7f68d7142dc3a6f9d3bcd0b036f084b6879ad
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Apr 11 10:02:51 2017 +0200

    LUCENE-7767: SortedDocValues.ordValue() now throws an IOException.

commit 9f300bb85da19635dc627db1c51f3dead10120e1
Author: Mike McCandless <mikemccand@apache.org>
Date:   Mon Apr 10 19:07:25 2017 -0400

    LUCENE-7775: fix exception handling to throw first exception hit

commit b0cb9cff99d3d0386193e7c84b50fd61c07c8dc0
Author: Steve Rowe <sarowe@gmail.com>
Date:   Mon Apr 10 15:26:28 2017 -0400

    SOLR-10474: TestPointFields.testPointFieldReturn() depends on order of unsorted hits

commit 6535a3cd40eb5e35cbf112d4e131584bb163fbee
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Mon Apr 10 09:48:07 2017 -0700

    SOLR-10437: Delete index after each test in TestUseDocValuesAsStored

commit 82f13b8c5e8bd7e3d476c0a6bbe8f5d7ebad057c
Author: jdyer1 <jdyer@apache.org>
Date:   Mon Apr 10 08:39:41 2017 -0500

    SOLR-8807: disable the CollapseQParser Plugin when testing spellcheck collations for hit-counts

commit e0a2890a6f35e26d045dfc9e37f028efcb6dfbd7
Author: Alan Woodward <romseygeek@apache.org>
Date:   Tue Mar 28 19:52:53 2017 +0100

    LUCENE-7701: Refactor grouping collectors

commit adf2819c16af26cb15b59afbdecf666fc8b78f66
Author: Noble Paul <noble@apache.org>
Date:   Mon Apr 10 06:43:27 2017 +0930

    SOLR-10429: UpdateRequest#getRoutes()should copy the response parser

commit 76a594a5677429b4e0d51a4e25d796d5a77a6120
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Fri Apr 7 14:11:25 2017 -0700

    SOLR-10443: Improvements to TestPointFields

    * Fixes testInternals, index needs to be cleaned after each field
    * Validate that SolrQueryParser generates a PointInSetQuery when possible

commit 94817b40a65bfbfb95dbed1a35ede606c1d30978
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Fri Apr 7 11:36:22 2017 -0700

    SOLR-10437: Improve test coverage of useDocValuesAsStored=false

commit 5b84bad617c21a2af1242ce3e098c28b58715397
Author: Joel Bernstein <jbernste@apache.org>
Date:   Fri Apr 7 12:36:19 2017 -0400

    Add version 6.5.1

commit b60de7380b49906096c6bdc40fbfccc78a3df1fe
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Fri Apr 7 11:11:23 2017 +0100

    SOLR-10264: Fixes multi-term synonym parsing in ManagedSynonymFilterFactory.
    (Jörg Rathlev, Steve Rowe, Christine Poerschke)

commit 315843008bf287c882361aec9847844c8fa9d273
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Fri Apr 7 11:04:41 2017 +0100

    SOLR-10440: LBHttpSolrClient.doRequest is now always wrapped in a Mapped Diagnostic Context (MDC).

commit 27bea7dc2738281fa1a3beb801f83920f48ce009
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Fri Apr 7 10:37:30 2017 +0100

    SOLR-10441: remove no longer used HttpShardHandlerFactory.USE_RETRIES

commit c2d73021182bbe5789e1ca5e3d08a16e1282e7f5
Author: Tommaso Teofili <tommaso@apache.org>
Date:   Fri Apr 7 10:58:49 2017 +0200

    LUCENE-5548 - improved testing for SNBC

commit 6af46f1195d899f542aa603684433b4360329423
Author: Tommaso Teofili <tommaso@apache.org>
Date:   Thu Apr 6 19:13:50 2017 +0200

    LUCENE-6853 - re-enabled test classification measures for bpc

commit 99c57f533eaf896ac7750f0833939fbd220fbd5b
Author: Tommaso Teofili <tommaso@apache.org>
Date:   Thu Apr 6 19:05:52 2017 +0200

    LUCENE-6853 - renamed threshold to bias, initialize to avg tf

commit 9a3c8c43af63de6a3e37f07df6aa2834b80a81e7
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Apr 6 22:06:07 2017 -0400

    SOLR-10341, SOLR-10444: Update CHANGES.txt

commit ed052b20c923f6b3883068b7cf570881686cdcc2
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Apr 6 21:33:47 2017 -0400

    SOLR-10444: Fix precommit

commit 47f2548a221482f071ad7facfb5d982d5d20dafe
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Apr 6 21:17:02 2017 -0400

    SOLR-10444: SQL interface does not use client cache

commit 878fdedba086a660c5483ba08a3cb05f02abafac
Author: Chris Hostetter <hossman@apache.org>
Date:   Thu Apr 6 12:07:41 2017 -0700

    SOLR-10425: Fix indexed="false" on numeric PointFields

commit 5a1d69782f30bf8b8fb4342ef2bfcf862777b8f6
Author: jdyer1 <jdyer@apache.org>
Date:   Thu Apr 6 12:48:19 2017 -0500

    SOLR-10323: fix to SpellingQueryConverter to properly strip out colons in field-specific queries

commit 338a5e6920a639ad0901079b9f7c4763174a1177
Author: yonik <yonik@apache.org>
Date:   Thu Apr 6 09:29:29 2017 -0400

    SOLR-7452: add support for refining missing allBuckets

commit ec5731a60905cbdcc648fbe30502e0d595b6bf00
Author: Cao Manh Dat <datcm@apache.org>
Date:   Thu Apr 6 16:02:20 2017 +0700

    SOLR-10239: Update CHANGES.txt

commit f89f9f4ea44d95c44dd2d4ad6d329053b5afd0c2
Author: Cao Manh Dat <datcm@apache.org>
Date:   Thu Apr 6 15:57:43 2017 +0700

    SOLR-10239: change empty lambda to null

commit 4942a94d7198af70091cb510695468f3802d6198
Author: Cao Manh Dat <datcm@apache.org>
Date:   Thu Apr 6 15:48:38 2017 +0700

    SOLR-10239: MOVEREPLICA API

commit 03c3282de1049589360e8e33110ae0ce697c9854
Author: Joel Bernstein <jbernste@apache.org>
Date:   Wed Apr 5 17:57:11 2017 -0400

    SOLR-10426: Add shuffle Streaming Expression

commit fd3b624e9f3f7c458ffbdec86ebe572ce1f7a7f7
Author: Steve Rowe <sarowe@gmail.com>
Date:   Wed Apr 5 16:23:26 2017 -0400

    SOLR-10423: Disable graph query production via schema configuration <fieldtype ... enableGraphQueries="false">.  This fixes broken queries for ShingleFilter-containing query-time analyzers when request param sow=false.

commit d0b5031ffdbd9f2ee2b45db5354d7b931fdce42c
Author: Nicholas Knize <nknize@gmail.com>
Date:   Wed Apr 5 11:10:15 2017 -0500

    LUCENE-7738: Fix min/max verification bug in InetAddressRange to correctly compare IPv4 and IPv6. Update tests.

commit 14231a9625f483cbd4ff8bc62aa52d22f6c64d63
Author: David Smiley <dsmiley@apache.org>
Date:   Wed Apr 5 08:56:50 2017 -0400

    SOLR-10404: fetch() streaming expression: escape values in generated query.

commit 62bf5c334c3cb9fddbb5fe2732bc53ba443b54fc
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Wed Apr 5 11:52:34 2017 +0100

    Remove unused (private static final) loggers in LTRQParserPlugin and LTRFeatureLoggerTransformerFactory.

commit 039e256b59b18e600cbef5443daf86056b6d0e1f
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Wed Apr 5 11:50:13 2017 +0100

    SOLR-10421: Fix params persistence for solr/contrib/ltr (MinMax|Standard)Normalizer classes.
    (Jianxiong Dong, Christine Poerschke)

commit ae603ddf5bc6e09e633e75059912f16f583aa726
Author: Shalin Shekhar Mangar <shalin@apache.org>
Date:   Wed Apr 5 16:01:44 2017 +0530

    SOLR-10277: On 'downnode', lots of wasteful mutations are done to ZK

commit 9f7ef3f878ee70c53531a26866da5447b4c61b97
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Tue Apr 4 13:11:02 2017 -0700

    SOLR-10347: Remove index level boost support from 'documents' section of the admin UI

commit 8cd9deff5b0619807254749d7300048c741c0807
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Tue Apr 4 12:52:09 2017 +0100

    SOLR-10394: Rename getSortWithinGroup to getWithinGroupSort in search.grouping.Command class.
    (Judith Silverman, Christine Poerschke)

commit 4288bc73e456925c0e4ab04fe99a061b98a10482
Author: Shalin Shekhar Mangar <shalin@apache.org>
Date:   Tue Apr 4 14:20:31 2017 +0530

    SOLR-10416: The JSON output of /admin/metrics is fixed to write the container as a map (SimpleOrderedMap) instead of an array (NamedList)

commit e8d9987d536b431e29d321b3cfb7222911d3574b
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Tue Apr 4 11:01:19 2017 +0300

    SOLR-9745: check exit code only if process has finished

commit c10dd97496eac37aa07cac28759f14c62c9f3438
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 30 09:12:45 2017 +0200

    LUCENE-7756: Only record the major Lucene version that created the index, and record the minimum Lucene version that contributed to segments.

commit 0c8d63373ac94a9deec5e1ec1991ab135dcc5707
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Tue Apr 4 08:42:31 2017 +0300

    SOLR-9745: bring back timeout value to fix tests

commit 4d6eb98561729b778b4feac9f8362d42f196beb1
Author: Mark Miller <markrmiller@apache.org>
Date:   Mon Apr 3 22:00:08 2017 -0300

    SOLR-10338: Revert configure SecureRandom non blocking for tests. (reverted from commit 0445f8200e)

commit 5430570655c8377d45479dffd50cfa0cad3af47a
Author: Joel Bernstein <jbernste@apache.org>
Date:   Mon Apr 3 20:39:37 2017 -0400

    SOLR-10351: Add try-with-resources clause around TokenStream

commit 4111f12e8bfcd6fa2076cb781bb85abdab144534
Author: Mikhail Khludnev <mkhl@apache.org>
Date:   Mon Apr 3 23:45:54 2017 +0300

    SOLR-9745: fix solr.cmd to print errors from invoked script

commit 5ffb14bf57992ede5370bc6ca168dfc75d062c41
Author: Erick Erickson <erick@apache.org>
Date:   Mon Apr 3 13:27:12 2017 -0700

    SOLR-8906: Make transient core cache pluggable

commit e1c6dd4b2fba4b709a39ec5d2a27650ce5f1b954
Author: Adrien Grand <jpountz@gmail.com>
Date:   Mon Apr 3 13:49:05 2017 +0200

    LUCENE-7749: Made LRUQueryCache delegate the scoreSupplier method.

commit cf9ce6ad127844d08f80386e69324d7b02beeb07
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Mon Apr 3 13:01:16 2017 +0100

    SOLR-10383: Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class.
    (Vitezslav Zak, Christine Poerschke)

commit a902df5f45ce05b315243bf352f9c63fa27e341e
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Mon Apr 3 12:10:09 2017 +0100

    SOLR-10383: reduce code duplication in TestOriginalScoreFeature

commit 780a5304487925d4c8e0a3270dd86c38dcb97a3b
Author: Mike McCandless <mikemccand@apache.org>
Date:   Sun Apr 2 16:25:09 2017 -0400

    switch to advanceExact

commit 4c85f560b074a350e3559505a9fc075866f13abc
Author: Dennis Gove <dpgove@gmail.com>
Date:   Fri Mar 31 20:52:42 2017 -0400

    SOLR-10393: Adds UUID Streaming Evaluator

commit ae138852fed59bae2a4106c218b76fca01538198
Author: Dennis Gove <dpgove@gmail.com>
Date:   Thu Mar 23 20:08:11 2017 -0400

    SOLR-10356: Adds basic math streaming evaluators

commit e027bc4b566acd34a04f75cdf4700a26713738ac
Author: Alexandre Rafalovitch <arafalov@apache.org>
Date:   Sat Apr 1 19:06:50 2017 -0400

    SOLR-9601: DIH Tika example is now minimal
    Only keep definitions and files required to show Tika-extraction in DIH

commit a78aef990d5f4ae9a7d6704d677bd000ce16f8a9
Author: Alexandre Rafalovitch <arafalov@apache.org>
Date:   Sat Apr 1 13:42:23 2017 -0400

    SOLR-7383: Replace DIH 'rss' example with 'atom'
    rss example was broken for multiple reasons.
    atom example showcases the same - and more - features
    and uses the smallest config file needed to make it work.

commit 3e5e6522f0c63abc860da812d59f2f1116746f5b
Author: Chris Hostetter <hossman@apache.org>
Date:   Fri Mar 31 18:16:13 2017 -0700

    SOLR-10399: cleanup unused imports

commit 95bddaf0bb431c2aef7e41b59a4777ecc044f3e2
Author: Chris Hostetter <hossman@apache.org>
Date:   Fri Mar 31 17:01:42 2017 -0700

    SOLR-10399: Generalize some internal facet logic to simplify points/non-points field handling

commit d6cdcceac7a857e983713d9839c4cddfb2e9916c
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Fri Mar 31 18:10:27 2017 +0100

    LUCENE-7763: Remove outdated comment in IndexWriterConfig.setIndexSort javadocs.
    (马可阳 via Christine Poerschke)

commit b865e31397e2a519a34606b493003a5952fafc08
Author: yonik <yonik@apache.org>
Date:   Fri Mar 31 12:55:15 2017 -0400

    SOLR-7452: add more tests for refinement of missing buckets

commit 8d91c30a67b8aca488b3640ec5d996075362828d
Author: yonik <yonik@apache.org>
Date:   Thu Mar 30 12:55:27 2017 -0400

    SOLR-7452: refinement of missing buckets and partial facets through missing buckets

commit 90c02315cb935b29051596061fb4ce7d7383d575
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Mar 31 16:22:45 2017 +0200

    LUCENE-7753: Make fields static when possible.

commit b9dd7bf4456cba88a21ac98f8587a03aa677bac5
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Mar 31 16:11:19 2017 +0200

    LUCENE-7761: Fixed comment in ReqExclScorer.

commit ae421e820ab45328d7229d756b16677d957385c3
Author: markrmiller <markrmiller@apache.org>
Date:   Fri Mar 31 10:53:20 2017 -0400

    SOLR-10338: Configure SecureRandom non blocking for tests.

commit 99ebb56439b29ab0a11c66d3afd9904995e8fe91
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Mar 30 17:52:16 2017 +0100

    SOLR-10351: Fix pre-commit

commit a45f36ef3d3ebdf97149fe405a9980145b79c133
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Mar 30 17:34:28 2017 +0100

    SOLR-10351: Add analyze Stream Evaluator to support streaming NLP

commit 5a401ab5262224052d0abf671a64bac99a5e4d7c
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 30 15:11:52 2017 +0200

    LUCENE-7755: Join queries should not reference IndexReaders.

commit 139f885554dc88863559dd43e7dc153ec256714c
Author: Erick Erickson <erick@apache.org>
Date:   Wed Mar 29 21:13:40 2017 -0700

    SOLR-10387: zkTransfer normalizes destination path incorrectly if source is a windows directory

commit 317ef93a17890c076846a69e1e630d983362aba0
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Wed Mar 29 19:22:02 2017 +0530

    SOLR-10352: Fixing available entropy warning limit to 300

commit 4269c23bd87234ac078bd5ff2f893e2086c15115
Author: Andrzej Bialecki <ab@apache.org>
Date:   Wed Mar 29 14:42:20 2017 +0200

    SOLR-10362 Be more specific when catching this exception.

commit b6b2aac3f63823819d81a9b9d8f9f77fd65c7ced
Author: Mike McCandless <mikemccand@apache.org>
Date:   Wed Mar 29 08:48:26 2017 -0400

    remove dead code

commit 00f2db48f7ebdf5d49181f8b72079897a3999e4a
Author: Jan Høydahl <janhoy@apache.org>
Date:   Wed Mar 29 10:51:34 2017 +0200

    SOLR-10147: Admin UI -> Cloud -> Graph: Impossible to see shard state

commit 3a9ae68230e98c46782e9fb190c4fb37f2517771
Author: Cao Manh Dat <datcm@apache.org>
Date:   Wed Mar 29 13:52:51 2017 +0700

    SOLR-9993: Add support for ExpandComponent with PointFields

commit 25b5d090455cc856d6bda5bd4129f6eefcb725dd
Author: Shai Erera <shaie@apache.org>
Date:   Thu Mar 23 08:28:05 2017 +0200

    SOLR-10349: Add totalTermFreq support to TermsComponent

    TermsComponent only returns docFreq information per requested term.
    This commit adds a terms.ttf parameter, which if set to true, will
    return both docFreq and totalTermFreq statistics for each requested
    term.

commit 90efd090bfa5e8bbf45bb8ccba0997f916b1d2f7
Author: Cao Manh Dat <datcm@apache.org>
Date:   Wed Mar 29 08:09:40 2017 +0700

    SOLR-10079: TestInPlaceUpdates(Distrib|Standalone) failures

commit 83fb5d7758e6c2653386d483ff2cc9495864ad81
Author: yonik <yonik@apache.org>
Date:   Tue Mar 28 19:52:51 2017 -0400

    SOLR-7452: change terminology from _m missing-bucket to _p partial-bucket for refinement

commit 3c2e9c913435bdb685f2bfdcc6e0059e0d5e1c0f
Author: Steve Rowe <sarowe@apache.org>
Date:   Tue Mar 28 18:39:28 2017 -0400

    SOLR-10357: Enable edismax and standard query parsers to handle the option combination sow=false / autoGeneratePhraseQueries=true by setting QueryBuilder.autoGenerateMultiTermSynonymsQuery

commit 2762e31bfe243144f7dbba305c60b66d8c43793e
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Wed Mar 29 00:44:27 2017 +0530

    SOLR-6736: Adding support for uploading zipped configsets using ConfigSets API

commit 94d69590e67259b14efcfebb54f748a381c37a82
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Wed Mar 29 00:26:31 2017 +0530

    SOLR-10365: Handle a SolrCoreInitializationException while publishing core state during SolrCore creation

commit a3d51aebe3724fbcc2162b5fc8862b336595fe57
Author: Joel Bernstein <jbernste@apache.org>
Date:   Tue Mar 28 09:25:25 2017 +0100

    SOLR-10341: SQL AVG function mis-interprets field type

commit 9a927620c0d59801dc633d8aace14aaf2aa86dc3
Author: Steve Rowe <sarowe@gmail.com>
Date:   Tue Mar 28 11:47:02 2017 -0400

    SOLR-10343: Update Solr default/example and test configs to use SynonymGraphFilterFactory

commit c2d8c05387c8085af2ae14e688daa979b5632ff0
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Mar 28 15:25:16 2017 +0200

    LUCENE-7743: Avoid calling new String(String).

commit 4e317da9ee19b17619dfbbf9f69026d61782c8c2
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Mar 28 15:21:35 2017 +0200

    LUCENE-7751: Avoid boxing primitives only to call compareTo.

commit 41f533559db6d442a633c5fcfda14fd11cf027af
Author: Adrien Grand <jpountz@gmail.com>
Date:   Tue Mar 28 15:15:45 2017 +0200

    LUCENE-7754: Inner classes should be static whenever possible.

commit b216a28f720c0cd6342e050a1a7b895248c65a6d
Author: Jan Høydahl <janhoy@apache.org>
Date:   Tue Mar 28 14:24:09 2017 +0200

    SOLR-10369: bin\solr.cmd delete and healthcheck now works again (fixed continuation chars ^)

commit a3eb7488a69f1824a02f9d3e7820f1e9d7f0531c
Author: Steve Rowe <sarowe@gmail.com>
Date:   Mon Mar 27 23:53:55 2017 -0400

    SOLR-10344: Update Solr default/example and test configs to use WordDelimiterGraphFilterFactory

commit b6fcaadbb9ac53557fca901b11af3801cab2ddad
Author: Andrzej Bialecki <ab@apache.org>
Date:   Mon Mar 27 22:17:34 2017 +0200

    SOLR-10362: "Memory Pool not found" error when reporting JVM metrics.

commit 398c9200876cdd0973fb9da0d35fd6765264d50d
Author: Erick Erickson <erick@apache.org>
Date:   Mon Mar 27 12:15:05 2017 -0700

    SLR-10108: bin/solr script recursive copy broken

commit 39584af1f0a3a1ace793b8ace45cb07cdff3edf4
Author: Ishan Chattopadhyaya <ishan@apache.org>
Date:   Mon Mar 27 23:56:23 2017 +0530

    SOLR-10352: bin/solr script now prints warning when available system entropy is lower than 300

commit ce6a309140a76af1a85a88362e53a57f047edc3b
Author: Erick Erickson <erick@apache.org>
Date:   Mon Mar 27 09:31:15 2017 -0700

    SOLR-10371: There is some spelling mistakes in the Java source code Thanks hu xiaodong"

commit e030eacba8f63d66fcfbcd5e94f89463de2a77f0
Author: markrmiller <markrmiller@apache.org>
Date:   Mon Mar 27 10:44:27 2017 -0400

    SOLR-10076: Move changes entry to 6.6 release.

commit 9e8a9df5076e98ca7af2272495a287fc5a916700
Author: Shalin Shekhar Mangar <shalin@apache.org>
Date:   Mon Mar 27 20:06:01 2017 +0530

    SOLR-9835: Fixed precommit failure.

commit 19df35c95f3042c676f311f26c299b3e1921cf0d
Author: Cao Manh Dat <datcm@apache.org>
Date:   Mon Mar 27 16:44:22 2017 +0700

    SOLR-9835: TestInjection.waitForInSyncWithLeader() should rely on commit point of searcher

commit 781c9e7855d4a830c8c9d0c0a5c813e856e8f18d
Author: Jim Ferenczi <jim.ferenczi@elastic.co>
Date:   Sat Mar 25 18:44:39 2017 +0100

    Update project doap files with 6.5.0 release

commit 0545d0886176a511a8dcb72872051f3b3488242e
Author: Jim Ferenczi <jim.ferenczi@elastic.co>
Date:   Sat Mar 25 18:37:25 2017 +0100

    Add 6.5.0 back compat test indexes

commit 51c16b81eca762a2348c9526694fdf07bab7c332
Author: Mike McCandless <mikemccand@apache.org>
Date:   Sat Mar 25 06:21:29 2017 -0400

    fix typo in comment

commit 9c0fc47bb88cf3823dc47ebb9d729b3c5ed3d21e
Author: David Smiley <dsmiley@apache.org>
Date:   Fri Mar 24 23:01:32 2017 -0400

    SOLR-10304: Refactor new SolrDocumentFetcher out of SolrIndexSearcher

commit a67d143cde7ddc26343e6b68f37acf3d6c7b2e3d
Author: David Smiley <dsmiley@apache.org>
Date:   Fri Mar 24 22:46:24 2017 -0400

    SOLR-10249: Refactor IndexFetcher to return detailed result

commit 75f4376e07b540291a22fe9956b50c9cb9160cb2
Author: yonik <yonik@apache.org>
Date:   Fri Mar 24 20:43:44 2017 -0400

    SOLR-7452: add support for _m buckets, missing and has sub-facets in need of refinement

commit c7d33661013543fe32a231d5b18e36e929a95125
Author: Steve Rowe <sarowe@apache.org>
Date:   Fri Mar 24 12:31:16 2017 -0400

    SOLR-9221: Remove Solr contribs: map-reduce, morphlines-core and morphlines-cell

commit d72d707f0406b51153423b19a3387a669467fc51
Author: Chris Hostetter <hossman@apache.org>
Date:   Thu Mar 23 18:28:10 2017 -0700

    Fix test to stop asserting specific order when secondary sort is unspecified, add new checks that do assert an explicit order when secondary sort IS specified.

commit 665d0a954cf981fdbcb5aead4f7bb1b31ec0b75c
Author: Shalin Shekhar Mangar <shalin@apache.org>
Date:   Thu Mar 23 19:33:45 2017 +0530

    SOLR-10281: ADMIN_PATHS is duplicated in two places and inconsistent

commit aefcdd1ab1a1fecbba7c8a31226b48363bb76313
Author: Noble Paul <noble@apache.org>
Date:   Thu Mar 23 18:12:20 2017 +1030

    SOLR-6615: use constants for 'sort', 'distrib'

commit 0e4562d6a29ad072b45ee9aae0d83b5febd5600f
Author: koji <koji@apache.org>
Date:   Thu Mar 23 14:57:45 2017 +0900

    SOLR-9184: Add a static convenience method ModifiableSolrParams#of(SolrParams) which returns the same instance if it already is modifiable, otherwise creates a new ModifiableSolrParams instance.

commit d762464e305b6b0fd70f921dcade41c5c25d1d2f
Author: Noble Paul <noble@apache.org>
Date:   Thu Mar 23 11:45:50 2017 +1030

    SOLR-6615: use constants for 'id', '_route_', '_version_'

commit 3ad6ffe8bffc3eee2ec9320d4f11300451fc3519
Author: yonik <yonik@apache.org>
Date:   Wed Mar 22 19:53:50 2017 -0400

    SOLR-7452: facet refinement - don't generate domain if skipping bucket

commit c913835cb701a4a4743840afe6b3ec28bf5e171b
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Wed Mar 22 10:52:14 2017 -0700

    SOLR-9986: Add javadoc to DatePointField class

commit 6747c82fbf846eaf3471ca58183502e2f818418b
Author: Cao Manh Dat <datcm@apache.org>
Date:   Wed Mar 22 15:00:33 2017 +0700

    Add support for CollapseQParser with PointFields

commit 532829cd79e203ad064ff6841eedba9e5dbd9799
Author: yonik <yonik@apache.org>
Date:   Tue Mar 21 08:42:33 2017 -0400

    SOLR-7452: json facet API, refine/skip through buckets already visited

commit 24caf3e39405137cec787e3f4f6f607aad82d28c
Author: Dennis Gove <dpgove@gmail.com>
Date:   Tue Mar 21 08:40:40 2017 -0400

    SOLR-10333: Fixes use of HashedMap in StreamEvaluator tests

commit 1d0e38d3d0b6f558220473578236daead6faff34
Author: Dennis Gove <dpgove@gmail.com>
Date:   Mon Mar 20 16:36:05 2017 -0400

    SOLR-10292: Adds CartesianProductStream to turn multivalued fields into multiple tuples

commit 21798848c6b3af14d9b43aa2d27eddcec8dd7679
Author: Christine Poerschke <cpoerschke@apache.org>
Date:   Mon Mar 20 19:00:33 2017 +0000

    SOLR-10046: move from 6.5.0 to 6.6.0 CHANGES.txt (backport yet to be completed)

commit fb3fb8c82e0067d53ddb2579acf97dc6a987e99e
Author: Andrzej Bialecki <ab@apache.org>
Date:   Mon Mar 20 19:03:55 2017 +0100

    SOLR-10319 SolrCore "instanceDir" metric not visible in JMX.

commit c18a11fc787cc8139578861e08bb703ddd2955a1
Author: Cao Manh Dat <datcm@apache.org>
Date:   Mon Mar 20 17:30:40 2017 +0700

    SOLR-9992: Update changes.txt

commit 39bb661c9fa69545137c0ae8e14538492b7bb3de
Author: Cao Manh Dat <datcm@apache.org>
Date:   Mon Mar 20 15:21:36 2017 +0700

    SOLR-9992: Add support for grouping with PointFIelds

commit 575535567dd3d13e69014a5f335400298bec4835
Author: Cao Manh Dat <datcm@apache.org>
Date:   Mon Mar 20 09:18:54 2017 +0700

    SOLR-10079: Speedup TestInPlaceUpdatesDistrib in new replication mode

commit c74cd14cc381da34ec52c8f1c09457c921fc4063
Author: Cao Manh Dat <datcm@apache.org>
Date:   Mon Mar 20 08:21:54 2017 +0700

    SOLR-9835: Fix OnlyLeaderIndexesTest failure, inplace updates is not copied over properly

commit 0932284a3b5cfef8e082d92bba4258ce6f337ea3
Author: Steve Rowe <sarowe@apache.org>
Date:   Sat Mar 18 15:09:43 2017 -0400

    LUCENE-7748: buildAndPushRelease.py should fail if the project DOAP files are missing releases that are less than the release being produced

commit 24d3e21f5039426c8c6f297bc71d6c540b2d8db5
Author: David Smiley <dsmiley@apache.org>
Date:   Sat Mar 18 10:42:34 2017 -0400

    SOLR-10286: fix test for Windows

commit 12b8b70cdad7b9f10c8e1e5f5baf17804cde7fd1
Author: Steve Rowe <sarowe@apache.org>
Date:   Sat Mar 18 00:00:59 2017 -0400

    SOLR-10218: The Schema API commands add-field-type and replace-field-type improperly specify SimilarityFactory params

commit cc0f133b50cbf22cfe3b3913c6ac463e85502849
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Fri Mar 17 13:45:29 2017 -0700

    Fix CHANGES.txt

commit 5e59c85583f2aa8adc62470b8008ab8fa1a2f536
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Fri Mar 17 11:55:15 2017 -0700

    SOLR-10237: Poly-Fields should work with subfield that have docValues=true

commit 3b0555639465c651fb731c3068bcd0f95ed35835
Author: yonik <yonik@apache.org>
Date:   Fri Mar 17 12:13:43 2017 -0400

    SOLR-7452: add refine param to json facets, implement for array field faceting

commit a671eb4b42fcb1010cd77e1acfd7c6cacfa859fe
Author: Jim Ferenczi <jimczi@apache.org>
Date:   Fri Mar 17 15:53:21 2017 +0100

    LUCENE_7747: QueryBuilder now iterates lazily over the possible paths when building a graph query

commit 0991eb27a9973692376814b58eab42d53afec40c
Author: Jim Ferenczi <jimczi@apache.org>
Date:   Fri Mar 17 11:55:59 2017 +0100

    Add 6.6 version

commit e6f215ad48bc0e3abef94d8a31b81da7d8d672f3
Author: David Smiley <dsmiley@apache.org>
Date:   Thu Mar 16 21:22:08 2017 -0400

    SOLR-10273: DocumentBuilder move longest field to last position

commit c718e7bdb697f5f2cf4c029cca99b8cfc01974bb
Author: David Smiley <dsmiley@apache.org>
Date:   Thu Mar 16 21:11:39 2017 -0400

    SOLR-10286: fix precommit (unused imports)

commit 754bf0cb0de13b49357b6e52c95af99323a7e1ef
Author: Steve Rowe <sarowe@apache.org>
Date:   Thu Mar 16 19:41:37 2017 -0400

    SOLR-9185: Solr's edismax and Lucene/standard query parsers should optionally not split on whitespace before sending terms to analysis

commit 15f4e69249ab28db4bf377dfd2b960f402b1bfe8
Author: David Smiley <dsmiley@apache.org>
Date:   Thu Mar 16 18:30:57 2017 -0400

    SOLR-10286: fix test; we were writing to read-only dir.
    Expand solrconfig-managed-schema.xml to have toggle-able elements vis system property flags

commit 8baf6bd8cad79974858ce13354d5a6c739e441cd
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Thu Mar 16 15:10:48 2017 -0700

    SOLR-9990: Avoid copyField in SolrExampleTests.testUpdateField

commit 30859ba48727f300a128b398ca263f3198ca7017
Author: David Smiley <dsmiley@apache.org>
Date:   Thu Mar 16 14:58:59 2017 -0400

    SOLR-10286: large fields.
    And refactored FieldType.checkSchemaField to call a new checkSupportsDocValues()

commit 3494d1c610f549f252c77c0a2e322e6293dd0b97
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Mar 16 14:18:43 2017 -0400

    Fixed typos in CHANGES.txt

commit 75a7faaeefd873a961fa48d7a8dd2daa1898c631
Author: Tomas Fernandez Lobbe <tflobbe@apache.org>
Date:   Thu Mar 16 11:08:50 2017 -0700

    SOLR-9990: Add PointFields in example/default schemas

commit 9cc480384205142b79e220fb0b19cb78b6ba1b4c
Author: Joel Bernstein <jbernste@apache.org>
Date:   Thu Mar 16 13:54:25 2017 -0400

    SOLR-10254, 10085: Update CHANGES.txt

commit 7b235fec21c3492d275c1bb98b1edcabd52365d3
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 18:28:22 2017 -0700

    use callouts instead of hackish fake-comments

commit e00da437099d67fa055b62b6dac9b296a9878b08
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 18:19:52 2017 -0700

    cleanup a bunch of [source,foo] tags (in o-z files) where foo was inaccurate

    files found by manually skimming grep -A4 '\[source,.*\]' src/[o-z]*.adoc

commit 00cffbae7ab21364cdea14ecbfb3dc695a2b9757
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 17:40:50 2017 -0700

    cleanup a bunch of [source,foo] tags (in a-n files) where foo was inaccurate

    files found by manually skimming grep -A4 '\[source,.*\]' src/[a-n]*.adoc

commit f7e6565d7fb1e759fbc4c1b607249579a8e6c262
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 17:05:44 2017 -0700

    rename CDCR page so it doesn't end in a silly '-'

commit f6182d8519a5b6204985833e53d41c9530a4b251
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 16:58:29 2017 -0700

    SOLR-10640: move the page.shortname based id to <body> so there is no jitter/scroll when clicking links in HTML, and update CheckLinksAndAnchors to account for it there

commit ccebeecd1e432f87f50bb98a18496fe7cad3688f
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 16:40:28 2017 -0700

    SOLR-10655: add TODO comments noting content that should be refactored into include snippets

commit 04bf430b7398e65385b26997bb9847f4aa052ed6
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 16:28:27 2017 -0700

    removing weird/unneeded src/field-type-definitions-and-properties.pdf

commit 296c2f48eba6a92466659a06b96f7dacb4d05ffd
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 14:04:59 2017 -0700

    use versioned link for online errata

commit 320aedd60aafb406dd83110cf3a43f906af7e361
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 14:02:01 2017 -0700

    Fixup last remaining TODOs/questions regarding cwiki URLs and hardcoded TOCs

commit add32b39ff051330c99f1ec8bc04cc3c4144a061
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue May 9 13:47:28 2017 -0700

    SOLR-10640: change the html page layout to include an id with teh page shortname (just like the pdf effectively does) and remove the HACK in CheckLinksAndAnchors that faked this

    only one page had a dup anchor that needed fixed as a result

commit 96058f824cdb75e61438c0f6e14085f91f6971a5
Author: Cassandra Targett <ctargett@apache.org>
Date:   Tue May 9 14:50:17 2017 -0500

    SOLR-10290: remove confluence-export dir with tools for conversion

commit 07d81790f00e25db28e495e16688cf6d70bfeeb5
Author: Cassandra Targett <ctargett@apache.org>
Date:   Tue May 9 14:00:49 2017 -0500

    SOLR-10296: fix glossary headings; fix callout font used in PDF

commit 8436b4050cbde7f62d3f1d9fbee58c63c334eab7
Author: Cassandra Targett <ctargett@apache.org>
Date:   Tue May 9 09:57:50 2017 -0500

    SOLR-10296: missed a few tables + unescaped http urls

commit a244450b27e1883acbbe13235990b373ce0e885f
Author: Cassandra Targett <ctargett@apache.org>
Date:   Tue May 9 09:48:01 2017 -0500

    SOLR-10296: conversion, table cleanup, last commit

commit 029b600d0c21ee140f5a8383c4dc7afe105e107e
Author: Cassandra Targett <ctargett@apache.org>
Date:   Tue May 9 08:20:45 2017 -0500

    SOLR-10296: conversion, table cleanups

commit 4a0b9b5395f07961ded6ed9e041a8772de690b24
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 21:36:34 2017 -0700

    batch remove OLD_CONFLUENCE_ID comments now that all inter-page links have been fixed

commit 10152ba8cf96c49f3f2e24acf5fbc99dcb508500
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 21:32:03 2017 -0700

    fix broken links/anchors found with improved link checker

commit 53f380125d022c99fa9f1200dc433b5f13f6ff87
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 20:29:26 2017 -0700

    Fix dup anchors found by improved check-links-and-anchors

commit 83b966e2cab5427e6afb2b4e468d4b3e833494a5
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 18:38:12 2017 -0700

    SOLR-10640: make CheckLinksAndAnchors smarter

    now checks for relative links to files/anchors that don't exist

    in the process of adding that, I realize the duplicat ID checking wasn't accounting for the 'implicit' page.shortname ids that are used in the PDF - so some dups were getting over looked

    example: suggester.adoc implicitly has a 'suggester' id, but glossary.adoc might explicitly definie a '[[suggester]]' anchor

commit db902f5c97c12628cb1ef6534e5bea4fe9bebaa4
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 17:06:04 2017 -0700

    beef up TODO with some more details for fixing in future

commit e00e2b8d2e3503422eafb045c6bb6a5fee0152ae
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 16:45:36 2017 -0700

    remove TODOs added by automated conversion that have already been manually dealt with

commit 172bf3963b6c7de4f89b29e6536d5255566b4936
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 16:04:40 2017 -0700

    add the (raw) SVG source for some diagrams

commit 5105c9bba5d00e79cea9ecd712409f4f05d86356
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 15:59:04 2017 -0700

    fix some absolute cwiki.apache.org links, add some TODO questions about anytihng still refering to cwiki.apache.org

commit df8c2597599756c5d572685cb5fbed9622748d07
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 14:54:00 2017 -0700

    update TODO with some thoughts on the errata URLs

commit 218265778182281e2aaea28aac83141f9b6fff6d
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 8 16:25:56 2017 -0500

    SOLR-10296: conversion, table cleanup

commit 09f53281cc6e7a3117a3d05adf282dfdc649237e
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 8 15:19:30 2017 -0500

    SOLR-10296: conversion, table cleanup

commit 02c2d45c15241cd325941c9acd253827c7e4e827
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 8 14:41:38 2017 -0500

    SOLR-10296: conversion, table cleanup

commit 1c3daa748e222a50b6938fdf92a42bdc19681c91
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 8 14:10:18 2017 -0500

    SOLR-10290: add fonts and rules for inline callouts

commit 2953940d2670fc4fad37d1c91ce30fcc2ec12b36
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 12:04:29 2017 -0700

    auto check links anytime html site is built

commit de663b5f98c8c7a37ef0ab4c570e46f990cd1938
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 12:01:44 2017 -0700

    fix anchor/link combo

commit 0bfa433d7ac1b0166069986e971588b2095b9d2d
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 11:54:24 2017 -0700

    Fix link checker to accept some absolute URLs that aren't valid URIs

commit 58c717cfe40e9f8b575d782210d624a5726f112a
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 11:34:12 2017 -0700

    Fix some duplicate anchors that were breaking check-links-and-anchors

commit 3de6709c9be74bcaca7933f2375d20e2ed54dad0
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 11:24:20 2017 -0700

    replace problematic footnoteref usage w/simple anchors/links

commit 33e8df25427276b0496e5f9882756678f0ef3f3c
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 10:35:22 2017 -0700

    Fix formatting glitches found via: egrep '' src/*.adoc

commit bd805af6354f662af7df6f5d67cee4f3ed6143aa
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 8 11:50:03 2017 -0500

    SOLR-10296: conversion, solr-glossary

commit ceabf15b458b9b34738b61209d2cf7c218ef76eb
Author: Chris Hostetter <hossman@apache.org>
Date:   Mon May 8 09:49:44 2017 -0700

    upgrade asciidoctor-ant = 1.6.0-alpha.3 and fix logging jar dep that was overlooked before

commit d05e3a4066fcf7446479a233f26a254c970fb95c
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 8 10:23:43 2017 -0500

    SOLR-10296: conversion, remaining letter S minus solr-glossary

commit 3f9dc385915c69dce15d224898908e8c25b26c3a
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 19:47:01 2017 -0500

    SOLR-10296: conversion, letter S part 1

commit ff9fdcf1fcfb68008064e902f7cca8c776516323
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 18:50:42 2017 -0500

    SOLR-10296: add glossary attribute to CDCR Glossary

commit f7859d7f32df43308b1d82914523d90723b415a4
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 18:46:57 2017 -0500

    SOLR-10296: conversion, letter R

commit e53c64a3508a2a2fd7c01b085d26f39cf89af03e
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 17:18:47 2017 -0500

    SOLR-10296: conversion, letter Q

commit 53b3eaac6073e817dd9cc85d9b6f51cc873fb2a7
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 17:00:47 2017 -0500

    SOLR-10296: conversion, letter P

commit 26a571e0bf22de52caced8d72f7badb1afbcb0b4
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 15:47:29 2017 -0500

    SOLR-10296: conversion, letter O

commit 0ffdf3d543dc009caaf10b616cc9ee8c84ef2dac
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 14:48:04 2017 -0500

    SOLR-10296: conversion, letters M + N

commit 7d7fb52ab134a1f4227cab448c5a82890ef4e855
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 14:03:53 2017 -0500

    SOLR-10296: conversion, letter L; some other cleanups

commit c4b547c55a0461f8a7d4191a0e233d8a59d113de
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 11:59:51 2017 -0500

    SOLR-10296: conversion, letters J + K

commit d77278df6bb7d0639c18d2cc6f8cee3ff9196c96
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 11:12:50 2017 -0500

    SOLR-10296: conversion, letter I

commit 8916dd05af036ac58bc0754c994e62e45c808249
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sun May 7 10:43:43 2017 -0500

    SOLR-10296: fix tables that declare specific column percentages

commit 7f1990c56950497a086fdfcbbccc5f3e16684cfd
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sat May 6 14:38:26 2017 -0500

    SOLR-10296: conversion, letter H

commit a967d5a392e3cda08a6699dd751f597b8473f71e
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sat May 6 13:58:00 2017 -0500

    SOLR-10296: conversion, letter G + some heading level cleanups

commit f060417a8345de17fa2c1c3e210da46ae5b6599e
Author: Cassandra Targett <ctargett@apache.org>
Date:   Sat May 6 08:09:41 2017 -0500

    SOLR-10296: conversion, done with letter F

commit ccf5dd28af555a35b3938ad8948afa6822887fbc
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri May 5 22:42:37 2017 -0500

    SOLR-10296: conversion, letter F, part 1

commit c0cc260ced47238e1342aa62ff9263ec48fe5f02
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri May 5 21:19:55 2017 -0500

    SOLR-10296: conversion, letter E

commit 645908e293298ccce31988a717dfb122c3bf3611
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri May 5 20:42:53 2017 -0500

    SOLR-10296: content conversion, letter D

commit 7905416153ff5726d97fde2774f13224c42759c1
Author: Chris Hostetter <hossman@apache.org>
Date:   Fri May 5 17:16:57 2017 -0700

    manual cleanup of the T files

commit 287ffe43c78e1cdccffb0d14c6f6cf78a0141b5e
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri May 5 18:51:16 2017 -0500

    SOLR-10296: conversion - C's

commit 7c2e7ce8c3d5f66b476e8167354f581667bbb88d
Author: Chris Hostetter <hossman@apache.org>
Date:   Fri May 5 15:07:15 2017 -0700

    manual cleanup of the remaining u files

commit bbe60af21393f6c8b59bdcae9c79a28a10545aa0
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri May 5 15:52:23 2017 -0500

    SOLR-10290: content conversion: B's and some C's

commit c7361afb83f3ae477c9f9a9fcbd901157735d390
Author: Chris Hostetter <hossman@apache.org>
Date:   Fri May 5 11:21:34 2017 -0700

    manual cleanup of using-* pages

commit 4697d96ca4bb0c6273865c88582907d2995c7d23
Author: Chris Hostetter <hossman@apache.org>
Date:   Fri May 5 10:30:46 2017 -0700

    manual cleanup of w pages

commit bf84e407e6111a694f689cd2a7fc34c1376d9b17
Author: Chris Hostetter <hossman@apache.org>
Date:   Fri May 5 10:09:26 2017 -0700

    manual cleanup of w-z

commit adf8959db733f516f81b5cb1b43e0b13218c4f53
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri May 5 12:01:52 2017 -0500

    SOLR-10290: content conversion, letter A

commit 4dca53bf59e138b90db7b89c7cf9ca37fcadbe46
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri May 5 09:43:37 2017 -0500

    SOLR-10290: commit changed pages since last Confluence export

commit 72e0d34bfdc3dbd8d0af51f1b55904d3b82fb215
Author: Cassandra Targett <ctargett@apache.org>
Date:   Wed May 3 13:35:24 2017 -0500

    SOLR-10295: Use index.html; fix pub process; add nav for PDFs

commit 737464ab897a424c2f6677e13d794ab136043de3
Author: Cassandra Targett <ctargett@apache.org>
Date:   Wed May 3 13:34:14 2017 -0500

    SOLR-10295: Use index.html; add nav for PDF versions

commit 7bd15b902a78756e333aa9db8363d7178aa53ac3
Author: Cassandra Targett <ctargett@apache.org>
Date:   Wed May 3 08:38:44 2017 -0500

    SOLR-10295: typos

commit 35e1acbfb537cf5540f5e576092664c8783fb29e
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 1 16:08:57 2017 -0500

    SOLR-10593: remove external link icons in comment threads

commit 1664ab87cf09f12a6f547425f05f4bd30f1af735
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 1 15:36:01 2017 -0500

    SOLR-10593: add comments.css to head include file so comments are styled properly

commit 276db77bf30e3c23f86ac8d5107014f84d2b7d16
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 1 14:55:06 2017 -0500

    SOLR-10593: change to "solr-refguide" site for comments

commit 42871f86bdaa12cc5fe68670a1145ee36d5571f4
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 1 14:33:48 2017 -0500

    SOLR-10301: small edits to how to write in asciidoc

commit 8b5f9693354b9019caf43c98badaf496ab9361e2
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon May 1 11:26:08 2017 -0500

    SOLR-10295: Add PDF instructions to publish guide

commit e375e250071a793a647926e1f0ab40d1e26031a0
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri Apr 28 15:06:13 2017 -0500

    SOLR-10295: missed some edits

commit 43e70c3f8f2c9612de359b29ef7127a81672ac5c
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri Apr 28 14:36:29 2017 -0500

    SOLR-10295: add re-publish process

commit f511fbf45a092f04829db14b24e6737e6f8a70fa
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri Apr 28 08:17:40 2017 -0500

    SOLR-10295: update publication process

commit 49c92bf03deda0e29c9a0e774ce05cf0bfbe71c5
Author: Steve Rowe <sarowe@apache.org>
Date:   Thu Apr 27 14:24:38 2017 -0400

    SOLR-10290: Added pygments.rb to list of required gems.  Fixed build directories.

commit a56e8c9b73c1a44b791db3c4b9f5840fa32b73c2
Author: Cassandra Targett <ctargett@apache.org>
Date:   Wed Apr 26 15:55:15 2017 -0500

    SOLR-10295: clarify some instructions; add more TODOs

commit 81c8215ce5dad3d5f06ad07fa9314133b9d0a542
Author: Cassandra Targett <ctargett@apache.org>
Date:   Wed Apr 26 14:13:45 2017 -0500

    SOLR-10295: strawman process for publishing HTML version

commit 73148be0baab123b93953f98d69d3b4517842f03
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Apr 20 14:35:53 2017 -0500

    SOLR-10290: update raw content files

commit 201d238ad61dfbbf5d7feb7d1750256fb67f39d3
Author: Cassandra Targett <ctargett@apache.org>
Date:   Wed Apr 5 17:47:56 2017 -0500

    SOLR-10290: remove stray pipe in feed.xml

commit d44e9ed8727844eaae4e7cba98d3f1e6639c7b87
Author: Cassandra Targett <ctargett@apache.org>
Date:   Wed Apr 5 15:14:15 2017 -0500

    SOLR-10422: Remove license attribution for now-unused Raleway font

commit 79e103756efc214c1ff3fca6b58c7621e077229f
Author: Cassandra Targett <ctargett@apache.org>
Date:   Wed Apr 5 14:49:16 2017 -0500

    SOLR-10290: Fix font-family references for consistency

commit 34829c59cfbf615c40e0717a59f6d25d5290a845
Author: Cassandra Targett <ctargett@apache.org>
Date:   Wed Apr 5 14:47:09 2017 -0500

    SOLR-10422: remove pdf/fonts directory; change HTML format to use Inconsolata & Noto Sans; other font-related dir cleanups

commit 6472b196372b387a43920781d3b2aad1d1d47544
Author: Cassandra Targett <ctargett@apache.org>
Date:   Tue Apr 4 15:52:58 2017 -0500

    SOLR-10290: get rid of unnecessary duplicate font directory

commit c6d29e33e41435824e217acf66a02ea3b14be7de
Author: Cassandra Targett <ctargett@apache.org>
Date:   Tue Apr 4 15:29:07 2017 -0500

    SOLR-10298: replace Noto Serif with Noto Sans, to reduce font size and line height and keep readability

commit d14977553f2597d84f907011a5c7275bfb79be77
Author: Chris Hostetter <hossman@apache.org>
Date:   Fri Mar 31 11:09:26 2017 -0700

    SOLR-10298: new pdfbox based tool to reduce PDF size (via FLATE_DECODE)

commit ec2cbb3eee23d94619e7c9e7e2b45fb77da61394
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri Mar 31 11:24:52 2017 -0500

    SOLR-10290: Fix feed.xml in case we want to have it in the online site; ensure URL is right everywhere

commit 5c29efc41a0213908f1b0ebd6d40c09927f6832f
Author: Cassandra Targett <ctargett@apache.org>
Date:   Fri Mar 31 11:23:34 2017 -0500

    SOLR-10290: Clean up _config.yml.template: update description; remove unused settings; move comments to above the thing they describe

commit 8071d3f4a2aac2b8a3b2e38bc1896f507a7e7763
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 30 15:04:32 2017 -0500

    Small edits to PDF meta-docs

commit 3b592f1be5f803c33fc170f6bf54b78e0597bef3
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 30 15:03:53 2017 -0500

    SOLR-10346: Remove topnav.yml data file & hard code top navigation into topnav template

commit 9bb9568b65cfec495d3effb570a65ba84fd2d9a3
Author: Cassandra Targett <ctargett@apache.org>
Date:   Tue Mar 28 15:58:47 2017 -0500

    SOLR-10300: add docs on building PDF and HTML versions

commit a214ff8c79a19f1babbea530caf2536a7e7f425e
Author: Cassandra Targett <ctargett@apache.org>
Date:   Tue Mar 28 15:58:00 2017 -0500

    SOLR-10298: remove some PDF build params that are not used

commit 017b890f7cdde1e1c7ec4b73ed8943db7eff370d
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed Mar 22 11:22:19 2017 -0700

    SOLR-10342: use ivy-version.properties and std build paths for ref-guide

commit 183bfbeea69c7f1c9d481440a7ccac21b18ee1fd
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed Mar 22 10:12:26 2017 -0700

    SOLR-10342: license headers

commit 85bb18003830846b036751ae3137a1efacaaa79d
Author: Chris Hostetter <hossman@apache.org>
Date:   Wed Mar 22 10:08:17 2017 -0700

    SOLR-10342: use common-build.xml for variables

commit 853c4ae0467607d508d8b3a13b60ad4414ef7234
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon Mar 20 16:17:45 2017 -0500

    SOLR-10300, SOLR-10301: add meta-docs directory and first pass at Asciidoc syntax & tools

commit 0e6f187daed4e571c6fa7f438a5ce7e9fd53ea54
Author: Cassandra Targett <ctargett@apache.org>
Date:   Mon Mar 20 10:20:31 2017 -0500

    Remove errant .DS_Store file

commit 46251028f4ca21305528b4a384aba64dbfc4d2cf
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 13:07:50 2017 -0500

    SOLR-10290: Add remaining files needed for complete builds

commit 08e6c9e99e426d3269af9cd3bb9697fd78c2da81
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 13:03:45 2017 -0500

    SOLR-10290: Really last batch of images

commit 66e7d9be95cde5a51716128ba1a33571132ff76d
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 13:00:58 2017 -0500

    SOLR-10290: Add last batch of images

commit 7012478b8bc1118f9acc0c1e34cc9b735a54e6a3
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 13:00:11 2017 -0500

    SOLR-10290: Add more images

commit fca9f3681452466941d7107d8813ba0c1e04edbb
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 12:59:10 2017 -0500

    SOLR-10290: Add more images

commit 7e8fd9ac0b87a5bdbcc589ec9c6f747561e9e7bc
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 12:57:04 2017 -0500

    SOLR-10290: Add some more images

commit 000fe8090da9c1d953fb0927b612a76de4f9ccf0
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 12:55:58 2017 -0500

    SOLR-10290: Add some of the images

commit 45a148a74805ae4723edab8230b1d2ba1f62284b
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 12:28:21 2017 -0500

    SOLR-10290: Add .adoc files

commit 2a6997b027ffa4cacab55ac8eb66d178b131a2fb
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 12:24:58 2017 -0500

    SOLR-10290: Add config.yml template for Jekyll build

commit 71001667e2781cedee3bc92f985b24fc3c99c3e3
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 12:22:35 2017 -0500

    SOLR-10290: Add files for PDF build

commit 659494333e2f0aa1700692e082a6a4f94e07720e
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 12:12:12 2017 -0500

    SOLR-10290: Add files needed for Jekyll HTML build

commit ec324b294ce858733dd014399a27ccb2cb513def
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 11:51:06 2017 -0500

    SOLR-10290: Add ivy and build files, etc.

commit 8736246ee51e8fbf08ee8d08a1dc22cdeba50d97
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 11:49:12 2017 -0500

    SOLR-10290: Add Confluence conversion tools

commit e825f0a7501802bd79004ed0ac44b62eba59954d
Author: Cassandra Targett <ctargett@apache.org>
Date:   Thu Mar 16 10:54:18 2017 -0500

    SOLR-10290: Initial commit of ref guide tools
This commit is contained in:
Chris Hostetter 2017-05-10 14:40:25 -07:00
parent 69783f6403
commit 95968c69fd
370 changed files with 51005 additions and 2 deletions

View File

@ -40,6 +40,7 @@ com.sun.jersey.version = 1.9
/com.sun.mail/javax.mail = 1.5.1
/com.tdunning/t-digest = 3.1
/com.vaadin.external.google/android-json = 0.0.20131108.vaadin1
/commons-beanutils/commons-beanutils = 1.8.3
/commons-cli/commons-cli = 1.2
/commons-codec/commons-codec = 1.10
@ -145,7 +146,7 @@ org.apache.hadoop.version = 2.7.2
/org.apache.htrace/htrace-core = 3.2.0-incubating
# The httpcore version is often different from the httpclient and httpmime versions,
# so the httpcore version value should not share the same symbolic name with them.
# so the httpcore version value should not share the same symbolic name with them.
/org.apache.httpcomponents/httpclient = 4.4.1
/org.apache.httpcomponents/httpcore = 4.4.1
/org.apache.httpcomponents/httpmime = 4.4.1
@ -187,6 +188,11 @@ org.apache.uima.version = 2.3.1
/org.apache.velocity/velocity-tools = 2.0
/org.apache.xmlbeans/xmlbeans = 2.6.0
/org.apache.zookeeper/zookeeper = 3.4.10
# v1.6.0-alpha.3 of asciidoctor-ant includes asciidoctorj-pdf 1.5.0-alpha.15,
# which is the same as asciidoctor-pdf 1.5.0-alpha.15
/org.asciidoctor/asciidoctor-ant = 1.6.0-alpha.3
/org.aspectj/aspectjrt = 1.8.0
org.bouncycastle.version = 1.45
@ -236,6 +242,8 @@ org.gagravarr.vorbis.java.version = 0.8
/org.gagravarr/vorbis-java-core = ${org.gagravarr.vorbis.java.version}
/org.gagravarr/vorbis-java-tika = ${org.gagravarr.vorbis.java.version}
/org.jsoup/jsoup = 1.8.2
/org.locationtech.spatial4j/spatial4j = 0.6
/org.mockito/mockito-core = 2.6.2
@ -262,6 +270,7 @@ org.slf4j.version = 1.7.7
/org.slf4j/jul-to-slf4j = ${org.slf4j.version}
/org.slf4j/slf4j-api = ${org.slf4j.version}
/org.slf4j/slf4j-log4j12 = ${org.slf4j.version}
/org.slf4j/slf4j-simple = ${org.slf4j.version}
/org.tukaani/xz = 1.5
/rome/rome = 1.0
@ -270,4 +279,3 @@ ua.net.nlp.morfologik-ukrainian-search.version = 3.7.5
/ua.net.nlp/morfologik-ukrainian-search = ${ua.net.nlp.morfologik-ukrainian-search.version}
/xerces/xercesImpl = 2.9.1

View File

@ -656,6 +656,9 @@
<ant dir="test-framework" target="resolve" inheritall="false">
<propertyset refid="uptodate.and.compiled.properties"/>
</ant>
<ant dir="solr-ref-guide" target="resolve" inheritall="false">
<propertyset refid="uptodate.and.compiled.properties"/>
</ant>
<contrib-crawl target="resolve"/>
</sequential>
</target>

View File

@ -0,0 +1 @@
fa26d351fe62a6a17f5cda1287c1c6110dec413f

View File

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

View File

@ -0,0 +1 @@
a7068544963ed46839c8352eddd87271fa93b967

View File

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@ -0,0 +1,5 @@
Apache [asciidoctor-ant]
Copyright [2013] The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

View File

@ -0,0 +1 @@
64238922c4006c3d0a9951c4c03983ecc6a1e1a0

View File

@ -0,0 +1,21 @@
The MIT License
© 2009-2017, Jonathan Hedley <jonathan@hedley.net>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

View File

@ -0,0 +1 @@
8095d0b9f7e0a9cd79a663c740e0f8fb31d0e2c8

View File

@ -0,0 +1,27 @@
= Solr Ref Guide
This is the source for the Solr Reference Guide.
Raw content is stored in Asciidoc (`.adoc`) formated files in the `src/` directory.
These files are processed with AsciiDoctor in 2 different ways:
* Via 'Jekyll' to build an HTML browsable version of the Ref Guide
** Prerequisites: `Ruby` and the following gems must be installed:
*** `jekyll`
*** `jekyll-asciidoc`
*** `pygments.rb`
* Via `asciidoctor-ant` to build the officially released PDF version of the Ref Guide
** Prerequisites: None (except for those required to use the Lucene/Solr build: Java, Ant)
For details on building the ref guide, see `ant -p`.
Key directories to be aware of:
* `src` - where all human edited `*.adoc` files realted to the Guide live, as well as various configuration, theme, and template files.
* `tools` - custom Java code for parsing metadata in our `src/*.adoc` files to produce some `_data/` files for site & pdf navigation purposes.
* `../build/solr-ref-guide/content` - a copy of the `src` dir generated by ant where:
** `*.template` files are processed to replace ant properties with their runtime values
** some `../build/solr-ref-guide/content/_data` files are generated by our java tools based header attributes from each of the `*.adoc` files
* `../build/solr-ref-guide/html-site` - HTML generated version of the ref guide
* `../build/solr-ref-guide/apache-solr-ref-guide-X.Y.pdf` - PDF generated version of the ref guide

View File

@ -0,0 +1,204 @@
<?xml version="1.0"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<project name="solr-ref-guide" default="default" xmlns:asciidoctor="antlib:org.asciidoctor.ant" >
<import file="../common-build.xml"/>
<!-- properties to use in our docs -->
<loadresource property="solr-docs-version">
<propertyresource name="version.base"/>
<filterchain>
<tokenfilter>
<filetokenizer/>
<replaceregex pattern="^(\d+\.\d+)(|\..*)$" replace="\1" flags="s"/>
</tokenfilter>
</filterchain>
</loadresource>
<loadresource property="solr-docs-version-path">
<propertyresource name="solr-docs-version"/>
<filterchain>
<tokenfilter>
<filetokenizer/>
<replaceregex pattern="^(\d+)\.(\d+)(|\..*)$" replace="\1_\2_0" flags="s"/>
</tokenfilter>
</filterchain>
</loadresource>
<property name="solr-javadocs" value="https://lucene.apache.org/solr/${solr-docs-version-path}/" />
<property name="lucene-javadocs" value="https://lucene.apache.org/core/${solr-docs-version-path}/" />
<property name="build.content.dir" location="${build.dir}/content" />
<property name="main-page" value="index" />
<property name="pdf-filename" value="apache-solr-ref-guide-${solr-docs-version}.pdf" />
<!-- ====== TOOLS FOR GENERATING/VALIDATING BITS OF THE SITE / PDF ======= -->
<property name="tools-jar-name" value="solr-ref-guide-tools.jar" />
<path id="tools-compile-classpath">
<fileset dir="lib">
<include name="**/*.jar"/>
<exclude name="**/${tools-jar-name}" />
</fileset>
</path>
<path id="tools-run-classpath">
<fileset dir="lib">
<include name="**/*.jar"/>
</fileset>
<fileset dir="${build.dir}">
<include name="**/*.jar"/>
</fileset>
</path>
<target name="clean">
<delete dir="${build.dir}"/>
</target>
<target name="build-tools-jar" depends="resolve" description="Builds the custom java tools use use for generating some data files from page metdata">
<mkdir dir="${build.dir}/classes"/>
<javac debug="yes"
debuglevel="source,lines,vars"
destdir="${build.dir}/classes"
includeantruntime="false">
<compilerarg value="-Xlint:all"/>
<classpath refid="tools-compile-classpath"/>
<src path="tools/"/>
</javac>
<jar destfile="${build.dir}/${tools-jar-name}">
<fileset dir="${build.dir}/classes"
includes="**/*.class"/>
</jar>
</target>
<target name="build-init" description="Prepares the build's 'content' dir, copying over src files and transforming *.template files in the process">
<delete dir="${build.content.dir}" />
<mkdir dir="${build.content.dir}" />
<echo>Copying all non template files from src ...</echo>
<copy todir="${build.content.dir}">
<fileset dir="src">
<exclude name="**/*.template"/>
</fileset>
</copy>
<echo>Copy (w/prop replacement) any template files from src...</echo>
<copy todir="${build.content.dir}">
<fileset dir="src">
<include name="**/*.template"/>
</fileset>
<mapper type="glob" from="*.template" to="*"/>
<filterchain>
<expandproperties/>
</filterchain>
</copy>
</target>
<target name="build-nav-data-files" depends="build-init,build-tools-jar" description="creates nav based data files needed by both the html and pdf artifacts">
<mkdir dir="${build.content.dir}/_data"/>
<java classname="BuildNavAndPDFBody"
failonerror="true"
fork="true">
<classpath refid="tools-run-classpath"/>
<arg value="${build.content.dir}"/>
<arg value="${main-page}"/>
</java>
</target>
<target name="check-links-and-anchors" depends="build-init,build-tools-jar" description="Parse the HTML site files to check for problematic links or anchors">
<java classname="CheckLinksAndAnchors"
failonerror="true"
fork="true">
<classpath refid="tools-run-classpath"/>
<arg value="${build.dir}/html-site"/>
</java>
</target>
<!-- ====== PDF Build ======= -->
<target name="build-pdf" depends="-build-raw-pdf,-reduce-pdf-size" description="Builds a PDF">
<echo>Finished Building ${build.dir}/${pdf-filename}</echo>
</target>
<target name="-build-raw-pdf"
depends="build-nav-data-files,resolve">
<mkdir dir="${build.dir}/pdf-tmp"/>
<taskdef uri="antlib:org.asciidoctor.ant" resource="org/asciidoctor/ant/antlib.xml"
classpathref="tools-run-classpath"/>
<asciidoctor:convert
sourceDirectory="${build.content.dir}/pdf"
sourceDocumentName="SolrRefGuide-all.adoc"
baseDir="${build.content.dir}"
outputDirectory="${build.dir}/pdf-tmp"
backend="pdf"
extensions="adoc"
sourceHighlighter="coderay"
imagesDir="${build.content.dir}"
doctype="book"
safemode="unsafe">
<attribute key="icons" value="font" />
<attribute key="icon-set" value="fa" />
<attribute key="pdf-stylesDir" value="./pdf/themes"/>
<attribute key="pdf-style" value="refguide"/>
<attribute key="pdf-fontsDir" value="./fonts"/>
<attribute key="figure-caption!" value='' />
<attribute key="idprefix" value='' />
<attribute key="idseparator" value='-' />
<!-- attributes used in adoc files -->
<!-- NOTE: If you add any attributes here for use in adoc files, you almost certainly need to also add
them to the _config.yml.template file for building the jekyll site as well
-->
<attribute key="solr-docs-version" value="${solr-docs-version}" />
<attribute key="solr-javadocs" value="${solr-javadocs}" />
<attribute key="lucene-javadocs" value="${lucene-javadocs}" />
<attribute key="build-date" value="${DSTAMP}" />
<attribute key="build-year" value="${current.year}" />
</asciidoctor:convert>
<move file="${build.dir}/pdf-tmp/SolrRefGuide-all.pdf" tofile="${build.dir}/pdf-tmp/RAW-${pdf-filename}" />
</target>
<target name="-reduce-pdf-size" depends="build-init,build-tools-jar">
<java classname="ReducePDFSize"
failonerror="true"
fork="true">
<classpath refid="tools-run-classpath"/>
<arg value="${build.dir}/pdf-tmp/RAW-${pdf-filename}"/>
<arg value="${build.dir}/${pdf-filename}"/>
</java>
</target>
<!-- ======= HTML Site Build =======
Builds site with Jekyll.
This (for now) assumes that Jekyll (http://jekyllrb.com) is installed locally. -->
<target name="build-site"
depends="-build-site,check-links-and-anchors"
description="Builds an HTML Site w/Jekyll and verifies the anchors+links are valid" >
<echo>Ready to browse site: ${build.dir}/html-site/${main-page}.html</echo>
</target>
<target name="-build-site"
depends="build-init,build-nav-data-files"
description="Builds an HTML Site w/Jekyll">
<echo>Running Jekyll...</echo>
<exec executable="jekyll" dir="${build.content.dir}">
<arg value="build"/>
</exec>
</target>
<target name="default"
description="Builds both a PDF and HTML versions of the ref guide"
depends="build-pdf,build-site">
<echo>PDF: ${build.dir}/${pdf-filename}</echo>
<echo>SITE: ${build.dir}/html-site/${main-page}.html</echo>
</target>
</project>

View File

@ -0,0 +1,34 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<ivy-module version="2.0">
<info organisation="org.apache.solr" module="ref-guide-tools"/>
<configurations defaultconfmapping="compile->master">
<conf name="compile" transitive="false" />
</configurations>
<dependencies>
<dependency org="org.asciidoctor" name="asciidoctor-ant" rev="${/org.asciidoctor/asciidoctor-ant}" conf="compile" />
<dependency org="com.vaadin.external.google" name="android-json" rev="${/com.vaadin.external.google/android-json}" conf="compile" />
<dependency org="org.jsoup" name="jsoup" rev="${/org.jsoup/jsoup}" conf="compile" />
<dependency org="org.apache.pdfbox" name="pdfbox" rev="${/org.apache.pdfbox/pdfbox}" conf="compile"/>
<dependency org="org.slf4j" name="jcl-over-slf4j" rev="${/org.slf4j/jcl-over-slf4j}" conf="compile"/>
<dependency org="org.slf4j" name="slf4j-api" rev="${/org.slf4j/slf4j-api}" conf="compile"/>
<dependency org="org.slf4j" name="slf4j-simple" rev="${/org.slf4j/slf4j-simple}" conf="compile"/>
<dependency org="log4j" name="log4j" rev="${/log4j/log4j}" conf="compile"/>
</dependencies>
</ivy-module>

View File

@ -0,0 +1,223 @@
= Asciidoc syntax
:toc:
The definitive manual on Asciidoc syntax is in the http://asciidoctor.org/docs/user-manual/[Asciidoctor User Manual]. To help people get started, however, here is a simpler cheat sheet.
== Basic syntax
=== Bold
Put asterisks around text to make it *bold*.
More info: http://asciidoctor.org/docs/user-manual/#bold-and-italic
=== Italics
Use underlines on either side of a string to put text into _italics_.
More info: http://asciidoctor.org/docs/user-manual/#bold-and-italic
=== Headings
Equal signs (`=`) are used for heading levels. Each equal sign is a level. Each page can *only* have one top level. Levels should be appropriately nested.
Validation occurs to ensure that level 3s are preceded by level 2s, level 4s are preceded by level 3s, etc. Including out-of-sequence heading levels (such as a level 3 then a level 5) will not fail the build, but will produce an error.
More info: http://asciidoctor.org/docs/user-manual/#sections
=== Code Examples
Use backticks ``` for text that should be monospaced, such as a code or class name in the body of a paragraph.
More info: http://asciidoctor.org/docs/user-manual/#mono
Longer code examples can be separated from text with `source` blocks. These allow defining the syntax being used so the code is properly highlighted.
[source]
.Example Source Block
----
[source,xml]
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
----
If your code block will include line breaks, put 4 hyphens (`----`) before and after the entire block.
More info: http://asciidoctor.org/docs/user-manual/#source-code-blocks
=== Block Titles
Titles can be added to most blocks (images, source blocks, tables, etc.) by simply prefacing the title with a period (`.`). For example, to add a title to the source block example above:
[source]
----
[source,xml]
.Example ID field
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
----
== Links
=== Link to Sites on the Internet
When converting content to HTML or PDF, Asciidoctor will automatically render many link types (such as `http:` and `mailto:`) without any additional syntax.
However, you can add a name to a link by adding the URI followed by square brackets:
[source]
http://lucene.apache.org/solr[Solr Website]
=== Link to Other Pages/Sections of the Guide
A warning up front, linking to other pages can be a little bit painful.
To link to an anchor (or heading) on the _same page_, you can simply use double angle brackets (`<< >>`) around the anchor/heading/section name. Note that any heading (aka section title) that starts with equal signs is automatically an anchor.
More info: http://asciidoctor.org/docs/user-manual/#internal-cross-references
To link to another page or even a heading on _another page_, you use a similar syntax, but you must refer to the full filename and refer to the section title you want to link to by changing it to lower case and including hyphens between each word.
For example, if I want to link from one page to the Schema API page's section "Modify the Schema", I need to create a link that looks like this:
[source]
For more information about modifying the schema with the API, see the section <<schema-api.adoc#modify-the-schema>>.
You can add text to appear by adding it after the file name and section reference, as in:
[source]
<<schema-api.adoc#modify-the-schema,Modify the Schema>>
More info: http://asciidoctor.org/docs/user-manual/#inter-document-cross-references
== Item Lists
Asciidoc supports three types of lists:
* Unordered lists
* Ordered lists
* Labeled lists
Each type of list can be mixed with the other types. So, you could have an ordered list inside a labeled list if necessary.
=== Unordered Lists
Simple bulleted lists need each line to start with an asterisk (`*`). It should be the first character of the line, and be followed by a space.
More info: http://asciidoctor.org/docs/user-manual/#unordered-lists
=== Ordered Lists
Numbered lists need each line to start with a period (`.`). It should be the first character of the line, and be followed by a space.
More info: http://asciidoctor.org/docs/user-manual/#ordered-lists
=== Labeled Lists
These are like question & answer lists or glossary definitions. Each line should start with the list item followed by double colons (`::`), then a space or new line.
Labeled lists can be nested by adding an additional colon (such as `:::`, etc.).
More info: http://asciidoctor.org/docs/user-manual/#labeled-list
== Images
There are two ways to include an image: inline or as a block.
Inline images are those where text will flow around the image. Block images are those that appear on their own line, set off from any other text on the page.
Both approaches use the `image` tag before the image filename, but the number of colons after `image` define if it is inline or a block. Inline images use one colon (`image:`), while block images use two colons (`image::`).
Block images automatically include a caption label and a number (such as `Figure 1`). If a block image includes a title, it will be included as the text of the caption.
Optional attributes allow you to set the alt text, the size of the image, if it should be a link, float and alignment.
More info: http://asciidoctor.org/docs/user-manual/#images
== Tables
Tables can be complex, but it is pretty easy to make a basic table that fits most needs.
=== Basic Tables
The basic structure of a table is similar to Markdown, with pipes (`|`) delimiting columns between rows:
[source]
----
|===
| col 1 row 1 | col 2 row 1|
| col 1 row 2 | col 2 row 2|
|===
----
Note the use of `|===` at the start and end. For basic tables that's not exactly required, but it does help to delimit the start and end of the table in case you accidentally introduce (or maybe prefer) spaces between the rows.
=== Header Rows
To add a header to a table, you need only set the `header` attribute at the start of the table:
[source]
----
[options="header"]
|===
| header col 1 | header col 2|
| col 1 row 1 | col 2 row 1|
| col 1 row 2 | col 2 row 2|
|===
----
=== Defining Column Styles
If you need to define specific styles to all rows in a column, you can do so with the attributes.
This example will center all content in all rows:
[source]
----
[cols="2*^" options="header"]
|===
| header col 1 | header col 2|
| col 1 row 1 | col 2 row 1|
| col 1 row 2 | col 2 row 2|
|===
----
Alignments or any other styles can be applied only to a specific column. For example, this would only center the last column of the table:
[source]
----
[cols="2*,^" options="header"]
|===
| header col 1 | header col 2|
| col 1 row 1 | col 2 row 1|
| col 1 row 2 | col 2 row 2|
|===
----
Many more examples of formatting:
* Columns: http://asciidoctor.org/docs/user-manual/#cols-format
* Cells: http://asciidoctor.org/docs/user-manual/#cell
=== More Options
Tables can also be given footer rows, borders, and captions. CSV or DSV can be used instead of formatting the data in pipes.
More info: http://asciidoctor.org/docs/user-manual/#tables
== Admonitions (Notes, Warnings)
Asciidoc supports several types of callout boxes, called "admonitions":
* NOTE
* TIP
* IMPORTANT
* CAUTION
* WARNING
It is enough to start a paragraph with one of these words followed by a colon (such as `NOTE:`). When it is converted to HTML or PDF, those sections will be formatted properly - indented from the main text and showing an icon inline.
You can add titles to admonitions by making it an admonition block. The structure of an admonition block is like this:
[source]
----
[NOTE]
.Title of Note
====
Text of note
====
----
In this example, the type of admonition is included in square brackets (`[NOTE]`), and the title is prefixed with a period. Four equal signs give the start and end points of the note text (which can include new lines, lists, etc.).
More info: http://asciidoctor.org/docs/user-manual/#admonition

View File

@ -0,0 +1,23 @@
= Tools for Working with Asciidoc Format
== Asciidoc vs Asciidoctor
The Solr Ref Guide is written in _Asciidoc_ format. This format is generally considered an extension of Markdown, because it has support for tables of contents, better table support, and other features that make it more appropriate for writing technical documentation.
We are using a version of the Asciidoc syntax along with tools from an open source project called https://asciidoctor.org[Asciidoctor]. This provides full support for the Asciidoc syntax, but replaces the original Python processor with one written in Ruby. There is a Java implementation, known as https://github.com/asciidoctor/asciidoctorj[AsciidoctorJ]. Further extensions from the original Asciidoc project include support for font-based icons and UI elements.
== Helpful Tools
You can write Asciidoc without any special tools. It's simply text, with familiar syntax for bold (`*`) and italics (`_`).
Having some tools in your editor is helpful, though.
=== Doc Preview
This allows you to see your document in something close to what it might appear like when output to HTML.
The following information is from http://asciidoctor.org/docs/editing-asciidoc-with-live-preview.
* Atom has AsciiDoc Preview, which gives you a panel that updates as you type. There are also a couple of other plugins to support Asciidoc format and auto-complete.
* There is a Live Preview browser plugin for Chrome, Firefox and Opera which allow you to open your Asciidoc page in the browser. It will also update as you type.
* There is also an Intellij IDEA plugin to support Asciidoc format.

View File

@ -0,0 +1,61 @@
= Working with Jekyll
:toc:
The Solr Ref Guide uses Jekyll to build the HTML version of the site.
== What is Jekyll?
Jekyll is a static site generator, meaning that it takes some set of documents and produces HTML pages. It allows for templating of the pages, so each page has the same look and feel without having to code headers, footers, logos, etc., into every page.
Jekyll is an open source project written in Ruby, online at https://jekyllrb.com/.
== Jekyll-Asciidoctor Plugin
We use a plugin for Jekyll from the Asciidoctor project to integrate Jekyll with Asciidoc formatted content. The source for the plugin is available at https://github.com/asciidoctor/jekyll-asciidoc.
This plugin allows us to use Asciidoctor-style variables with Jekyll, instead of having to maintain two sets of the same variables (one for HTML version and another for PDF version).
== Jekyll Basics
The following sections describe the main features of Jekyll that you will encounter while working with the Solr Ref Guide.
=== _config.yml
The `_config.yml` is a global configuration file that drives many of the options used when building the site (particularly in our use of Jekyll).
We have templatized `_config.yml` so in our use of Jekyll you will find it as `solr-ref-guide/_config.yml.template`. This allows us to define some variables during the build, and use common Lucene/Solr build parameters (such as versions, etc.).
=== Front Matter
Front matter for Jekyll is like a header that defines the title of the page, and any other variables that may be helpful or even required when rendering the page.
Every document that will be converted to HTML *must* include at least the page title at the top of the page.
The Solr Ref Guide uses the front matter to define the "short name" and permanent URL of a page, and to define the children of a page. The list of children is used to build the site navigation menu that appears to the left of each page's content.
Many guides to Jekyll also say that defining the layout in the front matter is required. However, since we only have one layout for all pages, we have defined this as a default.
=== Layouts
Layouts define the "look and feel" of each page.
Jekyll uses Liquid for page templates.
For our implementation of Jekyll, layouts are found in `solr-ref-guide/src/_layouts`
=== Includes
Include files are usually small files that are pulled into a layout when a page is being built. They are Liquid templates that define an area of the page. This allows flexibility across layouts - all pages can have the same header without duplicating code, but different pages could have different menu options.
Include files that we use define the top navigation, the page header, the page footer, and tables of contents.
For our implementation of Jekyll, include files are found in `solr-ref-guide/src/_includes`.
=== Data Files
Data files include data such as lists, that should be included in each page. The left-hand navigation is an example of a data file.
For our implementation of Jekyll, data files are found in `solr-ref-guide/src/_data`.
== Building the HTML Site
An Ant target `build-site` will build the full HTML site. This target builds the navigation for the left-hand menu, and converts all `.adoc` files to `.html`, including navigation and inter-document links.

View File

@ -0,0 +1,131 @@
= Generating PDF
The main published artifact of the Solr Reference Guide is a PDF document.
The PDF version of the Ref Guide uses `asciidoctor-pdf` (indirectly) to convert the `.adoc` files to PDF.
In order for all of the various files to be converted into a single, large, PDF file, another `.adoc` file must be created.
== About asciidoctor-pdf
Our PDF build process uses the https://github.com/asciidoctor/asciidoctor-ant[`asciidoctor-ant`] project to integrate Asciidoctor.
The `asciidoctor-ant` project is really an Ant wrapper around https://github.com/asciidoctor/asciidoctorj[`asciidoctorj`]. This is a Java binding of several Asciidoctor projects that produces a .jar that includes Asciidoctor, Asciidoctor PDF and Asciidoctor EPub.
Since it is a binding for the Ruby implementations of these three projects, they each work the same way they would if we were using the Ruby versions.
== Configuring the PDF Theme
Nearly all of the styling and layout of the PDF is controlled by a single YAML file that is called a theme.
See the section <<Modifying the Theme>> for information on how to update the theme.
=== Declaring the Theme to Use
Our theme is `refguide-theme.yml`, found in `solr/solr-ref-guide/pdf/themes`.
New themes must be named to include `-theme.yml` at the end. They must also be in proper YAML format.
The theme to use when generating the PDF is defined with the `pdf-style` attribute. Only the first part of the file name is used. So, a theme file named `refguide-theme.yml` is selected by defining `pdf-style=refguide`.
In the section <<Asciidoctor PDF Attributes>> below, we can see how we've declared this in our Ant target.
=== Modifying the Theme
All of the styling capabilities are described in the https://github.com/asciidoctor/asciidoctor-pdf/blob/master/docs/theming-guide.adoc[Asciidoctor PDF Theming Guide].
== About the Uber-Document
In order for all of the files in `.adoc` format to be compiled into a single PDF, we need to give the process one `.adoc` file (an "uber-document") that includes all of the content we want in the PDF. In other words, there is no command that says "take all the files in this directory and make one PDF".
Asciidoctor has a nice feature called the _include directive_, which allows you to insert content from another file into the current file (more details in the http://asciidoctor.org/docs/user-manual/#include-directive[Asciidoctor documentation]).
We wanted to make sure we didn't add to the burden of maintaining the PDF by asking everyone to update the uber-document or trying to institute some sort of check on if all the pages are included and in the right place in the hierarchy. Instead, the uber-document is build programmatically at build time.
The uber-document is found in `solr/solr-ref-guide/src/pdf/SolrRefGuide-all.adoc`. This document is very simple - it includes only the Apache license statement and a single `include` directive to another file:
[source]
\include::_data/pdf-main-body.adoc[]
The Ant target `build-pdf` includes a dependency on another target, `build-nav-data-files`. This target looks at each document and builds the `pdf-main-body.adoc` file with the proper hierarchy of all pages.
NOTE: You will not see the `pdf-main-body.adoc` in `solr/solr-ref-guide/src/_data` directory. This file exists only once it has been built, and it is placed in the build directory, at `solr/solr-ref-guide/build/content_data`.
== Configuring the build-pdf Ant Target
Since most of the styling and layout is defined by the theme, the options that must be configured in the Ant target relate to where to find referenced files, font directories, image directories, and similar top-level document settings.
There are two types of settings at work in our Ant target. First are the settings which are part of the `asciidoctor-ant` jar. Second, we are able to declare any Asciidoctor PDF attribute (setting) that we want to apply to the entire PDF.
Our Ant target (`build-pdf`) uses the following settings:
[source,xml]
----
<asciidoctor:convert
sourceDirectory="${build.content.dir}/pdf"
sourceDocumentName="SolrRefGuide-all.adoc"
baseDir="${build.content.dir}"
outputDirectory="${build.dir}"
backend="pdf"
extensions="adoc"
sourceHighlighter="coderay"
embedAssets="true"
imagesDir="${build.content.dir}"
doctype="book"
safemode="unsafe">
<attribute key="icons" value="font" />
<attribute key="icon-set" value="fa" />
<attribute key="pdf-stylesDir" value="./pdf/themes"/>
<attribute key="pdf-style" value="refguide"/>
<attribute key="pdf-fontsDir" value="./pdf/fonts"/>
<attribute key="pagenums" value='' />
<attribute key="figure-caption!" value='' />
<attribute key="idprefix" value='' />
<attribute key="idseparator" value='-' />
<!-- attributes used in adoc files -->
<!-- NOTE: If you add any attributes here for use in adoc files, you almost certainly need to also add
them to the _config.yml.template file for building the jekyll site as well
-->
<attribute key="solr-docs-version" value="${solr-docs-version}" />
<attribute key="solr-javadocs" value="${solr-javadocs}" />
<attribute key="lucene-javadocs" value="${lucene-javadocs}" />
<attribute key="build-date" value="${DSTAMP}" />
<attribute key="build-year" value="${current.year}" />
</asciidoctor:convert>
----
There are a lot of options here. Note that some include the `<attribute>` tag and some do not. Those that do not are options provided by `asciidoctor-ant` so don't need any special syntax. The options with the `<attribute>` tag are not provided by `asciidoctor-ant` but are supported by `asciidoctor-pdf` so will be applied when the target is run. A few of these are custom variables that we have defined ourselves.
=== Asciidoctor Ant Attributes
`sourceDirectory="${build.content.dir}/pdf"`:: Defines where to find the source file to build the PDF. The variable is declared earlier in `build.xml`.
`sourceDocumentName="SolrRefGuide-all.adoc"`:: The source file name, which in our case will be built as described in the section <<Creating the Uber-Document>>.
`baseDir="${build.content.dir}"`:: The root path for any included files.
`outputDirectory="${build.dir}"`:: Defines where the resulting PDF file should go. In this case, the
`backend="pdf"`:: The output format. The `asciidoctor-ant` jar is also capable of outputting DocBook format, so we must declare we want a PDF.
`extensions="adoc"`:: The file extensions to allow for the source document.
`sourceHighlighter="coderay"`:: The library to use for syntax highlighting source code.
`imagesDir="${build.content.dir}"`:: The directory to use to find images referenced in the documents.
`doctype="book"`:: Adds support for book-style format and sections, such as a preface, colophon, glossary, index, etc.
`safemode="unsafe">`:: Allows including resources that are external to the parent directory of the source file. For example, source examples could be pulled from Solr's source code instead of copied to documentation. This setting allows that to happen.
=== Asciidoctor PDF Attributes
`<attribute key="icons" value="font" />`:: The style of icons.
`<attribute key="icon-set" value="fa" />`:: The icon set to use. We use the Font Awesome font set.
`<attribute key="pdf-stylesDir" value="./pdf/themes"/>`:: The directory to find PDF themes. See the section <<Configuring the PDF Theme>> for more details on themes.
`<attribute key="pdf-style" value="refguide"/>`:: The theme to use. The theme must be saved in the directory referenced with the `pdf-stylesDir` attribute, and must be named `<pdf-style>-theme.yml`.
`<attribute key="pdf-fontsDir" value="./pdf/fonts"/>`:: The directory where to find fonts declared in the theme.
`<attribute key="figure-caption!" value='' />`:: Sets caption labels and numbers (such as "Figure 1") to block images. The exclamation at the end of this setting in our config _disables_ figure captioning.
`<attribute key="idprefix" value='' />`:: Sets the prefix for auto-generated section IDs, such as those from headings in a page. In our config, this is effectively "null", so auto-generated section IDs do not have any prefix.
`<attribute key="idseparator" value='-' />`:: Sets the separator between words in auto-generated section IDs to a hyphen (`-`).
=== Custom Attributes
These attributes use variables that are inserted by Ant during the PDF creation process. This allows us to pull from standard Lucene/Solr build files, and not have to update several places for any release. The Ant build process updates the `_config.yml` file from the `_config.yml.template`, then these attributes pull the proper value from that file.
`<attribute key="solr-docs-version" value="${solr-docs-version}" />`:: Sets the version to the current release version.
`<attribute key="solr-javadocs" value="${solr-javadocs}" />`:: Sets the path for Solr javadoc links to include the right path for the current release version.
`<attribute key="lucene-javadocs" value="${lucene-javadocs}" />`:: Sets the path for Lucene javadoc links to the right path for the current release version.
`<attribute key="build-date" value="${DSTAMP}" />`:: Sets the date of the build to add the date to the footer of each page of the PDF.
`<attribute key="build-year" value="${current.year}" />`:: Sets the year of the build to add the date to the copyright notice.

View File

@ -0,0 +1,196 @@
= Publication Process
:toc:
== About the Formats
The Solr Ref Guide is published in two formats: PDF and HTML.
The PDF version is the *official* release, and requires a vote before release. See <<Publishing PDF Version>> for details on how to generate the PDF and hold a vote.
The HTML version is considered a "convenience" version, and does not require a vote. See <<Publishing HTML Version>> for details on how to publish the HTML.
It's strongly preferred that both PDF and HTML versions are available during the vote for the PDF. However, since the HTML version is not an official release, it is more of an unwritten rule to publish the HTML at the same time as producing a release candidate for the PDF.
== Publishing PDF Version
The PDF version of the Solr Reference Guide is the *official* version. As such, it is voted on by the community before release, and is treated as an official artifact of the Lucene/Solr project.
=== Generate the PDF
No local dependencies are required to build the PDF. The Ant target will download the jars and other items it requires.
The build process generates the PDF, including the page hierarchy, and then runs an optimization script on the PDF to make it smaller.
To build the PDF:
. Run `ant build-pdf`
. The resulting PDF will be in `solr/build/solr-ref-guide`.
=== Prerequisites
* You have checked out the Lucene/Solr source code on the machine you will be doing the release from. You will need scripts in the `dev-tools` directory.
* You have generated a GPG key. See the Apache documentation on https://www.apache.org/dev/release-signing.html#generate[generating a code signing key].
* You have Python 3 installed. This is needed to poll the mirrors after release to be sure it's propagated enough to make the announcement.
=== Prepare and Upload Release Candidate
The `dist/dev` Subversion repository includes a directory for the Solr Ref Guide at https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/[`lucene/solr/ref-guide`] which can host the release candidate (RC) during the VOTE stage of the process.
These steps walk through checking out this directory and uploading the Guide to it.
. Checkout the directory: `svn co https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide solr-ref-guide-rc`
* If you have already checked out this directory, you can simply update it: `svn update solr-ref-guide-rc`
. Change directories so `solr-ref-guide-rc` is your working directory (`cd solr-ref-guide-rc`).
IMPORTANT: The next step requires that you have already generated your GPG keys. Your GPG passphrase will be required.
[start=3]
. Run the Prep Ref Guide script to prepare the RC. This script ensures proper naming of the PDF file, generates `.sha1` and `.asc` files and creates the proper RC sub-directories under `solr-ref-guide-rc`.
.. The structure of the input is: `prep-solr-ref-guide-rc.sh <path/PDFfilename> <Solrversion-RC#> GPGkey`.
.. From the `solr-ref-guide-rc` directory, it will look something like this:
+
[source,bash]
----
$ ~/lucene-source/dev-tools/scripts/prep-solr-ref-guide-rc.sh apache-solr-ref-guide-7.0.pdf 7.0-RC0
+ mkdir apache-solr-ref-guide-7.0-RC0
+ mv apache-solr-ref-guide-7.0.pdf apache-solr-ref-guide-7.0-RC0/apache-solr-ref-guide-7.0.pdf
+ cd apache-solr-ref-guide-7.0-RC0
+ sha1sum apache-solr-ref-guide-7.0.pdf
+ gpg -u DEADBEEF --armor --output apache-solr-ref-guide-7.0.pdf.asc --detach-sig apache-solr-ref-guide-7.0.pdf
You need a passphrase to unlock the secret key for
user: "Your Name <you@apache.org>"
4096-bit RSA key, ID DEADBEEF, created 1969-07-04
----
+
. Add and commit the new release candidate to the `dist/dev` with these steps:
.. `svn add apache-solr-ref-guide-7.0-RC0`
.. `svn commit -m "7.0 ref guide RC0"`
=== Hold a VOTE
Votes must be sent to the lucene-dev mailing list (`dev@lucene.apache.org`).
. Send an email to `dev@lucene.apache.org` with subject, "VOTE: Release Apache Solr Ref Guide for Solr X.Y".
. The body of the email should include the full URL of the RC directory in the `dist/dev` repo. Such as: https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-7.0-RC0
. You can add your own +1 to the vote announcement email.
. If there are issues that need to be resolved, you can start the process over, using RC1, RC2, etc., as needed.
Ideally, the HTML version will also be available for voters to evaluate, see the section <<Publishing HTML Version>> below for details of how to do that.
=== Publish to Production & Archive Old Versions
Once at least three PMC members have voted for release (see https://www.apache.org/foundation/voting.html#ReleaseVotes[Apache Voting Process] for details on the rules for votes), the release candidate can be released.
. Use the Publish Solr Ref Guide script (`publish-solr-ref-guide.sh`) to generate the proper SVN commands to be run to execute a remote move of the RC files to the final `dist/releases` repository.
.. The script takes only the version and _RC number that passed the vote_ as inputs, such as `7.0-RC0`.
.. The input and output of the script will look like this:
+
[source,bash]
----
$ ~/lucene-source/dev-tools/scripts/publish-solr-ref-guide-rc.sh X.Y-RCZ
## Run the following commands when ready...
svn move -m 'publishing apache-solr-ref-guide-X.Y-RCZ' https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RCZ/apache-solr-ref-guide-X.Y.pdf https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RCZ/apache-solr-ref-guide-X.Y.pdf.asc https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RCZ/apache-solr-ref-guide-X.Y.pdf.sha1 https://dist.apache.org/repos/dist/release/lucene/solr/ref-guide/
svn rm -m 'cleaning up apache-solr-ref-guide-X.Y-RCZ' https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RCZ
----
[start=2]
. The release should propagate to as many mirrors as possible before announcing the release, generally 24 hours is long enough. Use the Poll Mirrors script (`poll-mirrors.py`) to check the status:
+
[source,bash]
python3 -u ~/lucene-source/dev-tools/scripts/poll-mirrors.py -details -p lucene/solr/ref-guide/apache-solr-ref-guide-X.Y.pdf
* This script requires Python 3 to be installed on your machine.
* If you have over 85% of the mirrors with the file, it's OK to go ahead with the announcement.
. You may get an automated email about updating the ASF release repository; you can safely ignore this email.
. The `dist/releases` repo is only meant to keep the latest releases. Shortly after new releases are mirrored, they are copied to `archive.apache.org`, so older releases can safely be deleted from `dist/releases` since they have been backed up in the archives.
.. Run the Archive Ref Guide script (`archive-solr-ref-guide.sh`) using the X.Y version of the Ref Guide that has just been published. Older RCs will also be removed.
.. Again, this script doesn't do any direct removal of files, it only outputs SVN commands for you to copy and paste:
+
[source,bash]
----
$ ~/lucene-source/dev-tools/scripts/archive-solr-ref-guide.sh X.Y
## Run the following commands when ready...
# Delete old releases
svn rm -m 'removing archived ref guide files prior to X.Y' https://dist.apache.org/repos/dist/release/lucene/solr/ref-guide/apache-solr-ref-guide-A.B.pdf https://dist.apache.org/repos/dist/release/lucene/solr/ref-guide/apache-solr-ref-guide-A.B.pdf.asc https://dist.apache.org/repos/dist/release/lucene/solr/ref-guide/apache-solr-ref-guide-A.B.pdf.sha1
# Delete old RC files
svn rm -m 'cleaning up old RCs now that X.Y has been released' https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RC0/ https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RC1/
----
=== Announce the Release
Announce the availability of the new Ref Guide on `solr-user@lucene.apache.org` and CC `general@lucene.apache.org` and `announce@apache.org`.
WARNING: You must send the announcement email from your @apache.org email address or announce@apache will reject it.
Always use the link to the download redirector for the announcement, as it will automatically direct users to the closest mirror for download: `https://www.apache.org/dyn/closer.cgi/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y.pdf`.
You could also include a link to the HTML version in your announcement, if the publication process for that has finished.
== Publishing HTML Version
The steps to publish the Guide differ depending on if it is the first time the Guide has been published or if it is an update to an already published Guide.
=== Building the HTML Version
If you have the required dependencies on your local machine, you can build the HTML version with `ant build-site`. The dependencies are listed in `solr-ref-guide/README.adoc`.
//TODO update Jenkins link
If you do not have the required dependencies, and don't choose to install them, you can download the files from the Jenkins (https://builds.apache.org/job/Solr-reference-guide-jira-SOLR-10290/lastSuccessfulBuild/artifact/solr/build/solr-ref-guide/html-site/[Solr Reference Guide job]).
=== Publish a New Guide
// A lot of this was copied from https://wiki.apache.org/lucene-java/ReleaseTodo#Website_.2B-.3D_javadocs. See that section for explanations for why some steps are required.
==== Step 1: Update extpaths.txt in CMS Staging
. Checkout CMS trunk:
+
[source,bash]
svn co --depth=immediates https://svn.apache.org/repos/asf/lucene/cms/trunk/content website-source
+
* If you already have this repo checked out, you can simply `svn up website-source` to update to the latest revision.
. `cd website-source`
. Add Guide branch dir: `echo solr/guide/X_Y_Z >> extpaths.txt`
. Commit changes:
+
[source,bash]
svn commit -m "Update CMS production sync exceptions for X_Y_Z Guide" extpaths.txt
==== Step 2: Push Guide to Website Production
Go to the checkout directory where you have built the Guide and push the documentation via subversion import. You must push it to the path you just added to `extpaths.txt`, so if the path you added was `solr/guide/6.5`, you'll use the path as shown in the below example:
[source,bash]
svn -m "Add Ref Guide for Solr 6.5" import <checkoutroot>/solr/build/solr-ref-guide/html-site https://svn.apache.org/repos/infra/websites/production/lucene/content/solr/guide/6_5
Confirm you can browse to these URLs manually, and especially that solr javadocs link back to lucene's correctly. Example:
https://lucene.apache.org/solr/guide/6_5
==== Step 3: Push Staging extpaths.txt to Production
The `extpaths.txt` works by listing paths that should be ignored when the CMS syncs the staging and production repositories. Publishing staging to production will only succeed if the paths listed in `extpaths.txt` exist in production. At the same time, if a path exists in production but not in staging it will be deleted unless it is defined in `extpaths.txt`. After pushing the content to production, check that the `extpaths.txt` in production includes the proper path to ensure that the Guide is not deleted incorrectly. If it does not exist in production, try to publish the site again to make sure it is updated.
Production URL: https://lucene.apache.org/extpaths.txt
==== Update Ref Guide Landing Page
Update the landing page at https://lucene.apache.org/solr/guide to link to the newest version.
You can use the CMS system for this since it is a small change, or you can edit the file locally and commit it to the staging repo.
=== Update a Published Guide
If you need to re-publish an existing online copy of the Guide, you will need to checkout the directory in production website repository and overwrite the existing files:
. Build the new HTML files locally (`ant clean build-site`), or download them from Jenkins.
. Checkout the directory you need to update from the production repo: `svn co https://svn.apache.org/repos/infra/websites/production/lucene/content/solr/guide/<dir>`.
* This command checks out the Guide version directory into a local subdirectory with the same name as the version (such as "6_5"). You can provide a better name locally if you prefer by adding it to the end of the command shown above.
* Don't shortcut this and download the whole production website. It will take an incredibly long time and that will feel like _forever_.
. Copy the files from the build location to the checked out Guide directory. For example, if we needed to replace the current Guide for Solr 6.5, we'd do `cp -r <checkoutroot>/solr/build/html-site 6_5/.`
. Use `svn status` to see the files modified.
. If there are any pages added or deleted, use `svn add <file>` or `svn rm <file>` as needed.
. Commit the changes: `svn commit -m "Update production 6.5 Ref Guide"`
// TODO:
// - figure out if redirects in .htaccess require any work here (probably)

3
solr/solr-ref-guide/src/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
_site
.sass-cache
.jekyll-metadata

6
solr/solr-ref-guide/src/404.md Executable file
View File

@ -0,0 +1,6 @@
---
title: "Page Not Found"
search: exclude
---
Sorry, but the page you were trying to view does not exist. Try searching for it or looking at the URL to see if it looks correct.

View File

@ -0,0 +1,3 @@
## Jekyll Documentation theme
Build the site to see the instructions for using it. Or just go here: [http://idratherbewriting.com/documentation-theme-jekyll/](http://idratherbewriting.com/documentation-theme-jekyll/)

View File

@ -0,0 +1,99 @@
#
#
#
# NOTE: Ant converts _config.yml.template into create _config.yml and performs ant property substitution.
#
#
#
# Gems that are included for building the site. jekyll-asciidoc allows Jekyll to use Asciidoctor for variables and settings
gems: [jekyll-asciidoc]
destination: ../html-site
# this property is useful for conditional filtering of content that is separate from the PDF.
output: web
# this appears on the top navigation bar next to the home button
topnav_title: Solr Ref Guide
# this appears in the html browser tab for the site title (seen mostly by search engines, not users)
site_title: Apache Solr Reference Guide
# this appears in the footer
company_name: Apache Software Foundation
# the preview server used. Leave as is.
host: 127.0.0.1
# the port where the preview is rendered. You can leave this as is unless you have other Jekyll builds using this same port that might cause conflicts. in that case, use another port such as 4006.
port: 4015
# these are the files and directories that jekyll will exclude from the build
exclude:
- .idea/
- .gitignore
- pdf/
# if you uncomment the next line, the Feedback link gets removed
feedback_disable: true
# used as a contact email for the Feedback link in the top navigation bar
# feedback_email: an_email@apache.org
# if you uncomment the next line, it changes the Feedback text
# feedback_text: "Need help?"
# if you uncomment the next line, it changes where the feedback link points to
# feedback_link: "http://helpy.io/"
# these are defaults used for the frontmatter for these file types
defaults:
-
scope:
path: ""
type: "pages"
values:
layout: "page"
search: true
-
scope:
path: ""
type: "posts"
values:
layout: "post"
search: true
# the description is used in the feed.xml file
description: "The Apache Solr Reference Guide is the official documentation for the Apache Solr project."
# needed for sitemap.xml and feed.xml files
url: https://home.apache.org/~ctargett/RefGuidePOC/jekyll-full
# Asciidoc settings - disabled so we can use asciidoctor instead
asciidoc: {}
# Custom Attributes for use in our templates & adoc files.
#
# Declared as a YAML reference so we can refer to them via site.solr-attributes.foo in liquid templates,
# in addition to using them below in our asciidoctor attribute configurations
# (see https://github.com/asciidoctor/jekyll-asciidoc/issues/137)
#
# NOTE: If you add any attributes here for use in adoc files, you almost certainly need to also add
# them to the <asciidoctor:convert/> ant task for building the PDF as well.
solr-attributes: &solr-attributes-ref
solr-docs-version: "${solr-docs-version}"
solr-javadocs: "${solr-javadocs}"
lucene-javadocs: "${lucene-javadocs}"
build-date: "${DSTAMP}"
build-year: "${current.year}"
asciidoctor:
attributes:
<<: *solr-attributes-ref
icons: "font"
source-highlighter: "pygments"
pygments-css: "style"

View File

@ -0,0 +1,5 @@
# placed here for translation purposes
search_placeholder_text: search...
search_no_results_text: No results found.

View File

@ -0,0 +1,7 @@
allowed-tags:
- getting_started
- content_types
- troubleshooting
- analysis
- languages
- scaling

View File

@ -0,0 +1,15 @@
---
layout: default
type: archive
---
<div class="post-header">
<h1 class="post-title-main">{{ page.title }}</h1>
</div>
<div class="post-content">
{{ content }}
</div>

View File

@ -0,0 +1,16 @@
<!-- Send feedback function -->
<script>
function SendLinkByMail(href) {
var subject= "{{site.site_title}} feedback";
var body = "I have some feedback about the {{page.title}} page: ";
body += window.location.href;
body += "";
var uri = "mailto:{{site.feedback_email}}?subject=";
uri += encodeURIComponent(subject);
uri += "&body=";
uri += encodeURIComponent(body);
window.location.href = uri;
}
</script>
<li><a href="{% if site.feedback_link %}{{site.feedback_link}}{% else %}javascript:(function()%7BSendLinkByMail()%3B%7D)()%3B{% endif %}" target="_blank">{% if site.feedback_link == null %}<i class="fa fa-envelope-o"></i>{% endif %} {% if site.feedback_text %}{{site.feedback_text}}{% else %}Feedback{% endif %}</a></li>

View File

@ -0,0 +1,9 @@
<footer>
<div class="row">
<div class="col-lg-12 footer">
&copy;{{ site.solr-attributes.build-year }} {{site.company_name}}. All rights reserved. <br />
{% if page.last_updated %}<p>Page last updated:</span> {{page.last_updated}}<br/>{% endif %} Site last generated: {{ site.solr-attributes.build-date }} <br />
<p><img src="{{ "solr-sunOnly-small.png" }}" alt="Apache Solr"/></p>
</div>
</div>
</footer>

View File

@ -0,0 +1,6 @@
<!-- the google_analytics_id gets auto inserted from the config file -->
{% if site.google_analytics %}
<script>(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)})(window,document,'script','//www.google-analytics.com/analytics.js','ga');ga('create','{{site.google_analytics}}','auto');ga('require','displayfeatures');ga('send','pageview');</script>
{% endif %}

View File

@ -0,0 +1,34 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="{% if page.description %}{{ page.description | strip_html | strip_newlines | truncate: 160 }}{% endif %}">
<meta name="keywords" content="{{page.tags}}{% if page.tags %}, {% endif %} {{page.keywords}}">
<title>{{ page.title }} | {{ site.site_title }}</title>
<link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css">
<!--<link rel="stylesheet" type="text/css" href="css/bootstrap.min.css">-->
<link rel="stylesheet" href="{{ "css/lavish-bootstrap.css" }}">
<link rel="stylesheet" href="{{ "css/customstyles.css" }}">
<link rel="stylesheet" href="{{ "css/theme-solr.css" }}">
<link rel="stylesheet" href="{{ "css/ref-guide.css" }}">
<link rel="stylesheet" href="{{ "css/comments.css" }}">
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.1.4/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery-cookie/1.4.1/jquery.cookie.min.js"></script>
<script src="{{ "js/jquery.navgoco.min.js" }}"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/anchor-js/2.0.0/anchor.min.js"></script>
<script src="{{ "js/toc.js" }}"></script>
<script src="{{ "js/customscripts.js" }}"></script>
<link rel="shortcut icon" href="{{ "images/favicon.ico" }}">
<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
<script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
<![endif]-->
<link rel="alternate" type="application/rss+xml" title="{{ site.site_title }}" href="{{ "/feed.xml" | prepend: site.url }}">

View File

@ -0,0 +1,33 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="{% if page.summary %}{{ page.summary | strip_html | strip_newlines | truncate: 160 }}{% endif %}">
<meta name="keywords" content="{{page.tags}}{% if page.tags %}, {% endif %} {{page.keywords}}">
<title>{% if page.homepage == true %} {{site.homepage_title}} {% elsif page.title %}{{ page.title }}{% endif %} | {{ site.site_title }}</title>
<link rel="stylesheet" href="{{ "/css/syntax.css" | prepend: site.baseurl | prepend: site.url }}">
<link rel="stylesheet" href="{{ "/css/font-awesome.min.css" | prepend: site.baseurl | prepend: site.url }}">
<link rel="stylesheet" href="{{ "/css/bootstrap.min.css" | prepend: site.baseurl | prepend: site.url }}">
<link rel="stylesheet" href="{{ "/css/modern-business.css" | prepend: site.baseurl | prepend: site.url }}">
<link rel="stylesheet" href="{{ "/css/lavish-bootstrap.css" | prepend: site.baseurl | prepend: site.url }}">
<link rel="stylesheet" href="{{ "/css/customstyles.css" | prepend: site.baseurl | prepend: site.url }}">
<link rel="stylesheet" href="{{ "/css/theme-green.css" | prepend: site.baseurl | prepend: site.url }}">
<link rel="stylesheet" href="{{ "/css/syntax.css" | prepend: site.baseurl | prepend: site.url }}">
<link rel="stylesheet" href="{{ "/css/printstyles.css" | prepend: site.baseurl }}">
<script>
Prince.addScriptFunc("datestamp", function() {
return "PDF last generated: {{ site.time | date: '%B %d, %Y' }}";
});
</script>
<script>
Prince.addScriptFunc("guideName", function() {
return "{{site.print_title}} User Guide";
});
</script>

View File

@ -0,0 +1 @@
<figure>{% if {{include.url}} %}<a class="no_icon" target="_blank" href="{{include.url}}">{% endif %}<img class="docimage" src="images/{{include.file}}" alt="{{include.alt}}" {% if {{include.max-width}} %}style="max-width: {{include.max-width}}px"{% endif %} />{% if {{include.url}} %}</a>{% endif %}{% if {{include.caption}} %}<figcaption>{{include.caption}}</figcaption></figure>{% endif %}

View File

@ -0,0 +1 @@
<img class="inline" src="images/{{include.file}}" alt="{{include.alt}}" />

View File

@ -0,0 +1,44 @@
{% comment %}Get links from each sidebar, as listed in the _config.yml file under sidebars{% endcomment %}
{% for sidebar in site.sidebars %}
{% for entry in site.data.sidebars[sidebar].entries %}
{% for folder in entry.folders %}
{% for folderitem in folder.folderitems %}
{% if folderitem.url contains "html#" %}
[{{folderitem.url | remove: "/" }}]: {{folderitem.url | remove: "/"}}
{% else %}
[{{folderitem.url | remove: "/" | remove: ".html"}}]: {{folderitem.url | remove: "/"}}
{% endif %}
{% for subfolders in folderitem.subfolders %}
{% for subfolderitem in subfolders.subfolderitems %}
[{{subfolderitem.url | remove: "/" | remove: ".html"}}]: {{subfolderitem.url | remove: "/"}}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
{% comment %} Get links from topnav {% endcomment %}
{% for entry in site.data.topnav.topnav %}
{% for item in entry.items %}
{% if item.external_url == null %}
[{{item.url | remove: "/" | remove: ".html"}}]: {{item.url | remove: "/"}}
{% endif %}
{% endfor %}
{% endfor %}
{% comment %}Get links from topnav dropdowns {% endcomment %}
{% for entry in site.data.topnav.topnav_dropdowns %}
{% for folder in entry.folders %}
{% for folderitem in folder.folderitems %}
{% if folderitem.external_url == null %}
[{{folderitem.url | remove: "/" | remove: ".html"}}]: {{folderitem.url | remove: "/"}}
{% endif %}
{% endfor %}
{% endfor %}
{% endfor %}

View File

@ -0,0 +1,59 @@
{% assign sidebar = site.data.sidebar %}
<ul id="mysidebar" class="nav">
<li class="sidebarTitle">{{sidebar.title}} <!-- TODO: version from site.data --></li>
{% for level1item in sidebar.kids %}
<li class="sb-level1">
<a href="{{level1item.url | remove: "/"}}">{{ level1item.title }}</a>
{% if level1item.kids.size > 0 %}
<ul>
{% for level2item in level1item.kids %}
<li class="sb-level2">
<a href="{{ level2item.url | remove: "/" }}">{{ level2item.title }}</a>
{% if level2item.kids.size > 0 %}
<ul>
{% for level3item in level2item.kids %}
<li class="sb-level3">
<a href="{{ level3item.url | remove: "/" }}">{{ level3item.title }}</a>
{% if level3item.kids.size > 0 %}
<ul>
{% for level4item in level3item.kids %}
<li class="sb-level4">
<a href="{{ level4item.url | remove: "/" }}">{{ level4item.title }}</a>
{% comment %}
<!-- NOTE: adding addiional levels here will also require additional changes to
ref-guide.css in order to indent them properly
-->
{% endcomment %}
</li>
{% endfor %}
</ul>
{% endif %}
</li>
{% endfor %}
</ul>
{% endif %}
</li>
{% endfor %}
</ul>
{% endif %}
</li>
{% endfor %}
</ul>
{% comment %}
<!-- if you aren't using the accordion, uncomment this block:
<p class="external">
<a href="#" id="collapseAll">Collapse All</a> | <a href="#" id="expandAll">Expand All</a>
</p>
-->
{% endcomment %}
<!-- set the 'active' class on the current page and ancestors -->
<!-- this highlights the active parent class in the navgoco sidebar. this is critical so that the parent expands when you're viewing a page. This must appear below the sidebar code above. Otherwise, if placed inside customscripts.js, the script runs before the sidebar code runs and the class never gets inserted.-->
<script>$("#mysidebar a[href='{{ page.shortname }}.html']").parents('li').toggleClass("active", true);</script>
<!-- set the 'current' class on the current page and 'current-tree' on the current page + it's ancestors -->
<!-- this can let us do css highlighting of the current page in the sidebar even if/when the user clicks around in the sidebar causing other sidebar elements to be 'active' -->
<script>
$("#mysidebar a[href='{{ page.shortname }}.html']").parent('li').toggleClass("current", true);
$("#mysidebar a[href='{{ page.shortname }}.html']").parents('li').toggleClass("current-tree", true);
</script>

View File

@ -0,0 +1,22 @@
<p>The following pages and posts are tagged with <button type="button" style="cursor: default" class="btn btn-default navbar-btn">{{page.tagName}}</button></p>
<table><thead><tr><th>Title</th><th>Type</th><th>Excerpt</th></tr></thead>
<tbody>
{% assign thisTag = page.tagName %}
{% for page in site.pages %}
{% for tag in page.tags %}
{% if tag == thisTag %}
<tr><td><a href="{{ page.url | remove: "/" }}">{{page.title}}</a></td>
<td><span class="label label-default">Page</span></td>
<td>{% if page.description %}
{{ page.description | strip_html | strip_newlines | truncate: 160 }}
{% else %}
{{ page.content | truncatewords: 50 | strip_html }}
{% endif %}</td>
</tr>
{% endif %}
{% endfor %}
{% endfor %}
</tbody>
</table>

View File

@ -0,0 +1,21 @@
<!-- this handles the automatic toc. use ## for subheads to auto-generate the on-page minitoc. if you use html tags, you must supply an ID for the heading element in order for it to appear in the minitoc. -->
<script>
$( document ).ready(function() {
// Handler for .ready() called.
$('#toc').toc({ minimumHeaders: 2, listType: 'ul', showSpeed: 0, headers: 'h2,h3' });
/* this offset helps account for the space taken up by the floating toolbar. */
$('#toc').on('click', 'a', function() {
var target = $(this.getAttribute('href'))
, scroll_target = target.offset().top
$(window).scrollTop(scroll_target - 10);
return false
})
});
</script>
<div id="toc"></div>

View File

@ -0,0 +1,63 @@
<!-- Navigation -->
<nav class="navbar navbar-inverse navbar-fixed-top">
<div class="container topnavlinks">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="fa fa-home fa-lg navbar-brand" href="index.html">&nbsp;<span class="projectTitle"> {{site.topnav_title}} {{ site.solr-attributes.solr-docs-version }}</span></a>
</div>
<div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
<ul class="nav navbar-nav navbar-right">
<!-- Link to Solr website -->
<li><a href="https://lucene.apache.org/solr/news" target="_blank">Solr News</a></li>
<!-- Other Guide Formats dropdown -->
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Other Formats<b class="caret"></b></a>
<ul class="dropdown-menu">
<li><a href="https://www.apache.org/dyn/closer.cgi/lucene/solr/ref-guide" target="_blank">PDF for Latest Release</a></li>
<li><a href="https://archive.apache.org/dist/lucene/solr/ref-guide/" target="_blank">Archived PDFs</a></li>
<li><a href="https://lucene.apache.org/solr/guide/" target="_blank">Other Versions Online</a></li>
</ul>
</li>
<!-- Solr Resources dropdown -->
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Solr Resources<b class="caret"></b></a>
<ul class="dropdown-menu">
<li><a href="{{site.solr-attributes.solr-javadocs}}/solr-core/index.html" target="_blank">Solr Javadocs</a></li>
<li><a href="https://git1-us-west.apache.org/repos/asf?p=lucene-solr.git" target="_blank">Lucene/Solr Source Code</a></li>
<li><a href="https://lucene.apache.org/solr/resources.html#community" target="_blank">Solr Community Links</a></li>
</ul>
</li>
{% if site.feedback_disable == null or site.feedback_disable == false %}
{% include feedback.html %}
{% endif %}
<!--comment out this block if you want to hide search-->
<li>
<!--start search-->
<div id="search-demo-container">
<input type="text" id="search-input" placeholder="{{site.data.strings.search_placeholder_text}}">
<ul id="results-container"></ul>
</div>
<script src="{{ "js/jekyll-search.js"}}" type="text/javascript"></script>
<script type="text/javascript">
SimpleJekyllSearch.init({
searchInput: document.getElementById('search-input'),
resultsContainer: document.getElementById('results-container'),
dataSource: '{{ "search.json" }}',
searchResultTemplate: '<li><a href="{url}" title="{{page.title | replace: "'", "\"}}">{title}</a></li>',
noResultsText: '{{site.data.strings.search_no_results_text}}',
limit: 10,
fuzzy: true,
})
</script>
<!--end search-->
</li>
</ul>
</div>
</div>
<!-- /.container -->
</nav>

View File

@ -0,0 +1,55 @@
<!DOCTYPE html>
<head>
{% include head.html %}
<script>
$(document).ready(function() {
// Initialize navgoco with default options
$("#mysidebar").navgoco({
caretHtml: '',
accordion: true,
openClass: 'active', // open
save: false, // we do *NOT* want cookies used to save the current stage of the sidebar
// instead the code in sidebar.html will ensure that the current page
// (and it's ancestors) are "active" on page load
slide: {
duration: 400,
easing: 'swing'
}
});
$("#collapseAll").click(function(e) {
e.preventDefault();
$("#mysidebar").navgoco('toggle', false);
});
$("#expandAll").click(function(e) {
e.preventDefault();
$("#mysidebar").navgoco('toggle', true);
});
});
</script>
</head>
<body id="{{ page.shortname }}">
{% include topnav.html %}
<!-- Page Content -->
<div class="container">
<div class="col-lg-12">&nbsp;</div>
<!-- Content Row -->
<div class="row">
<!-- Sidebar Column -->
<div class="col-md-3">
{% include sidebar.html %}
</div>
<!-- Content Column -->
<div class="col-md-9">
{{content}}
</div>
<!-- /.row -->
</div>
<!-- /.container -->
</div>
</body>
</html>

View File

@ -0,0 +1,25 @@
<!DOCTYPE html>
<html lang="en">
<html>
<head>
{% include head_print.html %}
</head>
<body class="{% if page.type == "title"%}title{% elsif page.type == "frontmatter" %}frontmatter{% elsif page.type == "first_page" %}first_page{% endif %} print">
<!-- Page Content -->
<div class="container">
<!-- Content Column -->
<div class="col-md-9">
{{content}}
</div>
</div> <!-- /.container -->
</body>
</html>

View File

@ -0,0 +1,3 @@
---
---
{{content}}

View File

@ -0,0 +1,80 @@
---
layout: default
---
<div class="post-header">
<h1 class="post-title-main">{{ page.title }}</h1>
</div>
<!-- This adds a workflow map, to a page
See http://idratherbewriting.com/documentation-theme-jekyll/mydoc_workflow_maps.html
Leaving it here commented out in case we want to implement this in the future
{% if page.simple_map == true %}
<script>
$(document).ready ( function(){
$('.box{{page.box_number}}').addClass('active');
});
</script>
{% include custom/{{page.map_name}}.html %}
{% elsif page.complex_map == true %}
<script>
$(document).ready ( function(){
$('.modalButton{{page.box_number}}').addClass('active');
});
</script>
{% include custom/{{page.map_name}}.html %}
{% endif %} -->
<div class="post-content">
{% if page.summary %}
<div class="summary">{{ page.summary }}</div>
{% endif %}
{% unless page.toc == false %}
{% include toc.html %}
{% endunless %}
<div class="main-content">
{{content}}
</div>
<!-- Adds tags, if any -->
<div class="tags">
{% if page.tags != null %}
<b>Tags: </b>
{% assign projectTags = site.data.tags.allowed-tags %}
{% for tag in page.tags %}
{% if projectTags contains tag %}
<a href="{{ "tag-" | append: tag | append: ".html" }}" class="btn btn-default navbar-btn cursorNorm" role="button">{{page.tagName}}{{tag}}</a>
{% endif %}
{% endfor %}
{% endif %}
</div>
<!-- Adds nav links on each page -->
{% assign scrollnav = site.data.scrollnav[page.shortname] %}
{% if scrollnav %}
<div class="scrollnav">
{% if scrollnav.prev %}
<a class="btn btn-primary prev" href="{{ scrollnav.prev.url }}">{{ scrollnav.prev.title }}</a>
{% endif %}
{% if scrollnav.next %}
<a class="btn btn-primary next" href="{{ scrollnav.next.url }}">{{ scrollnav.next.title }}</a>
{% endif %}
</div>
{% endif %}
</div>
<!-- Adds comments from Apache's Comment system -->
<div id="comments_thread">
</div>
<script type="text/javascript" src="https://comments.apache.org/show_comments.lua?site=solr-refguide&style=css/comments.css&page={{ page.shortname }}" async="true">
</script>
<noscript>
<iframe width="100%" height="500" src="https://comments.apache.org/iframe.lua?site=solr-refguide&style=css/comments.css&page={{ page.shortname }}"></iframe>
</noscript>
{% include footer.html %}

View File

@ -0,0 +1,15 @@
---
layout: default_print
comments: true
---
<div class="post-header">
<h1 class="post-title-main" id="{{page.permalink | replace: '/', '' }}">{{ page.title }}</h1>
</div>
<div class="post-content">
{% if page.summary %}
<div class="summary">{{page.summary}}</div>
{% endif %}
{{ content }}
</div>

View File

@ -0,0 +1,41 @@
---
layout: default
---
<article class="post" itemscope itemtype="http://schema.org/BlogPosting">
<header class="post-header">
<h1 class="post-title" itemprop="name headline">{{ page.title }}</h1>
<p class="post-meta"><time datetime="{{ page.date | date_to_xmlschema }}" itemprop="datePublished">{{ page.date | date: "%b %-d, %Y" }}</time> {% if page.author %}<span itemprop="author" itemscope itemtype="http://schema.org/Person"><span itemprop="name">/ {{ page.author }}</span></span>{% endif %}{% if page.tags != null %}/
{% assign projectTags = site.data.tags.allowed-tags %}
{% for tag in page.tags %}
{% if projectTags contains tag %}
<a href="{{ "../tag_" | append: tag | append: ".html" }}">{{tag}}</a>{% unless forloop.last %}, {% endunless%}
{% endif %}
{% endfor %}
{% endif %}
</p>
</header>
<div class="post-content" itemprop="articleBody">
{% if page.summary %}
<div class="summary">{{page.summary}}</div>
{% endif %}
{{ content }}
</div>
</article>
{% if site.disqus %}
{% include disqus.html %}
{% endif %}
{{site.data.alerts.hr_shaded}}
{% include footer.html %}

View File

@ -0,0 +1,33 @@
= A Quick Overview
:page-shortname: a-quick-overview
:page-permalink: a-quick-overview.html
Having had some fun with Solr, you will now learn about all the cool things it can do.
Here is a example of how Solr might be integrated into an application:
.Solr integration with applications
image::images/a-quick-overview/sample-client-app-arch.png[image,width=500,height=379]
In the scenario above, Solr runs along side other server applications. For example, an online store application would provide a user interface, a shopping cart, and a way to make purchases for end users; while an inventory management application would allow store employees to edit product information. The product metadata would be kept in some kind of database, as well as in Solr.
Solr makes it easy to add the capability to search through the online store through the following steps:
. Define a _schema_. The schema tells Solr about the contents of documents it will be indexing. In the online store example, the schema would define fields for the product name, description, price, manufacturer, and so on. Solr's schema is powerful and flexible and allows you to tailor Solr's behavior to your application. See <<documents-fields-and-schema-design.adoc#documents-fields-and-schema-design,Documents, Fields, and Schema Design>> for all the details.
. Deploy Solr.
. Feed Solr documents for which your users will search.
. Expose search functionality in your application.
Because Solr is based on open standards, it is highly extensible. Solr queries are RESTful, which means, in essence, that a query is a simple HTTP request URL and the response is a structured document: mainly XML, but it could also be JSON, CSV, or some other format. This means that a wide variety of clients will be able to use Solr, from other web applications to browser clients, rich client applications, and mobile devices. Any platform capable of HTTP can talk to Solr. See <<client-apis.adoc#client-apis,Client APIs>> for details on client APIs.
Solr is based on the Apache Lucene project, a high-performance, full-featured search engine. Solr offers support for the simplest keyword searching through to complex queries on multiple fields and faceted search results. <<searching.adoc#searching,Searching>> has more information about searching and queries.
If Solr's capabilities are not impressive enough, its ability to handle very high-volume applications should do the trick.
A relatively common scenario is that you have so much data, or so many queries, that a single Solr server is unable to handle your entire workload. In this case, you can scale up the capabilities of your application using <<solrcloud.adoc#solrcloud,SolrCloud>> to better distribute the data, and the processing of requests, across many servers. Multiple options can be mixed and matched depending on the type of scalability you need.
For example: "Sharding" is a scaling technique in which a collection is split into multiple logical pieces called "shards" in order to scale up the number of documents in a collection beyond what could physically fit on a single server. Incoming queries are distributed to every shard in the collection, which respond with merged results. Another technique available is to increase the "Replication Factor" of your collection, which allows you to add servers with additional copies of your collection to handle higher concurrent query load by spreading the requests around to multiple machines. Sharding and Replication are not mutually exclusive, and together make Solr an extremely powerful and scalable platform.
Best of all, this talk about high-volume applications is not just hypothetical: some of the famous Internet sites that use Solr today are Macy's, EBay, and Zappo's.
For more information, take a look at https://wiki.apache.org/solr/PublicServers.

View File

@ -0,0 +1,54 @@
= A Step Closer
:page-shortname: a-step-closer
:page-permalink: a-step-closer.html
You already have some idea of Solr's schema. This section describes Solr's home directory and other configuration options.
When Solr runs in an application server, it needs access to a home directory. The home directory contains important configuration information and is the place where Solr will store its index. The layout of the home directory will look a little different when you are running Solr in standalone mode vs when you are running in SolrCloud mode.
The crucial parts of the Solr home directory are shown in these examples:
.Standalone Mode
[source,plain]
----
<solr-home-directory>/
solr.xml
core_name1/
core.properties
conf/
solrconfig.xml
managed-schema
data/
core_name2/
core.properties
conf/
solrconfig.xml
managed-schema
data/
----
.SolrCloud Mode
[source,plain]
----
<solr-home-directory>/
solr.xml
core_name1/
core.properties
data/
core_name2/
core.properties
data/
----
You may see other files, but the main ones you need to know are:
* `solr.xml` specifies configuration options for your Solr server instance. For more information on `solr.xml` see <<solr-cores-and-solr-xml.adoc#solr-cores-and-solr-xml,Solr Cores and solr.xml>>.
* Per Solr Core:
** `core.properties` defines specific properties for each core such as its name, the collection the core belongs to, the location of the schema, and other parameters. For more details on `core.properties`, see the section <<defining-core-properties.adoc#defining-core-properties,Defining core.properties>>.
** `solrconfig.xml` controls high-level behavior. You can, for example, specify an alternate location for the data directory. For more information on `solrconfig.xml`, see <<configuring-solrconfig-xml.adoc#configuring-solrconfig-xml,Configuring solrconfig.xml>>.
** `managed-schema` (or `schema.xml` instead) describes the documents you will ask Solr to index. The Schema define a document as a collection of fields. You get to define both the field types and the fields themselves. Field type definitions are powerful and include information about how Solr processes incoming field values and query values. For more information on Solr Schemas, see <<documents-fields-and-schema-design.adoc#documents-fields-and-schema-design,Documents, Fields, and Schema Design>> and the <<schema-api.adoc#schema-api,Schema API>>.
** `data/` The directory containing the low level index files.
Note that the SolrCloud example does not include a `conf` directory for each Solr Core (so there is no `solrconfig.xml` or Schema file). This is because the configuration files usually found in the `conf` directory are stored in ZooKeeper so they can be propagated across the cluster.
If you are using SolrCloud with the embedded ZooKeeper instance, you may also see `zoo.cfg` and `zoo.data` which are ZooKeeper configuration and data files. However, if you are running your own ZooKeeper ensemble, you would supply your own ZooKeeper configuration file when you start it and the copies in Solr would be unused. For more information about ZooKeeper and SolrCloud, see the section <<solrcloud.adoc#solrcloud,SolrCloud>>.

View File

@ -0,0 +1,29 @@
= About Filters
:page-shortname: about-filters
:page-permalink: about-filters.html
Like <<tokenizers.adoc#tokenizers,tokenizers>>, <<filter-descriptions.adoc#filter-descriptions,filters>> consume input and produce a stream of tokens. Filters also derive from `org.apache.lucene.analysis.TokenStream`. Unlike tokenizers, a filter's input is another TokenStream. The job of a filter is usually easier than that of a tokenizer since in most cases a filter looks at each token in the stream sequentially and decides whether to pass it along, replace it or discard it.
A filter may also do more complex analysis by looking ahead to consider multiple tokens at once, although this is less common. One hypothetical use for such a filter might be to normalize state names that would be tokenized as two words. For example, the single token "california" would be replaced with "CA", while the token pair "rhode" followed by "island" would become the single token "RI".
Because filters consume one `TokenStream` and produce a new `TokenStream`, they can be chained one after another indefinitely. Each filter in the chain in turn processes the tokens produced by its predecessor. The order in which you specify the filters is therefore significant. Typically, the most general filtering is done first, and later filtering stages are more specialized.
[source,xml]
----
<fieldType name="text" class="solr.TextField">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"/>
</analyzer>
</fieldType>
----
This example starts with Solr's standard tokenizer, which breaks the field's text into tokens. Those tokens then pass through Solr's standard filter, which removes dots from acronyms, and performs a few other common operations. All the tokens are then set to lowercase, which will facilitate case-insensitive matching at query time.
The last filter in the above example is a stemmer filter that uses the Porter stemming algorithm. A stemmer is basically a set of mapping rules that maps the various forms of a word back to the base, or _stem_, word from which they derive. For example, in English the words "hugs", "hugging" and "hugged" are all forms of the stem word "hug". The stemmer will replace all of these terms with "hug", which is what will be indexed. This means that a query for "hug" will match the term "hugged", but not "huge".
Conversely, applying a stemmer to your query terms will allow queries containing non stem terms, like "hugging", to match documents with different variations of the same stem word, such as "hugged". This works because both the indexer and the query will map to the same stem ("hug").
Word stemming is, obviously, very language specific. Solr includes several language-specific stemmers created by the http://snowball.tartarus.org/[Snowball] generator that are based on the Porter stemming algorithm. The generic Snowball Porter Stemmer Filter can be used to configure any of these language stemmers. Solr also includes a convenience wrapper for the English Snowball stemmer. There are also several purpose-built stemmers for non-English languages. These stemmers are described in <<language-analysis.adoc#language-analysis,Language Analysis>>.

View File

@ -0,0 +1,57 @@
= About This Guide
:page-shortname: about-this-guide
:page-permalink: about-this-guide.html
This guide describes all of the important features and functions of Apache Solr.
Solr is free to download from http://lucene.apache.org/solr/.
Designed to provide high-level documentation, this guide is intended to be more encyclopedic and less of a cookbook. It is structured to address a broad spectrum of needs, ranging from new developers getting started to well-experienced developers extending their application or troubleshooting. It will be of use at any point in the application life cycle, for whenever you need authoritative information about Solr.
The material as presented assumes that you are familiar with some basic search concepts and that you can read XML. It does not assume that you are a Java programmer, although knowledge of Java is helpful when working directly with Lucene or when developing custom extensions to a Lucene/Solr installation.
[[AboutThisGuide-SpecialInlineNotes]]
== Special Inline Notes
Special notes are included throughout these pages. There are several types of notes:
Information blocks::
+
NOTE: These provide additional information that's useful for you to know.
Important::
+
IMPORTANT: These provide information that is critical for you to know.
Tip::
+
TIP: These provide helpful tips.
Caution::
+
CAUTION: These provide details on scenarios or configurations you should be careful with.
Warning::
+
WARNING: These are meant to warn you from a possibly dangerous change or action.
[[AboutThisGuide-HostsandPortExamples]]
== Hosts and Port Examples
The default port when running Solr is 8983. The samples, URLs and screenshots in this guide may show different ports, because the port number that Solr uses is configurable. If you have not customized your installation of Solr, please make sure that you use port 8983 when following the examples, or configure your own installation to use the port numbers shown in the examples. For information about configuring port numbers, see the section <<managing-solr.adoc#managing-solr,Managing Solr>>.
Similarly, URL examples use 'localhost' throughout; if you are accessing Solr from a location remote to the server hosting Solr, replace 'localhost' with the proper domain or IP where Solr is running.
For example, we might provide a sample query like:
`\http://localhost:8983/solr/gettingstarted/select?q=brown+cow`
There are several items in this URL you might need to change locally. First, if your server is running at "www.example.com", you'll replace "localhost" with the proper domain. If you aren't using port 8983, you'll replace that also. Finally, you'll want to replace "gettingstarted" (the collection or core name) with the proper one in use in your implementation. The URL would then become:
`\http://www.example.com/solr/mycollection/select?q=brown+cow`
[[AboutThisGuide-Paths]]
== Paths
Path information is given relative to `solr.home`, which is the location under the main Solr installation where Solr's collections and their `conf` and `data` directories are stored. When running the various examples mentioned through out this tutorial (i.e., `bin/solr -e techproducts`) the `solr.home` will be a sub-directory of `example/` created for you automatically.

View File

@ -0,0 +1,31 @@
= About Tokenizers
:page-shortname: about-tokenizers
:page-permalink: about-tokenizers.html
The job of a <<tokenizers.adoc#tokenizers,tokenizer>> is to break up a stream of text into tokens, where each token is (usually) a sub-sequence of the characters in the text. An analyzer is aware of the field it is configured for, but a tokenizer is not. Tokenizers read from a character stream (a Reader) and produce a sequence of Token objects (a TokenStream).
Characters in the input stream may be discarded, such as whitespace or other delimiters. They may also be added to or replaced, such as mapping aliases or abbreviations to normalized forms. A token contains various metadata in addition to its text value, such as the location at which the token occurs in the field. Because a tokenizer may produce tokens that diverge from the input text, you should not assume that the text of the token is the same text that occurs in the field, or that its length is the same as the original text. It's also possible for more than one token to have the same position or refer to the same offset in the original text. Keep this in mind if you use token metadata for things like highlighting search results in the field text.
[source,xml]
----
<fieldType name="text" class="solr.TextField">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
</analyzer>
</fieldType>
----
The class named in the tokenizer element is not the actual tokenizer, but rather a class that implements the `TokenizerFactory` API. This factory class will be called upon to create new tokenizer instances as needed. Objects created by the factory must derive from `Tokenizer`, which indicates that they produce sequences of tokens. If the tokenizer produces tokens that are usable as is, it may be the only component of the analyzer. Otherwise, the tokenizer's output tokens will serve as input to the first filter stage in the pipeline.
A `TypeTokenFilterFactory` is available that creates a `TypeTokenFilter` that filters tokens based on their TypeAttribute, which is set in `factory.getStopTypes`.
For a complete list of the available TokenFilters, see the section <<tokenizers.adoc#tokenizers,Tokenizers>>.
[[AboutTokenizers-WhenTouseaCharFiltervs.aTokenFilter]]
== When To use a CharFilter vs. a TokenFilter
There are several pairs of CharFilters and TokenFilters that have related (ie: `MappingCharFilter` and `ASCIIFoldingFilter`) or nearly identical (ie: `PatternReplaceCharFilterFactory` and `PatternReplaceFilterFactory`) functionality and it may not always be obvious which is the best choice.
The decision about which to use depends largely on which Tokenizer you are using, and whether you need to preprocess the stream of characters.
For example, suppose you have a tokenizer such as `StandardTokenizer` and although you are pretty happy with how it works overall, you want to customize how some specific characters behave. You could modify the rules and re-build your own tokenizer with JFlex, but it might be easier to simply map some of the characters before tokenization with a `CharFilter`.

View File

@ -0,0 +1,156 @@
= Adding Custom Plugins in SolrCloud Mode
:page-shortname: adding-custom-plugins-in-solrcloud-mode
:page-permalink: adding-custom-plugins-in-solrcloud-mode.html
In SolrCloud mode, custom plugins need to be shared across all nodes of the cluster.
When running Solr in SolrCloud mode and you want to use custom code (such as custom analyzers, tokenizers, query parsers, and other plugins), it can be cumbersome to add jars to the classpath on all nodes in your cluster. Using the <<blob-store-api.adoc#blob-store-api,Blob Store API>> and special commands with the <<config-api.adoc#config-api,Config API>>, you can upload jars to a special system-level collection and dynamically load plugins from them at runtime with out needing to restart any nodes.
.This Feature is Disabled By Default
[IMPORTANT]
====
In addition to requiring that Solr by running in <<solrcloud.adoc#solrcloud,SolrCloud>> mode, this feature is also disabled by default unless all Solr nodes are run with the `-Denable.runtime.lib=true` option on startup.
Before enabling this feature, users should carefully consider the issues discussed in the <<Securing Runtime Libraries>> section below.
====
[[AddingCustomPluginsinSolrCloudMode-UploadingJarFiles]]
== Uploading Jar Files
The first step is to use the <<blob-store-api.adoc#blob-store-api,Blob Store API>> to upload your jar files. This will to put your jars in the `.system` collection and distribute them across your SolrCloud nodes. These jars are added to a separate classloader and only accessible to components that are configured with the property `runtimeLib=true`. These components are loaded lazily because the `.system` collection may not be loaded when a particular core is loaded.
[[AddingCustomPluginsinSolrCloudMode-ConfigAPICommandstouseJarsasRuntimeLibraries]]
== Config API Commands to use Jars as Runtime Libraries
The runtime library feature uses a special set of commands for the <<config-api.adoc#config-api,Config API>> to add, update, or remove jar files currently available in the blob store to the list of runtime libraries.
The following commands are used to manage runtime libs:
* `add-runtimelib`
* `update-runtimelib`
* `delete-runtimelib`
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
"add-runtimelib": { "name":"jarblobname", "version":2 },
"update-runtimelib": { "name":"jarblobname", "version":3 },
"delete-runtimelib": "jarblobname"
}'
----
The name to use is the name of the blob that you specified when you uploaded your jar to the blob store. You should also include the version of the jar found in the blob store that you want to use. These details are added to `configoverlay.json`.
The default `SolrResourceLoader` does not have visibility to the jars that have been defined as runtime libraries. There is a classloader that can access these jars which is made available only to those components which are specially annotated.
Every pluggable component can have an optional extra attribute called `runtimeLib=true`, which means that the components are not loaded at core load time. Instead, they will be loaded on demand. If all the dependent jars are not available when the component is loaded, an error is thrown.
This example shows creating a ValueSourceParser using a jar that has been loaded to the Blob store.
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
"create-valuesourceparser": {
"name": "nvl",
"runtimeLib": true,
"class": "solr.org.apache.solr.search.function.NvlValueSourceParser,
"nvlFloatValue": 0.0 }
}'
----
[[AddingCustomPluginsinSolrCloudMode-SecuringRuntimeLibraries]]
== Securing Runtime Libraries
A drawback of this feature is that it could be used to load malicious executable code into the system. However, it is possible to restrict the system to load only trusted jars using http://en.wikipedia.org/wiki/Public_key_infrastructure[PKI] to verify that the executables loaded into the system are trustworthy.
The following steps will allow you enable security for this feature. The instructions assume you have started all your Solr nodes with the `-Denable.runtime.lib=true`.
[[Step1_GenerateanRSAPrivateKey]]
=== Step 1: Generate an RSA Private Key
The first step is to generate an RSA private key. The example below uses a 512-bit key, but you should use the strength appropriate to your needs.
[source,bash]
----
$ openssl genrsa -out priv_key.pem 512
----
[[Step2_OutputthePublicKey]]
=== Step 2: Output the Public Key
The public portion of the key should be output in DER format so Java can read it.
[source,bash]
----
$ openssl rsa -in priv_key.pem -pubout -outform DER -out pub_key.der
----
[[Step3_LoadtheKeytoZooKeeper]]
=== Step 3: Load the Key to ZooKeeper
The `.der` files that are output from Step 2 should then be loaded to ZooKeeper under a node `/keys/exe` so they are available throughout every node. You can load any number of public keys to that node and all are valid. If a key is removed from the directory, the signatures of that key will cease to be valid. So, before removing the a key, make sure to update your runtime library configurations with valid signatures with the `update-runtimelib` command.
At the current time, you can only use the ZooKeeper `zkCli.sh` (or `zkCli.cmd` on Windows) script to issue these commands (the Solr version has the same name, but is not the same). If you have your own ZooKeeper ensemble running already, you can find the script in `$ZK_INSTALL/bin/zkCli.sh` (or `zkCli.cmd` if you are using Windows).
NOTE: If you are running the embedded ZooKeeper that is included with Solr, you *do not* have this script already; in order to use it, you will need to download a copy of ZooKeeper v3.4.6 from http://zookeeper.apache.org/. Don't worry about configuring the download, you're just trying to get the command line utility script. When you start the script, you will connect to the embedded ZooKeeper.
To load the keys, you will need to connect to ZooKeeper with `zkCli.sh`, create the directories, and then create the key file, as in the following example.
[source,bash]
----
# Connect to ZooKeeper
# Replace the server location below with the correct ZooKeeper connect string for your installation.
$ .bin/zkCli.sh -server localhost:9983
# After connection, you will interact with the ZK prompt.
# Create the directories
[zk: localhost:9983(CONNECTED) 5] create /keys
[zk: localhost:9983(CONNECTED) 5] create /keys/exe
# Now create the public key file in ZooKeeper
# The second path is the path to the .der file on your local machine
[zk: localhost:9983(CONNECTED) 5] create /keys/exe/pub_key.der /myLocal/pathTo/pub_key.der
----
After this, any attempt to load a jar will fail. All your jars must be signed with one of your private keys for Solr to trust it. The process to sign your jars and use the signature is outlined in Steps 4-6.
[[Step4_SignthejarFile]]
=== Step 4: Sign the jar File
Next you need to sign the sha1 digest of your jar file and get the base64 string.
[source,bash]
----
$ openssl dgst -sha1 -sign priv_key.pem myjar.jar | openssl enc -base64
----
The output of this step will be a string that you will need to add the jar to your classpath in Step 6 below.
[[Step5_LoadthejartotheBlobStore]]
=== Step 5: Load the jar to the Blob Store
Load your jar to the Blob store, using the <<blob-store-api.adoc#blob-store-api,Blob Store API>>. This step does not require a signature; you will need the signature in Step 6 to add it to your classpath.
[source,bash]
----
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @{filename}
http://localhost:8983/solr/.system/blob/{blobname}
----
The blob name that you give the jar file in this step will be used as the name in the next step.
[[Step6_AddthejartotheClasspath]]
=== Step 6: Add the jar to the Classpath
Finally, add the jar to the classpath using the Config API as detailed above. In this step, you will need to provide the signature of the jar that you got in Step 4.
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
"add-runtimelib": {
"name":"blobname",
"version":2,
"sig":"mW1Gwtz2QazjfVdrLFHfbGwcr8xzFYgUOLu68LHqWRDvLG0uLcy1McQ+AzVmeZFBf1yLPDEHBWJb5KXr8bdbHN/
PYgUB1nsr9pk4EFyD9KfJ8TqeH/ijQ9waa/vjqyiKEI9U550EtSzruLVZ32wJ7smvV0fj2YYhrUaaPzOn9g0=" }
}'
----

View File

@ -0,0 +1,17 @@
= Analysis Screen
:page-shortname: analysis-screen
:page-permalink: analysis-screen.html
The Analysis screen lets you inspect how data will be handled according to the field, field type and dynamic field configurations found in your Schema. You can analyze how content would be handled during indexing or during query processing and view the results separately or at the same time. Ideally, you would want content to be handled consistently, and this screen allows you to validate the settings in the field type or field analysis chains.
Enter content in one or both boxes at the top of the screen, and then choose the field or field type definitions to use for analysis.
image::images/analysis-screen/analysis_normal.png[image,height=400]
If you click the *Verbose Output* check box, you see more information, including more details on the transformations to the input (such as, convert to lower case, strip extra characters, etc.) including the raw bytes, type and detailed position information at each stage. The information displayed will vary depending on the settings of the field or field type. Each step of the process is displayed in a separate section, with an abbreviation for the tokenizer or filter that is applied in that step. Hover or click on the abbreviation, and you'll see the name and path of the tokenizer or filter.
image::images/analysis-screen/analysis_verbose.png[image,height=400]
In the example screenshot above, several transformations are applied to the input "Running is a sport." The words "is" and "a" have been removed and the word "running" has been changed to its basic form, "run". This is because we are using the field type `text_en` in this scenario, which is configured to remove stop words (small words that usually do not provide a great deal of context) and "stem" terms when possible to find more possible matches (this is particularly helpful with plural forms of words). If you click the question mark next to the *Analyze Fieldname/Field Type* pull-down menu, the <<schema-browser-screen.adoc#schema-browser-screen,Schema Browser window>> will open, showing you the settings for the field specified.
The section <<understanding-analyzers-tokenizers-and-filters.adoc#understanding-analyzers-tokenizers-and-filters,Understanding Analyzers, Tokenizers, and Filters>> describes in detail what each option is and how it may transform your data and the section <<running-your-analyzer.adoc#running-your-analyzer,Running Your Analyzer>> has specific examples for using the Analysis screen.

View File

@ -0,0 +1,103 @@
= Analyzers
:page-shortname: analyzers
:page-permalink: analyzers.html
An analyzer examines the text of fields and generates a token stream.
Analyzers are specified as a child of the `<fieldType>` element in the `schema.xml` configuration file (in the same `conf/` directory as `solrconfig.xml`).
In normal usage, only fields of type `solr.TextField` will specify an analyzer. The simplest way to configure an analyzer is with a single `<analyzer>` element whose class attribute is a fully qualified Java class name. The named class must derive from `org.apache.lucene.analysis.Analyzer`. For example:
[source,xml]
----
<fieldType name="nametext" class="solr.TextField">
<analyzer class="org.apache.lucene.analysis.core.WhitespaceAnalyzer"/>
</fieldType>
----
In this case a single class, `WhitespaceAnalyzer`, is responsible for analyzing the content of the named text field and emitting the corresponding tokens. For simple cases, such as plain English prose, a single analyzer class like this may be sufficient. But it's often necessary to do more complex analysis of the field content.
Even the most complex analysis requirements can usually be decomposed into a series of discrete, relatively simple processing steps. As you will soon discover, the Solr distribution comes with a large selection of tokenizers and filters that covers most scenarios you are likely to encounter. Setting up an analyzer chain is very straightforward; you specify a simple `<analyzer>` element (no class attribute) with child elements that name factory classes for the tokenizer and filters to use, in the order you want them to run.
For example:
[source,xml]
----
<fieldType name="nametext" class="solr.TextField">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"/>
</analyzer>
</fieldType>
----
Note that classes in the `org.apache.solr.analysis` package may be referred to here with the shorthand `solr.` prefix.
In this case, no Analyzer class was specified on the `<analyzer>` element. Rather, a sequence of more specialized classes are wired together and collectively act as the Analyzer for the field. The text of the field is passed to the first item in the list (`solr.StandardTokenizerFactory`), and the tokens that emerge from the last one (`solr.EnglishPorterFilterFactory`) are the terms that are used for indexing or querying any fields that use the "nametext" `fieldType`.
.Field Values versus Indexed Terms
[IMPORTANT]
====
The output of an Analyzer affects the _terms_ indexed in a given field (and the terms used when parsing queries against those fields) but it has no impact on the _stored_ value for the fields. For example: an analyzer might split "Brown Cow" into two indexed terms "brown" and "cow", but the stored value will still be a single String: "Brown Cow"
====
[[Analyzers-AnalysisPhases]]
== Analysis Phases
Analysis takes place in two contexts. At index time, when a field is being created, the token stream that results from analysis is added to an index and defines the set of terms (including positions, sizes, and so on) for the field. At query time, the values being searched for are analyzed and the terms that result are matched against those that are stored in the field's index.
In many cases, the same analysis should be applied to both phases. This is desirable when you want to query for exact string matches, possibly with case-insensitivity, for example. In other cases, you may want to apply slightly different analysis steps during indexing than those used at query time.
If you provide a simple `<analyzer>` definition for a field type, as in the examples above, then it will be used for both indexing and queries. If you want distinct analyzers for each phase, you may include two `<analyzer>` definitions distinguished with a type attribute. For example:
[source,xml]
----
<fieldType name="nametext" class="solr.TextField">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.KeepWordFilterFactory" words="keepwords.txt"/>
<filter class="solr.SynonymFilterFactory" synonyms="syns.txt"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
----
In this theoretical example, at index time the text is tokenized, the tokens are set to lowercase, any that are not listed in `keepwords.txt` are discarded and those that remain are mapped to alternate values as defined by the synonym rules in the file `syns.txt`. This essentially builds an index from a restricted set of possible values and then normalizes them to values that may not even occur in the original text.
At query time, the only normalization that happens is to convert the query terms to lowercase. The filtering and mapping steps that occur at index time are not applied to the query terms. Queries must then, in this example, be very precise, using only the normalized terms that were stored at index time.
[[Analyzers-AnalysisforMulti-TermExpansion]]
=== Analysis for Multi-Term Expansion
In some types of queries (ie: Prefix, Wildcard, Regex, etc...) the input provided by the user is not natural language intended for Analysis. Things like Synonyms or Stop word filtering do not work in a logical way in these types of Queries.
The analysis factories that _can_ work in these types of queries (such as Lowercasing, or Normalizing Factories) are known as {lucene-javadocs}/analyzers-common/org/apache/lucene/analysis/util/MultiTermAwareComponent.html[`MultiTermAwareComponents`]. When Solr needs to perform analysis for a query that results in Multi-Term expansion, only the `MultiTermAwareComponents` used in the `query` analyzer are used, Factory that is not Multi-Term aware will be skipped.
For most use cases, this provides the best possible behavior, but if you wish for absolute control over the analysis performed on these types of queries, you may explicitly define a `multiterm` analyzer to use, such as in the following example:
[source,xml]
----
<fieldType name="nametext" class="solr.TextField">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.KeepWordFilterFactory" words="keepwords.txt"/>
<filter class="solr.SynonymFilterFactory" synonyms="syns.txt"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<!-- No analysis at all when doing queries that involved Multi-Term expansion -->
<analyzer type="multiterm">
<tokenizer class="solr.KeywordTokenizerFactory" />
</analyzer>
</fieldType>
----

View File

@ -0,0 +1,170 @@
= Authentication and Authorization Plugins
:page-shortname: authentication-and-authorization-plugins
:page-permalink: authentication-and-authorization-plugins.html
:page-children: basic-authentication-plugin, hadoop-authentication-plugin, kerberos-authentication-plugin, rule-based-authorization-plugin
Solr has security frameworks for supporting authentication and authorization of users. This allows for verifying a user's identity and for restricting access to resources in a Solr cluster.
Solr includes some plugins out of the box, and additional plugins can be developed using the authentication and authorization frameworks described below.
All authentication and authorization plugins can work with Solr whether they are running in SolrCloud mode or standalone mode. All authentication and authorization configuration, including users and permission rules, are stored in a file named `security.json`. When using Solr in standalone mode, this file must be in the `$SOLR_HOME` directory (usually `server/solr`). When using SolrCloud, this file must be located in ZooKeeper.
The following section describes how to enable plugins with `security.json` and place them in the proper locations for your mode of operation.
[[AuthenticationandAuthorizationPlugins-EnablePluginswithsecurity.json]]
== Enable Plugins with security.json
All of the information required to initialize either type of security plugin is stored in a `security.json` file. This file contains 2 sections, one each for authentication and authorization.
.Sample security.json
[source,json]
----
{
"authentication" : {
"class": "class.that.implements.authentication"
},
"authorization": {
"class": "class.that.implements.authorization"
}
}
----
The `/security.json` file needs to be in the proper location before a Solr instance comes up so Solr starts with the security plugin enabled. See the section <<AuthenticationandAuthorizationPlugins-Usingsecurity.jsonwithSolr,Using security.json with Solr>> below for information on how to do this.
Depending on the plugin(s) in use, other information will be stored in `security.json` such as user information or rules to create roles and permissions. This information is added through the APIs for each plugin provided by Solr, or, in the case of a custom plugin, the approach designed by you.
Here is a more detailed `security.json` example. In this, the Basic authentication and rule-based authorization plugins are enabled, and some data has been added:
[source,json]
----
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{"name":"security-edit",
"role":"admin"}],
"user-role":{"solr":"admin"}
}}
----
[[AuthenticationandAuthorizationPlugins-Usingsecurity.jsonwithSolr]]
== Using security.json with Solr
[[AuthenticationandAuthorizationPlugins-InSolrCloudmode]]
=== In SolrCloud Mode
While configuring Solr to use an authentication or authorization plugin, you will need to upload a `security.json` file to ZooKeeper. The following command writes the file as it uploads it - you could also upload a file that you have already created locally.
[source,bash]
----
>server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd put /security.json
'{"authentication": {"class": "org.apache.solr.security.KerberosPlugin"}}'
----
Note that this example defines the `KerberosPlugin` for authentication. You will want to modify this section as appropriate for the plugin you are using.
This example also defines `security.json` on the command line, but you can also define a file locally and upload it to ZooKeeper.
[WARNING]
====
Depending on the authentication and authorization plugin that you use, you may have user information stored in `security.json`. If so, we highly recommend that you implement access control in your ZooKeeper nodes. Information about how to enable this is available in the section <<zookeeper-access-control.adoc#zookeeper-access-control,ZooKeeper Access Control>>.
====
Once `security.json` has been uploaded to ZooKeeper, you should use the appropriate APIs for the plugins you're using to update it. You can edit it manually, but you must take care to remove any version data so it will be properly updated across all ZooKeeper nodes. The version data is found at the end of the `security.json` file, and will appear as the letter "v" followed by a number, such as `{"v":138}`.
[[AuthenticationandAuthorizationPlugins-InStandaloneMode]]
=== In Standalone Mode
When running Solr in standalone mode, you need to create the `security.json` file and put it in the `$SOLR_HOME` directory for your installation (this is the same place you have located `solr.xml` and is usually `server/solr`).
If you are using <<legacy-scaling-and-distribution.adoc#legacy-scaling-and-distribution,Legacy Scaling and Distribution>>, you will need to place `security.json` on each node of the cluster.
You can use the authentication and authorization APIs, but if you are using the legacy scaling model, you will need to make the same API requests on each node separately. You can also edit `security.json` by hand if you prefer.
[[AuthenticationandAuthorizationPlugins-Authentication]]
== Authentication
Authentication plugins help in securing the endpoints of Solr by authenticating incoming requests. A custom plugin can be implemented by extending the AuthenticationPlugin class.
An authentication plugin consists of two parts:
. Server-side component, which intercepts and authenticates incoming requests to Solr using a mechanism defined in the plugin, such as Kerberos, Basic Auth or others.
. Client-side component, i.e., an extension of `HttpClientConfigurer`, which enables a SolrJ client to make requests to a secure Solr instance using the authentication mechanism which the server understands.
[[AuthenticationandAuthorizationPlugins-EnablingaPlugin]]
=== Enabling a Plugin
* Specify the authentication plugin in `/security.json` as in this example:
+
[source,json]
----
{
"authentication": {
"class": "class.that.implements.authentication",
"other_data" : "..."}
}
----
* All of the content in the authentication block of `security.json` would be passed on as a map to the plugin during initialization.
* An authentication plugin can also be used with a standalone Solr instance by passing in `-DauthenticationPlugin=<plugin class name>` during startup.
[[AuthenticationandAuthorizationPlugins-AvailableAuthenticationPlugins]]
=== Available Authentication Plugins
Solr has the following implementations of authentication plugins:
* <<kerberos-authentication-plugin.adoc#kerberos-authentication-plugin,Kerberos Authentication Plugin>>
* <<basic-authentication-plugin.adoc#basic-authentication-plugin,Basic Authentication Plugin>>
* <<hadoop-authentication-plugin.adoc#hadoop-authentication-plugin,Hadoop Authentication Plugin>>
[[AuthenticationandAuthorizationPlugins-Authorization]]
== Authorization
An authorization plugin can be written for Solr by extending the {solr-javadocs}/solr-core/org/apache/solr/security/AuthorizationPlugin.html[AuthorizationPlugin] interface.
[[AuthenticationandAuthorizationPlugins-LoadingaCustomPlugin]]
=== Loading a Custom Plugin
* Make sure that the plugin implementation is in the classpath.
* The plugin can then be initialized by specifying the same in `security.json` in the following manner:
[source,json]
----
{
"authorization": {
"class": "org.apache.solr.security.MockAuthorizationPlugin",
"other_data" : "..."}
}
----
All of the content in the `authorization` block of `security.json` would be passed on as a map to the plugin during initialization.
[IMPORTANT]
====
The authorization plugin is only supported in SolrCloud mode. Also, reloading the plugin isn't yet supported and requires a restart of the Solr installation (meaning, the JVM should be restarted, not simply a core reload).
====
[[AuthenticationandAuthorizationPlugins-AvailableAuthorizationPlugins]]
=== Available Authorization Plugins
Solr has one implementation of an authorization plugin:
* <<rule-based-authorization-plugin.adoc#rule-based-authorization-plugin,Rule-Based Authorization Plugin>>
[[AuthenticationandAuthorizationPlugins-PKISecuringInter-NodeRequests]]
[[AuthenticationandAuthorizationPlugins-PKI]]
== Securing Inter-Node Requests
There are a lot of requests that originate from the Solr nodes itself. For example, requests from overseer to nodes, recovery threads, etc. Each Authentication plugin declares whether it is capable of securing inter-node requests or not. If not, Solr will fall back to using a special internode authentication mechanism where each Solr node is a super user and is fully trusted by other Solr nodes, described below.
[[AuthenticationandAuthorizationPlugins-PKIAuthenticationPlugin]]
=== PKIAuthenticationPlugin
The PKIAuthenticationPlugin is used when there is any request going on between two Solr nodes, and the configured Authentication plugin does not wish to handle inter-node security.
For each outgoing request `PKIAuthenticationPlugin` adds a special header `'SolrAuth'` which carries the timestamp and principal encrypted using the private key of that node. The public key is exposed through an API so that any node can read it whenever it needs it. Any node who gets the request with that header, would get the public key from the sender and decrypt the information. If it is able to decrypt the data, the request trusted. It is invalid if the timestamp is more than 5 secs old. This assumes that the clocks of different nodes in the cluster are synchronized.
The timeout is configurable through a system property called `pkiauth.ttl`. For example, if you wish to bump up the time-to-live to 10 seconds (10000 milliseconds), start each node with a property `'-Dpkiauth.ttl=10000'`.

View File

@ -0,0 +1,140 @@
= Basic Authentication Plugin
:page-shortname: basic-authentication-plugin
:page-permalink: basic-authentication-plugin.html
Solr can support Basic authentication for users with the use of the BasicAuthPlugin.
An authorization plugin is also available to configure Solr with permissions to perform various activities in the system. The authorization plugin is described in the section <<rule-based-authorization-plugin.adoc#rule-based-authorization-plugin,Rule-Based Authorization Plugin>>.
[[BasicAuthenticationPlugin-EnableBasicAuthentication]]
== Enable Basic Authentication
To use Basic authentication, you must first create a `security.json` file. This file and where to put it is described in detail in the section <<authentication-and-authorization-plugins.adoc#AuthenticationandAuthorizationPlugins-EnablePluginswithsecurity.json,Enable Plugins with security.json>>.
For Basic authentication, the `security.json` file must have an `authentication` part which defines the class being used for authentication. Usernames and passwords (as a sha256(password+salt) hash) could be added when the file is created, or can be added later with the Basic authentication API, described below.
The `authorization` part is not related to Basic authentication, but is a separate authorization plugin designed to support fine-grained user access control. For more information, see the section <<rule-based-authorization-plugin.adoc#rule-based-authorization-plugin,Rule-Based Authorization Plugin>>.
An example `security.json` showing both sections is shown below to show how these plugins can work together:
[source,json]
----
{
"authentication":{
"blockUnknown": true,
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{"name":"security-edit",
"role":"admin"}],
"user-role":{"solr":"admin"}
}}
----
There are several things defined in this file:
* Basic authentication and rule-based authorization plugins are enabled.
* A user called 'solr', with a password `'SolrRocks'` has been defined.
* The parameter `"blockUnknown": true` means that unauthenticated requests are not allowed to pass through.
* The 'admin' role has been defined, and it has permission to edit security settings.
* The 'solr' user has been defined to the 'admin' role.
Save your settings to a file called `security.json` locally. If you are using Solr in standalone mode, you should put this file in `$SOLR_HOME`.
If `blockUnknown` does not appear in the `security.json` file, it will default to `false`. This has the effect of not requiring authentication at all. In some cases, you may want this; for example, if you want to have `security.json` in place but aren't ready to enable authentication. However, you will want to ensure that this parameter is set to `true` in order for authentication to be truly enabled in your system.
If you are using SolrCloud, you must upload `security.json` to ZooKeeper. You can use this example command, ensuring that the ZooKeeper port is correct:
[source,bash]
----
bin/solr zk cp file:path_to_local_security.json zk:/security.json -z localhost:9983
----
[[BasicAuthenticationPlugin-Caveats]]
=== Caveats
There are a few things to keep in mind when using the Basic authentication plugin.
* Credentials are sent in plain text by default. It's recommended to use SSL for communication when Basic authentication is enabled, as described in the section <<enabling-ssl.adoc#enabling-ssl,Enabling SSL>>.
* A user who has access to write permissions to `security.json` will be able to modify all the permissions and how users have been assigned permissions. Special care should be taken to only grant access to editing security to appropriate users.
* Your network should, of course, be secure. Even with Basic authentication enabled, you should not unnecessarily expose Solr to the outside world.
[[BasicAuthenticationPlugin-EditingAuthenticationPluginConfiguration]]
== Editing Authentication Plugin Configuration
An Authentication API allows modifying user IDs and passwords. The API provides an endpoint with specific commands to set user details or delete a user.
[[BasicAuthenticationPlugin-APIEntryPoint]]
=== API Entry Point
`admin/authentication`
This endpoint is not collection-specific, so users are created for the entire Solr cluster. If users need to be restricted to a specific collection, that can be done with the authorization rules.
[[BasicAuthenticationPlugin-AddaUserorEditaPassword]]
=== Add a User or Edit a Password
The `set-user` command allows you to add users and change their passwords. For example, the following defines two users and their passwords:
[source,bash]
----
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{
"set-user": {"tom" : "TomIsCool" ,
"harry":"HarrysSecret"}}'
----
[[BasicAuthenticationPlugin-DeleteaUser]]
=== Delete a User
The `delete-user` command allows you to remove a user. The user password does not need to be sent to remove a user. In the following example, we've asked that user IDs 'tom' and 'harry' be removed from the system.
[source,bash]
----
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{
"delete-user": ["tom","harry"]}'
----
[[BasicAuthenticationPlugin-Setaproperty]]
=== Set a property
Set arbitrary properties for authentication plugin. The only supported property is `'blockUnknown'`
[source,bash]
----
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{
"set-property": {"blockUnknown":false}}'
----
[[BasicAuthenticationPlugin-UsingBasicAuthwithSolrJ]]
=== Using BasicAuth with SolrJ
In SolrJ, the basic authentication credentials need to be set for each request as in this example:
[source,java]
----
SolrRequest req ;//create a new request object
req.setBasicAuthCredentials(userName, password);
solrClient.request(req);
----
Query example:
[source,java]
----
QueryRequest req = new QueryRequest(new SolrQuery("*:*"));
req.setBasicAuthCredentials(userName, password);
QueryResponse rsp = req.process(solrClient);
----
[[BasicAuthenticationPlugin-UsingCommandLinescriptswithBasicAuth]]
=== Using Command Line scripts with BasicAuth
Add the following line to the `solr.in.sh` or `solr.in.cmd` file. This example tells the `bin/solr` command line to to use "basic" as the type of authentication, and to pass credentials with the user-name "solr" and password "SolrRocks":
[source,bash]
----
SOLR_AUTH_TYPE="basic"
SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
----

View File

@ -0,0 +1,135 @@
= Blob Store API
:page-shortname: blob-store-api
:page-permalink: blob-store-api.html
The Blob Store REST API provides REST methods to store, retrieve or list files in a Lucene index.
It can be used to upload a jar file which contains standard solr components such as RequestHandlers, SearchComponents, or other custom code you have written for Solr. Schema components _do not_ yet support the Blob Store.
When using the blob store, note that the API does not delete or overwrite a previous object if a new one is uploaded with the same name. It always adds a new version of the blob to the index. Deletes can be performed with standard REST delete commands.
*The blob store is only available when running in SolrCloud mode.* Solr in standalone mode does not support use of a blob store.
The blob store API is implemented as a requestHandler. A special collection named ".system" is used to store the blobs. This collection can be created in advance, but if it does not exist it will be created automatically.
[[BlobStoreAPI-Aboutthe.systemCollection]]
== About the .system Collection
Before uploading blobs to the blob store, a special collection must be created and it must be named `.system`. Solr will automatically create this collection if it does not already exist, but you can also create it manually if you choose.
The BlobHandler is automatically registered in the .system collection. The `solrconfig.xml`, Schema, and other configuration files for the collection are automatically provided by the system and don't need to be defined specifically.
If you do not use the `-shards` or `-replicationFactor` options, then defaults of numShards=1 and replicationFactor=3 (or maximum nodes in the cluster) will be used.
You can create the `.system` collection with the <<collections-api.adoc#collections-api,Collections API>>, as in this example:
[source,bash]
----
curl http://localhost:8983/solr/admin/collections?action=CREATE&name=.system&replicationFactor=2
----
IMPORTANT: The `bin/solr` script cannot be used to create the `.system` collection.
[[BlobStoreAPI-UploadFilestoBlobStore]]
== Upload Files to Blob Store
After the `.system` collection has been created, files can be uploaded to the blob store with a request similar to the following:
[source,bash]
----
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @{filename} http://localhost:8983/solr/.system/blob/{blobname}
----
For example, to upload a file named "test1.jar" as a blob named "test", you would make a POST request like:
[source,bash]
----
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @test1.jar http://localhost:8983/solr/.system/blob/test
----
A GET request will return the list of blobs and other details:
[source,bash]
----
curl http://localhost:8983/solr/.system/blob?omitHeader=true
----
Output:
[source,json]
----
{
"response":{"numFound":1,"start":0,"docs":[
{
"id":"test/1",
"md5":"20ff915fa3f5a5d66216081ae705c41b",
"blobName":"test",
"version":1,
"timestamp":"2015-02-04T16:45:48.374Z",
"size":13108}]
}
}
----
Details on individual blobs can be accessed with a request similar to:
[source,bash]
----
curl http://localhost:8983/solr/.system/blob/{blobname}
----
For example, this request will return only the blob named 'test':
[source,bash]
----
curl http://localhost:8983/solr/.system/blob/test?omitHeader=true
----
Output:
[source,json]
----
{
"response":{"numFound":1,"start":0,"docs":[
{
"id":"test/1",
"md5":"20ff915fa3f5a5d66216081ae705c41b",
"blobName":"test",
"version":1,
"timestamp":"2015-02-04T16:45:48.374Z",
"size":13108}]
}
}
----
The filestream response writer can return a particular version of a blob for download, as in:
[source,bash]
----
curl http://localhost:8983/solr/.system/blob/{blobname}/{version}?wt=filestream > {outputfilename}
----
For the latest version of a blob, the \{version} can be omitted,
[source,bash]
----
curl http://localhost:8983/solr/.system/blob/{blobname}?wt=filestream > {outputfilename}
----
[[BlobStoreAPI-UseaBlobinaHandlerorComponent]]
== Use a Blob in a Handler or Component
To use the blob as the class for a request handler or search component, you create a request handler in `solrconfig.xml` as usual. You will need to define the following parameters:
`class`:: the fully qualified class name. For example, if you created a new request handler class called CRUDHandler, you would enter `org.apache.solr.core.CRUDHandler`.
`runtimeLib`:: Set to true to require that this component should be loaded from the classloader that loads the runtime jars.
For example, to use a blob named test, you would configure `solrconfig.xml` like this:
[source,xml]
----
<requestHandler name="/myhandler" class="org.apache.solr.core.myHandler" runtimeLib="true" version="1">
</requestHandler>
----
If there are parameters available in the custom handler, you can define them in the same way as any other request handler definition.

View File

@ -0,0 +1,99 @@
= BlockJoin Faceting
:page-shortname: blockjoin-faceting
:page-permalink: blockjoin-faceting.html
BlockJoin facets allow you to aggregate children facet counts by their parents.
It is a common requirement that if a parent document has several children documents, all of them need to increment facet value count only once. This functionality is provided by `BlockJoinDocSetFacetComponent`, and `BlockJoinFacetComponent` just an alias for compatibility.
CAUTION: This component is considered experimental, and must be explicitly enabled for a request handler in `solrconfig.xml`, in the same way as any other <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,search component>>.
This example shows how you could add this search components to `solrconfig.xml` and define it in request handler:
[source,xml]
----
<searchComponent name="bjqFacetComponent" class="org.apache.solr.search.join.BlockJoinDocSetFacetComponent"/>
<requestHandler name="/bjqfacet" class="org.apache.solr.handler.component.SearchHandler">
<lst name="defaults">
<str name="shards.qt">/bjqfacet</str>
</lst>
<arr name="last-components">
<str>bjqFacetComponent</str>
</arr>
</requestHandler>
----
This component can be added into any search request handler. This component work with distributed search in SolrCloud mode.
Documents should be added in children-parent blocks as described in <<uploading-data-with-index-handlers.adoc#UploadingDatawithIndexHandlers-NestedChildDocuments,indexing nested child documents>>. Examples:
.Sample document
[source,xml]
----
<add>
<doc>
<field name="id">1</field>
<field name="type_s">parent</field>
<doc>
<field name="id">11</field>
<field name="COLOR_s">Red</field>
<field name="SIZE_s">XL</field>
<field name="PRICE_i">6</field>
</doc>
<doc>
<field name="id">12</field>
<field name="COLOR_s">Red</field>
<field name="SIZE_s">XL</field>
<field name="PRICE_i">7</field>
</doc>
<doc>
<field name="id">13</field>
<field name="COLOR_s">Blue</field>
<field name="SIZE_s">L</field>
<field name="PRICE_i">5</field>
</doc>
</doc>
<doc>
<field name="id">2</field>
<field name="type_s">parent</field>
<doc>
<field name="id">21</field>
<field name="COLOR_s">Blue</field>
<field name="SIZE_s">XL</field>
<field name="PRICE_i">6</field>
</doc>
<doc>
<field name="id">22</field>
<field name="COLOR_s">Blue</field>
<field name="SIZE_s">XL</field>
<field name="PRICE_i">7</field>
</doc>
<doc>
<field name="id">23</field>
<field name="COLOR_s">Red</field>
<field name="SIZE_s">L</field>
<field name="PRICE_i">5</field>
</doc>
</doc>
</add>
----
Queries are constructed the same way as for a <<other-parsers.adoc#OtherParsers-BlockJoinQueryParsers,Parent Block Join query>>. For example:
[source,text]
----
http://localhost:8983/solr/bjqfacet?q={!parent which=type_s:parent}SIZE_s:XL&child.facet.field=COLOR_s
----
As a result we should have facets for Red(1) and Blue(1), because matches on children `id=11` and `id=12` are aggregated into single hit into parent with `id=1`. The key components of the request are:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="30,70",options="header"]
|===
|URL Part | Meaning
|`/bjqfacet` |The name of the request handler that has been defined with one of block join facet components enabled.
|`q={!parent ...}..` |The mandatory parent query as a main query. The parent query could also be a subordinate clause in a more complex query.
|`child.facet.field=...` |The child document field, which might be repeated many times with several fields, as necessary.
|===

View File

@ -0,0 +1,159 @@
= CharFilterFactories
:page-shortname: charfilterfactories
:page-permalink: charfilterfactories.html
CharFilter is a component that pre-processes input characters.
CharFilters can be chained like Token Filters and placed in front of a Tokenizer. CharFilters can add, change, or remove characters while preserving the original character offsets to support features like highlighting.
[[CharFilterFactories-solr.MappingCharFilterFactory]]
== solr.MappingCharFilterFactory
This filter creates `org.apache.lucene.analysis.MappingCharFilter`, which can be used for changing one string to another (for example, for normalizing `é` to `e`.).
This filter requires specifying a `mapping` argument, which is the path and name of a file containing the mappings to perform.
Example:
[source,xml]
----
<analyzer>
<charFilter class="solr.MappingCharFilterFactory" mapping="mapping-FoldToASCII.txt"/>
<tokenizer ...>
[...]
</analyzer>
----
Mapping file syntax:
* Comment lines beginning with a hash mark (`#`), as well as blank lines, are ignored.
* Each non-comment, non-blank line consists of a mapping of the form: `"source" => "target"`
** Double-quoted source string, optional whitespace, an arrow (`=>`), optional whitespace, double-quoted target string.
* Trailing comments on mapping lines are not allowed.
* The source string must contain at least one character, but the target string may be empty.
* The following character escape sequences are recognized within source and target strings:
+
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
[cols="20,30,20,30",options="header"]
|===
|Escape Sequence |Resulting Character (http://www.ecma-international.org/publications/standards/Ecma-048.htm[ECMA-48] alias) |Unicode Character |Example Mapping Line
|`\\` |`\` |U+005C |`"\\" => "/"`
|`\"` |`"` |U+0022 |`"\"and\"" => "'and'"`
|`\b` |backspace (BS) |U+0008 |`"\b" => " "`
|`\t` |tab (HT) |U+0009 |`"\t" => ","`
|`\n` |newline (LF) |U+000A |`"\n" => "<br>"`
|`\f` |form feed (FF) |U+000C |`"\f" => "\n"`
|`\r` |carriage return (CR) |U+000D |`"\r" => "/carriage-return/"`
|`\uXXXX` |Unicode char referenced by the 4 hex digits |U+XXXX |`"\uFEFF" => ""`
|===
** A backslash followed by any other character is interpreted as if the character were present without the backslash.
[[CharFilterFactories-solr.HTMLStripCharFilterFactory]]
== solr.HTMLStripCharFilterFactory
This filter creates `org.apache.solr.analysis.HTMLStripCharFilter`. This CharFilter strips HTML from the input stream and passes the result to another CharFilter or a Tokenizer.
This filter:
* Removes HTML/XML tags while preserving other content.
* Removes attributes within tags and supports optional attribute quoting.
* Removes XML processing instructions, such as: <?foo bar?>
* Removes XML comments.
* Removes XML elements starting with <!>.
* Removes contents of <script> and <style> elements.
* Handles XML comments inside these elements (normal comment processing will not always work).
* Replaces numeric character entities references like `&#65`; or `&#x7f`; with the corresponding character.
* The terminating ';' is optional if the entity reference at the end of the input; otherwise the terminating ';' is mandatory, to avoid false matches on something like "Alpha&Omega Corp".
* Replaces all named character entity references with the corresponding character.
* `&nbsp`; is replaced with a space instead of the 0xa0 character.
* Newlines are substituted for block-level elements.
* <CDATA> sections are recognized.
* Inline tags, such as `<b>`, `<i>`, or `<span>` will be removed.
* Uppercase character entities like `quot`, `gt`, `lt` and `amp` are recognized and handled as lowercase.
TIP: The input need not be an HTML document. The filter removes only constructs that look like HTML. If the input doesn't include anything that looks like HTML, the filter won't remove any input.
The table below presents examples of HTML stripping.
[width="100%",options="header",]
|===
|Input |Output
|`my <a href="www.foo.bar">link</a>` |my link
|`<br>hello<!--comment-->` |hello
|`hello<script><!-- f('<!--internal--></script>'); --></script>` |hello
|`if a<b then print a;` |if a<b then print a;
|`hello <td height=22 nowrap align="left">` |hello
|`a<b &#65 Alpha&Omega Ω` |a<b A Alpha&Omega Ω
|===
Example:
[source,xml]
----
<analyzer>
<charFilter class="solr.HTMLStripCharFilterFactory"/>
<tokenizer ...>
[...]
</analyzer>
----
[[CharFilterFactories-solr.ICUNormalizer2CharFilterFactory]]
== solr.ICUNormalizer2CharFilterFactory
This filter performs pre-tokenization Unicode normalization using http://site.icu-project.org[ICU4J].
Arguments:
`name`:: A http://unicode.org/reports/tr15/[Unicode Normalization Form], one of `nfc`, `nfkc`, `nfkc_cf`. Default is `nfkc_cf`.
`mode`:: Either `compose` or `decompose`. Default is `compose`. Use `decompose` with `name="nfc"` or `name="nfkc"` to get NFD or NFKD, respectively.
`filter`:: A http://www.icu-project.org/apiref/icu4j/com/ibm/icu/text/UnicodeSet.html[UnicodeSet] pattern. Codepoints outside the set are always left unchanged. Default is `[]` (the null set, no filtering - all codepoints are subject to normalization).
Example:
[source,xml]
----
<analyzer>
<charFilter class="solr.ICUNormalizer2CharFilterFactory"/>
<tokenizer ...>
[...]
</analyzer>
----
[[CharFilterFactories-solr.PatternReplaceCharFilterFactory]]
== solr.PatternReplaceCharFilterFactory
This filter uses http://www.regular-expressions.info/reference.html[regular expressions] to replace or change character patterns.
Arguments:
`pattern`:: the regular expression pattern to apply to the incoming text.
`replacement`:: the text to use to replace matching patterns.
You can configure this filter in `schema.xml` like this:
[source,xml]
----
<analyzer>
<charFilter class="solr.PatternReplaceCharFilterFactory"
pattern="([nN][oO]\.)\s*(\d+)" replacement="$1$2"/>
<tokenizer ...>
[...]
</analyzer>
----
The table below presents examples of regex-based pattern replacement:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,20,10,20,30",options="header"]
|===
|Input |Pattern |Replacement |Output |Description
|see-ing looking |`(\w+)(ing)` |`$1` |see-ing look |Removes "ing" from the end of word.
|see-ing looking |`(\w+)ing` |`$1` |see-ing look |Same as above. 2nd parentheses can be omitted.
|No.1 NO. no. 543 |`[nN][oO]\.\s*(\d+)` |`#$1` |#1 NO. #543 |Replace some string literals
|abc=1234=5678 |`(\w+)=(\d+)=(\d+)` |`$3=$1=$2` |5678=abc=1234 |Change the order of the groups.
|===

View File

@ -0,0 +1,9 @@
= Choosing an Output Format
:page-shortname: choosing-an-output-format
:page-permalink: choosing-an-output-format.html
Many programming environments are able to send HTTP requests and retrieve responses. Parsing the responses is a slightly more thorny problem. Fortunately, Solr makes it easy to choose an output format that will be easy to handle on the client side.
Specify a response format using the `wt` parameter in a query. The available response formats are documented in <<response-writers.adoc#response-writers,Response Writers>>.
Most client APIs hide this detail for you, so for many types of client applications, you won't ever have to specify a `wt` parameter. In JavaScript, however, the interface to Solr is a little closer to the metal, so you will need to add this parameter yourself.

View File

@ -0,0 +1,29 @@
= Client API Lineup
:page-shortname: client-api-lineup
:page-permalink: client-api-lineup.html
The Solr Wiki contains a list of client APIs at http://wiki.apache.org/solr/IntegratingSolr.
Here is the list of client APIs, current at this writing (November 2011):
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,20,60",options="header"]
|===
|Name |Environment |URL
|SolRuby |Ruby |https://github.com/rsolr/rsolr
|DelSolr |Ruby |https://github.com/avvo/delsolr
|acts_as_solr |Rails |http://acts-as-solr.rubyforge.org/, http://rubyforge.org/projects/background-solr/
|Flare |Rails |http://wiki.apache.org/solr/Flare
|SolPHP |PHP |http://wiki.apache.org/solr/SolPHP
|SolrJ |Java |http://wiki.apache.org/solr/SolJava
|Python API |Python |http://wiki.apache.org/solr/SolPython
|PySolr |Python |http://code.google.com/p/pysolr/
|SolPerl |Perl |http://wiki.apache.org/solr/SolPerl
|Solr.pm |Perl |http://search.cpan.org/~garafola/Solr-0.03/lib/Solr.pm
|SolrForrest |Forrest/Cocoon |http://wiki.apache.org/solr/SolrForrest
|SolrSharp |C# |http://www.codeplex.com/solrsharp
|SolColdfusion |ColdFusion |http://solcoldfusion.riaforge.org/
|SolrNet |.NET |https://github.com/mausch/SolrNet
|AJAX Solr |AJAX |http://github.com/evolvingweb/ajax-solr/wiki
|===

View File

@ -0,0 +1,22 @@
= Client APIs
:page-shortname: client-apis
:page-permalink: client-apis.html
:page-children: introduction-to-client-apis, choosing-an-output-format, client-api-lineup, using-javascript, using-python, using-solrj, using-solr-from-ruby
This section discusses the available client APIs for Solr. It covers the following topics:
<<introduction-to-client-apis.adoc#introduction-to-client-apis,Introduction to Client APIs>>: A conceptual overview of Solr client APIs.
<<choosing-an-output-format.adoc#choosing-an-output-format,Choosing an Output Format>>: Information about choosing a response format in Solr.
<<using-javascript.adoc#using-javascript,Using JavaScript>>: Explains why a client API is not needed for JavaScript responses.
<<using-python.adoc#using-python,Using Python>>: Information about Python and JSON responses.
<<client-api-lineup.adoc#client-api-lineup,Client API Lineup>>: A list of all Solr Client APIs, with links.
<<using-solrj.adoc#using-solrj,Using SolrJ>>: Detailed information about SolrJ, an API for working with Java applications.
<<using-solr-from-ruby.adoc#using-solr-from-ruby,Using Solr From Ruby>>: Detailed information about using Solr with Ruby applications.
<<mbean-request-handler.adoc#mbean-request-handler,MBean Request Handler>>: Describes the MBean request handler for programmatic access to Solr server statistics and information.

View File

@ -0,0 +1,29 @@
= Cloud Screens
:page-shortname: cloud-screens
:page-permalink: cloud-screens.html
When running in <<solrcloud.adoc#solrcloud,SolrCloud>> mode, a "Cloud" option will appear in the Admin UI between <<logging.adoc#logging,Logging>> and <<collections-core-admin.adoc#collections-core-admin,Collections/Core Admin>>.
This screen provides status information about each collection & node in your cluster, as well as access to the low level data being stored in <<using-zookeeper-to-manage-configuration-files.adoc#using-zookeeper-to-manage-configuration-files,Zookeeper>>.
.Only Visible When using SolrCloud
[NOTE]
====
The "Cloud" menu option is only available on Solr instances running in <<getting-started-with-solrcloud.adoc#getting-started-with-solrcloud,SolrCloud mode>>. Single node or master/slave replication instances of Solr will not display this option.
====
Click on the Cloud option in the left-hand navigation, and a small sub-menu appears with options called "Tree", "Graph", "Graph (Radial)" and "Dump". The default view ("Graph") shows a graph of each collection, the shards that make up those collections, and the addresses of each replica for each shard.
This example shows the very simple two-node cluster created using the `bin/solr -e cloud -noprompt` example command. In addition to the 2 shard, 2 replica "gettingstarted" collection, there is an additional "films" collection consisting of a single shard/replica:
image::images/cloud-screens/cloud-graph.png[image,width=512,height=250]
The "Graph (Radial)" option provides a different visual view of each node. Using the same example cluster, the radial graph view looks like:
image::images/cloud-screens/cloud-radial.png[image,width=478,height=250]
The "Tree" option shows a directory structure of the data in ZooKeeper, including cluster wide information regarding the `live_nodes` and `overseer` status, as well as collection specific information such as the `state.json`, current shard leaders, and configuration files in use. In this example, we see the `state.json` file definition for the "films" collection:
image::images/cloud-screens/cloud-tree.png[image,width=487,height=250]
The final option is "Dump", which returns a JSON document containing all nodes, their contents and their children (recursively). This can be used to export a snapshot of all the data that Solr has kept inside ZooKeeper and can aid in debugging SolrCloud problems.

View File

@ -0,0 +1,21 @@
= Codec Factory
:page-shortname: codec-factory
:page-permalink: codec-factory.html
A `codecFactory` can be specified in `solrconfig.xml` to determine which Lucene {lucene-javadocs}/core/org/apache/lucene/codecs/Codec.html[`Codec`] is used when writing the index to disk.
If not specified, Lucene's default codec is implicitly used, but a {solr-javadocs}/solr-core/org/apache/solr/core/SchemaCodecFactory.html[`solr.SchemaCodecFactory`] is also available which supports 2 key features:
* Schema based per-fieldtype configuration for `docValuesFormat` and `postingsFormat` - see the <<field-type-definitions-and-properties.adoc#field-type-properties,Field Type Properties>> section for more details.
* A `compressionMode` option:
** `BEST_SPEED` (default) is optimized for search speed performance
** `BEST_COMPRESSION` is optimized for disk space usage
Example:
[source,xml]
----
<codecFactory class="solr.SchemaCodecFactory">
<str name="compressionMode">BEST_COMPRESSION</str>
</codecFactory>
----

View File

@ -0,0 +1,133 @@
= Collapse and Expand Results
:page-shortname: collapse-and-expand-results
:page-permalink: collapse-and-expand-results.html
The Collapsing query parser and the Expand component combine to form an approach to grouping documents for field collapsing in search results.
The Collapsing query parser groups documents (collapsing the result set) according to your parameters, while the Expand component provides access to documents in the collapsed group for use in results display or other processing by a client application. Collapse & Expand can together do what the older <<result-grouping.adoc#result-grouping,Result Grouping>> (`group=true`) does for _most_ use-cases but not all. Generally, you should prefer Collapse & Expand.
[IMPORTANT]
====
In order to use these features with SolrCloud, the documents must be located on the same shard. To ensure document co-location, you can define the `router.name` parameter as `compositeId` when creating the collection. For more information on this option, see the section <<shards-and-indexing-data-in-solrcloud.adoc#ShardsandIndexingDatainSolrCloud-DocumentRouting,Document Routing>>.
====
[[CollapseandExpandResults-CollapsingQueryParser]]
== Collapsing Query Parser
The `CollapsingQParser` is really a _post filter_ that provides more performant field collapsing than Solr's standard approach when the number of distinct groups in the result set is high. This parser collapses the result set to a single document per group before it forwards the result set to the rest of the search components. So all downstream components (faceting, highlighting, etc...) will work with the collapsed result set.
The CollapsingQParser accepts the following local parameters:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,60,20",options="header"]
|===
|Parameter |Description |Default
|field |The field that is being collapsed on. The field must be a single valued String, Int or Float |none
|min \| max a|
Selects the group head document for each group based on which document has the min or max value of the specified numeric field or <<function-queries.adoc#function-queries,function query>>.
At most only one of the min, max, or sort (see below) parameters may be specified.
If none are specified, the group head document of each group will be selected based on the highest scoring document in that group. |none
|sort a|
Selects the group head document for each group based on which document comes first according to the specified <<common-query-parameters.adoc#CommonQueryParameters-ThesortParameter,sort string>>.
At most only one of the min, max, (see above) or sort parameters may be specified.
If none are specified, the group head document of each group will be selected based on the highest scoring document in that group. |none
|nullPolicy a|
There are three null policies:
* *ignore*: removes documents with a null value in the collapse field. This is the default.
* *expand*: treats each document with a null value in the collapse field as a separate group.
* *collapse*: collapses all documents with a null value into a single group using either highest score, or minimum/maximum.
|ignore
|hint |Currently there is only one hint available: `top_fc`, which stands for top level FieldCache. The `top_fc` hint is only available when collapsing on String fields. `top_fc` usually provides the best query time speed but takes the longest to warm on startup or following a commit. `top_fc` will also result in having the collapsed field cached in memory twice if it's used for faceting or sorting. For very high cardinality (high distinct count) fields, `top_fc` may not fare so well. |none
|size |Sets the initial size of the collapse data structures when collapsing on a *numeric field only*. The data structures used for collapsing grow dynamically when collapsing on numeric fields. Setting the size above the number of results expected in the result set will eliminate the resizing cost. |100,000
|===
*Sample Syntax:*
Collapse on `group_field` selecting the document in each group with the highest scoring document:
[source,text]
----
fq={!collapse field=group_field}
----
Collapse on `group_field` selecting the document in each group with the minimum value of `numeric_field`:
[source,text]
----
fq={!collapse field=group_field min=numeric_field}
----
Collapse on `group_field` selecting the document in each group with the maximum value of `numeric_field`:
[source,text]
----
fq={!collapse field=group_field max=numeric_field}
----
Collapse on `group_field` selecting the document in each group with the maximum value of a function. Note that the *cscore()* function can be used with the min/max options to use the score of the current document being collapsed.
[source,text]
----
fq={!collapse field=group_field max=sum(cscore(),numeric_field)}
----
Collapse on `group_field` with a null policy so that all docs that do not have a value in the `group_field` will be treated as a single group. For each group, the selected document will be based first on a `numeric_field`, but ties will be broken by score:
[source,text]
----
fq={!collapse field=group_field nullPolicy=collapse sort='numeric_field asc, score desc'}
----
Collapse on `group_field` with a hint to use the top level field cache:
[source,text]
----
fq={!collapse field=group_field hint=top_fc}
----
The CollapsingQParserPlugin fully supports the QueryElevationComponent.
[[CollapseandExpandResults-ExpandComponent]]
== Expand Component
The ExpandComponent can be used to expand the groups that were collapsed by the http://heliosearch.org/the-collapsingqparserplugin-solrs-new-high-performance-field-collapsing-postfilter/[CollapsingQParserPlugin].
Example usage with the CollapsingQParserPlugin:
[source,text]
----
q=foo&fq={!collapse field=ISBN}
----
In the query above, the CollapsingQParserPlugin will collapse the search results on the _ISBN_ field. The main search results will contain the highest ranking document from each book.
The ExpandComponent can now be used to expand the results so you can see the documents grouped by ISBN. For example:
[source,text]
----
q=foo&fq={!collapse field=ISBN}&expand=true
----
The “expand=true” parameter turns on the ExpandComponent. The ExpandComponent adds a new section to the search output labeled “expanded”.
Inside the expanded section there is a _map_ with each group head pointing to the expanded documents that are within the group. As applications iterate the main collapsed result set, they can access the _expanded_ map to retrieve the expanded groups.
The ExpandComponent has the following parameters:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,60,20",options="header"]
|===
|Parameter |Description |Default
|expand.sort |Orders the documents within the expanded groups |score desc
|expand.rows |The number of rows to display in each group |5
|expand.q |Overrides the main q parameter, determines which documents to include in the main group. |main q
|expand.fq |Overrides main fq's, determines which documents to include in the main group. |main fq's
|===

View File

@ -0,0 +1,30 @@
= Collection-Specific Tools
:page-shortname: collection-specific-tools
:page-permalink: collection-specific-tools.html
:page-children: analysis-screen, dataimport-screen, documents-screen, files-screen, query-screen, stream-screen, schema-browser-screen
In the left-hand navigation bar, you will see a pull-down menu titled "Collection Selector" that can be used to access collection specific administration screens.
.Only Visible When Using SolrCloud
[NOTE]
====
The "Collection Selector" pull-down menu is only available on Solr instances running in <<solrcloud.adoc#solrcloud,SolrCloud mode>>.
Single node or master/slave replication instances of Solr will not display this menu, instead the Collection specific UI pages described in this section will be available in the <<core-specific-tools.adoc#core-specific-tools,Core Selector pull-down menu>>.
====
Clicking on the Collection Selector pull-down menu will show a list of the collections in your Solr cluster, with a search box that can be used to find a specific collection by name. When you select a collection from the pull-down, the main display of the page will display some basic metadata about the collection, and a secondary menu will appear in the left nav with links to additional collection specific administration screens.
image::images/collection-specific-tools/collection_dashboard.png[image,width=482,height=250]
The collection-specific UI screens are listed below, with a link to the section of this guide to find out more:
// TODO: SOLR-10655 BEGIN: refactor this into a 'collection-screens-list.include.adoc' file for reuse
* <<analysis-screen.adoc#analysis-screen,Analysis>> - lets you analyze the data found in specific fields.
* <<dataimport-screen.adoc#dataimport-screen,Dataimport>> - shows you information about the current status of the Data Import Handler.
* <<documents-screen.adoc#documents-screen,Documents>> - provides a simple form allowing you to execute various Solr indexing commands directly from the browser.
* <<files-screen.adoc#files-screen,Files>> - shows the current core configuration files such as `solrconfig.xml`.
* <<query-screen.adoc#query-screen,Query>> - lets you submit a structured query about various elements of a core.
* <<stream-screen.adoc#stream-screen,Stream>> - allows you to submit streaming expressions and see results and parsing explanations.
* <<schema-browser-screen.adoc#schema-browser-screen,Schema Browser>> - displays schema data in a browser window.
// TODO: SOLR-10655 END

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,25 @@
= Collections / Core Admin
:page-shortname: collections-core-admin
:page-permalink: collections-core-admin.html
The Collections screen provides some basic functionality for managing your Collections, powered by the <<collections-api.adoc#collections-api,Collections API>>.
[NOTE]
====
If you are running a single node Solr instance, you will not see a Collections option in the left nav menu of the Admin UI.
You will instead see a "Core Admin" screen that supports some comparable Core level information & manipulation via the <<coreadmin-api.adoc#coreadmin-api,CoreAdmin API>> instead.
====
The main display of this page provides a list of collections that exist in your cluster. Clicking on a collection name provides some basic metadata about how the collection is defined, and its current shards & replicas, with options for adding and deleting individual replicas.
The buttons at the top of the screen let you make various collection level changes to your cluster, from add new collections or aliases to reloading or deleting a single collection.
image::images/collections-core-admin/collection-admin.png[image,width=653,height=250]
Replicas can be deleted by clicking the red "X" next to the replica name.
If the shard is inactive, for example after a <<collections-api.adoc#CollectionsAPI-splitshard,SPLITSHARD action>>, an option to delete the shard will appear as a red "X" next to the shard name.
image::images/collections-core-admin/DeleteShard.png[image,width=486,height=250]

View File

@ -0,0 +1,19 @@
= Combining Distribution and Replication
:page-shortname: combining-distribution-and-replication
:page-permalink: combining-distribution-and-replication.html
When your index is too large for a single machine and you have a query volume that single shards cannot keep up with, it's time to replicate each shard in your distributed search setup.
The idea is to combine distributed search with replication. As shown in the figure below, a combined distributed-replication configuration features a master server for each shard and then 1-_n_ slaves that are replicated from the master. As in a standard replicated configuration, the master server handles updates and optimizations without adversely affecting query handling performance.
Query requests should be load balanced across each of the shard slaves. This gives you both increased query handling capacity and fail-over backup if a server goes down.
.A Solr configuration combining both replication and master-slave distribution.
image::images/combining-distribution-and-replication/worddav4101c16174820e932b44baa22abcfcd1.png[image,width=312,height=344]
None of the master shards in this configuration know about each other. You index to each master, the index is replicated to each slave, and then searches are distributed across the slaves, using one slave from each master/slave shard.
For high availability you can use a load balancer to set up a virtual IP for each shard's set of slaves. If you are new to load balancing, HAProxy (http://haproxy.1wt.eu/) is a good open source software load-balancer. If a slave server goes down, a good load-balancer will detect the failure using some technique (generally a heartbeat system), and forward all requests to the remaining live slaves that served with the failed slave. A single virtual IP should then be set up so that requests can hit a single IP, and get load balanced to each of the virtual IPs for the search slaves.
With this configuration you will have a fully load balanced, search-side fault-tolerant system (Solr does not yet support fault-tolerant indexing). Incoming searches will be handed off to one of the functioning slaves, then the slave will distribute the search request across a slave for each of the shards in your configuration. The slave will issue a request to each of the virtual IPs for each shard, and the load balancer will choose one of the available slaves. Finally, the results will be combined into a single results set and returned. If any of the slaves go down, they will be taken out of rotation and the remaining slaves will be used. If a shard master goes down, searches can still be served from the slaves until you have corrected the problem and put the master back into production.

View File

@ -0,0 +1,120 @@
= Command Line Utilities
:page-shortname: command-line-utilities
:page-permalink: command-line-utilities.html
Solr's Administration page (found by default at `\http://hostname:8983/solr/`), provides a section with menu items for monitoring indexing and performance statistics.
This pag also includes information about index distribution and replication, and information on all threads running in the JVM at the time.
There is also a section where you can run queries, and an assistance area.
In addition, SolrCloud provides its own administration page (found at http://localhost:8983/solr/#/~cloud), as well as a few tools available via a ZooKeeper Command Line Utility (CLI). The CLI scripts found in `server/scripts/cloud-scripts` let you upload configuration information to ZooKeeper, in the same two ways that were shown in the examples in <<parameter-reference.adoc#parameter-reference,Parameter Reference>>. It also provides a few other commands that let you link collection sets to collections, make ZooKeeper paths or clear them, and download configurations from ZooKeeper to the local filesystem.
.Solr's zkcli.sh vs ZooKeeper's zkCli.sh vs Solr Start Script
[IMPORTANT]
====
The `zkcli.sh` provided by Solr is not the same as the https://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_ConnectingToZooKeeper[`zkCli.sh` included in ZooKeeper distributions].
ZooKeeper's `zkCli.sh` provides a completely general, application-agnostic shell for manipulating data in ZooKeeper. Solr's `zkcli.sh` discussed in this section is specific to Solr, and has command line arguments specific to dealing with Solr data in ZooKeeper.
Many of the functions provided by the zkCli.sh script are also provided by the <<solr-control-script-reference.adoc#solr-control-script-reference,Solr Control Script Reference>>, which may be more familiar as the start script ZooKeeper maintenance commands are very similar to Unix commands.
====
[[CommandLineUtilities-UsingSolr_sZooKeeperCLI]]
== Using Solr's ZooKeeper CLI
Both `zkcli.sh` (for Unix environments) and `zkcli.bat` (for Windows environments) support the following command line options:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="10,30,60",options="header"]
|===
|Short |Parameter Usage |Meaning
| |`-cmd <arg>` |CLI Command to be executed: `bootstrap`, `upconfig`, `downconfig`, `linkconfig`, `makepath`, `get`, `getfile`, `put`, `putfile`, `list`, `clear` or `clusterprop`. This parameter is **mandatory**.
|`-z` |`-zkhost <locations>` |ZooKeeper host address. This parameter is *mandatory* for all CLI commands.
|`-c` |`-collection <name>` |For `linkconfig`: name of the collection.
|`-d` |`-confdir <path>` |For `upconfig`: a directory of configuration files. For downconfig: the destination of files pulled from ZooKeeper
|`-h` |`-help` |Display help text.
|`-n` |`-confname <arg>` |For `upconfig`, `linkconfig`, `downconfig`: name of the configuration set.
|`-r` |`-runzk <port>` |Run ZooKeeper internally by passing the Solr run port; only for clusters on one machine.
|`-s` |`-solrhome <path>` |For `bootstrap` or when using `-runzk`: the *mandatory* solrhome location.
| |`-name <name>` |For `clusterprop`: the *mandatory* cluster property name.
| |`-val <value>` |For `clusterprop`: the cluster property value. If not specified, *null* will be used as value.
|===
The short form parameter options may be specified with a single dash (eg: `-c mycollection`).
The long form parameter options may be specified using either a single dash (eg: `-collection mycollection`) or a double dash (eg: `--collection mycollection`)
[[CommandLineUtilities-ZooKeeperCLIExamples]]
== ZooKeeper CLI Examples
Below are some examples of using the `zkcli.sh` CLI which assume you have already started the SolrCloud example (`bin/solr -e cloud -noprompt`)
If you are on Windows machine, simply replace `zkcli.sh` with `zkcli.bat` in these examples.
[[CommandLineUtilities-Uploadaconfigurationdirectory]]
=== Upload a configuration directory
[source,bash]
----
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd upconfig -confname my_new_config -confdir server/solr/configsets/basic_configs/conf
----
[[CommandLineUtilities-BootstrapZooKeeperfromexistingSOLR_HOME]]
=== Bootstrap ZooKeeper from existing SOLR_HOME
[source,bash]
----
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd bootstrap -solrhome /var/solr/data
----
.Bootstrap with chroot
[NOTE]
====
Using the boostrap command with a zookeeper chroot in the -zkhost parameter, e.g. `-zkhost 127.0.0.1:2181/solr`, will automatically create the chroot path before uploading the configs.
====
[[CommandLineUtilities-PutarbitrarydataintoanewZooKeeperfile]]
=== Put arbitrary data into a new ZooKeeper file
[source,bash]
----
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd put /my_zk_file.txt 'some data'
----
[[CommandLineUtilities-PutalocalfileintoanewZooKeeperfile]]
=== Put a local file into a new ZooKeeper file
[source,bash]
----
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd putfile /my_zk_file.txt /tmp/my_local_file.txt
----
[[CommandLineUtilities-Linkacollectiontoaconfigurationset]]
=== Link a collection to a configuration set
[source,bash]
----
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd linkconfig -collection gettingstarted -confname my_new_config
----
[[CommandLineUtilities-CreateanewZooKeeperpath]]
=== Create a new ZooKeeper path
[source,bash]
----
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd makepath /solr
----
This can be useful to create a chroot path in ZooKeeper before first cluster start.
[[CommandLineUtilities-Setaclusterproperty]]
=== Set a cluster property
This command will add or modify a single cluster property in `clusterprops.json`. Use this command instead of the usual getfile -> edit -> putfile cycle. Unlike the CLUSTERPROP REST API, this command does *not* require a running Solr cluster.
[source,bash]
----
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd clusterprop -name urlScheme -val https
----

View File

@ -0,0 +1,365 @@
= Common Query Parameters
:page-shortname: common-query-parameters
:page-permalink: common-query-parameters.html
Several query parsers share supported query parameters.
The table below summarizes Solr's common query parameters, which are supported by the <<requesthandlers-and-searchcomponents-in-solrconfig#RequestHandlersandSearchComponentsinSolrConfig-SearchHandlers,Search RequestHandlers>>
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="30,70",options="header"]
|===
|Parameter |Description
|<<CommonQueryParameters-ThedefTypeParameter,defType>> |Selects the query parser to be used to process the query.
|<<CommonQueryParameters-ThesortParameter,sort>> |Sorts the response to a query in either ascending or descending order based on the response's score or another specified characteristic.
|<<CommonQueryParameters-ThestartParameter,start>> |Specifies an offset (by default, 0) into the responses at which Solr should begin displaying content.
|<<CommonQueryParameters-TherowsParameter,rows>> |Controls how many rows of responses are displayed at a time (default value: 10)
|<<CommonQueryParameters-Thefq_FilterQuery_Parameter,fq>> |Applies a filter query to the search results.
|<<CommonQueryParameters-Thefl_FieldList_Parameter,fl>> |Limits the information included in a query response to a specified list of fields. The fields need to either be `stored="true"` or `docValues="true"`
|<<CommonQueryParameters-ThedebugParameter,debug>> |Request additional debugging information in the response. Specifying the `debug=timing` parameter returns just the timing information; specifying the `debug=results` parameter returns "explain" information for each of the documents returned; specifying the `debug=query parameter` returns all of the debug information.
|<<CommonQueryParameters-TheexplainOtherParameter,explainOther>> |Allows clients to specify a Lucene query to identify a set of documents. If non-blank, the explain info of each document which matches this query, relative to the main query (specified by the q parameter) will be returned along with the rest of the debugging information.
|<<CommonQueryParameters-ThetimeAllowedParameter,timeAllowed>> |Defines the time allowed for the query to be processed. If the time elapses before the query response is complete, partial information may be returned.
|<<CommonQueryParameters-ThesegmentTerminateEarlyParameter,segmentTerminateEarly>> |Indicates that, if possible, Solr should stop collecting documents from each individual (sorted) segment once it can determine that any subsequent documents in that segment will not be candidates for the `rows` being returned. The default is false.
|<<CommonQueryParameters-TheomitHeaderParameter,omitHeader>> |Excludes the header from the returned results, if set to true. The header contains information about the request, such as the time the request took to complete. The default is false.
|<<CommonQueryParameters-ThewtParameter,wt>> |Specifies the Response Writer to be used to format the query response.
|<<CommonQueryParameters-ThelogParamsListParameter,logParamsList>> |By default, Solr logs all parameters. Set this parameter to restrict which parameters are logged. Valid entries are the parameters to be logged, separated by commas (i.e., `logParamsList=param1,param2`). An empty list will log no parameters, so if logging all parameters is desired, do not define this additional parameter at all.
|<<CommonQueryParameters-TheechoParamsParameter,echoParams>> |The response header can include parameters sent with the query request. This parameter controls what is contained in that section of the response header. Valid values are `none`, `all`, and `explicit`. The default value is `explicit.`
|===
The following sections describe these parameters in detail.
[[CommonQueryParameters-ThedefTypeParameter]]
== The `defType` Parameter
The defType parameter selects the query parser that Solr should use to process the main query parameter (`q`) in the request. For example:
`defType=dismax`
If no defType param is specified, then by default, the <<the-standard-query-parser.adoc#the-standard-query-parser,The Standard Query Parser>> is used. (eg: `defType=lucene`)
[[CommonQueryParameters-ThesortParameter]]
== The `sort` Parameter
The `sort` parameter arranges search results in either ascending (`asc`) or descending (`desc`) order. The parameter can be used with either numerical or alphabetical content. The directions can be entered in either all lowercase or all uppercase letters (i.e., both `asc` or `ASC`).
Solr can sort query responses according to document scores or the value of any field with a single value that is either indexed or uses <<docvalues.adoc#docvalues,DocValues>> (that is, any field whose attributes in the Schema include `multiValued="false"` and either `docValues="true"` or `indexed="true"` if the field does not have DocValues enabled, the indexed terms are used to build them on the fly at runtime), provided that:
* the field is non-tokenized (that is, the field has no analyzer and its contents have been parsed into tokens, which would make the sorting inconsistent), or
* the field uses an analyzer (such as the KeywordTokenizer) that produces only a single term.
If you want to be able to sort on a field whose contents you want to tokenize to facilitate searching, <<copying-fields.adoc#copying-fields,use a `copyField` directive>> in the the Schema to clone the field. Then search on the field and sort on its clone.
The table explains how Solr responds to various settings of the `sort` parameter.
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="30,70",options="header"]
|===
|Example |Result
| |If the sort parameter is omitted, sorting is performed as though the parameter were set to score `desc`.
|score desc |Sorts in descending order from the highest score to the lowest score.
|price asc |Sorts in ascending order of the price field
|inStock desc, price asc |Sorts by the contents of the `inStock` field in descending order, then within those results sorts in ascending order by the contents of the price field.
|===
Regarding the sort parameter's arguments:
* A sort ordering must include a field name (or `score` as a pseudo field), followed by whitespace (escaped as + or `%20` in URL strings), followed by a sort direction (`asc` or `desc`).
* Multiple sort orderings can be separated by a comma, using this syntax: `sort=<field name>+<direction>,<field name>+<direction>],...`
** When more than one sort criteria is provided, the second entry will only be used if the first entry results in a tie. If there is a third entry, it will only be used if the first AND second entries are tied. This pattern continues with further entries.
[[CommonQueryParameters-ThestartParameter]]
== The `start` Parameter
When specified, the `start` parameter specifies an offset into a query's result set and instructs Solr to begin displaying results from this offset.
The default value is "0". In other words, by default, Solr returns results without an offset, beginning where the results themselves begin.
Setting the `start` parameter to some other number, such as 3, causes Solr to skip over the preceding records and start at the document identified by the offset.
You can use the `start` parameter this way for paging. For example, if the `rows` parameter is set to 10, you could display three successive pages of results by setting start to 0, then re-issuing the same query and setting start to 10, then issuing the query again and setting start to 20.
[[CommonQueryParameters-TherowsParameter]]
== The `rows` Parameter
You can use the rows parameter to paginate results from a query. The parameter specifies the maximum number of documents from the complete result set that Solr should return to the client at one time.
The default value is 10. That is, by default, Solr returns 10 documents at a time in response to a query.
[[CommonQueryParameters-Thefq_FilterQuery_Parameter]]
== The `fq` (Filter Query) Parameter
The `fq` parameter defines a query that can be used to restrict the superset of documents that can be returned, without influencing score. It can be very useful for speeding up complex queries, since the queries specified with `fq` are cached independently of the main query. When a later query uses the same filter, there's a cache hit, and filter results are returned quickly from the cache.
When using the `fq` parameter, keep in mind the following:
* The `fq` parameter can be specified multiple times in a query. Documents will only be included in the result if they are in the intersection of the document sets resulting from each instance of the parameter. In the example below, only documents which have a popularity greater then 10 and have a section of 0 will match.
+
[source,text]
----
fq=popularity:[10 TO *]&fq=section:0
----
* Filter queries can involve complicated Boolean queries. The above example could also be written as a single `fq` with two mandatory clauses like so:
+
[source,text]
----
fq=+popularity:[10 TO *] +section:0
----
* The document sets from each filter query are cached independently. Thus, concerning the previous examples: use a single `fq` containing two mandatory clauses if those clauses appear together often, and use two separate `fq` parameters if they are relatively independent. (To learn about tuning cache sizes and making sure a filter cache actually exists, see <<the-well-configured-solr-instance.adoc#the-well-configured-solr-instance,The Well-Configured Solr Instance>>.)
* It is also possible to use <<the-standard-query-parser.adoc#TheStandardQueryParser-DifferencesbetweenLuceneQueryParserandtheSolrStandardQueryParser,filter(condition) syntax>> inside the `fq` to cache clauses individually and - among other things - to achieve union of cached filter queries.
* As with all parameters: special characters in an URL need to be properly escaped and encoded as hex values. Online tools are available to help you with URL-encoding. For example: http://meyerweb.com/eric/tools/dencoder/.
[[CommonQueryParameters-Thefl_FieldList_Parameter]]
== The `fl` (Field List) Parameter
The `fl` parameter limits the information included in a query response to a specified list of fields. The fields need to either be `stored="true"` or `docValues="true"``.`
The field list can be specified as a space-separated or comma-separated list of field names. The string "score" can be used to indicate that the score of each document for the particular query should be returned as a field. The wildcard character `*` selects all the fields in the document which are either `stored="true"` or `docValues="true"` and `useDocValuesAsStored="true"` (which is the default when docValues are enabled). You can also add pseudo-fields, functions and transformers to the field list request.
This table shows some basic examples of how to use `fl`:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="30,70",options="header"]
|===
|Field List |Result
|id name price |Return only the id, name, and price fields.
|id,name,price |Return only the id, name, and price fields.
|id name, price |Return only the id, name, and price fields.
|id score |Return the id field and the score.
|* |Return all the `stored` fields in each document, as well as any `docValues` fields that have `useDocValuesAsStored="true"`. This is the default value of the fl parameter.
|* score |Return all the fields in each document, along with each field's score.
|*,dv_field_name |Return all the `stored` fields in each document, and any `docValues` fields that have `useDocValuesAsStored="true`" and the docValues from dv_field_name even if it has `useDocValuesAsStored="false`"
|===
[[CommonQueryParameters-FunctionValues]]
=== Function Values
<<function-queries.adoc#function-queries,Functions>> can be computed for each document in the result and returned as a pseudo-field:
[source,text]
----
fl=id,title,product(price,popularity)
----
[[CommonQueryParameters-DocumentTransformers]]
=== Document Transformers
<<transforming-result-documents.adoc#transforming-result-documents,Document Transformers>> can be used to modify the information returned about each documents in the results of a query:
[source,text]
----
fl=id,title,[explain]
----
[[CommonQueryParameters-FieldNameAliases]]
=== Field Name Aliases
You can change the key used to in the response for a field, function, or transformer by prefixing it with a `_"displayName_:`". For example:
[source,text]
----
fl=id,sales_price:price,secret_sauce:prod(price,popularity),why_score:[explain style=nl]
----
[source,json]
----
{
"response": {
"numFound": 2,
"start": 0,
"docs": [{
"id": "6H500F0",
"secret_sauce": 2100.0,
"sales_price": 350.0,
"why_score": {
"match": true,
"value": 1.052226,
"description": "weight(features:cache in 2) [DefaultSimilarity], result of:",
"details": [{
"..."
}]}}]}}
----
[[CommonQueryParameters-ThedebugParameter]]
== The `debug` Parameter
The `debug` parameter can be specified multiple times and supports the following arguments:
* `debug=query`: return debug information about the query only.
* `debug=timing`: return debug information about how long the query took to process.
* `debug=results`: return debug information about the score results (also known as "explain").
** By default, score explanations are returned as large string values, using newlines and tab indenting for structure & readability, but an additional `debug.explain.structured=true` parameter may be specified to return this information as nested data structures native to the response format requested by `wt`.
* `debug=all`: return all available debug information about the request request. (alternatively usage: `debug=true`)
For backwards compatibility with older versions of Solr, `debugQuery=true` may instead be specified as an alternative way to indicate `debug=all`
The default behavior is not to include debugging information.
[[CommonQueryParameters-TheexplainOtherParameter]]
== The `explainOther` Parameter
The `explainOther` parameter specifies a Lucene query in order to identify a set of documents. If this parameter is included and is set to a non-blank value, the query will return debugging information, along with the "explain info" of each document that matches the Lucene query, relative to the main query (which is specified by the q parameter). For example:
[source,text]
----
q=supervillians&debugQuery=on&explainOther=id:juggernaut
----
The query above allows you to examine the scoring explain info of the top matching documents, compare it to the explain info for documents matching `id:juggernaut`, and determine why the rankings are not as you expect.
The default value of this parameter is blank, which causes no extra "explain info" to be returned.
[[CommonQueryParameters-ThetimeAllowedParameter]]
== The `timeAllowed` Parameter
This parameter specifies the amount of time, in milliseconds, allowed for a search to complete. If this time expires before the search is complete, any partial results will be returned, but values such as `numFound`, <<faceting.adoc#faceting,facet>> counts, and result <<the-stats-component.adoc#the-stats-component,stats>> may not be accurate for the entire result set.
This value is only checked at the time of:
1. Query Expansion, and
2. Document collection
As this check is periodically performed, the actual time for which a request can be processed before it is aborted would be marginally greater than or equal to the value of `timeAllowed`. If the request consumes more time in other stages, e.g., custom components, etc., this parameter is not expected to abort the request.
[[CommonQueryParameters-ThesegmentTerminateEarlyParameter]]
== The `segmentTerminateEarly` Parameter
This parameter may be set to either true or false.
If set to true, and if <<indexconfig-in-solrconfig.adoc#IndexConfiginSolrConfig-mergePolicyFactory,the mergePolicyFactory>> for this collection is a {solr-javadocs}/solr-core/org/apache/solr/index/SortingMergePolicyFactory.html[`SortingMergePolicyFactory`] which uses a `sort` option which is compatible with <<CommonQueryParameters-ThesortParameter,the sort parameter>> specified for this query, then Solr will attempt to use an {lucene-javadocs}/core/org/apache/lucene/search/EarlyTerminatingSortingCollector.html[`EarlyTerminatingSortingCollector`].
If early termination is used, a `segmentTerminatedEarly` header will be included in the `responseHeader`.
Similar to using <<CommonQueryParameters-ThetimeAllowedParameter,the `timeAllowed `Parameter>>, when early segment termination happens values such as `numFound`, <<faceting.adoc#faceting,Facet>> counts, and result <<the-stats-component.adoc#the-stats-component,Stats>> may not be accurate for the entire result set.
The default value of this parameter is false.
[[CommonQueryParameters-TheomitHeaderParameter]]
== The `omitHeader` Parameter
This parameter may be set to either true or false.
If set to true, this parameter excludes the header from the returned results. The header contains information about the request, such as the time it took to complete. The default value for this parameter is false.
[[CommonQueryParameters-ThewtParameter]]
== The `wt` Parameter
The `wt` parameter selects the Response Writer that Solr should use to format the query's response. For detailed descriptions of Response Writers, see <<response-writers.adoc#response-writers,Response Writers>>.
[[CommonQueryParameters-Thecache_falseParameter]]
== The `cache=false` Parameter
Solr caches the results of all queries and filter queries by default. To disable result caching, set the `cache=false` parameter.
You can also use the `cost` option to control the order in which non-cached filter queries are evaluated. This allows you to order less expensive non-cached filters before expensive non-cached filters.
For very high cost filters, if `cache=false` and `cost>=100` and the query implements the `PostFilter` interface, a Collector will be requested from that query and used to filter documents after they have matched the main query and all other filter queries. There can be multiple post filters; they are also ordered by cost.
For example:
// TODO: fix this, it looks horrible (CT)
[source,text]
----
// normal function range query used as a filter, all matching documents
// generated up front and cached
fq={!frange l=10 u=100}mul(popularity,price)
// function range query run in parallel with the main query like a traditional
// lucene filter
fq={!frange l=10 u=100 cache=false}mul(popularity,price)
// function range query checked after each document that already matches the query
// and all other filters. Good for really expensive function queries.
fq={!frange l=10 u=100 cache=false cost=100}mul(popularity,price)
----
[[CommonQueryParameters-ThelogParamsListParameter]]
== The `logParamsList` Parameter
By default, Solr logs all parameters of requests. Set this parameter to restrict which parameters of a request are logged. This may help control logging to only those parameters considered important to your organization.
For example, you could define this like:
`logParamsList=q,fq`
And only the 'q' and 'fq' parameters will be logged.
If no parameters should be logged, you can send `logParamsList` as empty (i.e., `logParamsList=`).
[TIP]
====
This parameter does not only apply to query requests, but to any kind of request to Solr.
====
[[CommonQueryParameters-TheechoParamsParameter]]
== The `echoParams` Parameter
The `echoParams` parameter controls what information about request parameters is included in the response header.
The table explains how Solr responds to various settings of the `echoParams` parameter:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="30,70",options="header"]
|===
|Value |Meaning
|explicit |This is the default value. Only parameters included in the actual request, plus the `_` parameter (which is a 64-bit numeric timestamp) will be added to the params section of the response header.
|all |Include all request parameters that contributed to the query. This will include everything defined in the request handler definition found in `solrconfig.xml` as well as parameters included with the request, plus the `_` parameter. If a parameter is included in the request handler definition AND the request, it will appear multiple times in the response header.
|none |Entirely removes the "params" section of the response header. No information about the request parameters will be available in the response.
|===
Here is an example of a JSON response where the echoParams parameter was not included, so the default of `explicit` is active. The request URL that created this response included three parameters - `q`, `wt`, and `indent`:
[source,json]
----
{
"responseHeader": {
"status": 0,
"QTime": 0,
"params": {
"q": "solr",
"indent": "true",
"wt": "json",
"_": "1458227751857"
}
},
"response": {
"numFound": 0,
"start": 0,
"docs": []
}
}
----
This is what happens if a similar request is sent that adds `echoParams=all` to the three parameters used in the previous example:
[source,json]
----
{
"responseHeader": {
"status": 0,
"QTime": 0,
"params": {
"q": "solr",
"df": "text",
"preferLocalShards": "false",
"indent": "true",
"echoParams": "all",
"rows": "10",
"wt": "json",
"_": "1458228887287"
}
},
"response": {
"numFound": 0,
"start": 0,
"docs": []
}
}
----

View File

@ -0,0 +1,521 @@
= Config API
:page-shortname: config-api
:page-permalink: config-api.html
The Config API enables manipulating various aspects of your `solrconfig.xml` using REST-like API calls.
This feature is enabled by default and works similarly in both SolrCloud and standalone mode. Many commonly edited properties (such as cache sizes and commit settings) and request handler definitions can be changed with this API.
When using this API, `solrconfig.xml` is not changed. Instead, all edited configuration is stored in a file called `configoverlay.json`. The values in `configoverlay.json` override the values in `solrconfig.xml`.
[[ConfigAPI-APIEntryPoints]]
== API Entry Points
* `/config`: retrieve or modify the config. GET to retrieve and POST for executing commands
* `/config/overlay`: retrieve the details in the `configoverlay.json` alone
* `/config/params` : allows creating parameter sets that can override or take the place of parameters defined in `solrconfig.xml`. See the <<request-parameters-api.adoc#request-parameters-api,Request Parameters API>> section for more details.
[[ConfigAPI-Retrievingtheconfig]]
== Retrieving the config
All configuration items, can be retrieved by sending a GET request to the `/config` endpoint - the results will be the effective configuration resulting from merging settings in `configoverlay.json` with those in `solrconfig.xml`:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config
----
To restrict the returned results to a top level section, e.g. `query`, `requestHandler` or `updateHandler`, append the name of the section to the `/config` endpoint following a slash. E.g. to retrieve configuration for all request handlers:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config/requestHandler
----
To further restrict returned results to a single component within a top level section, use the `componentName` request param, e.g. to return configuration for the `/select` request handler:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config/requestHandler?componentName=/select
----
[[ConfigAPI-Commandstomodifytheconfig]]
== Commands to modify the config
This API uses specific commands to tell Solr what property or type of property to add to `configoverlay.json`. The commands are passed as part of the data sent with the request.
The config commands are categorized into 3 different sections which manipulate various data structures in `solrconfig.xml`. Each of these is described below.
* <<ConfigAPI-CommandsforCommonProperties,Common Properties>>
* <<ConfigAPI-CommandsforCustomHandlersandLocalComponents,Components>>
* <<ConfigAPI-CommandsforUser-DefinedProperties,User-defined properties>>
[[ConfigAPI-CommandsforCommonProperties]]
=== Commands for Common Properties
The common properties are those that are frequently need to be customized in a Solr instance. They are manipulated with two commands:
* `set-property`: Set a well known property. The names of the properties are predefined and fixed. If the property has already been set, this command will overwrite the previous setting.
* `unset-property`: Remove a property set using the `set-property` command.
The properties that are configured with these commands are predefined and listed below. The names of these properties are derived from their XML paths as found in `solrconfig.xml`.
* `updateHandler.autoCommit.maxDocs`
* `updateHandler.autoCommit.maxTime`
* `updateHandler.autoCommit.openSearcher`
* `updateHandler.autoSoftCommit.maxDocs`
* `updateHandler.autoSoftCommit.maxTime`
* `updateHandler.commitWithin.softCommit`
* `updateHandler.indexWriter.closeWaitsForMerges`
* `query.filterCache.class`
* `query.filterCache.size`
* `query.filterCache.initialSize`
* `query.filterCache.autowarmCount`
* `query.filterCache.regenerator`
* `query.queryResultCache.class`
* `query.queryResultCache.size`
* `query.queryResultCache.initialSize`
* `query.queryResultCache.autowarmCount`
* `query.queryResultCache.regenerator`
* `query.documentCache.class`
* `query.documentCache.size`
* `query.documentCache.initialSize`
* `query.documentCache.autowarmCount`
* `query.documentCache.regenerator`
* `query.fieldValueCache.class`
* `query.fieldValueCache.size`
* `query.fieldValueCache.initialSize`
* `query.fieldValueCache.autowarmCount`
* `query.fieldValueCache.regenerator`
* `query.useFilterForSortedQuery`
* `query.queryResultWindowSize`
* `query.queryResultMaxDocCached`
* `query.enableLazyFieldLoading`
* `query.boolToFilterOptimizer`
* `query.maxBooleanClauses`
* `jmx.agentId`
* `jmx.serviceUrl`
* `jmx.rootName`
* `requestDispatcher.handleSelect`
* `requestDispatcher.requestParsers.multipartUploadLimitInKB`
* `requestDispatcher.requestParsers.formdataUploadLimitInKB`
* `requestDispatcher.requestParsers.enableRemoteStreaming`
* `requestDispatcher.requestParsers.addHttpRequestToContext`
[[ConfigAPI-CommandsforCustomHandlersandLocalComponents]]
=== Commands for Custom Handlers and Local Components
Custom request handlers, search components, and other types of localized Solr components (such as custom query parsers, update processors, etc.) can be added, updated and deleted with specific commands for the component being modified.
The syntax is similar in each case: `add-<component-name>`, `update-<component-name>`, and `delete-<component-name>`. The command name is not case sensitive, so `Add-RequestHandler`, `ADD-REQUESTHANDLER` and `add-requesthandler` are all equivalent.
In each case, `add-` commands add the new configuration to `configoverlay.json`, which will override any other settings for the component in `solrconfig.xml`; `update-` commands overwrite an existing setting in `configoverlay.json`; and `delete-` commands remove the setting from `configoverlay.json`.
Settings removed from `configoverlay.json` are not removed from `solrconfig.xml`.
The full list of available commands follows below:
[[ConfigAPI-GeneralPurposeCommands]]
==== General Purpose Commands
These commands are the most commonly used:
* `add-requesthandler`
* `update-requesthandler`
* `delete-requesthandler`
* `add-searchcomponent`
* `update-searchcomponent`
* `delete-searchcomponent`
* `add-initparams`
* `update-initparams`
* `delete-initparams`
* `add-queryresponsewriter`
* `update-queryresponsewriter`
* `delete-queryresponsewriter`
[[ConfigAPI-AdvancedCommands]]
==== Advanced Commands
These commands allow registering more advanced customizations to Solr:
* `add-queryparser`
* `update-queryparser`
* `delete-queryparser`
* `add-valuesourceparser`
* `update-valuesourceparser`
* `delete-valuesourceparser`
* `add-transformer`
* `update-transformer`
* `delete-transformer`
* `add-updateprocessor`
* `update-updateprocessor`
* `delete-updateprocessor`
* `add-queryconverter`
* `update-queryconverter`
* `delete-queryconverter`
* `add-listener`
* `update-listener`
* `delete-listener`
* `add-runtimelib`
* `update-runtimelib`
* `delete-runtimelib`
See the section <<ConfigAPI-CreatingandUpdatingRequestHandlers,Creating and Updating Request Handlers>> below for examples of using these commands.
[[ConfigAPI-Whatabout_updateRequestProcessorChain_]]
==== What about updateRequestProcessorChain?
The Config API does not let you create or edit `updateRequestProcessorChain` elements. However, it is possible to create `updateProcessor` entries and can use them by name to create a chain.
example:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
"add-updateprocessor" : { "name" : "firstFld",
"class": "solr.FirstFieldValueUpdateProcessorFactory",
"fieldName":"test_s"}}'
----
You can use this directly in your request by adding a parameter in the `updateRequestProcessorChain` for the specific update processor called `processor=firstFld`.
[[ConfigAPI-CommandsforUser-DefinedProperties]]
=== Commands for User-Defined Properties
Solr lets users templatize the `solrconfig.xml` using the place holder format `${variable_name:default_val}`. You could set the values using system properties, for example, `-Dvariable_name= my_customvalue`. The same can be achieved during runtime using these commands:
* `set-user-property`: Set a user-defined property. If the property has already been set, this command will overwrite the previous setting.
* `unset-user-property`: Remove a user-defined property.
The structure of the request is similar to the structure of requests using other commands, in the format of `"command":{"variable_name":"property_value"}`. You can add more than one variable at a time if necessary.
For more information about user-defined properties, see the section <<configuring-solrconfig-xml.adoc#Configuringsolrconfig.xml-Userdefinedpropertiesfromcore.properties,User defined properties from core.properties>>.
See also the section <<ConfigAPI-CreatingandUpdatingUser-DefinedProperties,Creating and Updating User-Defined Properties>> below for examples of how to use this type of command.
[[ConfigAPI-HowtoMapsolrconfig.xmlPropertiestoJSON]]
== How to Map `solrconfig.xml` Properties to JSON
By using this API, you will be generating JSON representations of properties defined in `solrconfig.xml`. To understand how properties should be represented with the API, let's take a look at a few examples.
Here is what a request handler looks like in `solrconfig.xml`:
[source,xml]
----
<requestHandler name="/query" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<str name="wt">json</str>
<str name="indent">true</str>
</lst>
</requestHandler>
----
The same request handler defined with the Config API would look like this:
[source,json]
----
{
"add-requesthandler":{
"name":"/query",
"class":"solr.SearchHandler",
"defaults":{
"echoParams":"explicit",
"wt":"json",
"indent":true
}
}
}
----
The QueryElevationComponent searchComponent in `solrconfig.xml` looks like this:
[source,xml]
----
<searchComponent name="elevator" class="solr.QueryElevationComponent" >
<str name="queryFieldType">string</str>
<str name="config-file">elevate.xml</str>
</searchComponent>
----
And the same searchComponent with the Config API:
[source,json]
----
{
"add-searchcomponent":{
"name":"elevator",
"class":"QueryElevationComponent",
"queryFieldType":"string",
"config-file":"elevate.xml"
}
}
----
Removing the searchComponent with the Config API:
[source,json]
----
{
"delete-searchcomponent":"elevator"
}
----
A simple highlighter looks like this in `solrconfig.xml` (example has been truncated for space):
[source,xml]
----
<searchComponent class="solr.HighlightComponent" name="highlight">
<highlighting>
<fragmenter name="gap"
default="true"
class="solr.highlight.GapFragmenter">
<lst name="defaults">
<int name="hl.fragsize">100</int>
</lst>
</fragmenter>
<formatter name="html"
default="true"
class="solr.highlight.HtmlFormatter">
<lst name="defaults">
<str name="hl.simple.pre"><![CDATA[<em>]]></str>
<str name="hl.simple.post"><![CDATA[</em>]]></str>
</lst>
</formatter>
<encoder name="html" class="solr.highlight.HtmlEncoder" />
...
</highlighting>
----
The same highlighter with the Config API:
[source,json]
----
{
"add-searchcomponent": {
"name": "highlight",
"class": "solr.HighlightComponent",
"": {
"gap": {
"default": "true",
"name": "gap",
"class": "solr.highlight.GapFragmenter",
"defaults": {
"hl.fragsize": 100
}
}
},
"html": [{
"default": "true",
"name": "html",
"class": "solr.highlight.HtmlFormatter",
"defaults": {
"hl.simple.pre": "before-",
"hl.simple.post": "-after"
}
}, {
"name": "html",
"class": "solr.highlight.HtmlEncoder"
}]
}
}
----
Set autoCommit properties in `solrconfig.xml`:
[source,xml]
----
<autoCommit>
<maxTime>15000</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
----
Define the same properties with the Config API:
[source,json]
----
{
"set-property": {
"updateHandler.autoCommit.maxTime":15000,
"updateHandler.autoCommit.openSearcher":false
}
}
----
[[ConfigAPI-NameComponentsfortheConfigAPI]]
=== Name Components for the Config API
The Config API always allows changing the configuration of any component by name. However, some configurations such as `listener` or `initParams` do not require a name in `solrconfig.xml`. In order to be able to `update` and `delete` of the same item in `configoverlay.json`, the name attribute becomes mandatory.
[[ConfigAPI-Examples]]
== Examples
[[ConfigAPI-CreatingandUpdatingCommonProperties]]
=== Creating and Updating Common Properties
This change sets the `query.filterCache.autowarmCount` to 1000 items and unsets the `query.filterCache.size`.
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d'{
"set-property" : {"query.filterCache.autowarmCount":1000},
"unset-property" :"query.filterCache.size"}'
----
Using the `/config/overlay` endpoint, you can verify the changes with a request like this:
[source,bash]
----
curl http://localhost:8983/solr/gettingstarted/config/overlay?omitHeader=true
----
And you should get a response like this:
[source,json]
----
{
"overlay":{
"znodeVersion":1,
"props":{"query":{"filterCache":{
"autowarmCount":1000,
"size":25}}}}}
----
[[ConfigAPI-CreatingandUpdatingRequestHandlers]]
=== Creating and Updating Request Handlers
To create a request handler, we can use the `add-requesthandler` command:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
"add-requesthandler" : {
"name": "/mypath",
"class":"solr.DumpRequestHandler",
"defaults":{ "x":"y" ,"a":"b", "wt":"json", "indent":true },
"useParams":"x"
}
}'
----
Make a call to the new request handler to check if it is registered:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/mypath?omitHeader=true
----
And you should see the following as output:
[source,json]
----
{
"params":{
"indent":"true",
"a":"b",
"x":"y",
"wt":"json"},
"context":{
"webapp":"/solr",
"path":"/mypath",
"httpMethod":"GET"}}
----
To update a request handler, you should use the `update-requesthandler` command :
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
"update-requesthandler": {
"name": "/mypath",
"class":"solr.DumpRequestHandler",
"defaults": {"x":"new value for X", "wt":"json", "indent":true},
"useParams":"x"
}
}'
----
As another example, we'll create another request handler, this time adding the 'terms' component as part of the definition:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
"add-requesthandler": {
"name": "/myterms",
"class":"solr.SearchHandler",
"defaults": {"terms":true, "distrib":false},
"components": [ "terms" ]
}
}'
----
[[ConfigAPI-CreatingandUpdatingUser-DefinedProperties]]
=== Creating and Updating User-Defined Properties
This command sets a user property.
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H'Content-type:application/json' -d '{
"set-user-property" : {"variable_name":"some_value"}}'
----
Again, we can use the `/config/overlay` endpoint to verify the changes have been made:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config/overlay?omitHeader=true
----
And we would expect to see output like this:
[source,json]
----
{"overlay":{
"znodeVersion":5,
"userProps":{
"variable_name":"some_value"}}
}
----
To unset the variable, issue a command like this:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H'Content-type:application/json' -d '{"unset-user-property" : "variable_name"}'
----
[[ConfigAPI-HowItWorks]]
== How It Works
Every core watches the ZooKeeper directory for the configset being used with that core. In standalone mode, however, there is no watch (because ZooKeeper is not running). If there are multiple cores in the same node using the same configset, only one ZooKeeper watch is used. For instance, if the configset 'myconf' is used by a core, the node would watch `/configs/myconf`. Every write operation performed through the API would 'touch' the directory (sets an empty byte[] to trigger watches) and all watchers are notified. Every core would check if the Schema file, `solrconfig.xml` or `configoverlay.json` is modified by comparing the `znode` versions and if modified, the core is reloaded.
If `params.json` is modified, the params object is just updated without a core reload (see the section <<request-parameters-api.adoc#request-parameters-api,Request Parameters API>> for more information about `params.json`).
[[ConfigAPI-EmptyCommand]]
=== Empty Command
If an empty command is sent to the `/config` endpoint, the watch is triggered on all cores using this configset. For example:
[source,bash]
----
curl http://localhost:8983/solr/techproducts/config -H'Content-type:application/json' -d '{}'
----
Directly editing any files without 'touching' the directory *will not* make it visible to all nodes.
It is possible for components to watch for the configset 'touch' events by registering a listener using `SolrCore#registerConfListener()` .
[[ConfigAPI-ListeningtoconfigChanges]]
=== Listening to config Changes
Any component can register a listener using:
`SolrCore#addConfListener(Runnable listener)`
to get notified for config changes. This is not very useful if the files modified result in core reloads (i.e., `configoverlay.xml` or Schema). Components can use this to reload the files they are interested in.

View File

@ -0,0 +1,26 @@
= Config Sets
:page-shortname: config-sets
:page-permalink: config-sets.html
On a multicore Solr instance, you may find that you want to share configuration between a number of different cores. You can achieve this using named configsets, which are essentially shared configuration directories stored under a configurable configset base directory.
To create a configset, simply add a new directory under the configset base directory. The configset will be identified by the name of this directory. Then into this copy the config directory you want to share. The structure should look something like this:
[source,bash]
----
/<configSetBaseDir>
/configset1
/conf
/managed-schema
/solrconfig.xml
/configset2
/conf
/managed-schema
/solrconfig.xml
----
The default base directory is `$SOLR_HOME/configsets`, and it can be configured in `solr.xml`.
To create a new core using a configset, pass `configSet` as one of the core properties. For example, if you do this via the core admin API:
`\http://localhost:8983/admin/cores?action=CREATE&name=mycore&instanceDir=path/to/instance&configSet=configset2`

View File

@ -0,0 +1,155 @@
= ConfigSets API
:page-shortname: configsets-api
:page-permalink: configsets-api.html
The ConfigSets API enables you to create, delete, and otherwise manage ConfigSets.
To use a ConfigSet created with this API as the configuration for a collection, use the <<collections-api.adoc#collections-api,Collections API>>.
This API can only be used with Solr running in SolrCloud mode. If you are not running Solr in SolrCloud mode but would still like to use shared configurations, please see the section <<config-sets.adoc#config-sets,Config Sets>>.
[[ConfigSetsAPI-APIEntryPoints]]
== API Entry Points
The base URL for all API calls is `\http://<hostname>:<port>/solr`.
* `/admin/configs?action=CREATE`: <<ConfigSetsAPI-create,create>> a ConfigSet, based on an existing ConfigSet
* `/admin/configs?action=DELETE`: <<ConfigSetsAPI-delete,delete>> a ConfigSet
* `/admin/configs?action=LIST`: <<ConfigSetsAPI-list,list>> all ConfigSets
[[ConfigSetsAPI-createCreateaConfigSet]]
[[ConfigSetsAPI-create]]
== Create a ConfigSet
`/admin/configs?action=CREATE&name=_name_&baseConfigSet=_baseConfigSet_`
Create a ConfigSet, based on an existing ConfigSet.
[[ConfigSetsAPI-Input]]
=== Input
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,15,10,15,40",options="header"]
|===
|Key |Type |Required |Default |Description
|name |String |Yes | |ConfigSet to be created
|baseConfigSet |String |Yes | |ConfigSet to copy as a base
|configSetProp._name=value_ |String |No | |ConfigSet property from base to override
|===
[[ConfigSetsAPI-Output]]
=== Output
*Output Content*
The output will include the status of the request. If the status is anything other than "success", an error message will explain why the request failed.
[[ConfigSetsAPI-Examples]]
=== Examples
*Input*
Create a ConfigSet named 'myConfigSet' based on a 'predefinedTemplate' ConfigSet, overriding the immutable property to false.
[source,text]
----
http://localhost:8983/solr/admin/configs?action=CREATE&name=myConfigSet&baseConfigSet=predefinedTemplate&configSetProp.immutable=false
----
*Output*
[source,xml]
----
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">323</int>
</lst>
</response>
----
[[ConfigSetsAPI-deleteDeleteaConfigSet]]
[[ConfigSetsAPI-delete]]
== Delete a ConfigSet
`/admin/configs?action=DELETE&name=_name_`
Delete a ConfigSet
[[ConfigSetsAPI-Input.1]]
=== Input
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,15,10,15,40",options="header"]
|===
|Key |Type |Required |Default |Description
|name |String |Yes | |ConfigSet to be deleted
|===
[[ConfigSetsAPI-Output.1]]
=== Output
*Output Content*
The output will include the status of the request. If the status is anything other than "success", an error message will explain why the request failed.
[[ConfigSetsAPI-Examples.1]]
=== Examples
*Input*
Delete ConfigSet 'myConfigSet'
[source,text]
----
http://localhost:8983/solr/admin/configs?action=DELETE&name=myConfigSet
----
*Output*
[source,xml]
----
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">170</int>
</lst>
</response>
----
[[ConfigSetsAPI-listListConfigSets]]
[[ConfigSetsAPI-list]]
== List ConfigSets
`/admin/configs?action=LIST`
Fetch the names of the ConfigSets in the cluster.
[[ConfigSetsAPI-Examples.2]]
=== Examples
*Input*
[source,text]
----
http://localhost:8983/solr/admin/configs?action=LIST&wt=json
----
*Output*
[source,json]
----
{
"responseHeader":{
"status":0,
"QTime":203},
"configSets":["myConfigSet1",
"myConfig2"]}
----

View File

@ -0,0 +1,11 @@
= Configuration APIs
:page-shortname: configuration-apis
:page-permalink: configuration-apis.html
:page-children: blob-store-api, config-api, request-parameters-api, managed-resources
Solr includes several APIs that can be used to modify settings in `solrconfig.xml`.
* <<blob-store-api.adoc#blob-store-api,Blob Store API>>
* <<config-api.adoc#config-api,Config API>>
* <<request-parameters-api.adoc#request-parameters-api,Request Parameters API>>
* <<managed-resources.adoc#managed-resources,Managed Resources>>

View File

@ -0,0 +1,110 @@
= Configuring Logging
:page-shortname: configuring-logging
:page-permalink: configuring-logging.html
Solr logs are a key way to know what's happening in the system. There are several ways to adjust the default logging configuration.
[IMPORTANT]
====
In addition to the logging options described below, there is a way to configure which request parameters (such as parameters sent as part of queries) are logged with an additional request parameter called `logParamsList`. See the section on <<common-query-parameters.adoc#CommonQueryParameters-ThelogParamsListParameter,Common Query Parameters>> for more information.
====
[[ConfiguringLogging-TemporaryLoggingSettings]]
== Temporary Logging Settings
You can control the amount of logging output in Solr by using the Admin Web interface. Select the *LOGGING* link. Note that this page only lets you change settings in the running system and is not saved for the next run. (For more information about the Admin Web interface, see <<using-the-solr-administration-user-interface.adoc#using-the-solr-administration-user-interface,Using the Solr Administration User Interface>>.)
.The Logging Screen
image::images/logging/logging.png[image]
This part of the Admin Web interface allows you to set the logging level for many different log categories. Fortunately, any categories that are *unset* will have the logging level of its parent. This makes it possible to change many categories at once by adjusting the logging level of their parent.
When you select **Level**, you see the following menu:
.The Log Level Menu
image::images/logging/level_menu.png[image,width=1159,height=577]
Directories are shown with their current logging levels. The Log Level Menu floats over these. To set a log level for a particular directory, select it and click the appropriate log level button.
Log levels settings are as follows:
[width="100%",options="header",]
|===
|Level |Result
|FINEST |Reports everything.
|FINE |Reports everything but the least important messages.
|CONFIG |Reports configuration errors.
|INFO |Reports everything but normal status.
|WARN |Reports all warnings.
|SEVERE |Reports only the most severe warnings.
|OFF |Turns off logging.
|UNSET |Removes the previous log setting.
|===
Multiple settings at one time are allowed.
[[ConfiguringLogging-LoglevelAPI]]
=== Log level API
There is also a way of sending REST commands to the logging endpoint to do the same. Example:
[source,bash]
----
# Set the root logger to level WARN
curl -s http://localhost:8983/solr/admin/info/logging --data-binary "set=root:WARN&wt=json"
----
[[ConfiguringLogging-ChoosingLogLevelatStartup]]
== Choosing Log Level at Startup
You can temporarily choose a different logging level as you start Solr. There are two ways:
The first way is to set the `SOLR_LOG_LEVEL` environment variable before you start Solr, or place the same variable in `bin/solr.in.sh` or `bin/solr.in.cmd`. The variable must contain an uppercase string with a supported log level (see above).
The second way is to start Solr with the -v or -q options, see <<solr-control-script-reference.adoc#solr-control-script-reference,Solr Control Script Reference>> for details. Examples:
[source,bash]
----
# Start with verbose (DEBUG) looging
bin/solr start -f -v
# Start with quiet (WARN) logging
bin/solr start -f -q
----
[[ConfiguringLogging-PermanentLoggingSettings]]
== Permanent Logging Settings
Solr uses http://logging.apache.org/log4j/1.2/[Log4J version 1.2] for logging which is configured using `server/resources/log4j.properties`. Take a moment to inspect the contents of the `log4j.properties` file so that you are familiar with its structure. By default, Solr log messages will be written to `SOLR_LOGS_DIR/solr.log`.
When you're ready to deploy Solr in production, set the variable `SOLR_LOGS_DIR` to the location where you want Solr to write log files, such as `/var/solr/logs`. You may also want to tweak `log4j.properties`. Note that if you installed Solr as a service using the instructions provided in <<taking-solr-to-production.adoc#taking-solr-to-production,Taking Solr to Production>>, then see `/var/solr/log4j.properties` instead of the default `server/resources` version.
When starting Solr in the foreground (`-f` option), all logs will be sent to the console, in addition to `solr.log`. When starting Solr in the background, it will write all `stdout` and `stderr` output to a log file in `solr-<port>-console.log`, and automatically disable the CONSOLE logger configured in `log4j.properties`, having the same effect as if you removed the CONSOLE appender from the rootLogger manually.
Also, in `log4j.properties` the default log rotation size threshold of 4MB is most likely too small for production servers and should be increased to a larger value (such as 100MB or more).
[source,text]
----
log4j.appender.file.MaxFileSize=100MB
----
Java Garbage Collection logs are rotated by the JVM when size hits 20M, for a max of 9 generations. Old GC logs are moved to `SOLR_LOGS_DIR/archived`. These settings can only be changed by editing the start scripts.
On every startup of Solr, the start script will clean up old logs and rotate the main `solr.log` file. If you changed the `log4j.appender.file.MaxBackupIndex` setting in `log4j.properties`, you also need to change the corresponding setting `-rotate_solr_logs 9` in the start script.
You can disable the automatic log rotation at startup by changing the setting `SOLR_LOG_PRESTART_ROTATION` found in `bin/solr.in.sh` or `bin/solr.in.cmd` to false.
[[ConfiguringLogging-LoggingSlowQueries]]
== Logging Slow Queries
For high-volume search applications, logging every query can generate a large amount of logs and, depending on the volume, potentially impact performance. If you mine these logs for additional insights into your application, then logging every query request may be useful.
On the other hand, if you're only concerned about warnings and error messages related to requests, then you can set the log verbosity to WARN. However, this poses a potential problem in that you won't know if any queries are slow, as slow queries are still logged at the INFO level.
Solr provides a way to set your log verbosity threshold to WARN and be able to set a latency threshold above which a request is considered "slow" and log that request at the WARN level to help you identify slow queries in your application. To enable this behavior, configure the `<slowQueryThresholdMillis>` element in the *query* section of solrconfig.xml:
[source,xml]
----
<slowQueryThresholdMillis>1000</slowQueryThresholdMillis>
----
Any queries that take longer than the specified threshold will be logged as "slow" queries at the WARN level.

View File

@ -0,0 +1,155 @@
= Configuring solrconfig.xml
:page-shortname: configuring-solrconfig-xml
:page-permalink: configuring-solrconfig-xml.html
:page-children: datadir-and-directoryfactory-in-solrconfig, lib-directives-in-solrconfig, schema-factory-definition-in-solrconfig, indexconfig-in-solrconfig, requesthandlers-and-searchcomponents-in-solrconfig, initparams-in-solrconfig, updatehandlers-in-solrconfig, query-settings-in-solrconfig, requestdispatcher-in-solrconfig, update-request-processors, codec-factory
The `solrconfig.xml` file is the configuration file with the most parameters affecting Solr itself.
While configuring Solr, you'll work with `solrconfig.xml` often, either directly or via the <<config-api.adoc#config-api,Config API>> to create "configuration overlays" (`configoverlay.json`) to override the values in `solrconfig.xml`.
In `solrconfig.xml`, you configure important features such as:
* request handlers, which process the requests to Solr, such as requests to add documents to the index or requests to return results for a query
* listeners, processes that "listen" for particular query-related events; listeners can be used to trigger the execution of special code, such as invoking some common queries to warm-up caches
* the Request Dispatcher for managing HTTP communications
* the Admin Web interface
* parameters related to replication and duplication (these parameters are covered in detail in <<legacy-scaling-and-distribution.adoc#legacy-scaling-and-distribution,Legacy Scaling and Distribution>>)
The `solrconfig.xml` file is located in the `conf/` directory for each collection. Several well-commented example files can be found in the `server/solr/configsets/` directories demonstrating best practices for many different types of installations.
We've covered the options in the following sections:
* <<datadir-and-directoryfactory-in-solrconfig.adoc#datadir-and-directoryfactory-in-solrconfig,DataDir and DirectoryFactory in SolrConfig>>
* <<lib-directives-in-solrconfig.adoc#lib-directives-in-solrconfig,Lib Directives in SolrConfig>>
* <<schema-factory-definition-in-solrconfig.adoc#schema-factory-definition-in-solrconfig,Schema Factory Definition in SolrConfig>>
* <<indexconfig-in-solrconfig.adoc#indexconfig-in-solrconfig,IndexConfig in SolrConfig>>
* <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,RequestHandlers and SearchComponents in SolrConfig>>
* <<initparams-in-solrconfig.adoc#initparams-in-solrconfig,InitParams in SolrConfig>>
* <<updatehandlers-in-solrconfig.adoc#updatehandlers-in-solrconfig,UpdateHandlers in SolrConfig>>
* <<query-settings-in-solrconfig.adoc#query-settings-in-solrconfig,Query Settings in SolrConfig>>
* <<requestdispatcher-in-solrconfig.adoc#requestdispatcher-in-solrconfig,RequestDispatcher in SolrConfig>>
* <<update-request-processors.adoc#update-request-processors,Update Request Processors>>
* <<codec-factory.adoc#codec-factory,Codec Factory>>
[[Configuringsolrconfig.xml-SubstitutingPropertiesinSolrConfigFiles]]
== Substituting Properties in Solr Config Files
Solr supports variable substitution of property values in config files, which allows runtime specification of various configuration options in `solrconfig.xml`. The syntax is `${propertyname[:option default value]`}. This allows defining a default that can be overridden when Solr is launched. If a default value is not specified, then the property _must_ be specified at runtime or the configuration file will generate an error when parsed.
There are multiple methods for specifying properties that can be used in configuration files. Of those below, strongly consider "config overlay" as the preferred approach, as it stays local to the config set and because it's easy to modify.
[[Configuringsolrconfig.xml-JVMSystemProperties]]
=== JVM System Properties
Any JVM System properties, usually specified using the `-D` flag when starting the JVM, can be used as variables in any XML configuration file in Solr.
For example, in the sample `solrconfig.xml` files, you will see this value which defines the locking type to use:
[source,xml]
----
<lockType>${solr.lock.type:native}</lockType>
----
Which means the lock type defaults to "native" but when starting Solr, you could override this using a JVM system property by launching the Solr it with:
[source,bash]
----
bin/solr start -Dsolr.lock.type=none
----
In general, any Java system property that you want to set can be passed through the `bin/solr` script using the standard `-Dproperty=value` syntax. Alternatively, you can add common system properties to the `SOLR_OPTS` environment variable defined in the Solr include file (`bin/solr.in.sh` or `bin/solr.in.cmd`). For more information about how the Solr include file works, refer to: <<taking-solr-to-production.adoc#taking-solr-to-production,Taking Solr to Production>>.
[[Configuringsolrconfig.xml-ConfigAPI]]
=== Config API
The <<config-api.adoc#config-api,Config API>> allows you to use an API to modify Solr's configuration, specifically user defined properties. Changes made with this API are stored in a file named `configoverlay.json`. This file should only be edited with the API, but will look like this example:
[source,json]
----
{"userProps":{
"dih.db.url":"jdbc:oracle:thin:@localhost:1521",
"dih.db.user":"username",
"dih.db.pass":"password"}}
----
For more details, see the section <<config-api.adoc#config-api,Config API>>.
[[Configuringsolrconfig.xml-solrcore.properties]]
=== solrcore.properties
If the configuration directory for a Solr core contains a file named `solrcore.properties` that file can contain any arbitrary user defined property names and values using the Java standard https://en.wikipedia.org/wiki/.properties[properties file format], and those properties can be used as variables in the XML configuration files for that Solr core.
For example, the following `solrcore.properties` file could be created in the `conf/` directory of a collection using one of the example configurations, to override the lockType used.
[source,bash]
----
#conf/solrcore.properties
solr.lock.type=none
----
.Deprecation
[WARNING]
====
`solrcore.properties` won't work in SolrCloud mode (it is not read from ZooKeeper). This feature is likely to be removed in the future. Instead, use another mechanism like a config overlay.
====
[IMPORTANT]
====
The path and name of the `solrcore.properties` file can be overridden using the `properties` property in <<defining-core-properties.adoc#defining-core-properties,`core.properties`>>.
====
[[Configuringsolrconfig.xml-Userdefinedpropertiesfromcore.properties]]
=== User-Defined Properties in `core.properties`
Every Solr core has a `core.properties` file, automatically created when using the APIs. When you create a SolrCloud collection, you can pass through custom parameters to go into each core.properties that will be created, by prefixing the parameter name with "property." as a URL parameter. Example:
http://localhost:8983/solr/admin/collections?action=CREATE&name=gettingstarted&numShards=1&property.my.custom.prop=edismax
That would create a `core.properties` file that has at least the following properties (others omitted for brevity):
[source,bash]
----
#core.properties
name=gettingstarted
my.custom.prop=edismax
----
The `my.custom.prop` property can then be used as a variable, such as in `solrconfig.xml`:
[source,xml]
----
<requestHandler name="/select">
<lst name="defaults">
<str name="defType">${my.custom.prop}</str>
</lst>
</requestHandler>
----
[[Configuringsolrconfig.xml-ImplicitCoreProperties]]
=== Implicit Core Properties
Several attributes of a Solr core are available as "implicit" properties that can be used in variable substitution, independent of where or how they underlying value is initialized. For example: regardless of whether the name for a particular Solr core is explicitly configured in `core.properties` or inferred from the name of the instance directory, the implicit property `solr.core.name` is available for use as a variable in that core's configuration file...
[source,xml]
----
<requestHandler name="/select">
<lst name="defaults">
<str name="collection_name">${solr.core.name}</str>
</lst>
</requestHandler>
----
All implicit properties use the `solr.core.` name prefix, and reflect the runtime value of the equivalent <<defining-core-properties.adoc#defining-core-properties,`core.properties` property>>:
* `solr.core.name`
* `solr.core.config`
* `solr.core.schema`
* `solr.core.dataDir`
* `solr.core.transient`
* `solr.core.loadOnStartup`

View File

@ -0,0 +1,48 @@
= Content Streams
:page-shortname: content-streams
:page-permalink: content-streams.html
Content streams are bulk data passed with a request to Solr.
When Solr RequestHandlers are accessed using path based URLs, the `SolrQueryRequest` object containing the parameters of the request may also contain a list of ContentStreams containing bulk data for the request. (The name SolrQueryRequest is a bit misleading: it is involved in all requests, regardless of whether it is a query request or an update request.)
[[ContentStreams-StreamSources]]
== Stream Sources
Currently request handlers can get content streams in a variety of ways:
* For multipart file uploads, each file is passed as a stream.
* For POST requests where the content-type is not `application/x-www-form-urlencoded`, the raw POST body is passed as a stream. The full POST body is parsed as parameters and included in the Solr parameters.
* The contents of parameter `stream.body` is passed as a stream.
* If remote streaming is enabled and URL content is called for during request handling, the contents of each `stream.url` and `stream.file` parameters are fetched and passed as a stream.
By default, curl sends a `contentType="application/x-www-form-urlencoded"` header. If you need to test a SolrContentHeader content stream, you will need to set the content type with curl's `-H` flag.
[[ContentStreams-RemoteStreaming]]
== RemoteStreaming
Remote streaming lets you send the contents of a URL as a stream to a given SolrRequestHandler. You could use remote streaming to send a remote or local file to an update plugin.
For convenience, remote streaming is enabled in most of the example `solrconfig.xml` files included with Solr, however it is not recommended in a production situation without additional security between you and untrusted remote clients.
[source,xml]
----
<!-- *** WARNING ***
The settings below authorize Solr to fetch remote files, You
should make sure your system has some authentication before
using enableRemoteStreaming="true"
-->
<requestParsers enableRemoteStreaming="true" />
----
The default behavior, when `enableRemoteStreaming` is not specified in `solrconfig.xml` is to _not_ allow remote streaming (i.e., `enableRemoteStreaming="false"`).
[IMPORTANT]
====
If you `enableRemoteStreaming="true"` is used, be aware that this allows _anyone_ to send a request to any URL or local file. If <<ContentStreams-DebuggingRequests,DumpRequestHandler>> is enabled, it will allow anyone to view any file on your system.
====
[[ContentStreams-DebuggingRequests]]
== Debugging Requests
The implicit "dump" RequestHandler (see <<implicit-requesthandlers.adoc#implicit-requesthandlers,Implicit RequestHandlers>>) simply outputs the contents of the SolrQueryRequest using the specified writer type `wt`. This is a useful tool to help understand what streams are available to the RequestHandlers.

View File

@ -0,0 +1,40 @@
= Copying Fields
:page-shortname: copying-fields
:page-permalink: copying-fields.html
You might want to interpret some document fields in more than one way. Solr has a mechanism for making copies of fields so that you can apply several distinct field types to a single piece of incoming information.
The name of the field you want to copy is the _source_, and the name of the copy is the _destination_. In `schema.xml`, it's very simple to make copies of fields:
[source,xml]
----
<copyField source="cat" dest="text" maxChars="30000" />
----
In this example, we want Solr to copy the `cat` field to a field named `text`. Fields are copied before <<understanding-analyzers-tokenizers-and-filters.adoc#understanding-analyzers-tokenizers-and-filters,analysis>> is done, meaning you can have two fields with identical original content, but which use different analysis chains and are stored in the index differently.
In the example above, if the `text` destination field has data of its own in the input documents, the contents of the `cat` field will be added as additional values just as if all of the values had originally been specified by the client. Remember to configure your fields as `multivalued="true"` if they will ultimately get multiple values (either from a multivalued source or from multiple `copyField` directives).
A common usage for this functionality is to create a single "search" field that will serve as the default query field when users or clients do not specify a field to query. For example, `title`, `author`, `keywords`, and `body` may all be fields that should be searched by default, with copy field rules for each field to copy to a `catchall` field (for example, it could be named anything). Later you can set a rule in `solrconfig.xml` to search the `catchall` field by default. One caveat to this is your index will grow when using copy fields. However, whether this becomes problematic for you and the final size will depend on the number of fields being copied, the number of destination fields being copied to, the analysis in use, and the available disk space.
The `maxChars` parameter, an `int` parameter, establishes an upper limit for the number of characters to be copied from the source value when constructing the value added to the destination field. This limit is useful for situations in which you want to copy some data from the source field, but also control the size of index files.
Both the source and the destination of `copyField` can contain either leading or trailing asterisks, which will match anything. For example, the following line will copy the contents of all incoming fields that match the wildcard pattern `*_t` to the text field.:
[source,xml]
----
<copyField source="*_t" dest="text" maxChars="25000" />
----
[IMPORTANT]
====
The `copyField` command can use a wildcard (*) character in the `dest` parameter only if the `source` parameter contains one as well. `copyField` uses the matching glob from the source field for the `dest` field name into which the source content is copied.
====
Copying is done at the stream source level and no copy feeds into another copy. This means that copy fields cannot be chained i.e. _you cannot_ copy from `here` to `there` and then from `there` to `elsewhere`. However, the same source field can be copied to multiple destination fields:
[source,xml]
----
<copyField source="here" dest="there"/>
<copyField source="here" dest="elsewhere"/>
----

View File

@ -0,0 +1,36 @@
= Core-Specific Tools
:page-shortname: core-specific-tools
:page-permalink: core-specific-tools.html
:page-children: ping, plugins-stats-screen, replication-screen, segments-info
The Core-Specific tools are a group of UI screens that allow you to see core-level information.
In the left-hand navigation bar, you will see a pull-down menu titled "Core Selector". Clicking on the menu will show a list of Solr cores hosted on this Solr node, with a search box that can be used to find a specific core by name.
When you select a core from the pull-down, the main display of the page will show some basic metadata about the core, and a secondary menu will appear in the left nav with links to additional core specific administration screens.
You can also define a configuration file called `admin-extra.html` that includes links or other information you would like to display in the "Admin Extra" part of this main screen.
.Core overview screen
image::images/core-specific-tools/core_dashboard.png[image,width=515,height=250]
The core-specific UI screens are listed below, with a link to the section of this guide to find out more:
// TODO: SOLR-10655 BEGIN: refactor this into a 'core-screens-list.include.adoc' file for reuse
* <<ping.adoc#ping,Ping>> - lets you ping a named core and determine whether the core is active.
* <<plugins-stats-screen.adoc#plugins-stats-screen,Plugins/Stats>> - shows statistics for plugins and other installed components.
* <<replication-screen.adoc#replication-screen,Replication>> - shows you the current replication status for the core, and lets you enable/disable replication.
* <<segments-info.adoc#segments-info,Segments Info>> - Provides a visualization of the underlying Lucene index segments.
// TODO: SOLR-10655 END
If you are running a single node instance of Solr, additional UI screens normally displayed on a per-collection bases will also be listed:
// TODO: SOLR-10655 BEGIN: refactor this into a 'collection-screens-list.include.adoc' file for reuse
* <<analysis-screen.adoc#analysis-screen,Analysis>> - lets you analyze the data found in specific fields.
* <<dataimport-screen.adoc#dataimport-screen,Dataimport>> - shows you information about the current status of the Data Import Handler.
* <<documents-screen.adoc#documents-screen,Documents>> - provides a simple form allowing you to execute various Solr indexing commands directly from the browser.
* <<files-screen.adoc#files-screen,Files>> - shows the current core configuration files such as `solrconfig.xml`.
* <<query-screen.adoc#query-screen,Query>> - lets you submit a structured query about various elements of a core.
* <<stream-screen.adoc#stream-screen,Stream>> - allows you to submit streaming expressions and see results and parsing explanations.
* <<schema-browser-screen.adoc#schema-browser-screen,Schema Browser>> - displays schema data in a browser window.
// TODO: SOLR-10655 END

View File

@ -0,0 +1,353 @@
= CoreAdmin API
:page-shortname: coreadmin-api
:page-permalink: coreadmin-api.html
The Core Admin API is primarily used under the covers by the <<collections-api.adoc#collections-api,Collections API>> when running a <<solrcloud.adoc#solrcloud,SolrCloud>> cluster.
SolrCloud users should not typically use the CoreAdmin API directly, but the API may be useful for users of single-node or master/slave Solr installations for core maintenance operations.
The CoreAdmin API is implemented by the CoreAdminHandler, which is a special purpose <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,request handler>> that is used to manage Solr cores. Unlike other request handlers, the CoreAdminHandler is not attached to a single core. Instead, there is a single instance of the CoreAdminHandler in each Solr node that manages all the cores running in that node and is accessible at the `/solr/admin/cores` path.
CoreAdmin actions can be executed by via HTTP requests that specify an `action` request parameter, with additional action specific arguments provided as additional parameters.
All action names are uppercase, and are defined in depth in the sections below.
[[CoreAdminAPI-STATUS]]
== STATUS
The `STATUS` action returns the status of all running Solr cores, or status for only the named core.
`admin/cores?action=STATUS&core=_core-name_`
[[CoreAdminAPI-Input]]
=== *Input*
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|core |string |No | |The name of a core, as listed in the "name" attribute of a `<core>` element in `solr.xml`.
|indexInfo |boolean |No |true |If **false**, information about the index will not be returned with a core STATUS request. In Solr implementations with a large number of cores (i.e., more than hundreds), retrieving the index information for each core can take a lot of time and isn't always required.
|===
[[CoreAdminAPI-CREATE]]
== CREATE
The `CREATE` action creates a new core and registers it.
If a Solr core with the given name already exists, it will continue to handle requests while the new core is initializing. When the new core is ready, it will take new requests and the old core will be unloaded.
`admin/cores?action=CREATE&name=_core-name_&instanceDir=_path/to/dir_&config=solrconfig.xml&dataDir=data`
Note that this command is the only one of the Core Admin API commands that *does not* support the `core` parameter. Instead, the `name` parameter is required, as shown below.
.CREATE must be able to find a configuration!
[WARNING]
====
Your CREATE call must be able to find a configuration, or it will not succeed.
When you are running SolrCloud and create a new core for a collection, the configuration will be inherited from the collection. Each collection is linked to a configName, which is stored in the ZooKeeper database. This satisfies the config requirement. There is something to note, though if you're running SolrCloud, you should *NOT* be using the CoreAdmin API at all. Use the Collections API.
When you are not running SolrCloud, if you have <<config-sets.adoc#config-sets,Config Sets>> defined, you can use the configSet parameter as documented below. If there are no config sets, then the instanceDir specified in the CREATE call must already exist, and it must contain a `conf` directory which in turn must contain `solrconfig.xml`, your schema, which is usually named either `managed-schema` or `schema.xml`, and any files referenced by those configs.
The config and schema filenames can be specified with the config and schema parameters, but these are expert options. One thing you could do to avoid creating the conf directory is use config and schema parameters that point at absolute paths, but this can lead to confusing configurations unless you fully understand what you are doing.
====
.CREATE and the core.properties file
[IMPORTANT]
====
The core.properties file is built as part of the CREATE command. If you create a core.properties file yourself in a core directory and then try to use CREATE to add that core to Solr, you will get an error telling you that another core is already defined there. The core.properties file must NOT exist before calling the CoreAdmin API with the CREATE command.
====
[[CoreAdminAPI-Input.1]]
=== *Input*
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|name |string |Yes |N/A |The name of the new core. Same as "name" on the `<core>` element.
|instanceDir |string |No |The value specified for "name" parameter |The directory where files for this SolrCore should be stored. Same as `instanceDir` on the `<core>` element.
|config |string |No | |Name of the config file (i.e., `solrconfig.xml`) relative to `instanceDir`.
|schema |string |No | |Name of the schema file to use for the core. Please note that if you are using a "managed schema" (the default behavior) then any value for this property which does not match the effective `managedSchemaResourceName` will be read once, backed up, and converted for managed schema use. See <<schema-factory-definition-in-solrconfig.adoc#schema-factory-definition-in-solrconfig,Schema Factory Definition in SolrConfig>> for details.
|dataDir |string |No | |Name of the data directory relative to `instanceDir`.
|configSet |string |No | |Name of the configset to use for this core. For more information, see the section <<config-sets.adoc#config-sets,Config Sets>>.
|collection |string |No | |The name of the collection to which this core belongs. The default is the name of the core. `collection.<param>=<value>` causes a property of `<param>=<value>` to be set if a new collection is being created. Use `collection.configName=<configname>` to point to the configuration for a new collection.
|shard |string |No | |The shard id this core represents. Normally you want to be auto-assigned a shard id.
|property.__name__=__value__ |string |No | |Sets the core property _name_ to __value__. See the section on defining <<defining-core-properties.adoc#Definingcore.properties-core.properties_files,core.properties file contents>>.
|async |string |No | |Request ID to track this action which will be processed asynchronously
|===
Use `collection.configName=<configname>` to point to the config for a new collection.
[[CoreAdminAPI-Example]]
=== Example
`\http://localhost:8983/solr/admin/cores?action=CREATE&name=my_core&collection=my_collection&shard=shard2`
[WARNING]
====
While it's possible to create a core for a non-existent collection, this approach is not supported and not recommended. Always create a collection using the <<collections-api.adoc#collections-api,Collections API>> before creating a core directly for it.
====
[[CoreAdminAPI-RELOAD]]
== RELOAD
The RELOAD action loads a new core from the configuration of an existing, registered Solr core. While the new core is initializing, the existing one will continue to handle requests. When the new Solr core is ready, it takes over and the old core is unloaded.
`admin/cores?action=RELOAD&core=_core-name_`
This is useful when you've made changes to a Solr core's configuration on disk, such as adding new field definitions. Calling the RELOAD action lets you apply the new configuration without having to restart the Web container.
[IMPORTANT]
====
RELOAD performs "live" reloads of SolrCore, reusing some existing objects. Some configuration options, such as the `dataDir` location and `IndexWriter`-related settings in `solrconfig.xml` can not be changed and made active with a simple RELOAD action.
====
[[CoreAdminAPI-Input.2]]
=== Input
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|core |string |Yes |N/A |The name of the core, as listed in the "name" attribute of a `<core>` element in `solr.xml`.
|===
[[CoreAdminAPI-RENAME]]
== RENAME
The `RENAME` action changes the name of a Solr core.
`admin/cores?action=RENAME&core=_core-name_&other=_other-core-name_`
[[CoreAdminAPI-Input.3]]
=== Input
**Query Parameters**
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|core |string |Yes | |The name of the Solr core to be renamed.
|other |string |Yes | |The new name for the Solr core. If the persistent attribute of `<solr>` is `true`, the new name will be written to `solr.xml` as the `name` attribute of the `<core>` attribute.
|async |string |No | |Request ID to track this action which will be processed asynchronously
|===
[[CoreAdminAPI-SWAP]]
== SWAP
`SWAP` atomically swaps the names used to access two existing Solr cores. This can be used to swap new content into production. The prior core remains available and can be swapped back, if necessary. Each core will be known by the name of the other, after the swap.
`admin/cores?action=SWAP&core=_core-name_&other=_other-core-name_`
[IMPORTANT]
====
Do not use `SWAP` with a SolrCloud node. It is not supported and can result in the core being unusable.
====
[[CoreAdminAPI-Input.4]]
=== Input
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|core |string |Yes | |The name of one of the cores to be swapped.
|other |string |Yes | |The name of one of the cores to be swapped.
|async |string |No | |Request ID to track this action which will be processed asynchronously
|===
[[CoreAdminAPI-UNLOAD]]
== UNLOAD
The `UNLOAD` action removes a core from Solr. Active requests will continue to be processed, but no new requests will be sent to the named core. If a core is registered under more than one name, only the given name is removed.
`admin/cores?action=UNLOAD&core=_core-name_`
The `UNLOAD` action requires a parameter (`core`) identifying the core to be removed. If the persistent attribute of `<solr>` is set to `true`, the `<core>` element with this `name` attribute will be removed from `solr.xml`.
[IMPORTANT]
====
Unloading all cores in a SolrCloud collection causes the removal of that collection's metadata from ZooKeeper.
====
[[CoreAdminAPI-Input.5]]
=== Input
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|core |string |Yes | |The name of one of the cores to be removed.
|deleteIndex |boolean |No |false |If true, will remove the index when unloading the core.
|deleteDataDir |boolean |No |false |If true, removes the `data` directory and all sub-directories.
|deleteInstanceDir |boolean |No |false |If true, removes everything related to the core, including the index directory, configuration files and other related files.
|async |string |No | |Request ID to track this action which will be processed asynchronously
|===
[[CoreAdminAPI-MERGEINDEXES]]
== MERGEINDEXES
The `MERGEINDEXES` action merges one or more indexes to another index. The indexes must have completed commits, and should be locked against writes until the merge is complete or the resulting merged index may become corrupted. The target core index must already exist and have a compatible schema with the one or more indexes that will be merged to it. Another commit on the target core should also be performed after the merge is complete.
`admin/cores?action=MERGEINDEXES&core=_new-core-name_&indexDir=_path/to/core1/data/index_&indexDir=_path/to/core2/data/index_`
In this example, we use the `indexDir` parameter to define the index locations of the source cores. The `core` parameter defines the target index. A benefit of this approach is that we can merge any Lucene-based index that may not be associated with a Solr core.
Alternatively, we can instead use a `srcCore` parameter, as in this example:
`admin/cores?action=mergeindexes&core=_new-core-name_&srcCore=_core1-name_&srcCore=_core2-name_`
This approach allows us to define cores that may not have an index path that is on the same physical server as the target core. However, we can only use Solr cores as the source indexes. Another benefit of this approach is that we don't have as high a risk for corruption if writes occur in parallel with the source index.
We can make this call run asynchronously by specifying the `async` parameter and passing a request-id. This id can then be used to check the status of the already submitted task using the REQUESTSTATUS API.
[[CoreAdminAPI-Input.6]]
=== Input
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|core |string |Yes | |The name of the target core/index.
|indexDir |string | | |Multi-valued, directories that would be merged.
|srcCore |string | | |Multi-valued, source cores that would be merged.
|async |string | | |Request ID to track this action which will be processed asynchronously
|===
[[CoreAdminAPI-SPLIT]]
== SPLIT
The `SPLIT` action splits an index into two or more indexes. The index being split can continue to handle requests. The split pieces can be placed into a specified directory on the server's filesystem or it can be merged into running Solr cores.
The `SPLIT` action supports five parameters, which are described in the table below.
[[CoreAdminAPI-Input.7]]
=== Input
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|core |string |Yes | |The name of the core to be split.
|path |string | | |Multi-valued, the directory path in which a piece of the index will be written.
|targetCore |string | | |Multi-valued, the target Solr core to which a piece of the index will be merged
|ranges |string |No | |A comma-separated list of hash ranges in hexadecimal format
|split.key |string |No | |The key to be used for splitting the index
|async |string |No | |Request ID to track this action which will be processed asynchronously
|===
[IMPORTANT]
====
Either `path` or `targetCore` parameter must be specified but not both. The ranges and split.key parameters are optional and only one of the two should be specified, if at all required.
====
[[CoreAdminAPI-Examples]]
=== Examples
The `core` index will be split into as many pieces as the number of `path` or `targetCore` parameters.
==== Usage with two `targetCore` parameters:
`\http://localhost:8983/solr/admin/cores?action=SPLIT&core=core0&targetCore=core1&targetCore=core2`
Here the `core` index will be split into two pieces and merged into the two `targetCore` indexes.
==== Usage with two `path` parameters:
`\http://localhost:8983/solr/admin/cores?action=SPLIT&core=core0&path=/path/to/index/1&path=/path/to/index/2`
The `core` index will be split into two pieces and written into the two directory paths specified.
==== Usage with the `split.key` parameter:
`\http://localhost:8983/solr/admin/cores?action=SPLIT&core=core0&targetCore=core1&split.key=A!`
Here all documents having the same route key as the `split.key` i.e. 'A!' will be split from the `core` index and written to the `targetCore`.
==== Usage with `ranges` parameter:
`\http://localhost:8983/solr/admin/cores?action=SPLIT&core=core0&targetCore=core1&targetCore=core2&targetCore=core3&ranges=0-1f4,1f5-3e8,3e9-5dc`
This example uses the `ranges` parameter with hash ranges 0-500, 501-1000 and 1001-1500 specified in hexadecimal. Here the index will be split into three pieces with each targetCore receiving documents matching the hash ranges specified i.e. core1 will get documents with hash range 0-500, core2 will receive documents with hash range 501-1000 and finally, core3 will receive documents with hash range 1001-1500. At least one hash range must be specified. Please note that using a single hash range equal to a route key's hash range is NOT equivalent to using the `split.key` parameter because multiple route keys can hash to the same range.
The `targetCore` must already exist and must have a compatible schema with the `core` index. A commit is automatically called on the `core` index before it is split.
This command is used as part of the <<collections-api.adoc#CollectionsAPI-splitshard,SPLITSHARD>> command but it can be used for non-cloud Solr cores as well. When used against a non-cloud core without `split.key` parameter, this action will split the source index and distribute its documents alternately so that each split piece contains an equal number of documents. If the `split.key` parameter is specified then only documents having the same route key will be split from the source index.
[[CoreAdminAPI-REQUESTSTATUS]]
== REQUESTSTATUS
Request the status of an already submitted asynchronous CoreAdmin API call.
`admin/cores?action=REQUESTSTATUS&requestid=_id_`
[[CoreAdminAPI-Input.8]]
=== Input
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|requestid |string |Yes | |The user defined request-id for the Asynchronous request.
|===
The call below will return the status of an already submitted Asynchronous CoreAdmin call.
`\http://localhost:8983/solr/admin/cores?action=REQUESTSTATUS&requestid=1`
[[CoreAdminAPI-REQUESTRECOVERY]]
== REQUESTRECOVERY
The `REQUESTRECOVERY` action manually asks a core to recover by synching with the leader. This should be considered an "expert" level command and should be used in situations where the node (SorlCloud replica) is unable to become active automatically.
`admin/cores?action=REQUESTRECOVERY&core=_core-name_`
[[CoreAdminAPI-Input.9]]
=== Input
*Query Parameters*
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="15,10,10,10,55",options="header"]
|===
|Parameter |Type |Required |Default |Description
|core |string |Yes | |The name of the core to re-sync.
|===
[[CoreAdminAPI-Examples.1]]
=== Examples
`\http://localhost:8981/solr/admin/cores?action=REQUESTRECOVERY&core=gettingstarted_shard1_replica1`
The core to specify can be found by expanding the appropriate ZooKeeper node via the admin UI.

View File

@ -0,0 +1,761 @@
= Cross Data Center Replication (CDCR)
:page-shortname: cross-data-center-replication-cdcr
:page-permalink: cross-data-center-replication-cdcr.html
Cross Data Center Replication (CDCR) allows you to create multiple SolrCloud data centers and keep them in sync in case they are needed at a future time.
The <<solrcloud.adoc#solrcloud,SolrCloud>> architecture is not particularly well suited for situations where a single SolrCloud cluster consists of nodes in separated data clusters connected by an expensive pipe. The root problem is that SolrCloud is designed to support <<near-real-time-searching.adoc#near-real-time-searching,Near Real Time Searching>> by immediately forwarding updates between nodes in the cluster on a per-shard basis. "CDCR" features exist to help mitigate the risk of an entire data center outage.
== What is CDCR?
CDCR supports replicating data from one data center to multiple data centers. The initial version of the solution supports an active-passive scenario where data updates are replicated from a Source data center to one or more target data centers.
The target data center(s) will not propagate updates such as adds, updates, or deletes to the source data center and updates should _not_ be sent to any of the target data center(s).
Source and target data centers can serve search queries when CDCR is operating. The target data centers will have slightly stale views of the corpus due to propagation delays, but this is minimal (perhaps a few seconds).
Data changes on the source data center are replicated to the target data center only after they are persisted to disk. The data changes can be replicated in near real-time (with a small delay) or could be scheduled to be sent in intervals to the target data center. This solution pre-supposes that the source and target data centers begin with the same documents indexed. Of course the indexes may be empty to start.
Each shard leader in the source data center will be responsible for replicating its updates to the corresponding leader in the target data center. When receiving updates from the source data center, shard leaders in the target data center will replicate the changes to their own replicas.
This replication model is designed to tolerate some degradation in connectivity, accommodate limited bandwidth, and support batch updates to optimize communication.
Replication supports both a new empty index and pre-built indexes. In the scenario where the replication is set up on a pre-built index, CDCR will ensure consistency of the replication of the updates, but cannot ensure consistency on the full index. Therefore any index created before CDCR was set up will have to be replicated by other means (described in the section <<Initial Startup>>) so source and target indexes are fully consistent.
The active-passive nature of the initial implementation implies a "push" model from the source collection to the target collection. Therefore, the source configuration must be able to "see" the ZooKeeper ensemble in the target cluster. The ZooKeeper ensemble is provided configured in the Source's `solrconfig.xml` file.
CDCR is configured to replicate from collections in the source cluster to collections in the target cluster on a collection-by-collection basis. Since CDCR is configured in `solrconfig.xml` (on both source and target clusters), the settings can be tailored for the needs of each collection.
CDCR can be configured to replicate from one collection to a second collection _within the same cluster_. That is a specialized scenario not covered in this document.
[glossary]
== CDCR Glossary
Terms used in this document include:
[glossary]
Node:: A JVM instance running Solr; a server.
Cluster:: A set of Solr nodes managed as a single unit by a ZooKeeper ensemble, hosting one or more Collections.
Data Center:: A group of networked servers hosting a Solr cluster. In this document, the terms _Cluster_ and _Data Center_ are interchangeable as we assume that each Solr cluster is hosted in a different group of networked servers.
Shard:: A sub-index of a single logical collection. This may be spread across multiple nodes of the cluster. Each shard can have as many replicas as needed.
Leader:: Each shard has one node identified as its leader. All the writes for documents belonging to a shard are routed through the leader.
Replica:: A copy of a shard for use in failover or load balancing. Replicas comprising a shard can either be leaders or non-leaders.
Follower:: A convenience term for a replica that is _not_ the leader of a shard.
Collection:: Multiple documents that make up one logical index. A cluster can have multiple collections.
Updates Log:: An append-only log of write operations maintained by each node.
== CDCR Architecture
Here is a picture of the data flow.
.CDCR Data Flow
image::images/cross-data-center-replication-cdcr-/CDCR_arch.png[image,width=700,height=525]
Updates and deletes are first written to the Source cluster, then forwarded to the Target cluster. The data flow sequence is:
. A shard leader receives a new data update that is processed by its update processor chain.
. The data update is first applied to the local index.
. Upon successful application of the data update on the local index, the data update is added to the Updates Log queue.
. After the data update is persisted to disk, the data update is sent to the replicas within the data center.
. After Step 4 is successful, CDCR reads the data update from the Updates Log and pushes it to the corresponding collection in the target data center. This is necessary in order to ensure consistency between the Source and target data centers.
. The leader on the target data center writes the data locally and forwards it to all its followers.
Steps 1, 2, 3 and 4 are performed synchronously by SolrCloud; Step 5 is performed asynchronously by a background thread. Given that CDCR replication is performed asynchronously, it becomes possible to push batch updates in order to minimize network communication overhead. Also, if CDCR is unable to push the update at a given time, for example, due to a degradation in connectivity, it can retry later without any impact on the source data center.
One implication of the architecture is that the leaders in the source cluster must be able to "see" the leaders in the target cluster. Since leaders may change, this effectively means that all nodes in the source cluster must be able to "see" all Solr nodes in the target cluster so firewalls, ACL rules, etc. must be configured with care.
The current design works most robustly if both the Source and target clusters have the same number of shards. There is no requirement that the shards in the Source and target collection have the same number of replicas.
Having different numbers of shards on the Source and target cluster is possible, but is also an "expert" configuration as that option imposes certain constraints and is not recommended. Most of the scenarios where having differing numbers of shards are contemplated are better accomplished by hosting multiple shards on each target Solr instance.
== Major Components of CDCR
There are a number of key features and components in CDCRs architecture:
=== CDCR Configuration
In order to configure CDCR, the Source data center requires the host address of the ZooKeeper cluster associated with the target data center. The ZooKeeper host address is the only information needed by CDCR to instantiate the communication with the target Solr cluster. The CDCR configuration file on the source cluster will therefore contain a list of ZooKeeper hosts. The CDCR configuration file might also contain secondary/optional configuration, such as the number of CDC Replicator threads, batch updates related settings, etc.
=== CDCR Initialization
CDCR supports incremental updates to either new or existing collections. CDCR may not be able to keep up with very high volume updates, especially if there are significant communications latencies due to a slow "pipe" between the data centers. Some scenarios:
* There is an initial bulk load of a corpus followed by lower volume incremental updates. In this case, one can do the initial bulk load and then enable CDCR. See the section <<Initial Startup>> for more information.
* The index is being built up from scratch, without a significant initial bulk load. CDCR can be set up on empty collections and keep them synchronized from the start.
* The index is always being updated at a volume too high for CDCR to keep up. This is especially possible in situations where the connection between the Source and target data centers is poor. This scenario is unsuitable for CDCR in its current form.
=== Inter-Data Center Communication
Communication between data centers will be achieved through HTTP and the Solr REST API using the SolrJ client. The SolrJ client will be instantiated with the ZooKeeper host of the target data center. SolrJ will manage the shard leader discovery process.
=== Updates Tracking & Pushing
CDCR replicates data updates from the source to the target data center by leveraging the Updates Log.
A background thread regularly checks the Updates Log for new entries, and then forwards them to the target data center. The thread therefore needs to keep a checkpoint in the form of a pointer to the last update successfully processed in the Updates Log. Upon acknowledgement from the target data center that updates have been successfully processed, the Updates Log pointer is updated to reflect the current checkpoint.
This pointer must be synchronized across all the replicas. In the case where the leader goes down and a new leader is elected, the new leader will be able to resume replication from the last update by using this synchronized pointer. The strategy to synchronize such a pointer across replicas will be explained next.
If for some reason, the target data center is offline or fails to process the updates, the thread will periodically try to contact the target data center and push the updates.
=== Synchronization of Update Checkpoints
A reliable synchronization of the update checkpoints between the shard leader and shard replicas is critical to avoid introducing inconsistency between the Source and target data centers. Another important requirement is that the synchronization must be performed with minimal network traffic to maximize scalability.
In order to achieve this, the strategy is to:
* Uniquely identify each update operation. This unique identifier will serve as pointer.
* Rely on two storages: an ephemeral storage on the Source shard leader, and a persistent storage on the target cluster.
The shard leader in the source cluster will be in charge of generating a unique identifier for each update operation, and will keep a copy of the identifier of the last processed updates in memory. The identifier will be sent to the target cluster as part of the update request. On the target data center side, the shard leader will receive the update request, store it along with the unique identifier in the Updates Log, and replicate it to the other shards.
SolrCloud already provides a unique identifier for each update operation, i.e., a “version” number. This version number is generated using a time-based lmport clock which is incremented for each update operation sent. This provides an “happened-before” ordering of the update operations that will be leveraged in (1) the initialization of the update checkpoint on the source cluster, and in (2) the maintenance strategy of the Updates Log.
The persistent storage on the target cluster is used only during the election of a new shard leader on the Source cluster. If a shard leader goes down on the source cluster and a new leader is elected, the new leader will contact the target cluster to retrieve the last update checkpoint and instantiate its ephemeral pointer. On such a request, the target cluster will retrieve the latest identifier received across all the shards, and send it back to the source cluster. To retrieve the latest identifier, every shard leader will look up the identifier of the first entry in its Update Logs and send it back to a coordinator. The coordinator will have to select the highest among them.
This strategy does not require any additional network traffic and ensures reliable pointer synchronization. Consistency is principally achieved by leveraging SolrCloud. The update workflow of SolrCloud ensures that every update is applied to the leader but also to any of the replicas. If the leader goes down, a new leader is elected. During the leader election, a synchronization is performed between the new leader and the other replicas. As a result, this ensures that the new leader has a consistent Update Logs with the previous leader. Having a consistent Updates Log means that:
* On the source cluster, the update checkpoint can be reused by the new leader.
* On the target cluster, the update checkpoint will be consistent between the previous and new leader. This ensures the correctness of the update checkpoint sent by a newly elected leader from the target cluster.
=== Maintenance of Updates Log
The CDCR replication logic requires modification to the maintenance logic of the Updates Log on the source data center. Initially, the Updates Log acts as a fixed size queue, limited to 100 update entries. In the CDCR scenario, the Update Logs must act as a queue of variable size as they need to keep track of all the updates up through the last processed update by the target data center. Entries in the Update Logs are removed only when all pointers (one pointer per target data center) are after them.
If the communication with one of the target data center is slow, the Updates Log on the source data center can grow to a substantial size. In such a scenario, it is necessary for the Updates Log to be able to efficiently find a given update operation given its identifier. Given that its identifier is an incremental number, it is possible to implement an efficient search strategy. Each transaction log file contains as part of its filename the version number of the first element. This is used to quickly traverse all the transaction log files and find the transaction log file containing one specific version number.
[[CrossDataCenterReplication_CDCR_-Monitoring]]
=== Monitoring
CDCR provides the following monitoring capabilities over the replication operations:
* Monitoring of the outgoing and incoming replications, with information such as the Source and target nodes, their status, etc.
* Statistics about the replication, with information such as operations (add/delete) per second, number of documents in the queue, etc.
Information about the lifecycle and statistics will be provided on a per-shard basis by the CDC Replicator thread. The CDCR API can then aggregate this information an a collection level.
=== CDC Replicator
The CDC Replicator is a background thread that is responsible for replicating updates from a Source data center to one or more target data centers. It is responsible in providing monitoring information on a per-shard basis. As there can be a large number of collections and shards in a cluster, we will use a fixed-size pool of CDC Replicator threads that will be shared across shards.
[[CrossDataCenterReplication_CDCR_-Limitations]]
=== Limitations
The current design of CDCR has some limitations. CDCR will continue to evolve over time and many of these limitations will be addressed. Among them are:
* CDCR is unlikely to be satisfactory for bulk-load situations where the update rate is high, especially if the bandwidth between the Source and target clusters is restricted. In this scenario, the initial bulk load should be performed, the Source and target data centers synchronized and CDCR be utilized for incremental updates.
* CDCR is currently only active-passive; data is pushed from the Source cluster to the target cluster. There is active work being done in this area in the 6x code line to remove this limitation.
* CDCR works most robustly with the same number of shards in the Source and target collection. The shards in the two collections may have different numbers of replicas.
[[CrossDataCenterReplication_CDCR_-Configuration]]
== Configuration
The source and target configurations differ in the case of the data centers being in separate clusters. "Cluster" here means separate ZooKeeper ensembles controlling disjoint Solr instances. Whether these data centers are physically separated or not is immaterial for this discussion.
[[CrossDataCenterReplication_CDCR_-SourceConfiguration]]
=== Source Configuration
Here is a sample of a source configuration file, a section in `solrconfig.xml`. The presence of the <replica> section causes CDCR to use this cluster as the Source and should not be present in the target collections in the cluster-to-cluster case. Details about each setting are after the two examples:
[source,xml]
----
<requestHandler name="/cdcr" class="solr.CdcrRequestHandler">
<lst name="replica">
<str name="zkHost">10.240.18.211:2181</str>
<str name="source">collection1</str>
<str name="target">collection1</str>
</lst>
<lst name="replicator">
<str name="threadPoolSize">8</str>
<str name="schedule">1000</str>
<str name="batchSize">128</str>
</lst>
<lst name="updateLogSynchronizer">
<str name="schedule">1000</str>
</lst>
</requestHandler>
<!-- Modify the <updateLog> section of your existing <updateHandler>
in your config as below -->
<updateHandler class="solr.DirectUpdateHandler2">
<updateLog class="solr.CdcrUpdateLog">
<str name="dir">${solr.ulog.dir:}</str>
<!--Any parameters from the original <updateLog> section -->
</updateLog>
</updateHandler>
----
[[CrossDataCenterReplication_CDCR_-TargetConfiguration]]
=== Target Configuration
Here is a typical target configuration.
Target instance must configure an update processor chain that is specific to CDCR. The update processor chain must include the *CdcrUpdateProcessorFactory*. The task of this processor is to ensure that the version numbers attached to update requests coming from a CDCR source SolrCloud are reused and not overwritten by the target. A properly configured Target configuration looks similar to this.
[source,xml]
----
<requestHandler name="/cdcr" class="solr.CdcrRequestHandler">
<lst name="buffer">
<str name="defaultState">disabled</str>
</lst>
</requestHandler>
<requestHandler name="/update" class="solr.UpdateRequestHandler">
<lst name="defaults">
<str name="update.chain">cdcr-processor-chain</str>
</lst>
</requestHandler>
<updateRequestProcessorChain name="cdcr-processor-chain">
<processor class="solr.CdcrUpdateProcessorFactory"/>
<processor class="solr.RunUpdateProcessorFactory"/>
</updateRequestProcessorChain>
<!-- Modify the <updateLog> section of your existing <updateHandler> in your
config as below -->
<updateHandler class="solr.DirectUpdateHandler2">
<updateLog class="solr.CdcrUpdateLog">
<str name="dir">${solr.ulog.dir:}</str>
<!--Any parameters from the original <updateLog> section -->
</updateLog>
</updateHandler>
----
=== Configuration Details
The configuration details, defaults and options are as follows:
==== The Replica Element
CDCR can be configured to forward update requests to one or more replicas. A replica is defined with a “replica” list as follows:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,10,15,55",options="header"]
|===
|Parameter |Required |Default |Description
|zkHost |Yes |none |The host address for ZooKeeper of the target SolrCloud. Usually this is a comma-separated list of addresses to each node in the target ZooKeeper ensemble.
|Source |Yes |none |The name of the collection on the Source SolrCloud to be replicated.
|Target |Yes |none |The name of the collection on the target SolrCloud to which updates will be forwarded.
|===
==== The Replicator Element
The CDC Replicator is the component in charge of forwarding updates to the replicas. The replicator will monitor the update logs of the Source collection and will forward any new updates to the target collection.
The replicator uses a fixed thread pool to forward updates to multiple replicas in parallel. If more than one replica is configured, one thread will forward a batch of updates from one replica at a time in a round-robin fashion. The replicator can be configured with a “replicator” list as follows:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,10,15,55",options="header"]
|===
|Parameter |Required |Default |Description
|threadPoolSize |No |2 |The number of threads to use for forwarding updates. One thread per replica is recommended.
|schedule |No |10 |The delay in milliseconds for the monitoring the update log(s).
|batchSize |No |128 |The number of updates to send in one batch. The optimal size depends on the size of the documents. Large batches of large documents can increase your memory usage significantly.
|===
==== The updateLogSynchronizer Element
Expert: Non-leader nodes need to synchronize their update logs with their leader node from time to time in order to clean deprecated transaction log files. By default, such a synchronization process is performed every minute. The schedule of the synchronization can be modified with a “updateLogSynchronizer” list as follows:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,10,15,55",options="header"]
|===
|Parameter |Required |Default |Description
|schedule |No |60000 |The delay in milliseconds for synchronizing the updates log.
|===
==== The Buffer Element
CDCR is configured by default to buffer any new incoming updates. When buffering updates, the updates log will store all the updates indefinitely. Replicas do not need to buffer updates, and it is recommended to disable buffer on the target SolrCloud. The buffer can be disabled at startup with a “buffer” list and the parameter “defaultState” as follows:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,10,15,55",options="header"]
|===
|Parameter |Required |Default |Description
|defaultState |No |enabled |The state of the buffer at startup.
|===
== CDCR API
The CDCR API is used to control and monitor the replication process. Control actions are performed at a collection level, i.e., by using the following base URL for API calls: `\http://localhost:8983/solr/<collection>`.
Monitor actions are performed at a core level, i.e., by using the following base URL for API calls: `\http://localhost:8983/solr/<collection>`.
Currently, none of the CDCR API calls have parameters.
=== API Entry Points (Control)
* `<collection>/cdcr?action=STATUS`: <<CrossDataCenterReplication_CDCR_-STATUS,Returns the current state>> of CDCR.
* `<collection>/cdcr?action=START`: <<CrossDataCenterReplication_CDCR_-START,Starts CDCR>> replication
* `<collection>/cdcr?action=STOP`: <<CrossDataCenterReplication_CDCR_-STOP,Stops CDCR>> replication.
* `<collection>/cdcr?action=ENABLEBUFFER`: <<CrossDataCenterReplication_CDCR_-ENABLEBUFFER,Enables the buffering>> of updates.
* `<collection>/cdcr?action=DISABLEBUFFER`: <<CrossDataCenterReplication_CDCR_-DISABLEBUFFER,Disables the buffering>> of updates.
=== API Entry Points (Monitoring)
* `core/cdcr?action=QUEUES`: <<CrossDataCenterReplication_CDCR_-QUEUES,Fetches statistics about the queue>> for each replica and about the update logs.
* `core/cdcr?action=OPS`: <<CrossDataCenterReplication_CDCR_-OPS,Fetches statistics about the replication performance>> (operations per second) for each replica.
* `core/cdcr?action=ERRORS`: <<CrossDataCenterReplication_CDCR_-ERRORS,Fetches statistics and other information about replication errors>> for each replica.
=== Control Commands
[[CrossDataCenterReplication_CDCR_-STATUS]]
==== STATUS
`/collection/cdcr?action=STATUS`
===== Input
*Query Parameters:* There are no parameters to this command.
===== Output
*Output Content*
The current state of the CDCR, which includes the state of the replication process and the state of the buffer.
[[cdcr_examples]]
===== Examples
*Input*
[source,text]
----
http://host:8983/solr/<collection_name>/cdcr?action=STATUS
----
*Output*
[source,json]
----
{
"responseHeader": {
"status": 0,
"QTime": 0
},
"status": {
"process": "stopped",
"buffer": "enabled"
}
}
----
[[CrossDataCenterReplication_CDCR_-ENABLEBUFFER]]
==== ENABLEBUFFER
`/collection/cdcr?action=ENABLEBUFFER`
===== Input
*Query Parameters:* There are no parameters to this command.
===== Output
*Output Content*
The status of the process and an indication of whether the buffer is enabled
===== Examples
*Input*
[source,text]
----
http://host:8983/solr/<collection_name>/cdcr?action=ENABLEBUFFER
----
*Output*
[source,json]
----
{
"responseHeader": {
"status": 0,
"QTime": 0
},
"status": {
"process": "started",
"buffer": "enabled"
}
}
----
[[CrossDataCenterReplication_CDCR_-DISABLEBUFFER]]
==== DISABLEBUFFER
`/collection/cdcr?action=DISABLEBUFFER`
===== Input
*Query Parameters:* There are no parameters to this command
===== Output
*Output Content:* The status of CDCR and an indication that the buffer is disabled.
===== Examples
*Input*
[source,text]
----
http://host:8983/solr/<collection_name>/cdcr?action=DISABLEBUFFER
----
*Output*
[source,json]
----
{
"responseHeader": {
"status": 0,
"QTime": 0
},
"status": {
"process": "started",
"buffer": "disabled"
}
}
----
[[CrossDataCenterReplication_CDCR_-START]]
==== START
`/collection/cdcr?action=START`
===== Input
*Query Parameters:* There are no parameters for this action
===== Output
*Output Content:* Confirmation that CDCR is started and the status of buffering
===== Examples
*Input*
[source,text]
----
http://host:8983/solr/<collection_name>/cdcr?action=START
----
*Output*
[source,json]
----
{
"responseHeader": {
"status": 0,
"QTime": 0
},
"status": {
"process": "started",
"buffer": "enabled"
}
}
----
[[CrossDataCenterReplication_CDCR_-STOP]]
==== STOP
`/collection/cdcr?action=STOP`
===== Input
*Query Parameters:* There are no parameters for this command.
===== Output
*Output Content:* The status of CDCR, including the confirmation that CDCR is stopped
===== Examples
*Input*
[source,text]
----
http://host:8983/solr/<collection_name>/cdcr?action=STOP
----
*Output*
[source,json]
----
{
"responseHeader": {
"status": 0,
"QTime": 0
},
"status": {
"process": "stopped",
"buffer": "enabled"
}
}
----
[[CrossDataCenterReplication_CDCR_-Monitoringcommands]]
=== Monitoring commands
[[CrossDataCenterReplication_CDCR_-QUEUES]]
==== QUEUES
`/core/cdcr?action=QUEUES`
===== Input
*Query Parameters:* There are no parameters for this command
===== Output
*Output Content*
The output is composed of a list “queues” which contains a list of (ZooKeeper) target hosts, themselves containing a list of target collections. For each collection, the current size of the queue and the timestamp of the last update operation successfully processed is provided. The timestamp of the update operation is the original timestamp, i.e., the time this operation was processed on the Source SolrCloud. This allows an estimate the latency of the replication process.
The “queues” object also contains information about the updates log, such as the size (in bytes) of the updates log on disk (“tlogTotalSize”), the number of transaction log files (“tlogTotalCount”) and the status of the updates log synchronizer (“updateLogSynchronizer”).
===== Examples
*Input*
[source,text]
----
http://host:8983/solr/<replica_name>/cdcr?action=QUEUES
----
*Output*
[source,json]
----
{
"responseHeader":{
"status": 0,
"QTime": 1
},
"queues":{
"127.0.0.1: 40342/solr":{
"Target_collection":{
"queueSize": 104,
"lastTimestamp": "2014-12-02T10:32:15.879Z"
}
}
},
"tlogTotalSize":3817,
"tlogTotalCount":1,
"updateLogSynchronizer": "stopped"
}
----
[[CrossDataCenterReplication_CDCR_-OPS]]
==== OPS
`/core/cdcr?action=OPS`
===== Input
*Query Parameters:* There are no parameters for this command.
===== Output
*Output Content:* The output is composed of a list “operationsPerSecond” which contains a list of (ZooKeeper) target hosts, themselves containing a list of target collections. For each collection, the average number of processed operations per second since the start of the replication process is provided. The operations are further broken down into two groups: add and delete operations.
===== Examples
*Input*
[source,text]
----
http://host:8983/solr/<collection_name>/cdcr?action=OPS
----
*Output*
[source,json]
----
{
"responseHeader":{
"status":0,
"QTime":1
},
"operationsPerSecond":{
"127.0.0.1: 59661/solr":{
"Target_collection":{
"all": 297.102944952749052,
"adds": 297.102944952749052,
"deletes": 0.0
}
}
}
}
----
[[CrossDataCenterReplication_CDCR_-ERRORS]]
==== ERRORS
`/core/cdcr?action=ERRORS`
===== Input
*Query Parameters:* There are no parameters for this command.
===== Output
*Output Content:* The output is composed of a list “errors” which contains a list of (ZooKeeper) target hosts, themselves containing a list of target collections. For each collection, information about errors encountered during the replication is provided, such as the number of consecutive errors encountered by the replicator thread, the number of bad requests or internal errors since the start of the replication process, and a list of the last errors encountered ordered by timestamp.
===== Examples
*Input*
[source,text]
----
http://host:8983/solr/<collection_name>/cdcr?action=ERRORS
----
*Output*
[source,json]
----
{
"responseHeader":{
"status":0,
"QTime":2
},
"errors": {
"127.0.0.1: 36872/solr":{
"Target_collection":{
"consecutiveErrors":3,
"bad_request":0,
"internal":3,
"last":{
"2014-12-02T11:04:42.523Z":"internal",
"2014-12-02T11:04:39.223Z":"internal",
"2014-12-02T11:04:38.22Z":"internal"
}
}
}
}
}
----
== Initial Startup
This is a general approach for initializing CDCR in a production environment based upon an approach taken by the initial working installation of CDCR and generously contributed to illustrate a "real world" scenario.
* Customer uses the CDCR approach to keep a remote disaster-recovery instance available for production backup. This is an active-passive solution.
* Customer has 26 clouds with 200 million assets per cloud (15GB indexes). Total document count is over 4.8 billion.
** Source and target clouds were synched in 2-3 hour maintenance windows to establish the base index for the targets.
As usual, it is good to start small. Sync a single cloud and monitor for a period of time before doing the others. You may need to adjust your settings several times before finding the right balance.
* Before starting, stop or pause the indexers. This is best done during a small maintenance window.
* Stop the SolrCloud instances at the Source
* Include the CDCR request handler configuration in `solrconfig.xml` as in the below example.
+
[source,xml]
----
<requestHandler name="/cdcr" class="solr.CdcrRequestHandler">
<lst name="replica">
<str name="zkHost">${TargetZk}</str>
<str name="Source">${SourceCollection}</str>
<str name="Target">${TargetCollection}</str>
</lst>
<lst name="replicator">
<str name="threadPoolSize">8</str>
<str name="schedule">10</str>
<str name="batchSize">2000</str>
</lst>
<lst name="updateLogSynchronizer">
<str name="schedule">1000</str>
</lst>
</requestHandler>
<updateRequestProcessorChain name="cdcr-processor-chain">
<processor class="solr.CdcrUpdateProcessorFactory" />
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
----
+
* Upload the modified `solrconfig.xml` to ZooKeeper on both Source and Target
* Sync the index directories from the Source collection to target collection across to the corresponding shard nodes. `rsync` works well for this.
+
For example, if there are 2 shards on collection1 with 2 replicas for each shard, copy the corresponding index directories from
+
[width="75%",cols="45,10,45"]
|===
|shard1replica1Source |to |shard1replica1Target
|shard1replica2Source |to |shard1replica2Target
|shard2replica1Source |to |shard2replica1Target
|shard2replica2Source |to |shard2replica2Target
|===
+
* Start the ZooKeeper on the Target (DR) side
* Start the SolrCloud on the Target (DR) side
* Start the ZooKeeper on the Source side
* Start the SolrCloud on the Source side. As a general rule, the Target (DR) side of the SolrCloud should be started before the Source side.
* Activate the CDCR on Source instance using the CDCR API: `\http://host:port/solr/collection_name/cdcr?action=START`
+
[source,text]
http://host:port/solr/<collection_name>/cdcr?action=START
+
* There is no need to run the /cdcr?action=START command on the Target
* Disable the buffer on the Target
+
[source,text]
http://host:port/solr/collection_name/cdcr?action=DISABLEBUFFER
+
* Renable indexing
[[CrossDataCenterReplication_CDCR_-Monitoring.1]]
== Monitoring
. Network and disk space monitoring are essential. Ensure that the system has plenty of available storage to queue up changes if there is a disconnect between the Source and Target. A network outage between the two data centers can cause your disk usage to grow.
.. Tip: Set a monitor for your disks to send alerts when the disk gets over a certain percentage (e.g., 70%)
.. Tip: Run a test. With moderate indexing, how long can the system queue changes before you run out of disk space?
. Create a simple way to check the counts between the Source and the Target.
.. Keep in mind that if indexing is running, the Source and Target may not match document for document. Set an alert to fire if the difference is greater than some percentage of the overall cloud size.
== ZooKeeper Settings
With CDCR, the target ZooKeepers will have connections from the Target clouds and the Source clouds. You may need to increase the `maxClientCnxns` setting in `zoo.cfg`.
[source,text]
----
## set numbers of connection to 200 from client
## is maxClientCnxns=0 that means no limit
maxClientCnxns=800
----
== Upgrading and Patching Production
When rolling in upgrades to your indexer or application, you should shutdown the Source (production) and the Target (DR). Depending on your setup, you may want to pause/stop indexing. Deploy the release or patch and renable indexing. Then start the Target (DR).
* There is no need to reissue the DISABLEBUFFERS or START commands. These are persisted.
* After starting the Target, run a simple test. Add a test document to each of the Source clouds. Then check for it on the Target.
[source,bash]
----
#send to the Source
curl http://<Source>/solr/cloud1/update -H 'Content-type:application/json' -d '[{"SKU":"ABC"}]'
#check the Target
curl "http://<Target>:8983/solr/<collection_name>/select?q=SKU:ABC&wt=json&indent=true"
----
[[CrossDataCenterReplication_CDCR_-Limitations.1]]
== Limitations
* Running CDCR with the indexes on HDFS is not currently supported, see: https://issues.apache.org/jira/browse/SOLR-9861[Solr CDCR over HDFS].

View File

@ -0,0 +1,164 @@
/* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* comments.css
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ */
#comments_thread a:link {
color: #5A88B5;
background-color: inherit;
}
#comments_thread a:visited {
color: #5A88B5;
background-color: inherit;
}
#comments_thread a:link:hover,
#comments_thread a:link:active,
#comments_thread a:visited:hover,
#comments_thread a:visited:active {
color: #0073c7;
background-color: #f0f0f0;
}
/* in general */
#comments_thread p {
line-height: 1.3em;
color: #003;
}
#comments_thread h4 {
font-size: 14px;
}
.apaste_menu {
float: right;
margin-right: 10px;
width: 80px;
}
.apaste_comment {
background: #FEFEFE;
border: 1px solid #AAA;
border-radius: 2px;
display: block;
white-space: pre-wrap;
font-weight: normal;
padding-left: 20px;
padding-right: 20px;
padding-bottom: 16px;
padding-top: 5px;
margin: 15px;
font-size: 13px
}
.comment_header {
color: #000000;
border-radius: 3px;
border: 1px solid #999;
min-height: 24px;
text-indent: 5px;
font-size: 12pt;
background: #ffe9a3; /* Old browsers */
background: -moz-linear-gradient(top, #ffe9a3 0%, #ffd08a 32%, #ff9d57 69%, #ff833d 100%); /* FF3.6-15 */
background: -webkit-linear-gradient(top, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* Chrome10-25,Safari5.1-6 */
background: linear-gradient(to bottom, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* W3C, IE10+, FF16+, Chrome26+, Opera12+, Safari7+ */
}
.comment_header_verified {
color: #000000;
border-radius: 3px;
border: 1px solid #999;
min-height: 24px;
text-indent: 5px;
font-size: 12pt;
background: #ffe9a3; /* Old browsers */
background: -moz-linear-gradient(top, #ffe9a3 0%, #ffd08a 32%, #ff9d57 69%, #ff833d 100%); /* FF3.6-15 */
background: -webkit-linear-gradient(top, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* Chrome10-25,Safari5.1-6 */
background: linear-gradient(to bottom, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* W3C, IE10+, FF16+, Chrome26+, Opera12+, Safari7+ */
}
.comment_header_sticky {
color: #000000;
border-radius: 3px;
border: 1px solid #999;
min-height: 24px;
text-indent: 5px;
font-size: 12pt;
background: #ffe9a3; /* Old browsers */
background: -moz-linear-gradient(top, #ffe9a3 0%, #ffd08a 32%, #ff9d57 69%, #ff833d 100%); /* FF3.6-15 */
background: -webkit-linear-gradient(top, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* Chrome10-25,Safari5.1-6 */
background: linear-gradient(to bottom, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* W3C, IE10+, FF16+, Chrome26+, Opera12+, Safari7+ */
}
.comment_header img {
padding-top: 3px;
padding-bottom: 2px;
}
.comment_header_verified img {
padding-top: 3px;
padding-bottom: 2px;
}
.comment_header_sticky img {
padding-top: 3px;
padding-bottom: 2px;
}
.apaste_comment img {
/* border-radius: 5px;*/
border: none;
}
.apaste_comment_selected {background: #F8F4E9;}
.apaste_comment_notapproved {background: #F8E0E0;}
.apaste_comment_resolved {background: #FAFCFA;}
.apaste_comment_sticky {background: #FFFFF6;}
.apaste_comment_verified {background: #FAFBFA;}
.apaste_comment_invalid {
color: #999;
background: #F8F8F8;
}
.apaste_comment textarea {
width: 480px;
height: 180px;
}
#apaste {
margin: 5px;
font-weight: normal;
font-size: 14px;
color: #024;
}
#apaste .section {
padding: 20px;
padding-left: 80px;
}
.notapproved {
background-color: #FEE;
padding: 5px;
}
#comments_thread textarea{
background-color: #ffffff;
width: auto;
border: 1px solid #1c1c1c;
border-radius: 3px;
box-shadow: 0pt 1px 3px rgba(0, 0, 0, 0.16) inset;
position: relative;
}
.apaste_honeypot {
display: none;
}
//* Remove external link icons when they appear in comments *//
a[href^="http://"]:after,
a[href^="https://"]:after {
content: none !important;
}

View File

@ -0,0 +1,869 @@
.gi-2x{font-size: 2em;}
.gi-3x{font-size: 3em;}
.gi-4x{font-size: 4em;}
.gi-5x{font-size: 5em;}
.breadcrumb > .active {color: #777 !important;}
.post-content img {
margin: 12px 0 3px 0;
width: auto;
height: auto;
max-width: 100%;
max-height: 100%;
}
.post-content ol li, .post-content ul li {
margin: 10px 0;
}
.pageSummary {
font-size:13px;
display:block;
margin-bottom:15px;
padding-left:20px;
}
.post-summary {
margin-bottom:12px;
}
.bs-example{
margin: 20px;
}
.breadcrumb li {
color: gray;
}
caption {
padding-top: 8px;
padding-bottom: 8px;
color: #777;
text-align: left;
}
p.external a {
text-align:right;
font-size:12px;
color: #0088cc;
display:inline;
}
#definition-box-container div a.active {
font-weight: bold;
}
p.post-meta {font-size: 80%; color: #777;}
.entry-date{font-size:14px;font-size:0.875rem;line-height:1.71429;margin-bottom:0;text-transform:uppercase;}
/* search area */
#search-demo-container ul#results-container {
list-style: none;
font-size: 12px;
background-color: white;
position: absolute;
top: 40px; /* if you change anything about the nav, you'll prob. need to reset the top and left values here.*/
left: 20px;
z-index: -1;
width:223px;
border-left: 1px solid #dedede;
box-shadow: 2px 3px 2px #dedede;
}
/* make room for the nav bar */
h1[id],
h2[id],
h3[id],
h4[id],
h5[id],
h6[id],
dt[id]{
padding-top: 60px;
margin-top: -40px
}
ul#results-container a {
background-color: transparent;
}
ul#results-container a:hover {
color: black;
}
#search-demo-container a:hover {
color: black;
}
#search-input {
padding: .5em;
margin-left:20px;
width:20em;
font-size: 0.8em;
-webkit-box-sizing: border-box;
-moz-box-sizing: border-box;
box-sizing: border-box;
float: right;
margin-top:10px;
}
/* end search */
.filter-options {
margin-bottom: 20px;
}
.filter-options button {
margin: 3px;
}
li.dropdownActive a {
font-weight: bold;
}
.post-content a.fa-rss {
color: orange;
}
.navbar-inverse .navbar-nav > li > a {
background-color: transparent;
margin-top:10px;
}
.post-content .rssfeedLink {
color: #248EC2;
}
footer {
font-size: smaller;
}
/* FAQ page */
#accordion .panel-heading {
font-size: 12px;
}
a.accordion-toggle, a.accordion-collapsed {
font-size: 14px;
text-decoration: none;
}
/* navgoco sidebar styles (customized) */
.nav, .nav ul, .nav li {
list-style: none;
}
.nav ul {
padding: 0;
/*margin: 0 0 0 18px;*/
margin:0;
}
.nav {
/* padding: 4px;*/
padding:0;
margin: 0;
}
.nav > li {
margin: 1px 0;
}
.nav > li li {
margin: 2px 0;
}
.nav a {
color: #333;
display: block;
outline: none;
text-decoration: none;
}
.nav li > a > span {
float: right;
font-size: 19px;
font-weight: bolder;
}
.nav li > a > span:after {
content: '\25be';
}
.nav li.open > a > span:after {
content: '\25b4';
}
.nav a:hover, .nav a:focus, .nav li.active > a {
background-color: #8D8D8D;
color: #f5f5f5;
}
.nav > li.active > a {
background-color: #347DBE;
}
.nav li a {
line-height: 18px;
padding: 2px 10px;
background-color: #f1f1f1;
}
.nav > li > a {
line-height: 20px;
padding: 4px 10px;
}
ul#mysidebar {
border-radius:0;
}
ul.nav li ul {
font-size: 10pt;
}
.nav ul li a {
background-color: #FAFAFA;
}
.nav li a {
padding-right:10px;
}
.nav li a:hover {
background-color: #8D8D8D;
}
.nav ul li a {
border-top:1px solid whitesmoke;
padding-left:10px;
}
/* end sidebar */
.navbar-inverse .navbar-nav > .active > a, .navbar-inverse .navbar-nav > .active > a:hover, .navbar-inverse .navbar-nav > .active > a:focus {
border-radius:5px;
}
.navbar-inverse .navbar-nav>.open>a, .navbar-inverse .navbar-nav>.open>a:focus, .navbar-inverse .navbar-nav>.open>a:hover {
border-radius: 5px;
}
.footer {
text-align: right;
}
.footerMeta {
background-color: whitesmoke;
padding: 10px;
max-width: 250px;
border-radius: 5px;
margin-top: 50px;
font-style:italic;
font-size:12px;
}
img.screenshotSmall {
max-width: 300px;
}
dl dt p {
margin-left:20px;
}
dl dd {
margin-top:10px;
margin-bottom:10px;
}
dl.dl-horizontal dd {
padding-top: 20px;
}
figcaption {
padding-bottom:12px;
padding-top:6px;
max-width: 90%;
margin-bottom:20px;
font-style: italic;
color: gray;
}
.testing {
color: orange;
}
.preference {
color: red;
}
table.dataTable thead {
background-color: #444;
}
section table tr.success {
background-color: #dff0d8 !important;
}
table tr.info {
background-color: #d9edf7 !important;
}
section table tr.warning, table tr.testing, table tr.testing > td.sorting_1 {
background-color: #fcf8e3 !important;
}
section table tr.danger, table tr.preference, table tr.preference > td.sorting_1 {
background-color: #f2dede !important;
}
.orange {
color: orange;
}
table.profile thead tr th {
background-color: #248ec2;
}
table.request thead tr th {
background-color: #ED1951;
}
.audienceLabel {
margin: 10px;
float: right;
border:1px solid #dedede;
padding:7px;
}
.prefaceAudienceLabel {
color: gray;
text-align: center;
margin:5px;
}
span.myLabel {
padding-left:10px;
padding-right:10px;
}
button.cursorNorm {
cursor: default;
}
a.dropdown-toggle, .navbar-inverse .navbar-nav > li > a {
margin-left: 10px;
}
hr.faded {
border: 0;
height: 1px;
background-image: -webkit-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));
background-image: -moz-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));
background-image: -ms-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));
background-image: -o-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));
}
hr.shaded {
height: 12px;
border: 0;
box-shadow: inset 0 6px 6px -6px rgba(0,0,0,0.5);
margin-top: 70px;
background: white;
width: 100%;
margin-bottom: 10px;
}
.fa-6x{font-size:900%;}
.fa-7x{font-size:1100%;}
.fa-8x{font-size:1300%;}
.fa-9x{font-size:1500%;}
.fa-10x{font-size:1700%;}
i.border {
padding: 10px 20px;
background-color: whitesmoke;
}
a[data-toggle] {
color: #248EC2;
}
.summary {
font-size:120%;
color: #808080;
margin:20px 0 20px 0;
border-left: 5px solid #ED1951;
padding-left: 10px;
}
.summary:before {
content: "Summary: ";
font-weight: bold;
}
a.fa.fa-envelope-o.mailto {
font-weight: 600;
}
.nav-tabs > li.active > a, .nav-tabs > li.active > a:hover, .nav-tabs > li.active > a:focus {
background-color: #248ec2;
color: white;
}
ol li ol li {list-style-type: lower-alpha;}
ol li ul li {list-style-type: disc;}
li img {clear:both; }
div#toc ul li ul li {
list-style-type: none;
margin: 5px 0 0 0;
}
.tab-content {
padding: 15px;
background-color: #FAFAFA;
}
span.tagTitle {font-weight: 500;}
li.activeSeries {
font-weight: bold;
}
.seriesContext .dropdown-menu li.active {
font-weight: bold;
margin-left: 43px;
font-size:18px;
}
div.tags {padding: 10px 5px;}
.tabLabel {
font-weight: normal;
}
hr {
border: 0;
border-bottom: 1px dashed #ccc;
background: #999;
margin: 30px 0;
width: 90%;
margin-left: auto;
margin-right: auto;
}
button.cursorNorm {
cursor: pointer;
}
span.otherProgrammingLanguages {
font-style: normal;
}
a[data-toggle="tooltip"] {
color: #649345;
font-style: italic;
cursor: default;
}
.seriesNext, .seriesContext {
margin-top: 15px;
margin-bottom: 15px;
}
.seriescontext ol li {
list-style-type: upper-roman;
}
ol.series li {
list-style-type: decimal;
margin-left: 40px;
padding-left: 0;
}
.siteTagline {
font-size: 200%;
font-weight: bold;
color: silver;
font-family: monospace;
text-align: center;
line-height: 10px;
margin: 20px 0;
display: block;
}
.versionTagline {
text-align: center;
margin-bottom: 20px;
font-family: courier;
color: silver;
color: #444;
display:block;
}
#mysidebar .nav ul {
background-color: #FAFAFA;
}
.nav ul.series li {
list-style: decimal;
font-size:12px;
}
.nav ul.series li a:hover {
background-color: gray;
}
.nav ul.series {
padding-left: 30px;
}
.nav ul.series {
background-color: #FAFAFA;
}
/*
a.dropdown-toggle.otherProgLangs {
color: #f7e68f !important;
}
*/
span.muted {color: #666;}
table code {background-color: transparent;}
.highlight .err {
color: #a61717;
background-color: transparent !important;
}
#json-box-container pre {
margin: 0;
}
.video-js {
margin: 30px 0;
}
video {
display: block;
margin: 30px 0;
border: 1px solid #c0c0c0;
}
p.required, p.dataType {display: block; color: #c0c0c0; font-size: 80%; margin-left:4px;}
dd {margin-left:20px;}
.post-content img.inline {
margin:0;
margin-bottom:6px;
}
.panel-heading {
font-weight: bold;
}
a.accordion-toggle {
font-style: normal;
}
span.red {
color: red;
font-family: Monaco, Menlo, Consolas, "Courier New", monospace;
}
h3.codeExplanation {
font-size:18px;
font-style:normal;
color: black;
line-height: 24px;
}
span.soft {
color: #c0c0c0;
}
.githubEditButton {
margin-bottom:7px;
}
.endpoint {
padding: 15px;
background-color: #f0f0f0;
font-family: courier;
font-size: 110%;
margin: 20px 0;
color: #444;
}
.parameter {
font-family: courier;
color: red !important;
}
.formBoundary {
border: 1px solid gray;
padding: 15px;
margin: 15px 0;
background-color: whitesmoke;
}
@media (max-width: 767px) {
.navbar-inverse .navbar-nav .open .dropdown-menu > li > a {
color: #444;
}
}
@media (max-width: 990px) {
#mysidebar {
position: relative;
}
}
@media (min-width: 1000px) {
ul#mysidebar {
width: 225px;
}
}
@media (max-width: 900px) {
ul#mysidebar {
max-width: 100%;
}
}
.col-md-9 img {
max-width: 100%;
max-height: 100%;
}
.videoThumbs img {
float: left;
margin:15px 15px 15px 0;
box-shadow: 2px 2px 1px #f0f0f0;
border: 1px solid #dedede;
}
@media only screen and (min-width: 900px)
{.col-md-9 img {
max-width: 700px;
max-height: 700px;
}
}
@media only screen and (min-device-width: 900px)
{.col-md-9 img {
max-width: 700px;
max-height: 700px;
}
}
*:hover > .anchorjs-link {
transition: color .25s linear;
text-decoration: none;
}
.kbCaption {
color: white;
background-color: #444;
padding:10px;
}
.btn-default {
margin-bottom: 10px;
}
/* algolia search */
.search {
text-align: left;
}
.search input {
font-size: 20px;
width: 300px;
}
.results {
margin: auto;
text-align: left;
}
.results ul {
list-style-type: none;
padding: 0;
}
/* algolia */
div.results {
position: absolute;
background-color: white;
width: 100%;
}
.post-meta {
font-size: 14px;
color: #828282;
}
.post-link {
font-size: 22px;
}
.post-list p {
margin: 10px 0;
}
time {
margin-right: 10px;
}
p.post-meta time {
margin-right: 0;
}
span.label.label-default {
background-color: gray;
}
span.label.label-primary {
background-color: #f0ad4e;
}
.col-lg-12 .nav li a {background-color: white}
a code {
color: ##2156a5;
}
table th code {
color: white;
}
ol li ul li ol li {
list-style: decimal;
}
ol li ul li ol li ul li{
list-style: disc;
}
.box {
padding: 10px;
border: 1px solid #888;
box-shadow: 2px 2px 4px #dedede;
width: 100px;
height: 80px;
background-color: #f5f5f5;
font-family: Arial;
font-size: 12px;
hyphens: auto;
float: left;
}
.box:hover {
background-color: #f0f0f0;
}
#userMap {
overflow-x: auto;
overflow-y: auto;
padding: 20px;
min-width: 770px;
}
#userMap .active {
background-color: #d6f5d6;
border:1px solid #555;
font-weight: bold;
}
h2.userMapTitle {
font-family: Arial;
}
#userMap a:hover {
text-decoration: none;
}
div.arrow {
max-width: 50px;
margin-left: 15px;
margin-right: 15px;
font-size: 20px;
}
#userMap div.arrow, #userMap div.content {
float: left;
}
.clearfix {
clear: both;
}
#userMap div.arrow {
position: relative;
top: 30px;
}
.box1 {
margin-left:0;
}
button.btn.btn-default.btn-lg.modalButton1 {
margin-left: -20px;
}
div.box.box1 {
margin-left: -20px;
}
#userMap .btn-lg {
width: 100px;
height: 80px;
}
#userMap .complexArrow {
font-size: 22px;
margin: 0 10px;
}
#userMap .btn-lg .active {
background-color: #d6f5d6;
}
#userMap .btn-lg {
white-space: pre-wrap; /* css-3 */
white-space: -moz-pre-wrap; /* Mozilla, since 1999 */
white-space: -pre-wrap; /* Opera 4-6 */
white-space: -o-pre-wrap; /* Opera 7 */
word-wrap: break-word; /* Internet Explorer 5.5+ */
font-size: 14px;
}
/*
* Let's target IE to respect aspect ratios and sizes for img tags containing SVG files
*
* [1] IE9
* [2] IE10+
*/
/* 1 */
.ie9 img[src$=".svg"] {
width: 100%;
}
/* 2 */
@media screen and (-ms-high-contrast: active), (-ms-high-contrast: none) {
img[src$=".svg"] {
width: 100%;
}
}

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,160 @@
/*body.print .container {max-width: 650px;}*/
body {
font-size:14px;
}
.nav ul li a {border-top:0px; background-color:transparent; color: #808080; }
#navig a[href] {color: #595959 !important;}
table .table {max-width:650px;}
#navig li.sectionHead {font-weight: bold; font-size: 18px; color: #595959 !important; }
#navig li {font-weight: normal; }
#navig a[href]::after { content: leader(".") target-counter(attr(href), page); }
a[href]::after {
content: " (page " target-counter(attr(href), page) ")"
}
a[href^="http:"]::after, a[href^="https:"]::after {
content: " (" attr(href) ")";
}
a[href] {
color: blue !important;
}
a[href*="mailto"]::after, a[data-toggle="tooltip"]::after, a[href].noCrossRef::after {
content: "";
}
@page {
margin: 60pt 90pt 60pt 90pt;
font-family: sans-serif;
font-style:none;
color: gray;
}
.printTitle {
line-height:30pt;
font-size:27pt;
font-weight: bold;
letter-spacing: -.5px;
margin-bottom:25px;
}
.printSubtitle {
font-size: 19pt;
color: #cccccc !important;
front-family: "Grotesque MT Light";
line-height: 22pt;
letter-spacing: -.5px;
margin-bottom:20px;
}
.printTitleArea hr {
color: #999999 !important;
height: 2px;
width: 100%;
}
.printTitleImage {
max-width:300px;
margin-bottom:200px;
}
.printTitleImage {
max-width: 250px;
}
#navig {
/*page-break-before: always;*/
}
.copyrightBoilerplate {
page-break-before:always;
font-size:14px;
}
.lastGeneratedDate {
font-style: italic;
font-size:14px;
color: gray;
}
.alert a {
text-decoration: none !important;
}
body.title { page: title }
@page title {
@top-left {
content: " ";
}
@top-right {
content: " "
}
@bottom-right {
content: " ";
}
@bottom-left {
content: " ";
}
}
body.frontmatter { page: frontmatter }
body.frontmatter {counter-reset: page 1}
@page frontmatter {
@top-left {
content: prince-script(guideName);
}
@top-right {
content: prince-script(datestamp);
}
@bottom-right {
content: counter(page, lower-roman);
}
@bottom-left {
content: "youremail@domain.com"; }
}
body.first_page {counter-reset: page 1}
h1 { string-set: doctitle content() }
@page {
@top-left {
content: string(doctitle);
font-size: 11px;
font-style: italic;
}
@top-right {
content: prince-script(datestamp);
font-size: 11px;
}
@bottom-right {
content: "Page " counter(page);
font-size: 11px;
}
@bottom-left {
content: prince-script(guideName);
font-size: 11px;
}
}
.alert {
background-color: #fafafa !important;
border-color: #dedede !important;
color: black;
}
pre {
background-color: #fafafa;
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,127 @@
.summary {
color: #808080;
border-left: 5px solid #d9411e;
font-size:16px;
}
.nav-tabs > li.active > a, .nav-tabs > li.active > a:hover, .nav-tabs > li.active > a:focus {
background-color: #FF833D;
color: white;
}
.nav > li.active > a {
background-color: #262130;
}
.nav > li > a:hover {
background-color: #FF833D;
}
div.navbar-collapse .dropdown-menu > li > a:hover {
background-color: #D9411E;
}
.nav li.thirdlevel > a {
background-color: #FAFAFA !important;
color: #FF833D;
font-weight: bold;
}
a[data-toggle="tooltip"] {
color: #649345;
font-style: italic;
cursor: default;
}
.navbar-inverse {
background-color: #D9411E;
border-color: #015CAE;
}
.navbar-inverse .navbar-nav > .open > a, .navbar-inverse .navbar-nav > .open > a:hover, .navbar-inverse .navbar-nav > .open > a:focus {
color: #015CAE;
}
.navbar-inverse .navbar-nav > .open > a, .navbar-inverse .navbar-nav > .open > a:hover, .navbar-inverse .navbar-nav > .open > a:focus {
background-color: #015CAE;
color: #ffffff;
}
/* not sure if using this ...*/
.navbar-inverse .navbar-collapse, .navbar-inverse .navbar-form {
border-color: #248ec2 !important;
}
/* Used for nav buttons */
.btn-primary {
color: #ffffff;
background-color: #0A7C38;
border-color: #E6E7E8;
}
.navbar-inverse .navbar-nav > .active > a, .navbar-inverse .navbar-nav > .active > a:hover, .navbar-inverse .navbar-nav > .active > a:focus {
background-color: #D9411E;
}
/* Used for nav buttons */
.btn-primary:hover,
.btn-primary:focus,
.btn-primary:active,
.btn-primary.active,
.open .dropdown-toggle.btn-primary {
background-color: #305CB3;
border-color: #E6E7E8;
}
.printTitle {
color: #015CAE !important;
}
body.print h1 {color: #015CAE !important; font-size:28px !important;}
body.print h2 {color: #595959 !important; font-size:20px !important;}
body.print h3 {color: #E50E51 !important; font-size:14px !important;}
body.print h4 {color: #679DCE !important; font-size:14px; font-style: italic !important;}
.anchorjs-link:hover {
color: #216f9b;
}
div.sidebarTitle {
color: #015CAE;
}
li.sidebarTitle {
margin-top:40px;
font-weight:normal;
font-size:120%;
color: #262130;
margin-bottom:10px;
margin-left: 5px;
}
.scrollnav {
font-size: larger;
/* why do we need both of these??? */
padding-bottom: 1em;
margin-bottom: 2em;
}
.scrollnav .prev {
text-align: left;
float: left;
font-size: inherit;
}
.scrollnav .prev:before {
padding-right: 0.5em;
content: "\25C0 ";
display: inline-block; /* text-decoration: none doesn't work, but this does */
}
.scrollnav .next {
text-align: right;
float: right;
font-size: inherit;
}
.scrollnav .next:after {
padding-left: 0.5em;
content: " \25B6";
display: inline-block; /* text-decoration: none doesn't work, but this does */
}

View File

@ -0,0 +1,45 @@
= DataDir and DirectoryFactory in SolrConfig
:page-shortname: datadir-and-directoryfactory-in-solrconfig
:page-permalink: datadir-and-directoryfactory-in-solrconfig.html
Where and how Solr stores its indexes are configurable options.
== Specifying a Location for Index Data with the `dataDir` Parameter
By default, Solr stores its index data in a directory called `/data` under the core's instance directory (`instanceDir`). If you would like to specify a different directory for storing index data, you can configure the `dataDir` in the `core.properties` file for the core, or use the `<dataDir>` parameter in the `solrconfig.xml` file. You can specify another directory either with an absolute path or a pathname relative to the instanceDir of the SolrCore. For example:
[source,xml]
----
<dataDir>/solr/data/${solr.core.name}</dataDir>
----
The `${solr.core.name}` substitution will cause the name of the current core to be substituted, which results in each core's data being kept in a separate subdirectory.
If you are using replication to replicate the Solr index (as described in <<legacy-scaling-and-distribution.adoc#legacy-scaling-and-distribution,Legacy Scaling and Distribution>>), then the `<dataDir>` directory should correspond to the index directory used in the replication configuration.
[[DataDirandDirectoryFactoryinSolrConfig-SpecifyingtheDirectoryFactoryForYourIndex]]
== Specifying the DirectoryFactory For Your Index
The default {solr-javadocs}/solr-core/org/apache/solr/core/StandardDirectoryFactory.html[`solr.StandardDirectoryFactory`] is filesystem based, and tries to pick the best implementation for the current JVM and platform. You can force a particular implementation and/or config options by specifying {solr-javadocs}/solr-core/org/apache/solr/core/MMapDirectoryFactory.html[`solr.MMapDirectoryFactory`], {solr-javadocs}/solr-core/org/apache/solr/core/NIOFSDirectoryFactory.html[`solr.NIOFSDirectoryFactory`], or {solr-javadocs}/solr-core/org/apache/solr/core/SimpleFSDirectoryFactory.html[`solr.SimpleFSDirectoryFactory`].
[source,xml]
----
<directoryFactory name="DirectoryFactory"
class="solr.MMapDirectoryFactory">
<bool name="preload">true</bool>
</directoryFactory>
----
The {solr-javadocs}/solr-core/org/apache/solr/core/RAMDirectoryFactory.html[`solr.RAMDirectoryFactory`] is memory based, not persistent, and does not work with replication. Use this DirectoryFactory to store your index in RAM.
[source,xml]
----
<directoryFactory class="org.apache.solr.core.RAMDirectoryFactory"/>
----
[NOTE]
====
If you are using Hadoop and would like to store your indexes in HDFS, you should use the {solr-javadocs}/solr-core/org/apache/solr/core/HdfsDirectoryFactory.html[`solr.HdfsDirectoryFactory`] instead of either of the above implementations. For more details, see the section <<running-solr-on-hdfs.adoc#running-solr-on-hdfs,Running Solr on HDFS>>.
====

View File

@ -0,0 +1,13 @@
= Dataimport Screen
:page-shortname: dataimport-screen
:page-permalink: dataimport-screen.html
The Dataimport screen shows the configuration of the DataImportHandler (DIH) and allows you start, and monitor the status of, import commands as defined by the options selected on the screen and defined in the configuration file.
.The Dataimport Screen
image::images/dataimport-screen/dataimport.png[image,width=485,height=250]
This screen also lets you adjust various options to control how the data is imported to Solr, and view the data import configuration file that controls the import.
For more information about data importing with DIH, see the section on <<uploading-structured-data-store-data-with-the-data-import-handler.adoc#uploading-structured-data-store-data-with-the-data-import-handler,Uploading Structured Data Store Data with the Data Import Handler>>.

View File

@ -0,0 +1,100 @@
= De-Duplication
:page-shortname: de-duplication
:page-permalink: de-duplication.html
If duplicate, or near-duplicate documents are a concern in your index, de-duplication may be worth implementing.
Preventing duplicate or near duplicate documents from entering an index or tagging documents with a signature/fingerprint for duplicate field collapsing can be efficiently achieved with a low collision or fuzzy hash algorithm. Solr natively supports de-duplication techniques of this type via the `Signature` class and allows for the easy addition of new hash/signature implementations. A Signature can be implemented several ways:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="30,70",options="header"]
|===
|Method |Description
|MD5Signature |128-bit hash used for exact duplicate detection.
|Lookup3Signature |64-bit hash used for exact duplicate detection. This is much faster than MD5 and smaller to index.
|http://wiki.apache.org/solr/TextProfileSignature[TextProfileSignature] |Fuzzy hashing implementation from Apache Nutch for near duplicate detection. It's tunable but works best on longer text.
|===
Other, more sophisticated algorithms for fuzzy/near hashing can be added later.
[IMPORTANT]
====
Adding in the de-duplication process will change the `allowDups` setting so that it applies to an update term (with `signatureField` in this case) rather than the unique field Term.
Of course the `signatureField` could be the unique field, but generally you want the unique field to be unique. When a document is added, a signature will automatically be generated and attached to the document in the specified `signatureField`.
====
[[De-Duplication-ConfigurationOptions]]
== Configuration Options
There are two places in Solr to configure de-duplication: in `solrconfig.xml` and in `schema.xml`.
[[De-Duplication-Insolrconfig.xml]]
=== In `solrconfig.xml`
The `SignatureUpdateProcessorFactory` has to be registered in `solrconfig.xml` as part of an <<update-request-processors.adoc#update-request-processors,Update Request Processor Chain>>, as in this example:
[source,xml]
----
<updateRequestProcessorChain name="dedupe">
<processor class="solr.processor.SignatureUpdateProcessorFactory">
<bool name="enabled">true</bool>
<str name="signatureField">id</str>
<bool name="overwriteDupes">false</bool>
<str name="fields">name,features,cat</str>
<str name="signatureClass">solr.processor.Lookup3Signature</str>
</processor>
<processor class="solr.LogUpdateProcessorFactory" />
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
----
The `SignatureUpdateProcessorFactory` takes several properties:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,30,50",options="header"]
|===
|Parameter |Default |Description
|signatureClass |`org.apache.solr.update.processor.Lookup3Signature` a|
A Signature implementation for generating a signature hash. The full classpath of the implementation must be specified. The available options are described above, the associated classpaths to use are:
* `org.apache.solr.update.processor.Lookup3Signature`
* `org.apache.solr.update.processor.MD5Signature`
* `org.apache.solr.update.process.TextProfileSignature`
|fields |all fields |The fields to use to generate the signature hash in a comma separated list. By default, all fields on the document will be used.
|signatureField |signatureField |The name of the field used to hold the fingerprint/signature. The field should be defined in schema.xml.
|enabled |true |Enable/disable de-duplication processing.
|overwriteDupes |true |If true, when a document exists that already matches this signature, it will be overwritten.
|===
[[De-Duplication-Inschema.xml]]
=== In `schema.xml`
If you are using a separate field for storing the signature, you must have it indexed:
[source,xml]
----
<field name="signatureField" type="string" stored="true" indexed="true" multiValued="false" />
----
Be sure to change your update handlers to use the defined chain, as below:
[source,xml]
----
<requestHandler name="/update" class="solr.UpdateRequestHandler" >
<lst name="defaults">
<str name="update.chain">dedupe</str>
</lst>
...
</requestHandler>
----
This example assumes you have other sections of your request handler defined.
[TIP]
====
The update processor can also be specified per request with a parameter of `update.chain=dedupe`.
====

View File

@ -0,0 +1,79 @@
= Defining core.properties
:page-shortname: defining-core-properties
:page-permalink: defining-core-properties.html
Core discovery means that creating a core is as simple as a `core.properties` file located on disk.
The `core.properties` file is a simple Java Properties file where each line is just a key=value pair, e.g., `name=core1`. Notice that no quotes are required.
A minimal `core.properties` file looks like the example below. However, it can also be empty, see information on placement of `core.properties` below.
[source,bash]
----
name=my_core_name
----
[[Definingcore.properties-Placementofcore.properties]]
== Placement of core.properties
Solr cores are configured by placing a file named `core.properties` in a sub-directory under `solr.home`. There are no a-priori limits to the depth of the tree, nor are there limits to the number of cores that can be defined. Cores may be anywhere in the tree with the exception that cores may _not_ be defined under an existing core. That is, the following is not allowed:
[source,text]
----
./cores/core1/core.properties
./cores/core1/coremore/core5/core.properties
----
In this example, the enumeration will stop at "core1".
The following is legal:
[source,text]
----
./cores/somecores/core1/core.properties
./cores/somecores/core2/core.properties
./cores/othercores/core3/core.properties
./cores/extracores/deepertree/core4/core.properties
----
It is possible to segment Solr into multiple cores, each with its own configuration and indices. Cores may be dedicated to a single application or to very different ones, but all are administered through a common administration interface. You can create new Solr cores on the fly, shutdown cores, even replace one running core with another, all without ever stopping or restarting Solr.
Your `core.properties` file can be empty if necessary. Suppose `core.properties` is located in `./cores/core1` (relative to `solr_home`) but is empty. In this case, the core name is assumed to be "core1". The instanceDir will be the folder containing `core.properties` (i.e., `./cores/core1`). The dataDir will be `../cores/core1/data`, etc.
[NOTE]
====
You can run Solr without configuring any cores.
====
[[Definingcore.properties-Definingcore.propertiesFiles]]
== Defining core.properties Files
[[Definingcore.properties-core.properties_files]]
The minimal `core.properties` file is an empty file, in which case all of the properties are defaulted appropriately.
Java properties files allow the hash (`#`) or bang (`!`) characters to specify comment-to-end-of-line.
This table defines the recognized properties:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="25,75",options="header"]
|===
|Property |Description
|`name` |The name of the SolrCore. You'll use this name to reference the SolrCore when running commands with the CoreAdminHandler.
|`config` |The configuration file name for a given core. The default is `solrconfig.xml`.
|`schema` |The schema file name for a given core. The default is `schema.xml` but please note that if you are using a "managed schema" (the default behavior) then any value for this property which does not match the effective `managedSchemaResourceName` will be read once, backed up, and converted for managed schema use. See <<schema-factory-definition-in-solrconfig.adoc#schema-factory-definition-in-solrconfig,Schema Factory Definition in SolrConfig>> for details.
|`dataDir` |The core's data directory (where indexes are stored) as either an absolute pathname, or a path relative to the value of `instanceDir`. This is `data` by default.
|`configSet` |The name of a defined configset, if desired, to use to configure the core (see the <<config-sets.adoc#config-sets,Config Sets>> for more details).
|`properties` |The name of the properties file for this core. The value can be an absolute pathname or a path relative to the value of `instanceDir`.
|`transient` |If *true*, the core can be unloaded if Solr reaches the `transientCacheSize`. The default if not specified is *false*. Cores are unloaded in order of least recently used first. _Setting to *true* is not recommended in SolrCloud mode._
|`loadOnStartup` |If *true*, the default if it is not specified, the core will loaded when Solr starts. _Setting to *false* is not recommended in SolrCloud mode._
|`coreNodeName` |Used only in SolrCloud, this is a unique identifier for the node hosting this replica. By default a coreNodeName is generated automatically, but setting this attribute explicitly allows you to manually assign a new core to replace an existing replica. For example: when replacing a machine that has had a hardware failure by restoring from backups on a new machine with a new hostname or port..
|`ulogDir` |The absolute or relative directory for the update log for this core (SolrCloud).
|`shard` |The shard to assign this core to (SolrCloud).
|`collection` |The name of the collection this core is part of (SolrCloud).
|`roles` |Future param for SolrCloud or a way for users to mark nodes for their own use.
|===
Additional "user defined" properties may be specified for use as variables. For more information on how to define local properties, see the section <<configuring-solrconfig-xml.adoc#Configuringsolrconfig.xml-SubstitutingPropertiesinSolrConfigFiles,Substituting Properties in Solr Config Files>>.

View File

@ -0,0 +1,57 @@
= Defining Fields
:page-shortname: defining-fields
:page-permalink: defining-fields.html
Fields are defined in the fields element of `schema.xml`. Once you have the field types set up, defining the fields themselves is simple.
[[DefiningFields-Example]]
== Example
The following example defines a field named `price` with a type named `float` and a default value of `0.0`; the `indexed` and `stored` properties are explicitly set to `true`, while any other properties specified on the `float` field type are inherited.
[source,xml]
----
<field name="price" type="float" default="0.0" indexed="true" stored="true"/>
----
[[DefiningFields-FieldProperties]]
== Field Properties
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="30,70",options="header"]
|===
|Property |Description
|name |The name of the field. Field names should consist of alphanumeric or underscore characters only and not start with a digit. This is not currently strictly enforced, but other field names will not have first class support from all components and back compatibility is not guaranteed. Names with both leading and trailing underscores (e.g., `\_version_`) are reserved. Every field must have a `name`.
|type |The name of the `fieldType` for this field. This will be found in the `name` attribute on the `fieldType` definition. Every field must have a `type`.
|default |A default value that will be added automatically to any document that does not have a value in this field when it is indexed. If this property is not specified, there is no default.
|===
[[DefiningFields-OptionalFieldTypeOverrideProperties]]
== Optional Field Type Override Properties
Fields can have many of the same properties as field types. Properties from the table below which are specified on an individual field will override any explicit value for that property specified on the the `fieldType` of the field, or any implicit default property value provided by the underlying `fieldType` implementation. The table below is reproduced from <<field-type-definitions-and-properties.adoc#field-type-definitions-and-properties,Field Type Definitions and Properties>>, which has more details:
// TODO: SOLR-10655 BEGIN: refactor this into a 'field-default-properties.include.adoc' file for reuse
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,40,20,20",options="header"]
|===
|Property |Description |Values |Implicit Default
|indexed |If true, the value of the field can be used in queries to retrieve matching documents. |true or false |true
|stored |If true, the actual value of the field can be retrieved by queries. |true or false |true
|docValues |If true, the value of the field will be put in a column-oriented <<docvalues.adoc#docvalues,DocValues>> structure. |true or false |false
|sortMissingFirst sortMissingLast |Control the placement of documents when a sort field is not present. |true or false |false
|multiValued |If true, indicates that a single document might contain multiple values for this field type. |true or false |false
|omitNorms |If true, omits the norms associated with this field (this disables length normalization for the field, and saves some memory). *Defaults to true for all primitive (non-analyzed) field types, such as int, float, data, bool, and string.* Only full-text fields or fields need norms. |true or false |*
|omitTermFreqAndPositions |If true, omits term frequency, positions, and payloads from postings for this field. This can be a performance boost for fields that don't require that information. It also reduces the storage space required for the index. Queries that rely on position that are issued on a field with this option will silently fail to find documents. *This property defaults to true for all field types that are not text fields.* |true or false |*
|omitPositions |Similar to `omitTermFreqAndPositions` but preserves term frequency information. |true or false |*
|termVectors termPositions termOffsets termPayloads |These options instruct Solr to maintain full term vectors for each document, optionally including position, offset and payload information for each term occurrence in those vectors. These can be used to accelerate highlighting and other ancillary functionality, but impose a substantial cost in terms of index size. They are not necessary for typical uses of Solr. |true or false |false
|required |Instructs Solr to reject any attempts to add a document which does not have a value for this field. This property defaults to false. |true or false |false
|useDocValuesAsStored |If the field has `<<docvalues.adoc#docvalues,docValues>>` enabled, setting this to true would allow the field to be returned as if it were a stored field (even if it has `stored=false`) when matching "`*`" in an <<common-query-parameters.adoc#CommonQueryParameters-Thefl_FieldList_Parameter,fl parameter>>. |true or false |true
|large |Large fields are always lazy loaded and will only take up space in the document cache if the actual value is < 512KB. This option requires `stored="true"` and `multiValued="false"`. It's intended for fields that might have very large values so that they don't get cached in memory. |true or false |false
|===
// TODO: SOLR-10655 END

View File

@ -0,0 +1,82 @@
= Detecting Languages During Indexing
:page-shortname: detecting-languages-during-indexing
:page-permalink: detecting-languages-during-indexing.html
Solr can identify languages and map text to language-specific fields during indexing using the `langid` UpdateRequestProcessor.
Solr supports two implementations of this feature:
* Tika's language detection feature: http://tika.apache.org/0.10/detection.html
* LangDetect language detection: https://github.com/shuyo/language-detection
You can see a comparison between the two implementations here: http://blog.mikemccandless.com/2011/10/accuracy-and-performance-of-googles.html. In general, the LangDetect implementation supports more languages with higher performance.
For specific information on each of these language identification implementations, including a list of supported languages for each, see the relevant project websites.
For more information about language analysis in Solr, see <<language-analysis.adoc#language-analysis,Language Analysis>>.
[[DetectingLanguagesDuringIndexing-ConfiguringLanguageDetection]]
== Configuring Language Detection
You can configure the `langid` UpdateRequestProcessor in `solrconfig.xml`. Both implementations take the same parameters, which are described in the following section. At a minimum, you must specify the fields for language identification and a field for the resulting language code.
[[DetectingLanguagesDuringIndexing-ConfiguringTikaLanguageDetection]]
=== Configuring Tika Language Detection
Here is an example of a minimal Tika `langid` configuration in `solrconfig.xml`:
[source,xml]
----
<processor class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
<lst name="defaults">
<str name="langid.fl">title,subject,text,keywords</str>
<str name="langid.langField">language_s</str>
</lst>
</processor>
----
[[DetectingLanguagesDuringIndexing-ConfiguringLangDetectLanguageDetection]]
=== Configuring LangDetect Language Detection
Here is an example of a minimal LangDetect `langid` configuration in `solrconfig.xml`:
[source,xml]
----
<processor class="org.apache.solr.update.processor.LangDetectLanguageIdentifierUpdateProcessorFactory">
<lst name="defaults">
<str name="langid.fl">title,subject,text,keywords</str>
<str name="langid.langField">language_s</str>
</lst>
</processor>
----
[[DetectingLanguagesDuringIndexing-langidParameters]]
== `langid` Parameters
As previously mentioned, both implementations of the `langid` UpdateRequestProcessor take the same parameters.
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,10,10,10,50",options="header"]
|===
|Parameter |Type |Default |Required |Description
|langid |Boolean |true |no |Enables and disables language detection.
|langid.fl |string |none |yes |A comma- or space-delimited list of fields to be processed by `langid`.
|langid.langField |string |none |yes |Specifies the field for the returned language code.
|langid.langsField |multivalued string |none |no |Specifies the field for a list of returned language codes. If you use `langid.map.individual`, each detected language will be added to this field.
|langid.overwrite |Boolean |false |no |Specifies whether the content of the `langField` and `langsField` fields will be overwritten if they already contain values.
|langid.lcmap |string |none |false |A space-separated list specifying colon delimited language code mappings to apply to the detected languages. For example, you might use this to map Chinese, Japanese, and Korean to a common `cjk` code, and map both American and British English to a single `en` code by using `langid.lcmap=ja:cjk zh:cjk ko:cjk en_GB:en en_US:en`. This affects both the values put into the `langField` and `langsField` fields, as well as the field suffixes when using `langid.map`, unless overridden by `langid.map.lcmap`
|langid.threshold |float |0.5 |no |Specifies a threshold value between 0 and 1 that the language identification score must reach before `langid` accepts it. With longer text fields, a high threshold such at 0.8 will give good results. For shorter text fields, you may need to lower the threshold for language identification, though you will be risking somewhat lower quality results. We recommend experimenting with your data to tune your results.
|langid.whitelist |string |none |no |Specifies a list of allowed language identification codes. Use this in combination with `langid.map` to ensure that you only index documents into fields that are in your schema.
|langid.map |Boolean |false |no |Enables field name mapping. If true, Solr will map field names for all fields listed in `langid.fl`.
|langid.map.fl |string |none |no |A comma-separated list of fields for `langid.map` that is different than the fields specified in `langid.fl`.
|langid.map.keepOrig |Boolean |false |no |If true, Solr will copy the field during the field name mapping process, leaving the original field in place.
|langid.map.individual |Boolean |false |no |If true, Solr will detect and map languages for each field individually.
|langid.map.individual.fl |string |none |no |A comma-separated list of fields for use with `langid.map.individual` that is different than the fields specified in `langid.fl`.
|langid.fallbackFields |string |none |no |If no language is detected that meets the `langid.threshold` score, or if the detected language is not on the `langid.whitelist`, this field specifies language codes to be used as fallback values. If no appropriate fallback languages are found, Solr will use the language code specified in `langid.fallback`.
|langid.fallback |string |none |no |Specifies a language code to use if no language is detected or specified in `langid.fallbackFields`.
|langid.map.lcmap |string |determined by `langid.lcmap` |no |A space-separated list specifying colon delimited language code mappings to use when mapping field names. For example, you might use this to make Chinese, Japanese, and Korean language fields use a common `*_cjk` suffix, and map both American and British English fields to a single `*_en` by using `langid.map.lcmap=ja:cjk zh:cjk ko:cjk en_GB:en en_US:en`.
|langid.map.pattern |Java regular expression |none |no |By default, fields are mapped as <field>_<language>. To change this pattern, you can specify a Java regular expression in this parameter.
|langid.map.replace |Java replace |none |no |By default, fields are mapped as <field>_<language>. To change this pattern, you can specify a Java replace in this parameter.
|langid.enforceSchema |Boolean |true |no |If false, the `langid` processor does not validate field names against your schema. This may be useful if you plan to rename or delete fields later in the UpdateChain.
|===

View File

@ -0,0 +1,123 @@
= Distributed Requests
:page-shortname: distributed-requests
:page-permalink: distributed-requests.html
When a Solr node receives a search request, the request is routed behind the scenes to a replica of a shard that is part of the collection being searched.
The chosen replica acts as an aggregator: it creates internal requests to randomly chosen replicas of every shard in the collection, coordinates the responses, issues any subsequent internal requests as needed (for example, to refine facets values, or request additional stored fields), and constructs the final response for the client.
[[DistributedRequests-LimitingWhichShardsareQueried]]
== Limiting Which Shards are Queried
While one of the advantages of using SolrCloud is the ability to query very large collections distributed among various shards, in some cases <<shards-and-indexing-data-in-solrcloud.adoc#ShardsandIndexingDatainSolrCloud-DocumentRouting,you may know that you are only interested in results from a subset of your shards>>. You have the option of searching over all of your data or just parts of it.
Querying all shards for a collection should look familiar; it's as though SolrCloud didn't even come into play:
[source,text]
----
http://localhost:8983/solr/gettingstarted/select?q=*:*
----
If, on the other hand, you wanted to search just one shard, you can specify that shard by its logical ID, as in:
[source,text]
----
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=shard1
----
If you want to search a group of shard Ids, you can specify them together:
[source,text]
----
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=shard1,shard2
----
In both of the above examples, the shard Id(s) will be used to pick a random replica of that shard.
Alternatively, you can specify the explicit replicas you wish to use in place of a shard Ids:
[source,text]
----
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=localhost:7574/solr/gettingstarted,localhost:8983/solr/gettingstarted
----
Or you can specify a list of replicas to choose from for a single shard (for load balancing purposes) by using the pipe symbol (|):
[source,text]
----
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=localhost:7574/solr/gettingstarted|localhost:7500/solr/gettingstarted
----
And of course, you can specify a list of shards (seperated by commas) each defined by a list of replicas (seperated by pipes). In this example, 2 shards are queried, the first being a random replica from shard1, the second being a random replica from the explicit pipe delimited list:
[source,text]
----
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=shard1,localhost:7574/solr/gettingstarted|localhost:7500/solr/gettingstarted
----
[[DistributedRequests-ConfiguringtheShardHandlerFactory]]
== Configuring the ShardHandlerFactory
You can directly configure aspects of the concurrency and thread-pooling used within distributed search in Solr. This allows for finer grained control and you can tune it to target your own specific requirements. The default configuration favors throughput over latency.
To configure the standard handler, provide a configuration like this in `solrconfig.xml`:
[source,xml]
----
<requestHandler name="standard" class="solr.SearchHandler" default="true">
<!-- other params go here -->
<shardHandler class="HttpShardHandlerFactory">
<int name="socketTimeOut">1000</int>
<int name="connTimeOut">5000</int>
</shardHandler>
</requestHandler>
----
The parameters that can be specified are as follows:
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
[cols="20,15,65",options="header"]
|===
|Parameter |Default |Explanation
|`socketTimeout` |0 (use OS default) |The amount of time in ms that a socket is allowed to wait.
|`connTimeout` |0 (use OS default) |The amount of time in ms that is accepted for binding / connecting a socket
|`maxConnectionsPerHost` |20 |The maximum number of concurrent connections that is made to each individual shard in a distributed search.
|`maxConnections` |`10000` |The total maximum number of concurrent connections in distributed searches.
|`corePoolSize` |0 |The retained lowest limit on the number of threads used in coordinating distributed search.
|`maximumPoolSize` |Integer.MAX_VALUE |The maximum number of threads used for coordinating distributed search.
|`maxThreadIdleTime` |5 seconds |The amount of time to wait for before threads are scaled back in response to a reduction in load.
|`sizeOfQueue` |-1 |If specified, the thread pool will use a backing queue instead of a direct handoff buffer. High throughput systems will want to configure this to be a direct hand off (with -1). Systems that desire better latency will want to configure a reasonable size of queue to handle variations in requests.
|`fairnessPolicy` |false |Chooses the JVM specifics dealing with fair policy queuing, if enabled distributed searches will be handled in a First in First out fashion at a cost to throughput. If disabled throughput will be favored over latency.
|===
[[DistributedRequests-ConfiguringstatsCache_DistributedIDF_]]
== Configuring statsCache (Distributed IDF)
Document and term statistics are needed in order to calculate relevancy. Solr provides four implementations out of the box when it comes to document stats calculation:
* `LocalStatsCache`: This only uses local term and document statistics to compute relevance. In cases with uniform term distribution across shards, this works reasonably well.This option is the default if no `<statsCache>` is configured.
* `ExactStatsCache`: This implementation uses global values (across the collection) for document frequency.
* `ExactSharedStatsCache`: This is exactly like the exact stats cache in its functionality but the global stats are reused for subsequent requests with the same terms.
* `LRUStatsCache`: This implementation uses an LRU cache to hold global stats, which are shared between requests.
The implementation can be selected by setting `<statsCache>` in `solrconfig.xml`. For example, the following line makes Solr use the `ExactStatsCache` implementation:
[source,xml]
----
<statsCache class="org.apache.solr.search.stats.ExactStatsCache"/>
----
[[DistributedRequests-AvoidingDistributedDeadlock]]
== Avoiding Distributed Deadlock
Each shard serves top-level query requests and then makes sub-requests to all of the other shards. Care should be taken to ensure that the max number of threads serving HTTP requests is greater than the possible number of requests from both top-level clients and other shards. If this is not the case, the configuration may result in a distributed deadlock.
For example, a deadlock might occur in the case of two shards, each with just a single thread to service HTTP requests. Both threads could receive a top-level request concurrently, and make sub-requests to each other. Because there are no more remaining threads to service requests, the incoming requests will be blocked until the other pending requests are finished, but they will not finish since they are waiting for the sub-requests. By ensuring that Solr is configured to handle a sufficient number of threads, you can avoid deadlock situations like this.
[[DistributedRequests-PreferLocalShards]]
== Prefer Local Shards
Solr allows you to pass an optional boolean parameter named `preferLocalShards` to indicate that a distributed query should prefer local replicas of a shard when available. In other words, if a query includes `preferLocalShards=true`, then the query controller will look for local replicas to service the query instead of selecting replicas at random from across the cluster. This is useful when a query requests many fields or large fields to be returned per document because it avoids moving large amounts of data over the network when it is available locally. In addition, this feature can be useful for minimizing the impact of a problematic replica with degraded performance, as it reduces the likelihood that the degraded replica will be hit by other healthy replicas.
Lastly, it follows that the value of this feature diminishes as the number of shards in a collection increases because the query controller will have to direct the query to non-local replicas for most of the shards. In other words, this feature is mostly useful for optimizing queries directed towards collections with a small number of shards and many replicas. Also, this option should only be used if you are load balancing requests across all nodes that host replicas for the collection you are querying, as Solr's CloudSolrClient will do. If not load-balancing, this feature can introduce a hotspot in the cluster since queries won't be evenly distributed across the cluster.

View File

@ -0,0 +1,165 @@
= Distributed Search with Index Sharding
:page-shortname: distributed-search-with-index-sharding
:page-permalink: distributed-search-with-index-sharding.html
When using traditional index sharding, you will need to consider how to query your documents.
It is highly recommended that you use <<solrcloud.adoc#solrcloud,SolrCloud>> when needing to scale up or scale out. The setup described below is legacy and was used prior to the existence of SolrCloud. SolrCloud provides for a truly distributed set of features with support for things like automatic routing, leader election, optimistic concurrency and other sanity checks that are expected out of a distributed system.
Everything on this page is specific to legacy setup of distributed search. Users trying out SolrCloud should not follow any of the steps or information below.
Update reorders (i.e., replica A may see update X then Y, and replica B may see update Y then X). *deleteByQuery* also handles reorders the same way, to ensure replicas are consistent. All replicas of a shard are consistent, even if the updates arrive in a different order on different replicas.
[[DistributedSearchwithIndexSharding-DistributingDocumentsacrossShards]]
== Distributing Documents across Shards
When not using SolrCloud, it is up to you to get all your documents indexed on each shard of your server farm. Solr supports distributed indexing (routing) in its true form only in the SolrCloud mode.
In the legacy distributed mode, Solr does not calculate universal term/doc frequencies. For most large-scale implementations, it is not likely to matter that Solr calculates TF/IDF at the shard level. However, if your collection is heavily skewed in its distribution across servers, you may find misleading relevancy results in your searches. In general, it is probably best to randomly distribute documents to your shards.
[[DistributedSearchwithIndexSharding-ExecutingDistributedSearcheswiththeshardsParameter]]
== Executing Distributed Searches with the `shards` Parameter
If a query request includes the `shards` parameter, the Solr server distributes the request across all the shards listed as arguments to the parameter. The `shards` parameter uses this syntax:
`host:port/base_url,host:port/base_url*`
For example, the `shards` parameter below causes the search to be distributed across two Solr servers: *solr1* and **solr2**, both of which are running on port 8983:
`\http://localhost:8983/solr/core1/select?shards=solr1:8983/solr/core1,solr2:8983/solr/core1&indent=true&q=ipod+solr`
Rather than require users to include the shards parameter explicitly, it is usually preferred to configure this parameter as a default in the RequestHandler section of `solrconfig.xml`.
[IMPORTANT]
====
Do not add the `shards` parameter to the standard request handler; doing so may cause search queries may enter an infinite loop. Instead, define a new request handler that uses the `shards` parameter, and pass distributed search requests to that handler.
====
With Legacy mode, only query requests are distributed. This includes requests to the SearchHandler (or any handler extending from `org.apache.solr.handler.component.SearchHandler`) using standard components that support distributed search.
As in SolrCloud mode, when `shards.info=true`, distributed responses will include information about the shard (where each shard represents a logically different index or physical location)
The following components support distributed search:
* The *Query* component, which returns documents matching a query
* The *Facet* component, which processes facet.query and facet.field requests where facets are sorted by count (the default).
* The *Highlighting* component, which enables Solr to include "highlighted" matches in field values.
* The *Stats* component, which returns simple statistics for numeric fields within the DocSet.
* The *Debug* component, which helps with debugging.
[[DistributedSearchwithIndexSharding-LimitationstoDistributedSearch]]
== Limitations to Distributed Search
Distributed searching in Solr has the following limitations:
* Each document indexed must have a unique key.
* If Solr discovers duplicate document IDs, Solr selects the first document and discards subsequent ones.
* The index for distributed searching may become momentarily out of sync if a commit happens between the first and second phase of the distributed search. This might cause a situation where a document that once matched a query and was subsequently changed may no longer match the query but will still be retrieved. This situation is expected to be quite rare, however, and is only possible for a single query request.
* The number of shards is limited by number of characters allowed for GET method's URI; most Web servers generally support at least 4000 characters, but many servers limit URI length to reduce their vulnerability to Denial of Service (DoS) attacks.
* Shard information can be returned with each document in a distributed search by including `fl=id, [shard]` in the search request. This returns the shard URL.
* In a distributed search, the data directory from the core descriptor overrides any data directory in `solrconfig.xml.`
* Update commands may be sent to any server with distributed indexing configured correctly. Document adds and deletes are forwarded to the appropriate server/shard based on a hash of the unique document id. *commit* commands and *deleteByQuery* commands are sent to every server in `shards`.
Formerly a limitation was that TF/IDF relevancy computations only used shard-local statistics. This is still the case by default. If your data isn't randomly distributed, or if you want more exact statistics, then remember to configure the ExactStatsCache.
[[DistributedSearchwithIndexSharding-AvoidingDistributedDeadlock]]
== Avoiding Distributed Deadlock
Like in SolrCloud mode, inter-shard requests could lead to a distributed deadlock. It can be avoided by following the instructions in the section <<distributed-requests.adoc#distributed-requests,Distributed Requests>>.
[[DistributedSearchwithIndexSharding-TestingIndexShardingonTwoLocalServers]]
== Testing Index Sharding on Two Local Servers
For simple functional testing, it's easiest to just set up two local Solr servers on different ports. (In a production environment, of course, these servers would be deployed on separate machines.)
. Make two Solr home directories and copy `solr.xml` into the new directories:
+
[source,bash]
----
mkdir example/nodes
mkdir example/nodes/node1
# Copy solr.xml into this solr.home
cp server/solr/solr.xml example/nodes/node1/.
# Repeat the above steps for the second node
mkdir example/nodes/node2
cp server/solr/solr.xml example/nodes/node2/.
----
. Start the two Solr instances
+
[source,bash]
----
# Start first node on port 8983
bin/solr start -s example/nodes/node1 -p 8983
# Start second node on port 8984
bin/solr start -s example/nodes/node2 -p 8984
----
. Create a core on both the nodes with the sample_techproducts_configs.
+
[source,bash]
----
bin/solr create_core -c core1 -p 8983 -d sample_techproducts_configs
# Create a core on the Solr node running on port 8984
bin/solr create_core -c core1 -p 8984 -d sample_techproducts_configs
----
. In a third window, index an example document to each of the server:
+
[source,bash]
----
bin/post -c core1 example/exampledocs/monitor.xml -port 8983
bin/post -c core1 example/exampledocs/monitor2.xml -port 8984
----
. Search on the node on port 8983:
+
[source,bash]
----
curl http://localhost:8983/solr/core1/select?q=*:*&wt=xml&indent=true
----
+
This should bring back one document.
+
Search on the node on port 8984:
+
[source,bash]
----
curl http://localhost:8984/solr/core1/select?q=*:*&wt=xml&indent=true
----
+
This should also bring back a single document.
+
Now do a distributed search across both servers with your browser or `curl.` In the example below, an extra parameter 'fl' is passed to restrict the returned fields to id and name.
+
[source,bash]
----
curl http://localhost:8983/solr/core1/select?q=*:*&indent=true&shards=localhost:8983/solr/core1,localhost:8984/solr/core1&fl=id,name
----
+
This should contain both the documents as shown below:
+
[source,xml]
----
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">8</int>
<lst name="params">
<str name="q">*:*</str>
<str name="shards">localhost:8983/solr/core1,localhost:8984/solr/core1</str>
<str name="indent">true</str>
<str name="fl">id,name</str>
<str name="wt">xml</str>
</lst>
</lst>
<result name="response" numFound="2" start="0" maxScore="1.0">
<doc>
<str name="id">3007WFP</str>
<str name="name">Dell Widescreen UltraSharp 3007WFP</str>
</doc>
<doc>
<str name="id">VA902B</str>
<str name="name">ViewSonic VA902B - flat panel display - TFT - 19"</str>
</doc>
</result>
</response>
----

View File

@ -0,0 +1,28 @@
= Documents, Fields, and Schema Design
:page-shortname: documents-fields-and-schema-design
:page-permalink: documents-fields-and-schema-design.html
:page-children: overview-of-documents-fields-and-schema-design, solr-field-types, defining-fields, copying-fields, dynamic-fields, other-schema-elements, schema-api, putting-the-pieces-together, docvalues, schemaless-mode
This section discusses how Solr organizes its data into documents and fields, as well as how to work with a schema in Solr.
This section includes the following topics:
<<overview-of-documents-fields-and-schema-design.adoc#overview-of-documents-fields-and-schema-design,Overview of Documents, Fields, and Schema Design>>: An introduction to the concepts covered in this section.
<<solr-field-types.adoc#solr-field-types,Solr Field Types>>: Detailed information about field types in Solr, including the field types in the default Solr schema.
<<defining-fields.adoc#defining-fields,Defining Fields>>: Describes how to define fields in Solr.
<<copying-fields.adoc#copying-fields,Copying Fields>>: Describes how to populate fields with data copied from another field.
<<dynamic-fields.adoc#dynamic-fields,Dynamic Fields>>: Information about using dynamic fields in order to catch and index fields that do not exactly conform to other field definitions in your schema.
<<schema-api.adoc#schema-api,Schema API>>: Use curl commands to read various parts of a schema or create new fields and copyField rules.
<<other-schema-elements.adoc#other-schema-elements,Other Schema Elements>>: Describes other important elements in the Solr schema.
<<putting-the-pieces-together.adoc#putting-the-pieces-together,Putting the Pieces Together>>: A higher-level view of the Solr schema and how its elements work together.
<<docvalues.adoc#docvalues,DocValues>>: Describes how to create a docValues index for faster lookups.
<<schemaless-mode.adoc#schemaless-mode,Schemaless Mode>>: Automatically add previously unknown schema fields using value-based field type guessing.

View File

@ -0,0 +1,73 @@
= Documents Screen
:page-shortname: documents-screen
:page-permalink: documents-screen.html
The Documents screen provides a simple form allowing you to execute various Solr indexing commands in a variety of formats directly from the browser.
.The Documents Screen
image::images/documents-screen/documents_add_screen.png[image,height=400]
The screen allows you to:
* Copy documents in JSON, CSV or XML and submit them to the index
* Upload documents (in JSON, CSV or XML)
* Construct documents by selecting fields and field values
[TIP]
====
There are other ways to load data, see also these sections:
* <<uploading-data-with-index-handlers.adoc#uploading-data-with-index-handlers,Uploading Data with Index Handlers>>
* <<uploading-data-with-solr-cell-using-apache-tika.adoc#uploading-data-with-solr-cell-using-apache-tika,Uploading Data with Solr Cell using Apache Tika>>
====
The first step is to define the RequestHandler to use (aka, 'qt'). By default `/update` will be defined. To use Solr Cell, for example, change the request handler to `/update/extract`.
Then choose the Document Type to define the type of document to load. The remaining parameters will change depending on the document type selected.
[[DocumentsScreen-JSON]]
== JSON
When using the JSON document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper JSON format.
Then you can choose when documents should be added to the index (Commit Within), & whether existing documents should be overwritten with incoming documents with the same id (if this is not *true*, then the incoming documents will be dropped).
This option will only add or overwrite documents to the index; for other update tasks, see the <<DocumentsScreen-SolrCommand,Solr Command>> option.
[[DocumentsScreen-CSV]]
== CSV
When using the CSV document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper CSV format, with columns delimited and one row per document.
Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not *true*, then the incoming documents will be dropped).
[[DocumentsScreen-DocumentBuilder]]
== Document Builder
The Document Builder provides a wizard-like interface to enter fields of a document
[[DocumentsScreen-FileUpload]]
== File Upload
The File Upload option allows choosing a prepared file and uploading it. If using only `/update` for the Request-Handler option, you will be limited to XML, CSV, and JSON.
However, to use the ExtractingRequestHandler (aka Solr Cell), you can modify the Request-Handler to `/update/extract`. You must have this defined in your `solrconfig.xml` file, with your desired defaults. You should also update the `&literal.id` shown in the Extracting Req. Handler Params so the file chosen is given a unique id.
Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not *true*, then the incoming documents will be dropped).
[[DocumentsScreen-SolrCommand]]
== Solr Command
The Solr Command option allows you use XML or JSON to perform specific actions on documents, such as defining documents to be added or deleted, updating only certain fields of documents, or commit and optimize commands on the index.
The documents should be structured as they would be if using `/update` on the command line.
[[DocumentsScreen-XML]]
== XML
When using the XML document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper Solr XML format, with each document separated by `<doc>` tags and each field defined.
Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not **true**, then the incoming documents will be dropped).
This option will only add or overwrite documents to the index; for other update tasks, see the <<DocumentsScreen-SolrCommand,Solr Command>> option.

View File

@ -0,0 +1,75 @@
= DocValues
:page-shortname: docvalues
:page-permalink: docvalues.html
DocValues are a way of recording field values internally that is more efficient for some purposes, such as sorting and faceting, than traditional indexing.
== Why DocValues?
The standard way that Solr builds the index is with an _inverted index_. This style builds a list of terms found in all the documents in the index and next to each term is a list of documents that the term appears in (as well as how many times the term appears in that document). This makes search very fast - since users search by terms, having a ready list of term-to-document values makes the query process faster.
For other features that we now commonly associate with search, such as sorting, faceting, and highlighting, this approach is not very efficient. The faceting engine, for example, must look up each term that appears in each document that will make up the result set and pull the document IDs in order to build the facet list. In Solr, this is maintained in memory, and can be slow to load (depending on the number of documents, terms, etc.).
In Lucene 4.0, a new approach was introduced. DocValue fields are now column-oriented fields with a document-to-value mapping built at index time. This approach promises to relieve some of the memory requirements of the fieldCache and make lookups for faceting, sorting, and grouping much faster.
[[DocValues-EnablingDocValues]]
== Enabling DocValues
To use docValues, you only need to enable it for a field that you will use it with. As with all schema design, you need to define a field type and then define fields of that type with docValues enabled. All of these actions are done in `schema.xml`.
Enabling a field for docValues only requires adding `docValues="true"` to the field (or field type) definition, as in this example from the `schema.xml` of Solr's `sample_techproducts_configs` <<config-sets.adoc#config-sets,config set>>:
[source,xml]
----
<field name="manu_exact" type="string" indexed="false" stored="false" docValues="true" />
----
[IMPORTANT]
If you have already indexed data into your Solr index, you will need to completely re-index your content after changing your field definitions in `schema.xml` in order to successfully use docValues.
DocValues are only available for specific field types. The types chosen determine the underlying Lucene docValue type that will be used. The available Solr field types are:
* `StrField` and `UUIDField`.
** If the field is single-valued (i.e., multi-valued is false), Lucene will use the SORTED type.
** If the field is multi-valued, Lucene will use the SORTED_SET type.
* Any `Trie*` numeric fields, date fields and `EnumField`.
** If the field is single-valued (i.e., multi-valued is false), Lucene will use the NUMERIC type.
** If the field is multi-valued, Lucene will use the SORTED_SET type.
* Boolean fields
* Int|Long|Float|Double|Date PointField
** If the field is single-valued (i.e., multi-valued is false), Lucene will use the NUMERIC type.
** If the field is multi-valued, Lucene will use the SORTED_NUMERIC type.
These Lucene types are related to how the {lucene-javadocs}/core/org/apache/lucene/index/DocValuesType.html[values are sorted and stored].
There is an additional configuration option available, which is to modify the `docValuesFormat` <<field-type-definitions-and-properties.adoc#FieldTypeDefinitionsandProperties-docValuesFormat,used by the field type>>. The default implementation employs a mixture of loading some things into memory and keeping some on disk. In some cases, however, you may choose to specify an alternative {lucene-javadocs}/core/org/apache/lucene/codecs/DocValuesFormat.html[DocValuesFormat implementation]. For example, you could choose to keep everything in memory by specifying `docValuesFormat="Memory"` on a field type:
[source,xml]
----
<fieldType name="string_in_mem_dv" class="solr.StrField" docValues="true" docValuesFormat="Memory" />
----
Please note that the `docValuesFormat` option may change in future releases.
[NOTE]
Lucene index back-compatibility is only supported for the default codec. If you choose to customize the `docValuesFormat` in your schema.xml, upgrading to a future version of Solr may require you to either switch back to the default codec and optimize your index to rewrite it into the default codec before upgrading, or re-build your entire index from scratch after upgrading.
== Using DocValues
=== Sorting, Faceting & Functions
If `docValues="true"` for a field, then DocValues will automatically be used any time the field is used for <<common-query-parameters.adoc#CommonQueryParameters-ThesortParameter,sorting>>, <<faceting.adoc#faceting,faceting>> or <<function-queries.adoc#function-queries,function queries>>.
[[DocValues-RetrievingDocValuesDuringSearch]]
=== Retrieving DocValues During Search
Field values retrieved during search queries are typically returned from stored values. However, non-stored docValues fields will be also returned along with other stored fields when all fields (or pattern matching globs) are specified to be returned (e.g. "`fl=*`") for search queries depending on the effective value of the `useDocValuesAsStored` parameter for each field. For schema versions >= 1.6, the implicit default is `useDocValuesAsStored="true"`. See <<field-type-definitions-and-properties.adoc#field-type-definitions-and-properties,Field Type Definitions and Properties>> & <<defining-fields.adoc#defining-fields,Defining Fields>> for more details.
When `useDocValuesAsStored="false"`, non-stored DocValues fields can still be explicitly requested by name in the <<common-query-parameters.adoc#CommonQueryParameters-Thefl_FieldList_Parameter,fl param>>, but will not match glob patterns (`"*"`). Note that returning DocValues along with "regular" stored fields at query time has performance implications that stored fields may not because DocValues are column-oriented and may therefore incur additional cost to retrieve for each returned document. Also note that while returning non-stored fields from DocValues, the values of a multi-valued field are returned in sorted order (and not insertion order). If you require the multi-valued fields to be returned in the original insertion order, then make your multi-valued field as stored (such a change requires re-indexing).
In cases where the query is returning _only_ docValues fields performance may improve since returning stored fields requires disk reads and decompression whereas returning docValues fields in the fl list only requires memory access.
When retrieving fields from their docValues form (using the <<exporting-result-sets.adoc#exporting-result-sets,/export handler>>, <<streaming-expressions.adoc#streaming-expressions,streaming expressions>> or if the field is requested in the `fl` parameter), two important differences between regular stored fields and docValues fields must be understood:
1. Order is _not_ preserved. For simply retrieving stored fields, the insertion order is the return order. For docValues, it is the _sorted_ order.
2. Multiple identical entries are collapsed into a single value. Thus if I insert values 4, 5, 2, 4, 1, my return will be 1, 2, 4, 5.

View File

@ -0,0 +1,20 @@
= Dynamic Fields
:page-shortname: dynamic-fields
:page-permalink: dynamic-fields.html
_Dynamic fields_ allow Solr to index fields that you did not explicitly define in your schema.
This is useful if you discover you have forgotten to define one or more fields. Dynamic fields can make your application less brittle by providing some flexibility in the documents you can add to Solr.
A dynamic field is just like a regular field except it has a name with a wildcard in it. When you are indexing documents, a field that does not match any explicitly defined fields can be matched with a dynamic field.
For example, suppose your schema includes a dynamic field with a name of `*_i`. If you attempt to index a document with a `cost_i` field, but no explicit `cost_i` field is defined in the schema, then the `cost_i` field will have the field type and analysis defined for `*_i`.
Like regular fields, dynamic fields have a name, a field type, and options.
[source,xml]
----
<dynamicField name="*_i" type="int" indexed="true" stored="true"/>
----
It is recommended that you include basic dynamic field mappings (like that shown above) in your `schema.xml`. The mappings can be very useful.

View File

@ -0,0 +1,345 @@
= Enabling SSL
:page-shortname: enabling-ssl
:page-permalink: enabling-ssl.html
Solr can encrypt communications to and from clients, and between nodes in SolrCloud mode, with SSL.
This section describes enabling SSL with the example Jetty server using a self-signed certificate.
For background on SSL certificates and keys, see http://www.tldp.org/HOWTO/SSL-Certificates-HOWTO/.
[[EnablingSSL-BasicSSLSetup]]
== Basic SSL Setup
[[EnablingSSL-Generateaself-signedcertificateandakey]]
=== Generate a Self-Signed Certificate and a Key
To generate a self-signed certificate and a single key that will be used to authenticate both the server and the client, we'll use the JDK https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html[`keytool`] command and create a separate keystore. This keystore will also be used as a truststore below. It's possible to use the keystore that comes with the JDK for these purposes, and to use a separate truststore, but those options aren't covered here.
Run the commands below in the `server/etc/` directory in the binary Solr distribution. It's assumed that you have the JDK `keytool` utility on your `PATH`, and that `openssl` is also on your `PATH`. See https://www.openssl.org/related/binaries.html for OpenSSL binaries for Windows and Solaris.
The `-ext SAN=...` `keytool` option allows you to specify all the DNS names and/or IP addresses that will be allowed during hostname verification (but see below for how to skip hostname verification between Solr nodes so that you don't have to specify all hosts here).
In addition to `localhost` and `127.0.0.1`, this example includes a LAN IP address `192.168.1.3` for the machine the Solr nodes will be running on:
[source,bash]
----
keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass secret -storepass secret -validity 9999 -keystore solr-ssl.keystore.jks -ext SAN=DNS:localhost,IP:192.168.1.3,IP:127.0.0.1 -dname "CN=localhost, OU=Organizational Unit, O=Organization, L=Location, ST=State, C=Country"
----
The above command will create a keystore file named `solr-ssl.keystore.jks` in the current directory.
[[EnablingSSL-ConvertthecertificateandkeytoPEMformatforusewithcURL]]
=== Convert the Certificate and Key to PEM Format for Use with cURL
cURL isn't capable of using JKS formatted keystores, so the JKS keystore needs to be converted to PEM format, which cURL understands.
First convert the JKS keystore into PKCS12 format using `keytool`:
[source,bash]
----
keytool -importkeystore -srckeystore solr-ssl.keystore.jks -destkeystore solr-ssl.keystore.p12 -srcstoretype jks -deststoretype pkcs12
----
The keytool application will prompt you to create a destination keystore password and for the source keystore password, which was set when creating the keystore ("secret" in the example shown above).
Next convert the PKCS12 format keystore, including both the certificate and the key, into PEM format using the http://www.openssl.org[`openssl`] command:
[source,bash]
----
openssl pkcs12 -in solr-ssl.keystore.p12 -out solr-ssl.pem
----
If you want to use cURL on OS X Yosemite (10.10), you'll need to create a certificate-only version of the PEM format, as follows:
[source,bash]
----
openssl pkcs12 -nokeys -in solr-ssl.keystore.p12 -out solr-ssl.cacert.pem
----
[[EnablingSSL-SetcommonSSLrelatedsystemproperties]]
=== Set common SSL related system properties
The Solr Control Script is already setup to pass SSL-related Java system properties to the JVM. To activate the SSL settings, uncomment and update the set of properties beginning with SOLR_SSL_* in `bin/solr.in.sh`. (or `bin\solr.in.cmd` on Windows).
NOTE: If you setup Solr as a service on Linux using the steps outlined in <<taking-solr-to-production.adoc#taking-solr-to-production,Taking Solr to Production>>, then make these changes in `/var/solr/solr.in.sh` instead.
.bin/solr.in.sh example SOLR_SSL_* configuration
[source,bash]
----
SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.jks
SOLR_SSL_KEY_STORE_PASSWORD=secret
SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.jks
SOLR_SSL_TRUST_STORE_PASSWORD=secret
# Require clients to authenticate
SOLR_SSL_NEED_CLIENT_AUTH=false
# Enable clients to authenticate (but not require)
SOLR_SSL_WANT_CLIENT_AUTH=false
# Define Key Store type if necessary
SOLR_SSL_KEY_STORE_TYPE=JKS
SOLR_SSL_TRUST_STORE_TYPE=JKS
----
When you start Solr, the `bin/solr` script includes the settings in `bin/solr.in.sh` and will pass these SSL-related system properties to the JVM.
.Client Authentication Settings
WARNING: Enable either SOLR_SSL_NEED_CLIENT_AUTH or SOLR_SSL_WANT_CLIENT_AUTH but not both at the same time. They are mutually exclusive and Jetty will select one of them which may not be what you expect.
Similarly, when you start Solr on Windows, the `bin\solr.cmd` script includes the settings in `bin\solr.in.cmd` - uncomment and update the set of properties beginning with `SOLR_SSL_*` to pass these SSL-related system properties to the JVM:
.bin\solr.in.cmd example SOLR_SSL_* configuration
[source,text]
----
set SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.jks
set SOLR_SSL_KEY_STORE_PASSWORD=secret
set SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.jks
set SOLR_SSL_TRUST_STORE_PASSWORD=secret
REM Require clients to authenticate
set SOLR_SSL_NEED_CLIENT_AUTH=false
REM Enable clients to authenticate (but not require)
set SOLR_SSL_WANT_CLIENT_AUTH=false
----
[[EnablingSSL-RunSingleNodeSolrusingSSL]]
=== Run Single Node Solr using SSL
Start Solr using the command shown below; by default clients will not be required to authenticate:
.*nix command
[source,bash]
----
bin/solr -p 8984
----
.Windows command
[source,text]
----
bin\solr.cmd -p 8984
----
[[EnablingSSL-SolrCloud]]
== SolrCloud
This section describes how to run a two-node SolrCloud cluster with no initial collections and a single-node external ZooKeeper. The commands below assume you have already created the keystore described above.
[[EnablingSSL-ConfigureZooKeeper]]
=== Configure ZooKeeper
NOTE: ZooKeeper does not support encrypted communication with clients like Solr. There are several related JIRA tickets where SSL support is being planned/worked on: https://issues.apache.org/jira/browse/ZOOKEEPER-235[ZOOKEEPER-235]; https://issues.apache.org/jira/browse/ZOOKEEPER-236[ZOOKEEPER-236]; https://issues.apache.org/jira/browse/ZOOKEEPER-1000[ZOOKEEPER-1000]; and https://issues.apache.org/jira/browse/ZOOKEEPER-2120[ZOOKEEPER-2120].
Before you start any SolrCloud nodes, you must configure your solr cluster properties in ZooKeeper, so that Solr nodes know to communicate via SSL.
This section assumes you have created and started a single-node external ZooKeeper on port 2181 on localhost - see <<setting-up-an-external-zookeeper-ensemble.adoc#setting-up-an-external-zookeeper-ensemble,Setting Up an External ZooKeeper Ensemble>>.
The `urlScheme` cluster-wide property needs to be set to `https` before any Solr node starts up. The example below uses the `zkcli` tool that comes with the binary Solr distribution to do this:
.*nix command
[source,bash]
----
server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd clusterprop -name urlScheme -val https
----
.Windows command
[source,text]
----
server\scripts\cloud-scripts\zkcli.bat -zkhost localhost:2181 -cmd clusterprop -name urlScheme -val https
----
If you have set up your ZooKeeper cluster to use a <<taking-solr-to-production.adoc#TakingSolrtoProduction-ZooKeeperchroot,chroot for Solr>> , make sure you use the correct `zkhost` string with `zkcli`, e.g. `-zkhost localhost:2181/solr`.
[[EnablingSSL-RunSolrCloudwithSSL]]
=== Run SolrCloud with SSL
[[EnablingSSL-CreateSolrhomedirectoriesfortwonodes]]
==== Create Solr home directories for two nodes
Create two copies of the `server/solr/` directory which will serve as the Solr home directories for each of your two SolrCloud nodes:
.*nix commands
[source,bash]
----
mkdir cloud
cp -r server/solr cloud/node1
cp -r server/solr cloud/node2
----
.Windows commands
[source,text]
----
mkdir cloud
xcopy /E server\solr cloud\node1\
xcopy /E server\solr cloud\node2\
----
[[EnablingSSL-StartthefirstSolrnode]]
==== Start the First Solr Node
Next, start the first Solr node on port 8984. Be sure to stop the standalone server first if you started it when working through the previous section on this page.
.*nix command
[source,bash]
----
bin/solr -cloud -s cloud/node1 -z localhost:2181 -p 8984
----
.Windows command
[source,text]
----
bin\solr.cmd -cloud -s cloud\node1 -z localhost:2181 -p 8984
----
Notice the use of the `-s` option to set the location of the Solr home directory for node1.
If you created your SSL key without all DNS names/IP addresses on which Solr nodes will run, you can tell Solr to skip hostname verification for inter-Solr-node communications by setting the `solr.ssl.checkPeerName` system property to `false`:
.*nix command
[source,bash]
----
bin/solr -cloud -s cloud/node1 -z localhost:2181 -p 8984 -Dsolr.ssl.checkPeerName=false
----
.Windows command
[source,text]
----
bin\solr.cmd -cloud -s cloud\node1 -z localhost:2181 -p 8984 -Dsolr.ssl.checkPeerName=false
----
[[EnablingSSL-StartthesecondSolrnode]]
==== Start the Second Solr Node
Finally, start the second Solr node on port 7574 - again, to skip hostname verification, add `-Dsolr.ssl.checkPeerName=false`;
.*nix command
[source,text]
----
bin/solr -cloud -s cloud/node2 -z localhost:2181 -p 7574
----
.Windows command
[source,text]
----
bin\solr.cmd -cloud -s cloud\node2 -z localhost:2181 -p 7574
----
[[EnablingSSL-ExampleClientActions]]
== Example Client Actions
[IMPORTANT]
====
cURL on OS X Mavericks (10.9) has degraded SSL support. For more information and workarounds to allow one-way SSL, see http://curl.haxx.se/mail/archive-2013-10/0036.html. cURL on OS X Yosemite (10.10) is improved - 2-way SSL is possible - see http://curl.haxx.se/mail/archive-2014-10/0053.html .
The cURL commands in the following sections will not work with the system `curl` on OS X Yosemite (10.10). Instead, the certificate supplied with the `-E` param must be in PKCS12 format, and the file supplied with the `--cacert` param must contain only the CA certificate, and no key (see <<EnablingSSL-ConvertthecertificateandkeytoPEMformatforusewithcURL,above>> for instructions on creating this file):
[source,bash]
curl -E solr-ssl.keystore.p12:secret --cacert solr-ssl.cacert.pem ...
====
NOTE: If your operating system does not include cURL, you can download binaries here: http://curl.haxx.se/download.html
=== Create a SolrCloud Collection using `bin/solr`
Create a 2-shard, replicationFactor=1 collection named mycollection using the default configset (data_driven_schema_configs):
.*nix command
[source,bash]
----
bin/solr create -c mycollection -shards 2
----
.Windows command
[source,text]
----
bin\solr.cmd create -c mycollection -shards 2
----
The `create` action will pass the `SOLR_SSL_*` properties set in your include file to the SolrJ code used to create the collection.
[[EnablingSSL-RetrieveSolrCloudclusterstatususingcURL]]
=== Retrieve SolrCloud Cluster Status using cURL
To get the resulting cluster status (again, if you have not enabled client authentication, remove the `-E solr-ssl.pem:secret` option):
[source,bash]
----
curl -E solr-ssl.pem:secret --cacert solr-ssl.pem "https://localhost:8984/solr/admin/collections?action=CLUSTERSTATUS&wt=json&indent=on"
----
You should get a response that looks like this:
[source,json]
----
{
"responseHeader":{
"status":0,
"QTime":2041},
"cluster":{
"collections":{
"mycollection":{
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{"core_node1":{
"state":"active",
"base_url":"https://127.0.0.1:8984/solr",
"core":"mycollection_shard1_replica1",
"node_name":"127.0.0.1:8984_solr",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{"core_node2":{
"state":"active",
"base_url":"https://127.0.0.1:7574/solr",
"core":"mycollection_shard2_replica1",
"node_name":"127.0.0.1:7574_solr",
"leader":"true"}}}},
"maxShardsPerNode":"1",
"router":{"name":"compositeId"},
"replicationFactor":"1"}},
"properties":{"urlScheme":"https"}}}
----
[[EnablingSSL-Indexdocumentsusingpost.jar]]
=== Index Documents using `post.jar`
Use `post.jar` to index some example documents to the SolrCloud collection created above:
[source,bash]
----
cd example/exampledocs
java -Djavax.net.ssl.keyStorePassword=secret -Djavax.net.ssl.keyStore=../../server/etc/solr-ssl.keystore.jks -Djavax.net.ssl.trustStore=../../server/etc/solr-ssl.keystore.jks -Djavax.net.ssl.trustStorePassword=secret -Durl=https://localhost:8984/solr/mycollection/update -jar post.jar *.xml
----
[[EnablingSSL-QueryusingcURL]]
=== Query Using cURL
Use cURL to query the SolrCloud collection created above, from a directory containing the PEM formatted certificate and key created above (e.g. `example/etc/`) - if you have not enabled client authentication (system property `-Djetty.ssl.clientAuth=true)`, then you can remove the `-E solr-ssl.pem:secret` option:
[source,bash]
----
curl -E solr-ssl.pem:secret --cacert solr-ssl.pem "https://localhost:8984/solr/mycollection/select?q=*:*&wt=json&indent=on"
----
[[EnablingSSL-IndexadocumentusingCloudSolrClient]]
=== Index a document using `CloudSolrClient`
From a java client using SolrJ, index a document. In the code below, the `javax.net.ssl.*` system properties are set programmatically, but you could instead specify them on the java command line, as in the `post.jar` example above:
[source,java]
----
System.setProperty("javax.net.ssl.keyStore", "/path/to/solr-ssl.keystore.jks");
System.setProperty("javax.net.ssl.keyStorePassword", "secret");
System.setProperty("javax.net.ssl.trustStore", "/path/to/solr-ssl.keystore.jks");
System.setProperty("javax.net.ssl.trustStorePassword", "secret");
String zkHost = "127.0.0.1:2181";
CloudSolrClient client = new CloudSolrClient.Builder().withZkHost(zkHost).build();
client.setDefaultCollection("mycollection");
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", "1234");
doc.addField("name", "A lovely summer holiday");
client.add(doc);
client.commit();
----

Some files were not shown because too many files have changed in this diff Show More