mirror of https://github.com/apache/lucene.git
squash merge jira/solr-10290 into master
Squashed commit of the following: commit babe763e9d0f9561622171a45ab78607955c5dab Merge: 5a4d75769783f6
Author: Chris Hostetter <hossman@apache.org> Date: Wed May 10 13:26:09 2017 -0700 Merge branch 'master' into jira/solr-10290 Conflicts: lucene/ivy-versions.properties solr/CHANGES.txt solr/core/src/java/org/apache/solr/handler/admin/CollectionHandlerApi.java solr/licenses/android-json-LICENSE-ASL.txt solr/licenses/asciidoctor-ant-LICENSE-ASL.txt solr/solr-ref-guide/src/fonts/Noto_Sans/LICENSE.txt commit 5a4d7579c7d7b84263fd96e2f0dbe0c9db2f0ca0 Author: Chris Hostetter <hossman@apache.org> Date: Wed May 10 12:08:30 2017 -0700 ref guide tools: replace org.json with (not quite) drop in clean room replacement from android, and add all missting licenes and checksums commit 4570315c38a11cf18a336bb68f5ba2e9195e53e8 Author: Chris Hostetter <hossman@apache.org> Date: Wed May 10 10:36:35 2017 -0700 license headers for tools commit bd3d1cfad6579fbcdca76d9a98ce6362174dcce7 Author: Chris Hostetter <hossman@apache.org> Date: Wed May 10 10:32:08 2017 -0700 fix solr 'resolve' target to also resolve ref guide commit f132c9634ddc6423a2760f3df602b4add6052de8 Author: Chris Hostetter <hossman@apache.org> Date: Wed May 10 10:22:44 2017 -0700 Fix some ivy related precommit issues commit 2210941839329422fa1e1de09f799678fc219cbc Author: Chris Hostetter <hossman@apache.org> Date: Wed May 10 10:09:38 2017 -0700 remove old nocommit debugging code (was tired that night) commit 4c80ac15abbae01b5adde2119dba450ae62f6698 Author: Chris Hostetter <hossman@apache.org> Date: Wed May 10 10:08:28 2017 -0700 tabs to spaces commit da77243dd315c63242ac4f986c85f348ea302baa Author: Chris Hostetter <hossman@apache.org> Date: Wed May 10 09:47:02 2017 -0700 fix broken anchors commit 7856fdf7a1ffb8ce038f3a40e8057f9f62dc89ec Author: Cassandra Targett <cassandra.targett@lucidworks.com> Date: Wed May 10 09:43:19 2017 -0500 SOLR-10290: make page TOC appear on page load commit 39fdd15700de7c3a23f332b11fd64b1393b37895 Author: Cassandra Targett <cassandra.targett@lucidworks.com> Date: Wed May 10 09:42:45 2017 -0500 SOLR-10290: remove unused usermaps and tab formats commit fb86e76d1a8caa6051a1a066496897abbe3e6830 Author: Cassandra Targett <cassandra.targett@lucidworks.com> Date: Wed May 10 08:38:57 2017 -0500 Add back json and jsoup dependencies commit 3d036dec3cd1e18e9048d1832f84207e625dd752 Author: Cassandra Targett <cassandra.targett@lucidworks.com> Date: Wed May 10 08:01:57 2017 -0500 Update comment for asciidoctor-ant commit ffdc0b4511b62642a01f9df16e1806199f17b818 Author: Alan Woodward <romseygeek@apache.org> Date: Wed May 10 08:29:55 2017 +0100 LUCENE-7741: Add explain() to DoubleValuesSource commit 370adaa6f1d033e94e051b50a9633a5e8737ea69 Author: Christine Poerschke <cpoerschke@apache.org> Date: Tue May 9 15:18:22 2017 +0100 SOLR-10535: remove six unused test files (Jason Gerlowski via Christine Poerschke) commit 017da204ed18dfbe4bd28b77d484a1a4b071b2d9 Author: Andrzej Bialecki <ab@apache.org> Date: Tue May 9 12:53:18 2017 +0200 SOLR-10262: Add support for configurable metrics implementations. commit dea9c334fc597953dcdff31ce6df73d584e744ec Author: Jan Høydahl <janhoy@apache.org> Date: Tue May 9 13:03:18 2017 +0200 SOLR-10644: solr.in.sh installed by install script should be writable by solr user commit 788b696cdc6f89479e200189b29646d293f83094 Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Tue May 9 12:42:41 2017 +0530 SOLR-8440: Support for enabling basic authentication using bin/solr|bin/solr.cmd Usage: bin/solr auth -enable -prompt bin/solr auth -enable -credentials solr:SolrRocks bin/solr auth -disable commit fba5c76fc2e830c44a1078ee492d3932e31b80dc Author: Cao Manh Dat <datcm@apache.org> Date: Tue May 9 13:56:49 2017 +0700 SOLR-10619: Optimize using cache for DistributedQueue in case of single-consumer commit 1a8bdc53dd733d18391251029d99209f1e7269fe Author: Joel Bernstein <jbernste@apache.org> Date: Mon May 8 22:21:10 2017 -0400 SOLR-10638: Update CHANGES.txt commit 33c2ffd91ed8bde2c1160e3642bcc7ffb227f442 Author: Joel Bernstein <jbernste@apache.org> Date: Mon May 8 22:04:51 2017 -0400 SOLR-10638: Add normalize Stream Evaluator commit b4646846e573bfeb53218dcb1944c82f3ea0abb0 Author: Scott Blum <dragonsinth@apache.org> Date: Mon May 8 20:32:38 2017 -0400 SOLR-10524: Build fix for NPE Introduced by ZkStateWriter batching optimizations. commit 5a253a428607ae1153852c9343e7871317d8789c Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Mon May 8 17:30:22 2017 -0700 SOLR-10639: Removed entry from CHANGES.txt commit 46e6133540bfb018da9f463bc970b041dc9f1b25 Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Mon May 8 17:11:35 2017 -0700 SOLR-10639: Fix NPE in LRU/LFU/FastLRU caches toString method commit 4b116a96f3efd40b62b9e400e52df4d25972883f Author: jdyer1 <jdyer@apache.org> Date: Mon May 8 13:28:55 2017 -0500 SOLR-10522: Revert "SOLR-9972: SpellCheckComponent collations and suggestions returned as a JSON object rather than a list" commit fd626bf0f17e1812bbded71e74446c1a7977fb02 Author: jdyer1 <jdyer@apache.org> Date: Mon May 8 13:26:42 2017 -0500 SOLR-10522: Revert "SOLR-9972: SpellCheckComponent collations and suggestions returned as a JSON object rather than a list" This reverts commit4cd3d15da8
. commit 42e4ea69c6d59bd7b36495149743af486129b42f Author: Joel Bernstein <jbernste@apache.org> Date: Mon May 8 10:19:41 2017 -0400 SOLR-10625: Update CHANGES.txt commit 8675ced8e36eb2fc15edc0aa4dae34caec26d933 Author: Joel Bernstein <jbernste@apache.org> Date: Mon May 8 09:53:12 2017 -0400 SOLR-10625: Add convolution Stream Evaluator commit f830eff0c03e61b2608f8b9d12593b04ace8262f Author: Shalin Shekhar Mangar <shalin@apache.org> Date: Mon May 8 16:11:01 2017 +0530 Fixed nsToMs calculation in OverseerTest commit 65d5ead9b0a443444529c5b0047b585353233e17 Author: Shalin Shekhar Mangar <shalin@apache.org> Date: Mon May 8 15:50:10 2017 +0530 SOLR-10630: HttpSolrCall.getAuthCtx().new AuthorizationContext() {...}.getParams() sometimes throws java.lang.NullPointerException commit 1df6d3b5e07edbe737473c72a1301b2859481671 Author: Cao Manh Dat <datcm@apache.org> Date: Mon May 8 16:21:19 2017 +0700 SOLR-10524: Explore in-memory partitioning for processing Overseer queue messages commit f627c9fbaf225684401fe2166c454f91999266b7 Author: Jan Høydahl <janhoy@apache.org> Date: Mon May 8 10:48:20 2017 +0200 SOLR-8149: Admin UI - Plugins / Stats - active item is now highlighted commit 09a7cba99c178636dc57c5ffc59c08c81b6504e0 Author: Joel Bernstein <jbernste@apache.org> Date: Sun May 7 21:02:41 2017 -0400 SOLR-10626: Update CHANGES.txt commit 0051bacaa35e05515c33b92c58828da3ea382d13 Author: Joel Bernstein <jbernste@apache.org> Date: Sun May 7 20:42:39 2017 -0400 SOLR-10626: Add covariance Stream Evaluator commit 07b707f4a2d153266a771dc9c5d3571db9f7628d Author: Joel Bernstein <jbernste@apache.org> Date: Sun May 7 16:26:53 2017 -0400 SOLR-10622: Update CHANGES.txt commit c2a68d152534a8bef37f65cddc696e7983691821 Author: Joel Bernstein <jbernste@apache.org> Date: Sun May 7 15:23:56 2017 -0400 SOLR-10622: Add regress and predict Stream Evaluators commit a95438e851570c37ca5652e9ce41e57eb5aaa0fd Author: Uwe Schindler <uschindler@apache.org> Date: Sun May 7 13:24:21 2017 +0200 LUCENE-5365, LUCENE-7818: Fix incorrect condition in queryparser's QueryNodeOperation#logicalAnd() commit 6de19d0f48fb1413a46379284928da8b35f0dd85 Author: Mikhail Khludnev <mkhl@apache.org> Date: Sat May 6 10:44:57 2017 +0300 SOLR-10614: remove static backdoor fields from SimplePostTool. Enabling testTechproductsExample commit 204b54b0ad15678a656c4544b52846ea69932153 Author: Joel Bernstein <jbernste@apache.org> Date: Sat May 6 12:53:29 2017 -0400 SOLR-10601: Update CHANGES.txt commit 264ec0e791c847717e41983fad2efa2f4a7851c3 Author: Joel Bernstein <jbernste@apache.org> Date: Sat May 6 12:45:53 2017 -0400 SOLR-10536: Update CHANGES.txt commit 1f6e30fef088c1629ad35ad149c0a655f105a414 Author: yonik <yonik@apache.org> Date: Sat May 6 05:49:19 2017 -0400 SOLR-10547: consolidate MinAgg+MaxAgg, add min/max support for single valued string fields commit 93962273a9fa79c9714edeb45df9cdb0f7e744e6 Author: Mikhail Khludnev <mkhl@apache.org> Date: Sat May 6 00:36:00 2017 +0300 SOLR-10615: latching SDF.doFilter() on init(); respond 404 instead of 500 in case of init failures or corecontainer shutdown. commit d22431b2bf782cbf11d1605e8f45b17f4b08643a Author: Noble Paul <noble@apache.org> Date: Sat May 6 07:54:22 2017 +0930 SOLR-9530: addressing test failures with seed 9F9128B8E3E8FAA7 commit 3da09c91c6729d5e9a349e831608e5d59c0de8e9 Author: Noble Paul <noble@apache.org> Date: Sat May 6 06:00:54 2017 +0930 SOLR-9530: fixing test error commit dd93e0f0ac07e3609e75e8fdef08cafb270dec38 Author: Joel Bernstein <jbernste@apache.org> Date: Fri May 5 14:45:11 2017 -0400 SOLR-10582: Update CHANGES.txt commit 45a7998d0d79a7e230b0c9fe4f8345b94529603c Author: Joel Bernstein <jbernste@apache.org> Date: Fri May 5 14:36:32 2017 -0400 SOLR-10559: Update CHANGES.txt commit 717da83895dc23ef7c7c291404eb54463318241c Author: Joel Bernstein <jbernste@apache.org> Date: Fri May 5 14:27:46 2017 -0400 SOLR-10566: Update CHANGES.txt commit 181d2821b1d2eed28172e748fc39415c679ada9d Author: Christine Poerschke <cpoerschke@apache.org> Date: Thu May 4 18:20:52 2017 +0100 SOLR-10572: Removed three "no longer supported in solrconfig.xml" asserts. commit 338bdf27fb021a5b34b538d0ed65db9a9d774fb2 Author: Joel Bernstein <jbernste@apache.org> Date: Fri May 5 13:57:03 2017 -0400 SOLR-10516: Update CHANGES.txt commit 15f2014bfd2899f404579fb4c9d4738755399fe5 Author: Joel Bernstein <jbernste@apache.org> Date: Fri May 5 13:48:47 2017 -0400 SOLR-10504: Update CHANGES.txt commit 2cf6c76b58dca10d6f77035f2c74a429991fc493 Author: Joel Bernstein <jbernste@apache.org> Date: Fri May 5 13:40:02 2017 -0400 SOLR-10274: Update CHANGES.txt commit 4d6a8d92447d97c74171edf9f21df304bc4e6b3c Author: Joel Bernstein <jbernste@apache.org> Date: Fri May 5 13:32:33 2017 -0400 SOLR-10426: Add shuffle Streaming Expression commit 9bf20444feb13c81047726c5f18cdb0145fd5beb Author: Joel Bernstein <jbernste@apache.org> Date: Fri May 5 13:24:59 2017 -0400 SOLR-10351: Update CHANGES.txt commit ec687156c6ebe1c58654ff0f29061bf02adad79d Author: Joel Bernstein <jbernste@apache.org> Date: Fri May 5 13:14:15 2017 -0400 SOLR-10303: Update CHANGES.txt commit 71f39e747b996bb734cbe4984ce651de59768d8d Author: yonik <yonik@apache.org> Date: Thu May 4 23:50:50 2017 -0400 tests: test reset for other bucket aggregations commit 1ad2f3932b81ddacce428dc4f9edd17e57a19218 Author: Noble Paul <noble@apache.org> Date: Fri May 5 10:47:57 2017 +0930 SOLR-9530: An Update Processor to convert normal update operation to an atomic operations such as add, set,inc, remove ,set, removeregex commit d99c9e64270f3e8510a4e6fd86d25482b5d4b3c3 Author: Mikhail Khludnev <mkhl@apache.org> Date: Fri May 5 02:17:44 2017 +0300 SOLR-9867: @Ignore TestSolrCLIRunExample.testTechproductsExample() commit b2495028201a7cae9c4d0c1edf903ddeda5a2e2b Author: Jan Høydahl <janhoy@apache.org> Date: Fri May 5 00:53:50 2017 +0200 SOLR-7041: Cut over tests from <defaultSearchField> in schema to df on requests commit f945c8608fe3d338d610fd573c0b1b6ad82d7610 Author: Mike McCandless <mikemccand@apache.org> Date: Thu May 4 15:16:05 2017 -0400 LUCENE-7811: add concurrent SortedSet facets implementation commit 60b27234bcf369b2c3e92137353b021119927a3c Author: Erick Erickson <erick@apache.org> Date: Thu May 4 08:10:30 2017 -0700 SOLR-10607: Improve RTimerTree documentation commit 14e3451ebae14a50a419de37cbb42111ef43e798 Author: Mikhail Khludnev <mkhl@apache.org> Date: Thu May 4 17:18:24 2017 +0300 SOLR-9867: fixing JvmMetricsTest broken earlier, bring back testTechproductsExample() and single SDF.cores assignment. commit 76fc383911c83f5166ee5f2ae2dcace63efd2c10 Author: Mikhail Khludnev <mkhl@apache.org> Date: Thu May 4 16:19:14 2017 +0300 SOLR-9867: rollback SDF.createCoreContainer(). disable testTechproductsExample commit e6f7dd4ed90b089b963caa90437c7a431e4b170d Author: Mikhail Khludnev <mkhl@apache.org> Date: Thu May 4 15:45:14 2017 +0300 SOLR-9867: make sure cores are assigned in the end of SolrDispatchFilter.createCoreContainer() only commit 487a804dcdf8b78f7f5a75eab8c94a0057c99c75 Author: Mikhail Khludnev <mkhl@apache.org> Date: Sat Apr 29 00:25:28 2017 +0300 SOLR-9867: fixing TestSolrCLIRunExample.testTechproductsExample - SolrDispatchFilter.doFilter rejects invocation until init() is completed. - introducing isLoaded=false, isLoading=true core status - blocking shutdown until core loading stops - looping run example tool while core is loading 1 min max. commit faa140d2ae46a04097b78fef8ab9cfaab77792ad Author: Noble Paul <noble@apache.org> Date: Thu May 4 15:13:01 2017 +0930 added extra check if it is a liveNode commit d0fdfb1e6036b49a4c1de68e3f806a59f4d74154 Author: yonik <yonik@apache.org> Date: Wed May 3 23:04:33 2017 -0400 SOLR-10596: fix unique/hll docvalue iterator reuse commit 018801ad393e4c09ea9cfcd73873a9622f4ec702 Author: David Smiley <dsmiley@apache.org> Date: Wed May 3 15:50:34 2017 -0400 LUCENE-7814: DateRangePrefixTree bug in years >= 292M commit 3231c205722e0a7a49aa24c38100153d30dfdca4 Author: Chris Hostetter <hossman@apache.org> Date: Wed May 3 10:30:02 2017 -0700 SOLR-10583: JSON Faceting now supports a query time 'join' domain change option commit 6a6063c6fbf4785043da0900321fbbce6f569787 Author: Joel Bernstein <jbernste@apache.org> Date: Wed May 3 10:25:30 2017 -0400 SOLR-10601: StreamExpressionParser should handle white space around = in named parameters commit 2b4835f24ff12f1b1b4bdcd9b5cab36fbcf75f3f Author: David Smiley <dsmiley@apache.org> Date: Wed May 3 10:07:12 2017 -0400 * SOLR-10549: (typo fix in CHANGES.txt) commit f0b15e198146cc97df23420741564e10ce5924c5 Author: David Smiley <dsmiley@apache.org> Date: Wed May 3 09:30:34 2017 -0400 * SOLR-10549: The new 'large' attribute had been forgotten in /schema/fields?showDefaults=true (David Smiley) commit b3dc62dc4c1b2d849d9f29f73ccdb173f31d9ee0 Author: Mikhail Khludnev <mkhl@apache.org> Date: Sun Apr 30 23:37:51 2017 +0300 SOLR-10588: Prevent redundant double core reload on config update. commit fd874be6497d97b18311b823f1407174736dcd22 Author: Joel Bernstein <jbernste@apache.org> Date: Tue May 2 15:44:15 2017 -0400 SOLR-10536: stats Streaming Expression should work in non-SolrCloud mode commit f94865ed36f2b1062615c28bca47ca0be8d65ef0 Author: Erik Hatcher <ehatcher@apache.org> Date: Tue May 2 19:41:06 2017 -0400 SOLR-1485: improve TokenStream API usage commit 858a9bb70b8e0a27af11accb65528883e9752978 Author: Erick Erickson <erick@apache.org> Date: Tue May 2 13:58:47 2017 -0700 SOLR-10519: SolrCLI.atPath cannot handle children that begin with a slash. commit e9431219e402b9726ee7b7cc14a2e828554bdf9b Author: Mark Miller <markrmiller@apache.org> Date: Tue May 2 15:31:57 2017 -0300 SOLR-10316: Unloading a core can remove a ZK SolrCore registration entry for the wrong SolrCore. commit 0418ea5b7a04e103d06abfeffd79bee4cf49ad24 Author: Mark Miller <markrmiller@apache.org> Date: Tue May 2 13:38:44 2017 -0300 SOLR-10430: Add ls command to ZkCLI for listing only sub-directories. commit 064172798acd2cc54bb9856a42118aade321786c Author: Erik Hatcher <ehatcher@apache.org> Date: Tue May 2 10:13:24 2017 -0400 SOLR-1485: remove unused import commit c25560d6f36fd403aecf7614c90a0cd418f489bd Author: Erik Hatcher <ehatcher@apache.org> Date: Tue May 2 07:50:12 2017 -0400 SOLR-1485: fix tests, removing unnecessary tie to Similarity in PayloadDecoder commit dd5da77c1ca89496daaafece9cb139932b957863 Author: Erik Hatcher <ehatcher@apache.org> Date: Mon May 1 21:35:29 2017 -0400 SOLR-1485: Add payload support commit 09fff4da5916f30a624d784bce55c0266716a0e3 Author: Joel Bernstein <jbernste@apache.org> Date: Mon May 1 12:32:37 2017 -0400 SOLR-10559: Fix TupStream to respect field order commit 235f303f9c33ef089dd2d08842012844afa995c2 Author: Joel Bernstein <jbernste@apache.org> Date: Mon May 1 12:06:00 2017 -0400 SOLR-10566: Fix error handling commit 1d20b518bf4a68405e909db66d49df7cfd9a40a5 Author: Steve Rowe <sarowe@apache.org> Date: Mon May 1 10:26:41 2017 -0400 SOLR-9386: Move default clientPort specification to before calling QuorumPeerConfig.parseProperties(), which requires that clientPort be specified. commit f890fbf106603310ca6c13c26d73906384ba2015 Author: Joel Bernstein <jbernste@apache.org> Date: Sun Apr 30 22:14:01 2017 -0400 SOLR-10559: Remove debuggin commit fdcde26d9d3a67a63a386466e68ec4ede6eb0eb7 Author: Jan Høydahl <janhoy@apache.org> Date: Fri Apr 28 22:12:17 2017 +0200 SOLR-7041: Remove a lot of defaultOperator and defaultSearchField from test configs (still more work to do) commit 97d80a1abfdf2014b1f6cebfab5d84181d01701d Author: Mikhail Khludnev <mkhl@apache.org> Date: Sat Apr 29 13:35:24 2017 +0300 LUCENE-7798: Add .equals and .hashCode to ToParentBlockJoinSortField commit 177880ea0d6309c50ff7ab241549510ae12569d0 Author: Dennis Gove <dpgove@gmail.com> Date: Fri Apr 28 21:45:56 2017 -0400 SOLR-10559: Updates TupStream and enhances evaluators to work over values in the SteamContext commit 730dc2f01ce3d0ca45ee29c433923c6ae5dc456f Author: Joel Bernstein <jbernste@apache.org> Date: Fri Apr 28 15:01:43 2017 -0400 SOLR-10582: Add Correlation Evaluator commit baa6cdaded4fd528bdd3801d64f28649416d1519 Author: Steve Rowe <sarowe@apache.org> Date: Fri Apr 28 15:36:50 2017 -0400 SOLR-9596: Add Solr support for SimpleTextCodec, via <codecFactory class=solr.SimpleTextCodecFactory/> in solrconfig.xml (per-field specification in the schema is not possible). commit eeeb947db5399abebcf361f20278fe17e5a61335 Author: Steve Rowe <sarowe@apache.org> Date: Fri Apr 28 11:24:53 2017 -0400 SOLR-9386: Upgrade Zookeeper to 3.4.10 commit 21bab68f2fdb341723801fef7317cb8e43cbaba8 Author: Steve Rowe <sarowe@apache.org> Date: Fri Apr 28 10:14:38 2017 -0400 LUCENE-7794: buildAndPushRelease.py should run validate and documentation-lint commit 0c9cf102e2d330f405e18072cebc11b6fba99404 Author: Steve Rowe <sarowe@apache.org> Date: Fri Apr 28 09:58:02 2017 -0400 LUCENE-7793: smokeTestRelease.py should run documentation-lint commit 2b687866a96878f38ab46de22cceb9bd92d7a0dc Author: Dawid Weiss <dweiss@apache.org> Date: Fri Apr 28 10:58:55 2017 +0200 LUCENE-7796: Make IOUtils.reThrow idiom declare Error return type so callers may use it in a way that compiler knows subsequent code is unreachable. reThrow is now deprecated in favor of IOUtils.rethrowAlways. commit 0b699d6a3338031d7812f1c0d07db86352ff1193 Author: Mikhail Khludnev <mkhl@apache.org> Date: Sat Apr 22 14:39:33 2017 +0300 SOLR-10521: adding sort=childfield(field,$q) asc for {!parent} query. commit 97c6ed53d19cda6a6c46209d858387949a34b088 Author: Joel Bernstein <jbernste@apache.org> Date: Thu Apr 27 17:03:01 2017 -0400 SOLR-10559: Fix precommit commit d8e991cecdd0d829ae97ee92f628ffa2aebda87d Author: Joel Bernstein <jbernste@apache.org> Date: Thu Apr 27 16:40:24 2017 -0400 SOLR-10559: Fixed compilation error commit ff9fd224db521a1009be7ff5ac734c267b110088 Author: Joel Bernstein <jbernste@apache.org> Date: Thu Apr 27 16:30:46 2017 -0400 SOLR-10559: Cleaner syntax commit 59c76a2e69701125a200f84a673129b24397d30b Author: Erik Hatcher <ehatcher@apache.org> Date: Thu Apr 27 15:34:47 2017 -0400 Remove unused imports commit 44dff45ad5ac0f10d0838e70173dad73457e48eb Author: Erik Hatcher <ehatcher@apache.org> Date: Thu Apr 27 15:16:43 2017 -0400 Add CHANGES entry for LUCENE-7481 commit 2ff15d575046f6b5cc0291e33a0270e2940d67d1 Author: Erik Hatcher <ehatcher@apache.org> Date: Thu Apr 27 15:08:51 2017 -0400 LUCENE-7481: Fix PayloadScoreQuery rewrite commit 8f37fb29e94211869c3046ae048dc462dd187f9d Author: Erik Hatcher <ehatcher@apache.org> Date: Thu Apr 27 08:55:05 2017 -0400 LUCENE-7481: Fix SpanPayloadCheckQuery rewrite commit 53c1c5fa205702425c7cfc45f480bc1ee1d2c92f Author: Jim Ferenczi <jimczi@apache.org> Date: Thu Apr 27 13:53:52 2017 +0200 Add 6.5.1 back compat test indexes commit 733829de9d293af95ff1ef230f81a8ed537cb29a Author: Jim Ferenczi <jimczi@apache.org> Date: Thu Apr 27 13:06:39 2017 +0200 update doap files with Lucene / Solr 6.5.1 release commit afc0a997d930c8a414850fa73a8616eb9dcc3d24 Author: Mike McCandless <mikemccand@apache.org> Date: Thu Apr 27 06:59:24 2017 -0400 fix wrong version in logging message commit 6082e9e1bd9eadc814132e39b096ea835830db9b Author: Mikhail Khludnev <mkhl@apache.org> Date: Thu Apr 27 13:01:07 2017 +0300 SOLR-10500: fix many parents with nested children per /update/json/docs request commit 10c1379bc3cd2f0fb75b61e72e8d3bc3a6283ab0 Author: Noble Paul <noble@apache.org> Date: Thu Apr 27 12:38:34 2017 +0930 SOLR-10507: Core Admin status command to emit collection details of each core commit a8d2792b21f182aaaf9bc0da3aea571507b0f8ae Author: Joel Bernstein <jbernste@apache.org> Date: Wed Apr 26 22:34:20 2017 -0400 SEARCH-313: Handled unescaped plus sign in gap commit df3e927ade3f6befdfde511abc4ef2c067df7268 Author: Erik Hatcher <ehatcher@apache.org> Date: Wed Apr 26 20:05:32 2017 -0400 LUCENE-7808: Fix PayloadScoreQuery and SpanPayloadCheckQuery .equals and .hashCode methods. commit 2e3ee9a40de51f4eb91bb80c870d95fb114d8e07 Author: David Smiley <dsmiley@apache.org> Date: Wed Apr 26 14:04:35 2017 -0400 SOLR-10526: fix facet.heatmap facet exclusion when distributed/sharded commit deb017464c5d63688848c722edd54df239eeedcd Author: Andrzej Bialecki <ab@apache.org> Date: Wed Apr 26 19:50:31 2017 +0200 SOLR-10569: "updateHandler" stats is null when queried via MBeans handler. commit 3ead4b38d9f6ec28d2c44fd9e255432cca646ab8 Author: Christine Poerschke <cpoerschke@apache.org> Date: Wed Apr 26 17:01:29 2017 +0100 SOLR-10046: move from 6.6.0 to 7.0.0 CHANGES.txt (backport yet to be completed) commit 28e3134910655c0a41bd241652f1f514c4fe9c20 Author: Andrzej Bialecki <ab@apache.org> Date: Wed Apr 26 17:47:34 2017 +0200 SOLR-10565: Make root names more unique. commit 92901c859bf2029c49e5eb10dad89280860a928c Author: Joel Bernstein <jbernste@apache.org> Date: Wed Apr 26 11:17:22 2017 -0400 SOLR-10566: Fix precommit commit 8a3cbb5c75df08858b04f3191d8b3cfbc748f28d Author: Joel Bernstein <jbernste@apache.org> Date: Wed Apr 26 10:57:52 2017 -0400 SOLR-10566: Add timeseries Streaming Expression commit def07596935e3192d82edc6ce70b76ef8997c021 Author: Mike McCandless <mikemccand@apache.org> Date: Wed Apr 26 09:36:14 2017 -0400 LUCENE-7792: add try/finally to make sure semaphore is released on exceptions commit 004f086c1a7eff00362ebea350a27e648c798c96 Author: David Smiley <dsmiley@apache.org> Date: Wed Apr 26 08:38:52 2017 -0400 SOLR-10537: Added SolrParams.toLocalParamsString() and moved QP.encodeLocalParamVal to ClientUtils commit 6febaa31715fe57b2794aae0325b29a64aa6223e Author: Mike McCandless <mikemccand@apache.org> Date: Wed Apr 26 06:38:28 2017 -0400 LUCENE-7792: add optional concurrency to OfflineSorter commit 0c4f83af1d5bcedafc6f2bf44ee5225ef9e7cd48 Author: Andrzej Bialecki <ab@apache.org> Date: Wed Apr 26 12:31:39 2017 +0200 SOLR-10489: Fix an occasional NPE. commit 0adfdba01e1790844375db4f319ca01a4060788c Author: yonik <yonik@apache.org> Date: Tue Apr 25 22:56:44 2017 -0400 SOLR-10480: fix offset param handling in JSON Facet API commit 4e20b288a8d70043e1ae4e209c0e5e43574ff928 Author: Mike McCandless <mikemccand@apache.org> Date: Tue Apr 25 20:34:00 2017 -0400 LUCENE-7801: SortedSetDocValuesReaderState now implements Accountable commit 78ae8b98764e08c901031ab8674954678aa187cc Author: Steve Rowe <sarowe@apache.org> Date: Tue Apr 25 17:07:42 2017 -0400 SOLR-10310: Fix CopyFieldTest failure commit 8c58414bf73a0949859d7f6eaea57519f9569e17 Author: Steve Rowe <sarowe@apache.org> Date: Tue Apr 25 12:02:25 2017 -0400 SOLR-10310: By default, stop splitting on whitespace prior to analysis in edismax and standard/"lucene" query parsers commit aff53529218c86f1e71dc1988fa277f72ab6eba8 Author: yonik <yonik@apache.org> Date: Tue Apr 25 11:00:30 2017 -0400 SOLR-7452: JSON Facet API - refining for numBuckets commit d19848665bb09fe0adc757ec0eb57b031db3d5fe Author: Mikhail Khludnev <mkhl@apache.org> Date: Tue Apr 25 12:27:18 2017 +0300 SOLR-10520: fix child.facet.field counts commit 61c121cffdf4e1cd3005a6cea69a22761b4356d4 Author: Joel Bernstein <jbernste@apache.org> Date: Mon Apr 24 17:27:37 2017 -0400 SOLR-10559: Add let and get Streaming Expressions commit 2c20d1d5dfb69af3faee10c5cfc52ecddde5f527 Author: yonik <yonik@apache.org> Date: Mon Apr 24 18:17:17 2017 -0400 SOLR-10548: SOLR-10552: numBuckets should use hll and ignore mincount>1 filtering commit 5d57d866e31b8e3575b254dcbef0792b524e5c97 Author: Andrzej Bialecki <ab@apache.org> Date: Mon Apr 24 22:34:46 2017 +0200 SOLR-10557: Make "compact" format default for /admin/metrics. commit 26e717b22d74ea635a4cd510d8b0ccfd1a69e399 Author: Erick Erickson <erick@apache.org> Date: Mon Apr 24 12:17:46 2017 -0700 SOLR-10493: Investigate SolrCloudExampleTest failures. (cherry picked from commit 0247acd) commit 24bea6c588320decad5bcfc4c0e51843eae12b12 Author: Shalin Shekhar Mangar <shalin@apache.org> Date: Tue Apr 25 00:26:21 2017 +0530 SOLR-10047: Move test into its own test class and force use of NoMergePolicy to fix test failures This closes #195 commit a11a49bbd644ffe157b75b295dbc981dbc8b0316 Author: Andrzej Bialecki <ab@apache.org> Date: Mon Apr 24 16:12:02 2017 +0200 SOLR-10489 Tentative fix for a test failure (Mikhail Khludnev via ab) commit b7e56dbbf41e914b78fb93afcbd32fe8dfc7bf1c Author: Mike McCandless <mikemccand@apache.org> Date: Sat Apr 22 18:45:27 2017 -0400 don't allow ExtrasFS for this test case commit 044e0fb915124d05faeb736835af235015e4893f Author: Mikhail Khludnev <mkhl@apache.org> Date: Sun Apr 23 00:28:20 2017 +0300 SOLR-9217: delay JoinUtil call to createWeight for score join commit 96f8100ad10d51e233a268a9afa07c376e1db047 Author: Joel Bernstein <jbernste@apache.org> Date: Sat Apr 22 17:26:19 2017 -0400 SOLR-10551: Improve tests commit 75a250d12d94caf2ff3077ea34bf6f0706642b33 Author: Joel Bernstein <jbernste@apache.org> Date: Sat Apr 22 16:38:38 2017 -0400 SOLR-10551: Add list and cell Streaming Expressions commit 744e5deaa4dbb7a97ab0aecf67c3f73e05032aba Author: Mike McCandless <mikemccand@apache.org> Date: Sat Apr 22 08:49:34 2017 -0400 LUCENE-7797: the static FSDirectory.listAll was always returning an empty array commit 4f5ea24213321f63ccdaaa6d2699c6db9152e03a Author: Jim Ferenczi <jimczi@apache.org> Date: Fri Apr 21 12:01:09 2017 +0200 LUCENE-7791: add tests for index sorting with sparse text fields and norms commit 3f59e233ee3f38fd4b543324fbe64a985833332a Author: Jim Ferenczi <jimczi@apache.org> Date: Fri Apr 21 04:41:24 2017 +0200 LUCENE-7791: add tests with index sorting and sparse docvalues fields commit f933c812e4cadf92b1bd356780c1d8763e2fd1ed Author: David Smiley <dsmiley@apache.org> Date: Thu Apr 20 17:46:28 2017 -0400 SOLR-10499: facet.heatmap DocSet to Bits optimizations commit 7270943e56bfff7d4be4bd5e2649b4078594a698 Author: Andrzej Bialecki <ab@apache.org> Date: Thu Apr 20 15:09:14 2017 +0200 SOLR-10514 Upgrade Metrics library to 3.2.2. commit a2b316e60fd3d10e8122ab9ac31b5127ec291a5e Author: Shai Erera <shaie@apache.org> Date: Thu Apr 20 15:32:27 2017 +0300 Shorten docFreq and totalTermFreq to df and ttf in TermsComponent commit 20674464e04a261eda815e4a1a01e0111f882d60 Author: Shai Erera <shaie@apache.org> Date: Tue Apr 18 06:33:18 2017 +0300 SOLR-10505: Add multi-field support to TermsComponent for terms stats commit 4d7efe1f0c735788f75cb5b2d968ba8c77d6f76d Author: Steve Rowe <sarowe@gmail.com> Date: Wed Apr 19 20:10:11 2017 -0400 SOLR-10527: move CHANGES entry to 6.5.1 section commit b07f6f4c4b0df53638eb1e672505fd57ba315335 Author: Steve Rowe <sarowe@gmail.com> Date: Wed Apr 19 19:02:32 2017 -0400 SOLR-10527: edismax with sow=false fails to create dismax-per-term queries when any field is boosted commit 7a9728df01cd7b5e7c42578290f427e7c44a64f9 Author: Christine Poerschke <cpoerschke@apache.org> Date: Wed Apr 19 12:05:00 2017 +0100 (part 1 of several) SOLR-10415: use parameterized debug logging in SearchHandler and RealTimeGetComponent (Michael Braun via Christine Poerschke) commit fd5f957bb0bc3c27cefe9ca26423ae6a440b0a74 Author: Christine Poerschke <cpoerschke@apache.org> Date: Wed Apr 19 11:50:24 2017 +0100 SOLR-10394: a few more essentially non-public sortWithinGroup to withinGroupSort renames commit 86fe1543b32db0c1316b49ef0b19cd36be03324f Author: Christine Poerschke <cpoerschke@apache.org> Date: Wed Apr 19 11:45:59 2017 +0100 SOLR-5127: Multiple highlight fields and wildcards are now supported e.g. hl.fl=title,text_* (Sven-S. Porst, Daniel Debray, Simon Endele, Christine Poerschke) commit 972dc3164581950f1b33e74fa9841f86413c2cd3 Author: Dawid Weiss <dweiss@apache.org> Date: Wed Apr 19 12:21:18 2017 +0200 LUCENE-7785: Move dictionary for Ukrainian analyzer to external dependency. (Andriy Rysin via Dawid Weiss) commit 432aa350b8e7d7e436aea3556c7c27e99f09b4cb Author: Joel Bernstein <jbernste@apache.org> Date: Tue Apr 18 20:56:21 2017 -0400 SOLR-10516: Add eval() Streaming Expression commit 036ce79066dbe1ff7b435bdf006ad069d6b73f8f Author: David Smiley <dsmiley@apache.org> Date: Tue Apr 18 17:02:07 2017 -0400 SOLR-10439: 'large' was forgotten in /schema/fields?showDefaults=true commit c0f50ad282be823c192c9650b8d5a0442eedb6f0 Author: Chris Hostetter <hossman@apache.org> Date: Tue Apr 18 11:58:35 2017 -0700 SOLR-10472: Fixed uninversion (aka: FieldCache) bugs with the numeric PointField classes, and CurrencyField commit 79ab781507d8d7728bbb89faae495d5174424676 Author: Scott Blum <dragonsinth@gmail.com> Date: Mon Apr 17 18:27:12 2017 -0400 SOLR-10420: fix watcher leak in DistributedQueue commit 2e9bea5c062da5e2efedd5158dcf84fab9978c50 Author: Scott Blum <dragonsinth@gmail.com> Date: Tue Apr 18 14:36:01 2017 -0400 changes commit f1ef3d1dbd95591460c7e00fcc31193ae70a1083 Author: David Smiley <dsmiley@apache.org> Date: Tue Apr 18 14:14:38 2017 -0400 LUCENE-7769: UnifiedHighlighter wasn't seeing inside BoostQuery or SpanBoostQuery commit 8a7401550a5f624286704736f9fdf957bdcce88e Author: David Smiley <dsmiley@apache.org> Date: Tue Apr 18 12:32:12 2017 -0400 LUCENE-7787: HeatmapFacetCounter Bits.MatchNoBits optimization commit 43d02330c96128320cfa905ffc85f81d2bd8cb2c Author: Joel Bernstein <jbernste@apache.org> Date: Tue Apr 18 11:21:15 2017 -0400 SOLR-10504: Add echo Streaming Expression commit d44f6698ed426f20833fb3f558afd6248c9c0456 Author: Christine Poerschke <cpoerschke@apache.org> Date: Tue Apr 18 15:53:13 2017 +0100 Remove unused imports. commit 44db665e9989550fe41261246d075889985fec91 Author: Andrzej Bialecki <ab@apache.org> Date: Tue Apr 18 12:46:39 2017 +0200 SOLR-10418: Metrics API should return JVM system properties. commit be6d90f5965b8408130dc14cace470a19f1a036b Author: yonik <yonik@apache.org> Date: Mon Apr 17 22:30:29 2017 -0400 SOLR-10082: JSON Facet API, add stddev and variance functions commit 3e9d4c64ed0145aeeb3c3f5ec32c056e71541f26 Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Tue Apr 18 03:29:45 2017 +0530 SOLR-10447, SOLR-8589: Adding Yago Riveiro to changelog commit edafee5838ac9ae469429a189aae2955ac955977 Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Tue Apr 18 03:12:41 2017 +0530 SOLR-10447, SOLR-4968, SOLR-8589: Adding contributors to CHANGES.txt commit 4c25637aa8dc7c48399c583fe9f3aa17d0674971 Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Tue Apr 18 03:09:46 2017 +0530 SOLR-10446: Making HttpClusterStateProvider work with server that doesn't have LISTALIASES commit 491d3231ac6221e6d7520013dcf28debea1be6f6 Author: Joel Bernstein <jbernste@apache.org> Date: Mon Apr 17 11:08:53 2017 -0400 SOLR-10486: Fix precommit commit a7a3cb69292d62824f54c2fc3dcff5a8c0b8c153 Author: Joel Bernstein <jbernste@apache.org> Date: Mon Apr 17 10:10:05 2017 -0400 SOLR-10486: Add Length Conversion Evaluators commit b903026ed4d72a423f39f3b28a3a55d2846b22c7 Author: Joel Bernstein <jbernste@apache.org> Date: Thu Apr 13 11:57:45 2017 -0400 SOLR-10485: Remove incorrect code comment commit 249baee1850b0a2608ec029f8865fba54ecaff44 Author: Shalin Shekhar Mangar <shalin@apache.org> Date: Mon Apr 17 13:59:26 2017 +0530 SOLR-10047: Mismatched Docvalues segments cause exception in Sorting/Faceting. Solr now uninverts per segment to avoid such exceptions Squashed commit of the following: commit c38f4cabc2828ee83b53b931dd829e29a3e1701c Author: Keith Laban <kelaban17@gmail.com> Date: Tue Apr 11 17:17:05 2017 -0400 reverted tests to using old wrap interface commit 806f33e092491cc6a2ee292d2934c76171e40dc7 Author: Keith Laban <kelaban17@gmail.com> Date: Tue Apr 11 17:13:34 2017 -0400 updated UninvertingReader.wrap / tests commit b10bcab338b362b909491fea1cf13de66f5f17c0 Author: Keith Laban <klaban1@bloomberg.net> Date: Wed Apr 5 14:57:28 2017 -0400 SOLR-10047 - Updated javadoc/renamed class/added getReaderCacheHelper commit 90ecf5a4ae4feaf3efc42a1ed8643ad21e1c73ce Author: Keith Laban <klaban1@bloomberg.net> Date: Wed Jan 18 16:39:51 2017 -0500 SOLR-10047 - SolrIndexSearcher, UninvertingReader, uninvert docvalues per segment commit 3dbc429295ae4f565c2a998c4f3122e2ffa6502e Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Mon Apr 17 10:11:18 2017 +0530 SOLR-10447, SOLR-10447: LISTALIASES Collections API command; CloudSolrClient can be initialized using Solr URL SOLR-10447: Collections API now supports a LISTALIASES command to return a list of all collection aliases. SOLR-10446: CloudSolrClient can now be initialized using the base URL of a Solr instance instead of ZooKeeper hosts. This is possible through the use of newly introduced HttpClusterStateProvider. To fetch a list of collection aliases, this depends on LISTALIASES command, and hence this way of initializing CloudSolrClient would not work with older versions of Solr that doesn't support LISTALIASES. commit 6f691363fe914157a9c39e4de23e4d0eb49e4c34 Author: Mike McCandless <mikemccand@apache.org> Date: Fri Apr 14 08:15:32 2017 -0400 LUCENE-7782: OfflineSorter now passes the number of items it will write to getWriter commit 3a84e9a55a4547009ca8e115759d5886dcc2f207 Author: markrmiller <markrmiller@apache.org> Date: Fri Apr 14 01:33:19 2017 -0400 SOLR-9936: Allow configuration for recoveryExecutor thread pool size. commit 07c09b6f84defe5be00d135005ac56652ef72d75 Author: markrmiller <markrmiller@apache.org> Date: Fri Apr 14 01:17:03 2017 -0400 SOLR-10151: Use monotonically incrementing counter for doc ids in TestRecovery. commit 9ca384377c41e16721e1c84bb32fe20865cd0eec Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Thu Apr 13 17:31:22 2017 +0530 SOLR-6736: Fix authorization permissions commit 664a4236e4133359b6857d45149f019b9354bc03 Author: Adrien Grand <jpountz@gmail.com> Date: Thu Apr 13 09:43:01 2017 +0200 LUCENE-7783: Fix NPE when getting points on a non-existing field. commit 3b41debf70f7f71df52b36a6de89f00545f454e6 Author: Adrien Grand <jpountz@gmail.com> Date: Thu Apr 13 08:21:39 2017 +0200 LUCENE-7780: Remove TermValComparator.isNull. commit a216928eafc25e8af24c6a1384d5d6dfa22d9ff9 Author: Adrien Grand <jpountz@gmail.com> Date: Thu Apr 13 08:20:56 2017 +0200 LUCENE-7781: Call ensureOpen when registering closed listeners. commit 58de889d2bc9de254c881121fa2bc6054261e600 Author: Erick Erickson <erick@apache.org> Date: Wed Apr 12 17:02:40 2017 -0700 SOLR-10007: Clean up references to CoreContainer and CoreDescriptors commit 27936c20edd5c828fe9c8ff69ab1a80cf011f8e0 Author: Joel Bernstein <jbernste@apache.org> Date: Wed Apr 12 17:26:30 2017 -0400 SOLR-10485: Fix precommit commit df9d1958d413499e657147459549a3ca8c821d71 Author: Joel Bernstein <jbernste@apache.org> Date: Wed Apr 12 17:09:36 2017 -0400 SOLR-10485: Add CalculateStream to allow Solr to behave like a scientific calculator commit 7da48bd9dfc0281e1fe2c7bc6feb316fb91d1aca Author: Mike McCandless <mikemccand@apache.org> Date: Wed Apr 12 17:17:30 2017 -0400 LUCENE-7779: don't call BytesSequencesReader.next again after it already returned null commit 88e85dc0a5e5bf5ec4bff76747040bc5ca4bb111 Author: Joel Bernstein <jbernste@apache.org> Date: Wed Apr 12 14:46:31 2017 -0400 SOLR-10303: Fix precommit commit 7799fcf8c48bf103730d6741041b3a4edf6d04ab Author: Joel Bernstein <jbernste@apache.org> Date: Wed Apr 12 13:18:19 2017 -0400 SOLR-10303: Add the tuple context to avoid creating multiple LocalDateTime instances for the same Tuple commit 8c67cbc345a01f198baaabec016fc33052d209cd Author: Gethin James <gethin.james@alfresco.com> Date: Wed Apr 12 18:00:35 2017 +0200 SOLR-10303: Removing the unused class DatePartEvaluator from the test commit 0f2259de6770163351f4e3f0ef4d2f85d34565bd Author: Gethin James <gethin.james@alfresco.com> Date: Thu Apr 6 17:19:31 2017 +0200 SOLR-10303: Error message formatting for TemporalEvaluator commit 3fc1c254efe6e87ce74a2afe49a7a581fb7f961a Author: Gethin James <gethin.james@alfresco.com> Date: Thu Apr 6 17:19:02 2017 +0200 SOLR-10303: Removing the unused class, replaced by TemporalEvaluator commit d138df5745ed2b1f05f132a480d2c63a4e9a0209 Author: Gethin James <gethin.james@alfresco.com> Date: Thu Apr 6 11:58:26 2017 +0200 SOLR-10303: Refactored to multiple TemporalEvaluator classes based on feedback commit 3c65492947d844070efbe00d7029bd1a9886010f Author: Gethin James <gethin.james@alfresco.com> Date: Mon Mar 20 17:08:15 2017 +0100 SOLR-10303: Switched to pascal casing commit a325c359b0188ea8dd3ceb87bccad6a50e56c507 Author: Gethin James <gethin.james@alfresco.com> Date: Mon Mar 20 17:02:41 2017 +0100 SOLR-10303: Supporting more datatypes via a TemporalAccessor commit 9f3f27d4eb403431878fb80fe08165341a6fed21 Author: Gethin James <gethin.james@alfresco.com> Date: Sat Mar 18 10:42:19 2017 +0100 SOLR-10303: Supporting epoch for LocalDateTime commit 7d97c1692d32d31b94116ec4458fe571e873149f Author: Gethin James <gethin.james@alfresco.com> Date: Fri Mar 17 18:11:00 2017 +0100 SOLR-10303: Switching from the fieldName param to subEvaluators commit 748bff204b7037ad85b6d2c061a5aba4920304a1 Author: Gethin James <gethin.james@alfresco.com> Date: Fri Mar 17 15:30:12 2017 +0100 SOLR-10303: Renamed to DatePartEvaluator and adding support for Instant, Date, LocalDateTime commit e11a40426a36152ccf02f32d0d7ccc6c178b6053 Author: Gethin James <gethin.james@alfresco.com> Date: Fri Mar 17 14:07:50 2017 +0100 SOLR-10303: Initial support for common date/time Stream Evaluators commit 0f3a35220a1097e0f314b0a800ebbadae28e3cc8 Author: Christine Poerschke <cpoerschke@apache.org> Date: Tue Apr 11 11:16:23 2017 +0100 LUCENE-7746: precommit to ignore less and (potentially) error more commit e1e9a71f976aaaa5e43d2615cc6479e5a5259321 Author: Christine Poerschke <cpoerschke@apache.org> Date: Tue Apr 11 11:14:34 2017 +0100 SOLR-10473: Correct LBHttpSolrClient's confusing SolrServerException message when timeAllowed is exceeded. commit 4ed424dc8e6ffdc1deb9f5b479dc74adfe5eb7f5 Author: Andrzej Bialecki <ab@apache.org> Date: Wed Apr 12 10:40:58 2017 +0200 SOLR-9959 Increase the timeout to allow searcher to register metrics. commit 6df755782089c82e94e2c763aa1d6f5d39a17269 Author: Tommaso Teofili <tommaso@apache.org> Date: Wed Apr 12 09:55:03 2017 +0200 LUCENE-7776 - adjusted failing tests in solr due to switching to bm25 in knn commit 4485f66d4a91e3387cb88b4229eace1055dfa889 Author: Mike McCandless <mikemccand@apache.org> Date: Tue Apr 11 15:37:42 2017 -0400 LUCENE-7760: improve setMaxTokenLength javadocs for StandardAnalyzer/Tokenizer and UAX29URLEmailAnalyzer/Tokenizer commit 6e1418c9f2f6d60b3f6462ecc7da0ed2d1d3228b Author: Joel Bernstein <jbernste@apache.org> Date: Tue Apr 11 15:36:03 2017 -0400 SOLR-10274: fix precommit commit 579b5f9199899225f40a82b95b4a213a2cfab7af Author: Joel Bernstein <jbernste@apache.org> Date: Tue Apr 11 15:17:03 2017 -0400 SOLR-10274: The search Streaming Expression should work in non-SolrCloud mode commit dea126033ef363a9736b1fbb568838a9741432f8 Author: Christine Poerschke <cpoerschke@apache.org> Date: Tue Apr 11 20:01:05 2017 +0100 Removed two unused imports. commit ed9f434f37ee64ef0e2bfa5a479feac2f970cb45 Author: Christine Poerschke <cpoerschke@apache.org> Date: Tue Apr 11 19:36:44 2017 +0100 LUCENE-7776: change javadocs default mention from Classic to BM25 (Also kinda added missing javadoc for new method to fix 'ant precommit'.) commit 72b4579b71e0a7958c8602eba0122f898d88a64b Author: Andrzej Bialecki <ab@apache.org> Date: Tue Apr 11 19:22:23 2017 +0200 SOLR-9959: SolrInfoMBean-s category and hierarchy cleanup. commit c21e8664e3ccffee96addb46bd2426d3db80835c Author: Mike McCandless <mikemccand@apache.org> Date: Tue Apr 11 11:47:31 2017 -0400 LUCENE-7777: fix AIOOBE from ByteBlockPool.readBytes when byte block exceeds 32 KB commit 6b3dcad5244406be51b03d33d1c1e988a3805120 Author: Tommaso Teofili <tommaso@apache.org> Date: Tue Apr 11 17:12:13 2017 +0200 LUCENE-7776 - visualize diff btwn BytesRef values in ClassificationTestBase commit 87d3b8cc5afd40512016407320b1d9356bfb543b Author: Christine Poerschke <cpoerschke@apache.org> Date: Tue Apr 11 11:25:39 2017 +0100 LUCENE-7776: remove unused import commit 6586d7ce9022e913271e3cc4d83031c555bb7d85 Author: Tommaso Teofili <tommaso@apache.org> Date: Tue Apr 11 10:44:36 2017 +0200 LUCENE-7776 - use bm25 for knn classifier commit 58f7f68d7142dc3a6f9d3bcd0b036f084b6879ad Author: Adrien Grand <jpountz@gmail.com> Date: Tue Apr 11 10:02:51 2017 +0200 LUCENE-7767: SortedDocValues.ordValue() now throws an IOException. commit 9f300bb85da19635dc627db1c51f3dead10120e1 Author: Mike McCandless <mikemccand@apache.org> Date: Mon Apr 10 19:07:25 2017 -0400 LUCENE-7775: fix exception handling to throw first exception hit commit b0cb9cff99d3d0386193e7c84b50fd61c07c8dc0 Author: Steve Rowe <sarowe@gmail.com> Date: Mon Apr 10 15:26:28 2017 -0400 SOLR-10474: TestPointFields.testPointFieldReturn() depends on order of unsorted hits commit 6535a3cd40eb5e35cbf112d4e131584bb163fbee Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Mon Apr 10 09:48:07 2017 -0700 SOLR-10437: Delete index after each test in TestUseDocValuesAsStored commit 82f13b8c5e8bd7e3d476c0a6bbe8f5d7ebad057c Author: jdyer1 <jdyer@apache.org> Date: Mon Apr 10 08:39:41 2017 -0500 SOLR-8807: disable the CollapseQParser Plugin when testing spellcheck collations for hit-counts commit e0a2890a6f35e26d045dfc9e37f028efcb6dfbd7 Author: Alan Woodward <romseygeek@apache.org> Date: Tue Mar 28 19:52:53 2017 +0100 LUCENE-7701: Refactor grouping collectors commit adf2819c16af26cb15b59afbdecf666fc8b78f66 Author: Noble Paul <noble@apache.org> Date: Mon Apr 10 06:43:27 2017 +0930 SOLR-10429: UpdateRequest#getRoutes()should copy the response parser commit 76a594a5677429b4e0d51a4e25d796d5a77a6120 Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Fri Apr 7 14:11:25 2017 -0700 SOLR-10443: Improvements to TestPointFields * Fixes testInternals, index needs to be cleaned after each field * Validate that SolrQueryParser generates a PointInSetQuery when possible commit 94817b40a65bfbfb95dbed1a35ede606c1d30978 Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Fri Apr 7 11:36:22 2017 -0700 SOLR-10437: Improve test coverage of useDocValuesAsStored=false commit 5b84bad617c21a2af1242ce3e098c28b58715397 Author: Joel Bernstein <jbernste@apache.org> Date: Fri Apr 7 12:36:19 2017 -0400 Add version 6.5.1 commit b60de7380b49906096c6bdc40fbfccc78a3df1fe Author: Christine Poerschke <cpoerschke@apache.org> Date: Fri Apr 7 11:11:23 2017 +0100 SOLR-10264: Fixes multi-term synonym parsing in ManagedSynonymFilterFactory. (Jörg Rathlev, Steve Rowe, Christine Poerschke) commit 315843008bf287c882361aec9847844c8fa9d273 Author: Christine Poerschke <cpoerschke@apache.org> Date: Fri Apr 7 11:04:41 2017 +0100 SOLR-10440: LBHttpSolrClient.doRequest is now always wrapped in a Mapped Diagnostic Context (MDC). commit 27bea7dc2738281fa1a3beb801f83920f48ce009 Author: Christine Poerschke <cpoerschke@apache.org> Date: Fri Apr 7 10:37:30 2017 +0100 SOLR-10441: remove no longer used HttpShardHandlerFactory.USE_RETRIES commit c2d73021182bbe5789e1ca5e3d08a16e1282e7f5 Author: Tommaso Teofili <tommaso@apache.org> Date: Fri Apr 7 10:58:49 2017 +0200 LUCENE-5548 - improved testing for SNBC commit 6af46f1195d899f542aa603684433b4360329423 Author: Tommaso Teofili <tommaso@apache.org> Date: Thu Apr 6 19:13:50 2017 +0200 LUCENE-6853 - re-enabled test classification measures for bpc commit 99c57f533eaf896ac7750f0833939fbd220fbd5b Author: Tommaso Teofili <tommaso@apache.org> Date: Thu Apr 6 19:05:52 2017 +0200 LUCENE-6853 - renamed threshold to bias, initialize to avg tf commit 9a3c8c43af63de6a3e37f07df6aa2834b80a81e7 Author: Joel Bernstein <jbernste@apache.org> Date: Thu Apr 6 22:06:07 2017 -0400 SOLR-10341, SOLR-10444: Update CHANGES.txt commit ed052b20c923f6b3883068b7cf570881686cdcc2 Author: Joel Bernstein <jbernste@apache.org> Date: Thu Apr 6 21:33:47 2017 -0400 SOLR-10444: Fix precommit commit 47f2548a221482f071ad7facfb5d982d5d20dafe Author: Joel Bernstein <jbernste@apache.org> Date: Thu Apr 6 21:17:02 2017 -0400 SOLR-10444: SQL interface does not use client cache commit 878fdedba086a660c5483ba08a3cb05f02abafac Author: Chris Hostetter <hossman@apache.org> Date: Thu Apr 6 12:07:41 2017 -0700 SOLR-10425: Fix indexed="false" on numeric PointFields commit 5a1d69782f30bf8b8fb4342ef2bfcf862777b8f6 Author: jdyer1 <jdyer@apache.org> Date: Thu Apr 6 12:48:19 2017 -0500 SOLR-10323: fix to SpellingQueryConverter to properly strip out colons in field-specific queries commit 338a5e6920a639ad0901079b9f7c4763174a1177 Author: yonik <yonik@apache.org> Date: Thu Apr 6 09:29:29 2017 -0400 SOLR-7452: add support for refining missing allBuckets commit ec5731a60905cbdcc648fbe30502e0d595b6bf00 Author: Cao Manh Dat <datcm@apache.org> Date: Thu Apr 6 16:02:20 2017 +0700 SOLR-10239: Update CHANGES.txt commit f89f9f4ea44d95c44dd2d4ad6d329053b5afd0c2 Author: Cao Manh Dat <datcm@apache.org> Date: Thu Apr 6 15:57:43 2017 +0700 SOLR-10239: change empty lambda to null commit 4942a94d7198af70091cb510695468f3802d6198 Author: Cao Manh Dat <datcm@apache.org> Date: Thu Apr 6 15:48:38 2017 +0700 SOLR-10239: MOVEREPLICA API commit 03c3282de1049589360e8e33110ae0ce697c9854 Author: Joel Bernstein <jbernste@apache.org> Date: Wed Apr 5 17:57:11 2017 -0400 SOLR-10426: Add shuffle Streaming Expression commit fd3b624e9f3f7c458ffbdec86ebe572ce1f7a7f7 Author: Steve Rowe <sarowe@gmail.com> Date: Wed Apr 5 16:23:26 2017 -0400 SOLR-10423: Disable graph query production via schema configuration <fieldtype ... enableGraphQueries="false">. This fixes broken queries for ShingleFilter-containing query-time analyzers when request param sow=false. commit d0b5031ffdbd9f2ee2b45db5354d7b931fdce42c Author: Nicholas Knize <nknize@gmail.com> Date: Wed Apr 5 11:10:15 2017 -0500 LUCENE-7738: Fix min/max verification bug in InetAddressRange to correctly compare IPv4 and IPv6. Update tests. commit 14231a9625f483cbd4ff8bc62aa52d22f6c64d63 Author: David Smiley <dsmiley@apache.org> Date: Wed Apr 5 08:56:50 2017 -0400 SOLR-10404: fetch() streaming expression: escape values in generated query. commit 62bf5c334c3cb9fddbb5fe2732bc53ba443b54fc Author: Christine Poerschke <cpoerschke@apache.org> Date: Wed Apr 5 11:52:34 2017 +0100 Remove unused (private static final) loggers in LTRQParserPlugin and LTRFeatureLoggerTransformerFactory. commit 039e256b59b18e600cbef5443daf86056b6d0e1f Author: Christine Poerschke <cpoerschke@apache.org> Date: Wed Apr 5 11:50:13 2017 +0100 SOLR-10421: Fix params persistence for solr/contrib/ltr (MinMax|Standard)Normalizer classes. (Jianxiong Dong, Christine Poerschke) commit ae603ddf5bc6e09e633e75059912f16f583aa726 Author: Shalin Shekhar Mangar <shalin@apache.org> Date: Wed Apr 5 16:01:44 2017 +0530 SOLR-10277: On 'downnode', lots of wasteful mutations are done to ZK commit 9f7ef3f878ee70c53531a26866da5447b4c61b97 Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Tue Apr 4 13:11:02 2017 -0700 SOLR-10347: Remove index level boost support from 'documents' section of the admin UI commit 8cd9deff5b0619807254749d7300048c741c0807 Author: Christine Poerschke <cpoerschke@apache.org> Date: Tue Apr 4 12:52:09 2017 +0100 SOLR-10394: Rename getSortWithinGroup to getWithinGroupSort in search.grouping.Command class. (Judith Silverman, Christine Poerschke) commit 4288bc73e456925c0e4ab04fe99a061b98a10482 Author: Shalin Shekhar Mangar <shalin@apache.org> Date: Tue Apr 4 14:20:31 2017 +0530 SOLR-10416: The JSON output of /admin/metrics is fixed to write the container as a map (SimpleOrderedMap) instead of an array (NamedList) commit e8d9987d536b431e29d321b3cfb7222911d3574b Author: Mikhail Khludnev <mkhl@apache.org> Date: Tue Apr 4 11:01:19 2017 +0300 SOLR-9745: check exit code only if process has finished commit c10dd97496eac37aa07cac28759f14c62c9f3438 Author: Adrien Grand <jpountz@gmail.com> Date: Thu Mar 30 09:12:45 2017 +0200 LUCENE-7756: Only record the major Lucene version that created the index, and record the minimum Lucene version that contributed to segments. commit 0c8d63373ac94a9deec5e1ec1991ab135dcc5707 Author: Mikhail Khludnev <mkhl@apache.org> Date: Tue Apr 4 08:42:31 2017 +0300 SOLR-9745: bring back timeout value to fix tests commit 4d6eb98561729b778b4feac9f8362d42f196beb1 Author: Mark Miller <markrmiller@apache.org> Date: Mon Apr 3 22:00:08 2017 -0300 SOLR-10338: Revert configure SecureRandom non blocking for tests. (reverted from commit0445f8200e
) commit 5430570655c8377d45479dffd50cfa0cad3af47a Author: Joel Bernstein <jbernste@apache.org> Date: Mon Apr 3 20:39:37 2017 -0400 SOLR-10351: Add try-with-resources clause around TokenStream commit 4111f12e8bfcd6fa2076cb781bb85abdab144534 Author: Mikhail Khludnev <mkhl@apache.org> Date: Mon Apr 3 23:45:54 2017 +0300 SOLR-9745: fix solr.cmd to print errors from invoked script commit 5ffb14bf57992ede5370bc6ca168dfc75d062c41 Author: Erick Erickson <erick@apache.org> Date: Mon Apr 3 13:27:12 2017 -0700 SOLR-8906: Make transient core cache pluggable commit e1c6dd4b2fba4b709a39ec5d2a27650ce5f1b954 Author: Adrien Grand <jpountz@gmail.com> Date: Mon Apr 3 13:49:05 2017 +0200 LUCENE-7749: Made LRUQueryCache delegate the scoreSupplier method. commit cf9ce6ad127844d08f80386e69324d7b02beeb07 Author: Christine Poerschke <cpoerschke@apache.org> Date: Mon Apr 3 13:01:16 2017 +0100 SOLR-10383: Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class. (Vitezslav Zak, Christine Poerschke) commit a902df5f45ce05b315243bf352f9c63fa27e341e Author: Christine Poerschke <cpoerschke@apache.org> Date: Mon Apr 3 12:10:09 2017 +0100 SOLR-10383: reduce code duplication in TestOriginalScoreFeature commit 780a5304487925d4c8e0a3270dd86c38dcb97a3b Author: Mike McCandless <mikemccand@apache.org> Date: Sun Apr 2 16:25:09 2017 -0400 switch to advanceExact commit 4c85f560b074a350e3559505a9fc075866f13abc Author: Dennis Gove <dpgove@gmail.com> Date: Fri Mar 31 20:52:42 2017 -0400 SOLR-10393: Adds UUID Streaming Evaluator commit ae138852fed59bae2a4106c218b76fca01538198 Author: Dennis Gove <dpgove@gmail.com> Date: Thu Mar 23 20:08:11 2017 -0400 SOLR-10356: Adds basic math streaming evaluators commit e027bc4b566acd34a04f75cdf4700a26713738ac Author: Alexandre Rafalovitch <arafalov@apache.org> Date: Sat Apr 1 19:06:50 2017 -0400 SOLR-9601: DIH Tika example is now minimal Only keep definitions and files required to show Tika-extraction in DIH commit a78aef990d5f4ae9a7d6704d677bd000ce16f8a9 Author: Alexandre Rafalovitch <arafalov@apache.org> Date: Sat Apr 1 13:42:23 2017 -0400 SOLR-7383: Replace DIH 'rss' example with 'atom' rss example was broken for multiple reasons. atom example showcases the same - and more - features and uses the smallest config file needed to make it work. commit 3e5e6522f0c63abc860da812d59f2f1116746f5b Author: Chris Hostetter <hossman@apache.org> Date: Fri Mar 31 18:16:13 2017 -0700 SOLR-10399: cleanup unused imports commit 95bddaf0bb431c2aef7e41b59a4777ecc044f3e2 Author: Chris Hostetter <hossman@apache.org> Date: Fri Mar 31 17:01:42 2017 -0700 SOLR-10399: Generalize some internal facet logic to simplify points/non-points field handling commit d6cdcceac7a857e983713d9839c4cddfb2e9916c Author: Christine Poerschke <cpoerschke@apache.org> Date: Fri Mar 31 18:10:27 2017 +0100 LUCENE-7763: Remove outdated comment in IndexWriterConfig.setIndexSort javadocs. (马可阳 via Christine Poerschke) commit b865e31397e2a519a34606b493003a5952fafc08 Author: yonik <yonik@apache.org> Date: Fri Mar 31 12:55:15 2017 -0400 SOLR-7452: add more tests for refinement of missing buckets commit 8d91c30a67b8aca488b3640ec5d996075362828d Author: yonik <yonik@apache.org> Date: Thu Mar 30 12:55:27 2017 -0400 SOLR-7452: refinement of missing buckets and partial facets through missing buckets commit 90c02315cb935b29051596061fb4ce7d7383d575 Author: Adrien Grand <jpountz@gmail.com> Date: Fri Mar 31 16:22:45 2017 +0200 LUCENE-7753: Make fields static when possible. commit b9dd7bf4456cba88a21ac98f8587a03aa677bac5 Author: Adrien Grand <jpountz@gmail.com> Date: Fri Mar 31 16:11:19 2017 +0200 LUCENE-7761: Fixed comment in ReqExclScorer. commit ae421e820ab45328d7229d756b16677d957385c3 Author: markrmiller <markrmiller@apache.org> Date: Fri Mar 31 10:53:20 2017 -0400 SOLR-10338: Configure SecureRandom non blocking for tests. commit 99ebb56439b29ab0a11c66d3afd9904995e8fe91 Author: Joel Bernstein <jbernste@apache.org> Date: Thu Mar 30 17:52:16 2017 +0100 SOLR-10351: Fix pre-commit commit a45f36ef3d3ebdf97149fe405a9980145b79c133 Author: Joel Bernstein <jbernste@apache.org> Date: Thu Mar 30 17:34:28 2017 +0100 SOLR-10351: Add analyze Stream Evaluator to support streaming NLP commit 5a401ab5262224052d0abf671a64bac99a5e4d7c Author: Adrien Grand <jpountz@gmail.com> Date: Thu Mar 30 15:11:52 2017 +0200 LUCENE-7755: Join queries should not reference IndexReaders. commit 139f885554dc88863559dd43e7dc153ec256714c Author: Erick Erickson <erick@apache.org> Date: Wed Mar 29 21:13:40 2017 -0700 SOLR-10387: zkTransfer normalizes destination path incorrectly if source is a windows directory commit 317ef93a17890c076846a69e1e630d983362aba0 Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Wed Mar 29 19:22:02 2017 +0530 SOLR-10352: Fixing available entropy warning limit to 300 commit 4269c23bd87234ac078bd5ff2f893e2086c15115 Author: Andrzej Bialecki <ab@apache.org> Date: Wed Mar 29 14:42:20 2017 +0200 SOLR-10362 Be more specific when catching this exception. commit b6b2aac3f63823819d81a9b9d8f9f77fd65c7ced Author: Mike McCandless <mikemccand@apache.org> Date: Wed Mar 29 08:48:26 2017 -0400 remove dead code commit 00f2db48f7ebdf5d49181f8b72079897a3999e4a Author: Jan Høydahl <janhoy@apache.org> Date: Wed Mar 29 10:51:34 2017 +0200 SOLR-10147: Admin UI -> Cloud -> Graph: Impossible to see shard state commit 3a9ae68230e98c46782e9fb190c4fb37f2517771 Author: Cao Manh Dat <datcm@apache.org> Date: Wed Mar 29 13:52:51 2017 +0700 SOLR-9993: Add support for ExpandComponent with PointFields commit 25b5d090455cc856d6bda5bd4129f6eefcb725dd Author: Shai Erera <shaie@apache.org> Date: Thu Mar 23 08:28:05 2017 +0200 SOLR-10349: Add totalTermFreq support to TermsComponent TermsComponent only returns docFreq information per requested term. This commit adds a terms.ttf parameter, which if set to true, will return both docFreq and totalTermFreq statistics for each requested term. commit 90efd090bfa5e8bbf45bb8ccba0997f916b1d2f7 Author: Cao Manh Dat <datcm@apache.org> Date: Wed Mar 29 08:09:40 2017 +0700 SOLR-10079: TestInPlaceUpdates(Distrib|Standalone) failures commit 83fb5d7758e6c2653386d483ff2cc9495864ad81 Author: yonik <yonik@apache.org> Date: Tue Mar 28 19:52:51 2017 -0400 SOLR-7452: change terminology from _m missing-bucket to _p partial-bucket for refinement commit 3c2e9c913435bdb685f2bfdcc6e0059e0d5e1c0f Author: Steve Rowe <sarowe@apache.org> Date: Tue Mar 28 18:39:28 2017 -0400 SOLR-10357: Enable edismax and standard query parsers to handle the option combination sow=false / autoGeneratePhraseQueries=true by setting QueryBuilder.autoGenerateMultiTermSynonymsQuery commit 2762e31bfe243144f7dbba305c60b66d8c43793e Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Wed Mar 29 00:44:27 2017 +0530 SOLR-6736: Adding support for uploading zipped configsets using ConfigSets API commit 94d69590e67259b14efcfebb54f748a381c37a82 Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Wed Mar 29 00:26:31 2017 +0530 SOLR-10365: Handle a SolrCoreInitializationException while publishing core state during SolrCore creation commit a3d51aebe3724fbcc2162b5fc8862b336595fe57 Author: Joel Bernstein <jbernste@apache.org> Date: Tue Mar 28 09:25:25 2017 +0100 SOLR-10341: SQL AVG function mis-interprets field type commit 9a927620c0d59801dc633d8aace14aaf2aa86dc3 Author: Steve Rowe <sarowe@gmail.com> Date: Tue Mar 28 11:47:02 2017 -0400 SOLR-10343: Update Solr default/example and test configs to use SynonymGraphFilterFactory commit c2d8c05387c8085af2ae14e688daa979b5632ff0 Author: Adrien Grand <jpountz@gmail.com> Date: Tue Mar 28 15:25:16 2017 +0200 LUCENE-7743: Avoid calling new String(String). commit 4e317da9ee19b17619dfbbf9f69026d61782c8c2 Author: Adrien Grand <jpountz@gmail.com> Date: Tue Mar 28 15:21:35 2017 +0200 LUCENE-7751: Avoid boxing primitives only to call compareTo. commit 41f533559db6d442a633c5fcfda14fd11cf027af Author: Adrien Grand <jpountz@gmail.com> Date: Tue Mar 28 15:15:45 2017 +0200 LUCENE-7754: Inner classes should be static whenever possible. commit b216a28f720c0cd6342e050a1a7b895248c65a6d Author: Jan Høydahl <janhoy@apache.org> Date: Tue Mar 28 14:24:09 2017 +0200 SOLR-10369: bin\solr.cmd delete and healthcheck now works again (fixed continuation chars ^) commit a3eb7488a69f1824a02f9d3e7820f1e9d7f0531c Author: Steve Rowe <sarowe@gmail.com> Date: Mon Mar 27 23:53:55 2017 -0400 SOLR-10344: Update Solr default/example and test configs to use WordDelimiterGraphFilterFactory commit b6fcaadbb9ac53557fca901b11af3801cab2ddad Author: Andrzej Bialecki <ab@apache.org> Date: Mon Mar 27 22:17:34 2017 +0200 SOLR-10362: "Memory Pool not found" error when reporting JVM metrics. commit 398c9200876cdd0973fb9da0d35fd6765264d50d Author: Erick Erickson <erick@apache.org> Date: Mon Mar 27 12:15:05 2017 -0700 SLR-10108: bin/solr script recursive copy broken commit 39584af1f0a3a1ace793b8ace45cb07cdff3edf4 Author: Ishan Chattopadhyaya <ishan@apache.org> Date: Mon Mar 27 23:56:23 2017 +0530 SOLR-10352: bin/solr script now prints warning when available system entropy is lower than 300 commit ce6a309140a76af1a85a88362e53a57f047edc3b Author: Erick Erickson <erick@apache.org> Date: Mon Mar 27 09:31:15 2017 -0700 SOLR-10371: There is some spelling mistakes in the Java source code Thanks hu xiaodong" commit e030eacba8f63d66fcfbcd5e94f89463de2a77f0 Author: markrmiller <markrmiller@apache.org> Date: Mon Mar 27 10:44:27 2017 -0400 SOLR-10076: Move changes entry to 6.6 release. commit 9e8a9df5076e98ca7af2272495a287fc5a916700 Author: Shalin Shekhar Mangar <shalin@apache.org> Date: Mon Mar 27 20:06:01 2017 +0530 SOLR-9835: Fixed precommit failure. commit 19df35c95f3042c676f311f26c299b3e1921cf0d Author: Cao Manh Dat <datcm@apache.org> Date: Mon Mar 27 16:44:22 2017 +0700 SOLR-9835: TestInjection.waitForInSyncWithLeader() should rely on commit point of searcher commit 781c9e7855d4a830c8c9d0c0a5c813e856e8f18d Author: Jim Ferenczi <jim.ferenczi@elastic.co> Date: Sat Mar 25 18:44:39 2017 +0100 Update project doap files with 6.5.0 release commit 0545d0886176a511a8dcb72872051f3b3488242e Author: Jim Ferenczi <jim.ferenczi@elastic.co> Date: Sat Mar 25 18:37:25 2017 +0100 Add 6.5.0 back compat test indexes commit 51c16b81eca762a2348c9526694fdf07bab7c332 Author: Mike McCandless <mikemccand@apache.org> Date: Sat Mar 25 06:21:29 2017 -0400 fix typo in comment commit 9c0fc47bb88cf3823dc47ebb9d729b3c5ed3d21e Author: David Smiley <dsmiley@apache.org> Date: Fri Mar 24 23:01:32 2017 -0400 SOLR-10304: Refactor new SolrDocumentFetcher out of SolrIndexSearcher commit a67d143cde7ddc26343e6b68f37acf3d6c7b2e3d Author: David Smiley <dsmiley@apache.org> Date: Fri Mar 24 22:46:24 2017 -0400 SOLR-10249: Refactor IndexFetcher to return detailed result commit 75f4376e07b540291a22fe9956b50c9cb9160cb2 Author: yonik <yonik@apache.org> Date: Fri Mar 24 20:43:44 2017 -0400 SOLR-7452: add support for _m buckets, missing and has sub-facets in need of refinement commit c7d33661013543fe32a231d5b18e36e929a95125 Author: Steve Rowe <sarowe@apache.org> Date: Fri Mar 24 12:31:16 2017 -0400 SOLR-9221: Remove Solr contribs: map-reduce, morphlines-core and morphlines-cell commit d72d707f0406b51153423b19a3387a669467fc51 Author: Chris Hostetter <hossman@apache.org> Date: Thu Mar 23 18:28:10 2017 -0700 Fix test to stop asserting specific order when secondary sort is unspecified, add new checks that do assert an explicit order when secondary sort IS specified. commit 665d0a954cf981fdbcb5aead4f7bb1b31ec0b75c Author: Shalin Shekhar Mangar <shalin@apache.org> Date: Thu Mar 23 19:33:45 2017 +0530 SOLR-10281: ADMIN_PATHS is duplicated in two places and inconsistent commit aefcdd1ab1a1fecbba7c8a31226b48363bb76313 Author: Noble Paul <noble@apache.org> Date: Thu Mar 23 18:12:20 2017 +1030 SOLR-6615: use constants for 'sort', 'distrib' commit 0e4562d6a29ad072b45ee9aae0d83b5febd5600f Author: koji <koji@apache.org> Date: Thu Mar 23 14:57:45 2017 +0900 SOLR-9184: Add a static convenience method ModifiableSolrParams#of(SolrParams) which returns the same instance if it already is modifiable, otherwise creates a new ModifiableSolrParams instance. commit d762464e305b6b0fd70f921dcade41c5c25d1d2f Author: Noble Paul <noble@apache.org> Date: Thu Mar 23 11:45:50 2017 +1030 SOLR-6615: use constants for 'id', '_route_', '_version_' commit 3ad6ffe8bffc3eee2ec9320d4f11300451fc3519 Author: yonik <yonik@apache.org> Date: Wed Mar 22 19:53:50 2017 -0400 SOLR-7452: facet refinement - don't generate domain if skipping bucket commit c913835cb701a4a4743840afe6b3ec28bf5e171b Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Wed Mar 22 10:52:14 2017 -0700 SOLR-9986: Add javadoc to DatePointField class commit 6747c82fbf846eaf3471ca58183502e2f818418b Author: Cao Manh Dat <datcm@apache.org> Date: Wed Mar 22 15:00:33 2017 +0700 Add support for CollapseQParser with PointFields commit 532829cd79e203ad064ff6841eedba9e5dbd9799 Author: yonik <yonik@apache.org> Date: Tue Mar 21 08:42:33 2017 -0400 SOLR-7452: json facet API, refine/skip through buckets already visited commit 24caf3e39405137cec787e3f4f6f607aad82d28c Author: Dennis Gove <dpgove@gmail.com> Date: Tue Mar 21 08:40:40 2017 -0400 SOLR-10333: Fixes use of HashedMap in StreamEvaluator tests commit 1d0e38d3d0b6f558220473578236daead6faff34 Author: Dennis Gove <dpgove@gmail.com> Date: Mon Mar 20 16:36:05 2017 -0400 SOLR-10292: Adds CartesianProductStream to turn multivalued fields into multiple tuples commit 21798848c6b3af14d9b43aa2d27eddcec8dd7679 Author: Christine Poerschke <cpoerschke@apache.org> Date: Mon Mar 20 19:00:33 2017 +0000 SOLR-10046: move from 6.5.0 to 6.6.0 CHANGES.txt (backport yet to be completed) commit fb3fb8c82e0067d53ddb2579acf97dc6a987e99e Author: Andrzej Bialecki <ab@apache.org> Date: Mon Mar 20 19:03:55 2017 +0100 SOLR-10319 SolrCore "instanceDir" metric not visible in JMX. commit c18a11fc787cc8139578861e08bb703ddd2955a1 Author: Cao Manh Dat <datcm@apache.org> Date: Mon Mar 20 17:30:40 2017 +0700 SOLR-9992: Update changes.txt commit 39bb661c9fa69545137c0ae8e14538492b7bb3de Author: Cao Manh Dat <datcm@apache.org> Date: Mon Mar 20 15:21:36 2017 +0700 SOLR-9992: Add support for grouping with PointFIelds commit 575535567dd3d13e69014a5f335400298bec4835 Author: Cao Manh Dat <datcm@apache.org> Date: Mon Mar 20 09:18:54 2017 +0700 SOLR-10079: Speedup TestInPlaceUpdatesDistrib in new replication mode commit c74cd14cc381da34ec52c8f1c09457c921fc4063 Author: Cao Manh Dat <datcm@apache.org> Date: Mon Mar 20 08:21:54 2017 +0700 SOLR-9835: Fix OnlyLeaderIndexesTest failure, inplace updates is not copied over properly commit 0932284a3b5cfef8e082d92bba4258ce6f337ea3 Author: Steve Rowe <sarowe@apache.org> Date: Sat Mar 18 15:09:43 2017 -0400 LUCENE-7748: buildAndPushRelease.py should fail if the project DOAP files are missing releases that are less than the release being produced commit 24d3e21f5039426c8c6f297bc71d6c540b2d8db5 Author: David Smiley <dsmiley@apache.org> Date: Sat Mar 18 10:42:34 2017 -0400 SOLR-10286: fix test for Windows commit 12b8b70cdad7b9f10c8e1e5f5baf17804cde7fd1 Author: Steve Rowe <sarowe@apache.org> Date: Sat Mar 18 00:00:59 2017 -0400 SOLR-10218: The Schema API commands add-field-type and replace-field-type improperly specify SimilarityFactory params commit cc0f133b50cbf22cfe3b3913c6ac463e85502849 Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Fri Mar 17 13:45:29 2017 -0700 Fix CHANGES.txt commit 5e59c85583f2aa8adc62470b8008ab8fa1a2f536 Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Fri Mar 17 11:55:15 2017 -0700 SOLR-10237: Poly-Fields should work with subfield that have docValues=true commit 3b0555639465c651fb731c3068bcd0f95ed35835 Author: yonik <yonik@apache.org> Date: Fri Mar 17 12:13:43 2017 -0400 SOLR-7452: add refine param to json facets, implement for array field faceting commit a671eb4b42fcb1010cd77e1acfd7c6cacfa859fe Author: Jim Ferenczi <jimczi@apache.org> Date: Fri Mar 17 15:53:21 2017 +0100 LUCENE_7747: QueryBuilder now iterates lazily over the possible paths when building a graph query commit 0991eb27a9973692376814b58eab42d53afec40c Author: Jim Ferenczi <jimczi@apache.org> Date: Fri Mar 17 11:55:59 2017 +0100 Add 6.6 version commit e6f215ad48bc0e3abef94d8a31b81da7d8d672f3 Author: David Smiley <dsmiley@apache.org> Date: Thu Mar 16 21:22:08 2017 -0400 SOLR-10273: DocumentBuilder move longest field to last position commit c718e7bdb697f5f2cf4c029cca99b8cfc01974bb Author: David Smiley <dsmiley@apache.org> Date: Thu Mar 16 21:11:39 2017 -0400 SOLR-10286: fix precommit (unused imports) commit 754bf0cb0de13b49357b6e52c95af99323a7e1ef Author: Steve Rowe <sarowe@apache.org> Date: Thu Mar 16 19:41:37 2017 -0400 SOLR-9185: Solr's edismax and Lucene/standard query parsers should optionally not split on whitespace before sending terms to analysis commit 15f4e69249ab28db4bf377dfd2b960f402b1bfe8 Author: David Smiley <dsmiley@apache.org> Date: Thu Mar 16 18:30:57 2017 -0400 SOLR-10286: fix test; we were writing to read-only dir. Expand solrconfig-managed-schema.xml to have toggle-able elements vis system property flags commit 8baf6bd8cad79974858ce13354d5a6c739e441cd Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Thu Mar 16 15:10:48 2017 -0700 SOLR-9990: Avoid copyField in SolrExampleTests.testUpdateField commit 30859ba48727f300a128b398ca263f3198ca7017 Author: David Smiley <dsmiley@apache.org> Date: Thu Mar 16 14:58:59 2017 -0400 SOLR-10286: large fields. And refactored FieldType.checkSchemaField to call a new checkSupportsDocValues() commit 3494d1c610f549f252c77c0a2e322e6293dd0b97 Author: Joel Bernstein <jbernste@apache.org> Date: Thu Mar 16 14:18:43 2017 -0400 Fixed typos in CHANGES.txt commit 75a7faaeefd873a961fa48d7a8dd2daa1898c631 Author: Tomas Fernandez Lobbe <tflobbe@apache.org> Date: Thu Mar 16 11:08:50 2017 -0700 SOLR-9990: Add PointFields in example/default schemas commit 9cc480384205142b79e220fb0b19cb78b6ba1b4c Author: Joel Bernstein <jbernste@apache.org> Date: Thu Mar 16 13:54:25 2017 -0400 SOLR-10254, 10085: Update CHANGES.txt commit 7b235fec21c3492d275c1bb98b1edcabd52365d3 Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 18:28:22 2017 -0700 use callouts instead of hackish fake-comments commit e00da437099d67fa055b62b6dac9b296a9878b08 Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 18:19:52 2017 -0700 cleanup a bunch of [source,foo] tags (in o-z files) where foo was inaccurate files found by manually skimming grep -A4 '\[source,.*\]' src/[o-z]*.adoc commit 00cffbae7ab21364cdea14ecbfb3dc695a2b9757 Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 17:40:50 2017 -0700 cleanup a bunch of [source,foo] tags (in a-n files) where foo was inaccurate files found by manually skimming grep -A4 '\[source,.*\]' src/[a-n]*.adoc commit f7e6565d7fb1e759fbc4c1b607249579a8e6c262 Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 17:05:44 2017 -0700 rename CDCR page so it doesn't end in a silly '-' commit f6182d8519a5b6204985833e53d41c9530a4b251 Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 16:58:29 2017 -0700 SOLR-10640: move the page.shortname based id to <body> so there is no jitter/scroll when clicking links in HTML, and update CheckLinksAndAnchors to account for it there commit ccebeecd1e432f87f50bb98a18496fe7cad3688f Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 16:40:28 2017 -0700 SOLR-10655: add TODO comments noting content that should be refactored into include snippets commit 04bf430b7398e65385b26997bb9847f4aa052ed6 Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 16:28:27 2017 -0700 removing weird/unneeded src/field-type-definitions-and-properties.pdf commit 296c2f48eba6a92466659a06b96f7dacb4d05ffd Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 14:04:59 2017 -0700 use versioned link for online errata commit 320aedd60aafb406dd83110cf3a43f906af7e361 Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 14:02:01 2017 -0700 Fixup last remaining TODOs/questions regarding cwiki URLs and hardcoded TOCs commit add32b39ff051330c99f1ec8bc04cc3c4144a061 Author: Chris Hostetter <hossman@apache.org> Date: Tue May 9 13:47:28 2017 -0700 SOLR-10640: change the html page layout to include an id with teh page shortname (just like the pdf effectively does) and remove the HACK in CheckLinksAndAnchors that faked this only one page had a dup anchor that needed fixed as a result commit 96058f824cdb75e61438c0f6e14085f91f6971a5 Author: Cassandra Targett <ctargett@apache.org> Date: Tue May 9 14:50:17 2017 -0500 SOLR-10290: remove confluence-export dir with tools for conversion commit 07d81790f00e25db28e495e16688cf6d70bfeeb5 Author: Cassandra Targett <ctargett@apache.org> Date: Tue May 9 14:00:49 2017 -0500 SOLR-10296: fix glossary headings; fix callout font used in PDF commit 8436b4050cbde7f62d3f1d9fbee58c63c334eab7 Author: Cassandra Targett <ctargett@apache.org> Date: Tue May 9 09:57:50 2017 -0500 SOLR-10296: missed a few tables + unescaped http urls commit a244450b27e1883acbbe13235990b373ce0e885f Author: Cassandra Targett <ctargett@apache.org> Date: Tue May 9 09:48:01 2017 -0500 SOLR-10296: conversion, table cleanup, last commit commit 029b600d0c21ee140f5a8383c4dc7afe105e107e Author: Cassandra Targett <ctargett@apache.org> Date: Tue May 9 08:20:45 2017 -0500 SOLR-10296: conversion, table cleanups commit 4a0b9b5395f07961ded6ed9e041a8772de690b24 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 21:36:34 2017 -0700 batch remove OLD_CONFLUENCE_ID comments now that all inter-page links have been fixed commit 10152ba8cf96c49f3f2e24acf5fbc99dcb508500 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 21:32:03 2017 -0700 fix broken links/anchors found with improved link checker commit 53f380125d022c99fa9f1200dc433b5f13f6ff87 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 20:29:26 2017 -0700 Fix dup anchors found by improved check-links-and-anchors commit 83b966e2cab5427e6afb2b4e468d4b3e833494a5 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 18:38:12 2017 -0700 SOLR-10640: make CheckLinksAndAnchors smarter now checks for relative links to files/anchors that don't exist in the process of adding that, I realize the duplicat ID checking wasn't accounting for the 'implicit' page.shortname ids that are used in the PDF - so some dups were getting over looked example: suggester.adoc implicitly has a 'suggester' id, but glossary.adoc might explicitly definie a '[[suggester]]' anchor commit db902f5c97c12628cb1ef6534e5bea4fe9bebaa4 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 17:06:04 2017 -0700 beef up TODO with some more details for fixing in future commit e00e2b8d2e3503422eafb045c6bb6a5fee0152ae Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 16:45:36 2017 -0700 remove TODOs added by automated conversion that have already been manually dealt with commit 172bf3963b6c7de4f89b29e6536d5255566b4936 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 16:04:40 2017 -0700 add the (raw) SVG source for some diagrams commit 5105c9bba5d00e79cea9ecd712409f4f05d86356 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 15:59:04 2017 -0700 fix some absolute cwiki.apache.org links, add some TODO questions about anytihng still refering to cwiki.apache.org commit df8c2597599756c5d572685cb5fbed9622748d07 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 14:54:00 2017 -0700 update TODO with some thoughts on the errata URLs commit 218265778182281e2aaea28aac83141f9b6fff6d Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 8 16:25:56 2017 -0500 SOLR-10296: conversion, table cleanup commit 09f53281cc6e7a3117a3d05adf282dfdc649237e Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 8 15:19:30 2017 -0500 SOLR-10296: conversion, table cleanup commit 02c2d45c15241cd325941c9acd253827c7e4e827 Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 8 14:41:38 2017 -0500 SOLR-10296: conversion, table cleanup commit 1c3daa748e222a50b6938fdf92a42bdc19681c91 Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 8 14:10:18 2017 -0500 SOLR-10290: add fonts and rules for inline callouts commit 2953940d2670fc4fad37d1c91ce30fcc2ec12b36 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 12:04:29 2017 -0700 auto check links anytime html site is built commit de663b5f98c8c7a37ef0ab4c570e46f990cd1938 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 12:01:44 2017 -0700 fix anchor/link combo commit 0bfa433d7ac1b0166069986e971588b2095b9d2d Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 11:54:24 2017 -0700 Fix link checker to accept some absolute URLs that aren't valid URIs commit 58c717cfe40e9f8b575d782210d624a5726f112a Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 11:34:12 2017 -0700 Fix some duplicate anchors that were breaking check-links-and-anchors commit 3de6709c9be74bcaca7933f2375d20e2ed54dad0 Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 11:24:20 2017 -0700 replace problematic footnoteref usage w/simple anchors/links commit 33e8df25427276b0496e5f9882756678f0ef3f3c Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 10:35:22 2017 -0700 Fix formatting glitches found via: egrep '' src/*.adoc commit bd805af6354f662af7df6f5d67cee4f3ed6143aa Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 8 11:50:03 2017 -0500 SOLR-10296: conversion, solr-glossary commit ceabf15b458b9b34738b61209d2cf7c218ef76eb Author: Chris Hostetter <hossman@apache.org> Date: Mon May 8 09:49:44 2017 -0700 upgrade asciidoctor-ant = 1.6.0-alpha.3 and fix logging jar dep that was overlooked before commit d05e3a4066fcf7446479a233f26a254c970fb95c Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 8 10:23:43 2017 -0500 SOLR-10296: conversion, remaining letter S minus solr-glossary commit 3f9dc385915c69dce15d224898908e8c25b26c3a Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 19:47:01 2017 -0500 SOLR-10296: conversion, letter S part 1 commit ff9fdcf1fcfb68008064e902f7cca8c776516323 Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 18:50:42 2017 -0500 SOLR-10296: add glossary attribute to CDCR Glossary commit f7859d7f32df43308b1d82914523d90723b415a4 Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 18:46:57 2017 -0500 SOLR-10296: conversion, letter R commit e53c64a3508a2a2fd7c01b085d26f39cf89af03e Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 17:18:47 2017 -0500 SOLR-10296: conversion, letter Q commit 53b3eaac6073e817dd9cc85d9b6f51cc873fb2a7 Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 17:00:47 2017 -0500 SOLR-10296: conversion, letter P commit 26a571e0bf22de52caced8d72f7badb1afbcb0b4 Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 15:47:29 2017 -0500 SOLR-10296: conversion, letter O commit 0ffdf3d543dc009caaf10b616cc9ee8c84ef2dac Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 14:48:04 2017 -0500 SOLR-10296: conversion, letters M + N commit 7d7fb52ab134a1f4227cab448c5a82890ef4e855 Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 14:03:53 2017 -0500 SOLR-10296: conversion, letter L; some other cleanups commit c4b547c55a0461f8a7d4191a0e233d8a59d113de Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 11:59:51 2017 -0500 SOLR-10296: conversion, letters J + K commit d77278df6bb7d0639c18d2cc6f8cee3ff9196c96 Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 11:12:50 2017 -0500 SOLR-10296: conversion, letter I commit 8916dd05af036ac58bc0754c994e62e45c808249 Author: Cassandra Targett <ctargett@apache.org> Date: Sun May 7 10:43:43 2017 -0500 SOLR-10296: fix tables that declare specific column percentages commit 7f1990c56950497a086fdfcbbccc5f3e16684cfd Author: Cassandra Targett <ctargett@apache.org> Date: Sat May 6 14:38:26 2017 -0500 SOLR-10296: conversion, letter H commit a967d5a392e3cda08a6699dd751f597b8473f71e Author: Cassandra Targett <ctargett@apache.org> Date: Sat May 6 13:58:00 2017 -0500 SOLR-10296: conversion, letter G + some heading level cleanups commit f060417a8345de17fa2c1c3e210da46ae5b6599e Author: Cassandra Targett <ctargett@apache.org> Date: Sat May 6 08:09:41 2017 -0500 SOLR-10296: conversion, done with letter F commit ccf5dd28af555a35b3938ad8948afa6822887fbc Author: Cassandra Targett <ctargett@apache.org> Date: Fri May 5 22:42:37 2017 -0500 SOLR-10296: conversion, letter F, part 1 commit c0cc260ced47238e1342aa62ff9263ec48fe5f02 Author: Cassandra Targett <ctargett@apache.org> Date: Fri May 5 21:19:55 2017 -0500 SOLR-10296: conversion, letter E commit 645908e293298ccce31988a717dfb122c3bf3611 Author: Cassandra Targett <ctargett@apache.org> Date: Fri May 5 20:42:53 2017 -0500 SOLR-10296: content conversion, letter D commit 7905416153ff5726d97fde2774f13224c42759c1 Author: Chris Hostetter <hossman@apache.org> Date: Fri May 5 17:16:57 2017 -0700 manual cleanup of the T files commit 287ffe43c78e1cdccffb0d14c6f6cf78a0141b5e Author: Cassandra Targett <ctargett@apache.org> Date: Fri May 5 18:51:16 2017 -0500 SOLR-10296: conversion - C's commit 7c2e7ce8c3d5f66b476e8167354f581667bbb88d Author: Chris Hostetter <hossman@apache.org> Date: Fri May 5 15:07:15 2017 -0700 manual cleanup of the remaining u files commit bbe60af21393f6c8b59bdcae9c79a28a10545aa0 Author: Cassandra Targett <ctargett@apache.org> Date: Fri May 5 15:52:23 2017 -0500 SOLR-10290: content conversion: B's and some C's commit c7361afb83f3ae477c9f9a9fcbd901157735d390 Author: Chris Hostetter <hossman@apache.org> Date: Fri May 5 11:21:34 2017 -0700 manual cleanup of using-* pages commit 4697d96ca4bb0c6273865c88582907d2995c7d23 Author: Chris Hostetter <hossman@apache.org> Date: Fri May 5 10:30:46 2017 -0700 manual cleanup of w pages commit bf84e407e6111a694f689cd2a7fc34c1376d9b17 Author: Chris Hostetter <hossman@apache.org> Date: Fri May 5 10:09:26 2017 -0700 manual cleanup of w-z commit adf8959db733f516f81b5cb1b43e0b13218c4f53 Author: Cassandra Targett <ctargett@apache.org> Date: Fri May 5 12:01:52 2017 -0500 SOLR-10290: content conversion, letter A commit 4dca53bf59e138b90db7b89c7cf9ca37fcadbe46 Author: Cassandra Targett <ctargett@apache.org> Date: Fri May 5 09:43:37 2017 -0500 SOLR-10290: commit changed pages since last Confluence export commit 72e0d34bfdc3dbd8d0af51f1b55904d3b82fb215 Author: Cassandra Targett <ctargett@apache.org> Date: Wed May 3 13:35:24 2017 -0500 SOLR-10295: Use index.html; fix pub process; add nav for PDFs commit 737464ab897a424c2f6677e13d794ab136043de3 Author: Cassandra Targett <ctargett@apache.org> Date: Wed May 3 13:34:14 2017 -0500 SOLR-10295: Use index.html; add nav for PDF versions commit 7bd15b902a78756e333aa9db8363d7178aa53ac3 Author: Cassandra Targett <ctargett@apache.org> Date: Wed May 3 08:38:44 2017 -0500 SOLR-10295: typos commit 35e1acbfb537cf5540f5e576092664c8783fb29e Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 1 16:08:57 2017 -0500 SOLR-10593: remove external link icons in comment threads commit 1664ab87cf09f12a6f547425f05f4bd30f1af735 Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 1 15:36:01 2017 -0500 SOLR-10593: add comments.css to head include file so comments are styled properly commit 276db77bf30e3c23f86ac8d5107014f84d2b7d16 Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 1 14:55:06 2017 -0500 SOLR-10593: change to "solr-refguide" site for comments commit 42871f86bdaa12cc5fe68670a1145ee36d5571f4 Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 1 14:33:48 2017 -0500 SOLR-10301: small edits to how to write in asciidoc commit 8b5f9693354b9019caf43c98badaf496ab9361e2 Author: Cassandra Targett <ctargett@apache.org> Date: Mon May 1 11:26:08 2017 -0500 SOLR-10295: Add PDF instructions to publish guide commit e375e250071a793a647926e1f0ab40d1e26031a0 Author: Cassandra Targett <ctargett@apache.org> Date: Fri Apr 28 15:06:13 2017 -0500 SOLR-10295: missed some edits commit 43e70c3f8f2c9612de359b29ef7127a81672ac5c Author: Cassandra Targett <ctargett@apache.org> Date: Fri Apr 28 14:36:29 2017 -0500 SOLR-10295: add re-publish process commit f511fbf45a092f04829db14b24e6737e6f8a70fa Author: Cassandra Targett <ctargett@apache.org> Date: Fri Apr 28 08:17:40 2017 -0500 SOLR-10295: update publication process commit 49c92bf03deda0e29c9a0e774ce05cf0bfbe71c5 Author: Steve Rowe <sarowe@apache.org> Date: Thu Apr 27 14:24:38 2017 -0400 SOLR-10290: Added pygments.rb to list of required gems. Fixed build directories. commit a56e8c9b73c1a44b791db3c4b9f5840fa32b73c2 Author: Cassandra Targett <ctargett@apache.org> Date: Wed Apr 26 15:55:15 2017 -0500 SOLR-10295: clarify some instructions; add more TODOs commit 81c8215ce5dad3d5f06ad07fa9314133b9d0a542 Author: Cassandra Targett <ctargett@apache.org> Date: Wed Apr 26 14:13:45 2017 -0500 SOLR-10295: strawman process for publishing HTML version commit 73148be0baab123b93953f98d69d3b4517842f03 Author: Cassandra Targett <ctargett@apache.org> Date: Thu Apr 20 14:35:53 2017 -0500 SOLR-10290: update raw content files commit 201d238ad61dfbbf5d7feb7d1750256fb67f39d3 Author: Cassandra Targett <ctargett@apache.org> Date: Wed Apr 5 17:47:56 2017 -0500 SOLR-10290: remove stray pipe in feed.xml commit d44e9ed8727844eaae4e7cba98d3f1e6639c7b87 Author: Cassandra Targett <ctargett@apache.org> Date: Wed Apr 5 15:14:15 2017 -0500 SOLR-10422: Remove license attribution for now-unused Raleway font commit 79e103756efc214c1ff3fca6b58c7621e077229f Author: Cassandra Targett <ctargett@apache.org> Date: Wed Apr 5 14:49:16 2017 -0500 SOLR-10290: Fix font-family references for consistency commit 34829c59cfbf615c40e0717a59f6d25d5290a845 Author: Cassandra Targett <ctargett@apache.org> Date: Wed Apr 5 14:47:09 2017 -0500 SOLR-10422: remove pdf/fonts directory; change HTML format to use Inconsolata & Noto Sans; other font-related dir cleanups commit 6472b196372b387a43920781d3b2aad1d1d47544 Author: Cassandra Targett <ctargett@apache.org> Date: Tue Apr 4 15:52:58 2017 -0500 SOLR-10290: get rid of unnecessary duplicate font directory commit c6d29e33e41435824e217acf66a02ea3b14be7de Author: Cassandra Targett <ctargett@apache.org> Date: Tue Apr 4 15:29:07 2017 -0500 SOLR-10298: replace Noto Serif with Noto Sans, to reduce font size and line height and keep readability commit d14977553f2597d84f907011a5c7275bfb79be77 Author: Chris Hostetter <hossman@apache.org> Date: Fri Mar 31 11:09:26 2017 -0700 SOLR-10298: new pdfbox based tool to reduce PDF size (via FLATE_DECODE) commit ec2cbb3eee23d94619e7c9e7e2b45fb77da61394 Author: Cassandra Targett <ctargett@apache.org> Date: Fri Mar 31 11:24:52 2017 -0500 SOLR-10290: Fix feed.xml in case we want to have it in the online site; ensure URL is right everywhere commit 5c29efc41a0213908f1b0ebd6d40c09927f6832f Author: Cassandra Targett <ctargett@apache.org> Date: Fri Mar 31 11:23:34 2017 -0500 SOLR-10290: Clean up _config.yml.template: update description; remove unused settings; move comments to above the thing they describe commit 8071d3f4a2aac2b8a3b2e38bc1896f507a7e7763 Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 30 15:04:32 2017 -0500 Small edits to PDF meta-docs commit 3b592f1be5f803c33fc170f6bf54b78e0597bef3 Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 30 15:03:53 2017 -0500 SOLR-10346: Remove topnav.yml data file & hard code top navigation into topnav template commit 9bb9568b65cfec495d3effb570a65ba84fd2d9a3 Author: Cassandra Targett <ctargett@apache.org> Date: Tue Mar 28 15:58:47 2017 -0500 SOLR-10300: add docs on building PDF and HTML versions commit a214ff8c79a19f1babbea530caf2536a7e7f425e Author: Cassandra Targett <ctargett@apache.org> Date: Tue Mar 28 15:58:00 2017 -0500 SOLR-10298: remove some PDF build params that are not used commit 017b890f7cdde1e1c7ec4b73ed8943db7eff370d Author: Chris Hostetter <hossman@apache.org> Date: Wed Mar 22 11:22:19 2017 -0700 SOLR-10342: use ivy-version.properties and std build paths for ref-guide commit 183bfbeea69c7f1c9d481440a7ccac21b18ee1fd Author: Chris Hostetter <hossman@apache.org> Date: Wed Mar 22 10:12:26 2017 -0700 SOLR-10342: license headers commit 85bb18003830846b036751ae3137a1efacaaa79d Author: Chris Hostetter <hossman@apache.org> Date: Wed Mar 22 10:08:17 2017 -0700 SOLR-10342: use common-build.xml for variables commit 853c4ae0467607d508d8b3a13b60ad4414ef7234 Author: Cassandra Targett <ctargett@apache.org> Date: Mon Mar 20 16:17:45 2017 -0500 SOLR-10300, SOLR-10301: add meta-docs directory and first pass at Asciidoc syntax & tools commit 0e6f187daed4e571c6fa7f438a5ce7e9fd53ea54 Author: Cassandra Targett <ctargett@apache.org> Date: Mon Mar 20 10:20:31 2017 -0500 Remove errant .DS_Store file commit 46251028f4ca21305528b4a384aba64dbfc4d2cf Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 13:07:50 2017 -0500 SOLR-10290: Add remaining files needed for complete builds commit 08e6c9e99e426d3269af9cd3bb9697fd78c2da81 Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 13:03:45 2017 -0500 SOLR-10290: Really last batch of images commit 66e7d9be95cde5a51716128ba1a33571132ff76d Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 13:00:58 2017 -0500 SOLR-10290: Add last batch of images commit 7012478b8bc1118f9acc0c1e34cc9b735a54e6a3 Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 13:00:11 2017 -0500 SOLR-10290: Add more images commit fca9f3681452466941d7107d8813ba0c1e04edbb Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 12:59:10 2017 -0500 SOLR-10290: Add more images commit 7e8fd9ac0b87a5bdbcc589ec9c6f747561e9e7bc Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 12:57:04 2017 -0500 SOLR-10290: Add some more images commit 000fe8090da9c1d953fb0927b612a76de4f9ccf0 Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 12:55:58 2017 -0500 SOLR-10290: Add some of the images commit 45a148a74805ae4723edab8230b1d2ba1f62284b Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 12:28:21 2017 -0500 SOLR-10290: Add .adoc files commit 2a6997b027ffa4cacab55ac8eb66d178b131a2fb Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 12:24:58 2017 -0500 SOLR-10290: Add config.yml template for Jekyll build commit 71001667e2781cedee3bc92f985b24fc3c99c3e3 Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 12:22:35 2017 -0500 SOLR-10290: Add files for PDF build commit 659494333e2f0aa1700692e082a6a4f94e07720e Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 12:12:12 2017 -0500 SOLR-10290: Add files needed for Jekyll HTML build commit ec324b294ce858733dd014399a27ccb2cb513def Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 11:51:06 2017 -0500 SOLR-10290: Add ivy and build files, etc. commit 8736246ee51e8fbf08ee8d08a1dc22cdeba50d97 Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 11:49:12 2017 -0500 SOLR-10290: Add Confluence conversion tools commit e825f0a7501802bd79004ed0ac44b62eba59954d Author: Cassandra Targett <ctargett@apache.org> Date: Thu Mar 16 10:54:18 2017 -0500 SOLR-10290: Initial commit of ref guide tools
This commit is contained in:
parent
69783f6403
commit
95968c69fd
|
@ -40,6 +40,7 @@ com.sun.jersey.version = 1.9
|
|||
/com.sun.mail/javax.mail = 1.5.1
|
||||
|
||||
/com.tdunning/t-digest = 3.1
|
||||
/com.vaadin.external.google/android-json = 0.0.20131108.vaadin1
|
||||
/commons-beanutils/commons-beanutils = 1.8.3
|
||||
/commons-cli/commons-cli = 1.2
|
||||
/commons-codec/commons-codec = 1.10
|
||||
|
@ -145,7 +146,7 @@ org.apache.hadoop.version = 2.7.2
|
|||
/org.apache.htrace/htrace-core = 3.2.0-incubating
|
||||
|
||||
# The httpcore version is often different from the httpclient and httpmime versions,
|
||||
# so the httpcore version value should not share the same symbolic name with them.
|
||||
# so the httpcore version value should not share the same symbolic name with them.
|
||||
/org.apache.httpcomponents/httpclient = 4.4.1
|
||||
/org.apache.httpcomponents/httpcore = 4.4.1
|
||||
/org.apache.httpcomponents/httpmime = 4.4.1
|
||||
|
@ -187,6 +188,11 @@ org.apache.uima.version = 2.3.1
|
|||
/org.apache.velocity/velocity-tools = 2.0
|
||||
/org.apache.xmlbeans/xmlbeans = 2.6.0
|
||||
/org.apache.zookeeper/zookeeper = 3.4.10
|
||||
|
||||
# v1.6.0-alpha.3 of asciidoctor-ant includes asciidoctorj-pdf 1.5.0-alpha.15,
|
||||
# which is the same as asciidoctor-pdf 1.5.0-alpha.15
|
||||
/org.asciidoctor/asciidoctor-ant = 1.6.0-alpha.3
|
||||
|
||||
/org.aspectj/aspectjrt = 1.8.0
|
||||
|
||||
org.bouncycastle.version = 1.45
|
||||
|
@ -236,6 +242,8 @@ org.gagravarr.vorbis.java.version = 0.8
|
|||
/org.gagravarr/vorbis-java-core = ${org.gagravarr.vorbis.java.version}
|
||||
/org.gagravarr/vorbis-java-tika = ${org.gagravarr.vorbis.java.version}
|
||||
|
||||
/org.jsoup/jsoup = 1.8.2
|
||||
|
||||
/org.locationtech.spatial4j/spatial4j = 0.6
|
||||
|
||||
/org.mockito/mockito-core = 2.6.2
|
||||
|
@ -262,6 +270,7 @@ org.slf4j.version = 1.7.7
|
|||
/org.slf4j/jul-to-slf4j = ${org.slf4j.version}
|
||||
/org.slf4j/slf4j-api = ${org.slf4j.version}
|
||||
/org.slf4j/slf4j-log4j12 = ${org.slf4j.version}
|
||||
/org.slf4j/slf4j-simple = ${org.slf4j.version}
|
||||
|
||||
/org.tukaani/xz = 1.5
|
||||
/rome/rome = 1.0
|
||||
|
@ -270,4 +279,3 @@ ua.net.nlp.morfologik-ukrainian-search.version = 3.7.5
|
|||
/ua.net.nlp/morfologik-ukrainian-search = ${ua.net.nlp.morfologik-ukrainian-search.version}
|
||||
|
||||
/xerces/xercesImpl = 2.9.1
|
||||
|
||||
|
|
|
@ -656,6 +656,9 @@
|
|||
<ant dir="test-framework" target="resolve" inheritall="false">
|
||||
<propertyset refid="uptodate.and.compiled.properties"/>
|
||||
</ant>
|
||||
<ant dir="solr-ref-guide" target="resolve" inheritall="false">
|
||||
<propertyset refid="uptodate.and.compiled.properties"/>
|
||||
</ant>
|
||||
<contrib-crawl target="resolve"/>
|
||||
</sequential>
|
||||
</target>
|
||||
|
|
|
@ -0,0 +1 @@
|
|||
fa26d351fe62a6a17f5cda1287c1c6110dec413f
|
|
@ -0,0 +1,202 @@
|
|||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
|
@ -0,0 +1 @@
|
|||
a7068544963ed46839c8352eddd87271fa93b967
|
|
@ -0,0 +1,202 @@
|
|||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
|
@ -0,0 +1,5 @@
|
|||
Apache [asciidoctor-ant]
|
||||
Copyright [2013] The Apache Software Foundation
|
||||
|
||||
This product includes software developed at
|
||||
The Apache Software Foundation (http://www.apache.org/).
|
|
@ -0,0 +1 @@
|
|||
64238922c4006c3d0a9951c4c03983ecc6a1e1a0
|
|
@ -0,0 +1,21 @@
|
|||
The MIT License
|
||||
|
||||
© 2009-2017, Jonathan Hedley <jonathan@hedley.net>
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
|
@ -0,0 +1 @@
|
|||
8095d0b9f7e0a9cd79a663c740e0f8fb31d0e2c8
|
|
@ -0,0 +1,27 @@
|
|||
= Solr Ref Guide
|
||||
|
||||
This is the source for the Solr Reference Guide.
|
||||
|
||||
Raw content is stored in Asciidoc (`.adoc`) formated files in the `src/` directory.
|
||||
|
||||
These files are processed with AsciiDoctor in 2 different ways:
|
||||
|
||||
* Via 'Jekyll' to build an HTML browsable version of the Ref Guide
|
||||
** Prerequisites: `Ruby` and the following gems must be installed:
|
||||
*** `jekyll`
|
||||
*** `jekyll-asciidoc`
|
||||
*** `pygments.rb`
|
||||
* Via `asciidoctor-ant` to build the officially released PDF version of the Ref Guide
|
||||
** Prerequisites: None (except for those required to use the Lucene/Solr build: Java, Ant)
|
||||
|
||||
For details on building the ref guide, see `ant -p`.
|
||||
|
||||
Key directories to be aware of:
|
||||
|
||||
* `src` - where all human edited `*.adoc` files realted to the Guide live, as well as various configuration, theme, and template files.
|
||||
* `tools` - custom Java code for parsing metadata in our `src/*.adoc` files to produce some `_data/` files for site & pdf navigation purposes.
|
||||
* `../build/solr-ref-guide/content` - a copy of the `src` dir generated by ant where:
|
||||
** `*.template` files are processed to replace ant properties with their runtime values
|
||||
** some `../build/solr-ref-guide/content/_data` files are generated by our java tools based header attributes from each of the `*.adoc` files
|
||||
* `../build/solr-ref-guide/html-site` - HTML generated version of the ref guide
|
||||
* `../build/solr-ref-guide/apache-solr-ref-guide-X.Y.pdf` - PDF generated version of the ref guide
|
|
@ -0,0 +1,204 @@
|
|||
<?xml version="1.0"?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
-->
|
||||
<project name="solr-ref-guide" default="default" xmlns:asciidoctor="antlib:org.asciidoctor.ant" >
|
||||
|
||||
<import file="../common-build.xml"/>
|
||||
|
||||
<!-- properties to use in our docs -->
|
||||
<loadresource property="solr-docs-version">
|
||||
<propertyresource name="version.base"/>
|
||||
<filterchain>
|
||||
<tokenfilter>
|
||||
<filetokenizer/>
|
||||
<replaceregex pattern="^(\d+\.\d+)(|\..*)$" replace="\1" flags="s"/>
|
||||
</tokenfilter>
|
||||
</filterchain>
|
||||
</loadresource>
|
||||
<loadresource property="solr-docs-version-path">
|
||||
<propertyresource name="solr-docs-version"/>
|
||||
<filterchain>
|
||||
<tokenfilter>
|
||||
<filetokenizer/>
|
||||
<replaceregex pattern="^(\d+)\.(\d+)(|\..*)$" replace="\1_\2_0" flags="s"/>
|
||||
</tokenfilter>
|
||||
</filterchain>
|
||||
</loadresource>
|
||||
<property name="solr-javadocs" value="https://lucene.apache.org/solr/${solr-docs-version-path}/" />
|
||||
<property name="lucene-javadocs" value="https://lucene.apache.org/core/${solr-docs-version-path}/" />
|
||||
|
||||
<property name="build.content.dir" location="${build.dir}/content" />
|
||||
<property name="main-page" value="index" />
|
||||
<property name="pdf-filename" value="apache-solr-ref-guide-${solr-docs-version}.pdf" />
|
||||
|
||||
<!-- ====== TOOLS FOR GENERATING/VALIDATING BITS OF THE SITE / PDF ======= -->
|
||||
<property name="tools-jar-name" value="solr-ref-guide-tools.jar" />
|
||||
<path id="tools-compile-classpath">
|
||||
<fileset dir="lib">
|
||||
<include name="**/*.jar"/>
|
||||
<exclude name="**/${tools-jar-name}" />
|
||||
</fileset>
|
||||
</path>
|
||||
<path id="tools-run-classpath">
|
||||
<fileset dir="lib">
|
||||
<include name="**/*.jar"/>
|
||||
</fileset>
|
||||
<fileset dir="${build.dir}">
|
||||
<include name="**/*.jar"/>
|
||||
</fileset>
|
||||
</path>
|
||||
|
||||
<target name="clean">
|
||||
<delete dir="${build.dir}"/>
|
||||
</target>
|
||||
|
||||
<target name="build-tools-jar" depends="resolve" description="Builds the custom java tools use use for generating some data files from page metdata">
|
||||
<mkdir dir="${build.dir}/classes"/>
|
||||
<javac debug="yes"
|
||||
debuglevel="source,lines,vars"
|
||||
destdir="${build.dir}/classes"
|
||||
includeantruntime="false">
|
||||
<compilerarg value="-Xlint:all"/>
|
||||
<classpath refid="tools-compile-classpath"/>
|
||||
<src path="tools/"/>
|
||||
</javac>
|
||||
<jar destfile="${build.dir}/${tools-jar-name}">
|
||||
<fileset dir="${build.dir}/classes"
|
||||
includes="**/*.class"/>
|
||||
</jar>
|
||||
</target>
|
||||
|
||||
<target name="build-init" description="Prepares the build's 'content' dir, copying over src files and transforming *.template files in the process">
|
||||
<delete dir="${build.content.dir}" />
|
||||
<mkdir dir="${build.content.dir}" />
|
||||
<echo>Copying all non template files from src ...</echo>
|
||||
<copy todir="${build.content.dir}">
|
||||
<fileset dir="src">
|
||||
<exclude name="**/*.template"/>
|
||||
</fileset>
|
||||
</copy>
|
||||
<echo>Copy (w/prop replacement) any template files from src...</echo>
|
||||
<copy todir="${build.content.dir}">
|
||||
<fileset dir="src">
|
||||
<include name="**/*.template"/>
|
||||
</fileset>
|
||||
<mapper type="glob" from="*.template" to="*"/>
|
||||
<filterchain>
|
||||
<expandproperties/>
|
||||
</filterchain>
|
||||
</copy>
|
||||
</target>
|
||||
|
||||
<target name="build-nav-data-files" depends="build-init,build-tools-jar" description="creates nav based data files needed by both the html and pdf artifacts">
|
||||
<mkdir dir="${build.content.dir}/_data"/>
|
||||
<java classname="BuildNavAndPDFBody"
|
||||
failonerror="true"
|
||||
fork="true">
|
||||
<classpath refid="tools-run-classpath"/>
|
||||
<arg value="${build.content.dir}"/>
|
||||
<arg value="${main-page}"/>
|
||||
</java>
|
||||
</target>
|
||||
|
||||
<target name="check-links-and-anchors" depends="build-init,build-tools-jar" description="Parse the HTML site files to check for problematic links or anchors">
|
||||
<java classname="CheckLinksAndAnchors"
|
||||
failonerror="true"
|
||||
fork="true">
|
||||
<classpath refid="tools-run-classpath"/>
|
||||
<arg value="${build.dir}/html-site"/>
|
||||
</java>
|
||||
</target>
|
||||
|
||||
<!-- ====== PDF Build ======= -->
|
||||
<target name="build-pdf" depends="-build-raw-pdf,-reduce-pdf-size" description="Builds a PDF">
|
||||
<echo>Finished Building ${build.dir}/${pdf-filename}</echo>
|
||||
</target>
|
||||
<target name="-build-raw-pdf"
|
||||
depends="build-nav-data-files,resolve">
|
||||
<mkdir dir="${build.dir}/pdf-tmp"/>
|
||||
<taskdef uri="antlib:org.asciidoctor.ant" resource="org/asciidoctor/ant/antlib.xml"
|
||||
classpathref="tools-run-classpath"/>
|
||||
<asciidoctor:convert
|
||||
sourceDirectory="${build.content.dir}/pdf"
|
||||
sourceDocumentName="SolrRefGuide-all.adoc"
|
||||
baseDir="${build.content.dir}"
|
||||
outputDirectory="${build.dir}/pdf-tmp"
|
||||
backend="pdf"
|
||||
extensions="adoc"
|
||||
sourceHighlighter="coderay"
|
||||
imagesDir="${build.content.dir}"
|
||||
doctype="book"
|
||||
safemode="unsafe">
|
||||
<attribute key="icons" value="font" />
|
||||
<attribute key="icon-set" value="fa" />
|
||||
<attribute key="pdf-stylesDir" value="./pdf/themes"/>
|
||||
<attribute key="pdf-style" value="refguide"/>
|
||||
<attribute key="pdf-fontsDir" value="./fonts"/>
|
||||
<attribute key="figure-caption!" value='' />
|
||||
<attribute key="idprefix" value='' />
|
||||
<attribute key="idseparator" value='-' />
|
||||
<!-- attributes used in adoc files -->
|
||||
<!-- NOTE: If you add any attributes here for use in adoc files, you almost certainly need to also add
|
||||
them to the _config.yml.template file for building the jekyll site as well
|
||||
-->
|
||||
<attribute key="solr-docs-version" value="${solr-docs-version}" />
|
||||
<attribute key="solr-javadocs" value="${solr-javadocs}" />
|
||||
<attribute key="lucene-javadocs" value="${lucene-javadocs}" />
|
||||
<attribute key="build-date" value="${DSTAMP}" />
|
||||
<attribute key="build-year" value="${current.year}" />
|
||||
</asciidoctor:convert>
|
||||
<move file="${build.dir}/pdf-tmp/SolrRefGuide-all.pdf" tofile="${build.dir}/pdf-tmp/RAW-${pdf-filename}" />
|
||||
</target>
|
||||
<target name="-reduce-pdf-size" depends="build-init,build-tools-jar">
|
||||
<java classname="ReducePDFSize"
|
||||
failonerror="true"
|
||||
fork="true">
|
||||
<classpath refid="tools-run-classpath"/>
|
||||
<arg value="${build.dir}/pdf-tmp/RAW-${pdf-filename}"/>
|
||||
<arg value="${build.dir}/${pdf-filename}"/>
|
||||
</java>
|
||||
</target>
|
||||
|
||||
|
||||
|
||||
<!-- ======= HTML Site Build =======
|
||||
Builds site with Jekyll.
|
||||
This (for now) assumes that Jekyll (http://jekyllrb.com) is installed locally. -->
|
||||
<target name="build-site"
|
||||
depends="-build-site,check-links-and-anchors"
|
||||
description="Builds an HTML Site w/Jekyll and verifies the anchors+links are valid" >
|
||||
<echo>Ready to browse site: ${build.dir}/html-site/${main-page}.html</echo>
|
||||
</target>
|
||||
<target name="-build-site"
|
||||
depends="build-init,build-nav-data-files"
|
||||
description="Builds an HTML Site w/Jekyll">
|
||||
<echo>Running Jekyll...</echo>
|
||||
<exec executable="jekyll" dir="${build.content.dir}">
|
||||
<arg value="build"/>
|
||||
</exec>
|
||||
</target>
|
||||
|
||||
<target name="default"
|
||||
description="Builds both a PDF and HTML versions of the ref guide"
|
||||
depends="build-pdf,build-site">
|
||||
<echo>PDF: ${build.dir}/${pdf-filename}</echo>
|
||||
<echo>SITE: ${build.dir}/html-site/${main-page}.html</echo>
|
||||
</target>
|
||||
|
||||
</project>
|
|
@ -0,0 +1,34 @@
|
|||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
-->
|
||||
<ivy-module version="2.0">
|
||||
<info organisation="org.apache.solr" module="ref-guide-tools"/>
|
||||
<configurations defaultconfmapping="compile->master">
|
||||
<conf name="compile" transitive="false" />
|
||||
</configurations>
|
||||
<dependencies>
|
||||
<dependency org="org.asciidoctor" name="asciidoctor-ant" rev="${/org.asciidoctor/asciidoctor-ant}" conf="compile" />
|
||||
<dependency org="com.vaadin.external.google" name="android-json" rev="${/com.vaadin.external.google/android-json}" conf="compile" />
|
||||
<dependency org="org.jsoup" name="jsoup" rev="${/org.jsoup/jsoup}" conf="compile" />
|
||||
<dependency org="org.apache.pdfbox" name="pdfbox" rev="${/org.apache.pdfbox/pdfbox}" conf="compile"/>
|
||||
<dependency org="org.slf4j" name="jcl-over-slf4j" rev="${/org.slf4j/jcl-over-slf4j}" conf="compile"/>
|
||||
<dependency org="org.slf4j" name="slf4j-api" rev="${/org.slf4j/slf4j-api}" conf="compile"/>
|
||||
<dependency org="org.slf4j" name="slf4j-simple" rev="${/org.slf4j/slf4j-simple}" conf="compile"/>
|
||||
<dependency org="log4j" name="log4j" rev="${/log4j/log4j}" conf="compile"/>
|
||||
</dependencies>
|
||||
</ivy-module>
|
|
@ -0,0 +1,223 @@
|
|||
= Asciidoc syntax
|
||||
:toc:
|
||||
|
||||
The definitive manual on Asciidoc syntax is in the http://asciidoctor.org/docs/user-manual/[Asciidoctor User Manual]. To help people get started, however, here is a simpler cheat sheet.
|
||||
|
||||
== Basic syntax
|
||||
|
||||
=== Bold
|
||||
|
||||
Put asterisks around text to make it *bold*.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#bold-and-italic
|
||||
|
||||
=== Italics
|
||||
|
||||
Use underlines on either side of a string to put text into _italics_.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#bold-and-italic
|
||||
|
||||
=== Headings
|
||||
|
||||
Equal signs (`=`) are used for heading levels. Each equal sign is a level. Each page can *only* have one top level. Levels should be appropriately nested.
|
||||
|
||||
Validation occurs to ensure that level 3s are preceded by level 2s, level 4s are preceded by level 3s, etc. Including out-of-sequence heading levels (such as a level 3 then a level 5) will not fail the build, but will produce an error.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#sections
|
||||
|
||||
=== Code Examples
|
||||
|
||||
Use backticks ``` for text that should be monospaced, such as a code or class name in the body of a paragraph.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#mono
|
||||
|
||||
Longer code examples can be separated from text with `source` blocks. These allow defining the syntax being used so the code is properly highlighted.
|
||||
|
||||
[source]
|
||||
.Example Source Block
|
||||
----
|
||||
[source,xml]
|
||||
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
|
||||
----
|
||||
|
||||
If your code block will include line breaks, put 4 hyphens (`----`) before and after the entire block.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#source-code-blocks
|
||||
|
||||
=== Block Titles
|
||||
|
||||
Titles can be added to most blocks (images, source blocks, tables, etc.) by simply prefacing the title with a period (`.`). For example, to add a title to the source block example above:
|
||||
|
||||
[source]
|
||||
----
|
||||
[source,xml]
|
||||
.Example ID field
|
||||
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
|
||||
----
|
||||
|
||||
== Links
|
||||
|
||||
=== Link to Sites on the Internet
|
||||
When converting content to HTML or PDF, Asciidoctor will automatically render many link types (such as `http:` and `mailto:`) without any additional syntax.
|
||||
|
||||
However, you can add a name to a link by adding the URI followed by square brackets:
|
||||
|
||||
[source]
|
||||
http://lucene.apache.org/solr[Solr Website]
|
||||
|
||||
=== Link to Other Pages/Sections of the Guide
|
||||
A warning up front, linking to other pages can be a little bit painful.
|
||||
|
||||
To link to an anchor (or heading) on the _same page_, you can simply use double angle brackets (`<< >>`) around the anchor/heading/section name. Note that any heading (aka section title) that starts with equal signs is automatically an anchor.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#internal-cross-references
|
||||
|
||||
To link to another page or even a heading on _another page_, you use a similar syntax, but you must refer to the full filename and refer to the section title you want to link to by changing it to lower case and including hyphens between each word.
|
||||
|
||||
For example, if I want to link from one page to the Schema API page's section "Modify the Schema", I need to create a link that looks like this:
|
||||
|
||||
[source]
|
||||
For more information about modifying the schema with the API, see the section <<schema-api.adoc#modify-the-schema>>.
|
||||
|
||||
You can add text to appear by adding it after the file name and section reference, as in:
|
||||
|
||||
[source]
|
||||
<<schema-api.adoc#modify-the-schema,Modify the Schema>>
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#inter-document-cross-references
|
||||
|
||||
== Item Lists
|
||||
|
||||
Asciidoc supports three types of lists:
|
||||
|
||||
* Unordered lists
|
||||
* Ordered lists
|
||||
* Labeled lists
|
||||
|
||||
Each type of list can be mixed with the other types. So, you could have an ordered list inside a labeled list if necessary.
|
||||
|
||||
=== Unordered Lists
|
||||
Simple bulleted lists need each line to start with an asterisk (`*`). It should be the first character of the line, and be followed by a space.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#unordered-lists
|
||||
|
||||
=== Ordered Lists
|
||||
Numbered lists need each line to start with a period (`.`). It should be the first character of the line, and be followed by a space.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#ordered-lists
|
||||
|
||||
=== Labeled Lists
|
||||
These are like question & answer lists or glossary definitions. Each line should start with the list item followed by double colons (`::`), then a space or new line.
|
||||
|
||||
Labeled lists can be nested by adding an additional colon (such as `:::`, etc.).
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#labeled-list
|
||||
|
||||
== Images
|
||||
|
||||
There are two ways to include an image: inline or as a block.
|
||||
|
||||
Inline images are those where text will flow around the image. Block images are those that appear on their own line, set off from any other text on the page.
|
||||
|
||||
Both approaches use the `image` tag before the image filename, but the number of colons after `image` define if it is inline or a block. Inline images use one colon (`image:`), while block images use two colons (`image::`).
|
||||
|
||||
Block images automatically include a caption label and a number (such as `Figure 1`). If a block image includes a title, it will be included as the text of the caption.
|
||||
|
||||
Optional attributes allow you to set the alt text, the size of the image, if it should be a link, float and alignment.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#images
|
||||
|
||||
== Tables
|
||||
|
||||
Tables can be complex, but it is pretty easy to make a basic table that fits most needs.
|
||||
|
||||
=== Basic Tables
|
||||
The basic structure of a table is similar to Markdown, with pipes (`|`) delimiting columns between rows:
|
||||
|
||||
[source]
|
||||
----
|
||||
|===
|
||||
| col 1 row 1 | col 2 row 1|
|
||||
| col 1 row 2 | col 2 row 2|
|
||||
|===
|
||||
----
|
||||
|
||||
Note the use of `|===` at the start and end. For basic tables that's not exactly required, but it does help to delimit the start and end of the table in case you accidentally introduce (or maybe prefer) spaces between the rows.
|
||||
|
||||
=== Header Rows
|
||||
To add a header to a table, you need only set the `header` attribute at the start of the table:
|
||||
|
||||
[source]
|
||||
----
|
||||
[options="header"]
|
||||
|===
|
||||
| header col 1 | header col 2|
|
||||
| col 1 row 1 | col 2 row 1|
|
||||
| col 1 row 2 | col 2 row 2|
|
||||
|===
|
||||
----
|
||||
|
||||
=== Defining Column Styles
|
||||
If you need to define specific styles to all rows in a column, you can do so with the attributes.
|
||||
|
||||
This example will center all content in all rows:
|
||||
|
||||
[source]
|
||||
----
|
||||
[cols="2*^" options="header"]
|
||||
|===
|
||||
| header col 1 | header col 2|
|
||||
| col 1 row 1 | col 2 row 1|
|
||||
| col 1 row 2 | col 2 row 2|
|
||||
|===
|
||||
----
|
||||
|
||||
Alignments or any other styles can be applied only to a specific column. For example, this would only center the last column of the table:
|
||||
|
||||
[source]
|
||||
----
|
||||
[cols="2*,^" options="header"]
|
||||
|===
|
||||
| header col 1 | header col 2|
|
||||
| col 1 row 1 | col 2 row 1|
|
||||
| col 1 row 2 | col 2 row 2|
|
||||
|===
|
||||
----
|
||||
|
||||
Many more examples of formatting:
|
||||
|
||||
* Columns: http://asciidoctor.org/docs/user-manual/#cols-format
|
||||
* Cells: http://asciidoctor.org/docs/user-manual/#cell
|
||||
|
||||
=== More Options
|
||||
|
||||
Tables can also be given footer rows, borders, and captions. CSV or DSV can be used instead of formatting the data in pipes.
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#tables
|
||||
|
||||
== Admonitions (Notes, Warnings)
|
||||
|
||||
Asciidoc supports several types of callout boxes, called "admonitions":
|
||||
|
||||
* NOTE
|
||||
* TIP
|
||||
* IMPORTANT
|
||||
* CAUTION
|
||||
* WARNING
|
||||
|
||||
It is enough to start a paragraph with one of these words followed by a colon (such as `NOTE:`). When it is converted to HTML or PDF, those sections will be formatted properly - indented from the main text and showing an icon inline.
|
||||
|
||||
You can add titles to admonitions by making it an admonition block. The structure of an admonition block is like this:
|
||||
|
||||
[source]
|
||||
----
|
||||
[NOTE]
|
||||
.Title of Note
|
||||
====
|
||||
Text of note
|
||||
====
|
||||
----
|
||||
|
||||
In this example, the type of admonition is included in square brackets (`[NOTE]`), and the title is prefixed with a period. Four equal signs give the start and end points of the note text (which can include new lines, lists, etc.).
|
||||
|
||||
More info: http://asciidoctor.org/docs/user-manual/#admonition
|
|
@ -0,0 +1,23 @@
|
|||
= Tools for Working with Asciidoc Format
|
||||
|
||||
== Asciidoc vs Asciidoctor
|
||||
|
||||
The Solr Ref Guide is written in _Asciidoc_ format. This format is generally considered an extension of Markdown, because it has support for tables of contents, better table support, and other features that make it more appropriate for writing technical documentation.
|
||||
|
||||
We are using a version of the Asciidoc syntax along with tools from an open source project called https://asciidoctor.org[Asciidoctor]. This provides full support for the Asciidoc syntax, but replaces the original Python processor with one written in Ruby. There is a Java implementation, known as https://github.com/asciidoctor/asciidoctorj[AsciidoctorJ]. Further extensions from the original Asciidoc project include support for font-based icons and UI elements.
|
||||
|
||||
== Helpful Tools
|
||||
|
||||
You can write Asciidoc without any special tools. It's simply text, with familiar syntax for bold (`*`) and italics (`_`).
|
||||
|
||||
Having some tools in your editor is helpful, though.
|
||||
|
||||
=== Doc Preview
|
||||
|
||||
This allows you to see your document in something close to what it might appear like when output to HTML.
|
||||
|
||||
The following information is from http://asciidoctor.org/docs/editing-asciidoc-with-live-preview.
|
||||
|
||||
* Atom has AsciiDoc Preview, which gives you a panel that updates as you type. There are also a couple of other plugins to support Asciidoc format and auto-complete.
|
||||
* There is a Live Preview browser plugin for Chrome, Firefox and Opera which allow you to open your Asciidoc page in the browser. It will also update as you type.
|
||||
* There is also an Intellij IDEA plugin to support Asciidoc format.
|
|
@ -0,0 +1,61 @@
|
|||
= Working with Jekyll
|
||||
:toc:
|
||||
|
||||
The Solr Ref Guide uses Jekyll to build the HTML version of the site.
|
||||
|
||||
== What is Jekyll?
|
||||
|
||||
Jekyll is a static site generator, meaning that it takes some set of documents and produces HTML pages. It allows for templating of the pages, so each page has the same look and feel without having to code headers, footers, logos, etc., into every page.
|
||||
|
||||
Jekyll is an open source project written in Ruby, online at https://jekyllrb.com/.
|
||||
|
||||
== Jekyll-Asciidoctor Plugin
|
||||
We use a plugin for Jekyll from the Asciidoctor project to integrate Jekyll with Asciidoc formatted content. The source for the plugin is available at https://github.com/asciidoctor/jekyll-asciidoc.
|
||||
|
||||
This plugin allows us to use Asciidoctor-style variables with Jekyll, instead of having to maintain two sets of the same variables (one for HTML version and another for PDF version).
|
||||
|
||||
== Jekyll Basics
|
||||
|
||||
The following sections describe the main features of Jekyll that you will encounter while working with the Solr Ref Guide.
|
||||
|
||||
=== _config.yml
|
||||
|
||||
The `_config.yml` is a global configuration file that drives many of the options used when building the site (particularly in our use of Jekyll).
|
||||
|
||||
We have templatized `_config.yml` so in our use of Jekyll you will find it as `solr-ref-guide/_config.yml.template`. This allows us to define some variables during the build, and use common Lucene/Solr build parameters (such as versions, etc.).
|
||||
|
||||
=== Front Matter
|
||||
|
||||
Front matter for Jekyll is like a header that defines the title of the page, and any other variables that may be helpful or even required when rendering the page.
|
||||
|
||||
Every document that will be converted to HTML *must* include at least the page title at the top of the page.
|
||||
|
||||
The Solr Ref Guide uses the front matter to define the "short name" and permanent URL of a page, and to define the children of a page. The list of children is used to build the site navigation menu that appears to the left of each page's content.
|
||||
|
||||
Many guides to Jekyll also say that defining the layout in the front matter is required. However, since we only have one layout for all pages, we have defined this as a default.
|
||||
|
||||
=== Layouts
|
||||
|
||||
Layouts define the "look and feel" of each page.
|
||||
|
||||
Jekyll uses Liquid for page templates.
|
||||
|
||||
For our implementation of Jekyll, layouts are found in `solr-ref-guide/src/_layouts`
|
||||
|
||||
=== Includes
|
||||
|
||||
Include files are usually small files that are pulled into a layout when a page is being built. They are Liquid templates that define an area of the page. This allows flexibility across layouts - all pages can have the same header without duplicating code, but different pages could have different menu options.
|
||||
|
||||
Include files that we use define the top navigation, the page header, the page footer, and tables of contents.
|
||||
|
||||
For our implementation of Jekyll, include files are found in `solr-ref-guide/src/_includes`.
|
||||
|
||||
=== Data Files
|
||||
|
||||
Data files include data such as lists, that should be included in each page. The left-hand navigation is an example of a data file.
|
||||
|
||||
For our implementation of Jekyll, data files are found in `solr-ref-guide/src/_data`.
|
||||
|
||||
== Building the HTML Site
|
||||
|
||||
An Ant target `build-site` will build the full HTML site. This target builds the navigation for the left-hand menu, and converts all `.adoc` files to `.html`, including navigation and inter-document links.
|
|
@ -0,0 +1,131 @@
|
|||
= Generating PDF
|
||||
|
||||
The main published artifact of the Solr Reference Guide is a PDF document.
|
||||
|
||||
The PDF version of the Ref Guide uses `asciidoctor-pdf` (indirectly) to convert the `.adoc` files to PDF.
|
||||
|
||||
In order for all of the various files to be converted into a single, large, PDF file, another `.adoc` file must be created.
|
||||
|
||||
== About asciidoctor-pdf
|
||||
|
||||
Our PDF build process uses the https://github.com/asciidoctor/asciidoctor-ant[`asciidoctor-ant`] project to integrate Asciidoctor.
|
||||
|
||||
The `asciidoctor-ant` project is really an Ant wrapper around https://github.com/asciidoctor/asciidoctorj[`asciidoctorj`]. This is a Java binding of several Asciidoctor projects that produces a .jar that includes Asciidoctor, Asciidoctor PDF and Asciidoctor EPub.
|
||||
|
||||
Since it is a binding for the Ruby implementations of these three projects, they each work the same way they would if we were using the Ruby versions.
|
||||
|
||||
== Configuring the PDF Theme
|
||||
|
||||
Nearly all of the styling and layout of the PDF is controlled by a single YAML file that is called a theme.
|
||||
|
||||
See the section <<Modifying the Theme>> for information on how to update the theme.
|
||||
|
||||
=== Declaring the Theme to Use
|
||||
|
||||
Our theme is `refguide-theme.yml`, found in `solr/solr-ref-guide/pdf/themes`.
|
||||
|
||||
New themes must be named to include `-theme.yml` at the end. They must also be in proper YAML format.
|
||||
|
||||
The theme to use when generating the PDF is defined with the `pdf-style` attribute. Only the first part of the file name is used. So, a theme file named `refguide-theme.yml` is selected by defining `pdf-style=refguide`.
|
||||
|
||||
In the section <<Asciidoctor PDF Attributes>> below, we can see how we've declared this in our Ant target.
|
||||
|
||||
=== Modifying the Theme
|
||||
|
||||
All of the styling capabilities are described in the https://github.com/asciidoctor/asciidoctor-pdf/blob/master/docs/theming-guide.adoc[Asciidoctor PDF Theming Guide].
|
||||
|
||||
== About the Uber-Document
|
||||
|
||||
In order for all of the files in `.adoc` format to be compiled into a single PDF, we need to give the process one `.adoc` file (an "uber-document") that includes all of the content we want in the PDF. In other words, there is no command that says "take all the files in this directory and make one PDF".
|
||||
|
||||
Asciidoctor has a nice feature called the _include directive_, which allows you to insert content from another file into the current file (more details in the http://asciidoctor.org/docs/user-manual/#include-directive[Asciidoctor documentation]).
|
||||
|
||||
We wanted to make sure we didn't add to the burden of maintaining the PDF by asking everyone to update the uber-document or trying to institute some sort of check on if all the pages are included and in the right place in the hierarchy. Instead, the uber-document is build programmatically at build time.
|
||||
|
||||
The uber-document is found in `solr/solr-ref-guide/src/pdf/SolrRefGuide-all.adoc`. This document is very simple - it includes only the Apache license statement and a single `include` directive to another file:
|
||||
|
||||
[source]
|
||||
\include::_data/pdf-main-body.adoc[]
|
||||
|
||||
The Ant target `build-pdf` includes a dependency on another target, `build-nav-data-files`. This target looks at each document and builds the `pdf-main-body.adoc` file with the proper hierarchy of all pages.
|
||||
|
||||
NOTE: You will not see the `pdf-main-body.adoc` in `solr/solr-ref-guide/src/_data` directory. This file exists only once it has been built, and it is placed in the build directory, at `solr/solr-ref-guide/build/content_data`.
|
||||
|
||||
== Configuring the build-pdf Ant Target
|
||||
|
||||
Since most of the styling and layout is defined by the theme, the options that must be configured in the Ant target relate to where to find referenced files, font directories, image directories, and similar top-level document settings.
|
||||
|
||||
There are two types of settings at work in our Ant target. First are the settings which are part of the `asciidoctor-ant` jar. Second, we are able to declare any Asciidoctor PDF attribute (setting) that we want to apply to the entire PDF.
|
||||
|
||||
Our Ant target (`build-pdf`) uses the following settings:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<asciidoctor:convert
|
||||
sourceDirectory="${build.content.dir}/pdf"
|
||||
sourceDocumentName="SolrRefGuide-all.adoc"
|
||||
baseDir="${build.content.dir}"
|
||||
outputDirectory="${build.dir}"
|
||||
backend="pdf"
|
||||
extensions="adoc"
|
||||
sourceHighlighter="coderay"
|
||||
embedAssets="true"
|
||||
imagesDir="${build.content.dir}"
|
||||
doctype="book"
|
||||
safemode="unsafe">
|
||||
<attribute key="icons" value="font" />
|
||||
<attribute key="icon-set" value="fa" />
|
||||
<attribute key="pdf-stylesDir" value="./pdf/themes"/>
|
||||
<attribute key="pdf-style" value="refguide"/>
|
||||
<attribute key="pdf-fontsDir" value="./pdf/fonts"/>
|
||||
<attribute key="pagenums" value='' />
|
||||
<attribute key="figure-caption!" value='' />
|
||||
<attribute key="idprefix" value='' />
|
||||
<attribute key="idseparator" value='-' />
|
||||
<!-- attributes used in adoc files -->
|
||||
<!-- NOTE: If you add any attributes here for use in adoc files, you almost certainly need to also add
|
||||
them to the _config.yml.template file for building the jekyll site as well
|
||||
-->
|
||||
<attribute key="solr-docs-version" value="${solr-docs-version}" />
|
||||
<attribute key="solr-javadocs" value="${solr-javadocs}" />
|
||||
<attribute key="lucene-javadocs" value="${lucene-javadocs}" />
|
||||
<attribute key="build-date" value="${DSTAMP}" />
|
||||
<attribute key="build-year" value="${current.year}" />
|
||||
</asciidoctor:convert>
|
||||
----
|
||||
|
||||
There are a lot of options here. Note that some include the `<attribute>` tag and some do not. Those that do not are options provided by `asciidoctor-ant` so don't need any special syntax. The options with the `<attribute>` tag are not provided by `asciidoctor-ant` but are supported by `asciidoctor-pdf` so will be applied when the target is run. A few of these are custom variables that we have defined ourselves.
|
||||
|
||||
=== Asciidoctor Ant Attributes
|
||||
|
||||
`sourceDirectory="${build.content.dir}/pdf"`:: Defines where to find the source file to build the PDF. The variable is declared earlier in `build.xml`.
|
||||
`sourceDocumentName="SolrRefGuide-all.adoc"`:: The source file name, which in our case will be built as described in the section <<Creating the Uber-Document>>.
|
||||
`baseDir="${build.content.dir}"`:: The root path for any included files.
|
||||
`outputDirectory="${build.dir}"`:: Defines where the resulting PDF file should go. In this case, the
|
||||
`backend="pdf"`:: The output format. The `asciidoctor-ant` jar is also capable of outputting DocBook format, so we must declare we want a PDF.
|
||||
`extensions="adoc"`:: The file extensions to allow for the source document.
|
||||
`sourceHighlighter="coderay"`:: The library to use for syntax highlighting source code.
|
||||
`imagesDir="${build.content.dir}"`:: The directory to use to find images referenced in the documents.
|
||||
`doctype="book"`:: Adds support for book-style format and sections, such as a preface, colophon, glossary, index, etc.
|
||||
`safemode="unsafe">`:: Allows including resources that are external to the parent directory of the source file. For example, source examples could be pulled from Solr's source code instead of copied to documentation. This setting allows that to happen.
|
||||
|
||||
=== Asciidoctor PDF Attributes
|
||||
|
||||
`<attribute key="icons" value="font" />`:: The style of icons.
|
||||
`<attribute key="icon-set" value="fa" />`:: The icon set to use. We use the Font Awesome font set.
|
||||
`<attribute key="pdf-stylesDir" value="./pdf/themes"/>`:: The directory to find PDF themes. See the section <<Configuring the PDF Theme>> for more details on themes.
|
||||
`<attribute key="pdf-style" value="refguide"/>`:: The theme to use. The theme must be saved in the directory referenced with the `pdf-stylesDir` attribute, and must be named `<pdf-style>-theme.yml`.
|
||||
`<attribute key="pdf-fontsDir" value="./pdf/fonts"/>`:: The directory where to find fonts declared in the theme.
|
||||
`<attribute key="figure-caption!" value='' />`:: Sets caption labels and numbers (such as "Figure 1") to block images. The exclamation at the end of this setting in our config _disables_ figure captioning.
|
||||
`<attribute key="idprefix" value='' />`:: Sets the prefix for auto-generated section IDs, such as those from headings in a page. In our config, this is effectively "null", so auto-generated section IDs do not have any prefix.
|
||||
`<attribute key="idseparator" value='-' />`:: Sets the separator between words in auto-generated section IDs to a hyphen (`-`).
|
||||
|
||||
=== Custom Attributes
|
||||
|
||||
These attributes use variables that are inserted by Ant during the PDF creation process. This allows us to pull from standard Lucene/Solr build files, and not have to update several places for any release. The Ant build process updates the `_config.yml` file from the `_config.yml.template`, then these attributes pull the proper value from that file.
|
||||
|
||||
`<attribute key="solr-docs-version" value="${solr-docs-version}" />`:: Sets the version to the current release version.
|
||||
`<attribute key="solr-javadocs" value="${solr-javadocs}" />`:: Sets the path for Solr javadoc links to include the right path for the current release version.
|
||||
`<attribute key="lucene-javadocs" value="${lucene-javadocs}" />`:: Sets the path for Lucene javadoc links to the right path for the current release version.
|
||||
`<attribute key="build-date" value="${DSTAMP}" />`:: Sets the date of the build to add the date to the footer of each page of the PDF.
|
||||
`<attribute key="build-year" value="${current.year}" />`:: Sets the year of the build to add the date to the copyright notice.
|
|
@ -0,0 +1,196 @@
|
|||
= Publication Process
|
||||
:toc:
|
||||
|
||||
== About the Formats
|
||||
|
||||
The Solr Ref Guide is published in two formats: PDF and HTML.
|
||||
|
||||
The PDF version is the *official* release, and requires a vote before release. See <<Publishing PDF Version>> for details on how to generate the PDF and hold a vote.
|
||||
|
||||
The HTML version is considered a "convenience" version, and does not require a vote. See <<Publishing HTML Version>> for details on how to publish the HTML.
|
||||
|
||||
It's strongly preferred that both PDF and HTML versions are available during the vote for the PDF. However, since the HTML version is not an official release, it is more of an unwritten rule to publish the HTML at the same time as producing a release candidate for the PDF.
|
||||
|
||||
== Publishing PDF Version
|
||||
The PDF version of the Solr Reference Guide is the *official* version. As such, it is voted on by the community before release, and is treated as an official artifact of the Lucene/Solr project.
|
||||
|
||||
=== Generate the PDF
|
||||
|
||||
No local dependencies are required to build the PDF. The Ant target will download the jars and other items it requires.
|
||||
|
||||
The build process generates the PDF, including the page hierarchy, and then runs an optimization script on the PDF to make it smaller.
|
||||
|
||||
To build the PDF:
|
||||
|
||||
. Run `ant build-pdf`
|
||||
. The resulting PDF will be in `solr/build/solr-ref-guide`.
|
||||
|
||||
=== Prerequisites
|
||||
|
||||
* You have checked out the Lucene/Solr source code on the machine you will be doing the release from. You will need scripts in the `dev-tools` directory.
|
||||
* You have generated a GPG key. See the Apache documentation on https://www.apache.org/dev/release-signing.html#generate[generating a code signing key].
|
||||
* You have Python 3 installed. This is needed to poll the mirrors after release to be sure it's propagated enough to make the announcement.
|
||||
|
||||
=== Prepare and Upload Release Candidate
|
||||
|
||||
The `dist/dev` Subversion repository includes a directory for the Solr Ref Guide at https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/[`lucene/solr/ref-guide`] which can host the release candidate (RC) during the VOTE stage of the process.
|
||||
|
||||
These steps walk through checking out this directory and uploading the Guide to it.
|
||||
|
||||
. Checkout the directory: `svn co https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide solr-ref-guide-rc`
|
||||
* If you have already checked out this directory, you can simply update it: `svn update solr-ref-guide-rc`
|
||||
. Change directories so `solr-ref-guide-rc` is your working directory (`cd solr-ref-guide-rc`).
|
||||
|
||||
IMPORTANT: The next step requires that you have already generated your GPG keys. Your GPG passphrase will be required.
|
||||
|
||||
[start=3]
|
||||
. Run the Prep Ref Guide script to prepare the RC. This script ensures proper naming of the PDF file, generates `.sha1` and `.asc` files and creates the proper RC sub-directories under `solr-ref-guide-rc`.
|
||||
.. The structure of the input is: `prep-solr-ref-guide-rc.sh <path/PDFfilename> <Solrversion-RC#> GPGkey`.
|
||||
.. From the `solr-ref-guide-rc` directory, it will look something like this:
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
$ ~/lucene-source/dev-tools/scripts/prep-solr-ref-guide-rc.sh apache-solr-ref-guide-7.0.pdf 7.0-RC0
|
||||
|
||||
+ mkdir apache-solr-ref-guide-7.0-RC0
|
||||
+ mv apache-solr-ref-guide-7.0.pdf apache-solr-ref-guide-7.0-RC0/apache-solr-ref-guide-7.0.pdf
|
||||
+ cd apache-solr-ref-guide-7.0-RC0
|
||||
+ sha1sum apache-solr-ref-guide-7.0.pdf
|
||||
+ gpg -u DEADBEEF --armor --output apache-solr-ref-guide-7.0.pdf.asc --detach-sig apache-solr-ref-guide-7.0.pdf
|
||||
|
||||
You need a passphrase to unlock the secret key for
|
||||
user: "Your Name <you@apache.org>"
|
||||
4096-bit RSA key, ID DEADBEEF, created 1969-07-04
|
||||
----
|
||||
+
|
||||
. Add and commit the new release candidate to the `dist/dev` with these steps:
|
||||
.. `svn add apache-solr-ref-guide-7.0-RC0`
|
||||
.. `svn commit -m "7.0 ref guide RC0"`
|
||||
|
||||
=== Hold a VOTE
|
||||
Votes must be sent to the lucene-dev mailing list (`dev@lucene.apache.org`).
|
||||
|
||||
. Send an email to `dev@lucene.apache.org` with subject, "VOTE: Release Apache Solr Ref Guide for Solr X.Y".
|
||||
. The body of the email should include the full URL of the RC directory in the `dist/dev` repo. Such as: https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-7.0-RC0
|
||||
. You can add your own +1 to the vote announcement email.
|
||||
. If there are issues that need to be resolved, you can start the process over, using RC1, RC2, etc., as needed.
|
||||
|
||||
Ideally, the HTML version will also be available for voters to evaluate, see the section <<Publishing HTML Version>> below for details of how to do that.
|
||||
|
||||
=== Publish to Production & Archive Old Versions
|
||||
|
||||
Once at least three PMC members have voted for release (see https://www.apache.org/foundation/voting.html#ReleaseVotes[Apache Voting Process] for details on the rules for votes), the release candidate can be released.
|
||||
|
||||
. Use the Publish Solr Ref Guide script (`publish-solr-ref-guide.sh`) to generate the proper SVN commands to be run to execute a remote move of the RC files to the final `dist/releases` repository.
|
||||
.. The script takes only the version and _RC number that passed the vote_ as inputs, such as `7.0-RC0`.
|
||||
.. The input and output of the script will look like this:
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
$ ~/lucene-source/dev-tools/scripts/publish-solr-ref-guide-rc.sh X.Y-RCZ
|
||||
|
||||
## Run the following commands when ready...
|
||||
svn move -m 'publishing apache-solr-ref-guide-X.Y-RCZ' https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RCZ/apache-solr-ref-guide-X.Y.pdf https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RCZ/apache-solr-ref-guide-X.Y.pdf.asc https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RCZ/apache-solr-ref-guide-X.Y.pdf.sha1 https://dist.apache.org/repos/dist/release/lucene/solr/ref-guide/
|
||||
|
||||
svn rm -m 'cleaning up apache-solr-ref-guide-X.Y-RCZ' https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RCZ
|
||||
----
|
||||
[start=2]
|
||||
. The release should propagate to as many mirrors as possible before announcing the release, generally 24 hours is long enough. Use the Poll Mirrors script (`poll-mirrors.py`) to check the status:
|
||||
+
|
||||
[source,bash]
|
||||
python3 -u ~/lucene-source/dev-tools/scripts/poll-mirrors.py -details -p lucene/solr/ref-guide/apache-solr-ref-guide-X.Y.pdf
|
||||
|
||||
* This script requires Python 3 to be installed on your machine.
|
||||
* If you have over 85% of the mirrors with the file, it's OK to go ahead with the announcement.
|
||||
. You may get an automated email about updating the ASF release repository; you can safely ignore this email.
|
||||
. The `dist/releases` repo is only meant to keep the latest releases. Shortly after new releases are mirrored, they are copied to `archive.apache.org`, so older releases can safely be deleted from `dist/releases` since they have been backed up in the archives.
|
||||
.. Run the Archive Ref Guide script (`archive-solr-ref-guide.sh`) using the X.Y version of the Ref Guide that has just been published. Older RCs will also be removed.
|
||||
.. Again, this script doesn't do any direct removal of files, it only outputs SVN commands for you to copy and paste:
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
$ ~/lucene-source/dev-tools/scripts/archive-solr-ref-guide.sh X.Y
|
||||
## Run the following commands when ready...
|
||||
|
||||
# Delete old releases
|
||||
svn rm -m 'removing archived ref guide files prior to X.Y' https://dist.apache.org/repos/dist/release/lucene/solr/ref-guide/apache-solr-ref-guide-A.B.pdf https://dist.apache.org/repos/dist/release/lucene/solr/ref-guide/apache-solr-ref-guide-A.B.pdf.asc https://dist.apache.org/repos/dist/release/lucene/solr/ref-guide/apache-solr-ref-guide-A.B.pdf.sha1
|
||||
|
||||
# Delete old RC files
|
||||
svn rm -m 'cleaning up old RCs now that X.Y has been released' https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RC0/ https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y-RC1/
|
||||
----
|
||||
|
||||
=== Announce the Release
|
||||
|
||||
Announce the availability of the new Ref Guide on `solr-user@lucene.apache.org` and CC `general@lucene.apache.org` and `announce@apache.org`.
|
||||
|
||||
WARNING: You must send the announcement email from your @apache.org email address or announce@apache will reject it.
|
||||
|
||||
Always use the link to the download redirector for the announcement, as it will automatically direct users to the closest mirror for download: `https://www.apache.org/dyn/closer.cgi/lucene/solr/ref-guide/apache-solr-ref-guide-X.Y.pdf`.
|
||||
|
||||
You could also include a link to the HTML version in your announcement, if the publication process for that has finished.
|
||||
|
||||
== Publishing HTML Version
|
||||
The steps to publish the Guide differ depending on if it is the first time the Guide has been published or if it is an update to an already published Guide.
|
||||
|
||||
=== Building the HTML Version
|
||||
|
||||
If you have the required dependencies on your local machine, you can build the HTML version with `ant build-site`. The dependencies are listed in `solr-ref-guide/README.adoc`.
|
||||
|
||||
//TODO update Jenkins link
|
||||
If you do not have the required dependencies, and don't choose to install them, you can download the files from the Jenkins (https://builds.apache.org/job/Solr-reference-guide-jira-SOLR-10290/lastSuccessfulBuild/artifact/solr/build/solr-ref-guide/html-site/[Solr Reference Guide job]).
|
||||
|
||||
=== Publish a New Guide
|
||||
// A lot of this was copied from https://wiki.apache.org/lucene-java/ReleaseTodo#Website_.2B-.3D_javadocs. See that section for explanations for why some steps are required.
|
||||
|
||||
==== Step 1: Update extpaths.txt in CMS Staging
|
||||
|
||||
. Checkout CMS trunk:
|
||||
+
|
||||
[source,bash]
|
||||
svn co --depth=immediates https://svn.apache.org/repos/asf/lucene/cms/trunk/content website-source
|
||||
+
|
||||
* If you already have this repo checked out, you can simply `svn up website-source` to update to the latest revision.
|
||||
. `cd website-source`
|
||||
. Add Guide branch dir: `echo solr/guide/X_Y_Z >> extpaths.txt`
|
||||
. Commit changes:
|
||||
+
|
||||
[source,bash]
|
||||
svn commit -m "Update CMS production sync exceptions for X_Y_Z Guide" extpaths.txt
|
||||
|
||||
==== Step 2: Push Guide to Website Production
|
||||
|
||||
Go to the checkout directory where you have built the Guide and push the documentation via subversion import. You must push it to the path you just added to `extpaths.txt`, so if the path you added was `solr/guide/6.5`, you'll use the path as shown in the below example:
|
||||
|
||||
[source,bash]
|
||||
svn -m "Add Ref Guide for Solr 6.5" import <checkoutroot>/solr/build/solr-ref-guide/html-site https://svn.apache.org/repos/infra/websites/production/lucene/content/solr/guide/6_5
|
||||
|
||||
Confirm you can browse to these URLs manually, and especially that solr javadocs link back to lucene's correctly. Example:
|
||||
https://lucene.apache.org/solr/guide/6_5
|
||||
|
||||
==== Step 3: Push Staging extpaths.txt to Production
|
||||
|
||||
The `extpaths.txt` works by listing paths that should be ignored when the CMS syncs the staging and production repositories. Publishing staging to production will only succeed if the paths listed in `extpaths.txt` exist in production. At the same time, if a path exists in production but not in staging it will be deleted unless it is defined in `extpaths.txt`. After pushing the content to production, check that the `extpaths.txt` in production includes the proper path to ensure that the Guide is not deleted incorrectly. If it does not exist in production, try to publish the site again to make sure it is updated.
|
||||
|
||||
Production URL: https://lucene.apache.org/extpaths.txt
|
||||
|
||||
==== Update Ref Guide Landing Page
|
||||
|
||||
Update the landing page at https://lucene.apache.org/solr/guide to link to the newest version.
|
||||
|
||||
You can use the CMS system for this since it is a small change, or you can edit the file locally and commit it to the staging repo.
|
||||
|
||||
=== Update a Published Guide
|
||||
|
||||
If you need to re-publish an existing online copy of the Guide, you will need to checkout the directory in production website repository and overwrite the existing files:
|
||||
|
||||
. Build the new HTML files locally (`ant clean build-site`), or download them from Jenkins.
|
||||
. Checkout the directory you need to update from the production repo: `svn co https://svn.apache.org/repos/infra/websites/production/lucene/content/solr/guide/<dir>`.
|
||||
* This command checks out the Guide version directory into a local subdirectory with the same name as the version (such as "6_5"). You can provide a better name locally if you prefer by adding it to the end of the command shown above.
|
||||
* Don't shortcut this and download the whole production website. It will take an incredibly long time and that will feel like _forever_.
|
||||
. Copy the files from the build location to the checked out Guide directory. For example, if we needed to replace the current Guide for Solr 6.5, we'd do `cp -r <checkoutroot>/solr/build/html-site 6_5/.`
|
||||
. Use `svn status` to see the files modified.
|
||||
. If there are any pages added or deleted, use `svn add <file>` or `svn rm <file>` as needed.
|
||||
. Commit the changes: `svn commit -m "Update production 6.5 Ref Guide"`
|
||||
|
||||
// TODO:
|
||||
// - figure out if redirects in .htaccess require any work here (probably)
|
|
@ -0,0 +1,3 @@
|
|||
_site
|
||||
.sass-cache
|
||||
.jekyll-metadata
|
|
@ -0,0 +1,6 @@
|
|||
---
|
||||
title: "Page Not Found"
|
||||
search: exclude
|
||||
---
|
||||
|
||||
Sorry, but the page you were trying to view does not exist. Try searching for it or looking at the URL to see if it looks correct.
|
|
@ -0,0 +1,3 @@
|
|||
## Jekyll Documentation theme
|
||||
|
||||
Build the site to see the instructions for using it. Or just go here: [http://idratherbewriting.com/documentation-theme-jekyll/](http://idratherbewriting.com/documentation-theme-jekyll/)
|
|
@ -0,0 +1,99 @@
|
|||
#
|
||||
#
|
||||
#
|
||||
# NOTE: Ant converts _config.yml.template into create _config.yml and performs ant property substitution.
|
||||
#
|
||||
#
|
||||
#
|
||||
|
||||
# Gems that are included for building the site. jekyll-asciidoc allows Jekyll to use Asciidoctor for variables and settings
|
||||
gems: [jekyll-asciidoc]
|
||||
|
||||
destination: ../html-site
|
||||
|
||||
# this property is useful for conditional filtering of content that is separate from the PDF.
|
||||
output: web
|
||||
|
||||
# this appears on the top navigation bar next to the home button
|
||||
topnav_title: Solr Ref Guide
|
||||
|
||||
# this appears in the html browser tab for the site title (seen mostly by search engines, not users)
|
||||
site_title: Apache Solr Reference Guide
|
||||
|
||||
# this appears in the footer
|
||||
company_name: Apache Software Foundation
|
||||
|
||||
|
||||
# the preview server used. Leave as is.
|
||||
host: 127.0.0.1
|
||||
|
||||
# the port where the preview is rendered. You can leave this as is unless you have other Jekyll builds using this same port that might cause conflicts. in that case, use another port such as 4006.
|
||||
port: 4015
|
||||
|
||||
# these are the files and directories that jekyll will exclude from the build
|
||||
exclude:
|
||||
- .idea/
|
||||
- .gitignore
|
||||
- pdf/
|
||||
|
||||
# if you uncomment the next line, the Feedback link gets removed
|
||||
feedback_disable: true
|
||||
|
||||
# used as a contact email for the Feedback link in the top navigation bar
|
||||
# feedback_email: an_email@apache.org
|
||||
|
||||
# if you uncomment the next line, it changes the Feedback text
|
||||
# feedback_text: "Need help?"
|
||||
|
||||
# if you uncomment the next line, it changes where the feedback link points to
|
||||
# feedback_link: "http://helpy.io/"
|
||||
|
||||
# these are defaults used for the frontmatter for these file types
|
||||
defaults:
|
||||
-
|
||||
scope:
|
||||
path: ""
|
||||
type: "pages"
|
||||
values:
|
||||
layout: "page"
|
||||
search: true
|
||||
|
||||
-
|
||||
scope:
|
||||
path: ""
|
||||
type: "posts"
|
||||
values:
|
||||
layout: "post"
|
||||
search: true
|
||||
|
||||
|
||||
# the description is used in the feed.xml file
|
||||
description: "The Apache Solr Reference Guide is the official documentation for the Apache Solr project."
|
||||
|
||||
# needed for sitemap.xml and feed.xml files
|
||||
url: https://home.apache.org/~ctargett/RefGuidePOC/jekyll-full
|
||||
|
||||
# Asciidoc settings - disabled so we can use asciidoctor instead
|
||||
asciidoc: {}
|
||||
|
||||
# Custom Attributes for use in our templates & adoc files.
|
||||
#
|
||||
# Declared as a YAML reference so we can refer to them via site.solr-attributes.foo in liquid templates,
|
||||
# in addition to using them below in our asciidoctor attribute configurations
|
||||
# (see https://github.com/asciidoctor/jekyll-asciidoc/issues/137)
|
||||
#
|
||||
# NOTE: If you add any attributes here for use in adoc files, you almost certainly need to also add
|
||||
# them to the <asciidoctor:convert/> ant task for building the PDF as well.
|
||||
solr-attributes: &solr-attributes-ref
|
||||
solr-docs-version: "${solr-docs-version}"
|
||||
solr-javadocs: "${solr-javadocs}"
|
||||
lucene-javadocs: "${lucene-javadocs}"
|
||||
build-date: "${DSTAMP}"
|
||||
build-year: "${current.year}"
|
||||
|
||||
asciidoctor:
|
||||
attributes:
|
||||
<<: *solr-attributes-ref
|
||||
icons: "font"
|
||||
source-highlighter: "pygments"
|
||||
pygments-css: "style"
|
|
@ -0,0 +1,5 @@
|
|||
|
||||
|
||||
# placed here for translation purposes
|
||||
search_placeholder_text: search...
|
||||
search_no_results_text: No results found.
|
|
@ -0,0 +1,7 @@
|
|||
allowed-tags:
|
||||
- getting_started
|
||||
- content_types
|
||||
- troubleshooting
|
||||
- analysis
|
||||
- languages
|
||||
- scaling
|
|
@ -0,0 +1,15 @@
|
|||
---
|
||||
layout: default
|
||||
type: archive
|
||||
---
|
||||
|
||||
<div class="post-header">
|
||||
<h1 class="post-title-main">{{ page.title }}</h1>
|
||||
</div>
|
||||
<div class="post-content">
|
||||
|
||||
{{ content }}
|
||||
</div>
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
<!-- Send feedback function -->
|
||||
<script>
|
||||
function SendLinkByMail(href) {
|
||||
var subject= "{{site.site_title}} feedback";
|
||||
var body = "I have some feedback about the {{page.title}} page: ";
|
||||
body += window.location.href;
|
||||
body += "";
|
||||
var uri = "mailto:{{site.feedback_email}}?subject=";
|
||||
uri += encodeURIComponent(subject);
|
||||
uri += "&body=";
|
||||
uri += encodeURIComponent(body);
|
||||
window.location.href = uri;
|
||||
}
|
||||
</script>
|
||||
|
||||
<li><a href="{% if site.feedback_link %}{{site.feedback_link}}{% else %}javascript:(function()%7BSendLinkByMail()%3B%7D)()%3B{% endif %}" target="_blank">{% if site.feedback_link == null %}<i class="fa fa-envelope-o"></i>{% endif %} {% if site.feedback_text %}{{site.feedback_text}}{% else %}Feedback{% endif %}</a></li>
|
|
@ -0,0 +1,9 @@
|
|||
<footer>
|
||||
<div class="row">
|
||||
<div class="col-lg-12 footer">
|
||||
©{{ site.solr-attributes.build-year }} {{site.company_name}}. All rights reserved. <br />
|
||||
{% if page.last_updated %}<p>Page last updated:</span> {{page.last_updated}}<br/>{% endif %} Site last generated: {{ site.solr-attributes.build-date }} <br />
|
||||
<p><img src="{{ "solr-sunOnly-small.png" }}" alt="Apache Solr"/></p>
|
||||
</div>
|
||||
</div>
|
||||
</footer>
|
|
@ -0,0 +1,6 @@
|
|||
<!-- the google_analytics_id gets auto inserted from the config file -->
|
||||
|
||||
{% if site.google_analytics %}
|
||||
|
||||
<script>(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)})(window,document,'script','//www.google-analytics.com/analytics.js','ga');ga('create','{{site.google_analytics}}','auto');ga('require','displayfeatures');ga('send','pageview');</script>
|
||||
{% endif %}
|
|
@ -0,0 +1,34 @@
|
|||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="{% if page.description %}{{ page.description | strip_html | strip_newlines | truncate: 160 }}{% endif %}">
|
||||
<meta name="keywords" content="{{page.tags}}{% if page.tags %}, {% endif %} {{page.keywords}}">
|
||||
<title>{{ page.title }} | {{ site.site_title }}</title>
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css">
|
||||
<!--<link rel="stylesheet" type="text/css" href="css/bootstrap.min.css">-->
|
||||
<link rel="stylesheet" href="{{ "css/lavish-bootstrap.css" }}">
|
||||
<link rel="stylesheet" href="{{ "css/customstyles.css" }}">
|
||||
<link rel="stylesheet" href="{{ "css/theme-solr.css" }}">
|
||||
<link rel="stylesheet" href="{{ "css/ref-guide.css" }}">
|
||||
<link rel="stylesheet" href="{{ "css/comments.css" }}">
|
||||
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.1.4/jquery.min.js"></script>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery-cookie/1.4.1/jquery.cookie.min.js"></script>
|
||||
<script src="{{ "js/jquery.navgoco.min.js" }}"></script>
|
||||
|
||||
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/anchor-js/2.0.0/anchor.min.js"></script>
|
||||
<script src="{{ "js/toc.js" }}"></script>
|
||||
<script src="{{ "js/customscripts.js" }}"></script>
|
||||
|
||||
<link rel="shortcut icon" href="{{ "images/favicon.ico" }}">
|
||||
|
||||
<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements and media queries -->
|
||||
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
|
||||
<!--[if lt IE 9]>
|
||||
<script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
|
||||
<script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
|
||||
<![endif]-->
|
||||
|
||||
<link rel="alternate" type="application/rss+xml" title="{{ site.site_title }}" href="{{ "/feed.xml" | prepend: site.url }}">
|
|
@ -0,0 +1,33 @@
|
|||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="{% if page.summary %}{{ page.summary | strip_html | strip_newlines | truncate: 160 }}{% endif %}">
|
||||
<meta name="keywords" content="{{page.tags}}{% if page.tags %}, {% endif %} {{page.keywords}}">
|
||||
<title>{% if page.homepage == true %} {{site.homepage_title}} {% elsif page.title %}{{ page.title }}{% endif %} | {{ site.site_title }}</title>
|
||||
|
||||
|
||||
<link rel="stylesheet" href="{{ "/css/syntax.css" | prepend: site.baseurl | prepend: site.url }}">
|
||||
<link rel="stylesheet" href="{{ "/css/font-awesome.min.css" | prepend: site.baseurl | prepend: site.url }}">
|
||||
<link rel="stylesheet" href="{{ "/css/bootstrap.min.css" | prepend: site.baseurl | prepend: site.url }}">
|
||||
<link rel="stylesheet" href="{{ "/css/modern-business.css" | prepend: site.baseurl | prepend: site.url }}">
|
||||
<link rel="stylesheet" href="{{ "/css/lavish-bootstrap.css" | prepend: site.baseurl | prepend: site.url }}">
|
||||
<link rel="stylesheet" href="{{ "/css/customstyles.css" | prepend: site.baseurl | prepend: site.url }}">
|
||||
<link rel="stylesheet" href="{{ "/css/theme-green.css" | prepend: site.baseurl | prepend: site.url }}">
|
||||
<link rel="stylesheet" href="{{ "/css/syntax.css" | prepend: site.baseurl | prepend: site.url }}">
|
||||
<link rel="stylesheet" href="{{ "/css/printstyles.css" | prepend: site.baseurl }}">
|
||||
|
||||
<script>
|
||||
Prince.addScriptFunc("datestamp", function() {
|
||||
return "PDF last generated: {{ site.time | date: '%B %d, %Y' }}";
|
||||
});
|
||||
</script>
|
||||
|
||||
<script>
|
||||
Prince.addScriptFunc("guideName", function() {
|
||||
return "{{site.print_title}} User Guide";
|
||||
});
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1 @@
|
|||
<figure>{% if {{include.url}} %}<a class="no_icon" target="_blank" href="{{include.url}}">{% endif %}<img class="docimage" src="images/{{include.file}}" alt="{{include.alt}}" {% if {{include.max-width}} %}style="max-width: {{include.max-width}}px"{% endif %} />{% if {{include.url}} %}</a>{% endif %}{% if {{include.caption}} %}<figcaption>{{include.caption}}</figcaption></figure>{% endif %}
|
|
@ -0,0 +1 @@
|
|||
<img class="inline" src="images/{{include.file}}" alt="{{include.alt}}" />
|
|
@ -0,0 +1,44 @@
|
|||
{% comment %}Get links from each sidebar, as listed in the _config.yml file under sidebars{% endcomment %}
|
||||
|
||||
{% for sidebar in site.sidebars %}
|
||||
{% for entry in site.data.sidebars[sidebar].entries %}
|
||||
{% for folder in entry.folders %}
|
||||
{% for folderitem in folder.folderitems %}
|
||||
{% if folderitem.url contains "html#" %}
|
||||
[{{folderitem.url | remove: "/" }}]: {{folderitem.url | remove: "/"}}
|
||||
{% else %}
|
||||
[{{folderitem.url | remove: "/" | remove: ".html"}}]: {{folderitem.url | remove: "/"}}
|
||||
{% endif %}
|
||||
{% for subfolders in folderitem.subfolders %}
|
||||
{% for subfolderitem in subfolders.subfolderitems %}
|
||||
[{{subfolderitem.url | remove: "/" | remove: ".html"}}]: {{subfolderitem.url | remove: "/"}}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
|
||||
{% comment %} Get links from topnav {% endcomment %}
|
||||
|
||||
{% for entry in site.data.topnav.topnav %}
|
||||
{% for item in entry.items %}
|
||||
{% if item.external_url == null %}
|
||||
[{{item.url | remove: "/" | remove: ".html"}}]: {{item.url | remove: "/"}}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
{% comment %}Get links from topnav dropdowns {% endcomment %}
|
||||
|
||||
{% for entry in site.data.topnav.topnav_dropdowns %}
|
||||
{% for folder in entry.folders %}
|
||||
{% for folderitem in folder.folderitems %}
|
||||
{% if folderitem.external_url == null %}
|
||||
[{{folderitem.url | remove: "/" | remove: ".html"}}]: {{folderitem.url | remove: "/"}}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
|
@ -0,0 +1,59 @@
|
|||
{% assign sidebar = site.data.sidebar %}
|
||||
|
||||
<ul id="mysidebar" class="nav">
|
||||
<li class="sidebarTitle">{{sidebar.title}} <!-- TODO: version from site.data --></li>
|
||||
{% for level1item in sidebar.kids %}
|
||||
<li class="sb-level1">
|
||||
<a href="{{level1item.url | remove: "/"}}">{{ level1item.title }}</a>
|
||||
{% if level1item.kids.size > 0 %}
|
||||
<ul>
|
||||
{% for level2item in level1item.kids %}
|
||||
<li class="sb-level2">
|
||||
<a href="{{ level2item.url | remove: "/" }}">{{ level2item.title }}</a>
|
||||
{% if level2item.kids.size > 0 %}
|
||||
<ul>
|
||||
{% for level3item in level2item.kids %}
|
||||
<li class="sb-level3">
|
||||
<a href="{{ level3item.url | remove: "/" }}">{{ level3item.title }}</a>
|
||||
{% if level3item.kids.size > 0 %}
|
||||
<ul>
|
||||
{% for level4item in level3item.kids %}
|
||||
<li class="sb-level4">
|
||||
<a href="{{ level4item.url | remove: "/" }}">{{ level4item.title }}</a>
|
||||
{% comment %}
|
||||
<!-- NOTE: adding addiional levels here will also require additional changes to
|
||||
ref-guide.css in order to indent them properly
|
||||
-->
|
||||
{% endcomment %}
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
{% endif %}
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
{% endif %}
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
{% endif %}
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
{% comment %}
|
||||
<!-- if you aren't using the accordion, uncomment this block:
|
||||
<p class="external">
|
||||
<a href="#" id="collapseAll">Collapse All</a> | <a href="#" id="expandAll">Expand All</a>
|
||||
</p>
|
||||
-->
|
||||
{% endcomment %}
|
||||
|
||||
<!-- set the 'active' class on the current page and ancestors -->
|
||||
<!-- this highlights the active parent class in the navgoco sidebar. this is critical so that the parent expands when you're viewing a page. This must appear below the sidebar code above. Otherwise, if placed inside customscripts.js, the script runs before the sidebar code runs and the class never gets inserted.-->
|
||||
<script>$("#mysidebar a[href='{{ page.shortname }}.html']").parents('li').toggleClass("active", true);</script>
|
||||
<!-- set the 'current' class on the current page and 'current-tree' on the current page + it's ancestors -->
|
||||
<!-- this can let us do css highlighting of the current page in the sidebar even if/when the user clicks around in the sidebar causing other sidebar elements to be 'active' -->
|
||||
<script>
|
||||
$("#mysidebar a[href='{{ page.shortname }}.html']").parent('li').toggleClass("current", true);
|
||||
$("#mysidebar a[href='{{ page.shortname }}.html']").parents('li').toggleClass("current-tree", true);
|
||||
</script>
|
|
@ -0,0 +1,22 @@
|
|||
<p>The following pages and posts are tagged with <button type="button" style="cursor: default" class="btn btn-default navbar-btn">{{page.tagName}}</button></p>
|
||||
<table><thead><tr><th>Title</th><th>Type</th><th>Excerpt</th></tr></thead>
|
||||
<tbody>
|
||||
{% assign thisTag = page.tagName %}
|
||||
{% for page in site.pages %}
|
||||
{% for tag in page.tags %}
|
||||
{% if tag == thisTag %}
|
||||
|
||||
<tr><td><a href="{{ page.url | remove: "/" }}">{{page.title}}</a></td>
|
||||
<td><span class="label label-default">Page</span></td>
|
||||
<td>{% if page.description %}
|
||||
{{ page.description | strip_html | strip_newlines | truncate: 160 }}
|
||||
{% else %}
|
||||
{{ page.content | truncatewords: 50 | strip_html }}
|
||||
{% endif %}</td>
|
||||
</tr>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
</tbody>
|
||||
</table>
|
|
@ -0,0 +1,21 @@
|
|||
|
||||
<!-- this handles the automatic toc. use ## for subheads to auto-generate the on-page minitoc. if you use html tags, you must supply an ID for the heading element in order for it to appear in the minitoc. -->
|
||||
<script>
|
||||
$( document ).ready(function() {
|
||||
// Handler for .ready() called.
|
||||
|
||||
$('#toc').toc({ minimumHeaders: 2, listType: 'ul', showSpeed: 0, headers: 'h2,h3' });
|
||||
|
||||
/* this offset helps account for the space taken up by the floating toolbar. */
|
||||
$('#toc').on('click', 'a', function() {
|
||||
var target = $(this.getAttribute('href'))
|
||||
, scroll_target = target.offset().top
|
||||
|
||||
$(window).scrollTop(scroll_target - 10);
|
||||
return false
|
||||
})
|
||||
|
||||
});
|
||||
</script>
|
||||
|
||||
<div id="toc"></div>
|
|
@ -0,0 +1,63 @@
|
|||
<!-- Navigation -->
|
||||
<nav class="navbar navbar-inverse navbar-fixed-top">
|
||||
<div class="container topnavlinks">
|
||||
<div class="navbar-header">
|
||||
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
|
||||
<span class="sr-only">Toggle navigation</span>
|
||||
<span class="icon-bar"></span>
|
||||
<span class="icon-bar"></span>
|
||||
<span class="icon-bar"></span>
|
||||
</button>
|
||||
<a class="fa fa-home fa-lg navbar-brand" href="index.html"> <span class="projectTitle"> {{site.topnav_title}} {{ site.solr-attributes.solr-docs-version }}</span></a>
|
||||
</div>
|
||||
<div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
|
||||
<ul class="nav navbar-nav navbar-right">
|
||||
<!-- Link to Solr website -->
|
||||
<li><a href="https://lucene.apache.org/solr/news" target="_blank">Solr News</a></li>
|
||||
<!-- Other Guide Formats dropdown -->
|
||||
<li class="dropdown">
|
||||
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Other Formats<b class="caret"></b></a>
|
||||
<ul class="dropdown-menu">
|
||||
<li><a href="https://www.apache.org/dyn/closer.cgi/lucene/solr/ref-guide" target="_blank">PDF for Latest Release</a></li>
|
||||
<li><a href="https://archive.apache.org/dist/lucene/solr/ref-guide/" target="_blank">Archived PDFs</a></li>
|
||||
<li><a href="https://lucene.apache.org/solr/guide/" target="_blank">Other Versions Online</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<!-- Solr Resources dropdown -->
|
||||
<li class="dropdown">
|
||||
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Solr Resources<b class="caret"></b></a>
|
||||
<ul class="dropdown-menu">
|
||||
<li><a href="{{site.solr-attributes.solr-javadocs}}/solr-core/index.html" target="_blank">Solr Javadocs</a></li>
|
||||
<li><a href="https://git1-us-west.apache.org/repos/asf?p=lucene-solr.git" target="_blank">Lucene/Solr Source Code</a></li>
|
||||
<li><a href="https://lucene.apache.org/solr/resources.html#community" target="_blank">Solr Community Links</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
{% if site.feedback_disable == null or site.feedback_disable == false %}
|
||||
{% include feedback.html %}
|
||||
{% endif %}
|
||||
<!--comment out this block if you want to hide search-->
|
||||
<li>
|
||||
<!--start search-->
|
||||
<div id="search-demo-container">
|
||||
<input type="text" id="search-input" placeholder="{{site.data.strings.search_placeholder_text}}">
|
||||
<ul id="results-container"></ul>
|
||||
</div>
|
||||
<script src="{{ "js/jekyll-search.js"}}" type="text/javascript"></script>
|
||||
<script type="text/javascript">
|
||||
SimpleJekyllSearch.init({
|
||||
searchInput: document.getElementById('search-input'),
|
||||
resultsContainer: document.getElementById('results-container'),
|
||||
dataSource: '{{ "search.json" }}',
|
||||
searchResultTemplate: '<li><a href="{url}" title="{{page.title | replace: "'", "\"}}">{title}</a></li>',
|
||||
noResultsText: '{{site.data.strings.search_no_results_text}}',
|
||||
limit: 10,
|
||||
fuzzy: true,
|
||||
})
|
||||
</script>
|
||||
<!--end search-->
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /.container -->
|
||||
</nav>
|
|
@ -0,0 +1,55 @@
|
|||
<!DOCTYPE html>
|
||||
<head>
|
||||
{% include head.html %}
|
||||
<script>
|
||||
$(document).ready(function() {
|
||||
// Initialize navgoco with default options
|
||||
$("#mysidebar").navgoco({
|
||||
caretHtml: '',
|
||||
accordion: true,
|
||||
openClass: 'active', // open
|
||||
save: false, // we do *NOT* want cookies used to save the current stage of the sidebar
|
||||
// instead the code in sidebar.html will ensure that the current page
|
||||
// (and it's ancestors) are "active" on page load
|
||||
slide: {
|
||||
duration: 400,
|
||||
easing: 'swing'
|
||||
}
|
||||
});
|
||||
|
||||
$("#collapseAll").click(function(e) {
|
||||
e.preventDefault();
|
||||
$("#mysidebar").navgoco('toggle', false);
|
||||
});
|
||||
|
||||
$("#expandAll").click(function(e) {
|
||||
e.preventDefault();
|
||||
$("#mysidebar").navgoco('toggle', true);
|
||||
});
|
||||
|
||||
});
|
||||
|
||||
</script>
|
||||
</head>
|
||||
<body id="{{ page.shortname }}">
|
||||
{% include topnav.html %}
|
||||
<!-- Page Content -->
|
||||
<div class="container">
|
||||
<div class="col-lg-12"> </div>
|
||||
<!-- Content Row -->
|
||||
<div class="row">
|
||||
<!-- Sidebar Column -->
|
||||
<div class="col-md-3">
|
||||
{% include sidebar.html %}
|
||||
</div>
|
||||
<!-- Content Column -->
|
||||
<div class="col-md-9">
|
||||
{{content}}
|
||||
</div>
|
||||
<!-- /.row -->
|
||||
</div>
|
||||
<!-- /.container -->
|
||||
</div>
|
||||
|
||||
</body>
|
||||
</html>
|
|
@ -0,0 +1,25 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<html>
|
||||
<head>
|
||||
{% include head_print.html %}
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
<body class="{% if page.type == "title"%}title{% elsif page.type == "frontmatter" %}frontmatter{% elsif page.type == "first_page" %}first_page{% endif %} print">
|
||||
|
||||
<!-- Page Content -->
|
||||
<div class="container">
|
||||
<!-- Content Column -->
|
||||
<div class="col-md-9">
|
||||
|
||||
{{content}}
|
||||
</div>
|
||||
|
||||
</div> <!-- /.container -->
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
---
|
||||
---
|
||||
{{content}}
|
|
@ -0,0 +1,80 @@
|
|||
---
|
||||
layout: default
|
||||
---
|
||||
|
||||
<div class="post-header">
|
||||
<h1 class="post-title-main">{{ page.title }}</h1>
|
||||
</div>
|
||||
|
||||
<!-- This adds a workflow map, to a page
|
||||
See http://idratherbewriting.com/documentation-theme-jekyll/mydoc_workflow_maps.html
|
||||
Leaving it here commented out in case we want to implement this in the future
|
||||
{% if page.simple_map == true %}
|
||||
<script>
|
||||
$(document).ready ( function(){
|
||||
$('.box{{page.box_number}}').addClass('active');
|
||||
});
|
||||
</script>
|
||||
{% include custom/{{page.map_name}}.html %}
|
||||
{% elsif page.complex_map == true %}
|
||||
<script>
|
||||
$(document).ready ( function(){
|
||||
$('.modalButton{{page.box_number}}').addClass('active');
|
||||
});
|
||||
</script>
|
||||
{% include custom/{{page.map_name}}.html %}
|
||||
{% endif %} -->
|
||||
|
||||
<div class="post-content">
|
||||
|
||||
{% if page.summary %}
|
||||
<div class="summary">{{ page.summary }}</div>
|
||||
{% endif %}
|
||||
|
||||
{% unless page.toc == false %}
|
||||
{% include toc.html %}
|
||||
{% endunless %}
|
||||
|
||||
<div class="main-content">
|
||||
{{content}}
|
||||
</div>
|
||||
|
||||
<!-- Adds tags, if any -->
|
||||
<div class="tags">
|
||||
{% if page.tags != null %}
|
||||
<b>Tags: </b>
|
||||
{% assign projectTags = site.data.tags.allowed-tags %}
|
||||
{% for tag in page.tags %}
|
||||
{% if projectTags contains tag %}
|
||||
<a href="{{ "tag-" | append: tag | append: ".html" }}" class="btn btn-default navbar-btn cursorNorm" role="button">{{page.tagName}}{{tag}}</a>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<!-- Adds nav links on each page -->
|
||||
{% assign scrollnav = site.data.scrollnav[page.shortname] %}
|
||||
{% if scrollnav %}
|
||||
<div class="scrollnav">
|
||||
{% if scrollnav.prev %}
|
||||
<a class="btn btn-primary prev" href="{{ scrollnav.prev.url }}">{{ scrollnav.prev.title }}</a>
|
||||
{% endif %}
|
||||
{% if scrollnav.next %}
|
||||
<a class="btn btn-primary next" href="{{ scrollnav.next.url }}">{{ scrollnav.next.title }}</a>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
</div>
|
||||
|
||||
<!-- Adds comments from Apache's Comment system -->
|
||||
|
||||
<div id="comments_thread">
|
||||
</div>
|
||||
<script type="text/javascript" src="https://comments.apache.org/show_comments.lua?site=solr-refguide&style=css/comments.css&page={{ page.shortname }}" async="true">
|
||||
</script>
|
||||
<noscript>
|
||||
<iframe width="100%" height="500" src="https://comments.apache.org/iframe.lua?site=solr-refguide&style=css/comments.css&page={{ page.shortname }}"></iframe>
|
||||
</noscript>
|
||||
|
||||
{% include footer.html %}
|
|
@ -0,0 +1,15 @@
|
|||
---
|
||||
layout: default_print
|
||||
comments: true
|
||||
---
|
||||
<div class="post-header">
|
||||
<h1 class="post-title-main" id="{{page.permalink | replace: '/', '' }}">{{ page.title }}</h1>
|
||||
</div>
|
||||
|
||||
<div class="post-content">
|
||||
|
||||
{% if page.summary %}
|
||||
<div class="summary">{{page.summary}}</div>
|
||||
{% endif %}
|
||||
{{ content }}
|
||||
</div>
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
layout: default
|
||||
---
|
||||
<article class="post" itemscope itemtype="http://schema.org/BlogPosting">
|
||||
|
||||
<header class="post-header">
|
||||
<h1 class="post-title" itemprop="name headline">{{ page.title }}</h1>
|
||||
<p class="post-meta"><time datetime="{{ page.date | date_to_xmlschema }}" itemprop="datePublished">{{ page.date | date: "%b %-d, %Y" }}</time> {% if page.author %}<span itemprop="author" itemscope itemtype="http://schema.org/Person"><span itemprop="name">/ {{ page.author }}</span></span>{% endif %}{% if page.tags != null %}/
|
||||
{% assign projectTags = site.data.tags.allowed-tags %}
|
||||
{% for tag in page.tags %}
|
||||
{% if projectTags contains tag %}
|
||||
<a href="{{ "../tag_" | append: tag | append: ".html" }}">{{tag}}</a>{% unless forloop.last %}, {% endunless%}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
|
||||
</p>
|
||||
|
||||
|
||||
</header>
|
||||
|
||||
<div class="post-content" itemprop="articleBody">
|
||||
|
||||
{% if page.summary %}
|
||||
<div class="summary">{{page.summary}}</div>
|
||||
{% endif %}
|
||||
|
||||
{{ content }}
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
</article>
|
||||
|
||||
{% if site.disqus %}
|
||||
{% include disqus.html %}
|
||||
{% endif %}
|
||||
|
||||
{{site.data.alerts.hr_shaded}}
|
||||
|
||||
{% include footer.html %}
|
|
@ -0,0 +1,33 @@
|
|||
= A Quick Overview
|
||||
:page-shortname: a-quick-overview
|
||||
:page-permalink: a-quick-overview.html
|
||||
|
||||
Having had some fun with Solr, you will now learn about all the cool things it can do.
|
||||
|
||||
Here is a example of how Solr might be integrated into an application:
|
||||
|
||||
.Solr integration with applications
|
||||
image::images/a-quick-overview/sample-client-app-arch.png[image,width=500,height=379]
|
||||
|
||||
In the scenario above, Solr runs along side other server applications. For example, an online store application would provide a user interface, a shopping cart, and a way to make purchases for end users; while an inventory management application would allow store employees to edit product information. The product metadata would be kept in some kind of database, as well as in Solr.
|
||||
|
||||
Solr makes it easy to add the capability to search through the online store through the following steps:
|
||||
|
||||
. Define a _schema_. The schema tells Solr about the contents of documents it will be indexing. In the online store example, the schema would define fields for the product name, description, price, manufacturer, and so on. Solr's schema is powerful and flexible and allows you to tailor Solr's behavior to your application. See <<documents-fields-and-schema-design.adoc#documents-fields-and-schema-design,Documents, Fields, and Schema Design>> for all the details.
|
||||
. Deploy Solr.
|
||||
. Feed Solr documents for which your users will search.
|
||||
. Expose search functionality in your application.
|
||||
|
||||
Because Solr is based on open standards, it is highly extensible. Solr queries are RESTful, which means, in essence, that a query is a simple HTTP request URL and the response is a structured document: mainly XML, but it could also be JSON, CSV, or some other format. This means that a wide variety of clients will be able to use Solr, from other web applications to browser clients, rich client applications, and mobile devices. Any platform capable of HTTP can talk to Solr. See <<client-apis.adoc#client-apis,Client APIs>> for details on client APIs.
|
||||
|
||||
Solr is based on the Apache Lucene project, a high-performance, full-featured search engine. Solr offers support for the simplest keyword searching through to complex queries on multiple fields and faceted search results. <<searching.adoc#searching,Searching>> has more information about searching and queries.
|
||||
|
||||
If Solr's capabilities are not impressive enough, its ability to handle very high-volume applications should do the trick.
|
||||
|
||||
A relatively common scenario is that you have so much data, or so many queries, that a single Solr server is unable to handle your entire workload. In this case, you can scale up the capabilities of your application using <<solrcloud.adoc#solrcloud,SolrCloud>> to better distribute the data, and the processing of requests, across many servers. Multiple options can be mixed and matched depending on the type of scalability you need.
|
||||
|
||||
For example: "Sharding" is a scaling technique in which a collection is split into multiple logical pieces called "shards" in order to scale up the number of documents in a collection beyond what could physically fit on a single server. Incoming queries are distributed to every shard in the collection, which respond with merged results. Another technique available is to increase the "Replication Factor" of your collection, which allows you to add servers with additional copies of your collection to handle higher concurrent query load by spreading the requests around to multiple machines. Sharding and Replication are not mutually exclusive, and together make Solr an extremely powerful and scalable platform.
|
||||
|
||||
Best of all, this talk about high-volume applications is not just hypothetical: some of the famous Internet sites that use Solr today are Macy's, EBay, and Zappo's.
|
||||
|
||||
For more information, take a look at https://wiki.apache.org/solr/PublicServers.
|
|
@ -0,0 +1,54 @@
|
|||
= A Step Closer
|
||||
:page-shortname: a-step-closer
|
||||
:page-permalink: a-step-closer.html
|
||||
|
||||
You already have some idea of Solr's schema. This section describes Solr's home directory and other configuration options.
|
||||
|
||||
When Solr runs in an application server, it needs access to a home directory. The home directory contains important configuration information and is the place where Solr will store its index. The layout of the home directory will look a little different when you are running Solr in standalone mode vs when you are running in SolrCloud mode.
|
||||
|
||||
The crucial parts of the Solr home directory are shown in these examples:
|
||||
|
||||
.Standalone Mode
|
||||
[source,plain]
|
||||
----
|
||||
<solr-home-directory>/
|
||||
solr.xml
|
||||
core_name1/
|
||||
core.properties
|
||||
conf/
|
||||
solrconfig.xml
|
||||
managed-schema
|
||||
data/
|
||||
core_name2/
|
||||
core.properties
|
||||
conf/
|
||||
solrconfig.xml
|
||||
managed-schema
|
||||
data/
|
||||
----
|
||||
|
||||
.SolrCloud Mode
|
||||
[source,plain]
|
||||
----
|
||||
<solr-home-directory>/
|
||||
solr.xml
|
||||
core_name1/
|
||||
core.properties
|
||||
data/
|
||||
core_name2/
|
||||
core.properties
|
||||
data/
|
||||
----
|
||||
|
||||
You may see other files, but the main ones you need to know are:
|
||||
|
||||
* `solr.xml` specifies configuration options for your Solr server instance. For more information on `solr.xml` see <<solr-cores-and-solr-xml.adoc#solr-cores-and-solr-xml,Solr Cores and solr.xml>>.
|
||||
* Per Solr Core:
|
||||
** `core.properties` defines specific properties for each core such as its name, the collection the core belongs to, the location of the schema, and other parameters. For more details on `core.properties`, see the section <<defining-core-properties.adoc#defining-core-properties,Defining core.properties>>.
|
||||
** `solrconfig.xml` controls high-level behavior. You can, for example, specify an alternate location for the data directory. For more information on `solrconfig.xml`, see <<configuring-solrconfig-xml.adoc#configuring-solrconfig-xml,Configuring solrconfig.xml>>.
|
||||
** `managed-schema` (or `schema.xml` instead) describes the documents you will ask Solr to index. The Schema define a document as a collection of fields. You get to define both the field types and the fields themselves. Field type definitions are powerful and include information about how Solr processes incoming field values and query values. For more information on Solr Schemas, see <<documents-fields-and-schema-design.adoc#documents-fields-and-schema-design,Documents, Fields, and Schema Design>> and the <<schema-api.adoc#schema-api,Schema API>>.
|
||||
** `data/` The directory containing the low level index files.
|
||||
|
||||
Note that the SolrCloud example does not include a `conf` directory for each Solr Core (so there is no `solrconfig.xml` or Schema file). This is because the configuration files usually found in the `conf` directory are stored in ZooKeeper so they can be propagated across the cluster.
|
||||
|
||||
If you are using SolrCloud with the embedded ZooKeeper instance, you may also see `zoo.cfg` and `zoo.data` which are ZooKeeper configuration and data files. However, if you are running your own ZooKeeper ensemble, you would supply your own ZooKeeper configuration file when you start it and the copies in Solr would be unused. For more information about ZooKeeper and SolrCloud, see the section <<solrcloud.adoc#solrcloud,SolrCloud>>.
|
|
@ -0,0 +1,29 @@
|
|||
= About Filters
|
||||
:page-shortname: about-filters
|
||||
:page-permalink: about-filters.html
|
||||
|
||||
Like <<tokenizers.adoc#tokenizers,tokenizers>>, <<filter-descriptions.adoc#filter-descriptions,filters>> consume input and produce a stream of tokens. Filters also derive from `org.apache.lucene.analysis.TokenStream`. Unlike tokenizers, a filter's input is another TokenStream. The job of a filter is usually easier than that of a tokenizer since in most cases a filter looks at each token in the stream sequentially and decides whether to pass it along, replace it or discard it.
|
||||
|
||||
A filter may also do more complex analysis by looking ahead to consider multiple tokens at once, although this is less common. One hypothetical use for such a filter might be to normalize state names that would be tokenized as two words. For example, the single token "california" would be replaced with "CA", while the token pair "rhode" followed by "island" would become the single token "RI".
|
||||
|
||||
Because filters consume one `TokenStream` and produce a new `TokenStream`, they can be chained one after another indefinitely. Each filter in the chain in turn processes the tokens produced by its predecessor. The order in which you specify the filters is therefore significant. Typically, the most general filtering is done first, and later filtering stages are more specialized.
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<fieldType name="text" class="solr.TextField">
|
||||
<analyzer>
|
||||
<tokenizer class="solr.StandardTokenizerFactory"/>
|
||||
<filter class="solr.StandardFilterFactory"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.EnglishPorterFilterFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
----
|
||||
|
||||
This example starts with Solr's standard tokenizer, which breaks the field's text into tokens. Those tokens then pass through Solr's standard filter, which removes dots from acronyms, and performs a few other common operations. All the tokens are then set to lowercase, which will facilitate case-insensitive matching at query time.
|
||||
|
||||
The last filter in the above example is a stemmer filter that uses the Porter stemming algorithm. A stemmer is basically a set of mapping rules that maps the various forms of a word back to the base, or _stem_, word from which they derive. For example, in English the words "hugs", "hugging" and "hugged" are all forms of the stem word "hug". The stemmer will replace all of these terms with "hug", which is what will be indexed. This means that a query for "hug" will match the term "hugged", but not "huge".
|
||||
|
||||
Conversely, applying a stemmer to your query terms will allow queries containing non stem terms, like "hugging", to match documents with different variations of the same stem word, such as "hugged". This works because both the indexer and the query will map to the same stem ("hug").
|
||||
|
||||
Word stemming is, obviously, very language specific. Solr includes several language-specific stemmers created by the http://snowball.tartarus.org/[Snowball] generator that are based on the Porter stemming algorithm. The generic Snowball Porter Stemmer Filter can be used to configure any of these language stemmers. Solr also includes a convenience wrapper for the English Snowball stemmer. There are also several purpose-built stemmers for non-English languages. These stemmers are described in <<language-analysis.adoc#language-analysis,Language Analysis>>.
|
|
@ -0,0 +1,57 @@
|
|||
= About This Guide
|
||||
:page-shortname: about-this-guide
|
||||
:page-permalink: about-this-guide.html
|
||||
|
||||
This guide describes all of the important features and functions of Apache Solr.
|
||||
|
||||
Solr is free to download from http://lucene.apache.org/solr/.
|
||||
|
||||
Designed to provide high-level documentation, this guide is intended to be more encyclopedic and less of a cookbook. It is structured to address a broad spectrum of needs, ranging from new developers getting started to well-experienced developers extending their application or troubleshooting. It will be of use at any point in the application life cycle, for whenever you need authoritative information about Solr.
|
||||
|
||||
The material as presented assumes that you are familiar with some basic search concepts and that you can read XML. It does not assume that you are a Java programmer, although knowledge of Java is helpful when working directly with Lucene or when developing custom extensions to a Lucene/Solr installation.
|
||||
|
||||
[[AboutThisGuide-SpecialInlineNotes]]
|
||||
== Special Inline Notes
|
||||
|
||||
Special notes are included throughout these pages. There are several types of notes:
|
||||
|
||||
Information blocks::
|
||||
+
|
||||
NOTE: These provide additional information that's useful for you to know.
|
||||
|
||||
Important::
|
||||
+
|
||||
IMPORTANT: These provide information that is critical for you to know.
|
||||
|
||||
Tip::
|
||||
+
|
||||
TIP: These provide helpful tips.
|
||||
|
||||
Caution::
|
||||
+
|
||||
CAUTION: These provide details on scenarios or configurations you should be careful with.
|
||||
|
||||
Warning::
|
||||
+
|
||||
WARNING: These are meant to warn you from a possibly dangerous change or action.
|
||||
|
||||
|
||||
[[AboutThisGuide-HostsandPortExamples]]
|
||||
== Hosts and Port Examples
|
||||
|
||||
The default port when running Solr is 8983. The samples, URLs and screenshots in this guide may show different ports, because the port number that Solr uses is configurable. If you have not customized your installation of Solr, please make sure that you use port 8983 when following the examples, or configure your own installation to use the port numbers shown in the examples. For information about configuring port numbers, see the section <<managing-solr.adoc#managing-solr,Managing Solr>>.
|
||||
|
||||
Similarly, URL examples use 'localhost' throughout; if you are accessing Solr from a location remote to the server hosting Solr, replace 'localhost' with the proper domain or IP where Solr is running.
|
||||
|
||||
For example, we might provide a sample query like:
|
||||
|
||||
`\http://localhost:8983/solr/gettingstarted/select?q=brown+cow`
|
||||
|
||||
There are several items in this URL you might need to change locally. First, if your server is running at "www.example.com", you'll replace "localhost" with the proper domain. If you aren't using port 8983, you'll replace that also. Finally, you'll want to replace "gettingstarted" (the collection or core name) with the proper one in use in your implementation. The URL would then become:
|
||||
|
||||
`\http://www.example.com/solr/mycollection/select?q=brown+cow`
|
||||
|
||||
[[AboutThisGuide-Paths]]
|
||||
== Paths
|
||||
|
||||
Path information is given relative to `solr.home`, which is the location under the main Solr installation where Solr's collections and their `conf` and `data` directories are stored. When running the various examples mentioned through out this tutorial (i.e., `bin/solr -e techproducts`) the `solr.home` will be a sub-directory of `example/` created for you automatically.
|
|
@ -0,0 +1,31 @@
|
|||
= About Tokenizers
|
||||
:page-shortname: about-tokenizers
|
||||
:page-permalink: about-tokenizers.html
|
||||
|
||||
The job of a <<tokenizers.adoc#tokenizers,tokenizer>> is to break up a stream of text into tokens, where each token is (usually) a sub-sequence of the characters in the text. An analyzer is aware of the field it is configured for, but a tokenizer is not. Tokenizers read from a character stream (a Reader) and produce a sequence of Token objects (a TokenStream).
|
||||
|
||||
Characters in the input stream may be discarded, such as whitespace or other delimiters. They may also be added to or replaced, such as mapping aliases or abbreviations to normalized forms. A token contains various metadata in addition to its text value, such as the location at which the token occurs in the field. Because a tokenizer may produce tokens that diverge from the input text, you should not assume that the text of the token is the same text that occurs in the field, or that its length is the same as the original text. It's also possible for more than one token to have the same position or refer to the same offset in the original text. Keep this in mind if you use token metadata for things like highlighting search results in the field text.
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<fieldType name="text" class="solr.TextField">
|
||||
<analyzer>
|
||||
<tokenizer class="solr.StandardTokenizerFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
----
|
||||
|
||||
The class named in the tokenizer element is not the actual tokenizer, but rather a class that implements the `TokenizerFactory` API. This factory class will be called upon to create new tokenizer instances as needed. Objects created by the factory must derive from `Tokenizer`, which indicates that they produce sequences of tokens. If the tokenizer produces tokens that are usable as is, it may be the only component of the analyzer. Otherwise, the tokenizer's output tokens will serve as input to the first filter stage in the pipeline.
|
||||
|
||||
A `TypeTokenFilterFactory` is available that creates a `TypeTokenFilter` that filters tokens based on their TypeAttribute, which is set in `factory.getStopTypes`.
|
||||
|
||||
For a complete list of the available TokenFilters, see the section <<tokenizers.adoc#tokenizers,Tokenizers>>.
|
||||
|
||||
[[AboutTokenizers-WhenTouseaCharFiltervs.aTokenFilter]]
|
||||
== When To use a CharFilter vs. a TokenFilter
|
||||
|
||||
There are several pairs of CharFilters and TokenFilters that have related (ie: `MappingCharFilter` and `ASCIIFoldingFilter`) or nearly identical (ie: `PatternReplaceCharFilterFactory` and `PatternReplaceFilterFactory`) functionality and it may not always be obvious which is the best choice.
|
||||
|
||||
The decision about which to use depends largely on which Tokenizer you are using, and whether you need to preprocess the stream of characters.
|
||||
|
||||
For example, suppose you have a tokenizer such as `StandardTokenizer` and although you are pretty happy with how it works overall, you want to customize how some specific characters behave. You could modify the rules and re-build your own tokenizer with JFlex, but it might be easier to simply map some of the characters before tokenization with a `CharFilter`.
|
|
@ -0,0 +1,156 @@
|
|||
= Adding Custom Plugins in SolrCloud Mode
|
||||
:page-shortname: adding-custom-plugins-in-solrcloud-mode
|
||||
:page-permalink: adding-custom-plugins-in-solrcloud-mode.html
|
||||
|
||||
In SolrCloud mode, custom plugins need to be shared across all nodes of the cluster.
|
||||
|
||||
When running Solr in SolrCloud mode and you want to use custom code (such as custom analyzers, tokenizers, query parsers, and other plugins), it can be cumbersome to add jars to the classpath on all nodes in your cluster. Using the <<blob-store-api.adoc#blob-store-api,Blob Store API>> and special commands with the <<config-api.adoc#config-api,Config API>>, you can upload jars to a special system-level collection and dynamically load plugins from them at runtime with out needing to restart any nodes.
|
||||
|
||||
.This Feature is Disabled By Default
|
||||
[IMPORTANT]
|
||||
====
|
||||
In addition to requiring that Solr by running in <<solrcloud.adoc#solrcloud,SolrCloud>> mode, this feature is also disabled by default unless all Solr nodes are run with the `-Denable.runtime.lib=true` option on startup.
|
||||
|
||||
Before enabling this feature, users should carefully consider the issues discussed in the <<Securing Runtime Libraries>> section below.
|
||||
====
|
||||
|
||||
[[AddingCustomPluginsinSolrCloudMode-UploadingJarFiles]]
|
||||
== Uploading Jar Files
|
||||
|
||||
The first step is to use the <<blob-store-api.adoc#blob-store-api,Blob Store API>> to upload your jar files. This will to put your jars in the `.system` collection and distribute them across your SolrCloud nodes. These jars are added to a separate classloader and only accessible to components that are configured with the property `runtimeLib=true`. These components are loaded lazily because the `.system` collection may not be loaded when a particular core is loaded.
|
||||
|
||||
[[AddingCustomPluginsinSolrCloudMode-ConfigAPICommandstouseJarsasRuntimeLibraries]]
|
||||
== Config API Commands to use Jars as Runtime Libraries
|
||||
|
||||
The runtime library feature uses a special set of commands for the <<config-api.adoc#config-api,Config API>> to add, update, or remove jar files currently available in the blob store to the list of runtime libraries.
|
||||
|
||||
The following commands are used to manage runtime libs:
|
||||
|
||||
* `add-runtimelib`
|
||||
* `update-runtimelib`
|
||||
* `delete-runtimelib`
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
|
||||
"add-runtimelib": { "name":"jarblobname", "version":2 },
|
||||
"update-runtimelib": { "name":"jarblobname", "version":3 },
|
||||
"delete-runtimelib": "jarblobname"
|
||||
}'
|
||||
----
|
||||
|
||||
The name to use is the name of the blob that you specified when you uploaded your jar to the blob store. You should also include the version of the jar found in the blob store that you want to use. These details are added to `configoverlay.json`.
|
||||
|
||||
The default `SolrResourceLoader` does not have visibility to the jars that have been defined as runtime libraries. There is a classloader that can access these jars which is made available only to those components which are specially annotated.
|
||||
|
||||
Every pluggable component can have an optional extra attribute called `runtimeLib=true`, which means that the components are not loaded at core load time. Instead, they will be loaded on demand. If all the dependent jars are not available when the component is loaded, an error is thrown.
|
||||
|
||||
This example shows creating a ValueSourceParser using a jar that has been loaded to the Blob store.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
|
||||
"create-valuesourceparser": {
|
||||
"name": "nvl",
|
||||
"runtimeLib": true,
|
||||
"class": "solr.org.apache.solr.search.function.NvlValueSourceParser,
|
||||
"nvlFloatValue": 0.0 }
|
||||
}'
|
||||
----
|
||||
|
||||
[[AddingCustomPluginsinSolrCloudMode-SecuringRuntimeLibraries]]
|
||||
== Securing Runtime Libraries
|
||||
|
||||
A drawback of this feature is that it could be used to load malicious executable code into the system. However, it is possible to restrict the system to load only trusted jars using http://en.wikipedia.org/wiki/Public_key_infrastructure[PKI] to verify that the executables loaded into the system are trustworthy.
|
||||
|
||||
The following steps will allow you enable security for this feature. The instructions assume you have started all your Solr nodes with the `-Denable.runtime.lib=true`.
|
||||
|
||||
[[Step1_GenerateanRSAPrivateKey]]
|
||||
=== Step 1: Generate an RSA Private Key
|
||||
|
||||
The first step is to generate an RSA private key. The example below uses a 512-bit key, but you should use the strength appropriate to your needs.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
$ openssl genrsa -out priv_key.pem 512
|
||||
----
|
||||
|
||||
[[Step2_OutputthePublicKey]]
|
||||
=== Step 2: Output the Public Key
|
||||
|
||||
The public portion of the key should be output in DER format so Java can read it.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
$ openssl rsa -in priv_key.pem -pubout -outform DER -out pub_key.der
|
||||
----
|
||||
|
||||
[[Step3_LoadtheKeytoZooKeeper]]
|
||||
=== Step 3: Load the Key to ZooKeeper
|
||||
|
||||
The `.der` files that are output from Step 2 should then be loaded to ZooKeeper under a node `/keys/exe` so they are available throughout every node. You can load any number of public keys to that node and all are valid. If a key is removed from the directory, the signatures of that key will cease to be valid. So, before removing the a key, make sure to update your runtime library configurations with valid signatures with the `update-runtimelib` command.
|
||||
|
||||
At the current time, you can only use the ZooKeeper `zkCli.sh` (or `zkCli.cmd` on Windows) script to issue these commands (the Solr version has the same name, but is not the same). If you have your own ZooKeeper ensemble running already, you can find the script in `$ZK_INSTALL/bin/zkCli.sh` (or `zkCli.cmd` if you are using Windows).
|
||||
|
||||
NOTE: If you are running the embedded ZooKeeper that is included with Solr, you *do not* have this script already; in order to use it, you will need to download a copy of ZooKeeper v3.4.6 from http://zookeeper.apache.org/. Don't worry about configuring the download, you're just trying to get the command line utility script. When you start the script, you will connect to the embedded ZooKeeper.
|
||||
|
||||
To load the keys, you will need to connect to ZooKeeper with `zkCli.sh`, create the directories, and then create the key file, as in the following example.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
# Connect to ZooKeeper
|
||||
# Replace the server location below with the correct ZooKeeper connect string for your installation.
|
||||
$ .bin/zkCli.sh -server localhost:9983
|
||||
|
||||
# After connection, you will interact with the ZK prompt.
|
||||
# Create the directories
|
||||
[zk: localhost:9983(CONNECTED) 5] create /keys
|
||||
[zk: localhost:9983(CONNECTED) 5] create /keys/exe
|
||||
|
||||
# Now create the public key file in ZooKeeper
|
||||
# The second path is the path to the .der file on your local machine
|
||||
[zk: localhost:9983(CONNECTED) 5] create /keys/exe/pub_key.der /myLocal/pathTo/pub_key.der
|
||||
----
|
||||
|
||||
After this, any attempt to load a jar will fail. All your jars must be signed with one of your private keys for Solr to trust it. The process to sign your jars and use the signature is outlined in Steps 4-6.
|
||||
|
||||
[[Step4_SignthejarFile]]
|
||||
=== Step 4: Sign the jar File
|
||||
|
||||
Next you need to sign the sha1 digest of your jar file and get the base64 string.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
$ openssl dgst -sha1 -sign priv_key.pem myjar.jar | openssl enc -base64
|
||||
----
|
||||
|
||||
The output of this step will be a string that you will need to add the jar to your classpath in Step 6 below.
|
||||
|
||||
[[Step5_LoadthejartotheBlobStore]]
|
||||
=== Step 5: Load the jar to the Blob Store
|
||||
|
||||
Load your jar to the Blob store, using the <<blob-store-api.adoc#blob-store-api,Blob Store API>>. This step does not require a signature; you will need the signature in Step 6 to add it to your classpath.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @{filename}
|
||||
http://localhost:8983/solr/.system/blob/{blobname}
|
||||
----
|
||||
|
||||
The blob name that you give the jar file in this step will be used as the name in the next step.
|
||||
|
||||
[[Step6_AddthejartotheClasspath]]
|
||||
=== Step 6: Add the jar to the Classpath
|
||||
|
||||
Finally, add the jar to the classpath using the Config API as detailed above. In this step, you will need to provide the signature of the jar that you got in Step 4.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
|
||||
"add-runtimelib": {
|
||||
"name":"blobname",
|
||||
"version":2,
|
||||
"sig":"mW1Gwtz2QazjfVdrLFHfbGwcr8xzFYgUOLu68LHqWRDvLG0uLcy1McQ+AzVmeZFBf1yLPDEHBWJb5KXr8bdbHN/
|
||||
PYgUB1nsr9pk4EFyD9KfJ8TqeH/ijQ9waa/vjqyiKEI9U550EtSzruLVZ32wJ7smvV0fj2YYhrUaaPzOn9g0=" }
|
||||
}'
|
||||
----
|
|
@ -0,0 +1,17 @@
|
|||
= Analysis Screen
|
||||
:page-shortname: analysis-screen
|
||||
:page-permalink: analysis-screen.html
|
||||
|
||||
The Analysis screen lets you inspect how data will be handled according to the field, field type and dynamic field configurations found in your Schema. You can analyze how content would be handled during indexing or during query processing and view the results separately or at the same time. Ideally, you would want content to be handled consistently, and this screen allows you to validate the settings in the field type or field analysis chains.
|
||||
|
||||
Enter content in one or both boxes at the top of the screen, and then choose the field or field type definitions to use for analysis.
|
||||
|
||||
image::images/analysis-screen/analysis_normal.png[image,height=400]
|
||||
|
||||
If you click the *Verbose Output* check box, you see more information, including more details on the transformations to the input (such as, convert to lower case, strip extra characters, etc.) including the raw bytes, type and detailed position information at each stage. The information displayed will vary depending on the settings of the field or field type. Each step of the process is displayed in a separate section, with an abbreviation for the tokenizer or filter that is applied in that step. Hover or click on the abbreviation, and you'll see the name and path of the tokenizer or filter.
|
||||
|
||||
image::images/analysis-screen/analysis_verbose.png[image,height=400]
|
||||
|
||||
In the example screenshot above, several transformations are applied to the input "Running is a sport." The words "is" and "a" have been removed and the word "running" has been changed to its basic form, "run". This is because we are using the field type `text_en` in this scenario, which is configured to remove stop words (small words that usually do not provide a great deal of context) and "stem" terms when possible to find more possible matches (this is particularly helpful with plural forms of words). If you click the question mark next to the *Analyze Fieldname/Field Type* pull-down menu, the <<schema-browser-screen.adoc#schema-browser-screen,Schema Browser window>> will open, showing you the settings for the field specified.
|
||||
|
||||
The section <<understanding-analyzers-tokenizers-and-filters.adoc#understanding-analyzers-tokenizers-and-filters,Understanding Analyzers, Tokenizers, and Filters>> describes in detail what each option is and how it may transform your data and the section <<running-your-analyzer.adoc#running-your-analyzer,Running Your Analyzer>> has specific examples for using the Analysis screen.
|
|
@ -0,0 +1,103 @@
|
|||
= Analyzers
|
||||
:page-shortname: analyzers
|
||||
:page-permalink: analyzers.html
|
||||
|
||||
An analyzer examines the text of fields and generates a token stream.
|
||||
|
||||
Analyzers are specified as a child of the `<fieldType>` element in the `schema.xml` configuration file (in the same `conf/` directory as `solrconfig.xml`).
|
||||
|
||||
In normal usage, only fields of type `solr.TextField` will specify an analyzer. The simplest way to configure an analyzer is with a single `<analyzer>` element whose class attribute is a fully qualified Java class name. The named class must derive from `org.apache.lucene.analysis.Analyzer`. For example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<fieldType name="nametext" class="solr.TextField">
|
||||
<analyzer class="org.apache.lucene.analysis.core.WhitespaceAnalyzer"/>
|
||||
</fieldType>
|
||||
----
|
||||
|
||||
In this case a single class, `WhitespaceAnalyzer`, is responsible for analyzing the content of the named text field and emitting the corresponding tokens. For simple cases, such as plain English prose, a single analyzer class like this may be sufficient. But it's often necessary to do more complex analysis of the field content.
|
||||
|
||||
Even the most complex analysis requirements can usually be decomposed into a series of discrete, relatively simple processing steps. As you will soon discover, the Solr distribution comes with a large selection of tokenizers and filters that covers most scenarios you are likely to encounter. Setting up an analyzer chain is very straightforward; you specify a simple `<analyzer>` element (no class attribute) with child elements that name factory classes for the tokenizer and filters to use, in the order you want them to run.
|
||||
|
||||
For example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<fieldType name="nametext" class="solr.TextField">
|
||||
<analyzer>
|
||||
<tokenizer class="solr.StandardTokenizerFactory"/>
|
||||
<filter class="solr.StandardFilterFactory"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.StopFilterFactory"/>
|
||||
<filter class="solr.EnglishPorterFilterFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
----
|
||||
|
||||
Note that classes in the `org.apache.solr.analysis` package may be referred to here with the shorthand `solr.` prefix.
|
||||
|
||||
In this case, no Analyzer class was specified on the `<analyzer>` element. Rather, a sequence of more specialized classes are wired together and collectively act as the Analyzer for the field. The text of the field is passed to the first item in the list (`solr.StandardTokenizerFactory`), and the tokens that emerge from the last one (`solr.EnglishPorterFilterFactory`) are the terms that are used for indexing or querying any fields that use the "nametext" `fieldType`.
|
||||
|
||||
.Field Values versus Indexed Terms
|
||||
[IMPORTANT]
|
||||
====
|
||||
The output of an Analyzer affects the _terms_ indexed in a given field (and the terms used when parsing queries against those fields) but it has no impact on the _stored_ value for the fields. For example: an analyzer might split "Brown Cow" into two indexed terms "brown" and "cow", but the stored value will still be a single String: "Brown Cow"
|
||||
====
|
||||
|
||||
[[Analyzers-AnalysisPhases]]
|
||||
== Analysis Phases
|
||||
|
||||
Analysis takes place in two contexts. At index time, when a field is being created, the token stream that results from analysis is added to an index and defines the set of terms (including positions, sizes, and so on) for the field. At query time, the values being searched for are analyzed and the terms that result are matched against those that are stored in the field's index.
|
||||
|
||||
In many cases, the same analysis should be applied to both phases. This is desirable when you want to query for exact string matches, possibly with case-insensitivity, for example. In other cases, you may want to apply slightly different analysis steps during indexing than those used at query time.
|
||||
|
||||
If you provide a simple `<analyzer>` definition for a field type, as in the examples above, then it will be used for both indexing and queries. If you want distinct analyzers for each phase, you may include two `<analyzer>` definitions distinguished with a type attribute. For example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<fieldType name="nametext" class="solr.TextField">
|
||||
<analyzer type="index">
|
||||
<tokenizer class="solr.StandardTokenizerFactory"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.KeepWordFilterFactory" words="keepwords.txt"/>
|
||||
<filter class="solr.SynonymFilterFactory" synonyms="syns.txt"/>
|
||||
</analyzer>
|
||||
<analyzer type="query">
|
||||
<tokenizer class="solr.StandardTokenizerFactory"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
----
|
||||
|
||||
In this theoretical example, at index time the text is tokenized, the tokens are set to lowercase, any that are not listed in `keepwords.txt` are discarded and those that remain are mapped to alternate values as defined by the synonym rules in the file `syns.txt`. This essentially builds an index from a restricted set of possible values and then normalizes them to values that may not even occur in the original text.
|
||||
|
||||
At query time, the only normalization that happens is to convert the query terms to lowercase. The filtering and mapping steps that occur at index time are not applied to the query terms. Queries must then, in this example, be very precise, using only the normalized terms that were stored at index time.
|
||||
|
||||
[[Analyzers-AnalysisforMulti-TermExpansion]]
|
||||
=== Analysis for Multi-Term Expansion
|
||||
|
||||
In some types of queries (ie: Prefix, Wildcard, Regex, etc...) the input provided by the user is not natural language intended for Analysis. Things like Synonyms or Stop word filtering do not work in a logical way in these types of Queries.
|
||||
|
||||
The analysis factories that _can_ work in these types of queries (such as Lowercasing, or Normalizing Factories) are known as {lucene-javadocs}/analyzers-common/org/apache/lucene/analysis/util/MultiTermAwareComponent.html[`MultiTermAwareComponents`]. When Solr needs to perform analysis for a query that results in Multi-Term expansion, only the `MultiTermAwareComponents` used in the `query` analyzer are used, Factory that is not Multi-Term aware will be skipped.
|
||||
|
||||
For most use cases, this provides the best possible behavior, but if you wish for absolute control over the analysis performed on these types of queries, you may explicitly define a `multiterm` analyzer to use, such as in the following example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<fieldType name="nametext" class="solr.TextField">
|
||||
<analyzer type="index">
|
||||
<tokenizer class="solr.StandardTokenizerFactory"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.KeepWordFilterFactory" words="keepwords.txt"/>
|
||||
<filter class="solr.SynonymFilterFactory" synonyms="syns.txt"/>
|
||||
</analyzer>
|
||||
<analyzer type="query">
|
||||
<tokenizer class="solr.StandardTokenizerFactory"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
</analyzer>
|
||||
<!-- No analysis at all when doing queries that involved Multi-Term expansion -->
|
||||
<analyzer type="multiterm">
|
||||
<tokenizer class="solr.KeywordTokenizerFactory" />
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
----
|
|
@ -0,0 +1,170 @@
|
|||
= Authentication and Authorization Plugins
|
||||
:page-shortname: authentication-and-authorization-plugins
|
||||
:page-permalink: authentication-and-authorization-plugins.html
|
||||
:page-children: basic-authentication-plugin, hadoop-authentication-plugin, kerberos-authentication-plugin, rule-based-authorization-plugin
|
||||
|
||||
Solr has security frameworks for supporting authentication and authorization of users. This allows for verifying a user's identity and for restricting access to resources in a Solr cluster.
|
||||
|
||||
Solr includes some plugins out of the box, and additional plugins can be developed using the authentication and authorization frameworks described below.
|
||||
|
||||
All authentication and authorization plugins can work with Solr whether they are running in SolrCloud mode or standalone mode. All authentication and authorization configuration, including users and permission rules, are stored in a file named `security.json`. When using Solr in standalone mode, this file must be in the `$SOLR_HOME` directory (usually `server/solr`). When using SolrCloud, this file must be located in ZooKeeper.
|
||||
|
||||
The following section describes how to enable plugins with `security.json` and place them in the proper locations for your mode of operation.
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-EnablePluginswithsecurity.json]]
|
||||
== Enable Plugins with security.json
|
||||
|
||||
All of the information required to initialize either type of security plugin is stored in a `security.json` file. This file contains 2 sections, one each for authentication and authorization.
|
||||
|
||||
.Sample security.json
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"authentication" : {
|
||||
"class": "class.that.implements.authentication"
|
||||
},
|
||||
"authorization": {
|
||||
"class": "class.that.implements.authorization"
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
The `/security.json` file needs to be in the proper location before a Solr instance comes up so Solr starts with the security plugin enabled. See the section <<AuthenticationandAuthorizationPlugins-Usingsecurity.jsonwithSolr,Using security.json with Solr>> below for information on how to do this.
|
||||
|
||||
Depending on the plugin(s) in use, other information will be stored in `security.json` such as user information or rules to create roles and permissions. This information is added through the APIs for each plugin provided by Solr, or, in the case of a custom plugin, the approach designed by you.
|
||||
|
||||
Here is a more detailed `security.json` example. In this, the Basic authentication and rule-based authorization plugins are enabled, and some data has been added:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"authentication":{
|
||||
"class":"solr.BasicAuthPlugin",
|
||||
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
|
||||
},
|
||||
"authorization":{
|
||||
"class":"solr.RuleBasedAuthorizationPlugin",
|
||||
"permissions":[{"name":"security-edit",
|
||||
"role":"admin"}],
|
||||
"user-role":{"solr":"admin"}
|
||||
}}
|
||||
----
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-Usingsecurity.jsonwithSolr]]
|
||||
== Using security.json with Solr
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-InSolrCloudmode]]
|
||||
=== In SolrCloud Mode
|
||||
|
||||
While configuring Solr to use an authentication or authorization plugin, you will need to upload a `security.json` file to ZooKeeper. The following command writes the file as it uploads it - you could also upload a file that you have already created locally.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
>server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd put /security.json
|
||||
'{"authentication": {"class": "org.apache.solr.security.KerberosPlugin"}}'
|
||||
----
|
||||
|
||||
Note that this example defines the `KerberosPlugin` for authentication. You will want to modify this section as appropriate for the plugin you are using.
|
||||
|
||||
This example also defines `security.json` on the command line, but you can also define a file locally and upload it to ZooKeeper.
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
Depending on the authentication and authorization plugin that you use, you may have user information stored in `security.json`. If so, we highly recommend that you implement access control in your ZooKeeper nodes. Information about how to enable this is available in the section <<zookeeper-access-control.adoc#zookeeper-access-control,ZooKeeper Access Control>>.
|
||||
====
|
||||
|
||||
Once `security.json` has been uploaded to ZooKeeper, you should use the appropriate APIs for the plugins you're using to update it. You can edit it manually, but you must take care to remove any version data so it will be properly updated across all ZooKeeper nodes. The version data is found at the end of the `security.json` file, and will appear as the letter "v" followed by a number, such as `{"v":138}`.
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-InStandaloneMode]]
|
||||
=== In Standalone Mode
|
||||
|
||||
When running Solr in standalone mode, you need to create the `security.json` file and put it in the `$SOLR_HOME` directory for your installation (this is the same place you have located `solr.xml` and is usually `server/solr`).
|
||||
|
||||
If you are using <<legacy-scaling-and-distribution.adoc#legacy-scaling-and-distribution,Legacy Scaling and Distribution>>, you will need to place `security.json` on each node of the cluster.
|
||||
|
||||
You can use the authentication and authorization APIs, but if you are using the legacy scaling model, you will need to make the same API requests on each node separately. You can also edit `security.json` by hand if you prefer.
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-Authentication]]
|
||||
== Authentication
|
||||
|
||||
Authentication plugins help in securing the endpoints of Solr by authenticating incoming requests. A custom plugin can be implemented by extending the AuthenticationPlugin class.
|
||||
|
||||
An authentication plugin consists of two parts:
|
||||
|
||||
. Server-side component, which intercepts and authenticates incoming requests to Solr using a mechanism defined in the plugin, such as Kerberos, Basic Auth or others.
|
||||
. Client-side component, i.e., an extension of `HttpClientConfigurer`, which enables a SolrJ client to make requests to a secure Solr instance using the authentication mechanism which the server understands.
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-EnablingaPlugin]]
|
||||
=== Enabling a Plugin
|
||||
|
||||
* Specify the authentication plugin in `/security.json` as in this example:
|
||||
+
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"authentication": {
|
||||
"class": "class.that.implements.authentication",
|
||||
"other_data" : "..."}
|
||||
}
|
||||
----
|
||||
* All of the content in the authentication block of `security.json` would be passed on as a map to the plugin during initialization.
|
||||
* An authentication plugin can also be used with a standalone Solr instance by passing in `-DauthenticationPlugin=<plugin class name>` during startup.
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-AvailableAuthenticationPlugins]]
|
||||
=== Available Authentication Plugins
|
||||
|
||||
Solr has the following implementations of authentication plugins:
|
||||
|
||||
* <<kerberos-authentication-plugin.adoc#kerberos-authentication-plugin,Kerberos Authentication Plugin>>
|
||||
* <<basic-authentication-plugin.adoc#basic-authentication-plugin,Basic Authentication Plugin>>
|
||||
* <<hadoop-authentication-plugin.adoc#hadoop-authentication-plugin,Hadoop Authentication Plugin>>
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-Authorization]]
|
||||
== Authorization
|
||||
|
||||
An authorization plugin can be written for Solr by extending the {solr-javadocs}/solr-core/org/apache/solr/security/AuthorizationPlugin.html[AuthorizationPlugin] interface.
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-LoadingaCustomPlugin]]
|
||||
=== Loading a Custom Plugin
|
||||
|
||||
* Make sure that the plugin implementation is in the classpath.
|
||||
* The plugin can then be initialized by specifying the same in `security.json` in the following manner:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"authorization": {
|
||||
"class": "org.apache.solr.security.MockAuthorizationPlugin",
|
||||
"other_data" : "..."}
|
||||
}
|
||||
----
|
||||
|
||||
All of the content in the `authorization` block of `security.json` would be passed on as a map to the plugin during initialization.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
The authorization plugin is only supported in SolrCloud mode. Also, reloading the plugin isn't yet supported and requires a restart of the Solr installation (meaning, the JVM should be restarted, not simply a core reload).
|
||||
====
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-AvailableAuthorizationPlugins]]
|
||||
=== Available Authorization Plugins
|
||||
|
||||
Solr has one implementation of an authorization plugin:
|
||||
|
||||
* <<rule-based-authorization-plugin.adoc#rule-based-authorization-plugin,Rule-Based Authorization Plugin>>
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-PKISecuringInter-NodeRequests]]
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-PKI]]
|
||||
== Securing Inter-Node Requests
|
||||
|
||||
There are a lot of requests that originate from the Solr nodes itself. For example, requests from overseer to nodes, recovery threads, etc. Each Authentication plugin declares whether it is capable of securing inter-node requests or not. If not, Solr will fall back to using a special internode authentication mechanism where each Solr node is a super user and is fully trusted by other Solr nodes, described below.
|
||||
|
||||
[[AuthenticationandAuthorizationPlugins-PKIAuthenticationPlugin]]
|
||||
=== PKIAuthenticationPlugin
|
||||
|
||||
The PKIAuthenticationPlugin is used when there is any request going on between two Solr nodes, and the configured Authentication plugin does not wish to handle inter-node security.
|
||||
|
||||
For each outgoing request `PKIAuthenticationPlugin` adds a special header `'SolrAuth'` which carries the timestamp and principal encrypted using the private key of that node. The public key is exposed through an API so that any node can read it whenever it needs it. Any node who gets the request with that header, would get the public key from the sender and decrypt the information. If it is able to decrypt the data, the request trusted. It is invalid if the timestamp is more than 5 secs old. This assumes that the clocks of different nodes in the cluster are synchronized.
|
||||
|
||||
The timeout is configurable through a system property called `pkiauth.ttl`. For example, if you wish to bump up the time-to-live to 10 seconds (10000 milliseconds), start each node with a property `'-Dpkiauth.ttl=10000'`.
|
|
@ -0,0 +1,140 @@
|
|||
= Basic Authentication Plugin
|
||||
:page-shortname: basic-authentication-plugin
|
||||
:page-permalink: basic-authentication-plugin.html
|
||||
|
||||
Solr can support Basic authentication for users with the use of the BasicAuthPlugin.
|
||||
|
||||
An authorization plugin is also available to configure Solr with permissions to perform various activities in the system. The authorization plugin is described in the section <<rule-based-authorization-plugin.adoc#rule-based-authorization-plugin,Rule-Based Authorization Plugin>>.
|
||||
|
||||
[[BasicAuthenticationPlugin-EnableBasicAuthentication]]
|
||||
== Enable Basic Authentication
|
||||
|
||||
To use Basic authentication, you must first create a `security.json` file. This file and where to put it is described in detail in the section <<authentication-and-authorization-plugins.adoc#AuthenticationandAuthorizationPlugins-EnablePluginswithsecurity.json,Enable Plugins with security.json>>.
|
||||
|
||||
For Basic authentication, the `security.json` file must have an `authentication` part which defines the class being used for authentication. Usernames and passwords (as a sha256(password+salt) hash) could be added when the file is created, or can be added later with the Basic authentication API, described below.
|
||||
|
||||
The `authorization` part is not related to Basic authentication, but is a separate authorization plugin designed to support fine-grained user access control. For more information, see the section <<rule-based-authorization-plugin.adoc#rule-based-authorization-plugin,Rule-Based Authorization Plugin>>.
|
||||
|
||||
An example `security.json` showing both sections is shown below to show how these plugins can work together:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"authentication":{
|
||||
"blockUnknown": true,
|
||||
"class":"solr.BasicAuthPlugin",
|
||||
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
|
||||
},
|
||||
"authorization":{
|
||||
"class":"solr.RuleBasedAuthorizationPlugin",
|
||||
"permissions":[{"name":"security-edit",
|
||||
"role":"admin"}],
|
||||
"user-role":{"solr":"admin"}
|
||||
}}
|
||||
----
|
||||
|
||||
There are several things defined in this file:
|
||||
|
||||
* Basic authentication and rule-based authorization plugins are enabled.
|
||||
* A user called 'solr', with a password `'SolrRocks'` has been defined.
|
||||
* The parameter `"blockUnknown": true` means that unauthenticated requests are not allowed to pass through.
|
||||
* The 'admin' role has been defined, and it has permission to edit security settings.
|
||||
* The 'solr' user has been defined to the 'admin' role.
|
||||
|
||||
Save your settings to a file called `security.json` locally. If you are using Solr in standalone mode, you should put this file in `$SOLR_HOME`.
|
||||
|
||||
If `blockUnknown` does not appear in the `security.json` file, it will default to `false`. This has the effect of not requiring authentication at all. In some cases, you may want this; for example, if you want to have `security.json` in place but aren't ready to enable authentication. However, you will want to ensure that this parameter is set to `true` in order for authentication to be truly enabled in your system.
|
||||
|
||||
If you are using SolrCloud, you must upload `security.json` to ZooKeeper. You can use this example command, ensuring that the ZooKeeper port is correct:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
bin/solr zk cp file:path_to_local_security.json zk:/security.json -z localhost:9983
|
||||
----
|
||||
|
||||
[[BasicAuthenticationPlugin-Caveats]]
|
||||
=== Caveats
|
||||
|
||||
There are a few things to keep in mind when using the Basic authentication plugin.
|
||||
|
||||
* Credentials are sent in plain text by default. It's recommended to use SSL for communication when Basic authentication is enabled, as described in the section <<enabling-ssl.adoc#enabling-ssl,Enabling SSL>>.
|
||||
* A user who has access to write permissions to `security.json` will be able to modify all the permissions and how users have been assigned permissions. Special care should be taken to only grant access to editing security to appropriate users.
|
||||
* Your network should, of course, be secure. Even with Basic authentication enabled, you should not unnecessarily expose Solr to the outside world.
|
||||
|
||||
[[BasicAuthenticationPlugin-EditingAuthenticationPluginConfiguration]]
|
||||
== Editing Authentication Plugin Configuration
|
||||
|
||||
An Authentication API allows modifying user IDs and passwords. The API provides an endpoint with specific commands to set user details or delete a user.
|
||||
|
||||
[[BasicAuthenticationPlugin-APIEntryPoint]]
|
||||
=== API Entry Point
|
||||
|
||||
`admin/authentication`
|
||||
|
||||
This endpoint is not collection-specific, so users are created for the entire Solr cluster. If users need to be restricted to a specific collection, that can be done with the authorization rules.
|
||||
|
||||
[[BasicAuthenticationPlugin-AddaUserorEditaPassword]]
|
||||
=== Add a User or Edit a Password
|
||||
|
||||
The `set-user` command allows you to add users and change their passwords. For example, the following defines two users and their passwords:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{
|
||||
"set-user": {"tom" : "TomIsCool" ,
|
||||
"harry":"HarrysSecret"}}'
|
||||
----
|
||||
|
||||
[[BasicAuthenticationPlugin-DeleteaUser]]
|
||||
=== Delete a User
|
||||
|
||||
The `delete-user` command allows you to remove a user. The user password does not need to be sent to remove a user. In the following example, we've asked that user IDs 'tom' and 'harry' be removed from the system.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{
|
||||
"delete-user": ["tom","harry"]}'
|
||||
----
|
||||
|
||||
[[BasicAuthenticationPlugin-Setaproperty]]
|
||||
=== Set a property
|
||||
|
||||
Set arbitrary properties for authentication plugin. The only supported property is `'blockUnknown'`
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{
|
||||
"set-property": {"blockUnknown":false}}'
|
||||
----
|
||||
|
||||
[[BasicAuthenticationPlugin-UsingBasicAuthwithSolrJ]]
|
||||
=== Using BasicAuth with SolrJ
|
||||
|
||||
In SolrJ, the basic authentication credentials need to be set for each request as in this example:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
SolrRequest req ;//create a new request object
|
||||
req.setBasicAuthCredentials(userName, password);
|
||||
solrClient.request(req);
|
||||
----
|
||||
|
||||
Query example:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
QueryRequest req = new QueryRequest(new SolrQuery("*:*"));
|
||||
req.setBasicAuthCredentials(userName, password);
|
||||
QueryResponse rsp = req.process(solrClient);
|
||||
----
|
||||
|
||||
[[BasicAuthenticationPlugin-UsingCommandLinescriptswithBasicAuth]]
|
||||
=== Using Command Line scripts with BasicAuth
|
||||
|
||||
Add the following line to the `solr.in.sh` or `solr.in.cmd` file. This example tells the `bin/solr` command line to to use "basic" as the type of authentication, and to pass credentials with the user-name "solr" and password "SolrRocks":
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
SOLR_AUTH_TYPE="basic"
|
||||
SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
|
||||
----
|
|
@ -0,0 +1,135 @@
|
|||
= Blob Store API
|
||||
:page-shortname: blob-store-api
|
||||
:page-permalink: blob-store-api.html
|
||||
|
||||
The Blob Store REST API provides REST methods to store, retrieve or list files in a Lucene index.
|
||||
|
||||
It can be used to upload a jar file which contains standard solr components such as RequestHandlers, SearchComponents, or other custom code you have written for Solr. Schema components _do not_ yet support the Blob Store.
|
||||
|
||||
When using the blob store, note that the API does not delete or overwrite a previous object if a new one is uploaded with the same name. It always adds a new version of the blob to the index. Deletes can be performed with standard REST delete commands.
|
||||
|
||||
*The blob store is only available when running in SolrCloud mode.* Solr in standalone mode does not support use of a blob store.
|
||||
|
||||
The blob store API is implemented as a requestHandler. A special collection named ".system" is used to store the blobs. This collection can be created in advance, but if it does not exist it will be created automatically.
|
||||
|
||||
[[BlobStoreAPI-Aboutthe.systemCollection]]
|
||||
== About the .system Collection
|
||||
|
||||
Before uploading blobs to the blob store, a special collection must be created and it must be named `.system`. Solr will automatically create this collection if it does not already exist, but you can also create it manually if you choose.
|
||||
|
||||
The BlobHandler is automatically registered in the .system collection. The `solrconfig.xml`, Schema, and other configuration files for the collection are automatically provided by the system and don't need to be defined specifically.
|
||||
|
||||
If you do not use the `-shards` or `-replicationFactor` options, then defaults of numShards=1 and replicationFactor=3 (or maximum nodes in the cluster) will be used.
|
||||
|
||||
You can create the `.system` collection with the <<collections-api.adoc#collections-api,Collections API>>, as in this example:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/admin/collections?action=CREATE&name=.system&replicationFactor=2
|
||||
----
|
||||
|
||||
IMPORTANT: The `bin/solr` script cannot be used to create the `.system` collection.
|
||||
|
||||
[[BlobStoreAPI-UploadFilestoBlobStore]]
|
||||
== Upload Files to Blob Store
|
||||
|
||||
After the `.system` collection has been created, files can be uploaded to the blob store with a request similar to the following:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @{filename} http://localhost:8983/solr/.system/blob/{blobname}
|
||||
----
|
||||
|
||||
For example, to upload a file named "test1.jar" as a blob named "test", you would make a POST request like:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @test1.jar http://localhost:8983/solr/.system/blob/test
|
||||
----
|
||||
|
||||
A GET request will return the list of blobs and other details:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/.system/blob?omitHeader=true
|
||||
----
|
||||
|
||||
Output:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"response":{"numFound":1,"start":0,"docs":[
|
||||
{
|
||||
"id":"test/1",
|
||||
"md5":"20ff915fa3f5a5d66216081ae705c41b",
|
||||
"blobName":"test",
|
||||
"version":1,
|
||||
"timestamp":"2015-02-04T16:45:48.374Z",
|
||||
"size":13108}]
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
Details on individual blobs can be accessed with a request similar to:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/.system/blob/{blobname}
|
||||
----
|
||||
|
||||
For example, this request will return only the blob named 'test':
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/.system/blob/test?omitHeader=true
|
||||
----
|
||||
|
||||
Output:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"response":{"numFound":1,"start":0,"docs":[
|
||||
{
|
||||
"id":"test/1",
|
||||
"md5":"20ff915fa3f5a5d66216081ae705c41b",
|
||||
"blobName":"test",
|
||||
"version":1,
|
||||
"timestamp":"2015-02-04T16:45:48.374Z",
|
||||
"size":13108}]
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
The filestream response writer can return a particular version of a blob for download, as in:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/.system/blob/{blobname}/{version}?wt=filestream > {outputfilename}
|
||||
----
|
||||
|
||||
For the latest version of a blob, the \{version} can be omitted,
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/.system/blob/{blobname}?wt=filestream > {outputfilename}
|
||||
----
|
||||
|
||||
[[BlobStoreAPI-UseaBlobinaHandlerorComponent]]
|
||||
== Use a Blob in a Handler or Component
|
||||
|
||||
To use the blob as the class for a request handler or search component, you create a request handler in `solrconfig.xml` as usual. You will need to define the following parameters:
|
||||
|
||||
`class`:: the fully qualified class name. For example, if you created a new request handler class called CRUDHandler, you would enter `org.apache.solr.core.CRUDHandler`.
|
||||
`runtimeLib`:: Set to true to require that this component should be loaded from the classloader that loads the runtime jars.
|
||||
|
||||
For example, to use a blob named test, you would configure `solrconfig.xml` like this:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<requestHandler name="/myhandler" class="org.apache.solr.core.myHandler" runtimeLib="true" version="1">
|
||||
</requestHandler>
|
||||
----
|
||||
|
||||
If there are parameters available in the custom handler, you can define them in the same way as any other request handler definition.
|
|
@ -0,0 +1,99 @@
|
|||
= BlockJoin Faceting
|
||||
:page-shortname: blockjoin-faceting
|
||||
:page-permalink: blockjoin-faceting.html
|
||||
|
||||
BlockJoin facets allow you to aggregate children facet counts by their parents.
|
||||
|
||||
It is a common requirement that if a parent document has several children documents, all of them need to increment facet value count only once. This functionality is provided by `BlockJoinDocSetFacetComponent`, and `BlockJoinFacetComponent` just an alias for compatibility.
|
||||
|
||||
CAUTION: This component is considered experimental, and must be explicitly enabled for a request handler in `solrconfig.xml`, in the same way as any other <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,search component>>.
|
||||
|
||||
This example shows how you could add this search components to `solrconfig.xml` and define it in request handler:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<searchComponent name="bjqFacetComponent" class="org.apache.solr.search.join.BlockJoinDocSetFacetComponent"/>
|
||||
|
||||
<requestHandler name="/bjqfacet" class="org.apache.solr.handler.component.SearchHandler">
|
||||
<lst name="defaults">
|
||||
<str name="shards.qt">/bjqfacet</str>
|
||||
</lst>
|
||||
<arr name="last-components">
|
||||
<str>bjqFacetComponent</str>
|
||||
</arr>
|
||||
</requestHandler>
|
||||
----
|
||||
|
||||
This component can be added into any search request handler. This component work with distributed search in SolrCloud mode.
|
||||
|
||||
Documents should be added in children-parent blocks as described in <<uploading-data-with-index-handlers.adoc#UploadingDatawithIndexHandlers-NestedChildDocuments,indexing nested child documents>>. Examples:
|
||||
|
||||
.Sample document
|
||||
[source,xml]
|
||||
----
|
||||
<add>
|
||||
<doc>
|
||||
<field name="id">1</field>
|
||||
<field name="type_s">parent</field>
|
||||
<doc>
|
||||
<field name="id">11</field>
|
||||
<field name="COLOR_s">Red</field>
|
||||
<field name="SIZE_s">XL</field>
|
||||
<field name="PRICE_i">6</field>
|
||||
</doc>
|
||||
<doc>
|
||||
<field name="id">12</field>
|
||||
<field name="COLOR_s">Red</field>
|
||||
<field name="SIZE_s">XL</field>
|
||||
<field name="PRICE_i">7</field>
|
||||
</doc>
|
||||
<doc>
|
||||
<field name="id">13</field>
|
||||
<field name="COLOR_s">Blue</field>
|
||||
<field name="SIZE_s">L</field>
|
||||
<field name="PRICE_i">5</field>
|
||||
</doc>
|
||||
</doc>
|
||||
<doc>
|
||||
<field name="id">2</field>
|
||||
<field name="type_s">parent</field>
|
||||
<doc>
|
||||
<field name="id">21</field>
|
||||
<field name="COLOR_s">Blue</field>
|
||||
<field name="SIZE_s">XL</field>
|
||||
<field name="PRICE_i">6</field>
|
||||
</doc>
|
||||
<doc>
|
||||
<field name="id">22</field>
|
||||
<field name="COLOR_s">Blue</field>
|
||||
<field name="SIZE_s">XL</field>
|
||||
<field name="PRICE_i">7</field>
|
||||
</doc>
|
||||
<doc>
|
||||
<field name="id">23</field>
|
||||
<field name="COLOR_s">Red</field>
|
||||
<field name="SIZE_s">L</field>
|
||||
<field name="PRICE_i">5</field>
|
||||
</doc>
|
||||
</doc>
|
||||
</add>
|
||||
----
|
||||
|
||||
Queries are constructed the same way as for a <<other-parsers.adoc#OtherParsers-BlockJoinQueryParsers,Parent Block Join query>>. For example:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/bjqfacet?q={!parent which=type_s:parent}SIZE_s:XL&child.facet.field=COLOR_s
|
||||
----
|
||||
|
||||
As a result we should have facets for Red(1) and Blue(1), because matches on children `id=11` and `id=12` are aggregated into single hit into parent with `id=1`. The key components of the request are:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="30,70",options="header"]
|
||||
|===
|
||||
|URL Part | Meaning
|
||||
|`/bjqfacet` |The name of the request handler that has been defined with one of block join facet components enabled.
|
||||
|`q={!parent ...}..` |The mandatory parent query as a main query. The parent query could also be a subordinate clause in a more complex query.
|
||||
|`child.facet.field=...` |The child document field, which might be repeated many times with several fields, as necessary.
|
||||
|===
|
|
@ -0,0 +1,159 @@
|
|||
= CharFilterFactories
|
||||
:page-shortname: charfilterfactories
|
||||
:page-permalink: charfilterfactories.html
|
||||
|
||||
CharFilter is a component that pre-processes input characters.
|
||||
|
||||
CharFilters can be chained like Token Filters and placed in front of a Tokenizer. CharFilters can add, change, or remove characters while preserving the original character offsets to support features like highlighting.
|
||||
|
||||
[[CharFilterFactories-solr.MappingCharFilterFactory]]
|
||||
== solr.MappingCharFilterFactory
|
||||
|
||||
This filter creates `org.apache.lucene.analysis.MappingCharFilter`, which can be used for changing one string to another (for example, for normalizing `é` to `e`.).
|
||||
|
||||
This filter requires specifying a `mapping` argument, which is the path and name of a file containing the mappings to perform.
|
||||
|
||||
Example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<analyzer>
|
||||
<charFilter class="solr.MappingCharFilterFactory" mapping="mapping-FoldToASCII.txt"/>
|
||||
<tokenizer ...>
|
||||
[...]
|
||||
</analyzer>
|
||||
----
|
||||
|
||||
Mapping file syntax:
|
||||
|
||||
* Comment lines beginning with a hash mark (`#`), as well as blank lines, are ignored.
|
||||
* Each non-comment, non-blank line consists of a mapping of the form: `"source" => "target"`
|
||||
** Double-quoted source string, optional whitespace, an arrow (`=>`), optional whitespace, double-quoted target string.
|
||||
* Trailing comments on mapping lines are not allowed.
|
||||
* The source string must contain at least one character, but the target string may be empty.
|
||||
* The following character escape sequences are recognized within source and target strings:
|
||||
+
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
+
|
||||
[cols="20,30,20,30",options="header"]
|
||||
|===
|
||||
|Escape Sequence |Resulting Character (http://www.ecma-international.org/publications/standards/Ecma-048.htm[ECMA-48] alias) |Unicode Character |Example Mapping Line
|
||||
|`\\` |`\` |U+005C |`"\\" => "/"`
|
||||
|`\"` |`"` |U+0022 |`"\"and\"" => "'and'"`
|
||||
|`\b` |backspace (BS) |U+0008 |`"\b" => " "`
|
||||
|`\t` |tab (HT) |U+0009 |`"\t" => ","`
|
||||
|`\n` |newline (LF) |U+000A |`"\n" => "<br>"`
|
||||
|`\f` |form feed (FF) |U+000C |`"\f" => "\n"`
|
||||
|`\r` |carriage return (CR) |U+000D |`"\r" => "/carriage-return/"`
|
||||
|`\uXXXX` |Unicode char referenced by the 4 hex digits |U+XXXX |`"\uFEFF" => ""`
|
||||
|===
|
||||
** A backslash followed by any other character is interpreted as if the character were present without the backslash.
|
||||
|
||||
[[CharFilterFactories-solr.HTMLStripCharFilterFactory]]
|
||||
== solr.HTMLStripCharFilterFactory
|
||||
|
||||
This filter creates `org.apache.solr.analysis.HTMLStripCharFilter`. This CharFilter strips HTML from the input stream and passes the result to another CharFilter or a Tokenizer.
|
||||
|
||||
This filter:
|
||||
|
||||
* Removes HTML/XML tags while preserving other content.
|
||||
* Removes attributes within tags and supports optional attribute quoting.
|
||||
* Removes XML processing instructions, such as: <?foo bar?>
|
||||
* Removes XML comments.
|
||||
* Removes XML elements starting with <!>.
|
||||
* Removes contents of <script> and <style> elements.
|
||||
* Handles XML comments inside these elements (normal comment processing will not always work).
|
||||
* Replaces numeric character entities references like `A`; or ``; with the corresponding character.
|
||||
* The terminating ';' is optional if the entity reference at the end of the input; otherwise the terminating ';' is mandatory, to avoid false matches on something like "Alpha&Omega Corp".
|
||||
* Replaces all named character entity references with the corresponding character.
|
||||
* ` `; is replaced with a space instead of the 0xa0 character.
|
||||
* Newlines are substituted for block-level elements.
|
||||
* <CDATA> sections are recognized.
|
||||
* Inline tags, such as `<b>`, `<i>`, or `<span>` will be removed.
|
||||
* Uppercase character entities like `quot`, `gt`, `lt` and `amp` are recognized and handled as lowercase.
|
||||
|
||||
TIP: The input need not be an HTML document. The filter removes only constructs that look like HTML. If the input doesn't include anything that looks like HTML, the filter won't remove any input.
|
||||
|
||||
The table below presents examples of HTML stripping.
|
||||
|
||||
[width="100%",options="header",]
|
||||
|===
|
||||
|Input |Output
|
||||
|`my <a href="www.foo.bar">link</a>` |my link
|
||||
|`<br>hello<!--comment-->` |hello
|
||||
|`hello<script><!-- f('<!--internal--></script>'); --></script>` |hello
|
||||
|`if a<b then print a;` |if a<b then print a;
|
||||
|`hello <td height=22 nowrap align="left">` |hello
|
||||
|`a<b A Alpha&Omega Ω` |a<b A Alpha&Omega Ω
|
||||
|===
|
||||
|
||||
Example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<analyzer>
|
||||
<charFilter class="solr.HTMLStripCharFilterFactory"/>
|
||||
<tokenizer ...>
|
||||
[...]
|
||||
</analyzer>
|
||||
----
|
||||
|
||||
[[CharFilterFactories-solr.ICUNormalizer2CharFilterFactory]]
|
||||
== solr.ICUNormalizer2CharFilterFactory
|
||||
|
||||
This filter performs pre-tokenization Unicode normalization using http://site.icu-project.org[ICU4J].
|
||||
|
||||
Arguments:
|
||||
|
||||
`name`:: A http://unicode.org/reports/tr15/[Unicode Normalization Form], one of `nfc`, `nfkc`, `nfkc_cf`. Default is `nfkc_cf`.
|
||||
|
||||
`mode`:: Either `compose` or `decompose`. Default is `compose`. Use `decompose` with `name="nfc"` or `name="nfkc"` to get NFD or NFKD, respectively.
|
||||
|
||||
`filter`:: A http://www.icu-project.org/apiref/icu4j/com/ibm/icu/text/UnicodeSet.html[UnicodeSet] pattern. Codepoints outside the set are always left unchanged. Default is `[]` (the null set, no filtering - all codepoints are subject to normalization).
|
||||
|
||||
Example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<analyzer>
|
||||
<charFilter class="solr.ICUNormalizer2CharFilterFactory"/>
|
||||
<tokenizer ...>
|
||||
[...]
|
||||
</analyzer>
|
||||
----
|
||||
|
||||
[[CharFilterFactories-solr.PatternReplaceCharFilterFactory]]
|
||||
== solr.PatternReplaceCharFilterFactory
|
||||
|
||||
This filter uses http://www.regular-expressions.info/reference.html[regular expressions] to replace or change character patterns.
|
||||
|
||||
Arguments:
|
||||
|
||||
`pattern`:: the regular expression pattern to apply to the incoming text.
|
||||
|
||||
`replacement`:: the text to use to replace matching patterns.
|
||||
|
||||
You can configure this filter in `schema.xml` like this:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<analyzer>
|
||||
<charFilter class="solr.PatternReplaceCharFilterFactory"
|
||||
pattern="([nN][oO]\.)\s*(\d+)" replacement="$1$2"/>
|
||||
<tokenizer ...>
|
||||
[...]
|
||||
</analyzer>
|
||||
----
|
||||
|
||||
The table below presents examples of regex-based pattern replacement:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,20,10,20,30",options="header"]
|
||||
|===
|
||||
|Input |Pattern |Replacement |Output |Description
|
||||
|see-ing looking |`(\w+)(ing)` |`$1` |see-ing look |Removes "ing" from the end of word.
|
||||
|see-ing looking |`(\w+)ing` |`$1` |see-ing look |Same as above. 2nd parentheses can be omitted.
|
||||
|No.1 NO. no. 543 |`[nN][oO]\.\s*(\d+)` |`#$1` |#1 NO. #543 |Replace some string literals
|
||||
|abc=1234=5678 |`(\w+)=(\d+)=(\d+)` |`$3=$1=$2` |5678=abc=1234 |Change the order of the groups.
|
||||
|===
|
|
@ -0,0 +1,9 @@
|
|||
= Choosing an Output Format
|
||||
:page-shortname: choosing-an-output-format
|
||||
:page-permalink: choosing-an-output-format.html
|
||||
|
||||
Many programming environments are able to send HTTP requests and retrieve responses. Parsing the responses is a slightly more thorny problem. Fortunately, Solr makes it easy to choose an output format that will be easy to handle on the client side.
|
||||
|
||||
Specify a response format using the `wt` parameter in a query. The available response formats are documented in <<response-writers.adoc#response-writers,Response Writers>>.
|
||||
|
||||
Most client APIs hide this detail for you, so for many types of client applications, you won't ever have to specify a `wt` parameter. In JavaScript, however, the interface to Solr is a little closer to the metal, so you will need to add this parameter yourself.
|
|
@ -0,0 +1,29 @@
|
|||
= Client API Lineup
|
||||
:page-shortname: client-api-lineup
|
||||
:page-permalink: client-api-lineup.html
|
||||
|
||||
The Solr Wiki contains a list of client APIs at http://wiki.apache.org/solr/IntegratingSolr.
|
||||
|
||||
Here is the list of client APIs, current at this writing (November 2011):
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,20,60",options="header"]
|
||||
|===
|
||||
|Name |Environment |URL
|
||||
|SolRuby |Ruby |https://github.com/rsolr/rsolr
|
||||
|DelSolr |Ruby |https://github.com/avvo/delsolr
|
||||
|acts_as_solr |Rails |http://acts-as-solr.rubyforge.org/, http://rubyforge.org/projects/background-solr/
|
||||
|Flare |Rails |http://wiki.apache.org/solr/Flare
|
||||
|SolPHP |PHP |http://wiki.apache.org/solr/SolPHP
|
||||
|SolrJ |Java |http://wiki.apache.org/solr/SolJava
|
||||
|Python API |Python |http://wiki.apache.org/solr/SolPython
|
||||
|PySolr |Python |http://code.google.com/p/pysolr/
|
||||
|SolPerl |Perl |http://wiki.apache.org/solr/SolPerl
|
||||
|Solr.pm |Perl |http://search.cpan.org/~garafola/Solr-0.03/lib/Solr.pm
|
||||
|SolrForrest |Forrest/Cocoon |http://wiki.apache.org/solr/SolrForrest
|
||||
|SolrSharp |C# |http://www.codeplex.com/solrsharp
|
||||
|SolColdfusion |ColdFusion |http://solcoldfusion.riaforge.org/
|
||||
|SolrNet |.NET |https://github.com/mausch/SolrNet
|
||||
|AJAX Solr |AJAX |http://github.com/evolvingweb/ajax-solr/wiki
|
||||
|===
|
|
@ -0,0 +1,22 @@
|
|||
= Client APIs
|
||||
:page-shortname: client-apis
|
||||
:page-permalink: client-apis.html
|
||||
:page-children: introduction-to-client-apis, choosing-an-output-format, client-api-lineup, using-javascript, using-python, using-solrj, using-solr-from-ruby
|
||||
|
||||
This section discusses the available client APIs for Solr. It covers the following topics:
|
||||
|
||||
<<introduction-to-client-apis.adoc#introduction-to-client-apis,Introduction to Client APIs>>: A conceptual overview of Solr client APIs.
|
||||
|
||||
<<choosing-an-output-format.adoc#choosing-an-output-format,Choosing an Output Format>>: Information about choosing a response format in Solr.
|
||||
|
||||
<<using-javascript.adoc#using-javascript,Using JavaScript>>: Explains why a client API is not needed for JavaScript responses.
|
||||
|
||||
<<using-python.adoc#using-python,Using Python>>: Information about Python and JSON responses.
|
||||
|
||||
<<client-api-lineup.adoc#client-api-lineup,Client API Lineup>>: A list of all Solr Client APIs, with links.
|
||||
|
||||
<<using-solrj.adoc#using-solrj,Using SolrJ>>: Detailed information about SolrJ, an API for working with Java applications.
|
||||
|
||||
<<using-solr-from-ruby.adoc#using-solr-from-ruby,Using Solr From Ruby>>: Detailed information about using Solr with Ruby applications.
|
||||
|
||||
<<mbean-request-handler.adoc#mbean-request-handler,MBean Request Handler>>: Describes the MBean request handler for programmatic access to Solr server statistics and information.
|
|
@ -0,0 +1,29 @@
|
|||
= Cloud Screens
|
||||
:page-shortname: cloud-screens
|
||||
:page-permalink: cloud-screens.html
|
||||
|
||||
When running in <<solrcloud.adoc#solrcloud,SolrCloud>> mode, a "Cloud" option will appear in the Admin UI between <<logging.adoc#logging,Logging>> and <<collections-core-admin.adoc#collections-core-admin,Collections/Core Admin>>.
|
||||
|
||||
This screen provides status information about each collection & node in your cluster, as well as access to the low level data being stored in <<using-zookeeper-to-manage-configuration-files.adoc#using-zookeeper-to-manage-configuration-files,Zookeeper>>.
|
||||
|
||||
.Only Visible When using SolrCloud
|
||||
[NOTE]
|
||||
====
|
||||
The "Cloud" menu option is only available on Solr instances running in <<getting-started-with-solrcloud.adoc#getting-started-with-solrcloud,SolrCloud mode>>. Single node or master/slave replication instances of Solr will not display this option.
|
||||
====
|
||||
|
||||
Click on the Cloud option in the left-hand navigation, and a small sub-menu appears with options called "Tree", "Graph", "Graph (Radial)" and "Dump". The default view ("Graph") shows a graph of each collection, the shards that make up those collections, and the addresses of each replica for each shard.
|
||||
|
||||
This example shows the very simple two-node cluster created using the `bin/solr -e cloud -noprompt` example command. In addition to the 2 shard, 2 replica "gettingstarted" collection, there is an additional "films" collection consisting of a single shard/replica:
|
||||
|
||||
image::images/cloud-screens/cloud-graph.png[image,width=512,height=250]
|
||||
|
||||
The "Graph (Radial)" option provides a different visual view of each node. Using the same example cluster, the radial graph view looks like:
|
||||
|
||||
image::images/cloud-screens/cloud-radial.png[image,width=478,height=250]
|
||||
|
||||
The "Tree" option shows a directory structure of the data in ZooKeeper, including cluster wide information regarding the `live_nodes` and `overseer` status, as well as collection specific information such as the `state.json`, current shard leaders, and configuration files in use. In this example, we see the `state.json` file definition for the "films" collection:
|
||||
|
||||
image::images/cloud-screens/cloud-tree.png[image,width=487,height=250]
|
||||
|
||||
The final option is "Dump", which returns a JSON document containing all nodes, their contents and their children (recursively). This can be used to export a snapshot of all the data that Solr has kept inside ZooKeeper and can aid in debugging SolrCloud problems.
|
|
@ -0,0 +1,21 @@
|
|||
= Codec Factory
|
||||
:page-shortname: codec-factory
|
||||
:page-permalink: codec-factory.html
|
||||
|
||||
A `codecFactory` can be specified in `solrconfig.xml` to determine which Lucene {lucene-javadocs}/core/org/apache/lucene/codecs/Codec.html[`Codec`] is used when writing the index to disk.
|
||||
|
||||
If not specified, Lucene's default codec is implicitly used, but a {solr-javadocs}/solr-core/org/apache/solr/core/SchemaCodecFactory.html[`solr.SchemaCodecFactory`] is also available which supports 2 key features:
|
||||
|
||||
* Schema based per-fieldtype configuration for `docValuesFormat` and `postingsFormat` - see the <<field-type-definitions-and-properties.adoc#field-type-properties,Field Type Properties>> section for more details.
|
||||
* A `compressionMode` option:
|
||||
** `BEST_SPEED` (default) is optimized for search speed performance
|
||||
** `BEST_COMPRESSION` is optimized for disk space usage
|
||||
|
||||
Example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<codecFactory class="solr.SchemaCodecFactory">
|
||||
<str name="compressionMode">BEST_COMPRESSION</str>
|
||||
</codecFactory>
|
||||
----
|
|
@ -0,0 +1,133 @@
|
|||
= Collapse and Expand Results
|
||||
:page-shortname: collapse-and-expand-results
|
||||
:page-permalink: collapse-and-expand-results.html
|
||||
|
||||
The Collapsing query parser and the Expand component combine to form an approach to grouping documents for field collapsing in search results.
|
||||
|
||||
The Collapsing query parser groups documents (collapsing the result set) according to your parameters, while the Expand component provides access to documents in the collapsed group for use in results display or other processing by a client application. Collapse & Expand can together do what the older <<result-grouping.adoc#result-grouping,Result Grouping>> (`group=true`) does for _most_ use-cases but not all. Generally, you should prefer Collapse & Expand.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
In order to use these features with SolrCloud, the documents must be located on the same shard. To ensure document co-location, you can define the `router.name` parameter as `compositeId` when creating the collection. For more information on this option, see the section <<shards-and-indexing-data-in-solrcloud.adoc#ShardsandIndexingDatainSolrCloud-DocumentRouting,Document Routing>>.
|
||||
====
|
||||
|
||||
[[CollapseandExpandResults-CollapsingQueryParser]]
|
||||
== Collapsing Query Parser
|
||||
|
||||
The `CollapsingQParser` is really a _post filter_ that provides more performant field collapsing than Solr's standard approach when the number of distinct groups in the result set is high. This parser collapses the result set to a single document per group before it forwards the result set to the rest of the search components. So all downstream components (faceting, highlighting, etc...) will work with the collapsed result set.
|
||||
|
||||
The CollapsingQParser accepts the following local parameters:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,60,20",options="header"]
|
||||
|===
|
||||
|Parameter |Description |Default
|
||||
|field |The field that is being collapsed on. The field must be a single valued String, Int or Float |none
|
||||
|min \| max a|
|
||||
Selects the group head document for each group based on which document has the min or max value of the specified numeric field or <<function-queries.adoc#function-queries,function query>>.
|
||||
|
||||
At most only one of the min, max, or sort (see below) parameters may be specified.
|
||||
|
||||
If none are specified, the group head document of each group will be selected based on the highest scoring document in that group. |none
|
||||
|sort a|
|
||||
Selects the group head document for each group based on which document comes first according to the specified <<common-query-parameters.adoc#CommonQueryParameters-ThesortParameter,sort string>>.
|
||||
|
||||
At most only one of the min, max, (see above) or sort parameters may be specified.
|
||||
|
||||
If none are specified, the group head document of each group will be selected based on the highest scoring document in that group. |none
|
||||
|nullPolicy a|
|
||||
There are three null policies:
|
||||
|
||||
* *ignore*: removes documents with a null value in the collapse field. This is the default.
|
||||
* *expand*: treats each document with a null value in the collapse field as a separate group.
|
||||
* *collapse*: collapses all documents with a null value into a single group using either highest score, or minimum/maximum.
|
||||
|
||||
|ignore
|
||||
|hint |Currently there is only one hint available: `top_fc`, which stands for top level FieldCache. The `top_fc` hint is only available when collapsing on String fields. `top_fc` usually provides the best query time speed but takes the longest to warm on startup or following a commit. `top_fc` will also result in having the collapsed field cached in memory twice if it's used for faceting or sorting. For very high cardinality (high distinct count) fields, `top_fc` may not fare so well. |none
|
||||
|size |Sets the initial size of the collapse data structures when collapsing on a *numeric field only*. The data structures used for collapsing grow dynamically when collapsing on numeric fields. Setting the size above the number of results expected in the result set will eliminate the resizing cost. |100,000
|
||||
|===
|
||||
|
||||
*Sample Syntax:*
|
||||
|
||||
Collapse on `group_field` selecting the document in each group with the highest scoring document:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
fq={!collapse field=group_field}
|
||||
----
|
||||
|
||||
Collapse on `group_field` selecting the document in each group with the minimum value of `numeric_field`:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
fq={!collapse field=group_field min=numeric_field}
|
||||
----
|
||||
|
||||
Collapse on `group_field` selecting the document in each group with the maximum value of `numeric_field`:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
fq={!collapse field=group_field max=numeric_field}
|
||||
----
|
||||
|
||||
Collapse on `group_field` selecting the document in each group with the maximum value of a function. Note that the *cscore()* function can be used with the min/max options to use the score of the current document being collapsed.
|
||||
|
||||
[source,text]
|
||||
----
|
||||
fq={!collapse field=group_field max=sum(cscore(),numeric_field)}
|
||||
----
|
||||
|
||||
Collapse on `group_field` with a null policy so that all docs that do not have a value in the `group_field` will be treated as a single group. For each group, the selected document will be based first on a `numeric_field`, but ties will be broken by score:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
fq={!collapse field=group_field nullPolicy=collapse sort='numeric_field asc, score desc'}
|
||||
----
|
||||
|
||||
Collapse on `group_field` with a hint to use the top level field cache:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
fq={!collapse field=group_field hint=top_fc}
|
||||
----
|
||||
|
||||
The CollapsingQParserPlugin fully supports the QueryElevationComponent.
|
||||
|
||||
[[CollapseandExpandResults-ExpandComponent]]
|
||||
== Expand Component
|
||||
|
||||
The ExpandComponent can be used to expand the groups that were collapsed by the http://heliosearch.org/the-collapsingqparserplugin-solrs-new-high-performance-field-collapsing-postfilter/[CollapsingQParserPlugin].
|
||||
|
||||
Example usage with the CollapsingQParserPlugin:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
q=foo&fq={!collapse field=ISBN}
|
||||
----
|
||||
|
||||
In the query above, the CollapsingQParserPlugin will collapse the search results on the _ISBN_ field. The main search results will contain the highest ranking document from each book.
|
||||
|
||||
The ExpandComponent can now be used to expand the results so you can see the documents grouped by ISBN. For example:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
q=foo&fq={!collapse field=ISBN}&expand=true
|
||||
----
|
||||
|
||||
The “expand=true” parameter turns on the ExpandComponent. The ExpandComponent adds a new section to the search output labeled “expanded”.
|
||||
|
||||
Inside the expanded section there is a _map_ with each group head pointing to the expanded documents that are within the group. As applications iterate the main collapsed result set, they can access the _expanded_ map to retrieve the expanded groups.
|
||||
|
||||
The ExpandComponent has the following parameters:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,60,20",options="header"]
|
||||
|===
|
||||
|Parameter |Description |Default
|
||||
|expand.sort |Orders the documents within the expanded groups |score desc
|
||||
|expand.rows |The number of rows to display in each group |5
|
||||
|expand.q |Overrides the main q parameter, determines which documents to include in the main group. |main q
|
||||
|expand.fq |Overrides main fq's, determines which documents to include in the main group. |main fq's
|
||||
|===
|
|
@ -0,0 +1,30 @@
|
|||
= Collection-Specific Tools
|
||||
:page-shortname: collection-specific-tools
|
||||
:page-permalink: collection-specific-tools.html
|
||||
:page-children: analysis-screen, dataimport-screen, documents-screen, files-screen, query-screen, stream-screen, schema-browser-screen
|
||||
|
||||
In the left-hand navigation bar, you will see a pull-down menu titled "Collection Selector" that can be used to access collection specific administration screens.
|
||||
|
||||
.Only Visible When Using SolrCloud
|
||||
[NOTE]
|
||||
====
|
||||
The "Collection Selector" pull-down menu is only available on Solr instances running in <<solrcloud.adoc#solrcloud,SolrCloud mode>>.
|
||||
|
||||
Single node or master/slave replication instances of Solr will not display this menu, instead the Collection specific UI pages described in this section will be available in the <<core-specific-tools.adoc#core-specific-tools,Core Selector pull-down menu>>.
|
||||
====
|
||||
|
||||
Clicking on the Collection Selector pull-down menu will show a list of the collections in your Solr cluster, with a search box that can be used to find a specific collection by name. When you select a collection from the pull-down, the main display of the page will display some basic metadata about the collection, and a secondary menu will appear in the left nav with links to additional collection specific administration screens.
|
||||
|
||||
image::images/collection-specific-tools/collection_dashboard.png[image,width=482,height=250]
|
||||
|
||||
The collection-specific UI screens are listed below, with a link to the section of this guide to find out more:
|
||||
|
||||
// TODO: SOLR-10655 BEGIN: refactor this into a 'collection-screens-list.include.adoc' file for reuse
|
||||
* <<analysis-screen.adoc#analysis-screen,Analysis>> - lets you analyze the data found in specific fields.
|
||||
* <<dataimport-screen.adoc#dataimport-screen,Dataimport>> - shows you information about the current status of the Data Import Handler.
|
||||
* <<documents-screen.adoc#documents-screen,Documents>> - provides a simple form allowing you to execute various Solr indexing commands directly from the browser.
|
||||
* <<files-screen.adoc#files-screen,Files>> - shows the current core configuration files such as `solrconfig.xml`.
|
||||
* <<query-screen.adoc#query-screen,Query>> - lets you submit a structured query about various elements of a core.
|
||||
* <<stream-screen.adoc#stream-screen,Stream>> - allows you to submit streaming expressions and see results and parsing explanations.
|
||||
* <<schema-browser-screen.adoc#schema-browser-screen,Schema Browser>> - displays schema data in a browser window.
|
||||
// TODO: SOLR-10655 END
|
File diff suppressed because it is too large
Load Diff
|
@ -0,0 +1,25 @@
|
|||
= Collections / Core Admin
|
||||
:page-shortname: collections-core-admin
|
||||
:page-permalink: collections-core-admin.html
|
||||
|
||||
The Collections screen provides some basic functionality for managing your Collections, powered by the <<collections-api.adoc#collections-api,Collections API>>.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
If you are running a single node Solr instance, you will not see a Collections option in the left nav menu of the Admin UI.
|
||||
|
||||
You will instead see a "Core Admin" screen that supports some comparable Core level information & manipulation via the <<coreadmin-api.adoc#coreadmin-api,CoreAdmin API>> instead.
|
||||
====
|
||||
|
||||
The main display of this page provides a list of collections that exist in your cluster. Clicking on a collection name provides some basic metadata about how the collection is defined, and its current shards & replicas, with options for adding and deleting individual replicas.
|
||||
|
||||
The buttons at the top of the screen let you make various collection level changes to your cluster, from add new collections or aliases to reloading or deleting a single collection.
|
||||
|
||||
image::images/collections-core-admin/collection-admin.png[image,width=653,height=250]
|
||||
|
||||
|
||||
Replicas can be deleted by clicking the red "X" next to the replica name.
|
||||
|
||||
If the shard is inactive, for example after a <<collections-api.adoc#CollectionsAPI-splitshard,SPLITSHARD action>>, an option to delete the shard will appear as a red "X" next to the shard name.
|
||||
|
||||
image::images/collections-core-admin/DeleteShard.png[image,width=486,height=250]
|
|
@ -0,0 +1,19 @@
|
|||
= Combining Distribution and Replication
|
||||
:page-shortname: combining-distribution-and-replication
|
||||
:page-permalink: combining-distribution-and-replication.html
|
||||
|
||||
When your index is too large for a single machine and you have a query volume that single shards cannot keep up with, it's time to replicate each shard in your distributed search setup.
|
||||
|
||||
The idea is to combine distributed search with replication. As shown in the figure below, a combined distributed-replication configuration features a master server for each shard and then 1-_n_ slaves that are replicated from the master. As in a standard replicated configuration, the master server handles updates and optimizations without adversely affecting query handling performance.
|
||||
|
||||
Query requests should be load balanced across each of the shard slaves. This gives you both increased query handling capacity and fail-over backup if a server goes down.
|
||||
|
||||
.A Solr configuration combining both replication and master-slave distribution.
|
||||
image::images/combining-distribution-and-replication/worddav4101c16174820e932b44baa22abcfcd1.png[image,width=312,height=344]
|
||||
|
||||
|
||||
None of the master shards in this configuration know about each other. You index to each master, the index is replicated to each slave, and then searches are distributed across the slaves, using one slave from each master/slave shard.
|
||||
|
||||
For high availability you can use a load balancer to set up a virtual IP for each shard's set of slaves. If you are new to load balancing, HAProxy (http://haproxy.1wt.eu/) is a good open source software load-balancer. If a slave server goes down, a good load-balancer will detect the failure using some technique (generally a heartbeat system), and forward all requests to the remaining live slaves that served with the failed slave. A single virtual IP should then be set up so that requests can hit a single IP, and get load balanced to each of the virtual IPs for the search slaves.
|
||||
|
||||
With this configuration you will have a fully load balanced, search-side fault-tolerant system (Solr does not yet support fault-tolerant indexing). Incoming searches will be handed off to one of the functioning slaves, then the slave will distribute the search request across a slave for each of the shards in your configuration. The slave will issue a request to each of the virtual IPs for each shard, and the load balancer will choose one of the available slaves. Finally, the results will be combined into a single results set and returned. If any of the slaves go down, they will be taken out of rotation and the remaining slaves will be used. If a shard master goes down, searches can still be served from the slaves until you have corrected the problem and put the master back into production.
|
|
@ -0,0 +1,120 @@
|
|||
= Command Line Utilities
|
||||
:page-shortname: command-line-utilities
|
||||
:page-permalink: command-line-utilities.html
|
||||
|
||||
Solr's Administration page (found by default at `\http://hostname:8983/solr/`), provides a section with menu items for monitoring indexing and performance statistics.
|
||||
|
||||
This pag also includes information about index distribution and replication, and information on all threads running in the JVM at the time.
|
||||
|
||||
There is also a section where you can run queries, and an assistance area.
|
||||
|
||||
In addition, SolrCloud provides its own administration page (found at http://localhost:8983/solr/#/~cloud), as well as a few tools available via a ZooKeeper Command Line Utility (CLI). The CLI scripts found in `server/scripts/cloud-scripts` let you upload configuration information to ZooKeeper, in the same two ways that were shown in the examples in <<parameter-reference.adoc#parameter-reference,Parameter Reference>>. It also provides a few other commands that let you link collection sets to collections, make ZooKeeper paths or clear them, and download configurations from ZooKeeper to the local filesystem.
|
||||
|
||||
.Solr's zkcli.sh vs ZooKeeper's zkCli.sh vs Solr Start Script
|
||||
[IMPORTANT]
|
||||
====
|
||||
The `zkcli.sh` provided by Solr is not the same as the https://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_ConnectingToZooKeeper[`zkCli.sh` included in ZooKeeper distributions].
|
||||
|
||||
ZooKeeper's `zkCli.sh` provides a completely general, application-agnostic shell for manipulating data in ZooKeeper. Solr's `zkcli.sh` – discussed in this section – is specific to Solr, and has command line arguments specific to dealing with Solr data in ZooKeeper.
|
||||
|
||||
Many of the functions provided by the zkCli.sh script are also provided by the <<solr-control-script-reference.adoc#solr-control-script-reference,Solr Control Script Reference>>, which may be more familiar as the start script ZooKeeper maintenance commands are very similar to Unix commands.
|
||||
====
|
||||
|
||||
[[CommandLineUtilities-UsingSolr_sZooKeeperCLI]]
|
||||
== Using Solr's ZooKeeper CLI
|
||||
|
||||
Both `zkcli.sh` (for Unix environments) and `zkcli.bat` (for Windows environments) support the following command line options:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="10,30,60",options="header"]
|
||||
|===
|
||||
|Short |Parameter Usage |Meaning
|
||||
| |`-cmd <arg>` |CLI Command to be executed: `bootstrap`, `upconfig`, `downconfig`, `linkconfig`, `makepath`, `get`, `getfile`, `put`, `putfile`, `list`, `clear` or `clusterprop`. This parameter is **mandatory**.
|
||||
|`-z` |`-zkhost <locations>` |ZooKeeper host address. This parameter is *mandatory* for all CLI commands.
|
||||
|`-c` |`-collection <name>` |For `linkconfig`: name of the collection.
|
||||
|`-d` |`-confdir <path>` |For `upconfig`: a directory of configuration files. For downconfig: the destination of files pulled from ZooKeeper
|
||||
|`-h` |`-help` |Display help text.
|
||||
|`-n` |`-confname <arg>` |For `upconfig`, `linkconfig`, `downconfig`: name of the configuration set.
|
||||
|`-r` |`-runzk <port>` |Run ZooKeeper internally by passing the Solr run port; only for clusters on one machine.
|
||||
|`-s` |`-solrhome <path>` |For `bootstrap` or when using `-runzk`: the *mandatory* solrhome location.
|
||||
| |`-name <name>` |For `clusterprop`: the *mandatory* cluster property name.
|
||||
| |`-val <value>` |For `clusterprop`: the cluster property value. If not specified, *null* will be used as value.
|
||||
|===
|
||||
|
||||
The short form parameter options may be specified with a single dash (eg: `-c mycollection`).
|
||||
|
||||
The long form parameter options may be specified using either a single dash (eg: `-collection mycollection`) or a double dash (eg: `--collection mycollection`)
|
||||
|
||||
[[CommandLineUtilities-ZooKeeperCLIExamples]]
|
||||
== ZooKeeper CLI Examples
|
||||
|
||||
Below are some examples of using the `zkcli.sh` CLI which assume you have already started the SolrCloud example (`bin/solr -e cloud -noprompt`)
|
||||
|
||||
If you are on Windows machine, simply replace `zkcli.sh` with `zkcli.bat` in these examples.
|
||||
|
||||
[[CommandLineUtilities-Uploadaconfigurationdirectory]]
|
||||
=== Upload a configuration directory
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd upconfig -confname my_new_config -confdir server/solr/configsets/basic_configs/conf
|
||||
----
|
||||
|
||||
[[CommandLineUtilities-BootstrapZooKeeperfromexistingSOLR_HOME]]
|
||||
=== Bootstrap ZooKeeper from existing SOLR_HOME
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd bootstrap -solrhome /var/solr/data
|
||||
----
|
||||
|
||||
.Bootstrap with chroot
|
||||
[NOTE]
|
||||
====
|
||||
Using the boostrap command with a zookeeper chroot in the -zkhost parameter, e.g. `-zkhost 127.0.0.1:2181/solr`, will automatically create the chroot path before uploading the configs.
|
||||
====
|
||||
|
||||
[[CommandLineUtilities-PutarbitrarydataintoanewZooKeeperfile]]
|
||||
=== Put arbitrary data into a new ZooKeeper file
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd put /my_zk_file.txt 'some data'
|
||||
----
|
||||
|
||||
[[CommandLineUtilities-PutalocalfileintoanewZooKeeperfile]]
|
||||
=== Put a local file into a new ZooKeeper file
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd putfile /my_zk_file.txt /tmp/my_local_file.txt
|
||||
----
|
||||
|
||||
[[CommandLineUtilities-Linkacollectiontoaconfigurationset]]
|
||||
=== Link a collection to a configuration set
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd linkconfig -collection gettingstarted -confname my_new_config
|
||||
----
|
||||
|
||||
[[CommandLineUtilities-CreateanewZooKeeperpath]]
|
||||
=== Create a new ZooKeeper path
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd makepath /solr
|
||||
----
|
||||
|
||||
This can be useful to create a chroot path in ZooKeeper before first cluster start.
|
||||
|
||||
[[CommandLineUtilities-Setaclusterproperty]]
|
||||
=== Set a cluster property
|
||||
|
||||
This command will add or modify a single cluster property in `clusterprops.json`. Use this command instead of the usual getfile -> edit -> putfile cycle. Unlike the CLUSTERPROP REST API, this command does *not* require a running Solr cluster.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd clusterprop -name urlScheme -val https
|
||||
----
|
|
@ -0,0 +1,365 @@
|
|||
= Common Query Parameters
|
||||
:page-shortname: common-query-parameters
|
||||
:page-permalink: common-query-parameters.html
|
||||
|
||||
Several query parsers share supported query parameters.
|
||||
|
||||
The table below summarizes Solr's common query parameters, which are supported by the <<requesthandlers-and-searchcomponents-in-solrconfig#RequestHandlersandSearchComponentsinSolrConfig-SearchHandlers,Search RequestHandlers>>
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="30,70",options="header"]
|
||||
|===
|
||||
|Parameter |Description
|
||||
|<<CommonQueryParameters-ThedefTypeParameter,defType>> |Selects the query parser to be used to process the query.
|
||||
|<<CommonQueryParameters-ThesortParameter,sort>> |Sorts the response to a query in either ascending or descending order based on the response's score or another specified characteristic.
|
||||
|<<CommonQueryParameters-ThestartParameter,start>> |Specifies an offset (by default, 0) into the responses at which Solr should begin displaying content.
|
||||
|<<CommonQueryParameters-TherowsParameter,rows>> |Controls how many rows of responses are displayed at a time (default value: 10)
|
||||
|<<CommonQueryParameters-Thefq_FilterQuery_Parameter,fq>> |Applies a filter query to the search results.
|
||||
|<<CommonQueryParameters-Thefl_FieldList_Parameter,fl>> |Limits the information included in a query response to a specified list of fields. The fields need to either be `stored="true"` or `docValues="true"`
|
||||
|<<CommonQueryParameters-ThedebugParameter,debug>> |Request additional debugging information in the response. Specifying the `debug=timing` parameter returns just the timing information; specifying the `debug=results` parameter returns "explain" information for each of the documents returned; specifying the `debug=query parameter` returns all of the debug information.
|
||||
|<<CommonQueryParameters-TheexplainOtherParameter,explainOther>> |Allows clients to specify a Lucene query to identify a set of documents. If non-blank, the explain info of each document which matches this query, relative to the main query (specified by the q parameter) will be returned along with the rest of the debugging information.
|
||||
|<<CommonQueryParameters-ThetimeAllowedParameter,timeAllowed>> |Defines the time allowed for the query to be processed. If the time elapses before the query response is complete, partial information may be returned.
|
||||
|<<CommonQueryParameters-ThesegmentTerminateEarlyParameter,segmentTerminateEarly>> |Indicates that, if possible, Solr should stop collecting documents from each individual (sorted) segment once it can determine that any subsequent documents in that segment will not be candidates for the `rows` being returned. The default is false.
|
||||
|<<CommonQueryParameters-TheomitHeaderParameter,omitHeader>> |Excludes the header from the returned results, if set to true. The header contains information about the request, such as the time the request took to complete. The default is false.
|
||||
|<<CommonQueryParameters-ThewtParameter,wt>> |Specifies the Response Writer to be used to format the query response.
|
||||
|<<CommonQueryParameters-ThelogParamsListParameter,logParamsList>> |By default, Solr logs all parameters. Set this parameter to restrict which parameters are logged. Valid entries are the parameters to be logged, separated by commas (i.e., `logParamsList=param1,param2`). An empty list will log no parameters, so if logging all parameters is desired, do not define this additional parameter at all.
|
||||
|<<CommonQueryParameters-TheechoParamsParameter,echoParams>> |The response header can include parameters sent with the query request. This parameter controls what is contained in that section of the response header. Valid values are `none`, `all`, and `explicit`. The default value is `explicit.`
|
||||
|===
|
||||
|
||||
The following sections describe these parameters in detail.
|
||||
|
||||
[[CommonQueryParameters-ThedefTypeParameter]]
|
||||
== The `defType` Parameter
|
||||
|
||||
The defType parameter selects the query parser that Solr should use to process the main query parameter (`q`) in the request. For example:
|
||||
|
||||
`defType=dismax`
|
||||
|
||||
If no defType param is specified, then by default, the <<the-standard-query-parser.adoc#the-standard-query-parser,The Standard Query Parser>> is used. (eg: `defType=lucene`)
|
||||
|
||||
[[CommonQueryParameters-ThesortParameter]]
|
||||
== The `sort` Parameter
|
||||
|
||||
The `sort` parameter arranges search results in either ascending (`asc`) or descending (`desc`) order. The parameter can be used with either numerical or alphabetical content. The directions can be entered in either all lowercase or all uppercase letters (i.e., both `asc` or `ASC`).
|
||||
|
||||
Solr can sort query responses according to document scores or the value of any field with a single value that is either indexed or uses <<docvalues.adoc#docvalues,DocValues>> (that is, any field whose attributes in the Schema include `multiValued="false"` and either `docValues="true"` or `indexed="true"` – if the field does not have DocValues enabled, the indexed terms are used to build them on the fly at runtime), provided that:
|
||||
|
||||
* the field is non-tokenized (that is, the field has no analyzer and its contents have been parsed into tokens, which would make the sorting inconsistent), or
|
||||
|
||||
* the field uses an analyzer (such as the KeywordTokenizer) that produces only a single term.
|
||||
|
||||
If you want to be able to sort on a field whose contents you want to tokenize to facilitate searching, <<copying-fields.adoc#copying-fields,use a `copyField` directive>> in the the Schema to clone the field. Then search on the field and sort on its clone.
|
||||
|
||||
The table explains how Solr responds to various settings of the `sort` parameter.
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="30,70",options="header"]
|
||||
|===
|
||||
|Example |Result
|
||||
| |If the sort parameter is omitted, sorting is performed as though the parameter were set to score `desc`.
|
||||
|score desc |Sorts in descending order from the highest score to the lowest score.
|
||||
|price asc |Sorts in ascending order of the price field
|
||||
|inStock desc, price asc |Sorts by the contents of the `inStock` field in descending order, then within those results sorts in ascending order by the contents of the price field.
|
||||
|===
|
||||
|
||||
Regarding the sort parameter's arguments:
|
||||
|
||||
* A sort ordering must include a field name (or `score` as a pseudo field), followed by whitespace (escaped as + or `%20` in URL strings), followed by a sort direction (`asc` or `desc`).
|
||||
|
||||
* Multiple sort orderings can be separated by a comma, using this syntax: `sort=<field name>+<direction>,<field name>+<direction>],...`
|
||||
** When more than one sort criteria is provided, the second entry will only be used if the first entry results in a tie. If there is a third entry, it will only be used if the first AND second entries are tied. This pattern continues with further entries.
|
||||
|
||||
[[CommonQueryParameters-ThestartParameter]]
|
||||
== The `start` Parameter
|
||||
|
||||
When specified, the `start` parameter specifies an offset into a query's result set and instructs Solr to begin displaying results from this offset.
|
||||
|
||||
The default value is "0". In other words, by default, Solr returns results without an offset, beginning where the results themselves begin.
|
||||
|
||||
Setting the `start` parameter to some other number, such as 3, causes Solr to skip over the preceding records and start at the document identified by the offset.
|
||||
|
||||
You can use the `start` parameter this way for paging. For example, if the `rows` parameter is set to 10, you could display three successive pages of results by setting start to 0, then re-issuing the same query and setting start to 10, then issuing the query again and setting start to 20.
|
||||
|
||||
[[CommonQueryParameters-TherowsParameter]]
|
||||
== The `rows` Parameter
|
||||
|
||||
You can use the rows parameter to paginate results from a query. The parameter specifies the maximum number of documents from the complete result set that Solr should return to the client at one time.
|
||||
|
||||
The default value is 10. That is, by default, Solr returns 10 documents at a time in response to a query.
|
||||
|
||||
[[CommonQueryParameters-Thefq_FilterQuery_Parameter]]
|
||||
== The `fq` (Filter Query) Parameter
|
||||
|
||||
The `fq` parameter defines a query that can be used to restrict the superset of documents that can be returned, without influencing score. It can be very useful for speeding up complex queries, since the queries specified with `fq` are cached independently of the main query. When a later query uses the same filter, there's a cache hit, and filter results are returned quickly from the cache.
|
||||
|
||||
When using the `fq` parameter, keep in mind the following:
|
||||
|
||||
* The `fq` parameter can be specified multiple times in a query. Documents will only be included in the result if they are in the intersection of the document sets resulting from each instance of the parameter. In the example below, only documents which have a popularity greater then 10 and have a section of 0 will match.
|
||||
+
|
||||
[source,text]
|
||||
----
|
||||
fq=popularity:[10 TO *]&fq=section:0
|
||||
----
|
||||
|
||||
* Filter queries can involve complicated Boolean queries. The above example could also be written as a single `fq` with two mandatory clauses like so:
|
||||
+
|
||||
[source,text]
|
||||
----
|
||||
fq=+popularity:[10 TO *] +section:0
|
||||
----
|
||||
|
||||
* The document sets from each filter query are cached independently. Thus, concerning the previous examples: use a single `fq` containing two mandatory clauses if those clauses appear together often, and use two separate `fq` parameters if they are relatively independent. (To learn about tuning cache sizes and making sure a filter cache actually exists, see <<the-well-configured-solr-instance.adoc#the-well-configured-solr-instance,The Well-Configured Solr Instance>>.)
|
||||
* It is also possible to use <<the-standard-query-parser.adoc#TheStandardQueryParser-DifferencesbetweenLuceneQueryParserandtheSolrStandardQueryParser,filter(condition) syntax>> inside the `fq` to cache clauses individually and - among other things - to achieve union of cached filter queries.
|
||||
|
||||
* As with all parameters: special characters in an URL need to be properly escaped and encoded as hex values. Online tools are available to help you with URL-encoding. For example: http://meyerweb.com/eric/tools/dencoder/.
|
||||
|
||||
[[CommonQueryParameters-Thefl_FieldList_Parameter]]
|
||||
== The `fl` (Field List) Parameter
|
||||
|
||||
The `fl` parameter limits the information included in a query response to a specified list of fields. The fields need to either be `stored="true"` or `docValues="true"``.`
|
||||
|
||||
The field list can be specified as a space-separated or comma-separated list of field names. The string "score" can be used to indicate that the score of each document for the particular query should be returned as a field. The wildcard character `*` selects all the fields in the document which are either `stored="true"` or `docValues="true"` and `useDocValuesAsStored="true"` (which is the default when docValues are enabled). You can also add pseudo-fields, functions and transformers to the field list request.
|
||||
|
||||
This table shows some basic examples of how to use `fl`:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="30,70",options="header"]
|
||||
|===
|
||||
|Field List |Result
|
||||
|id name price |Return only the id, name, and price fields.
|
||||
|id,name,price |Return only the id, name, and price fields.
|
||||
|id name, price |Return only the id, name, and price fields.
|
||||
|id score |Return the id field and the score.
|
||||
|* |Return all the `stored` fields in each document, as well as any `docValues` fields that have `useDocValuesAsStored="true"`. This is the default value of the fl parameter.
|
||||
|* score |Return all the fields in each document, along with each field's score.
|
||||
|*,dv_field_name |Return all the `stored` fields in each document, and any `docValues` fields that have `useDocValuesAsStored="true`" and the docValues from dv_field_name even if it has `useDocValuesAsStored="false`"
|
||||
|===
|
||||
|
||||
[[CommonQueryParameters-FunctionValues]]
|
||||
=== Function Values
|
||||
|
||||
<<function-queries.adoc#function-queries,Functions>> can be computed for each document in the result and returned as a pseudo-field:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
fl=id,title,product(price,popularity)
|
||||
----
|
||||
|
||||
[[CommonQueryParameters-DocumentTransformers]]
|
||||
=== Document Transformers
|
||||
|
||||
<<transforming-result-documents.adoc#transforming-result-documents,Document Transformers>> can be used to modify the information returned about each documents in the results of a query:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
fl=id,title,[explain]
|
||||
----
|
||||
|
||||
[[CommonQueryParameters-FieldNameAliases]]
|
||||
=== Field Name Aliases
|
||||
|
||||
You can change the key used to in the response for a field, function, or transformer by prefixing it with a `_"displayName_:`". For example:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
fl=id,sales_price:price,secret_sauce:prod(price,popularity),why_score:[explain style=nl]
|
||||
----
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"response": {
|
||||
"numFound": 2,
|
||||
"start": 0,
|
||||
"docs": [{
|
||||
"id": "6H500F0",
|
||||
"secret_sauce": 2100.0,
|
||||
"sales_price": 350.0,
|
||||
"why_score": {
|
||||
"match": true,
|
||||
"value": 1.052226,
|
||||
"description": "weight(features:cache in 2) [DefaultSimilarity], result of:",
|
||||
"details": [{
|
||||
"..."
|
||||
}]}}]}}
|
||||
----
|
||||
|
||||
[[CommonQueryParameters-ThedebugParameter]]
|
||||
== The `debug` Parameter
|
||||
|
||||
The `debug` parameter can be specified multiple times and supports the following arguments:
|
||||
|
||||
* `debug=query`: return debug information about the query only.
|
||||
* `debug=timing`: return debug information about how long the query took to process.
|
||||
* `debug=results`: return debug information about the score results (also known as "explain").
|
||||
** By default, score explanations are returned as large string values, using newlines and tab indenting for structure & readability, but an additional `debug.explain.structured=true` parameter may be specified to return this information as nested data structures native to the response format requested by `wt`.
|
||||
* `debug=all`: return all available debug information about the request request. (alternatively usage: `debug=true`)
|
||||
|
||||
For backwards compatibility with older versions of Solr, `debugQuery=true` may instead be specified as an alternative way to indicate `debug=all`
|
||||
|
||||
The default behavior is not to include debugging information.
|
||||
|
||||
[[CommonQueryParameters-TheexplainOtherParameter]]
|
||||
== The `explainOther` Parameter
|
||||
|
||||
The `explainOther` parameter specifies a Lucene query in order to identify a set of documents. If this parameter is included and is set to a non-blank value, the query will return debugging information, along with the "explain info" of each document that matches the Lucene query, relative to the main query (which is specified by the q parameter). For example:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
q=supervillians&debugQuery=on&explainOther=id:juggernaut
|
||||
----
|
||||
|
||||
The query above allows you to examine the scoring explain info of the top matching documents, compare it to the explain info for documents matching `id:juggernaut`, and determine why the rankings are not as you expect.
|
||||
|
||||
The default value of this parameter is blank, which causes no extra "explain info" to be returned.
|
||||
|
||||
[[CommonQueryParameters-ThetimeAllowedParameter]]
|
||||
== The `timeAllowed` Parameter
|
||||
|
||||
This parameter specifies the amount of time, in milliseconds, allowed for a search to complete. If this time expires before the search is complete, any partial results will be returned, but values such as `numFound`, <<faceting.adoc#faceting,facet>> counts, and result <<the-stats-component.adoc#the-stats-component,stats>> may not be accurate for the entire result set.
|
||||
|
||||
This value is only checked at the time of:
|
||||
|
||||
1. Query Expansion, and
|
||||
2. Document collection
|
||||
|
||||
As this check is periodically performed, the actual time for which a request can be processed before it is aborted would be marginally greater than or equal to the value of `timeAllowed`. If the request consumes more time in other stages, e.g., custom components, etc., this parameter is not expected to abort the request.
|
||||
|
||||
[[CommonQueryParameters-ThesegmentTerminateEarlyParameter]]
|
||||
== The `segmentTerminateEarly` Parameter
|
||||
|
||||
This parameter may be set to either true or false.
|
||||
|
||||
If set to true, and if <<indexconfig-in-solrconfig.adoc#IndexConfiginSolrConfig-mergePolicyFactory,the mergePolicyFactory>> for this collection is a {solr-javadocs}/solr-core/org/apache/solr/index/SortingMergePolicyFactory.html[`SortingMergePolicyFactory`] which uses a `sort` option which is compatible with <<CommonQueryParameters-ThesortParameter,the sort parameter>> specified for this query, then Solr will attempt to use an {lucene-javadocs}/core/org/apache/lucene/search/EarlyTerminatingSortingCollector.html[`EarlyTerminatingSortingCollector`].
|
||||
|
||||
If early termination is used, a `segmentTerminatedEarly` header will be included in the `responseHeader`.
|
||||
|
||||
Similar to using <<CommonQueryParameters-ThetimeAllowedParameter,the `timeAllowed `Parameter>>, when early segment termination happens values such as `numFound`, <<faceting.adoc#faceting,Facet>> counts, and result <<the-stats-component.adoc#the-stats-component,Stats>> may not be accurate for the entire result set.
|
||||
|
||||
The default value of this parameter is false.
|
||||
|
||||
[[CommonQueryParameters-TheomitHeaderParameter]]
|
||||
== The `omitHeader` Parameter
|
||||
|
||||
This parameter may be set to either true or false.
|
||||
|
||||
If set to true, this parameter excludes the header from the returned results. The header contains information about the request, such as the time it took to complete. The default value for this parameter is false.
|
||||
|
||||
[[CommonQueryParameters-ThewtParameter]]
|
||||
== The `wt` Parameter
|
||||
|
||||
The `wt` parameter selects the Response Writer that Solr should use to format the query's response. For detailed descriptions of Response Writers, see <<response-writers.adoc#response-writers,Response Writers>>.
|
||||
|
||||
[[CommonQueryParameters-Thecache_falseParameter]]
|
||||
== The `cache=false` Parameter
|
||||
|
||||
Solr caches the results of all queries and filter queries by default. To disable result caching, set the `cache=false` parameter.
|
||||
|
||||
You can also use the `cost` option to control the order in which non-cached filter queries are evaluated. This allows you to order less expensive non-cached filters before expensive non-cached filters.
|
||||
|
||||
For very high cost filters, if `cache=false` and `cost>=100` and the query implements the `PostFilter` interface, a Collector will be requested from that query and used to filter documents after they have matched the main query and all other filter queries. There can be multiple post filters; they are also ordered by cost.
|
||||
|
||||
For example:
|
||||
// TODO: fix this, it looks horrible (CT)
|
||||
[source,text]
|
||||
----
|
||||
// normal function range query used as a filter, all matching documents
|
||||
// generated up front and cached
|
||||
fq={!frange l=10 u=100}mul(popularity,price)
|
||||
|
||||
// function range query run in parallel with the main query like a traditional
|
||||
// lucene filter
|
||||
fq={!frange l=10 u=100 cache=false}mul(popularity,price)
|
||||
|
||||
// function range query checked after each document that already matches the query
|
||||
// and all other filters. Good for really expensive function queries.
|
||||
fq={!frange l=10 u=100 cache=false cost=100}mul(popularity,price)
|
||||
----
|
||||
|
||||
[[CommonQueryParameters-ThelogParamsListParameter]]
|
||||
== The `logParamsList` Parameter
|
||||
|
||||
By default, Solr logs all parameters of requests. Set this parameter to restrict which parameters of a request are logged. This may help control logging to only those parameters considered important to your organization.
|
||||
|
||||
For example, you could define this like:
|
||||
|
||||
`logParamsList=q,fq`
|
||||
|
||||
And only the 'q' and 'fq' parameters will be logged.
|
||||
|
||||
If no parameters should be logged, you can send `logParamsList` as empty (i.e., `logParamsList=`).
|
||||
|
||||
[TIP]
|
||||
====
|
||||
This parameter does not only apply to query requests, but to any kind of request to Solr.
|
||||
====
|
||||
|
||||
[[CommonQueryParameters-TheechoParamsParameter]]
|
||||
== The `echoParams` Parameter
|
||||
|
||||
The `echoParams` parameter controls what information about request parameters is included in the response header.
|
||||
|
||||
The table explains how Solr responds to various settings of the `echoParams` parameter:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="30,70",options="header"]
|
||||
|===
|
||||
|Value |Meaning
|
||||
|explicit |This is the default value. Only parameters included in the actual request, plus the `_` parameter (which is a 64-bit numeric timestamp) will be added to the params section of the response header.
|
||||
|all |Include all request parameters that contributed to the query. This will include everything defined in the request handler definition found in `solrconfig.xml` as well as parameters included with the request, plus the `_` parameter. If a parameter is included in the request handler definition AND the request, it will appear multiple times in the response header.
|
||||
|none |Entirely removes the "params" section of the response header. No information about the request parameters will be available in the response.
|
||||
|===
|
||||
|
||||
Here is an example of a JSON response where the echoParams parameter was not included, so the default of `explicit` is active. The request URL that created this response included three parameters - `q`, `wt`, and `indent`:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader": {
|
||||
"status": 0,
|
||||
"QTime": 0,
|
||||
"params": {
|
||||
"q": "solr",
|
||||
"indent": "true",
|
||||
"wt": "json",
|
||||
"_": "1458227751857"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"numFound": 0,
|
||||
"start": 0,
|
||||
"docs": []
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
This is what happens if a similar request is sent that adds `echoParams=all` to the three parameters used in the previous example:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader": {
|
||||
"status": 0,
|
||||
"QTime": 0,
|
||||
"params": {
|
||||
"q": "solr",
|
||||
"df": "text",
|
||||
"preferLocalShards": "false",
|
||||
"indent": "true",
|
||||
"echoParams": "all",
|
||||
"rows": "10",
|
||||
"wt": "json",
|
||||
"_": "1458228887287"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"numFound": 0,
|
||||
"start": 0,
|
||||
"docs": []
|
||||
}
|
||||
}
|
||||
----
|
|
@ -0,0 +1,521 @@
|
|||
= Config API
|
||||
:page-shortname: config-api
|
||||
:page-permalink: config-api.html
|
||||
|
||||
The Config API enables manipulating various aspects of your `solrconfig.xml` using REST-like API calls.
|
||||
|
||||
This feature is enabled by default and works similarly in both SolrCloud and standalone mode. Many commonly edited properties (such as cache sizes and commit settings) and request handler definitions can be changed with this API.
|
||||
|
||||
When using this API, `solrconfig.xml` is not changed. Instead, all edited configuration is stored in a file called `configoverlay.json`. The values in `configoverlay.json` override the values in `solrconfig.xml`.
|
||||
|
||||
[[ConfigAPI-APIEntryPoints]]
|
||||
== API Entry Points
|
||||
|
||||
* `/config`: retrieve or modify the config. GET to retrieve and POST for executing commands
|
||||
* `/config/overlay`: retrieve the details in the `configoverlay.json` alone
|
||||
* `/config/params` : allows creating parameter sets that can override or take the place of parameters defined in `solrconfig.xml`. See the <<request-parameters-api.adoc#request-parameters-api,Request Parameters API>> section for more details.
|
||||
|
||||
[[ConfigAPI-Retrievingtheconfig]]
|
||||
== Retrieving the config
|
||||
|
||||
All configuration items, can be retrieved by sending a GET request to the `/config` endpoint - the results will be the effective configuration resulting from merging settings in `configoverlay.json` with those in `solrconfig.xml`:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config
|
||||
----
|
||||
|
||||
To restrict the returned results to a top level section, e.g. `query`, `requestHandler` or `updateHandler`, append the name of the section to the `/config` endpoint following a slash. E.g. to retrieve configuration for all request handlers:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config/requestHandler
|
||||
----
|
||||
|
||||
To further restrict returned results to a single component within a top level section, use the `componentName` request param, e.g. to return configuration for the `/select` request handler:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config/requestHandler?componentName=/select
|
||||
----
|
||||
|
||||
[[ConfigAPI-Commandstomodifytheconfig]]
|
||||
== Commands to modify the config
|
||||
|
||||
This API uses specific commands to tell Solr what property or type of property to add to `configoverlay.json`. The commands are passed as part of the data sent with the request.
|
||||
|
||||
The config commands are categorized into 3 different sections which manipulate various data structures in `solrconfig.xml`. Each of these is described below.
|
||||
|
||||
* <<ConfigAPI-CommandsforCommonProperties,Common Properties>>
|
||||
* <<ConfigAPI-CommandsforCustomHandlersandLocalComponents,Components>>
|
||||
* <<ConfigAPI-CommandsforUser-DefinedProperties,User-defined properties>>
|
||||
|
||||
[[ConfigAPI-CommandsforCommonProperties]]
|
||||
=== Commands for Common Properties
|
||||
|
||||
The common properties are those that are frequently need to be customized in a Solr instance. They are manipulated with two commands:
|
||||
|
||||
* `set-property`: Set a well known property. The names of the properties are predefined and fixed. If the property has already been set, this command will overwrite the previous setting.
|
||||
* `unset-property`: Remove a property set using the `set-property` command.
|
||||
|
||||
The properties that are configured with these commands are predefined and listed below. The names of these properties are derived from their XML paths as found in `solrconfig.xml`.
|
||||
|
||||
* `updateHandler.autoCommit.maxDocs`
|
||||
* `updateHandler.autoCommit.maxTime`
|
||||
* `updateHandler.autoCommit.openSearcher`
|
||||
* `updateHandler.autoSoftCommit.maxDocs`
|
||||
* `updateHandler.autoSoftCommit.maxTime`
|
||||
* `updateHandler.commitWithin.softCommit`
|
||||
* `updateHandler.indexWriter.closeWaitsForMerges`
|
||||
* `query.filterCache.class`
|
||||
* `query.filterCache.size`
|
||||
* `query.filterCache.initialSize`
|
||||
* `query.filterCache.autowarmCount`
|
||||
* `query.filterCache.regenerator`
|
||||
* `query.queryResultCache.class`
|
||||
* `query.queryResultCache.size`
|
||||
* `query.queryResultCache.initialSize`
|
||||
* `query.queryResultCache.autowarmCount`
|
||||
* `query.queryResultCache.regenerator`
|
||||
* `query.documentCache.class`
|
||||
* `query.documentCache.size`
|
||||
* `query.documentCache.initialSize`
|
||||
* `query.documentCache.autowarmCount`
|
||||
|
||||
* `query.documentCache.regenerator`
|
||||
* `query.fieldValueCache.class`
|
||||
* `query.fieldValueCache.size`
|
||||
* `query.fieldValueCache.initialSize`
|
||||
* `query.fieldValueCache.autowarmCount`
|
||||
* `query.fieldValueCache.regenerator`
|
||||
* `query.useFilterForSortedQuery`
|
||||
* `query.queryResultWindowSize`
|
||||
* `query.queryResultMaxDocCached`
|
||||
* `query.enableLazyFieldLoading`
|
||||
* `query.boolToFilterOptimizer`
|
||||
* `query.maxBooleanClauses`
|
||||
* `jmx.agentId`
|
||||
* `jmx.serviceUrl`
|
||||
* `jmx.rootName`
|
||||
* `requestDispatcher.handleSelect`
|
||||
* `requestDispatcher.requestParsers.multipartUploadLimitInKB`
|
||||
* `requestDispatcher.requestParsers.formdataUploadLimitInKB`
|
||||
* `requestDispatcher.requestParsers.enableRemoteStreaming`
|
||||
* `requestDispatcher.requestParsers.addHttpRequestToContext`
|
||||
|
||||
[[ConfigAPI-CommandsforCustomHandlersandLocalComponents]]
|
||||
=== Commands for Custom Handlers and Local Components
|
||||
|
||||
Custom request handlers, search components, and other types of localized Solr components (such as custom query parsers, update processors, etc.) can be added, updated and deleted with specific commands for the component being modified.
|
||||
|
||||
The syntax is similar in each case: `add-<component-name>`, `update-<component-name>`, and `delete-<component-name>`. The command name is not case sensitive, so `Add-RequestHandler`, `ADD-REQUESTHANDLER` and `add-requesthandler` are all equivalent.
|
||||
|
||||
In each case, `add-` commands add the new configuration to `configoverlay.json`, which will override any other settings for the component in `solrconfig.xml`; `update-` commands overwrite an existing setting in `configoverlay.json`; and `delete-` commands remove the setting from `configoverlay.json`.
|
||||
|
||||
Settings removed from `configoverlay.json` are not removed from `solrconfig.xml`.
|
||||
|
||||
The full list of available commands follows below:
|
||||
|
||||
[[ConfigAPI-GeneralPurposeCommands]]
|
||||
==== General Purpose Commands
|
||||
|
||||
These commands are the most commonly used:
|
||||
|
||||
* `add-requesthandler`
|
||||
* `update-requesthandler`
|
||||
* `delete-requesthandler`
|
||||
* `add-searchcomponent`
|
||||
* `update-searchcomponent`
|
||||
* `delete-searchcomponent`
|
||||
* `add-initparams`
|
||||
* `update-initparams`
|
||||
* `delete-initparams`
|
||||
* `add-queryresponsewriter`
|
||||
* `update-queryresponsewriter`
|
||||
* `delete-queryresponsewriter`
|
||||
|
||||
[[ConfigAPI-AdvancedCommands]]
|
||||
==== Advanced Commands
|
||||
|
||||
These commands allow registering more advanced customizations to Solr:
|
||||
|
||||
* `add-queryparser`
|
||||
* `update-queryparser`
|
||||
* `delete-queryparser`
|
||||
* `add-valuesourceparser`
|
||||
* `update-valuesourceparser`
|
||||
* `delete-valuesourceparser`
|
||||
* `add-transformer`
|
||||
* `update-transformer`
|
||||
* `delete-transformer`
|
||||
* `add-updateprocessor`
|
||||
* `update-updateprocessor`
|
||||
* `delete-updateprocessor`
|
||||
|
||||
* `add-queryconverter`
|
||||
* `update-queryconverter`
|
||||
* `delete-queryconverter`
|
||||
* `add-listener`
|
||||
* `update-listener`
|
||||
* `delete-listener`
|
||||
* `add-runtimelib`
|
||||
* `update-runtimelib`
|
||||
* `delete-runtimelib`
|
||||
|
||||
See the section <<ConfigAPI-CreatingandUpdatingRequestHandlers,Creating and Updating Request Handlers>> below for examples of using these commands.
|
||||
|
||||
[[ConfigAPI-Whatabout_updateRequestProcessorChain_]]
|
||||
==== What about updateRequestProcessorChain?
|
||||
|
||||
The Config API does not let you create or edit `updateRequestProcessorChain` elements. However, it is possible to create `updateProcessor` entries and can use them by name to create a chain.
|
||||
|
||||
example:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
|
||||
"add-updateprocessor" : { "name" : "firstFld",
|
||||
"class": "solr.FirstFieldValueUpdateProcessorFactory",
|
||||
"fieldName":"test_s"}}'
|
||||
----
|
||||
|
||||
You can use this directly in your request by adding a parameter in the `updateRequestProcessorChain` for the specific update processor called `processor=firstFld`.
|
||||
|
||||
[[ConfigAPI-CommandsforUser-DefinedProperties]]
|
||||
=== Commands for User-Defined Properties
|
||||
|
||||
Solr lets users templatize the `solrconfig.xml` using the place holder format `${variable_name:default_val}`. You could set the values using system properties, for example, `-Dvariable_name= my_customvalue`. The same can be achieved during runtime using these commands:
|
||||
|
||||
* `set-user-property`: Set a user-defined property. If the property has already been set, this command will overwrite the previous setting.
|
||||
* `unset-user-property`: Remove a user-defined property.
|
||||
|
||||
The structure of the request is similar to the structure of requests using other commands, in the format of `"command":{"variable_name":"property_value"}`. You can add more than one variable at a time if necessary.
|
||||
|
||||
For more information about user-defined properties, see the section <<configuring-solrconfig-xml.adoc#Configuringsolrconfig.xml-Userdefinedpropertiesfromcore.properties,User defined properties from core.properties>>.
|
||||
|
||||
See also the section <<ConfigAPI-CreatingandUpdatingUser-DefinedProperties,Creating and Updating User-Defined Properties>> below for examples of how to use this type of command.
|
||||
|
||||
[[ConfigAPI-HowtoMapsolrconfig.xmlPropertiestoJSON]]
|
||||
== How to Map `solrconfig.xml` Properties to JSON
|
||||
|
||||
By using this API, you will be generating JSON representations of properties defined in `solrconfig.xml`. To understand how properties should be represented with the API, let's take a look at a few examples.
|
||||
|
||||
Here is what a request handler looks like in `solrconfig.xml`:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<requestHandler name="/query" class="solr.SearchHandler">
|
||||
<lst name="defaults">
|
||||
<str name="echoParams">explicit</str>
|
||||
<str name="wt">json</str>
|
||||
<str name="indent">true</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
----
|
||||
|
||||
The same request handler defined with the Config API would look like this:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"add-requesthandler":{
|
||||
"name":"/query",
|
||||
"class":"solr.SearchHandler",
|
||||
"defaults":{
|
||||
"echoParams":"explicit",
|
||||
"wt":"json",
|
||||
"indent":true
|
||||
}
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
The QueryElevationComponent searchComponent in `solrconfig.xml` looks like this:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<searchComponent name="elevator" class="solr.QueryElevationComponent" >
|
||||
<str name="queryFieldType">string</str>
|
||||
<str name="config-file">elevate.xml</str>
|
||||
</searchComponent>
|
||||
----
|
||||
|
||||
And the same searchComponent with the Config API:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"add-searchcomponent":{
|
||||
"name":"elevator",
|
||||
"class":"QueryElevationComponent",
|
||||
"queryFieldType":"string",
|
||||
"config-file":"elevate.xml"
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
Removing the searchComponent with the Config API:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"delete-searchcomponent":"elevator"
|
||||
}
|
||||
----
|
||||
|
||||
A simple highlighter looks like this in `solrconfig.xml` (example has been truncated for space):
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<searchComponent class="solr.HighlightComponent" name="highlight">
|
||||
<highlighting>
|
||||
<fragmenter name="gap"
|
||||
default="true"
|
||||
class="solr.highlight.GapFragmenter">
|
||||
<lst name="defaults">
|
||||
<int name="hl.fragsize">100</int>
|
||||
</lst>
|
||||
</fragmenter>
|
||||
|
||||
<formatter name="html"
|
||||
default="true"
|
||||
class="solr.highlight.HtmlFormatter">
|
||||
<lst name="defaults">
|
||||
<str name="hl.simple.pre"><![CDATA[<em>]]></str>
|
||||
<str name="hl.simple.post"><![CDATA[</em>]]></str>
|
||||
</lst>
|
||||
</formatter>
|
||||
|
||||
<encoder name="html" class="solr.highlight.HtmlEncoder" />
|
||||
...
|
||||
</highlighting>
|
||||
----
|
||||
|
||||
The same highlighter with the Config API:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"add-searchcomponent": {
|
||||
"name": "highlight",
|
||||
"class": "solr.HighlightComponent",
|
||||
"": {
|
||||
"gap": {
|
||||
"default": "true",
|
||||
"name": "gap",
|
||||
"class": "solr.highlight.GapFragmenter",
|
||||
"defaults": {
|
||||
"hl.fragsize": 100
|
||||
}
|
||||
}
|
||||
},
|
||||
"html": [{
|
||||
"default": "true",
|
||||
"name": "html",
|
||||
"class": "solr.highlight.HtmlFormatter",
|
||||
"defaults": {
|
||||
"hl.simple.pre": "before-",
|
||||
"hl.simple.post": "-after"
|
||||
}
|
||||
}, {
|
||||
"name": "html",
|
||||
"class": "solr.highlight.HtmlEncoder"
|
||||
}]
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
Set autoCommit properties in `solrconfig.xml`:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<autoCommit>
|
||||
<maxTime>15000</maxTime>
|
||||
<openSearcher>false</openSearcher>
|
||||
</autoCommit>
|
||||
----
|
||||
|
||||
Define the same properties with the Config API:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"set-property": {
|
||||
"updateHandler.autoCommit.maxTime":15000,
|
||||
"updateHandler.autoCommit.openSearcher":false
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
[[ConfigAPI-NameComponentsfortheConfigAPI]]
|
||||
=== Name Components for the Config API
|
||||
|
||||
The Config API always allows changing the configuration of any component by name. However, some configurations such as `listener` or `initParams` do not require a name in `solrconfig.xml`. In order to be able to `update` and `delete` of the same item in `configoverlay.json`, the name attribute becomes mandatory.
|
||||
|
||||
[[ConfigAPI-Examples]]
|
||||
== Examples
|
||||
|
||||
[[ConfigAPI-CreatingandUpdatingCommonProperties]]
|
||||
=== Creating and Updating Common Properties
|
||||
|
||||
This change sets the `query.filterCache.autowarmCount` to 1000 items and unsets the `query.filterCache.size`.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d'{
|
||||
"set-property" : {"query.filterCache.autowarmCount":1000},
|
||||
"unset-property" :"query.filterCache.size"}'
|
||||
----
|
||||
|
||||
Using the `/config/overlay` endpoint, you can verify the changes with a request like this:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/gettingstarted/config/overlay?omitHeader=true
|
||||
----
|
||||
|
||||
And you should get a response like this:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"overlay":{
|
||||
"znodeVersion":1,
|
||||
"props":{"query":{"filterCache":{
|
||||
"autowarmCount":1000,
|
||||
"size":25}}}}}
|
||||
----
|
||||
|
||||
[[ConfigAPI-CreatingandUpdatingRequestHandlers]]
|
||||
=== Creating and Updating Request Handlers
|
||||
|
||||
To create a request handler, we can use the `add-requesthandler` command:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
|
||||
"add-requesthandler" : {
|
||||
"name": "/mypath",
|
||||
"class":"solr.DumpRequestHandler",
|
||||
"defaults":{ "x":"y" ,"a":"b", "wt":"json", "indent":true },
|
||||
"useParams":"x"
|
||||
}
|
||||
}'
|
||||
----
|
||||
|
||||
Make a call to the new request handler to check if it is registered:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/mypath?omitHeader=true
|
||||
----
|
||||
|
||||
And you should see the following as output:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"params":{
|
||||
"indent":"true",
|
||||
"a":"b",
|
||||
"x":"y",
|
||||
"wt":"json"},
|
||||
"context":{
|
||||
"webapp":"/solr",
|
||||
"path":"/mypath",
|
||||
"httpMethod":"GET"}}
|
||||
----
|
||||
|
||||
To update a request handler, you should use the `update-requesthandler` command :
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
|
||||
"update-requesthandler": {
|
||||
"name": "/mypath",
|
||||
"class":"solr.DumpRequestHandler",
|
||||
"defaults": {"x":"new value for X", "wt":"json", "indent":true},
|
||||
"useParams":"x"
|
||||
}
|
||||
}'
|
||||
----
|
||||
|
||||
As another example, we'll create another request handler, this time adding the 'terms' component as part of the definition:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
|
||||
"add-requesthandler": {
|
||||
"name": "/myterms",
|
||||
"class":"solr.SearchHandler",
|
||||
"defaults": {"terms":true, "distrib":false},
|
||||
"components": [ "terms" ]
|
||||
}
|
||||
}'
|
||||
----
|
||||
|
||||
[[ConfigAPI-CreatingandUpdatingUser-DefinedProperties]]
|
||||
=== Creating and Updating User-Defined Properties
|
||||
|
||||
This command sets a user property.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H'Content-type:application/json' -d '{
|
||||
"set-user-property" : {"variable_name":"some_value"}}'
|
||||
----
|
||||
|
||||
Again, we can use the `/config/overlay` endpoint to verify the changes have been made:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config/overlay?omitHeader=true
|
||||
----
|
||||
|
||||
And we would expect to see output like this:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{"overlay":{
|
||||
"znodeVersion":5,
|
||||
"userProps":{
|
||||
"variable_name":"some_value"}}
|
||||
}
|
||||
----
|
||||
|
||||
To unset the variable, issue a command like this:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H'Content-type:application/json' -d '{"unset-user-property" : "variable_name"}'
|
||||
----
|
||||
|
||||
[[ConfigAPI-HowItWorks]]
|
||||
== How It Works
|
||||
|
||||
Every core watches the ZooKeeper directory for the configset being used with that core. In standalone mode, however, there is no watch (because ZooKeeper is not running). If there are multiple cores in the same node using the same configset, only one ZooKeeper watch is used. For instance, if the configset 'myconf' is used by a core, the node would watch `/configs/myconf`. Every write operation performed through the API would 'touch' the directory (sets an empty byte[] to trigger watches) and all watchers are notified. Every core would check if the Schema file, `solrconfig.xml` or `configoverlay.json` is modified by comparing the `znode` versions and if modified, the core is reloaded.
|
||||
|
||||
If `params.json` is modified, the params object is just updated without a core reload (see the section <<request-parameters-api.adoc#request-parameters-api,Request Parameters API>> for more information about `params.json`).
|
||||
|
||||
[[ConfigAPI-EmptyCommand]]
|
||||
=== Empty Command
|
||||
|
||||
If an empty command is sent to the `/config` endpoint, the watch is triggered on all cores using this configset. For example:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/techproducts/config -H'Content-type:application/json' -d '{}'
|
||||
----
|
||||
|
||||
Directly editing any files without 'touching' the directory *will not* make it visible to all nodes.
|
||||
|
||||
It is possible for components to watch for the configset 'touch' events by registering a listener using `SolrCore#registerConfListener()` .
|
||||
|
||||
[[ConfigAPI-ListeningtoconfigChanges]]
|
||||
=== Listening to config Changes
|
||||
|
||||
Any component can register a listener using:
|
||||
|
||||
`SolrCore#addConfListener(Runnable listener)`
|
||||
|
||||
to get notified for config changes. This is not very useful if the files modified result in core reloads (i.e., `configoverlay.xml` or Schema). Components can use this to reload the files they are interested in.
|
|
@ -0,0 +1,26 @@
|
|||
= Config Sets
|
||||
:page-shortname: config-sets
|
||||
:page-permalink: config-sets.html
|
||||
|
||||
On a multicore Solr instance, you may find that you want to share configuration between a number of different cores. You can achieve this using named configsets, which are essentially shared configuration directories stored under a configurable configset base directory.
|
||||
|
||||
To create a configset, simply add a new directory under the configset base directory. The configset will be identified by the name of this directory. Then into this copy the config directory you want to share. The structure should look something like this:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
/<configSetBaseDir>
|
||||
/configset1
|
||||
/conf
|
||||
/managed-schema
|
||||
/solrconfig.xml
|
||||
/configset2
|
||||
/conf
|
||||
/managed-schema
|
||||
/solrconfig.xml
|
||||
----
|
||||
|
||||
The default base directory is `$SOLR_HOME/configsets`, and it can be configured in `solr.xml`.
|
||||
|
||||
To create a new core using a configset, pass `configSet` as one of the core properties. For example, if you do this via the core admin API:
|
||||
|
||||
`\http://localhost:8983/admin/cores?action=CREATE&name=mycore&instanceDir=path/to/instance&configSet=configset2`
|
|
@ -0,0 +1,155 @@
|
|||
= ConfigSets API
|
||||
:page-shortname: configsets-api
|
||||
:page-permalink: configsets-api.html
|
||||
|
||||
The ConfigSets API enables you to create, delete, and otherwise manage ConfigSets.
|
||||
|
||||
To use a ConfigSet created with this API as the configuration for a collection, use the <<collections-api.adoc#collections-api,Collections API>>.
|
||||
|
||||
This API can only be used with Solr running in SolrCloud mode. If you are not running Solr in SolrCloud mode but would still like to use shared configurations, please see the section <<config-sets.adoc#config-sets,Config Sets>>.
|
||||
|
||||
[[ConfigSetsAPI-APIEntryPoints]]
|
||||
== API Entry Points
|
||||
|
||||
The base URL for all API calls is `\http://<hostname>:<port>/solr`.
|
||||
|
||||
* `/admin/configs?action=CREATE`: <<ConfigSetsAPI-create,create>> a ConfigSet, based on an existing ConfigSet
|
||||
* `/admin/configs?action=DELETE`: <<ConfigSetsAPI-delete,delete>> a ConfigSet
|
||||
* `/admin/configs?action=LIST`: <<ConfigSetsAPI-list,list>> all ConfigSets
|
||||
|
||||
[[ConfigSetsAPI-createCreateaConfigSet]]
|
||||
|
||||
[[ConfigSetsAPI-create]]
|
||||
== Create a ConfigSet
|
||||
|
||||
`/admin/configs?action=CREATE&name=_name_&baseConfigSet=_baseConfigSet_`
|
||||
|
||||
Create a ConfigSet, based on an existing ConfigSet.
|
||||
|
||||
[[ConfigSetsAPI-Input]]
|
||||
=== Input
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,15,10,15,40",options="header"]
|
||||
|===
|
||||
|Key |Type |Required |Default |Description
|
||||
|name |String |Yes | |ConfigSet to be created
|
||||
|baseConfigSet |String |Yes | |ConfigSet to copy as a base
|
||||
|configSetProp._name=value_ |String |No | |ConfigSet property from base to override
|
||||
|===
|
||||
|
||||
[[ConfigSetsAPI-Output]]
|
||||
=== Output
|
||||
|
||||
*Output Content*
|
||||
|
||||
The output will include the status of the request. If the status is anything other than "success", an error message will explain why the request failed.
|
||||
|
||||
[[ConfigSetsAPI-Examples]]
|
||||
=== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
Create a ConfigSet named 'myConfigSet' based on a 'predefinedTemplate' ConfigSet, overriding the immutable property to false.
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/admin/configs?action=CREATE&name=myConfigSet&baseConfigSet=predefinedTemplate&configSetProp.immutable=false
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<response>
|
||||
<lst name="responseHeader">
|
||||
<int name="status">0</int>
|
||||
<int name="QTime">323</int>
|
||||
</lst>
|
||||
</response>
|
||||
----
|
||||
|
||||
[[ConfigSetsAPI-deleteDeleteaConfigSet]]
|
||||
|
||||
[[ConfigSetsAPI-delete]]
|
||||
== Delete a ConfigSet
|
||||
|
||||
`/admin/configs?action=DELETE&name=_name_`
|
||||
|
||||
Delete a ConfigSet
|
||||
|
||||
[[ConfigSetsAPI-Input.1]]
|
||||
=== Input
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,15,10,15,40",options="header"]
|
||||
|===
|
||||
|Key |Type |Required |Default |Description
|
||||
|name |String |Yes | |ConfigSet to be deleted
|
||||
|===
|
||||
|
||||
[[ConfigSetsAPI-Output.1]]
|
||||
=== Output
|
||||
|
||||
*Output Content*
|
||||
|
||||
The output will include the status of the request. If the status is anything other than "success", an error message will explain why the request failed.
|
||||
|
||||
[[ConfigSetsAPI-Examples.1]]
|
||||
=== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
Delete ConfigSet 'myConfigSet'
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/admin/configs?action=DELETE&name=myConfigSet
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<response>
|
||||
<lst name="responseHeader">
|
||||
<int name="status">0</int>
|
||||
<int name="QTime">170</int>
|
||||
</lst>
|
||||
</response>
|
||||
----
|
||||
|
||||
[[ConfigSetsAPI-listListConfigSets]]
|
||||
|
||||
[[ConfigSetsAPI-list]]
|
||||
== List ConfigSets
|
||||
|
||||
`/admin/configs?action=LIST`
|
||||
|
||||
Fetch the names of the ConfigSets in the cluster.
|
||||
|
||||
[[ConfigSetsAPI-Examples.2]]
|
||||
=== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/admin/configs?action=LIST&wt=json
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader":{
|
||||
"status":0,
|
||||
"QTime":203},
|
||||
"configSets":["myConfigSet1",
|
||||
"myConfig2"]}
|
||||
----
|
|
@ -0,0 +1,11 @@
|
|||
= Configuration APIs
|
||||
:page-shortname: configuration-apis
|
||||
:page-permalink: configuration-apis.html
|
||||
:page-children: blob-store-api, config-api, request-parameters-api, managed-resources
|
||||
|
||||
Solr includes several APIs that can be used to modify settings in `solrconfig.xml`.
|
||||
|
||||
* <<blob-store-api.adoc#blob-store-api,Blob Store API>>
|
||||
* <<config-api.adoc#config-api,Config API>>
|
||||
* <<request-parameters-api.adoc#request-parameters-api,Request Parameters API>>
|
||||
* <<managed-resources.adoc#managed-resources,Managed Resources>>
|
|
@ -0,0 +1,110 @@
|
|||
= Configuring Logging
|
||||
:page-shortname: configuring-logging
|
||||
:page-permalink: configuring-logging.html
|
||||
|
||||
Solr logs are a key way to know what's happening in the system. There are several ways to adjust the default logging configuration.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
In addition to the logging options described below, there is a way to configure which request parameters (such as parameters sent as part of queries) are logged with an additional request parameter called `logParamsList`. See the section on <<common-query-parameters.adoc#CommonQueryParameters-ThelogParamsListParameter,Common Query Parameters>> for more information.
|
||||
====
|
||||
|
||||
[[ConfiguringLogging-TemporaryLoggingSettings]]
|
||||
== Temporary Logging Settings
|
||||
|
||||
You can control the amount of logging output in Solr by using the Admin Web interface. Select the *LOGGING* link. Note that this page only lets you change settings in the running system and is not saved for the next run. (For more information about the Admin Web interface, see <<using-the-solr-administration-user-interface.adoc#using-the-solr-administration-user-interface,Using the Solr Administration User Interface>>.)
|
||||
|
||||
.The Logging Screen
|
||||
image::images/logging/logging.png[image]
|
||||
|
||||
This part of the Admin Web interface allows you to set the logging level for many different log categories. Fortunately, any categories that are *unset* will have the logging level of its parent. This makes it possible to change many categories at once by adjusting the logging level of their parent.
|
||||
|
||||
When you select **Level**, you see the following menu:
|
||||
|
||||
.The Log Level Menu
|
||||
image::images/logging/level_menu.png[image,width=1159,height=577]
|
||||
|
||||
Directories are shown with their current logging levels. The Log Level Menu floats over these. To set a log level for a particular directory, select it and click the appropriate log level button.
|
||||
|
||||
Log levels settings are as follows:
|
||||
|
||||
[width="100%",options="header",]
|
||||
|===
|
||||
|Level |Result
|
||||
|FINEST |Reports everything.
|
||||
|FINE |Reports everything but the least important messages.
|
||||
|CONFIG |Reports configuration errors.
|
||||
|INFO |Reports everything but normal status.
|
||||
|WARN |Reports all warnings.
|
||||
|SEVERE |Reports only the most severe warnings.
|
||||
|OFF |Turns off logging.
|
||||
|UNSET |Removes the previous log setting.
|
||||
|===
|
||||
|
||||
Multiple settings at one time are allowed.
|
||||
|
||||
[[ConfiguringLogging-LoglevelAPI]]
|
||||
=== Log level API
|
||||
|
||||
There is also a way of sending REST commands to the logging endpoint to do the same. Example:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
# Set the root logger to level WARN
|
||||
curl -s http://localhost:8983/solr/admin/info/logging --data-binary "set=root:WARN&wt=json"
|
||||
----
|
||||
|
||||
[[ConfiguringLogging-ChoosingLogLevelatStartup]]
|
||||
== Choosing Log Level at Startup
|
||||
|
||||
You can temporarily choose a different logging level as you start Solr. There are two ways:
|
||||
|
||||
The first way is to set the `SOLR_LOG_LEVEL` environment variable before you start Solr, or place the same variable in `bin/solr.in.sh` or `bin/solr.in.cmd`. The variable must contain an uppercase string with a supported log level (see above).
|
||||
|
||||
The second way is to start Solr with the -v or -q options, see <<solr-control-script-reference.adoc#solr-control-script-reference,Solr Control Script Reference>> for details. Examples:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
# Start with verbose (DEBUG) looging
|
||||
bin/solr start -f -v
|
||||
# Start with quiet (WARN) logging
|
||||
bin/solr start -f -q
|
||||
----
|
||||
|
||||
[[ConfiguringLogging-PermanentLoggingSettings]]
|
||||
== Permanent Logging Settings
|
||||
|
||||
Solr uses http://logging.apache.org/log4j/1.2/[Log4J version 1.2] for logging which is configured using `server/resources/log4j.properties`. Take a moment to inspect the contents of the `log4j.properties` file so that you are familiar with its structure. By default, Solr log messages will be written to `SOLR_LOGS_DIR/solr.log`.
|
||||
|
||||
When you're ready to deploy Solr in production, set the variable `SOLR_LOGS_DIR` to the location where you want Solr to write log files, such as `/var/solr/logs`. You may also want to tweak `log4j.properties`. Note that if you installed Solr as a service using the instructions provided in <<taking-solr-to-production.adoc#taking-solr-to-production,Taking Solr to Production>>, then see `/var/solr/log4j.properties` instead of the default `server/resources` version.
|
||||
|
||||
When starting Solr in the foreground (`-f` option), all logs will be sent to the console, in addition to `solr.log`. When starting Solr in the background, it will write all `stdout` and `stderr` output to a log file in `solr-<port>-console.log`, and automatically disable the CONSOLE logger configured in `log4j.properties`, having the same effect as if you removed the CONSOLE appender from the rootLogger manually.
|
||||
|
||||
Also, in `log4j.properties` the default log rotation size threshold of 4MB is most likely too small for production servers and should be increased to a larger value (such as 100MB or more).
|
||||
|
||||
[source,text]
|
||||
----
|
||||
log4j.appender.file.MaxFileSize=100MB
|
||||
----
|
||||
|
||||
Java Garbage Collection logs are rotated by the JVM when size hits 20M, for a max of 9 generations. Old GC logs are moved to `SOLR_LOGS_DIR/archived`. These settings can only be changed by editing the start scripts.
|
||||
|
||||
On every startup of Solr, the start script will clean up old logs and rotate the main `solr.log` file. If you changed the `log4j.appender.file.MaxBackupIndex` setting in `log4j.properties`, you also need to change the corresponding setting `-rotate_solr_logs 9` in the start script.
|
||||
|
||||
You can disable the automatic log rotation at startup by changing the setting `SOLR_LOG_PRESTART_ROTATION` found in `bin/solr.in.sh` or `bin/solr.in.cmd` to false.
|
||||
|
||||
[[ConfiguringLogging-LoggingSlowQueries]]
|
||||
== Logging Slow Queries
|
||||
|
||||
For high-volume search applications, logging every query can generate a large amount of logs and, depending on the volume, potentially impact performance. If you mine these logs for additional insights into your application, then logging every query request may be useful.
|
||||
|
||||
On the other hand, if you're only concerned about warnings and error messages related to requests, then you can set the log verbosity to WARN. However, this poses a potential problem in that you won't know if any queries are slow, as slow queries are still logged at the INFO level.
|
||||
|
||||
Solr provides a way to set your log verbosity threshold to WARN and be able to set a latency threshold above which a request is considered "slow" and log that request at the WARN level to help you identify slow queries in your application. To enable this behavior, configure the `<slowQueryThresholdMillis>` element in the *query* section of solrconfig.xml:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<slowQueryThresholdMillis>1000</slowQueryThresholdMillis>
|
||||
----
|
||||
|
||||
Any queries that take longer than the specified threshold will be logged as "slow" queries at the WARN level.
|
|
@ -0,0 +1,155 @@
|
|||
= Configuring solrconfig.xml
|
||||
:page-shortname: configuring-solrconfig-xml
|
||||
:page-permalink: configuring-solrconfig-xml.html
|
||||
:page-children: datadir-and-directoryfactory-in-solrconfig, lib-directives-in-solrconfig, schema-factory-definition-in-solrconfig, indexconfig-in-solrconfig, requesthandlers-and-searchcomponents-in-solrconfig, initparams-in-solrconfig, updatehandlers-in-solrconfig, query-settings-in-solrconfig, requestdispatcher-in-solrconfig, update-request-processors, codec-factory
|
||||
|
||||
The `solrconfig.xml` file is the configuration file with the most parameters affecting Solr itself.
|
||||
|
||||
While configuring Solr, you'll work with `solrconfig.xml` often, either directly or via the <<config-api.adoc#config-api,Config API>> to create "configuration overlays" (`configoverlay.json`) to override the values in `solrconfig.xml`.
|
||||
|
||||
In `solrconfig.xml`, you configure important features such as:
|
||||
|
||||
* request handlers, which process the requests to Solr, such as requests to add documents to the index or requests to return results for a query
|
||||
|
||||
* listeners, processes that "listen" for particular query-related events; listeners can be used to trigger the execution of special code, such as invoking some common queries to warm-up caches
|
||||
|
||||
* the Request Dispatcher for managing HTTP communications
|
||||
|
||||
* the Admin Web interface
|
||||
|
||||
* parameters related to replication and duplication (these parameters are covered in detail in <<legacy-scaling-and-distribution.adoc#legacy-scaling-and-distribution,Legacy Scaling and Distribution>>)
|
||||
|
||||
The `solrconfig.xml` file is located in the `conf/` directory for each collection. Several well-commented example files can be found in the `server/solr/configsets/` directories demonstrating best practices for many different types of installations.
|
||||
|
||||
We've covered the options in the following sections:
|
||||
|
||||
* <<datadir-and-directoryfactory-in-solrconfig.adoc#datadir-and-directoryfactory-in-solrconfig,DataDir and DirectoryFactory in SolrConfig>>
|
||||
* <<lib-directives-in-solrconfig.adoc#lib-directives-in-solrconfig,Lib Directives in SolrConfig>>
|
||||
* <<schema-factory-definition-in-solrconfig.adoc#schema-factory-definition-in-solrconfig,Schema Factory Definition in SolrConfig>>
|
||||
* <<indexconfig-in-solrconfig.adoc#indexconfig-in-solrconfig,IndexConfig in SolrConfig>>
|
||||
* <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,RequestHandlers and SearchComponents in SolrConfig>>
|
||||
* <<initparams-in-solrconfig.adoc#initparams-in-solrconfig,InitParams in SolrConfig>>
|
||||
* <<updatehandlers-in-solrconfig.adoc#updatehandlers-in-solrconfig,UpdateHandlers in SolrConfig>>
|
||||
* <<query-settings-in-solrconfig.adoc#query-settings-in-solrconfig,Query Settings in SolrConfig>>
|
||||
* <<requestdispatcher-in-solrconfig.adoc#requestdispatcher-in-solrconfig,RequestDispatcher in SolrConfig>>
|
||||
* <<update-request-processors.adoc#update-request-processors,Update Request Processors>>
|
||||
* <<codec-factory.adoc#codec-factory,Codec Factory>>
|
||||
|
||||
[[Configuringsolrconfig.xml-SubstitutingPropertiesinSolrConfigFiles]]
|
||||
== Substituting Properties in Solr Config Files
|
||||
|
||||
Solr supports variable substitution of property values in config files, which allows runtime specification of various configuration options in `solrconfig.xml`. The syntax is `${propertyname[:option default value]`}. This allows defining a default that can be overridden when Solr is launched. If a default value is not specified, then the property _must_ be specified at runtime or the configuration file will generate an error when parsed.
|
||||
|
||||
There are multiple methods for specifying properties that can be used in configuration files. Of those below, strongly consider "config overlay" as the preferred approach, as it stays local to the config set and because it's easy to modify.
|
||||
|
||||
[[Configuringsolrconfig.xml-JVMSystemProperties]]
|
||||
=== JVM System Properties
|
||||
|
||||
Any JVM System properties, usually specified using the `-D` flag when starting the JVM, can be used as variables in any XML configuration file in Solr.
|
||||
|
||||
For example, in the sample `solrconfig.xml` files, you will see this value which defines the locking type to use:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<lockType>${solr.lock.type:native}</lockType>
|
||||
----
|
||||
|
||||
Which means the lock type defaults to "native" but when starting Solr, you could override this using a JVM system property by launching the Solr it with:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
bin/solr start -Dsolr.lock.type=none
|
||||
----
|
||||
|
||||
In general, any Java system property that you want to set can be passed through the `bin/solr` script using the standard `-Dproperty=value` syntax. Alternatively, you can add common system properties to the `SOLR_OPTS` environment variable defined in the Solr include file (`bin/solr.in.sh` or `bin/solr.in.cmd`). For more information about how the Solr include file works, refer to: <<taking-solr-to-production.adoc#taking-solr-to-production,Taking Solr to Production>>.
|
||||
|
||||
[[Configuringsolrconfig.xml-ConfigAPI]]
|
||||
=== Config API
|
||||
|
||||
The <<config-api.adoc#config-api,Config API>> allows you to use an API to modify Solr's configuration, specifically user defined properties. Changes made with this API are stored in a file named `configoverlay.json`. This file should only be edited with the API, but will look like this example:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{"userProps":{
|
||||
"dih.db.url":"jdbc:oracle:thin:@localhost:1521",
|
||||
"dih.db.user":"username",
|
||||
"dih.db.pass":"password"}}
|
||||
----
|
||||
|
||||
For more details, see the section <<config-api.adoc#config-api,Config API>>.
|
||||
|
||||
[[Configuringsolrconfig.xml-solrcore.properties]]
|
||||
=== solrcore.properties
|
||||
|
||||
If the configuration directory for a Solr core contains a file named `solrcore.properties` that file can contain any arbitrary user defined property names and values using the Java standard https://en.wikipedia.org/wiki/.properties[properties file format], and those properties can be used as variables in the XML configuration files for that Solr core.
|
||||
|
||||
For example, the following `solrcore.properties` file could be created in the `conf/` directory of a collection using one of the example configurations, to override the lockType used.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
#conf/solrcore.properties
|
||||
solr.lock.type=none
|
||||
----
|
||||
|
||||
.Deprecation
|
||||
[WARNING]
|
||||
====
|
||||
`solrcore.properties` won't work in SolrCloud mode (it is not read from ZooKeeper). This feature is likely to be removed in the future. Instead, use another mechanism like a config overlay.
|
||||
====
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
|
||||
The path and name of the `solrcore.properties` file can be overridden using the `properties` property in <<defining-core-properties.adoc#defining-core-properties,`core.properties`>>.
|
||||
|
||||
====
|
||||
|
||||
[[Configuringsolrconfig.xml-Userdefinedpropertiesfromcore.properties]]
|
||||
=== User-Defined Properties in `core.properties`
|
||||
|
||||
Every Solr core has a `core.properties` file, automatically created when using the APIs. When you create a SolrCloud collection, you can pass through custom parameters to go into each core.properties that will be created, by prefixing the parameter name with "property." as a URL parameter. Example:
|
||||
|
||||
http://localhost:8983/solr/admin/collections?action=CREATE&name=gettingstarted&numShards=1&property.my.custom.prop=edismax
|
||||
|
||||
That would create a `core.properties` file that has at least the following properties (others omitted for brevity):
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
#core.properties
|
||||
name=gettingstarted
|
||||
my.custom.prop=edismax
|
||||
----
|
||||
|
||||
The `my.custom.prop` property can then be used as a variable, such as in `solrconfig.xml`:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<requestHandler name="/select">
|
||||
<lst name="defaults">
|
||||
<str name="defType">${my.custom.prop}</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
----
|
||||
|
||||
[[Configuringsolrconfig.xml-ImplicitCoreProperties]]
|
||||
=== Implicit Core Properties
|
||||
|
||||
Several attributes of a Solr core are available as "implicit" properties that can be used in variable substitution, independent of where or how they underlying value is initialized. For example: regardless of whether the name for a particular Solr core is explicitly configured in `core.properties` or inferred from the name of the instance directory, the implicit property `solr.core.name` is available for use as a variable in that core's configuration file...
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<requestHandler name="/select">
|
||||
<lst name="defaults">
|
||||
<str name="collection_name">${solr.core.name}</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
----
|
||||
|
||||
All implicit properties use the `solr.core.` name prefix, and reflect the runtime value of the equivalent <<defining-core-properties.adoc#defining-core-properties,`core.properties` property>>:
|
||||
|
||||
* `solr.core.name`
|
||||
* `solr.core.config`
|
||||
* `solr.core.schema`
|
||||
* `solr.core.dataDir`
|
||||
* `solr.core.transient`
|
||||
* `solr.core.loadOnStartup`
|
|
@ -0,0 +1,48 @@
|
|||
= Content Streams
|
||||
:page-shortname: content-streams
|
||||
:page-permalink: content-streams.html
|
||||
|
||||
Content streams are bulk data passed with a request to Solr.
|
||||
|
||||
When Solr RequestHandlers are accessed using path based URLs, the `SolrQueryRequest` object containing the parameters of the request may also contain a list of ContentStreams containing bulk data for the request. (The name SolrQueryRequest is a bit misleading: it is involved in all requests, regardless of whether it is a query request or an update request.)
|
||||
|
||||
[[ContentStreams-StreamSources]]
|
||||
== Stream Sources
|
||||
|
||||
Currently request handlers can get content streams in a variety of ways:
|
||||
|
||||
* For multipart file uploads, each file is passed as a stream.
|
||||
* For POST requests where the content-type is not `application/x-www-form-urlencoded`, the raw POST body is passed as a stream. The full POST body is parsed as parameters and included in the Solr parameters.
|
||||
* The contents of parameter `stream.body` is passed as a stream.
|
||||
* If remote streaming is enabled and URL content is called for during request handling, the contents of each `stream.url` and `stream.file` parameters are fetched and passed as a stream.
|
||||
|
||||
By default, curl sends a `contentType="application/x-www-form-urlencoded"` header. If you need to test a SolrContentHeader content stream, you will need to set the content type with curl's `-H` flag.
|
||||
|
||||
[[ContentStreams-RemoteStreaming]]
|
||||
== RemoteStreaming
|
||||
|
||||
Remote streaming lets you send the contents of a URL as a stream to a given SolrRequestHandler. You could use remote streaming to send a remote or local file to an update plugin.
|
||||
|
||||
For convenience, remote streaming is enabled in most of the example `solrconfig.xml` files included with Solr, however it is not recommended in a production situation without additional security between you and untrusted remote clients.
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<!-- *** WARNING ***
|
||||
The settings below authorize Solr to fetch remote files, You
|
||||
should make sure your system has some authentication before
|
||||
using enableRemoteStreaming="true"
|
||||
-->
|
||||
<requestParsers enableRemoteStreaming="true" />
|
||||
----
|
||||
|
||||
The default behavior, when `enableRemoteStreaming` is not specified in `solrconfig.xml` is to _not_ allow remote streaming (i.e., `enableRemoteStreaming="false"`).
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
If you `enableRemoteStreaming="true"` is used, be aware that this allows _anyone_ to send a request to any URL or local file. If <<ContentStreams-DebuggingRequests,DumpRequestHandler>> is enabled, it will allow anyone to view any file on your system.
|
||||
====
|
||||
|
||||
[[ContentStreams-DebuggingRequests]]
|
||||
== Debugging Requests
|
||||
|
||||
The implicit "dump" RequestHandler (see <<implicit-requesthandlers.adoc#implicit-requesthandlers,Implicit RequestHandlers>>) simply outputs the contents of the SolrQueryRequest using the specified writer type `wt`. This is a useful tool to help understand what streams are available to the RequestHandlers.
|
|
@ -0,0 +1,40 @@
|
|||
= Copying Fields
|
||||
:page-shortname: copying-fields
|
||||
:page-permalink: copying-fields.html
|
||||
|
||||
You might want to interpret some document fields in more than one way. Solr has a mechanism for making copies of fields so that you can apply several distinct field types to a single piece of incoming information.
|
||||
|
||||
The name of the field you want to copy is the _source_, and the name of the copy is the _destination_. In `schema.xml`, it's very simple to make copies of fields:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<copyField source="cat" dest="text" maxChars="30000" />
|
||||
----
|
||||
|
||||
In this example, we want Solr to copy the `cat` field to a field named `text`. Fields are copied before <<understanding-analyzers-tokenizers-and-filters.adoc#understanding-analyzers-tokenizers-and-filters,analysis>> is done, meaning you can have two fields with identical original content, but which use different analysis chains and are stored in the index differently.
|
||||
|
||||
In the example above, if the `text` destination field has data of its own in the input documents, the contents of the `cat` field will be added as additional values – just as if all of the values had originally been specified by the client. Remember to configure your fields as `multivalued="true"` if they will ultimately get multiple values (either from a multivalued source or from multiple `copyField` directives).
|
||||
|
||||
A common usage for this functionality is to create a single "search" field that will serve as the default query field when users or clients do not specify a field to query. For example, `title`, `author`, `keywords`, and `body` may all be fields that should be searched by default, with copy field rules for each field to copy to a `catchall` field (for example, it could be named anything). Later you can set a rule in `solrconfig.xml` to search the `catchall` field by default. One caveat to this is your index will grow when using copy fields. However, whether this becomes problematic for you and the final size will depend on the number of fields being copied, the number of destination fields being copied to, the analysis in use, and the available disk space.
|
||||
|
||||
The `maxChars` parameter, an `int` parameter, establishes an upper limit for the number of characters to be copied from the source value when constructing the value added to the destination field. This limit is useful for situations in which you want to copy some data from the source field, but also control the size of index files.
|
||||
|
||||
Both the source and the destination of `copyField` can contain either leading or trailing asterisks, which will match anything. For example, the following line will copy the contents of all incoming fields that match the wildcard pattern `*_t` to the text field.:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<copyField source="*_t" dest="text" maxChars="25000" />
|
||||
----
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
The `copyField` command can use a wildcard (*) character in the `dest` parameter only if the `source` parameter contains one as well. `copyField` uses the matching glob from the source field for the `dest` field name into which the source content is copied.
|
||||
====
|
||||
|
||||
Copying is done at the stream source level and no copy feeds into another copy. This means that copy fields cannot be chained i.e. _you cannot_ copy from `here` to `there` and then from `there` to `elsewhere`. However, the same source field can be copied to multiple destination fields:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<copyField source="here" dest="there"/>
|
||||
<copyField source="here" dest="elsewhere"/>
|
||||
----
|
|
@ -0,0 +1,36 @@
|
|||
= Core-Specific Tools
|
||||
:page-shortname: core-specific-tools
|
||||
:page-permalink: core-specific-tools.html
|
||||
:page-children: ping, plugins-stats-screen, replication-screen, segments-info
|
||||
|
||||
The Core-Specific tools are a group of UI screens that allow you to see core-level information.
|
||||
|
||||
In the left-hand navigation bar, you will see a pull-down menu titled "Core Selector". Clicking on the menu will show a list of Solr cores hosted on this Solr node, with a search box that can be used to find a specific core by name.
|
||||
|
||||
When you select a core from the pull-down, the main display of the page will show some basic metadata about the core, and a secondary menu will appear in the left nav with links to additional core specific administration screens.
|
||||
|
||||
You can also define a configuration file called `admin-extra.html` that includes links or other information you would like to display in the "Admin Extra" part of this main screen.
|
||||
|
||||
.Core overview screen
|
||||
image::images/core-specific-tools/core_dashboard.png[image,width=515,height=250]
|
||||
|
||||
The core-specific UI screens are listed below, with a link to the section of this guide to find out more:
|
||||
|
||||
// TODO: SOLR-10655 BEGIN: refactor this into a 'core-screens-list.include.adoc' file for reuse
|
||||
* <<ping.adoc#ping,Ping>> - lets you ping a named core and determine whether the core is active.
|
||||
* <<plugins-stats-screen.adoc#plugins-stats-screen,Plugins/Stats>> - shows statistics for plugins and other installed components.
|
||||
* <<replication-screen.adoc#replication-screen,Replication>> - shows you the current replication status for the core, and lets you enable/disable replication.
|
||||
* <<segments-info.adoc#segments-info,Segments Info>> - Provides a visualization of the underlying Lucene index segments.
|
||||
// TODO: SOLR-10655 END
|
||||
|
||||
If you are running a single node instance of Solr, additional UI screens normally displayed on a per-collection bases will also be listed:
|
||||
|
||||
// TODO: SOLR-10655 BEGIN: refactor this into a 'collection-screens-list.include.adoc' file for reuse
|
||||
* <<analysis-screen.adoc#analysis-screen,Analysis>> - lets you analyze the data found in specific fields.
|
||||
* <<dataimport-screen.adoc#dataimport-screen,Dataimport>> - shows you information about the current status of the Data Import Handler.
|
||||
* <<documents-screen.adoc#documents-screen,Documents>> - provides a simple form allowing you to execute various Solr indexing commands directly from the browser.
|
||||
* <<files-screen.adoc#files-screen,Files>> - shows the current core configuration files such as `solrconfig.xml`.
|
||||
* <<query-screen.adoc#query-screen,Query>> - lets you submit a structured query about various elements of a core.
|
||||
* <<stream-screen.adoc#stream-screen,Stream>> - allows you to submit streaming expressions and see results and parsing explanations.
|
||||
* <<schema-browser-screen.adoc#schema-browser-screen,Schema Browser>> - displays schema data in a browser window.
|
||||
// TODO: SOLR-10655 END
|
|
@ -0,0 +1,353 @@
|
|||
= CoreAdmin API
|
||||
:page-shortname: coreadmin-api
|
||||
:page-permalink: coreadmin-api.html
|
||||
|
||||
The Core Admin API is primarily used under the covers by the <<collections-api.adoc#collections-api,Collections API>> when running a <<solrcloud.adoc#solrcloud,SolrCloud>> cluster.
|
||||
|
||||
SolrCloud users should not typically use the CoreAdmin API directly, but the API may be useful for users of single-node or master/slave Solr installations for core maintenance operations.
|
||||
|
||||
The CoreAdmin API is implemented by the CoreAdminHandler, which is a special purpose <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,request handler>> that is used to manage Solr cores. Unlike other request handlers, the CoreAdminHandler is not attached to a single core. Instead, there is a single instance of the CoreAdminHandler in each Solr node that manages all the cores running in that node and is accessible at the `/solr/admin/cores` path.
|
||||
|
||||
CoreAdmin actions can be executed by via HTTP requests that specify an `action` request parameter, with additional action specific arguments provided as additional parameters.
|
||||
|
||||
All action names are uppercase, and are defined in depth in the sections below.
|
||||
|
||||
[[CoreAdminAPI-STATUS]]
|
||||
== STATUS
|
||||
|
||||
The `STATUS` action returns the status of all running Solr cores, or status for only the named core.
|
||||
|
||||
`admin/cores?action=STATUS&core=_core-name_`
|
||||
|
||||
[[CoreAdminAPI-Input]]
|
||||
=== *Input*
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|core |string |No | |The name of a core, as listed in the "name" attribute of a `<core>` element in `solr.xml`.
|
||||
|indexInfo |boolean |No |true |If **false**, information about the index will not be returned with a core STATUS request. In Solr implementations with a large number of cores (i.e., more than hundreds), retrieving the index information for each core can take a lot of time and isn't always required.
|
||||
|===
|
||||
|
||||
[[CoreAdminAPI-CREATE]]
|
||||
== CREATE
|
||||
|
||||
The `CREATE` action creates a new core and registers it.
|
||||
|
||||
If a Solr core with the given name already exists, it will continue to handle requests while the new core is initializing. When the new core is ready, it will take new requests and the old core will be unloaded.
|
||||
|
||||
`admin/cores?action=CREATE&name=_core-name_&instanceDir=_path/to/dir_&config=solrconfig.xml&dataDir=data`
|
||||
|
||||
Note that this command is the only one of the Core Admin API commands that *does not* support the `core` parameter. Instead, the `name` parameter is required, as shown below.
|
||||
|
||||
.CREATE must be able to find a configuration!
|
||||
[WARNING]
|
||||
====
|
||||
Your CREATE call must be able to find a configuration, or it will not succeed.
|
||||
|
||||
When you are running SolrCloud and create a new core for a collection, the configuration will be inherited from the collection. Each collection is linked to a configName, which is stored in the ZooKeeper database. This satisfies the config requirement. There is something to note, though – if you're running SolrCloud, you should *NOT* be using the CoreAdmin API at all. Use the Collections API.
|
||||
|
||||
When you are not running SolrCloud, if you have <<config-sets.adoc#config-sets,Config Sets>> defined, you can use the configSet parameter as documented below. If there are no config sets, then the instanceDir specified in the CREATE call must already exist, and it must contain a `conf` directory which in turn must contain `solrconfig.xml`, your schema, which is usually named either `managed-schema` or `schema.xml`, and any files referenced by those configs.
|
||||
|
||||
The config and schema filenames can be specified with the config and schema parameters, but these are expert options. One thing you could do to avoid creating the conf directory is use config and schema parameters that point at absolute paths, but this can lead to confusing configurations unless you fully understand what you are doing.
|
||||
====
|
||||
|
||||
.CREATE and the core.properties file
|
||||
[IMPORTANT]
|
||||
====
|
||||
The core.properties file is built as part of the CREATE command. If you create a core.properties file yourself in a core directory and then try to use CREATE to add that core to Solr, you will get an error telling you that another core is already defined there. The core.properties file must NOT exist before calling the CoreAdmin API with the CREATE command.
|
||||
====
|
||||
|
||||
[[CoreAdminAPI-Input.1]]
|
||||
=== *Input*
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|name |string |Yes |N/A |The name of the new core. Same as "name" on the `<core>` element.
|
||||
|instanceDir |string |No |The value specified for "name" parameter |The directory where files for this SolrCore should be stored. Same as `instanceDir` on the `<core>` element.
|
||||
|config |string |No | |Name of the config file (i.e., `solrconfig.xml`) relative to `instanceDir`.
|
||||
|schema |string |No | |Name of the schema file to use for the core. Please note that if you are using a "managed schema" (the default behavior) then any value for this property which does not match the effective `managedSchemaResourceName` will be read once, backed up, and converted for managed schema use. See <<schema-factory-definition-in-solrconfig.adoc#schema-factory-definition-in-solrconfig,Schema Factory Definition in SolrConfig>> for details.
|
||||
|dataDir |string |No | |Name of the data directory relative to `instanceDir`.
|
||||
|configSet |string |No | |Name of the configset to use for this core. For more information, see the section <<config-sets.adoc#config-sets,Config Sets>>.
|
||||
|collection |string |No | |The name of the collection to which this core belongs. The default is the name of the core. `collection.<param>=<value>` causes a property of `<param>=<value>` to be set if a new collection is being created. Use `collection.configName=<configname>` to point to the configuration for a new collection.
|
||||
|shard |string |No | |The shard id this core represents. Normally you want to be auto-assigned a shard id.
|
||||
|property.__name__=__value__ |string |No | |Sets the core property _name_ to __value__. See the section on defining <<defining-core-properties.adoc#Definingcore.properties-core.properties_files,core.properties file contents>>.
|
||||
|async |string |No | |Request ID to track this action which will be processed asynchronously
|
||||
|===
|
||||
|
||||
Use `collection.configName=<configname>` to point to the config for a new collection.
|
||||
|
||||
[[CoreAdminAPI-Example]]
|
||||
=== Example
|
||||
|
||||
`\http://localhost:8983/solr/admin/cores?action=CREATE&name=my_core&collection=my_collection&shard=shard2`
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
While it's possible to create a core for a non-existent collection, this approach is not supported and not recommended. Always create a collection using the <<collections-api.adoc#collections-api,Collections API>> before creating a core directly for it.
|
||||
====
|
||||
|
||||
[[CoreAdminAPI-RELOAD]]
|
||||
== RELOAD
|
||||
|
||||
The RELOAD action loads a new core from the configuration of an existing, registered Solr core. While the new core is initializing, the existing one will continue to handle requests. When the new Solr core is ready, it takes over and the old core is unloaded.
|
||||
|
||||
`admin/cores?action=RELOAD&core=_core-name_`
|
||||
|
||||
This is useful when you've made changes to a Solr core's configuration on disk, such as adding new field definitions. Calling the RELOAD action lets you apply the new configuration without having to restart the Web container.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
RELOAD performs "live" reloads of SolrCore, reusing some existing objects. Some configuration options, such as the `dataDir` location and `IndexWriter`-related settings in `solrconfig.xml` can not be changed and made active with a simple RELOAD action.
|
||||
====
|
||||
|
||||
[[CoreAdminAPI-Input.2]]
|
||||
=== Input
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|core |string |Yes |N/A |The name of the core, as listed in the "name" attribute of a `<core>` element in `solr.xml`.
|
||||
|===
|
||||
|
||||
[[CoreAdminAPI-RENAME]]
|
||||
== RENAME
|
||||
|
||||
The `RENAME` action changes the name of a Solr core.
|
||||
|
||||
`admin/cores?action=RENAME&core=_core-name_&other=_other-core-name_`
|
||||
|
||||
[[CoreAdminAPI-Input.3]]
|
||||
=== Input
|
||||
|
||||
**Query Parameters**
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|core |string |Yes | |The name of the Solr core to be renamed.
|
||||
|other |string |Yes | |The new name for the Solr core. If the persistent attribute of `<solr>` is `true`, the new name will be written to `solr.xml` as the `name` attribute of the `<core>` attribute.
|
||||
|async |string |No | |Request ID to track this action which will be processed asynchronously
|
||||
|===
|
||||
|
||||
[[CoreAdminAPI-SWAP]]
|
||||
== SWAP
|
||||
|
||||
`SWAP` atomically swaps the names used to access two existing Solr cores. This can be used to swap new content into production. The prior core remains available and can be swapped back, if necessary. Each core will be known by the name of the other, after the swap.
|
||||
|
||||
`admin/cores?action=SWAP&core=_core-name_&other=_other-core-name_`
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
|
||||
Do not use `SWAP` with a SolrCloud node. It is not supported and can result in the core being unusable.
|
||||
|
||||
====
|
||||
|
||||
[[CoreAdminAPI-Input.4]]
|
||||
=== Input
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|core |string |Yes | |The name of one of the cores to be swapped.
|
||||
|other |string |Yes | |The name of one of the cores to be swapped.
|
||||
|async |string |No | |Request ID to track this action which will be processed asynchronously
|
||||
|===
|
||||
|
||||
[[CoreAdminAPI-UNLOAD]]
|
||||
== UNLOAD
|
||||
|
||||
The `UNLOAD` action removes a core from Solr. Active requests will continue to be processed, but no new requests will be sent to the named core. If a core is registered under more than one name, only the given name is removed.
|
||||
|
||||
`admin/cores?action=UNLOAD&core=_core-name_`
|
||||
|
||||
The `UNLOAD` action requires a parameter (`core`) identifying the core to be removed. If the persistent attribute of `<solr>` is set to `true`, the `<core>` element with this `name` attribute will be removed from `solr.xml`.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
Unloading all cores in a SolrCloud collection causes the removal of that collection's metadata from ZooKeeper.
|
||||
====
|
||||
|
||||
[[CoreAdminAPI-Input.5]]
|
||||
=== Input
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|core |string |Yes | |The name of one of the cores to be removed.
|
||||
|deleteIndex |boolean |No |false |If true, will remove the index when unloading the core.
|
||||
|deleteDataDir |boolean |No |false |If true, removes the `data` directory and all sub-directories.
|
||||
|deleteInstanceDir |boolean |No |false |If true, removes everything related to the core, including the index directory, configuration files and other related files.
|
||||
|async |string |No | |Request ID to track this action which will be processed asynchronously
|
||||
|===
|
||||
|
||||
[[CoreAdminAPI-MERGEINDEXES]]
|
||||
== MERGEINDEXES
|
||||
|
||||
The `MERGEINDEXES` action merges one or more indexes to another index. The indexes must have completed commits, and should be locked against writes until the merge is complete or the resulting merged index may become corrupted. The target core index must already exist and have a compatible schema with the one or more indexes that will be merged to it. Another commit on the target core should also be performed after the merge is complete.
|
||||
|
||||
`admin/cores?action=MERGEINDEXES&core=_new-core-name_&indexDir=_path/to/core1/data/index_&indexDir=_path/to/core2/data/index_`
|
||||
|
||||
In this example, we use the `indexDir` parameter to define the index locations of the source cores. The `core` parameter defines the target index. A benefit of this approach is that we can merge any Lucene-based index that may not be associated with a Solr core.
|
||||
|
||||
Alternatively, we can instead use a `srcCore` parameter, as in this example:
|
||||
|
||||
`admin/cores?action=mergeindexes&core=_new-core-name_&srcCore=_core1-name_&srcCore=_core2-name_`
|
||||
|
||||
This approach allows us to define cores that may not have an index path that is on the same physical server as the target core. However, we can only use Solr cores as the source indexes. Another benefit of this approach is that we don't have as high a risk for corruption if writes occur in parallel with the source index.
|
||||
|
||||
We can make this call run asynchronously by specifying the `async` parameter and passing a request-id. This id can then be used to check the status of the already submitted task using the REQUESTSTATUS API.
|
||||
|
||||
[[CoreAdminAPI-Input.6]]
|
||||
=== Input
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|core |string |Yes | |The name of the target core/index.
|
||||
|indexDir |string | | |Multi-valued, directories that would be merged.
|
||||
|srcCore |string | | |Multi-valued, source cores that would be merged.
|
||||
|async |string | | |Request ID to track this action which will be processed asynchronously
|
||||
|===
|
||||
|
||||
[[CoreAdminAPI-SPLIT]]
|
||||
== SPLIT
|
||||
|
||||
The `SPLIT` action splits an index into two or more indexes. The index being split can continue to handle requests. The split pieces can be placed into a specified directory on the server's filesystem or it can be merged into running Solr cores.
|
||||
|
||||
The `SPLIT` action supports five parameters, which are described in the table below.
|
||||
|
||||
[[CoreAdminAPI-Input.7]]
|
||||
=== Input
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|core |string |Yes | |The name of the core to be split.
|
||||
|path |string | | |Multi-valued, the directory path in which a piece of the index will be written.
|
||||
|targetCore |string | | |Multi-valued, the target Solr core to which a piece of the index will be merged
|
||||
|ranges |string |No | |A comma-separated list of hash ranges in hexadecimal format
|
||||
|split.key |string |No | |The key to be used for splitting the index
|
||||
|async |string |No | |Request ID to track this action which will be processed asynchronously
|
||||
|===
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
Either `path` or `targetCore` parameter must be specified but not both. The ranges and split.key parameters are optional and only one of the two should be specified, if at all required.
|
||||
====
|
||||
|
||||
[[CoreAdminAPI-Examples]]
|
||||
=== Examples
|
||||
|
||||
The `core` index will be split into as many pieces as the number of `path` or `targetCore` parameters.
|
||||
|
||||
==== Usage with two `targetCore` parameters:
|
||||
|
||||
`\http://localhost:8983/solr/admin/cores?action=SPLIT&core=core0&targetCore=core1&targetCore=core2`
|
||||
|
||||
Here the `core` index will be split into two pieces and merged into the two `targetCore` indexes.
|
||||
|
||||
==== Usage with two `path` parameters:
|
||||
|
||||
`\http://localhost:8983/solr/admin/cores?action=SPLIT&core=core0&path=/path/to/index/1&path=/path/to/index/2`
|
||||
|
||||
The `core` index will be split into two pieces and written into the two directory paths specified.
|
||||
|
||||
==== Usage with the `split.key` parameter:
|
||||
|
||||
`\http://localhost:8983/solr/admin/cores?action=SPLIT&core=core0&targetCore=core1&split.key=A!`
|
||||
|
||||
Here all documents having the same route key as the `split.key` i.e. 'A!' will be split from the `core` index and written to the `targetCore`.
|
||||
|
||||
==== Usage with `ranges` parameter:
|
||||
|
||||
`\http://localhost:8983/solr/admin/cores?action=SPLIT&core=core0&targetCore=core1&targetCore=core2&targetCore=core3&ranges=0-1f4,1f5-3e8,3e9-5dc`
|
||||
|
||||
This example uses the `ranges` parameter with hash ranges 0-500, 501-1000 and 1001-1500 specified in hexadecimal. Here the index will be split into three pieces with each targetCore receiving documents matching the hash ranges specified i.e. core1 will get documents with hash range 0-500, core2 will receive documents with hash range 501-1000 and finally, core3 will receive documents with hash range 1001-1500. At least one hash range must be specified. Please note that using a single hash range equal to a route key's hash range is NOT equivalent to using the `split.key` parameter because multiple route keys can hash to the same range.
|
||||
|
||||
The `targetCore` must already exist and must have a compatible schema with the `core` index. A commit is automatically called on the `core` index before it is split.
|
||||
|
||||
This command is used as part of the <<collections-api.adoc#CollectionsAPI-splitshard,SPLITSHARD>> command but it can be used for non-cloud Solr cores as well. When used against a non-cloud core without `split.key` parameter, this action will split the source index and distribute its documents alternately so that each split piece contains an equal number of documents. If the `split.key` parameter is specified then only documents having the same route key will be split from the source index.
|
||||
|
||||
[[CoreAdminAPI-REQUESTSTATUS]]
|
||||
== REQUESTSTATUS
|
||||
|
||||
Request the status of an already submitted asynchronous CoreAdmin API call.
|
||||
|
||||
`admin/cores?action=REQUESTSTATUS&requestid=_id_`
|
||||
|
||||
[[CoreAdminAPI-Input.8]]
|
||||
=== Input
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|requestid |string |Yes | |The user defined request-id for the Asynchronous request.
|
||||
|===
|
||||
|
||||
The call below will return the status of an already submitted Asynchronous CoreAdmin call.
|
||||
|
||||
`\http://localhost:8983/solr/admin/cores?action=REQUESTSTATUS&requestid=1`
|
||||
|
||||
[[CoreAdminAPI-REQUESTRECOVERY]]
|
||||
== REQUESTRECOVERY
|
||||
|
||||
The `REQUESTRECOVERY` action manually asks a core to recover by synching with the leader. This should be considered an "expert" level command and should be used in situations where the node (SorlCloud replica) is unable to become active automatically.
|
||||
|
||||
`admin/cores?action=REQUESTRECOVERY&core=_core-name_`
|
||||
|
||||
[[CoreAdminAPI-Input.9]]
|
||||
=== Input
|
||||
|
||||
*Query Parameters*
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="15,10,10,10,55",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Required |Default |Description
|
||||
|core |string |Yes | |The name of the core to re-sync.
|
||||
|===
|
||||
|
||||
[[CoreAdminAPI-Examples.1]]
|
||||
=== Examples
|
||||
|
||||
`\http://localhost:8981/solr/admin/cores?action=REQUESTRECOVERY&core=gettingstarted_shard1_replica1`
|
||||
|
||||
The core to specify can be found by expanding the appropriate ZooKeeper node via the admin UI.
|
|
@ -0,0 +1,761 @@
|
|||
= Cross Data Center Replication (CDCR)
|
||||
:page-shortname: cross-data-center-replication-cdcr
|
||||
:page-permalink: cross-data-center-replication-cdcr.html
|
||||
|
||||
Cross Data Center Replication (CDCR) allows you to create multiple SolrCloud data centers and keep them in sync in case they are needed at a future time.
|
||||
|
||||
The <<solrcloud.adoc#solrcloud,SolrCloud>> architecture is not particularly well suited for situations where a single SolrCloud cluster consists of nodes in separated data clusters connected by an expensive pipe. The root problem is that SolrCloud is designed to support <<near-real-time-searching.adoc#near-real-time-searching,Near Real Time Searching>> by immediately forwarding updates between nodes in the cluster on a per-shard basis. "CDCR" features exist to help mitigate the risk of an entire data center outage.
|
||||
|
||||
== What is CDCR?
|
||||
|
||||
CDCR supports replicating data from one data center to multiple data centers. The initial version of the solution supports an active-passive scenario where data updates are replicated from a Source data center to one or more target data centers.
|
||||
|
||||
The target data center(s) will not propagate updates such as adds, updates, or deletes to the source data center and updates should _not_ be sent to any of the target data center(s).
|
||||
|
||||
Source and target data centers can serve search queries when CDCR is operating. The target data centers will have slightly stale views of the corpus due to propagation delays, but this is minimal (perhaps a few seconds).
|
||||
|
||||
Data changes on the source data center are replicated to the target data center only after they are persisted to disk. The data changes can be replicated in near real-time (with a small delay) or could be scheduled to be sent in intervals to the target data center. This solution pre-supposes that the source and target data centers begin with the same documents indexed. Of course the indexes may be empty to start.
|
||||
|
||||
Each shard leader in the source data center will be responsible for replicating its updates to the corresponding leader in the target data center. When receiving updates from the source data center, shard leaders in the target data center will replicate the changes to their own replicas.
|
||||
|
||||
This replication model is designed to tolerate some degradation in connectivity, accommodate limited bandwidth, and support batch updates to optimize communication.
|
||||
|
||||
Replication supports both a new empty index and pre-built indexes. In the scenario where the replication is set up on a pre-built index, CDCR will ensure consistency of the replication of the updates, but cannot ensure consistency on the full index. Therefore any index created before CDCR was set up will have to be replicated by other means (described in the section <<Initial Startup>>) so source and target indexes are fully consistent.
|
||||
|
||||
The active-passive nature of the initial implementation implies a "push" model from the source collection to the target collection. Therefore, the source configuration must be able to "see" the ZooKeeper ensemble in the target cluster. The ZooKeeper ensemble is provided configured in the Source's `solrconfig.xml` file.
|
||||
|
||||
CDCR is configured to replicate from collections in the source cluster to collections in the target cluster on a collection-by-collection basis. Since CDCR is configured in `solrconfig.xml` (on both source and target clusters), the settings can be tailored for the needs of each collection.
|
||||
|
||||
CDCR can be configured to replicate from one collection to a second collection _within the same cluster_. That is a specialized scenario not covered in this document.
|
||||
|
||||
[glossary]
|
||||
== CDCR Glossary
|
||||
|
||||
Terms used in this document include:
|
||||
|
||||
[glossary]
|
||||
Node:: A JVM instance running Solr; a server.
|
||||
Cluster:: A set of Solr nodes managed as a single unit by a ZooKeeper ensemble, hosting one or more Collections.
|
||||
Data Center:: A group of networked servers hosting a Solr cluster. In this document, the terms _Cluster_ and _Data Center_ are interchangeable as we assume that each Solr cluster is hosted in a different group of networked servers.
|
||||
Shard:: A sub-index of a single logical collection. This may be spread across multiple nodes of the cluster. Each shard can have as many replicas as needed.
|
||||
Leader:: Each shard has one node identified as its leader. All the writes for documents belonging to a shard are routed through the leader.
|
||||
Replica:: A copy of a shard for use in failover or load balancing. Replicas comprising a shard can either be leaders or non-leaders.
|
||||
Follower:: A convenience term for a replica that is _not_ the leader of a shard.
|
||||
Collection:: Multiple documents that make up one logical index. A cluster can have multiple collections.
|
||||
Updates Log:: An append-only log of write operations maintained by each node.
|
||||
|
||||
== CDCR Architecture
|
||||
|
||||
Here is a picture of the data flow.
|
||||
|
||||
.CDCR Data Flow
|
||||
image::images/cross-data-center-replication-cdcr-/CDCR_arch.png[image,width=700,height=525]
|
||||
|
||||
Updates and deletes are first written to the Source cluster, then forwarded to the Target cluster. The data flow sequence is:
|
||||
|
||||
. A shard leader receives a new data update that is processed by its update processor chain.
|
||||
. The data update is first applied to the local index.
|
||||
. Upon successful application of the data update on the local index, the data update is added to the Updates Log queue.
|
||||
. After the data update is persisted to disk, the data update is sent to the replicas within the data center.
|
||||
. After Step 4 is successful, CDCR reads the data update from the Updates Log and pushes it to the corresponding collection in the target data center. This is necessary in order to ensure consistency between the Source and target data centers.
|
||||
. The leader on the target data center writes the data locally and forwards it to all its followers.
|
||||
|
||||
Steps 1, 2, 3 and 4 are performed synchronously by SolrCloud; Step 5 is performed asynchronously by a background thread. Given that CDCR replication is performed asynchronously, it becomes possible to push batch updates in order to minimize network communication overhead. Also, if CDCR is unable to push the update at a given time, for example, due to a degradation in connectivity, it can retry later without any impact on the source data center.
|
||||
|
||||
One implication of the architecture is that the leaders in the source cluster must be able to "see" the leaders in the target cluster. Since leaders may change, this effectively means that all nodes in the source cluster must be able to "see" all Solr nodes in the target cluster so firewalls, ACL rules, etc. must be configured with care.
|
||||
|
||||
The current design works most robustly if both the Source and target clusters have the same number of shards. There is no requirement that the shards in the Source and target collection have the same number of replicas.
|
||||
|
||||
Having different numbers of shards on the Source and target cluster is possible, but is also an "expert" configuration as that option imposes certain constraints and is not recommended. Most of the scenarios where having differing numbers of shards are contemplated are better accomplished by hosting multiple shards on each target Solr instance.
|
||||
|
||||
== Major Components of CDCR
|
||||
|
||||
There are a number of key features and components in CDCR’s architecture:
|
||||
|
||||
=== CDCR Configuration
|
||||
|
||||
In order to configure CDCR, the Source data center requires the host address of the ZooKeeper cluster associated with the target data center. The ZooKeeper host address is the only information needed by CDCR to instantiate the communication with the target Solr cluster. The CDCR configuration file on the source cluster will therefore contain a list of ZooKeeper hosts. The CDCR configuration file might also contain secondary/optional configuration, such as the number of CDC Replicator threads, batch updates related settings, etc.
|
||||
|
||||
=== CDCR Initialization
|
||||
|
||||
CDCR supports incremental updates to either new or existing collections. CDCR may not be able to keep up with very high volume updates, especially if there are significant communications latencies due to a slow "pipe" between the data centers. Some scenarios:
|
||||
|
||||
* There is an initial bulk load of a corpus followed by lower volume incremental updates. In this case, one can do the initial bulk load and then enable CDCR. See the section <<Initial Startup>> for more information.
|
||||
* The index is being built up from scratch, without a significant initial bulk load. CDCR can be set up on empty collections and keep them synchronized from the start.
|
||||
* The index is always being updated at a volume too high for CDCR to keep up. This is especially possible in situations where the connection between the Source and target data centers is poor. This scenario is unsuitable for CDCR in its current form.
|
||||
|
||||
=== Inter-Data Center Communication
|
||||
|
||||
Communication between data centers will be achieved through HTTP and the Solr REST API using the SolrJ client. The SolrJ client will be instantiated with the ZooKeeper host of the target data center. SolrJ will manage the shard leader discovery process.
|
||||
|
||||
=== Updates Tracking & Pushing
|
||||
|
||||
CDCR replicates data updates from the source to the target data center by leveraging the Updates Log.
|
||||
|
||||
A background thread regularly checks the Updates Log for new entries, and then forwards them to the target data center. The thread therefore needs to keep a checkpoint in the form of a pointer to the last update successfully processed in the Updates Log. Upon acknowledgement from the target data center that updates have been successfully processed, the Updates Log pointer is updated to reflect the current checkpoint.
|
||||
|
||||
This pointer must be synchronized across all the replicas. In the case where the leader goes down and a new leader is elected, the new leader will be able to resume replication from the last update by using this synchronized pointer. The strategy to synchronize such a pointer across replicas will be explained next.
|
||||
|
||||
If for some reason, the target data center is offline or fails to process the updates, the thread will periodically try to contact the target data center and push the updates.
|
||||
|
||||
=== Synchronization of Update Checkpoints
|
||||
|
||||
A reliable synchronization of the update checkpoints between the shard leader and shard replicas is critical to avoid introducing inconsistency between the Source and target data centers. Another important requirement is that the synchronization must be performed with minimal network traffic to maximize scalability.
|
||||
|
||||
In order to achieve this, the strategy is to:
|
||||
|
||||
* Uniquely identify each update operation. This unique identifier will serve as pointer.
|
||||
* Rely on two storages: an ephemeral storage on the Source shard leader, and a persistent storage on the target cluster.
|
||||
|
||||
The shard leader in the source cluster will be in charge of generating a unique identifier for each update operation, and will keep a copy of the identifier of the last processed updates in memory. The identifier will be sent to the target cluster as part of the update request. On the target data center side, the shard leader will receive the update request, store it along with the unique identifier in the Updates Log, and replicate it to the other shards.
|
||||
|
||||
SolrCloud already provides a unique identifier for each update operation, i.e., a “version” number. This version number is generated using a time-based lmport clock which is incremented for each update operation sent. This provides an “happened-before” ordering of the update operations that will be leveraged in (1) the initialization of the update checkpoint on the source cluster, and in (2) the maintenance strategy of the Updates Log.
|
||||
|
||||
The persistent storage on the target cluster is used only during the election of a new shard leader on the Source cluster. If a shard leader goes down on the source cluster and a new leader is elected, the new leader will contact the target cluster to retrieve the last update checkpoint and instantiate its ephemeral pointer. On such a request, the target cluster will retrieve the latest identifier received across all the shards, and send it back to the source cluster. To retrieve the latest identifier, every shard leader will look up the identifier of the first entry in its Update Logs and send it back to a coordinator. The coordinator will have to select the highest among them.
|
||||
|
||||
This strategy does not require any additional network traffic and ensures reliable pointer synchronization. Consistency is principally achieved by leveraging SolrCloud. The update workflow of SolrCloud ensures that every update is applied to the leader but also to any of the replicas. If the leader goes down, a new leader is elected. During the leader election, a synchronization is performed between the new leader and the other replicas. As a result, this ensures that the new leader has a consistent Update Logs with the previous leader. Having a consistent Updates Log means that:
|
||||
|
||||
* On the source cluster, the update checkpoint can be reused by the new leader.
|
||||
* On the target cluster, the update checkpoint will be consistent between the previous and new leader. This ensures the correctness of the update checkpoint sent by a newly elected leader from the target cluster.
|
||||
|
||||
=== Maintenance of Updates Log
|
||||
|
||||
The CDCR replication logic requires modification to the maintenance logic of the Updates Log on the source data center. Initially, the Updates Log acts as a fixed size queue, limited to 100 update entries. In the CDCR scenario, the Update Logs must act as a queue of variable size as they need to keep track of all the updates up through the last processed update by the target data center. Entries in the Update Logs are removed only when all pointers (one pointer per target data center) are after them.
|
||||
|
||||
If the communication with one of the target data center is slow, the Updates Log on the source data center can grow to a substantial size. In such a scenario, it is necessary for the Updates Log to be able to efficiently find a given update operation given its identifier. Given that its identifier is an incremental number, it is possible to implement an efficient search strategy. Each transaction log file contains as part of its filename the version number of the first element. This is used to quickly traverse all the transaction log files and find the transaction log file containing one specific version number.
|
||||
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-Monitoring]]
|
||||
=== Monitoring
|
||||
|
||||
CDCR provides the following monitoring capabilities over the replication operations:
|
||||
|
||||
* Monitoring of the outgoing and incoming replications, with information such as the Source and target nodes, their status, etc.
|
||||
* Statistics about the replication, with information such as operations (add/delete) per second, number of documents in the queue, etc.
|
||||
|
||||
Information about the lifecycle and statistics will be provided on a per-shard basis by the CDC Replicator thread. The CDCR API can then aggregate this information an a collection level.
|
||||
|
||||
=== CDC Replicator
|
||||
|
||||
The CDC Replicator is a background thread that is responsible for replicating updates from a Source data center to one or more target data centers. It is responsible in providing monitoring information on a per-shard basis. As there can be a large number of collections and shards in a cluster, we will use a fixed-size pool of CDC Replicator threads that will be shared across shards.
|
||||
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-Limitations]]
|
||||
=== Limitations
|
||||
|
||||
The current design of CDCR has some limitations. CDCR will continue to evolve over time and many of these limitations will be addressed. Among them are:
|
||||
|
||||
* CDCR is unlikely to be satisfactory for bulk-load situations where the update rate is high, especially if the bandwidth between the Source and target clusters is restricted. In this scenario, the initial bulk load should be performed, the Source and target data centers synchronized and CDCR be utilized for incremental updates.
|
||||
* CDCR is currently only active-passive; data is pushed from the Source cluster to the target cluster. There is active work being done in this area in the 6x code line to remove this limitation.
|
||||
* CDCR works most robustly with the same number of shards in the Source and target collection. The shards in the two collections may have different numbers of replicas.
|
||||
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-Configuration]]
|
||||
== Configuration
|
||||
|
||||
The source and target configurations differ in the case of the data centers being in separate clusters. "Cluster" here means separate ZooKeeper ensembles controlling disjoint Solr instances. Whether these data centers are physically separated or not is immaterial for this discussion.
|
||||
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-SourceConfiguration]]
|
||||
=== Source Configuration
|
||||
|
||||
Here is a sample of a source configuration file, a section in `solrconfig.xml`. The presence of the <replica> section causes CDCR to use this cluster as the Source and should not be present in the target collections in the cluster-to-cluster case. Details about each setting are after the two examples:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<requestHandler name="/cdcr" class="solr.CdcrRequestHandler">
|
||||
<lst name="replica">
|
||||
<str name="zkHost">10.240.18.211:2181</str>
|
||||
<str name="source">collection1</str>
|
||||
<str name="target">collection1</str>
|
||||
</lst>
|
||||
|
||||
<lst name="replicator">
|
||||
<str name="threadPoolSize">8</str>
|
||||
<str name="schedule">1000</str>
|
||||
<str name="batchSize">128</str>
|
||||
</lst>
|
||||
|
||||
<lst name="updateLogSynchronizer">
|
||||
<str name="schedule">1000</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
|
||||
<!-- Modify the <updateLog> section of your existing <updateHandler>
|
||||
in your config as below -->
|
||||
<updateHandler class="solr.DirectUpdateHandler2">
|
||||
<updateLog class="solr.CdcrUpdateLog">
|
||||
<str name="dir">${solr.ulog.dir:}</str>
|
||||
<!--Any parameters from the original <updateLog> section -->
|
||||
</updateLog>
|
||||
</updateHandler>
|
||||
----
|
||||
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-TargetConfiguration]]
|
||||
=== Target Configuration
|
||||
|
||||
Here is a typical target configuration.
|
||||
|
||||
Target instance must configure an update processor chain that is specific to CDCR. The update processor chain must include the *CdcrUpdateProcessorFactory*. The task of this processor is to ensure that the version numbers attached to update requests coming from a CDCR source SolrCloud are reused and not overwritten by the target. A properly configured Target configuration looks similar to this.
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<requestHandler name="/cdcr" class="solr.CdcrRequestHandler">
|
||||
<lst name="buffer">
|
||||
<str name="defaultState">disabled</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
|
||||
<requestHandler name="/update" class="solr.UpdateRequestHandler">
|
||||
<lst name="defaults">
|
||||
<str name="update.chain">cdcr-processor-chain</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
|
||||
<updateRequestProcessorChain name="cdcr-processor-chain">
|
||||
<processor class="solr.CdcrUpdateProcessorFactory"/>
|
||||
<processor class="solr.RunUpdateProcessorFactory"/>
|
||||
</updateRequestProcessorChain>
|
||||
|
||||
<!-- Modify the <updateLog> section of your existing <updateHandler> in your
|
||||
config as below -->
|
||||
<updateHandler class="solr.DirectUpdateHandler2">
|
||||
<updateLog class="solr.CdcrUpdateLog">
|
||||
<str name="dir">${solr.ulog.dir:}</str>
|
||||
<!--Any parameters from the original <updateLog> section -->
|
||||
</updateLog>
|
||||
</updateHandler>
|
||||
----
|
||||
|
||||
=== Configuration Details
|
||||
|
||||
The configuration details, defaults and options are as follows:
|
||||
|
||||
==== The Replica Element
|
||||
|
||||
CDCR can be configured to forward update requests to one or more replicas. A replica is defined with a “replica” list as follows:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,10,15,55",options="header"]
|
||||
|===
|
||||
|Parameter |Required |Default |Description
|
||||
|zkHost |Yes |none |The host address for ZooKeeper of the target SolrCloud. Usually this is a comma-separated list of addresses to each node in the target ZooKeeper ensemble.
|
||||
|Source |Yes |none |The name of the collection on the Source SolrCloud to be replicated.
|
||||
|Target |Yes |none |The name of the collection on the target SolrCloud to which updates will be forwarded.
|
||||
|===
|
||||
|
||||
==== The Replicator Element
|
||||
|
||||
The CDC Replicator is the component in charge of forwarding updates to the replicas. The replicator will monitor the update logs of the Source collection and will forward any new updates to the target collection.
|
||||
|
||||
The replicator uses a fixed thread pool to forward updates to multiple replicas in parallel. If more than one replica is configured, one thread will forward a batch of updates from one replica at a time in a round-robin fashion. The replicator can be configured with a “replicator” list as follows:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,10,15,55",options="header"]
|
||||
|===
|
||||
|Parameter |Required |Default |Description
|
||||
|threadPoolSize |No |2 |The number of threads to use for forwarding updates. One thread per replica is recommended.
|
||||
|schedule |No |10 |The delay in milliseconds for the monitoring the update log(s).
|
||||
|batchSize |No |128 |The number of updates to send in one batch. The optimal size depends on the size of the documents. Large batches of large documents can increase your memory usage significantly.
|
||||
|===
|
||||
|
||||
==== The updateLogSynchronizer Element
|
||||
|
||||
Expert: Non-leader nodes need to synchronize their update logs with their leader node from time to time in order to clean deprecated transaction log files. By default, such a synchronization process is performed every minute. The schedule of the synchronization can be modified with a “updateLogSynchronizer” list as follows:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,10,15,55",options="header"]
|
||||
|===
|
||||
|Parameter |Required |Default |Description
|
||||
|schedule |No |60000 |The delay in milliseconds for synchronizing the updates log.
|
||||
|===
|
||||
|
||||
==== The Buffer Element
|
||||
|
||||
CDCR is configured by default to buffer any new incoming updates. When buffering updates, the updates log will store all the updates indefinitely. Replicas do not need to buffer updates, and it is recommended to disable buffer on the target SolrCloud. The buffer can be disabled at startup with a “buffer” list and the parameter “defaultState” as follows:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,10,15,55",options="header"]
|
||||
|===
|
||||
|Parameter |Required |Default |Description
|
||||
|defaultState |No |enabled |The state of the buffer at startup.
|
||||
|===
|
||||
|
||||
== CDCR API
|
||||
|
||||
The CDCR API is used to control and monitor the replication process. Control actions are performed at a collection level, i.e., by using the following base URL for API calls: `\http://localhost:8983/solr/<collection>`.
|
||||
|
||||
Monitor actions are performed at a core level, i.e., by using the following base URL for API calls: `\http://localhost:8983/solr/<collection>`.
|
||||
|
||||
Currently, none of the CDCR API calls have parameters.
|
||||
|
||||
|
||||
=== API Entry Points (Control)
|
||||
|
||||
* `<collection>/cdcr?action=STATUS`: <<CrossDataCenterReplication_CDCR_-STATUS,Returns the current state>> of CDCR.
|
||||
* `<collection>/cdcr?action=START`: <<CrossDataCenterReplication_CDCR_-START,Starts CDCR>> replication
|
||||
* `<collection>/cdcr?action=STOP`: <<CrossDataCenterReplication_CDCR_-STOP,Stops CDCR>> replication.
|
||||
* `<collection>/cdcr?action=ENABLEBUFFER`: <<CrossDataCenterReplication_CDCR_-ENABLEBUFFER,Enables the buffering>> of updates.
|
||||
* `<collection>/cdcr?action=DISABLEBUFFER`: <<CrossDataCenterReplication_CDCR_-DISABLEBUFFER,Disables the buffering>> of updates.
|
||||
|
||||
|
||||
=== API Entry Points (Monitoring)
|
||||
|
||||
* `core/cdcr?action=QUEUES`: <<CrossDataCenterReplication_CDCR_-QUEUES,Fetches statistics about the queue>> for each replica and about the update logs.
|
||||
* `core/cdcr?action=OPS`: <<CrossDataCenterReplication_CDCR_-OPS,Fetches statistics about the replication performance>> (operations per second) for each replica.
|
||||
* `core/cdcr?action=ERRORS`: <<CrossDataCenterReplication_CDCR_-ERRORS,Fetches statistics and other information about replication errors>> for each replica.
|
||||
|
||||
=== Control Commands
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-STATUS]]
|
||||
==== STATUS
|
||||
|
||||
`/collection/cdcr?action=STATUS`
|
||||
|
||||
===== Input
|
||||
|
||||
*Query Parameters:* There are no parameters to this command.
|
||||
|
||||
===== Output
|
||||
|
||||
*Output Content*
|
||||
|
||||
The current state of the CDCR, which includes the state of the replication process and the state of the buffer.
|
||||
|
||||
[[cdcr_examples]]
|
||||
===== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://host:8983/solr/<collection_name>/cdcr?action=STATUS
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader": {
|
||||
"status": 0,
|
||||
"QTime": 0
|
||||
},
|
||||
"status": {
|
||||
"process": "stopped",
|
||||
"buffer": "enabled"
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-ENABLEBUFFER]]
|
||||
==== ENABLEBUFFER
|
||||
|
||||
`/collection/cdcr?action=ENABLEBUFFER`
|
||||
|
||||
===== Input
|
||||
|
||||
*Query Parameters:* There are no parameters to this command.
|
||||
|
||||
===== Output
|
||||
|
||||
*Output Content*
|
||||
|
||||
The status of the process and an indication of whether the buffer is enabled
|
||||
|
||||
===== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://host:8983/solr/<collection_name>/cdcr?action=ENABLEBUFFER
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader": {
|
||||
"status": 0,
|
||||
"QTime": 0
|
||||
},
|
||||
"status": {
|
||||
"process": "started",
|
||||
"buffer": "enabled"
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-DISABLEBUFFER]]
|
||||
==== DISABLEBUFFER
|
||||
|
||||
`/collection/cdcr?action=DISABLEBUFFER`
|
||||
|
||||
===== Input
|
||||
|
||||
*Query Parameters:* There are no parameters to this command
|
||||
|
||||
===== Output
|
||||
|
||||
*Output Content:* The status of CDCR and an indication that the buffer is disabled.
|
||||
|
||||
===== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://host:8983/solr/<collection_name>/cdcr?action=DISABLEBUFFER
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader": {
|
||||
"status": 0,
|
||||
"QTime": 0
|
||||
},
|
||||
"status": {
|
||||
"process": "started",
|
||||
"buffer": "disabled"
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-START]]
|
||||
==== START
|
||||
|
||||
`/collection/cdcr?action=START`
|
||||
|
||||
===== Input
|
||||
|
||||
*Query Parameters:* There are no parameters for this action
|
||||
|
||||
===== Output
|
||||
|
||||
*Output Content:* Confirmation that CDCR is started and the status of buffering
|
||||
|
||||
===== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://host:8983/solr/<collection_name>/cdcr?action=START
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader": {
|
||||
"status": 0,
|
||||
"QTime": 0
|
||||
},
|
||||
"status": {
|
||||
"process": "started",
|
||||
"buffer": "enabled"
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-STOP]]
|
||||
==== STOP
|
||||
|
||||
`/collection/cdcr?action=STOP`
|
||||
|
||||
===== Input
|
||||
|
||||
*Query Parameters:* There are no parameters for this command.
|
||||
|
||||
===== Output
|
||||
|
||||
*Output Content:* The status of CDCR, including the confirmation that CDCR is stopped
|
||||
|
||||
===== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://host:8983/solr/<collection_name>/cdcr?action=STOP
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader": {
|
||||
"status": 0,
|
||||
"QTime": 0
|
||||
},
|
||||
"status": {
|
||||
"process": "stopped",
|
||||
"buffer": "enabled"
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-Monitoringcommands]]
|
||||
=== Monitoring commands
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-QUEUES]]
|
||||
==== QUEUES
|
||||
|
||||
`/core/cdcr?action=QUEUES`
|
||||
|
||||
===== Input
|
||||
|
||||
*Query Parameters:* There are no parameters for this command
|
||||
|
||||
===== Output
|
||||
|
||||
*Output Content*
|
||||
|
||||
The output is composed of a list “queues” which contains a list of (ZooKeeper) target hosts, themselves containing a list of target collections. For each collection, the current size of the queue and the timestamp of the last update operation successfully processed is provided. The timestamp of the update operation is the original timestamp, i.e., the time this operation was processed on the Source SolrCloud. This allows an estimate the latency of the replication process.
|
||||
|
||||
The “queues” object also contains information about the updates log, such as the size (in bytes) of the updates log on disk (“tlogTotalSize”), the number of transaction log files (“tlogTotalCount”) and the status of the updates log synchronizer (“updateLogSynchronizer”).
|
||||
|
||||
===== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://host:8983/solr/<replica_name>/cdcr?action=QUEUES
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader":{
|
||||
"status": 0,
|
||||
"QTime": 1
|
||||
},
|
||||
"queues":{
|
||||
"127.0.0.1: 40342/solr":{
|
||||
"Target_collection":{
|
||||
"queueSize": 104,
|
||||
"lastTimestamp": "2014-12-02T10:32:15.879Z"
|
||||
}
|
||||
}
|
||||
},
|
||||
"tlogTotalSize":3817,
|
||||
"tlogTotalCount":1,
|
||||
"updateLogSynchronizer": "stopped"
|
||||
}
|
||||
----
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-OPS]]
|
||||
==== OPS
|
||||
|
||||
`/core/cdcr?action=OPS`
|
||||
|
||||
===== Input
|
||||
|
||||
*Query Parameters:* There are no parameters for this command.
|
||||
|
||||
===== Output
|
||||
|
||||
*Output Content:* The output is composed of a list “operationsPerSecond” which contains a list of (ZooKeeper) target hosts, themselves containing a list of target collections. For each collection, the average number of processed operations per second since the start of the replication process is provided. The operations are further broken down into two groups: add and delete operations.
|
||||
|
||||
===== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://host:8983/solr/<collection_name>/cdcr?action=OPS
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader":{
|
||||
"status":0,
|
||||
"QTime":1
|
||||
},
|
||||
"operationsPerSecond":{
|
||||
"127.0.0.1: 59661/solr":{
|
||||
"Target_collection":{
|
||||
"all": 297.102944952749052,
|
||||
"adds": 297.102944952749052,
|
||||
"deletes": 0.0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-ERRORS]]
|
||||
==== ERRORS
|
||||
|
||||
`/core/cdcr?action=ERRORS`
|
||||
|
||||
===== Input
|
||||
|
||||
*Query Parameters:* There are no parameters for this command.
|
||||
|
||||
===== Output
|
||||
|
||||
*Output Content:* The output is composed of a list “errors” which contains a list of (ZooKeeper) target hosts, themselves containing a list of target collections. For each collection, information about errors encountered during the replication is provided, such as the number of consecutive errors encountered by the replicator thread, the number of bad requests or internal errors since the start of the replication process, and a list of the last errors encountered ordered by timestamp.
|
||||
|
||||
===== Examples
|
||||
|
||||
*Input*
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://host:8983/solr/<collection_name>/cdcr?action=ERRORS
|
||||
----
|
||||
|
||||
*Output*
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader":{
|
||||
"status":0,
|
||||
"QTime":2
|
||||
},
|
||||
"errors": {
|
||||
"127.0.0.1: 36872/solr":{
|
||||
"Target_collection":{
|
||||
"consecutiveErrors":3,
|
||||
"bad_request":0,
|
||||
"internal":3,
|
||||
"last":{
|
||||
"2014-12-02T11:04:42.523Z":"internal",
|
||||
"2014-12-02T11:04:39.223Z":"internal",
|
||||
"2014-12-02T11:04:38.22Z":"internal"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Initial Startup
|
||||
|
||||
This is a general approach for initializing CDCR in a production environment based upon an approach taken by the initial working installation of CDCR and generously contributed to illustrate a "real world" scenario.
|
||||
|
||||
* Customer uses the CDCR approach to keep a remote disaster-recovery instance available for production backup. This is an active-passive solution.
|
||||
* Customer has 26 clouds with 200 million assets per cloud (15GB indexes). Total document count is over 4.8 billion.
|
||||
** Source and target clouds were synched in 2-3 hour maintenance windows to establish the base index for the targets.
|
||||
|
||||
As usual, it is good to start small. Sync a single cloud and monitor for a period of time before doing the others. You may need to adjust your settings several times before finding the right balance.
|
||||
|
||||
* Before starting, stop or pause the indexers. This is best done during a small maintenance window.
|
||||
* Stop the SolrCloud instances at the Source
|
||||
* Include the CDCR request handler configuration in `solrconfig.xml` as in the below example.
|
||||
+
|
||||
[source,xml]
|
||||
----
|
||||
<requestHandler name="/cdcr" class="solr.CdcrRequestHandler">
|
||||
<lst name="replica">
|
||||
<str name="zkHost">${TargetZk}</str>
|
||||
<str name="Source">${SourceCollection}</str>
|
||||
<str name="Target">${TargetCollection}</str>
|
||||
</lst>
|
||||
<lst name="replicator">
|
||||
<str name="threadPoolSize">8</str>
|
||||
<str name="schedule">10</str>
|
||||
<str name="batchSize">2000</str>
|
||||
</lst>
|
||||
<lst name="updateLogSynchronizer">
|
||||
<str name="schedule">1000</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
|
||||
<updateRequestProcessorChain name="cdcr-processor-chain">
|
||||
<processor class="solr.CdcrUpdateProcessorFactory" />
|
||||
<processor class="solr.RunUpdateProcessorFactory" />
|
||||
</updateRequestProcessorChain>
|
||||
----
|
||||
+
|
||||
* Upload the modified `solrconfig.xml` to ZooKeeper on both Source and Target
|
||||
* Sync the index directories from the Source collection to target collection across to the corresponding shard nodes. `rsync` works well for this.
|
||||
+
|
||||
For example, if there are 2 shards on collection1 with 2 replicas for each shard, copy the corresponding index directories from
|
||||
+
|
||||
[width="75%",cols="45,10,45"]
|
||||
|===
|
||||
|shard1replica1Source |to |shard1replica1Target
|
||||
|shard1replica2Source |to |shard1replica2Target
|
||||
|shard2replica1Source |to |shard2replica1Target
|
||||
|shard2replica2Source |to |shard2replica2Target
|
||||
|===
|
||||
+
|
||||
* Start the ZooKeeper on the Target (DR) side
|
||||
* Start the SolrCloud on the Target (DR) side
|
||||
* Start the ZooKeeper on the Source side
|
||||
* Start the SolrCloud on the Source side. As a general rule, the Target (DR) side of the SolrCloud should be started before the Source side.
|
||||
* Activate the CDCR on Source instance using the CDCR API: `\http://host:port/solr/collection_name/cdcr?action=START`
|
||||
+
|
||||
[source,text]
|
||||
http://host:port/solr/<collection_name>/cdcr?action=START
|
||||
+
|
||||
* There is no need to run the /cdcr?action=START command on the Target
|
||||
* Disable the buffer on the Target
|
||||
+
|
||||
[source,text]
|
||||
http://host:port/solr/collection_name/cdcr?action=DISABLEBUFFER
|
||||
+
|
||||
* Renable indexing
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-Monitoring.1]]
|
||||
== Monitoring
|
||||
|
||||
. Network and disk space monitoring are essential. Ensure that the system has plenty of available storage to queue up changes if there is a disconnect between the Source and Target. A network outage between the two data centers can cause your disk usage to grow.
|
||||
.. Tip: Set a monitor for your disks to send alerts when the disk gets over a certain percentage (e.g., 70%)
|
||||
.. Tip: Run a test. With moderate indexing, how long can the system queue changes before you run out of disk space?
|
||||
. Create a simple way to check the counts between the Source and the Target.
|
||||
.. Keep in mind that if indexing is running, the Source and Target may not match document for document. Set an alert to fire if the difference is greater than some percentage of the overall cloud size.
|
||||
|
||||
== ZooKeeper Settings
|
||||
|
||||
With CDCR, the target ZooKeepers will have connections from the Target clouds and the Source clouds. You may need to increase the `maxClientCnxns` setting in `zoo.cfg`.
|
||||
|
||||
[source,text]
|
||||
----
|
||||
## set numbers of connection to 200 from client
|
||||
## is maxClientCnxns=0 that means no limit
|
||||
maxClientCnxns=800
|
||||
----
|
||||
|
||||
== Upgrading and Patching Production
|
||||
|
||||
When rolling in upgrades to your indexer or application, you should shutdown the Source (production) and the Target (DR). Depending on your setup, you may want to pause/stop indexing. Deploy the release or patch and renable indexing. Then start the Target (DR).
|
||||
|
||||
* There is no need to reissue the DISABLEBUFFERS or START commands. These are persisted.
|
||||
* After starting the Target, run a simple test. Add a test document to each of the Source clouds. Then check for it on the Target.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
#send to the Source
|
||||
curl http://<Source>/solr/cloud1/update -H 'Content-type:application/json' -d '[{"SKU":"ABC"}]'
|
||||
|
||||
#check the Target
|
||||
curl "http://<Target>:8983/solr/<collection_name>/select?q=SKU:ABC&wt=json&indent=true"
|
||||
----
|
||||
|
||||
[[CrossDataCenterReplication_CDCR_-Limitations.1]]
|
||||
== Limitations
|
||||
|
||||
* Running CDCR with the indexes on HDFS is not currently supported, see: https://issues.apache.org/jira/browse/SOLR-9861[Solr CDCR over HDFS].
|
|
@ -0,0 +1,164 @@
|
|||
/* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
* comments.css
|
||||
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ */
|
||||
|
||||
#comments_thread a:link {
|
||||
color: #5A88B5;
|
||||
background-color: inherit;
|
||||
}
|
||||
|
||||
#comments_thread a:visited {
|
||||
color: #5A88B5;
|
||||
background-color: inherit;
|
||||
}
|
||||
|
||||
#comments_thread a:link:hover,
|
||||
#comments_thread a:link:active,
|
||||
#comments_thread a:visited:hover,
|
||||
#comments_thread a:visited:active {
|
||||
color: #0073c7;
|
||||
background-color: #f0f0f0;
|
||||
}
|
||||
|
||||
|
||||
/* in general */
|
||||
#comments_thread p {
|
||||
line-height: 1.3em;
|
||||
color: #003;
|
||||
}
|
||||
|
||||
#comments_thread h4 {
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.apaste_menu {
|
||||
float: right;
|
||||
margin-right: 10px;
|
||||
width: 80px;
|
||||
}
|
||||
|
||||
.apaste_comment {
|
||||
background: #FEFEFE;
|
||||
border: 1px solid #AAA;
|
||||
border-radius: 2px;
|
||||
display: block;
|
||||
white-space: pre-wrap;
|
||||
font-weight: normal;
|
||||
padding-left: 20px;
|
||||
padding-right: 20px;
|
||||
padding-bottom: 16px;
|
||||
padding-top: 5px;
|
||||
margin: 15px;
|
||||
font-size: 13px
|
||||
}
|
||||
.comment_header {
|
||||
color: #000000;
|
||||
border-radius: 3px;
|
||||
border: 1px solid #999;
|
||||
min-height: 24px;
|
||||
text-indent: 5px;
|
||||
font-size: 12pt;
|
||||
background: #ffe9a3; /* Old browsers */
|
||||
background: -moz-linear-gradient(top, #ffe9a3 0%, #ffd08a 32%, #ff9d57 69%, #ff833d 100%); /* FF3.6-15 */
|
||||
background: -webkit-linear-gradient(top, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* Chrome10-25,Safari5.1-6 */
|
||||
background: linear-gradient(to bottom, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* W3C, IE10+, FF16+, Chrome26+, Opera12+, Safari7+ */
|
||||
}
|
||||
|
||||
.comment_header_verified {
|
||||
color: #000000;
|
||||
border-radius: 3px;
|
||||
border: 1px solid #999;
|
||||
min-height: 24px;
|
||||
text-indent: 5px;
|
||||
font-size: 12pt;
|
||||
background: #ffe9a3; /* Old browsers */
|
||||
background: -moz-linear-gradient(top, #ffe9a3 0%, #ffd08a 32%, #ff9d57 69%, #ff833d 100%); /* FF3.6-15 */
|
||||
background: -webkit-linear-gradient(top, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* Chrome10-25,Safari5.1-6 */
|
||||
background: linear-gradient(to bottom, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* W3C, IE10+, FF16+, Chrome26+, Opera12+, Safari7+ */
|
||||
}
|
||||
|
||||
.comment_header_sticky {
|
||||
color: #000000;
|
||||
border-radius: 3px;
|
||||
border: 1px solid #999;
|
||||
min-height: 24px;
|
||||
text-indent: 5px;
|
||||
font-size: 12pt;
|
||||
background: #ffe9a3; /* Old browsers */
|
||||
background: -moz-linear-gradient(top, #ffe9a3 0%, #ffd08a 32%, #ff9d57 69%, #ff833d 100%); /* FF3.6-15 */
|
||||
background: -webkit-linear-gradient(top, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* Chrome10-25,Safari5.1-6 */
|
||||
background: linear-gradient(to bottom, #ffe9a3 0%,#ffd08a 32%,#ff9d57 69%,#ff833d 100%); /* W3C, IE10+, FF16+, Chrome26+, Opera12+, Safari7+ */
|
||||
}
|
||||
|
||||
.comment_header img {
|
||||
padding-top: 3px;
|
||||
padding-bottom: 2px;
|
||||
}
|
||||
|
||||
.comment_header_verified img {
|
||||
padding-top: 3px;
|
||||
padding-bottom: 2px;
|
||||
}
|
||||
|
||||
.comment_header_sticky img {
|
||||
padding-top: 3px;
|
||||
padding-bottom: 2px;
|
||||
}
|
||||
|
||||
.apaste_comment img {
|
||||
/* border-radius: 5px;*/
|
||||
border: none;
|
||||
}
|
||||
|
||||
.apaste_comment_selected {background: #F8F4E9;}
|
||||
.apaste_comment_notapproved {background: #F8E0E0;}
|
||||
.apaste_comment_resolved {background: #FAFCFA;}
|
||||
.apaste_comment_sticky {background: #FFFFF6;}
|
||||
.apaste_comment_verified {background: #FAFBFA;}
|
||||
|
||||
.apaste_comment_invalid {
|
||||
color: #999;
|
||||
background: #F8F8F8;
|
||||
}
|
||||
|
||||
|
||||
.apaste_comment textarea {
|
||||
width: 480px;
|
||||
height: 180px;
|
||||
}
|
||||
|
||||
#apaste {
|
||||
margin: 5px;
|
||||
font-weight: normal;
|
||||
font-size: 14px;
|
||||
color: #024;
|
||||
|
||||
}
|
||||
#apaste .section {
|
||||
padding: 20px;
|
||||
padding-left: 80px;
|
||||
}
|
||||
|
||||
.notapproved {
|
||||
background-color: #FEE;
|
||||
padding: 5px;
|
||||
}
|
||||
|
||||
#comments_thread textarea{
|
||||
background-color: #ffffff;
|
||||
width: auto;
|
||||
border: 1px solid #1c1c1c;
|
||||
border-radius: 3px;
|
||||
box-shadow: 0pt 1px 3px rgba(0, 0, 0, 0.16) inset;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.apaste_honeypot {
|
||||
display: none;
|
||||
}
|
||||
|
||||
//* Remove external link icons when they appear in comments *//
|
||||
a[href^="http://"]:after,
|
||||
a[href^="https://"]:after {
|
||||
content: none !important;
|
||||
}
|
|
@ -0,0 +1,869 @@
|
|||
|
||||
.gi-2x{font-size: 2em;}
|
||||
.gi-3x{font-size: 3em;}
|
||||
.gi-4x{font-size: 4em;}
|
||||
.gi-5x{font-size: 5em;}
|
||||
|
||||
.breadcrumb > .active {color: #777 !important;}
|
||||
|
||||
.post-content img {
|
||||
margin: 12px 0 3px 0;
|
||||
width: auto;
|
||||
height: auto;
|
||||
max-width: 100%;
|
||||
max-height: 100%;
|
||||
}
|
||||
|
||||
.post-content ol li, .post-content ul li {
|
||||
margin: 10px 0;
|
||||
}
|
||||
|
||||
.pageSummary {
|
||||
font-size:13px;
|
||||
display:block;
|
||||
margin-bottom:15px;
|
||||
padding-left:20px;
|
||||
}
|
||||
|
||||
.post-summary {
|
||||
margin-bottom:12px;
|
||||
}
|
||||
|
||||
.bs-example{
|
||||
margin: 20px;
|
||||
}
|
||||
|
||||
.breadcrumb li {
|
||||
color: gray;
|
||||
}
|
||||
|
||||
caption {
|
||||
padding-top: 8px;
|
||||
padding-bottom: 8px;
|
||||
color: #777;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
p.external a {
|
||||
text-align:right;
|
||||
font-size:12px;
|
||||
color: #0088cc;
|
||||
display:inline;
|
||||
}
|
||||
|
||||
#definition-box-container div a.active {
|
||||
font-weight: bold;
|
||||
}
|
||||
p.post-meta {font-size: 80%; color: #777;}
|
||||
|
||||
.entry-date{font-size:14px;font-size:0.875rem;line-height:1.71429;margin-bottom:0;text-transform:uppercase;}
|
||||
|
||||
/* search area */
|
||||
#search-demo-container ul#results-container {
|
||||
list-style: none;
|
||||
font-size: 12px;
|
||||
background-color: white;
|
||||
position: absolute;
|
||||
top: 40px; /* if you change anything about the nav, you'll prob. need to reset the top and left values here.*/
|
||||
left: 20px;
|
||||
z-index: -1;
|
||||
width:223px;
|
||||
border-left: 1px solid #dedede;
|
||||
box-shadow: 2px 3px 2px #dedede;
|
||||
}
|
||||
|
||||
/* make room for the nav bar */
|
||||
h1[id],
|
||||
h2[id],
|
||||
h3[id],
|
||||
h4[id],
|
||||
h5[id],
|
||||
h6[id],
|
||||
dt[id]{
|
||||
padding-top: 60px;
|
||||
margin-top: -40px
|
||||
}
|
||||
|
||||
ul#results-container a {
|
||||
background-color: transparent;
|
||||
}
|
||||
|
||||
ul#results-container a:hover {
|
||||
color: black;
|
||||
}
|
||||
|
||||
|
||||
#search-demo-container a:hover {
|
||||
color: black;
|
||||
}
|
||||
#search-input {
|
||||
padding: .5em;
|
||||
margin-left:20px;
|
||||
width:20em;
|
||||
font-size: 0.8em;
|
||||
-webkit-box-sizing: border-box;
|
||||
-moz-box-sizing: border-box;
|
||||
box-sizing: border-box;
|
||||
float: right;
|
||||
margin-top:10px;
|
||||
}
|
||||
/* end search */
|
||||
|
||||
.filter-options {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
.filter-options button {
|
||||
margin: 3px;
|
||||
}
|
||||
|
||||
|
||||
|
||||
li.dropdownActive a {
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
|
||||
.post-content a.fa-rss {
|
||||
color: orange;
|
||||
}
|
||||
|
||||
|
||||
.navbar-inverse .navbar-nav > li > a {
|
||||
background-color: transparent;
|
||||
margin-top:10px;
|
||||
}
|
||||
|
||||
.post-content .rssfeedLink {
|
||||
color: #248EC2;
|
||||
}
|
||||
|
||||
footer {
|
||||
font-size: smaller;
|
||||
}
|
||||
|
||||
/* FAQ page */
|
||||
#accordion .panel-heading {
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
a.accordion-toggle, a.accordion-collapsed {
|
||||
font-size: 14px;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
/* navgoco sidebar styles (customized) */
|
||||
.nav, .nav ul, .nav li {
|
||||
list-style: none;
|
||||
}
|
||||
|
||||
.nav ul {
|
||||
padding: 0;
|
||||
/*margin: 0 0 0 18px;*/
|
||||
margin:0;
|
||||
}
|
||||
|
||||
.nav {
|
||||
/* padding: 4px;*/
|
||||
padding:0;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.nav > li {
|
||||
margin: 1px 0;
|
||||
}
|
||||
|
||||
.nav > li li {
|
||||
margin: 2px 0;
|
||||
}
|
||||
|
||||
.nav a {
|
||||
color: #333;
|
||||
display: block;
|
||||
outline: none;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
.nav li > a > span {
|
||||
float: right;
|
||||
font-size: 19px;
|
||||
font-weight: bolder;
|
||||
}
|
||||
|
||||
|
||||
.nav li > a > span:after {
|
||||
content: '\25be';
|
||||
}
|
||||
.nav li.open > a > span:after {
|
||||
content: '\25b4';
|
||||
}
|
||||
|
||||
.nav a:hover, .nav a:focus, .nav li.active > a {
|
||||
background-color: #8D8D8D;
|
||||
color: #f5f5f5;
|
||||
}
|
||||
|
||||
.nav > li.active > a {
|
||||
background-color: #347DBE;
|
||||
}
|
||||
|
||||
.nav li a {
|
||||
line-height: 18px;
|
||||
padding: 2px 10px;
|
||||
background-color: #f1f1f1;
|
||||
}
|
||||
|
||||
.nav > li > a {
|
||||
line-height: 20px;
|
||||
padding: 4px 10px;
|
||||
}
|
||||
|
||||
ul#mysidebar {
|
||||
border-radius:0;
|
||||
}
|
||||
|
||||
ul.nav li ul {
|
||||
font-size: 10pt;
|
||||
}
|
||||
|
||||
.nav ul li a {
|
||||
background-color: #FAFAFA;
|
||||
}
|
||||
|
||||
.nav li a {
|
||||
padding-right:10px;
|
||||
}
|
||||
|
||||
.nav li a:hover {
|
||||
background-color: #8D8D8D;
|
||||
}
|
||||
|
||||
.nav ul li a {
|
||||
border-top:1px solid whitesmoke;
|
||||
padding-left:10px;
|
||||
}
|
||||
/* end sidebar */
|
||||
|
||||
.navbar-inverse .navbar-nav > .active > a, .navbar-inverse .navbar-nav > .active > a:hover, .navbar-inverse .navbar-nav > .active > a:focus {
|
||||
border-radius:5px;
|
||||
}
|
||||
|
||||
.navbar-inverse .navbar-nav>.open>a, .navbar-inverse .navbar-nav>.open>a:focus, .navbar-inverse .navbar-nav>.open>a:hover {
|
||||
border-radius: 5px;
|
||||
}
|
||||
|
||||
.footer {
|
||||
text-align: right;
|
||||
}
|
||||
|
||||
.footerMeta {
|
||||
background-color: whitesmoke;
|
||||
padding: 10px;
|
||||
max-width: 250px;
|
||||
border-radius: 5px;
|
||||
margin-top: 50px;
|
||||
font-style:italic;
|
||||
font-size:12px;
|
||||
}
|
||||
|
||||
img.screenshotSmall {
|
||||
max-width: 300px;
|
||||
}
|
||||
|
||||
|
||||
dl dt p {
|
||||
margin-left:20px;
|
||||
}
|
||||
|
||||
|
||||
dl dd {
|
||||
margin-top:10px;
|
||||
margin-bottom:10px;
|
||||
}
|
||||
|
||||
dl.dl-horizontal dd {
|
||||
padding-top: 20px;
|
||||
}
|
||||
|
||||
figcaption {
|
||||
|
||||
padding-bottom:12px;
|
||||
padding-top:6px;
|
||||
max-width: 90%;
|
||||
margin-bottom:20px;
|
||||
font-style: italic;
|
||||
color: gray;
|
||||
|
||||
}
|
||||
|
||||
.testing {
|
||||
color: orange;
|
||||
}
|
||||
|
||||
.preference {
|
||||
color: red;
|
||||
}
|
||||
|
||||
|
||||
table.dataTable thead {
|
||||
background-color: #444;
|
||||
}
|
||||
|
||||
section table tr.success {
|
||||
background-color: #dff0d8 !important;
|
||||
}
|
||||
|
||||
table tr.info {
|
||||
background-color: #d9edf7 !important;
|
||||
}
|
||||
|
||||
section table tr.warning, table tr.testing, table tr.testing > td.sorting_1 {
|
||||
background-color: #fcf8e3 !important;
|
||||
}
|
||||
section table tr.danger, table tr.preference, table tr.preference > td.sorting_1 {
|
||||
background-color: #f2dede !important;
|
||||
}
|
||||
|
||||
.orange {
|
||||
color: orange;
|
||||
}
|
||||
|
||||
table.profile thead tr th {
|
||||
background-color: #248ec2;
|
||||
}
|
||||
|
||||
table.request thead tr th {
|
||||
background-color: #ED1951;
|
||||
}
|
||||
|
||||
.audienceLabel {
|
||||
margin: 10px;
|
||||
float: right;
|
||||
border:1px solid #dedede;
|
||||
padding:7px;
|
||||
}
|
||||
|
||||
.prefaceAudienceLabel {
|
||||
color: gray;
|
||||
text-align: center;
|
||||
margin:5px;
|
||||
}
|
||||
span.myLabel {
|
||||
padding-left:10px;
|
||||
padding-right:10px;
|
||||
}
|
||||
|
||||
button.cursorNorm {
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
a.dropdown-toggle, .navbar-inverse .navbar-nav > li > a {
|
||||
margin-left: 10px;
|
||||
}
|
||||
|
||||
hr.faded {
|
||||
border: 0;
|
||||
height: 1px;
|
||||
background-image: -webkit-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));
|
||||
background-image: -moz-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));
|
||||
background-image: -ms-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));
|
||||
background-image: -o-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));
|
||||
}
|
||||
|
||||
hr.shaded {
|
||||
height: 12px;
|
||||
border: 0;
|
||||
box-shadow: inset 0 6px 6px -6px rgba(0,0,0,0.5);
|
||||
margin-top: 70px;
|
||||
background: white;
|
||||
width: 100%;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
|
||||
.fa-6x{font-size:900%;}
|
||||
.fa-7x{font-size:1100%;}
|
||||
.fa-8x{font-size:1300%;}
|
||||
.fa-9x{font-size:1500%;}
|
||||
.fa-10x{font-size:1700%;}
|
||||
|
||||
i.border {
|
||||
padding: 10px 20px;
|
||||
background-color: whitesmoke;
|
||||
}
|
||||
|
||||
a[data-toggle] {
|
||||
color: #248EC2;
|
||||
}
|
||||
|
||||
.summary {
|
||||
font-size:120%;
|
||||
color: #808080;
|
||||
margin:20px 0 20px 0;
|
||||
border-left: 5px solid #ED1951;
|
||||
padding-left: 10px;
|
||||
|
||||
}
|
||||
|
||||
.summary:before {
|
||||
content: "Summary: ";
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
|
||||
a.fa.fa-envelope-o.mailto {
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.nav-tabs > li.active > a, .nav-tabs > li.active > a:hover, .nav-tabs > li.active > a:focus {
|
||||
background-color: #248ec2;
|
||||
color: white;
|
||||
}
|
||||
|
||||
ol li ol li {list-style-type: lower-alpha;}
|
||||
ol li ul li {list-style-type: disc;}
|
||||
|
||||
li img {clear:both; }
|
||||
|
||||
div#toc ul li ul li {
|
||||
list-style-type: none;
|
||||
margin: 5px 0 0 0;
|
||||
}
|
||||
|
||||
.tab-content {
|
||||
padding: 15px;
|
||||
background-color: #FAFAFA;
|
||||
}
|
||||
|
||||
span.tagTitle {font-weight: 500;}
|
||||
|
||||
li.activeSeries {
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
.seriesContext .dropdown-menu li.active {
|
||||
font-weight: bold;
|
||||
margin-left: 43px;
|
||||
font-size:18px;
|
||||
}
|
||||
|
||||
div.tags {padding: 10px 5px;}
|
||||
|
||||
.tabLabel {
|
||||
font-weight: normal;
|
||||
}
|
||||
|
||||
hr {
|
||||
border: 0;
|
||||
border-bottom: 1px dashed #ccc;
|
||||
background: #999;
|
||||
margin: 30px 0;
|
||||
width: 90%;
|
||||
margin-left: auto;
|
||||
margin-right: auto;
|
||||
}
|
||||
|
||||
button.cursorNorm {
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
span.otherProgrammingLanguages {
|
||||
font-style: normal;
|
||||
}
|
||||
|
||||
a[data-toggle="tooltip"] {
|
||||
color: #649345;
|
||||
font-style: italic;
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
.seriesNext, .seriesContext {
|
||||
margin-top: 15px;
|
||||
margin-bottom: 15px;
|
||||
}
|
||||
|
||||
.seriescontext ol li {
|
||||
list-style-type: upper-roman;
|
||||
}
|
||||
|
||||
ol.series li {
|
||||
list-style-type: decimal;
|
||||
margin-left: 40px;
|
||||
padding-left: 0;
|
||||
}
|
||||
|
||||
.siteTagline {
|
||||
font-size: 200%;
|
||||
font-weight: bold;
|
||||
color: silver;
|
||||
font-family: monospace;
|
||||
text-align: center;
|
||||
line-height: 10px;
|
||||
margin: 20px 0;
|
||||
display: block;
|
||||
}
|
||||
|
||||
.versionTagline {
|
||||
text-align: center;
|
||||
margin-bottom: 20px;
|
||||
font-family: courier;
|
||||
color: silver;
|
||||
color: #444;
|
||||
display:block;
|
||||
}
|
||||
|
||||
#mysidebar .nav ul {
|
||||
background-color: #FAFAFA;
|
||||
}
|
||||
.nav ul.series li {
|
||||
list-style: decimal;
|
||||
font-size:12px;
|
||||
}
|
||||
|
||||
.nav ul.series li a:hover {
|
||||
background-color: gray;
|
||||
}
|
||||
.nav ul.series {
|
||||
padding-left: 30px;
|
||||
}
|
||||
|
||||
.nav ul.series {
|
||||
background-color: #FAFAFA;
|
||||
}
|
||||
|
||||
/*
|
||||
a.dropdown-toggle.otherProgLangs {
|
||||
color: #f7e68f !important;
|
||||
}
|
||||
*/
|
||||
|
||||
span.muted {color: #666;}
|
||||
|
||||
table code {background-color: transparent;}
|
||||
|
||||
.highlight .err {
|
||||
color: #a61717;
|
||||
background-color: transparent !important;
|
||||
}
|
||||
|
||||
#json-box-container pre {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.video-js {
|
||||
margin: 30px 0;
|
||||
}
|
||||
|
||||
video {
|
||||
display: block;
|
||||
margin: 30px 0;
|
||||
border: 1px solid #c0c0c0;
|
||||
}
|
||||
|
||||
|
||||
p.required, p.dataType {display: block; color: #c0c0c0; font-size: 80%; margin-left:4px;}
|
||||
|
||||
dd {margin-left:20px;}
|
||||
|
||||
.post-content img.inline {
|
||||
margin:0;
|
||||
margin-bottom:6px;
|
||||
}
|
||||
.panel-heading {
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
a.accordion-toggle {
|
||||
font-style: normal;
|
||||
}
|
||||
|
||||
span.red {
|
||||
color: red;
|
||||
font-family: Monaco, Menlo, Consolas, "Courier New", monospace;
|
||||
}
|
||||
|
||||
h3.codeExplanation {
|
||||
font-size:18px;
|
||||
font-style:normal;
|
||||
color: black;
|
||||
line-height: 24px;
|
||||
}
|
||||
|
||||
span.soft {
|
||||
color: #c0c0c0;
|
||||
}
|
||||
|
||||
.githubEditButton {
|
||||
margin-bottom:7px;
|
||||
}
|
||||
|
||||
.endpoint {
|
||||
padding: 15px;
|
||||
background-color: #f0f0f0;
|
||||
font-family: courier;
|
||||
font-size: 110%;
|
||||
margin: 20px 0;
|
||||
color: #444;
|
||||
}
|
||||
|
||||
.parameter {
|
||||
font-family: courier;
|
||||
color: red !important;
|
||||
}
|
||||
|
||||
.formBoundary {
|
||||
border: 1px solid gray;
|
||||
padding: 15px;
|
||||
margin: 15px 0;
|
||||
background-color: whitesmoke;
|
||||
}
|
||||
|
||||
@media (max-width: 767px) {
|
||||
.navbar-inverse .navbar-nav .open .dropdown-menu > li > a {
|
||||
color: #444;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 990px) {
|
||||
#mysidebar {
|
||||
position: relative;
|
||||
}
|
||||
}
|
||||
|
||||
@media (min-width: 1000px) {
|
||||
|
||||
ul#mysidebar {
|
||||
width: 225px;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 900px) {
|
||||
|
||||
ul#mysidebar {
|
||||
max-width: 100%;
|
||||
}
|
||||
}
|
||||
|
||||
.col-md-9 img {
|
||||
max-width: 100%;
|
||||
max-height: 100%;
|
||||
}
|
||||
|
||||
.videoThumbs img {
|
||||
float: left;
|
||||
margin:15px 15px 15px 0;
|
||||
box-shadow: 2px 2px 1px #f0f0f0;
|
||||
border: 1px solid #dedede;
|
||||
}
|
||||
|
||||
@media only screen and (min-width: 900px)
|
||||
{.col-md-9 img {
|
||||
max-width: 700px;
|
||||
max-height: 700px;
|
||||
}
|
||||
}
|
||||
|
||||
@media only screen and (min-device-width: 900px)
|
||||
{.col-md-9 img {
|
||||
max-width: 700px;
|
||||
max-height: 700px;
|
||||
}
|
||||
}
|
||||
*:hover > .anchorjs-link {
|
||||
transition: color .25s linear;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
.kbCaption {
|
||||
color: white;
|
||||
background-color: #444;
|
||||
padding:10px;
|
||||
}
|
||||
|
||||
.btn-default {
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
|
||||
/* algolia search */
|
||||
|
||||
.search {
|
||||
text-align: left;
|
||||
}
|
||||
.search input {
|
||||
font-size: 20px;
|
||||
width: 300px;
|
||||
}
|
||||
.results {
|
||||
margin: auto;
|
||||
text-align: left;
|
||||
}
|
||||
.results ul {
|
||||
list-style-type: none;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
/* algolia */
|
||||
|
||||
div.results {
|
||||
position: absolute;
|
||||
background-color: white;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.post-meta {
|
||||
font-size: 14px;
|
||||
color: #828282;
|
||||
}
|
||||
|
||||
.post-link {
|
||||
font-size: 22px;
|
||||
}
|
||||
|
||||
.post-list p {
|
||||
margin: 10px 0;
|
||||
}
|
||||
|
||||
time {
|
||||
margin-right: 10px;
|
||||
}
|
||||
|
||||
p.post-meta time {
|
||||
margin-right: 0;
|
||||
}
|
||||
|
||||
span.label.label-default {
|
||||
background-color: gray;
|
||||
}
|
||||
|
||||
span.label.label-primary {
|
||||
background-color: #f0ad4e;
|
||||
}
|
||||
.col-lg-12 .nav li a {background-color: white}
|
||||
|
||||
a code {
|
||||
color: ##2156a5;
|
||||
}
|
||||
|
||||
table th code {
|
||||
color: white;
|
||||
}
|
||||
|
||||
ol li ul li ol li {
|
||||
list-style: decimal;
|
||||
}
|
||||
|
||||
ol li ul li ol li ul li{
|
||||
list-style: disc;
|
||||
}
|
||||
|
||||
|
||||
.box {
|
||||
padding: 10px;
|
||||
border: 1px solid #888;
|
||||
box-shadow: 2px 2px 4px #dedede;
|
||||
width: 100px;
|
||||
height: 80px;
|
||||
background-color: #f5f5f5;
|
||||
font-family: Arial;
|
||||
font-size: 12px;
|
||||
hyphens: auto;
|
||||
float: left;
|
||||
}
|
||||
|
||||
.box:hover {
|
||||
background-color: #f0f0f0;
|
||||
}
|
||||
|
||||
#userMap {
|
||||
overflow-x: auto;
|
||||
overflow-y: auto;
|
||||
padding: 20px;
|
||||
min-width: 770px;
|
||||
}
|
||||
|
||||
#userMap .active {
|
||||
background-color: #d6f5d6;
|
||||
border:1px solid #555;
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
h2.userMapTitle {
|
||||
font-family: Arial;
|
||||
}
|
||||
|
||||
#userMap a:hover {
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
div.arrow {
|
||||
max-width: 50px;
|
||||
margin-left: 15px;
|
||||
margin-right: 15px;
|
||||
font-size: 20px;
|
||||
}
|
||||
|
||||
#userMap div.arrow, #userMap div.content {
|
||||
float: left;
|
||||
}
|
||||
|
||||
.clearfix {
|
||||
clear: both;
|
||||
}
|
||||
|
||||
|
||||
#userMap div.arrow {
|
||||
position: relative;
|
||||
top: 30px;
|
||||
}
|
||||
|
||||
.box1 {
|
||||
margin-left:0;
|
||||
}
|
||||
|
||||
button.btn.btn-default.btn-lg.modalButton1 {
|
||||
margin-left: -20px;
|
||||
}
|
||||
|
||||
div.box.box1 {
|
||||
margin-left: -20px;
|
||||
}
|
||||
|
||||
#userMap .btn-lg {
|
||||
width: 100px;
|
||||
height: 80px;
|
||||
|
||||
}
|
||||
|
||||
#userMap .complexArrow {
|
||||
font-size: 22px;
|
||||
margin: 0 10px;
|
||||
}
|
||||
|
||||
|
||||
#userMap .btn-lg .active {
|
||||
background-color: #d6f5d6;
|
||||
}
|
||||
|
||||
#userMap .btn-lg {
|
||||
white-space: pre-wrap; /* css-3 */
|
||||
white-space: -moz-pre-wrap; /* Mozilla, since 1999 */
|
||||
white-space: -pre-wrap; /* Opera 4-6 */
|
||||
white-space: -o-pre-wrap; /* Opera 7 */
|
||||
word-wrap: break-word; /* Internet Explorer 5.5+ */
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
/*
|
||||
* Let's target IE to respect aspect ratios and sizes for img tags containing SVG files
|
||||
*
|
||||
* [1] IE9
|
||||
* [2] IE10+
|
||||
*/
|
||||
/* 1 */
|
||||
.ie9 img[src$=".svg"] {
|
||||
width: 100%;
|
||||
}
|
||||
/* 2 */
|
||||
@media screen and (-ms-high-contrast: active), (-ms-high-contrast: none) {
|
||||
img[src$=".svg"] {
|
||||
width: 100%;
|
||||
}
|
||||
}
|
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
|
@ -0,0 +1,160 @@
|
|||
|
||||
/*body.print .container {max-width: 650px;}*/
|
||||
|
||||
body {
|
||||
font-size:14px;
|
||||
}
|
||||
.nav ul li a {border-top:0px; background-color:transparent; color: #808080; }
|
||||
#navig a[href] {color: #595959 !important;}
|
||||
table .table {max-width:650px;}
|
||||
|
||||
#navig li.sectionHead {font-weight: bold; font-size: 18px; color: #595959 !important; }
|
||||
#navig li {font-weight: normal; }
|
||||
|
||||
#navig a[href]::after { content: leader(".") target-counter(attr(href), page); }
|
||||
|
||||
a[href]::after {
|
||||
content: " (page " target-counter(attr(href), page) ")"
|
||||
}
|
||||
|
||||
a[href^="http:"]::after, a[href^="https:"]::after {
|
||||
content: " (" attr(href) ")";
|
||||
}
|
||||
|
||||
a[href] {
|
||||
color: blue !important;
|
||||
}
|
||||
a[href*="mailto"]::after, a[data-toggle="tooltip"]::after, a[href].noCrossRef::after {
|
||||
content: "";
|
||||
}
|
||||
|
||||
|
||||
@page {
|
||||
margin: 60pt 90pt 60pt 90pt;
|
||||
font-family: sans-serif;
|
||||
font-style:none;
|
||||
color: gray;
|
||||
|
||||
}
|
||||
|
||||
.printTitle {
|
||||
line-height:30pt;
|
||||
font-size:27pt;
|
||||
font-weight: bold;
|
||||
letter-spacing: -.5px;
|
||||
margin-bottom:25px;
|
||||
}
|
||||
|
||||
.printSubtitle {
|
||||
font-size: 19pt;
|
||||
color: #cccccc !important;
|
||||
front-family: "Grotesque MT Light";
|
||||
line-height: 22pt;
|
||||
letter-spacing: -.5px;
|
||||
margin-bottom:20px;
|
||||
}
|
||||
.printTitleArea hr {
|
||||
color: #999999 !important;
|
||||
height: 2px;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.printTitleImage {
|
||||
max-width:300px;
|
||||
margin-bottom:200px;
|
||||
}
|
||||
|
||||
|
||||
.printTitleImage {
|
||||
max-width: 250px;
|
||||
}
|
||||
|
||||
#navig {
|
||||
/*page-break-before: always;*/
|
||||
}
|
||||
|
||||
.copyrightBoilerplate {
|
||||
page-break-before:always;
|
||||
font-size:14px;
|
||||
}
|
||||
|
||||
.lastGeneratedDate {
|
||||
font-style: italic;
|
||||
font-size:14px;
|
||||
color: gray;
|
||||
}
|
||||
|
||||
.alert a {
|
||||
text-decoration: none !important;
|
||||
}
|
||||
|
||||
|
||||
body.title { page: title }
|
||||
|
||||
@page title {
|
||||
@top-left {
|
||||
content: " ";
|
||||
}
|
||||
@top-right {
|
||||
content: " "
|
||||
}
|
||||
@bottom-right {
|
||||
content: " ";
|
||||
}
|
||||
@bottom-left {
|
||||
content: " ";
|
||||
}
|
||||
}
|
||||
|
||||
body.frontmatter { page: frontmatter }
|
||||
body.frontmatter {counter-reset: page 1}
|
||||
|
||||
|
||||
@page frontmatter {
|
||||
@top-left {
|
||||
content: prince-script(guideName);
|
||||
}
|
||||
@top-right {
|
||||
content: prince-script(datestamp);
|
||||
}
|
||||
@bottom-right {
|
||||
content: counter(page, lower-roman);
|
||||
}
|
||||
@bottom-left {
|
||||
content: "youremail@domain.com"; }
|
||||
}
|
||||
|
||||
body.first_page {counter-reset: page 1}
|
||||
|
||||
h1 { string-set: doctitle content() }
|
||||
|
||||
@page {
|
||||
@top-left {
|
||||
content: string(doctitle);
|
||||
font-size: 11px;
|
||||
font-style: italic;
|
||||
}
|
||||
@top-right {
|
||||
content: prince-script(datestamp);
|
||||
font-size: 11px;
|
||||
}
|
||||
|
||||
@bottom-right {
|
||||
content: "Page " counter(page);
|
||||
font-size: 11px;
|
||||
}
|
||||
@bottom-left {
|
||||
content: prince-script(guideName);
|
||||
font-size: 11px;
|
||||
}
|
||||
}
|
||||
.alert {
|
||||
background-color: #fafafa !important;
|
||||
border-color: #dedede !important;
|
||||
color: black;
|
||||
}
|
||||
|
||||
pre {
|
||||
background-color: #fafafa;
|
||||
}
|
||||
|
File diff suppressed because it is too large
Load Diff
|
@ -0,0 +1,127 @@
|
|||
.summary {
|
||||
color: #808080;
|
||||
border-left: 5px solid #d9411e;
|
||||
font-size:16px;
|
||||
}
|
||||
|
||||
.nav-tabs > li.active > a, .nav-tabs > li.active > a:hover, .nav-tabs > li.active > a:focus {
|
||||
background-color: #FF833D;
|
||||
color: white;
|
||||
}
|
||||
|
||||
.nav > li.active > a {
|
||||
background-color: #262130;
|
||||
}
|
||||
|
||||
.nav > li > a:hover {
|
||||
background-color: #FF833D;
|
||||
}
|
||||
|
||||
div.navbar-collapse .dropdown-menu > li > a:hover {
|
||||
background-color: #D9411E;
|
||||
}
|
||||
|
||||
.nav li.thirdlevel > a {
|
||||
background-color: #FAFAFA !important;
|
||||
color: #FF833D;
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
a[data-toggle="tooltip"] {
|
||||
color: #649345;
|
||||
font-style: italic;
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
.navbar-inverse {
|
||||
background-color: #D9411E;
|
||||
border-color: #015CAE;
|
||||
}
|
||||
|
||||
.navbar-inverse .navbar-nav > .open > a, .navbar-inverse .navbar-nav > .open > a:hover, .navbar-inverse .navbar-nav > .open > a:focus {
|
||||
color: #015CAE;
|
||||
}
|
||||
|
||||
.navbar-inverse .navbar-nav > .open > a, .navbar-inverse .navbar-nav > .open > a:hover, .navbar-inverse .navbar-nav > .open > a:focus {
|
||||
background-color: #015CAE;
|
||||
color: #ffffff;
|
||||
}
|
||||
|
||||
/* not sure if using this ...*/
|
||||
.navbar-inverse .navbar-collapse, .navbar-inverse .navbar-form {
|
||||
border-color: #248ec2 !important;
|
||||
}
|
||||
|
||||
/* Used for nav buttons */
|
||||
.btn-primary {
|
||||
color: #ffffff;
|
||||
background-color: #0A7C38;
|
||||
border-color: #E6E7E8;
|
||||
}
|
||||
|
||||
.navbar-inverse .navbar-nav > .active > a, .navbar-inverse .navbar-nav > .active > a:hover, .navbar-inverse .navbar-nav > .active > a:focus {
|
||||
background-color: #D9411E;
|
||||
}
|
||||
|
||||
/* Used for nav buttons */
|
||||
.btn-primary:hover,
|
||||
.btn-primary:focus,
|
||||
.btn-primary:active,
|
||||
.btn-primary.active,
|
||||
.open .dropdown-toggle.btn-primary {
|
||||
background-color: #305CB3;
|
||||
border-color: #E6E7E8;
|
||||
}
|
||||
|
||||
.printTitle {
|
||||
color: #015CAE !important;
|
||||
}
|
||||
|
||||
body.print h1 {color: #015CAE !important; font-size:28px !important;}
|
||||
body.print h2 {color: #595959 !important; font-size:20px !important;}
|
||||
body.print h3 {color: #E50E51 !important; font-size:14px !important;}
|
||||
body.print h4 {color: #679DCE !important; font-size:14px; font-style: italic !important;}
|
||||
|
||||
.anchorjs-link:hover {
|
||||
color: #216f9b;
|
||||
}
|
||||
|
||||
div.sidebarTitle {
|
||||
color: #015CAE;
|
||||
}
|
||||
|
||||
li.sidebarTitle {
|
||||
margin-top:40px;
|
||||
font-weight:normal;
|
||||
font-size:120%;
|
||||
color: #262130;
|
||||
margin-bottom:10px;
|
||||
margin-left: 5px;
|
||||
|
||||
}
|
||||
.scrollnav {
|
||||
font-size: larger;
|
||||
/* why do we need both of these??? */
|
||||
padding-bottom: 1em;
|
||||
margin-bottom: 2em;
|
||||
}
|
||||
.scrollnav .prev {
|
||||
text-align: left;
|
||||
float: left;
|
||||
font-size: inherit;
|
||||
}
|
||||
.scrollnav .prev:before {
|
||||
padding-right: 0.5em;
|
||||
content: "\25C0 ";
|
||||
display: inline-block; /* text-decoration: none doesn't work, but this does */
|
||||
}
|
||||
.scrollnav .next {
|
||||
text-align: right;
|
||||
float: right;
|
||||
font-size: inherit;
|
||||
}
|
||||
.scrollnav .next:after {
|
||||
padding-left: 0.5em;
|
||||
content: " \25B6";
|
||||
display: inline-block; /* text-decoration: none doesn't work, but this does */
|
||||
}
|
|
@ -0,0 +1,45 @@
|
|||
= DataDir and DirectoryFactory in SolrConfig
|
||||
:page-shortname: datadir-and-directoryfactory-in-solrconfig
|
||||
:page-permalink: datadir-and-directoryfactory-in-solrconfig.html
|
||||
|
||||
Where and how Solr stores its indexes are configurable options.
|
||||
|
||||
== Specifying a Location for Index Data with the `dataDir` Parameter
|
||||
|
||||
By default, Solr stores its index data in a directory called `/data` under the core's instance directory (`instanceDir`). If you would like to specify a different directory for storing index data, you can configure the `dataDir` in the `core.properties` file for the core, or use the `<dataDir>` parameter in the `solrconfig.xml` file. You can specify another directory either with an absolute path or a pathname relative to the instanceDir of the SolrCore. For example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dataDir>/solr/data/${solr.core.name}</dataDir>
|
||||
----
|
||||
|
||||
The `${solr.core.name}` substitution will cause the name of the current core to be substituted, which results in each core's data being kept in a separate subdirectory.
|
||||
|
||||
If you are using replication to replicate the Solr index (as described in <<legacy-scaling-and-distribution.adoc#legacy-scaling-and-distribution,Legacy Scaling and Distribution>>), then the `<dataDir>` directory should correspond to the index directory used in the replication configuration.
|
||||
|
||||
[[DataDirandDirectoryFactoryinSolrConfig-SpecifyingtheDirectoryFactoryForYourIndex]]
|
||||
== Specifying the DirectoryFactory For Your Index
|
||||
|
||||
The default {solr-javadocs}/solr-core/org/apache/solr/core/StandardDirectoryFactory.html[`solr.StandardDirectoryFactory`] is filesystem based, and tries to pick the best implementation for the current JVM and platform. You can force a particular implementation and/or config options by specifying {solr-javadocs}/solr-core/org/apache/solr/core/MMapDirectoryFactory.html[`solr.MMapDirectoryFactory`], {solr-javadocs}/solr-core/org/apache/solr/core/NIOFSDirectoryFactory.html[`solr.NIOFSDirectoryFactory`], or {solr-javadocs}/solr-core/org/apache/solr/core/SimpleFSDirectoryFactory.html[`solr.SimpleFSDirectoryFactory`].
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<directoryFactory name="DirectoryFactory"
|
||||
class="solr.MMapDirectoryFactory">
|
||||
<bool name="preload">true</bool>
|
||||
</directoryFactory>
|
||||
----
|
||||
|
||||
The {solr-javadocs}/solr-core/org/apache/solr/core/RAMDirectoryFactory.html[`solr.RAMDirectoryFactory`] is memory based, not persistent, and does not work with replication. Use this DirectoryFactory to store your index in RAM.
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<directoryFactory class="org.apache.solr.core.RAMDirectoryFactory"/>
|
||||
----
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
|
||||
If you are using Hadoop and would like to store your indexes in HDFS, you should use the {solr-javadocs}/solr-core/org/apache/solr/core/HdfsDirectoryFactory.html[`solr.HdfsDirectoryFactory`] instead of either of the above implementations. For more details, see the section <<running-solr-on-hdfs.adoc#running-solr-on-hdfs,Running Solr on HDFS>>.
|
||||
|
||||
====
|
|
@ -0,0 +1,13 @@
|
|||
= Dataimport Screen
|
||||
:page-shortname: dataimport-screen
|
||||
:page-permalink: dataimport-screen.html
|
||||
|
||||
The Dataimport screen shows the configuration of the DataImportHandler (DIH) and allows you start, and monitor the status of, import commands as defined by the options selected on the screen and defined in the configuration file.
|
||||
|
||||
.The Dataimport Screen
|
||||
image::images/dataimport-screen/dataimport.png[image,width=485,height=250]
|
||||
|
||||
|
||||
This screen also lets you adjust various options to control how the data is imported to Solr, and view the data import configuration file that controls the import.
|
||||
|
||||
For more information about data importing with DIH, see the section on <<uploading-structured-data-store-data-with-the-data-import-handler.adoc#uploading-structured-data-store-data-with-the-data-import-handler,Uploading Structured Data Store Data with the Data Import Handler>>.
|
|
@ -0,0 +1,100 @@
|
|||
= De-Duplication
|
||||
:page-shortname: de-duplication
|
||||
:page-permalink: de-duplication.html
|
||||
|
||||
If duplicate, or near-duplicate documents are a concern in your index, de-duplication may be worth implementing.
|
||||
|
||||
Preventing duplicate or near duplicate documents from entering an index or tagging documents with a signature/fingerprint for duplicate field collapsing can be efficiently achieved with a low collision or fuzzy hash algorithm. Solr natively supports de-duplication techniques of this type via the `Signature` class and allows for the easy addition of new hash/signature implementations. A Signature can be implemented several ways:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="30,70",options="header"]
|
||||
|===
|
||||
|Method |Description
|
||||
|MD5Signature |128-bit hash used for exact duplicate detection.
|
||||
|Lookup3Signature |64-bit hash used for exact duplicate detection. This is much faster than MD5 and smaller to index.
|
||||
|http://wiki.apache.org/solr/TextProfileSignature[TextProfileSignature] |Fuzzy hashing implementation from Apache Nutch for near duplicate detection. It's tunable but works best on longer text.
|
||||
|===
|
||||
|
||||
Other, more sophisticated algorithms for fuzzy/near hashing can be added later.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
Adding in the de-duplication process will change the `allowDups` setting so that it applies to an update term (with `signatureField` in this case) rather than the unique field Term.
|
||||
|
||||
Of course the `signatureField` could be the unique field, but generally you want the unique field to be unique. When a document is added, a signature will automatically be generated and attached to the document in the specified `signatureField`.
|
||||
====
|
||||
|
||||
[[De-Duplication-ConfigurationOptions]]
|
||||
== Configuration Options
|
||||
|
||||
There are two places in Solr to configure de-duplication: in `solrconfig.xml` and in `schema.xml`.
|
||||
|
||||
[[De-Duplication-Insolrconfig.xml]]
|
||||
=== In `solrconfig.xml`
|
||||
|
||||
The `SignatureUpdateProcessorFactory` has to be registered in `solrconfig.xml` as part of an <<update-request-processors.adoc#update-request-processors,Update Request Processor Chain>>, as in this example:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<updateRequestProcessorChain name="dedupe">
|
||||
<processor class="solr.processor.SignatureUpdateProcessorFactory">
|
||||
<bool name="enabled">true</bool>
|
||||
<str name="signatureField">id</str>
|
||||
<bool name="overwriteDupes">false</bool>
|
||||
<str name="fields">name,features,cat</str>
|
||||
<str name="signatureClass">solr.processor.Lookup3Signature</str>
|
||||
</processor>
|
||||
<processor class="solr.LogUpdateProcessorFactory" />
|
||||
<processor class="solr.RunUpdateProcessorFactory" />
|
||||
</updateRequestProcessorChain>
|
||||
----
|
||||
|
||||
The `SignatureUpdateProcessorFactory` takes several properties:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,30,50",options="header"]
|
||||
|===
|
||||
|Parameter |Default |Description
|
||||
|signatureClass |`org.apache.solr.update.processor.Lookup3Signature` a|
|
||||
A Signature implementation for generating a signature hash. The full classpath of the implementation must be specified. The available options are described above, the associated classpaths to use are:
|
||||
|
||||
* `org.apache.solr.update.processor.Lookup3Signature`
|
||||
* `org.apache.solr.update.processor.MD5Signature`
|
||||
* `org.apache.solr.update.process.TextProfileSignature`
|
||||
|
||||
|fields |all fields |The fields to use to generate the signature hash in a comma separated list. By default, all fields on the document will be used.
|
||||
|signatureField |signatureField |The name of the field used to hold the fingerprint/signature. The field should be defined in schema.xml.
|
||||
|enabled |true |Enable/disable de-duplication processing.
|
||||
|overwriteDupes |true |If true, when a document exists that already matches this signature, it will be overwritten.
|
||||
|===
|
||||
|
||||
[[De-Duplication-Inschema.xml]]
|
||||
=== In `schema.xml`
|
||||
|
||||
If you are using a separate field for storing the signature, you must have it indexed:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<field name="signatureField" type="string" stored="true" indexed="true" multiValued="false" />
|
||||
----
|
||||
|
||||
Be sure to change your update handlers to use the defined chain, as below:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<requestHandler name="/update" class="solr.UpdateRequestHandler" >
|
||||
<lst name="defaults">
|
||||
<str name="update.chain">dedupe</str>
|
||||
</lst>
|
||||
...
|
||||
</requestHandler>
|
||||
----
|
||||
|
||||
This example assumes you have other sections of your request handler defined.
|
||||
|
||||
[TIP]
|
||||
====
|
||||
The update processor can also be specified per request with a parameter of `update.chain=dedupe`.
|
||||
====
|
|
@ -0,0 +1,79 @@
|
|||
= Defining core.properties
|
||||
:page-shortname: defining-core-properties
|
||||
:page-permalink: defining-core-properties.html
|
||||
|
||||
Core discovery means that creating a core is as simple as a `core.properties` file located on disk.
|
||||
|
||||
The `core.properties` file is a simple Java Properties file where each line is just a key=value pair, e.g., `name=core1`. Notice that no quotes are required.
|
||||
|
||||
A minimal `core.properties` file looks like the example below. However, it can also be empty, see information on placement of `core.properties` below.
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
name=my_core_name
|
||||
----
|
||||
|
||||
[[Definingcore.properties-Placementofcore.properties]]
|
||||
== Placement of core.properties
|
||||
|
||||
Solr cores are configured by placing a file named `core.properties` in a sub-directory under `solr.home`. There are no a-priori limits to the depth of the tree, nor are there limits to the number of cores that can be defined. Cores may be anywhere in the tree with the exception that cores may _not_ be defined under an existing core. That is, the following is not allowed:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
./cores/core1/core.properties
|
||||
./cores/core1/coremore/core5/core.properties
|
||||
----
|
||||
|
||||
In this example, the enumeration will stop at "core1".
|
||||
|
||||
The following is legal:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
./cores/somecores/core1/core.properties
|
||||
./cores/somecores/core2/core.properties
|
||||
./cores/othercores/core3/core.properties
|
||||
./cores/extracores/deepertree/core4/core.properties
|
||||
----
|
||||
|
||||
It is possible to segment Solr into multiple cores, each with its own configuration and indices. Cores may be dedicated to a single application or to very different ones, but all are administered through a common administration interface. You can create new Solr cores on the fly, shutdown cores, even replace one running core with another, all without ever stopping or restarting Solr.
|
||||
|
||||
Your `core.properties` file can be empty if necessary. Suppose `core.properties` is located in `./cores/core1` (relative to `solr_home`) but is empty. In this case, the core name is assumed to be "core1". The instanceDir will be the folder containing `core.properties` (i.e., `./cores/core1`). The dataDir will be `../cores/core1/data`, etc.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
You can run Solr without configuring any cores.
|
||||
====
|
||||
|
||||
[[Definingcore.properties-Definingcore.propertiesFiles]]
|
||||
== Defining core.properties Files
|
||||
|
||||
[[Definingcore.properties-core.properties_files]]
|
||||
|
||||
The minimal `core.properties` file is an empty file, in which case all of the properties are defaulted appropriately.
|
||||
|
||||
Java properties files allow the hash (`#`) or bang (`!`) characters to specify comment-to-end-of-line.
|
||||
|
||||
This table defines the recognized properties:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="25,75",options="header"]
|
||||
|===
|
||||
|Property |Description
|
||||
|`name` |The name of the SolrCore. You'll use this name to reference the SolrCore when running commands with the CoreAdminHandler.
|
||||
|`config` |The configuration file name for a given core. The default is `solrconfig.xml`.
|
||||
|`schema` |The schema file name for a given core. The default is `schema.xml` but please note that if you are using a "managed schema" (the default behavior) then any value for this property which does not match the effective `managedSchemaResourceName` will be read once, backed up, and converted for managed schema use. See <<schema-factory-definition-in-solrconfig.adoc#schema-factory-definition-in-solrconfig,Schema Factory Definition in SolrConfig>> for details.
|
||||
|`dataDir` |The core's data directory (where indexes are stored) as either an absolute pathname, or a path relative to the value of `instanceDir`. This is `data` by default.
|
||||
|`configSet` |The name of a defined configset, if desired, to use to configure the core (see the <<config-sets.adoc#config-sets,Config Sets>> for more details).
|
||||
|`properties` |The name of the properties file for this core. The value can be an absolute pathname or a path relative to the value of `instanceDir`.
|
||||
|`transient` |If *true*, the core can be unloaded if Solr reaches the `transientCacheSize`. The default if not specified is *false*. Cores are unloaded in order of least recently used first. _Setting to *true* is not recommended in SolrCloud mode._
|
||||
|`loadOnStartup` |If *true*, the default if it is not specified, the core will loaded when Solr starts. _Setting to *false* is not recommended in SolrCloud mode._
|
||||
|`coreNodeName` |Used only in SolrCloud, this is a unique identifier for the node hosting this replica. By default a coreNodeName is generated automatically, but setting this attribute explicitly allows you to manually assign a new core to replace an existing replica. For example: when replacing a machine that has had a hardware failure by restoring from backups on a new machine with a new hostname or port..
|
||||
|`ulogDir` |The absolute or relative directory for the update log for this core (SolrCloud).
|
||||
|`shard` |The shard to assign this core to (SolrCloud).
|
||||
|`collection` |The name of the collection this core is part of (SolrCloud).
|
||||
|`roles` |Future param for SolrCloud or a way for users to mark nodes for their own use.
|
||||
|===
|
||||
|
||||
Additional "user defined" properties may be specified for use as variables. For more information on how to define local properties, see the section <<configuring-solrconfig-xml.adoc#Configuringsolrconfig.xml-SubstitutingPropertiesinSolrConfigFiles,Substituting Properties in Solr Config Files>>.
|
|
@ -0,0 +1,57 @@
|
|||
= Defining Fields
|
||||
:page-shortname: defining-fields
|
||||
:page-permalink: defining-fields.html
|
||||
|
||||
Fields are defined in the fields element of `schema.xml`. Once you have the field types set up, defining the fields themselves is simple.
|
||||
|
||||
[[DefiningFields-Example]]
|
||||
== Example
|
||||
|
||||
The following example defines a field named `price` with a type named `float` and a default value of `0.0`; the `indexed` and `stored` properties are explicitly set to `true`, while any other properties specified on the `float` field type are inherited.
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<field name="price" type="float" default="0.0" indexed="true" stored="true"/>
|
||||
----
|
||||
|
||||
[[DefiningFields-FieldProperties]]
|
||||
== Field Properties
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="30,70",options="header"]
|
||||
|===
|
||||
|Property |Description
|
||||
|name |The name of the field. Field names should consist of alphanumeric or underscore characters only and not start with a digit. This is not currently strictly enforced, but other field names will not have first class support from all components and back compatibility is not guaranteed. Names with both leading and trailing underscores (e.g., `\_version_`) are reserved. Every field must have a `name`.
|
||||
|type |The name of the `fieldType` for this field. This will be found in the `name` attribute on the `fieldType` definition. Every field must have a `type`.
|
||||
|default |A default value that will be added automatically to any document that does not have a value in this field when it is indexed. If this property is not specified, there is no default.
|
||||
|===
|
||||
|
||||
[[DefiningFields-OptionalFieldTypeOverrideProperties]]
|
||||
== Optional Field Type Override Properties
|
||||
|
||||
Fields can have many of the same properties as field types. Properties from the table below which are specified on an individual field will override any explicit value for that property specified on the the `fieldType` of the field, or any implicit default property value provided by the underlying `fieldType` implementation. The table below is reproduced from <<field-type-definitions-and-properties.adoc#field-type-definitions-and-properties,Field Type Definitions and Properties>>, which has more details:
|
||||
|
||||
// TODO: SOLR-10655 BEGIN: refactor this into a 'field-default-properties.include.adoc' file for reuse
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,40,20,20",options="header"]
|
||||
|===
|
||||
|Property |Description |Values |Implicit Default
|
||||
|indexed |If true, the value of the field can be used in queries to retrieve matching documents. |true or false |true
|
||||
|stored |If true, the actual value of the field can be retrieved by queries. |true or false |true
|
||||
|docValues |If true, the value of the field will be put in a column-oriented <<docvalues.adoc#docvalues,DocValues>> structure. |true or false |false
|
||||
|sortMissingFirst sortMissingLast |Control the placement of documents when a sort field is not present. |true or false |false
|
||||
|multiValued |If true, indicates that a single document might contain multiple values for this field type. |true or false |false
|
||||
|omitNorms |If true, omits the norms associated with this field (this disables length normalization for the field, and saves some memory). *Defaults to true for all primitive (non-analyzed) field types, such as int, float, data, bool, and string.* Only full-text fields or fields need norms. |true or false |*
|
||||
|omitTermFreqAndPositions |If true, omits term frequency, positions, and payloads from postings for this field. This can be a performance boost for fields that don't require that information. It also reduces the storage space required for the index. Queries that rely on position that are issued on a field with this option will silently fail to find documents. *This property defaults to true for all field types that are not text fields.* |true or false |*
|
||||
|omitPositions |Similar to `omitTermFreqAndPositions` but preserves term frequency information. |true or false |*
|
||||
|termVectors termPositions termOffsets termPayloads |These options instruct Solr to maintain full term vectors for each document, optionally including position, offset and payload information for each term occurrence in those vectors. These can be used to accelerate highlighting and other ancillary functionality, but impose a substantial cost in terms of index size. They are not necessary for typical uses of Solr. |true or false |false
|
||||
|required |Instructs Solr to reject any attempts to add a document which does not have a value for this field. This property defaults to false. |true or false |false
|
||||
|useDocValuesAsStored |If the field has `<<docvalues.adoc#docvalues,docValues>>` enabled, setting this to true would allow the field to be returned as if it were a stored field (even if it has `stored=false`) when matching "`*`" in an <<common-query-parameters.adoc#CommonQueryParameters-Thefl_FieldList_Parameter,fl parameter>>. |true or false |true
|
||||
|large |Large fields are always lazy loaded and will only take up space in the document cache if the actual value is < 512KB. This option requires `stored="true"` and `multiValued="false"`. It's intended for fields that might have very large values so that they don't get cached in memory. |true or false |false
|
||||
|===
|
||||
|
||||
// TODO: SOLR-10655 END
|
||||
|
|
@ -0,0 +1,82 @@
|
|||
= Detecting Languages During Indexing
|
||||
:page-shortname: detecting-languages-during-indexing
|
||||
:page-permalink: detecting-languages-during-indexing.html
|
||||
|
||||
Solr can identify languages and map text to language-specific fields during indexing using the `langid` UpdateRequestProcessor.
|
||||
|
||||
Solr supports two implementations of this feature:
|
||||
|
||||
* Tika's language detection feature: http://tika.apache.org/0.10/detection.html
|
||||
* LangDetect language detection: https://github.com/shuyo/language-detection
|
||||
|
||||
You can see a comparison between the two implementations here: http://blog.mikemccandless.com/2011/10/accuracy-and-performance-of-googles.html. In general, the LangDetect implementation supports more languages with higher performance.
|
||||
|
||||
For specific information on each of these language identification implementations, including a list of supported languages for each, see the relevant project websites.
|
||||
|
||||
For more information about language analysis in Solr, see <<language-analysis.adoc#language-analysis,Language Analysis>>.
|
||||
|
||||
[[DetectingLanguagesDuringIndexing-ConfiguringLanguageDetection]]
|
||||
== Configuring Language Detection
|
||||
|
||||
You can configure the `langid` UpdateRequestProcessor in `solrconfig.xml`. Both implementations take the same parameters, which are described in the following section. At a minimum, you must specify the fields for language identification and a field for the resulting language code.
|
||||
|
||||
[[DetectingLanguagesDuringIndexing-ConfiguringTikaLanguageDetection]]
|
||||
=== Configuring Tika Language Detection
|
||||
|
||||
Here is an example of a minimal Tika `langid` configuration in `solrconfig.xml`:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<processor class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
|
||||
<lst name="defaults">
|
||||
<str name="langid.fl">title,subject,text,keywords</str>
|
||||
<str name="langid.langField">language_s</str>
|
||||
</lst>
|
||||
</processor>
|
||||
----
|
||||
|
||||
[[DetectingLanguagesDuringIndexing-ConfiguringLangDetectLanguageDetection]]
|
||||
=== Configuring LangDetect Language Detection
|
||||
|
||||
Here is an example of a minimal LangDetect `langid` configuration in `solrconfig.xml`:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<processor class="org.apache.solr.update.processor.LangDetectLanguageIdentifierUpdateProcessorFactory">
|
||||
<lst name="defaults">
|
||||
<str name="langid.fl">title,subject,text,keywords</str>
|
||||
<str name="langid.langField">language_s</str>
|
||||
</lst>
|
||||
</processor>
|
||||
----
|
||||
|
||||
[[DetectingLanguagesDuringIndexing-langidParameters]]
|
||||
== `langid` Parameters
|
||||
|
||||
As previously mentioned, both implementations of the `langid` UpdateRequestProcessor take the same parameters.
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,10,10,10,50",options="header"]
|
||||
|===
|
||||
|Parameter |Type |Default |Required |Description
|
||||
|langid |Boolean |true |no |Enables and disables language detection.
|
||||
|langid.fl |string |none |yes |A comma- or space-delimited list of fields to be processed by `langid`.
|
||||
|langid.langField |string |none |yes |Specifies the field for the returned language code.
|
||||
|langid.langsField |multivalued string |none |no |Specifies the field for a list of returned language codes. If you use `langid.map.individual`, each detected language will be added to this field.
|
||||
|langid.overwrite |Boolean |false |no |Specifies whether the content of the `langField` and `langsField` fields will be overwritten if they already contain values.
|
||||
|langid.lcmap |string |none |false |A space-separated list specifying colon delimited language code mappings to apply to the detected languages. For example, you might use this to map Chinese, Japanese, and Korean to a common `cjk` code, and map both American and British English to a single `en` code by using `langid.lcmap=ja:cjk zh:cjk ko:cjk en_GB:en en_US:en`. This affects both the values put into the `langField` and `langsField` fields, as well as the field suffixes when using `langid.map`, unless overridden by `langid.map.lcmap`
|
||||
|langid.threshold |float |0.5 |no |Specifies a threshold value between 0 and 1 that the language identification score must reach before `langid` accepts it. With longer text fields, a high threshold such at 0.8 will give good results. For shorter text fields, you may need to lower the threshold for language identification, though you will be risking somewhat lower quality results. We recommend experimenting with your data to tune your results.
|
||||
|langid.whitelist |string |none |no |Specifies a list of allowed language identification codes. Use this in combination with `langid.map` to ensure that you only index documents into fields that are in your schema.
|
||||
|langid.map |Boolean |false |no |Enables field name mapping. If true, Solr will map field names for all fields listed in `langid.fl`.
|
||||
|langid.map.fl |string |none |no |A comma-separated list of fields for `langid.map` that is different than the fields specified in `langid.fl`.
|
||||
|langid.map.keepOrig |Boolean |false |no |If true, Solr will copy the field during the field name mapping process, leaving the original field in place.
|
||||
|langid.map.individual |Boolean |false |no |If true, Solr will detect and map languages for each field individually.
|
||||
|langid.map.individual.fl |string |none |no |A comma-separated list of fields for use with `langid.map.individual` that is different than the fields specified in `langid.fl`.
|
||||
|langid.fallbackFields |string |none |no |If no language is detected that meets the `langid.threshold` score, or if the detected language is not on the `langid.whitelist`, this field specifies language codes to be used as fallback values. If no appropriate fallback languages are found, Solr will use the language code specified in `langid.fallback`.
|
||||
|langid.fallback |string |none |no |Specifies a language code to use if no language is detected or specified in `langid.fallbackFields`.
|
||||
|langid.map.lcmap |string |determined by `langid.lcmap` |no |A space-separated list specifying colon delimited language code mappings to use when mapping field names. For example, you might use this to make Chinese, Japanese, and Korean language fields use a common `*_cjk` suffix, and map both American and British English fields to a single `*_en` by using `langid.map.lcmap=ja:cjk zh:cjk ko:cjk en_GB:en en_US:en`.
|
||||
|langid.map.pattern |Java regular expression |none |no |By default, fields are mapped as <field>_<language>. To change this pattern, you can specify a Java regular expression in this parameter.
|
||||
|langid.map.replace |Java replace |none |no |By default, fields are mapped as <field>_<language>. To change this pattern, you can specify a Java replace in this parameter.
|
||||
|langid.enforceSchema |Boolean |true |no |If false, the `langid` processor does not validate field names against your schema. This may be useful if you plan to rename or delete fields later in the UpdateChain.
|
||||
|===
|
|
@ -0,0 +1,123 @@
|
|||
= Distributed Requests
|
||||
:page-shortname: distributed-requests
|
||||
:page-permalink: distributed-requests.html
|
||||
|
||||
When a Solr node receives a search request, the request is routed behind the scenes to a replica of a shard that is part of the collection being searched.
|
||||
|
||||
The chosen replica acts as an aggregator: it creates internal requests to randomly chosen replicas of every shard in the collection, coordinates the responses, issues any subsequent internal requests as needed (for example, to refine facets values, or request additional stored fields), and constructs the final response for the client.
|
||||
|
||||
[[DistributedRequests-LimitingWhichShardsareQueried]]
|
||||
== Limiting Which Shards are Queried
|
||||
|
||||
While one of the advantages of using SolrCloud is the ability to query very large collections distributed among various shards, in some cases <<shards-and-indexing-data-in-solrcloud.adoc#ShardsandIndexingDatainSolrCloud-DocumentRouting,you may know that you are only interested in results from a subset of your shards>>. You have the option of searching over all of your data or just parts of it.
|
||||
|
||||
Querying all shards for a collection should look familiar; it's as though SolrCloud didn't even come into play:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/gettingstarted/select?q=*:*
|
||||
----
|
||||
|
||||
If, on the other hand, you wanted to search just one shard, you can specify that shard by its logical ID, as in:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=shard1
|
||||
----
|
||||
|
||||
If you want to search a group of shard Ids, you can specify them together:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=shard1,shard2
|
||||
----
|
||||
|
||||
In both of the above examples, the shard Id(s) will be used to pick a random replica of that shard.
|
||||
|
||||
Alternatively, you can specify the explicit replicas you wish to use in place of a shard Ids:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=localhost:7574/solr/gettingstarted,localhost:8983/solr/gettingstarted
|
||||
----
|
||||
|
||||
Or you can specify a list of replicas to choose from for a single shard (for load balancing purposes) by using the pipe symbol (|):
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=localhost:7574/solr/gettingstarted|localhost:7500/solr/gettingstarted
|
||||
----
|
||||
|
||||
And of course, you can specify a list of shards (seperated by commas) each defined by a list of replicas (seperated by pipes). In this example, 2 shards are queried, the first being a random replica from shard1, the second being a random replica from the explicit pipe delimited list:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=shard1,localhost:7574/solr/gettingstarted|localhost:7500/solr/gettingstarted
|
||||
----
|
||||
|
||||
[[DistributedRequests-ConfiguringtheShardHandlerFactory]]
|
||||
== Configuring the ShardHandlerFactory
|
||||
|
||||
You can directly configure aspects of the concurrency and thread-pooling used within distributed search in Solr. This allows for finer grained control and you can tune it to target your own specific requirements. The default configuration favors throughput over latency.
|
||||
|
||||
To configure the standard handler, provide a configuration like this in `solrconfig.xml`:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<requestHandler name="standard" class="solr.SearchHandler" default="true">
|
||||
<!-- other params go here -->
|
||||
<shardHandler class="HttpShardHandlerFactory">
|
||||
<int name="socketTimeOut">1000</int>
|
||||
<int name="connTimeOut">5000</int>
|
||||
</shardHandler>
|
||||
</requestHandler>
|
||||
----
|
||||
|
||||
The parameters that can be specified are as follows:
|
||||
|
||||
// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
|
||||
|
||||
[cols="20,15,65",options="header"]
|
||||
|===
|
||||
|Parameter |Default |Explanation
|
||||
|`socketTimeout` |0 (use OS default) |The amount of time in ms that a socket is allowed to wait.
|
||||
|`connTimeout` |0 (use OS default) |The amount of time in ms that is accepted for binding / connecting a socket
|
||||
|`maxConnectionsPerHost` |20 |The maximum number of concurrent connections that is made to each individual shard in a distributed search.
|
||||
|`maxConnections` |`10000` |The total maximum number of concurrent connections in distributed searches.
|
||||
|`corePoolSize` |0 |The retained lowest limit on the number of threads used in coordinating distributed search.
|
||||
|`maximumPoolSize` |Integer.MAX_VALUE |The maximum number of threads used for coordinating distributed search.
|
||||
|`maxThreadIdleTime` |5 seconds |The amount of time to wait for before threads are scaled back in response to a reduction in load.
|
||||
|`sizeOfQueue` |-1 |If specified, the thread pool will use a backing queue instead of a direct handoff buffer. High throughput systems will want to configure this to be a direct hand off (with -1). Systems that desire better latency will want to configure a reasonable size of queue to handle variations in requests.
|
||||
|`fairnessPolicy` |false |Chooses the JVM specifics dealing with fair policy queuing, if enabled distributed searches will be handled in a First in First out fashion at a cost to throughput. If disabled throughput will be favored over latency.
|
||||
|===
|
||||
|
||||
[[DistributedRequests-ConfiguringstatsCache_DistributedIDF_]]
|
||||
== Configuring statsCache (Distributed IDF)
|
||||
|
||||
Document and term statistics are needed in order to calculate relevancy. Solr provides four implementations out of the box when it comes to document stats calculation:
|
||||
|
||||
* `LocalStatsCache`: This only uses local term and document statistics to compute relevance. In cases with uniform term distribution across shards, this works reasonably well.This option is the default if no `<statsCache>` is configured.
|
||||
* `ExactStatsCache`: This implementation uses global values (across the collection) for document frequency.
|
||||
* `ExactSharedStatsCache`: This is exactly like the exact stats cache in its functionality but the global stats are reused for subsequent requests with the same terms.
|
||||
* `LRUStatsCache`: This implementation uses an LRU cache to hold global stats, which are shared between requests.
|
||||
|
||||
The implementation can be selected by setting `<statsCache>` in `solrconfig.xml`. For example, the following line makes Solr use the `ExactStatsCache` implementation:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<statsCache class="org.apache.solr.search.stats.ExactStatsCache"/>
|
||||
----
|
||||
|
||||
[[DistributedRequests-AvoidingDistributedDeadlock]]
|
||||
== Avoiding Distributed Deadlock
|
||||
|
||||
Each shard serves top-level query requests and then makes sub-requests to all of the other shards. Care should be taken to ensure that the max number of threads serving HTTP requests is greater than the possible number of requests from both top-level clients and other shards. If this is not the case, the configuration may result in a distributed deadlock.
|
||||
|
||||
For example, a deadlock might occur in the case of two shards, each with just a single thread to service HTTP requests. Both threads could receive a top-level request concurrently, and make sub-requests to each other. Because there are no more remaining threads to service requests, the incoming requests will be blocked until the other pending requests are finished, but they will not finish since they are waiting for the sub-requests. By ensuring that Solr is configured to handle a sufficient number of threads, you can avoid deadlock situations like this.
|
||||
|
||||
[[DistributedRequests-PreferLocalShards]]
|
||||
== Prefer Local Shards
|
||||
|
||||
Solr allows you to pass an optional boolean parameter named `preferLocalShards` to indicate that a distributed query should prefer local replicas of a shard when available. In other words, if a query includes `preferLocalShards=true`, then the query controller will look for local replicas to service the query instead of selecting replicas at random from across the cluster. This is useful when a query requests many fields or large fields to be returned per document because it avoids moving large amounts of data over the network when it is available locally. In addition, this feature can be useful for minimizing the impact of a problematic replica with degraded performance, as it reduces the likelihood that the degraded replica will be hit by other healthy replicas.
|
||||
|
||||
Lastly, it follows that the value of this feature diminishes as the number of shards in a collection increases because the query controller will have to direct the query to non-local replicas for most of the shards. In other words, this feature is mostly useful for optimizing queries directed towards collections with a small number of shards and many replicas. Also, this option should only be used if you are load balancing requests across all nodes that host replicas for the collection you are querying, as Solr's CloudSolrClient will do. If not load-balancing, this feature can introduce a hotspot in the cluster since queries won't be evenly distributed across the cluster.
|
|
@ -0,0 +1,165 @@
|
|||
= Distributed Search with Index Sharding
|
||||
:page-shortname: distributed-search-with-index-sharding
|
||||
:page-permalink: distributed-search-with-index-sharding.html
|
||||
|
||||
When using traditional index sharding, you will need to consider how to query your documents.
|
||||
|
||||
It is highly recommended that you use <<solrcloud.adoc#solrcloud,SolrCloud>> when needing to scale up or scale out. The setup described below is legacy and was used prior to the existence of SolrCloud. SolrCloud provides for a truly distributed set of features with support for things like automatic routing, leader election, optimistic concurrency and other sanity checks that are expected out of a distributed system.
|
||||
|
||||
Everything on this page is specific to legacy setup of distributed search. Users trying out SolrCloud should not follow any of the steps or information below.
|
||||
|
||||
Update reorders (i.e., replica A may see update X then Y, and replica B may see update Y then X). *deleteByQuery* also handles reorders the same way, to ensure replicas are consistent. All replicas of a shard are consistent, even if the updates arrive in a different order on different replicas.
|
||||
|
||||
[[DistributedSearchwithIndexSharding-DistributingDocumentsacrossShards]]
|
||||
== Distributing Documents across Shards
|
||||
|
||||
When not using SolrCloud, it is up to you to get all your documents indexed on each shard of your server farm. Solr supports distributed indexing (routing) in its true form only in the SolrCloud mode.
|
||||
|
||||
In the legacy distributed mode, Solr does not calculate universal term/doc frequencies. For most large-scale implementations, it is not likely to matter that Solr calculates TF/IDF at the shard level. However, if your collection is heavily skewed in its distribution across servers, you may find misleading relevancy results in your searches. In general, it is probably best to randomly distribute documents to your shards.
|
||||
|
||||
[[DistributedSearchwithIndexSharding-ExecutingDistributedSearcheswiththeshardsParameter]]
|
||||
== Executing Distributed Searches with the `shards` Parameter
|
||||
|
||||
If a query request includes the `shards` parameter, the Solr server distributes the request across all the shards listed as arguments to the parameter. The `shards` parameter uses this syntax:
|
||||
|
||||
`host:port/base_url,host:port/base_url*`
|
||||
|
||||
For example, the `shards` parameter below causes the search to be distributed across two Solr servers: *solr1* and **solr2**, both of which are running on port 8983:
|
||||
|
||||
`\http://localhost:8983/solr/core1/select?shards=solr1:8983/solr/core1,solr2:8983/solr/core1&indent=true&q=ipod+solr`
|
||||
|
||||
Rather than require users to include the shards parameter explicitly, it is usually preferred to configure this parameter as a default in the RequestHandler section of `solrconfig.xml`.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
Do not add the `shards` parameter to the standard request handler; doing so may cause search queries may enter an infinite loop. Instead, define a new request handler that uses the `shards` parameter, and pass distributed search requests to that handler.
|
||||
====
|
||||
|
||||
With Legacy mode, only query requests are distributed. This includes requests to the SearchHandler (or any handler extending from `org.apache.solr.handler.component.SearchHandler`) using standard components that support distributed search.
|
||||
|
||||
As in SolrCloud mode, when `shards.info=true`, distributed responses will include information about the shard (where each shard represents a logically different index or physical location)
|
||||
|
||||
The following components support distributed search:
|
||||
|
||||
* The *Query* component, which returns documents matching a query
|
||||
* The *Facet* component, which processes facet.query and facet.field requests where facets are sorted by count (the default).
|
||||
* The *Highlighting* component, which enables Solr to include "highlighted" matches in field values.
|
||||
* The *Stats* component, which returns simple statistics for numeric fields within the DocSet.
|
||||
* The *Debug* component, which helps with debugging.
|
||||
|
||||
[[DistributedSearchwithIndexSharding-LimitationstoDistributedSearch]]
|
||||
== Limitations to Distributed Search
|
||||
|
||||
Distributed searching in Solr has the following limitations:
|
||||
|
||||
* Each document indexed must have a unique key.
|
||||
* If Solr discovers duplicate document IDs, Solr selects the first document and discards subsequent ones.
|
||||
* The index for distributed searching may become momentarily out of sync if a commit happens between the first and second phase of the distributed search. This might cause a situation where a document that once matched a query and was subsequently changed may no longer match the query but will still be retrieved. This situation is expected to be quite rare, however, and is only possible for a single query request.
|
||||
* The number of shards is limited by number of characters allowed for GET method's URI; most Web servers generally support at least 4000 characters, but many servers limit URI length to reduce their vulnerability to Denial of Service (DoS) attacks.
|
||||
* Shard information can be returned with each document in a distributed search by including `fl=id, [shard]` in the search request. This returns the shard URL.
|
||||
* In a distributed search, the data directory from the core descriptor overrides any data directory in `solrconfig.xml.`
|
||||
* Update commands may be sent to any server with distributed indexing configured correctly. Document adds and deletes are forwarded to the appropriate server/shard based on a hash of the unique document id. *commit* commands and *deleteByQuery* commands are sent to every server in `shards`.
|
||||
|
||||
Formerly a limitation was that TF/IDF relevancy computations only used shard-local statistics. This is still the case by default. If your data isn't randomly distributed, or if you want more exact statistics, then remember to configure the ExactStatsCache.
|
||||
|
||||
[[DistributedSearchwithIndexSharding-AvoidingDistributedDeadlock]]
|
||||
== Avoiding Distributed Deadlock
|
||||
|
||||
Like in SolrCloud mode, inter-shard requests could lead to a distributed deadlock. It can be avoided by following the instructions in the section <<distributed-requests.adoc#distributed-requests,Distributed Requests>>.
|
||||
|
||||
[[DistributedSearchwithIndexSharding-TestingIndexShardingonTwoLocalServers]]
|
||||
== Testing Index Sharding on Two Local Servers
|
||||
|
||||
For simple functional testing, it's easiest to just set up two local Solr servers on different ports. (In a production environment, of course, these servers would be deployed on separate machines.)
|
||||
|
||||
. Make two Solr home directories and copy `solr.xml` into the new directories:
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
mkdir example/nodes
|
||||
mkdir example/nodes/node1
|
||||
# Copy solr.xml into this solr.home
|
||||
cp server/solr/solr.xml example/nodes/node1/.
|
||||
# Repeat the above steps for the second node
|
||||
mkdir example/nodes/node2
|
||||
cp server/solr/solr.xml example/nodes/node2/.
|
||||
----
|
||||
. Start the two Solr instances
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
# Start first node on port 8983
|
||||
bin/solr start -s example/nodes/node1 -p 8983
|
||||
|
||||
# Start second node on port 8984
|
||||
bin/solr start -s example/nodes/node2 -p 8984
|
||||
----
|
||||
. Create a core on both the nodes with the sample_techproducts_configs.
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
bin/solr create_core -c core1 -p 8983 -d sample_techproducts_configs
|
||||
# Create a core on the Solr node running on port 8984
|
||||
bin/solr create_core -c core1 -p 8984 -d sample_techproducts_configs
|
||||
----
|
||||
. In a third window, index an example document to each of the server:
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
bin/post -c core1 example/exampledocs/monitor.xml -port 8983
|
||||
|
||||
bin/post -c core1 example/exampledocs/monitor2.xml -port 8984
|
||||
----
|
||||
. Search on the node on port 8983:
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/core1/select?q=*:*&wt=xml&indent=true
|
||||
----
|
||||
+
|
||||
This should bring back one document.
|
||||
+
|
||||
Search on the node on port 8984:
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8984/solr/core1/select?q=*:*&wt=xml&indent=true
|
||||
----
|
||||
+
|
||||
This should also bring back a single document.
|
||||
+
|
||||
Now do a distributed search across both servers with your browser or `curl.` In the example below, an extra parameter 'fl' is passed to restrict the returned fields to id and name.
|
||||
+
|
||||
[source,bash]
|
||||
----
|
||||
curl http://localhost:8983/solr/core1/select?q=*:*&indent=true&shards=localhost:8983/solr/core1,localhost:8984/solr/core1&fl=id,name
|
||||
----
|
||||
+
|
||||
This should contain both the documents as shown below:
|
||||
+
|
||||
[source,xml]
|
||||
----
|
||||
<response>
|
||||
<lst name="responseHeader">
|
||||
<int name="status">0</int>
|
||||
<int name="QTime">8</int>
|
||||
<lst name="params">
|
||||
<str name="q">*:*</str>
|
||||
<str name="shards">localhost:8983/solr/core1,localhost:8984/solr/core1</str>
|
||||
<str name="indent">true</str>
|
||||
<str name="fl">id,name</str>
|
||||
<str name="wt">xml</str>
|
||||
</lst>
|
||||
</lst>
|
||||
<result name="response" numFound="2" start="0" maxScore="1.0">
|
||||
<doc>
|
||||
<str name="id">3007WFP</str>
|
||||
<str name="name">Dell Widescreen UltraSharp 3007WFP</str>
|
||||
</doc>
|
||||
<doc>
|
||||
<str name="id">VA902B</str>
|
||||
<str name="name">ViewSonic VA902B - flat panel display - TFT - 19"</str>
|
||||
</doc>
|
||||
</result>
|
||||
</response>
|
||||
----
|
|
@ -0,0 +1,28 @@
|
|||
= Documents, Fields, and Schema Design
|
||||
:page-shortname: documents-fields-and-schema-design
|
||||
:page-permalink: documents-fields-and-schema-design.html
|
||||
:page-children: overview-of-documents-fields-and-schema-design, solr-field-types, defining-fields, copying-fields, dynamic-fields, other-schema-elements, schema-api, putting-the-pieces-together, docvalues, schemaless-mode
|
||||
|
||||
This section discusses how Solr organizes its data into documents and fields, as well as how to work with a schema in Solr.
|
||||
|
||||
This section includes the following topics:
|
||||
|
||||
<<overview-of-documents-fields-and-schema-design.adoc#overview-of-documents-fields-and-schema-design,Overview of Documents, Fields, and Schema Design>>: An introduction to the concepts covered in this section.
|
||||
|
||||
<<solr-field-types.adoc#solr-field-types,Solr Field Types>>: Detailed information about field types in Solr, including the field types in the default Solr schema.
|
||||
|
||||
<<defining-fields.adoc#defining-fields,Defining Fields>>: Describes how to define fields in Solr.
|
||||
|
||||
<<copying-fields.adoc#copying-fields,Copying Fields>>: Describes how to populate fields with data copied from another field.
|
||||
|
||||
<<dynamic-fields.adoc#dynamic-fields,Dynamic Fields>>: Information about using dynamic fields in order to catch and index fields that do not exactly conform to other field definitions in your schema.
|
||||
|
||||
<<schema-api.adoc#schema-api,Schema API>>: Use curl commands to read various parts of a schema or create new fields and copyField rules.
|
||||
|
||||
<<other-schema-elements.adoc#other-schema-elements,Other Schema Elements>>: Describes other important elements in the Solr schema.
|
||||
|
||||
<<putting-the-pieces-together.adoc#putting-the-pieces-together,Putting the Pieces Together>>: A higher-level view of the Solr schema and how its elements work together.
|
||||
|
||||
<<docvalues.adoc#docvalues,DocValues>>: Describes how to create a docValues index for faster lookups.
|
||||
|
||||
<<schemaless-mode.adoc#schemaless-mode,Schemaless Mode>>: Automatically add previously unknown schema fields using value-based field type guessing.
|
|
@ -0,0 +1,73 @@
|
|||
= Documents Screen
|
||||
:page-shortname: documents-screen
|
||||
:page-permalink: documents-screen.html
|
||||
|
||||
The Documents screen provides a simple form allowing you to execute various Solr indexing commands in a variety of formats directly from the browser.
|
||||
|
||||
.The Documents Screen
|
||||
image::images/documents-screen/documents_add_screen.png[image,height=400]
|
||||
|
||||
The screen allows you to:
|
||||
|
||||
* Copy documents in JSON, CSV or XML and submit them to the index
|
||||
* Upload documents (in JSON, CSV or XML)
|
||||
* Construct documents by selecting fields and field values
|
||||
|
||||
|
||||
[TIP]
|
||||
====
|
||||
There are other ways to load data, see also these sections:
|
||||
|
||||
* <<uploading-data-with-index-handlers.adoc#uploading-data-with-index-handlers,Uploading Data with Index Handlers>>
|
||||
* <<uploading-data-with-solr-cell-using-apache-tika.adoc#uploading-data-with-solr-cell-using-apache-tika,Uploading Data with Solr Cell using Apache Tika>>
|
||||
====
|
||||
|
||||
The first step is to define the RequestHandler to use (aka, 'qt'). By default `/update` will be defined. To use Solr Cell, for example, change the request handler to `/update/extract`.
|
||||
|
||||
Then choose the Document Type to define the type of document to load. The remaining parameters will change depending on the document type selected.
|
||||
|
||||
[[DocumentsScreen-JSON]]
|
||||
== JSON
|
||||
|
||||
When using the JSON document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper JSON format.
|
||||
|
||||
Then you can choose when documents should be added to the index (Commit Within), & whether existing documents should be overwritten with incoming documents with the same id (if this is not *true*, then the incoming documents will be dropped).
|
||||
|
||||
This option will only add or overwrite documents to the index; for other update tasks, see the <<DocumentsScreen-SolrCommand,Solr Command>> option.
|
||||
|
||||
[[DocumentsScreen-CSV]]
|
||||
== CSV
|
||||
|
||||
When using the CSV document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper CSV format, with columns delimited and one row per document.
|
||||
|
||||
Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not *true*, then the incoming documents will be dropped).
|
||||
|
||||
[[DocumentsScreen-DocumentBuilder]]
|
||||
== Document Builder
|
||||
|
||||
The Document Builder provides a wizard-like interface to enter fields of a document
|
||||
|
||||
[[DocumentsScreen-FileUpload]]
|
||||
== File Upload
|
||||
|
||||
The File Upload option allows choosing a prepared file and uploading it. If using only `/update` for the Request-Handler option, you will be limited to XML, CSV, and JSON.
|
||||
|
||||
However, to use the ExtractingRequestHandler (aka Solr Cell), you can modify the Request-Handler to `/update/extract`. You must have this defined in your `solrconfig.xml` file, with your desired defaults. You should also update the `&literal.id` shown in the Extracting Req. Handler Params so the file chosen is given a unique id.
|
||||
|
||||
Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not *true*, then the incoming documents will be dropped).
|
||||
|
||||
[[DocumentsScreen-SolrCommand]]
|
||||
== Solr Command
|
||||
|
||||
The Solr Command option allows you use XML or JSON to perform specific actions on documents, such as defining documents to be added or deleted, updating only certain fields of documents, or commit and optimize commands on the index.
|
||||
|
||||
The documents should be structured as they would be if using `/update` on the command line.
|
||||
|
||||
[[DocumentsScreen-XML]]
|
||||
== XML
|
||||
|
||||
When using the XML document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper Solr XML format, with each document separated by `<doc>` tags and each field defined.
|
||||
|
||||
Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not **true**, then the incoming documents will be dropped).
|
||||
|
||||
This option will only add or overwrite documents to the index; for other update tasks, see the <<DocumentsScreen-SolrCommand,Solr Command>> option.
|
|
@ -0,0 +1,75 @@
|
|||
= DocValues
|
||||
:page-shortname: docvalues
|
||||
:page-permalink: docvalues.html
|
||||
|
||||
DocValues are a way of recording field values internally that is more efficient for some purposes, such as sorting and faceting, than traditional indexing.
|
||||
|
||||
== Why DocValues?
|
||||
|
||||
The standard way that Solr builds the index is with an _inverted index_. This style builds a list of terms found in all the documents in the index and next to each term is a list of documents that the term appears in (as well as how many times the term appears in that document). This makes search very fast - since users search by terms, having a ready list of term-to-document values makes the query process faster.
|
||||
|
||||
For other features that we now commonly associate with search, such as sorting, faceting, and highlighting, this approach is not very efficient. The faceting engine, for example, must look up each term that appears in each document that will make up the result set and pull the document IDs in order to build the facet list. In Solr, this is maintained in memory, and can be slow to load (depending on the number of documents, terms, etc.).
|
||||
|
||||
In Lucene 4.0, a new approach was introduced. DocValue fields are now column-oriented fields with a document-to-value mapping built at index time. This approach promises to relieve some of the memory requirements of the fieldCache and make lookups for faceting, sorting, and grouping much faster.
|
||||
|
||||
[[DocValues-EnablingDocValues]]
|
||||
== Enabling DocValues
|
||||
|
||||
To use docValues, you only need to enable it for a field that you will use it with. As with all schema design, you need to define a field type and then define fields of that type with docValues enabled. All of these actions are done in `schema.xml`.
|
||||
|
||||
Enabling a field for docValues only requires adding `docValues="true"` to the field (or field type) definition, as in this example from the `schema.xml` of Solr's `sample_techproducts_configs` <<config-sets.adoc#config-sets,config set>>:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<field name="manu_exact" type="string" indexed="false" stored="false" docValues="true" />
|
||||
----
|
||||
|
||||
[IMPORTANT]
|
||||
If you have already indexed data into your Solr index, you will need to completely re-index your content after changing your field definitions in `schema.xml` in order to successfully use docValues.
|
||||
|
||||
DocValues are only available for specific field types. The types chosen determine the underlying Lucene docValue type that will be used. The available Solr field types are:
|
||||
|
||||
* `StrField` and `UUIDField`.
|
||||
** If the field is single-valued (i.e., multi-valued is false), Lucene will use the SORTED type.
|
||||
** If the field is multi-valued, Lucene will use the SORTED_SET type.
|
||||
* Any `Trie*` numeric fields, date fields and `EnumField`.
|
||||
** If the field is single-valued (i.e., multi-valued is false), Lucene will use the NUMERIC type.
|
||||
** If the field is multi-valued, Lucene will use the SORTED_SET type.
|
||||
* Boolean fields
|
||||
* Int|Long|Float|Double|Date PointField
|
||||
** If the field is single-valued (i.e., multi-valued is false), Lucene will use the NUMERIC type.
|
||||
** If the field is multi-valued, Lucene will use the SORTED_NUMERIC type.
|
||||
|
||||
These Lucene types are related to how the {lucene-javadocs}/core/org/apache/lucene/index/DocValuesType.html[values are sorted and stored].
|
||||
|
||||
There is an additional configuration option available, which is to modify the `docValuesFormat` <<field-type-definitions-and-properties.adoc#FieldTypeDefinitionsandProperties-docValuesFormat,used by the field type>>. The default implementation employs a mixture of loading some things into memory and keeping some on disk. In some cases, however, you may choose to specify an alternative {lucene-javadocs}/core/org/apache/lucene/codecs/DocValuesFormat.html[DocValuesFormat implementation]. For example, you could choose to keep everything in memory by specifying `docValuesFormat="Memory"` on a field type:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<fieldType name="string_in_mem_dv" class="solr.StrField" docValues="true" docValuesFormat="Memory" />
|
||||
----
|
||||
|
||||
Please note that the `docValuesFormat` option may change in future releases.
|
||||
|
||||
[NOTE]
|
||||
Lucene index back-compatibility is only supported for the default codec. If you choose to customize the `docValuesFormat` in your schema.xml, upgrading to a future version of Solr may require you to either switch back to the default codec and optimize your index to rewrite it into the default codec before upgrading, or re-build your entire index from scratch after upgrading.
|
||||
|
||||
== Using DocValues
|
||||
|
||||
=== Sorting, Faceting & Functions
|
||||
|
||||
If `docValues="true"` for a field, then DocValues will automatically be used any time the field is used for <<common-query-parameters.adoc#CommonQueryParameters-ThesortParameter,sorting>>, <<faceting.adoc#faceting,faceting>> or <<function-queries.adoc#function-queries,function queries>>.
|
||||
|
||||
[[DocValues-RetrievingDocValuesDuringSearch]]
|
||||
=== Retrieving DocValues During Search
|
||||
|
||||
Field values retrieved during search queries are typically returned from stored values. However, non-stored docValues fields will be also returned along with other stored fields when all fields (or pattern matching globs) are specified to be returned (e.g. "`fl=*`") for search queries depending on the effective value of the `useDocValuesAsStored` parameter for each field. For schema versions >= 1.6, the implicit default is `useDocValuesAsStored="true"`. See <<field-type-definitions-and-properties.adoc#field-type-definitions-and-properties,Field Type Definitions and Properties>> & <<defining-fields.adoc#defining-fields,Defining Fields>> for more details.
|
||||
|
||||
When `useDocValuesAsStored="false"`, non-stored DocValues fields can still be explicitly requested by name in the <<common-query-parameters.adoc#CommonQueryParameters-Thefl_FieldList_Parameter,fl param>>, but will not match glob patterns (`"*"`). Note that returning DocValues along with "regular" stored fields at query time has performance implications that stored fields may not because DocValues are column-oriented and may therefore incur additional cost to retrieve for each returned document. Also note that while returning non-stored fields from DocValues, the values of a multi-valued field are returned in sorted order (and not insertion order). If you require the multi-valued fields to be returned in the original insertion order, then make your multi-valued field as stored (such a change requires re-indexing).
|
||||
|
||||
In cases where the query is returning _only_ docValues fields performance may improve since returning stored fields requires disk reads and decompression whereas returning docValues fields in the fl list only requires memory access.
|
||||
|
||||
When retrieving fields from their docValues form (using the <<exporting-result-sets.adoc#exporting-result-sets,/export handler>>, <<streaming-expressions.adoc#streaming-expressions,streaming expressions>> or if the field is requested in the `fl` parameter), two important differences between regular stored fields and docValues fields must be understood:
|
||||
|
||||
1. Order is _not_ preserved. For simply retrieving stored fields, the insertion order is the return order. For docValues, it is the _sorted_ order.
|
||||
2. Multiple identical entries are collapsed into a single value. Thus if I insert values 4, 5, 2, 4, 1, my return will be 1, 2, 4, 5.
|
|
@ -0,0 +1,20 @@
|
|||
= Dynamic Fields
|
||||
:page-shortname: dynamic-fields
|
||||
:page-permalink: dynamic-fields.html
|
||||
|
||||
_Dynamic fields_ allow Solr to index fields that you did not explicitly define in your schema.
|
||||
|
||||
This is useful if you discover you have forgotten to define one or more fields. Dynamic fields can make your application less brittle by providing some flexibility in the documents you can add to Solr.
|
||||
|
||||
A dynamic field is just like a regular field except it has a name with a wildcard in it. When you are indexing documents, a field that does not match any explicitly defined fields can be matched with a dynamic field.
|
||||
|
||||
For example, suppose your schema includes a dynamic field with a name of `*_i`. If you attempt to index a document with a `cost_i` field, but no explicit `cost_i` field is defined in the schema, then the `cost_i` field will have the field type and analysis defined for `*_i`.
|
||||
|
||||
Like regular fields, dynamic fields have a name, a field type, and options.
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dynamicField name="*_i" type="int" indexed="true" stored="true"/>
|
||||
----
|
||||
|
||||
It is recommended that you include basic dynamic field mappings (like that shown above) in your `schema.xml`. The mappings can be very useful.
|
|
@ -0,0 +1,345 @@
|
|||
= Enabling SSL
|
||||
:page-shortname: enabling-ssl
|
||||
:page-permalink: enabling-ssl.html
|
||||
|
||||
Solr can encrypt communications to and from clients, and between nodes in SolrCloud mode, with SSL.
|
||||
|
||||
This section describes enabling SSL with the example Jetty server using a self-signed certificate.
|
||||
|
||||
For background on SSL certificates and keys, see http://www.tldp.org/HOWTO/SSL-Certificates-HOWTO/.
|
||||
|
||||
[[EnablingSSL-BasicSSLSetup]]
|
||||
== Basic SSL Setup
|
||||
|
||||
[[EnablingSSL-Generateaself-signedcertificateandakey]]
|
||||
=== Generate a Self-Signed Certificate and a Key
|
||||
|
||||
To generate a self-signed certificate and a single key that will be used to authenticate both the server and the client, we'll use the JDK https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html[`keytool`] command and create a separate keystore. This keystore will also be used as a truststore below. It's possible to use the keystore that comes with the JDK for these purposes, and to use a separate truststore, but those options aren't covered here.
|
||||
|
||||
Run the commands below in the `server/etc/` directory in the binary Solr distribution. It's assumed that you have the JDK `keytool` utility on your `PATH`, and that `openssl` is also on your `PATH`. See https://www.openssl.org/related/binaries.html for OpenSSL binaries for Windows and Solaris.
|
||||
|
||||
The `-ext SAN=...` `keytool` option allows you to specify all the DNS names and/or IP addresses that will be allowed during hostname verification (but see below for how to skip hostname verification between Solr nodes so that you don't have to specify all hosts here).
|
||||
|
||||
In addition to `localhost` and `127.0.0.1`, this example includes a LAN IP address `192.168.1.3` for the machine the Solr nodes will be running on:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass secret -storepass secret -validity 9999 -keystore solr-ssl.keystore.jks -ext SAN=DNS:localhost,IP:192.168.1.3,IP:127.0.0.1 -dname "CN=localhost, OU=Organizational Unit, O=Organization, L=Location, ST=State, C=Country"
|
||||
----
|
||||
|
||||
The above command will create a keystore file named `solr-ssl.keystore.jks` in the current directory.
|
||||
|
||||
[[EnablingSSL-ConvertthecertificateandkeytoPEMformatforusewithcURL]]
|
||||
=== Convert the Certificate and Key to PEM Format for Use with cURL
|
||||
|
||||
cURL isn't capable of using JKS formatted keystores, so the JKS keystore needs to be converted to PEM format, which cURL understands.
|
||||
|
||||
First convert the JKS keystore into PKCS12 format using `keytool`:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
keytool -importkeystore -srckeystore solr-ssl.keystore.jks -destkeystore solr-ssl.keystore.p12 -srcstoretype jks -deststoretype pkcs12
|
||||
----
|
||||
|
||||
The keytool application will prompt you to create a destination keystore password and for the source keystore password, which was set when creating the keystore ("secret" in the example shown above).
|
||||
|
||||
Next convert the PKCS12 format keystore, including both the certificate and the key, into PEM format using the http://www.openssl.org[`openssl`] command:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
openssl pkcs12 -in solr-ssl.keystore.p12 -out solr-ssl.pem
|
||||
----
|
||||
|
||||
If you want to use cURL on OS X Yosemite (10.10), you'll need to create a certificate-only version of the PEM format, as follows:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
openssl pkcs12 -nokeys -in solr-ssl.keystore.p12 -out solr-ssl.cacert.pem
|
||||
----
|
||||
|
||||
[[EnablingSSL-SetcommonSSLrelatedsystemproperties]]
|
||||
=== Set common SSL related system properties
|
||||
|
||||
The Solr Control Script is already setup to pass SSL-related Java system properties to the JVM. To activate the SSL settings, uncomment and update the set of properties beginning with SOLR_SSL_* in `bin/solr.in.sh`. (or `bin\solr.in.cmd` on Windows).
|
||||
|
||||
NOTE: If you setup Solr as a service on Linux using the steps outlined in <<taking-solr-to-production.adoc#taking-solr-to-production,Taking Solr to Production>>, then make these changes in `/var/solr/solr.in.sh` instead.
|
||||
|
||||
.bin/solr.in.sh example SOLR_SSL_* configuration
|
||||
[source,bash]
|
||||
----
|
||||
SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.jks
|
||||
SOLR_SSL_KEY_STORE_PASSWORD=secret
|
||||
SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.jks
|
||||
SOLR_SSL_TRUST_STORE_PASSWORD=secret
|
||||
# Require clients to authenticate
|
||||
SOLR_SSL_NEED_CLIENT_AUTH=false
|
||||
# Enable clients to authenticate (but not require)
|
||||
SOLR_SSL_WANT_CLIENT_AUTH=false
|
||||
# Define Key Store type if necessary
|
||||
SOLR_SSL_KEY_STORE_TYPE=JKS
|
||||
SOLR_SSL_TRUST_STORE_TYPE=JKS
|
||||
----
|
||||
|
||||
When you start Solr, the `bin/solr` script includes the settings in `bin/solr.in.sh` and will pass these SSL-related system properties to the JVM.
|
||||
|
||||
.Client Authentication Settings
|
||||
WARNING: Enable either SOLR_SSL_NEED_CLIENT_AUTH or SOLR_SSL_WANT_CLIENT_AUTH but not both at the same time. They are mutually exclusive and Jetty will select one of them which may not be what you expect.
|
||||
|
||||
Similarly, when you start Solr on Windows, the `bin\solr.cmd` script includes the settings in `bin\solr.in.cmd` - uncomment and update the set of properties beginning with `SOLR_SSL_*` to pass these SSL-related system properties to the JVM:
|
||||
|
||||
.bin\solr.in.cmd example SOLR_SSL_* configuration
|
||||
[source,text]
|
||||
----
|
||||
set SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.jks
|
||||
set SOLR_SSL_KEY_STORE_PASSWORD=secret
|
||||
set SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.jks
|
||||
set SOLR_SSL_TRUST_STORE_PASSWORD=secret
|
||||
REM Require clients to authenticate
|
||||
set SOLR_SSL_NEED_CLIENT_AUTH=false
|
||||
REM Enable clients to authenticate (but not require)
|
||||
set SOLR_SSL_WANT_CLIENT_AUTH=false
|
||||
----
|
||||
|
||||
[[EnablingSSL-RunSingleNodeSolrusingSSL]]
|
||||
=== Run Single Node Solr using SSL
|
||||
|
||||
Start Solr using the command shown below; by default clients will not be required to authenticate:
|
||||
|
||||
.*nix command
|
||||
[source,bash]
|
||||
----
|
||||
bin/solr -p 8984
|
||||
----
|
||||
|
||||
.Windows command
|
||||
[source,text]
|
||||
----
|
||||
bin\solr.cmd -p 8984
|
||||
----
|
||||
|
||||
[[EnablingSSL-SolrCloud]]
|
||||
== SolrCloud
|
||||
|
||||
This section describes how to run a two-node SolrCloud cluster with no initial collections and a single-node external ZooKeeper. The commands below assume you have already created the keystore described above.
|
||||
|
||||
[[EnablingSSL-ConfigureZooKeeper]]
|
||||
=== Configure ZooKeeper
|
||||
|
||||
NOTE: ZooKeeper does not support encrypted communication with clients like Solr. There are several related JIRA tickets where SSL support is being planned/worked on: https://issues.apache.org/jira/browse/ZOOKEEPER-235[ZOOKEEPER-235]; https://issues.apache.org/jira/browse/ZOOKEEPER-236[ZOOKEEPER-236]; https://issues.apache.org/jira/browse/ZOOKEEPER-1000[ZOOKEEPER-1000]; and https://issues.apache.org/jira/browse/ZOOKEEPER-2120[ZOOKEEPER-2120].
|
||||
|
||||
Before you start any SolrCloud nodes, you must configure your solr cluster properties in ZooKeeper, so that Solr nodes know to communicate via SSL.
|
||||
|
||||
This section assumes you have created and started a single-node external ZooKeeper on port 2181 on localhost - see <<setting-up-an-external-zookeeper-ensemble.adoc#setting-up-an-external-zookeeper-ensemble,Setting Up an External ZooKeeper Ensemble>>.
|
||||
|
||||
The `urlScheme` cluster-wide property needs to be set to `https` before any Solr node starts up. The example below uses the `zkcli` tool that comes with the binary Solr distribution to do this:
|
||||
|
||||
.*nix command
|
||||
[source,bash]
|
||||
----
|
||||
server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd clusterprop -name urlScheme -val https
|
||||
----
|
||||
|
||||
.Windows command
|
||||
[source,text]
|
||||
----
|
||||
server\scripts\cloud-scripts\zkcli.bat -zkhost localhost:2181 -cmd clusterprop -name urlScheme -val https
|
||||
----
|
||||
|
||||
If you have set up your ZooKeeper cluster to use a <<taking-solr-to-production.adoc#TakingSolrtoProduction-ZooKeeperchroot,chroot for Solr>> , make sure you use the correct `zkhost` string with `zkcli`, e.g. `-zkhost localhost:2181/solr`.
|
||||
|
||||
[[EnablingSSL-RunSolrCloudwithSSL]]
|
||||
=== Run SolrCloud with SSL
|
||||
|
||||
[[EnablingSSL-CreateSolrhomedirectoriesfortwonodes]]
|
||||
==== Create Solr home directories for two nodes
|
||||
|
||||
Create two copies of the `server/solr/` directory which will serve as the Solr home directories for each of your two SolrCloud nodes:
|
||||
|
||||
.*nix commands
|
||||
[source,bash]
|
||||
----
|
||||
mkdir cloud
|
||||
cp -r server/solr cloud/node1
|
||||
cp -r server/solr cloud/node2
|
||||
----
|
||||
|
||||
.Windows commands
|
||||
[source,text]
|
||||
----
|
||||
mkdir cloud
|
||||
xcopy /E server\solr cloud\node1\
|
||||
xcopy /E server\solr cloud\node2\
|
||||
----
|
||||
|
||||
[[EnablingSSL-StartthefirstSolrnode]]
|
||||
==== Start the First Solr Node
|
||||
|
||||
Next, start the first Solr node on port 8984. Be sure to stop the standalone server first if you started it when working through the previous section on this page.
|
||||
|
||||
.*nix command
|
||||
[source,bash]
|
||||
----
|
||||
bin/solr -cloud -s cloud/node1 -z localhost:2181 -p 8984
|
||||
----
|
||||
|
||||
.Windows command
|
||||
[source,text]
|
||||
----
|
||||
bin\solr.cmd -cloud -s cloud\node1 -z localhost:2181 -p 8984
|
||||
----
|
||||
|
||||
Notice the use of the `-s` option to set the location of the Solr home directory for node1.
|
||||
|
||||
If you created your SSL key without all DNS names/IP addresses on which Solr nodes will run, you can tell Solr to skip hostname verification for inter-Solr-node communications by setting the `solr.ssl.checkPeerName` system property to `false`:
|
||||
|
||||
.*nix command
|
||||
[source,bash]
|
||||
----
|
||||
bin/solr -cloud -s cloud/node1 -z localhost:2181 -p 8984 -Dsolr.ssl.checkPeerName=false
|
||||
----
|
||||
|
||||
.Windows command
|
||||
[source,text]
|
||||
----
|
||||
bin\solr.cmd -cloud -s cloud\node1 -z localhost:2181 -p 8984 -Dsolr.ssl.checkPeerName=false
|
||||
----
|
||||
|
||||
[[EnablingSSL-StartthesecondSolrnode]]
|
||||
==== Start the Second Solr Node
|
||||
|
||||
Finally, start the second Solr node on port 7574 - again, to skip hostname verification, add `-Dsolr.ssl.checkPeerName=false`;
|
||||
|
||||
.*nix command
|
||||
[source,text]
|
||||
----
|
||||
bin/solr -cloud -s cloud/node2 -z localhost:2181 -p 7574
|
||||
----
|
||||
|
||||
.Windows command
|
||||
[source,text]
|
||||
----
|
||||
bin\solr.cmd -cloud -s cloud\node2 -z localhost:2181 -p 7574
|
||||
----
|
||||
|
||||
[[EnablingSSL-ExampleClientActions]]
|
||||
== Example Client Actions
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
cURL on OS X Mavericks (10.9) has degraded SSL support. For more information and workarounds to allow one-way SSL, see http://curl.haxx.se/mail/archive-2013-10/0036.html. cURL on OS X Yosemite (10.10) is improved - 2-way SSL is possible - see http://curl.haxx.se/mail/archive-2014-10/0053.html .
|
||||
|
||||
The cURL commands in the following sections will not work with the system `curl` on OS X Yosemite (10.10). Instead, the certificate supplied with the `-E` param must be in PKCS12 format, and the file supplied with the `--cacert` param must contain only the CA certificate, and no key (see <<EnablingSSL-ConvertthecertificateandkeytoPEMformatforusewithcURL,above>> for instructions on creating this file):
|
||||
|
||||
[source,bash]
|
||||
curl -E solr-ssl.keystore.p12:secret --cacert solr-ssl.cacert.pem ...
|
||||
|
||||
====
|
||||
|
||||
NOTE: If your operating system does not include cURL, you can download binaries here: http://curl.haxx.se/download.html
|
||||
|
||||
=== Create a SolrCloud Collection using `bin/solr`
|
||||
|
||||
Create a 2-shard, replicationFactor=1 collection named mycollection using the default configset (data_driven_schema_configs):
|
||||
|
||||
.*nix command
|
||||
[source,bash]
|
||||
----
|
||||
bin/solr create -c mycollection -shards 2
|
||||
----
|
||||
|
||||
.Windows command
|
||||
[source,text]
|
||||
----
|
||||
bin\solr.cmd create -c mycollection -shards 2
|
||||
----
|
||||
|
||||
The `create` action will pass the `SOLR_SSL_*` properties set in your include file to the SolrJ code used to create the collection.
|
||||
|
||||
[[EnablingSSL-RetrieveSolrCloudclusterstatususingcURL]]
|
||||
=== Retrieve SolrCloud Cluster Status using cURL
|
||||
|
||||
To get the resulting cluster status (again, if you have not enabled client authentication, remove the `-E solr-ssl.pem:secret` option):
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -E solr-ssl.pem:secret --cacert solr-ssl.pem "https://localhost:8984/solr/admin/collections?action=CLUSTERSTATUS&wt=json&indent=on"
|
||||
----
|
||||
|
||||
You should get a response that looks like this:
|
||||
|
||||
[source,json]
|
||||
----
|
||||
{
|
||||
"responseHeader":{
|
||||
"status":0,
|
||||
"QTime":2041},
|
||||
"cluster":{
|
||||
"collections":{
|
||||
"mycollection":{
|
||||
"shards":{
|
||||
"shard1":{
|
||||
"range":"80000000-ffffffff",
|
||||
"state":"active",
|
||||
"replicas":{"core_node1":{
|
||||
"state":"active",
|
||||
"base_url":"https://127.0.0.1:8984/solr",
|
||||
"core":"mycollection_shard1_replica1",
|
||||
"node_name":"127.0.0.1:8984_solr",
|
||||
"leader":"true"}}},
|
||||
"shard2":{
|
||||
"range":"0-7fffffff",
|
||||
"state":"active",
|
||||
"replicas":{"core_node2":{
|
||||
"state":"active",
|
||||
"base_url":"https://127.0.0.1:7574/solr",
|
||||
"core":"mycollection_shard2_replica1",
|
||||
"node_name":"127.0.0.1:7574_solr",
|
||||
"leader":"true"}}}},
|
||||
"maxShardsPerNode":"1",
|
||||
"router":{"name":"compositeId"},
|
||||
"replicationFactor":"1"}},
|
||||
"properties":{"urlScheme":"https"}}}
|
||||
----
|
||||
|
||||
[[EnablingSSL-Indexdocumentsusingpost.jar]]
|
||||
=== Index Documents using `post.jar`
|
||||
|
||||
Use `post.jar` to index some example documents to the SolrCloud collection created above:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
cd example/exampledocs
|
||||
|
||||
java -Djavax.net.ssl.keyStorePassword=secret -Djavax.net.ssl.keyStore=../../server/etc/solr-ssl.keystore.jks -Djavax.net.ssl.trustStore=../../server/etc/solr-ssl.keystore.jks -Djavax.net.ssl.trustStorePassword=secret -Durl=https://localhost:8984/solr/mycollection/update -jar post.jar *.xml
|
||||
----
|
||||
|
||||
[[EnablingSSL-QueryusingcURL]]
|
||||
=== Query Using cURL
|
||||
|
||||
Use cURL to query the SolrCloud collection created above, from a directory containing the PEM formatted certificate and key created above (e.g. `example/etc/`) - if you have not enabled client authentication (system property `-Djetty.ssl.clientAuth=true)`, then you can remove the `-E solr-ssl.pem:secret` option:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
curl -E solr-ssl.pem:secret --cacert solr-ssl.pem "https://localhost:8984/solr/mycollection/select?q=*:*&wt=json&indent=on"
|
||||
----
|
||||
|
||||
[[EnablingSSL-IndexadocumentusingCloudSolrClient]]
|
||||
=== Index a document using `CloudSolrClient`
|
||||
|
||||
From a java client using SolrJ, index a document. In the code below, the `javax.net.ssl.*` system properties are set programmatically, but you could instead specify them on the java command line, as in the `post.jar` example above:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
System.setProperty("javax.net.ssl.keyStore", "/path/to/solr-ssl.keystore.jks");
|
||||
System.setProperty("javax.net.ssl.keyStorePassword", "secret");
|
||||
System.setProperty("javax.net.ssl.trustStore", "/path/to/solr-ssl.keystore.jks");
|
||||
System.setProperty("javax.net.ssl.trustStorePassword", "secret");
|
||||
String zkHost = "127.0.0.1:2181";
|
||||
CloudSolrClient client = new CloudSolrClient.Builder().withZkHost(zkHost).build();
|
||||
client.setDefaultCollection("mycollection");
|
||||
SolrInputDocument doc = new SolrInputDocument();
|
||||
doc.addField("id", "1234");
|
||||
doc.addField("name", "A lovely summer holiday");
|
||||
client.add(doc);
|
||||
client.commit();
|
||||
----
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue