SOLR-12118: Solr Ref-Guide can now use some ivy version props directly as attributes in content

This commit is contained in:
Chris Hostetter 2018-03-19 10:13:34 -07:00
parent aae07d9572
commit d6ed71b5c4
14 changed files with 56 additions and 34 deletions

View File

@ -82,7 +82,11 @@ io.prometheus.version = 0.2.0
/javax.servlet/servlet-api = 2.4
/joda-time/joda-time = 2.2
/junit/junit = 4.10
/log4j/log4j = 1.2.17
# NOTE: split out from the full version to also use as solr-ref-guide attribute
org.log4j.major.version = 1.2
/log4j/log4j = ${org.log4j.major.version}.17
/mecab/mecab-ipadic = 2.7.0-20070801
/mecab/mecab-naist-jdic = 0.6.3b-20111013
/net.arnx/jsonic = 1.2.7

View File

@ -92,6 +92,8 @@ Other Changes
Also being able to run all Lucene and Solr tests as a configuration (Karthik Ramachandran via
Varun Thacker, Uwe Schindler)
* SOLR-12118: Solr Ref-Guide can now use some ivy version props directly as attributes in content (hossman)
================== 7.3.0 ==================
Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.

View File

@ -89,6 +89,8 @@
<property name="build.content.dir" location="${build.dir}/content" />
<property name="main-page" value="index" />
<property name="pdf-filename" value="apache-solr-ref-guide-${solr-guide-version}.pdf" />
<!-- for pulling in versions of major deps -->
<property prefix="ivyversions" file="${common.dir}/ivy-versions.properties"/>
<!-- ====== TOOLS FOR GENERATING/VALIDATING BITS OF THE SITE / PDF ======= -->
<property name="tools-jar-name" value="solr-ref-guide-tools.jar" />
@ -214,6 +216,13 @@
<attribute key="lucene-javadocs" value="${lucene-javadocs}" />
<attribute key="build-date" value="${DSTAMP}" />
<attribute key="build-year" value="${current.year}" />
<attribute key="ivy-commons-codec-version" value="${ivyversions./commons-codec/commons-codec}" />
<attribute key="ivy-dropwizard-version" value="${ivyversions.io.dropwizard.metrics.version}" />
<attribute key="ivy-log4j-version" value="${ivyversions.org.log4j.major.version}" />
<attribute key="ivy-opennlp-version" value="${ivyversions./org.apache.opennlp/opennlp-tools}" />
<attribute key="ivy-tika-version" value="${ivyversions.org.apache.tika.version}" />
<attribute key="ivy-velocity-tools-version" value="${ivyversions./org.apache.velocity/velocity-tools}" />
<attribute key="ivy-zookeeper-version" value="${ivyversions./org.apache.zookeeper/zookeeper}" />
</asciidoctor:convert>
</sequential>
</macrodef>

View File

@ -78,7 +78,14 @@ solr-attributes: &solr-attributes-ref
lucene-javadocs: "${lucene-javadocs}"
build-date: "${DSTAMP}"
build-year: "${current.year}"
ivy-commons-codec-version: "${ivyversions./commons-codec/commons-codec}"
ivy-dropwizard-version: "${ivyversions.io.dropwizard.metrics.version}"
ivy-log4j-version: "${ivyversions.org.log4j.major.version}"
ivy-opennlp-version: "${ivyversions./org.apache.opennlp/opennlp-tools}"
ivy-tika-version: "${ivyversions.org.apache.tika.version}"
ivy-velocity-tools-version: "${ivyversions./org.apache.velocity/velocity-tools}"
ivy-zookeeper-version: "${ivyversions./org.apache.zookeeper/zookeeper}"
asciidoctor:
safe: 0
attributes:

View File

@ -146,7 +146,7 @@ The `.der` files that are output from Step 2 should then be loaded to ZooKeeper
At the current time, you can only use the ZooKeeper `zkCli.sh` (or `zkCli.cmd` on Windows) script to issue these commands (the Solr version has the same name, but is not the same). If you have your own ZooKeeper ensemble running already, you can find the script in `$ZK_INSTALL/bin/zkCli.sh` (or `zkCli.cmd` if you are using Windows).
NOTE: If you are running the embedded ZooKeeper that is included with Solr, you *do not* have this script already; in order to use it, you will need to download a copy of ZooKeeper v3.4.11 from http://zookeeper.apache.org/. Don't worry about configuring the download, you're just trying to get the command line utility script. When you start the script, you will connect to the embedded ZooKeeper.
NOTE: If you are running the embedded ZooKeeper that is included with Solr, you *do not* have this script already; in order to use it, you will need to download a copy of ZooKeeper v{ivy-zookeeper-version} from http://zookeeper.apache.org/. Don't worry about configuring the download, you're just trying to get the command line utility script. When you start the script, you will connect to the embedded ZooKeeper.
To load the keys, you will need to connect to ZooKeeper with `zkCli.sh`, create the directories, and then create the key file, as in the following example.

View File

@ -194,24 +194,24 @@ $ sudo vim /etc/hosts
These steps will help you install and configure a single instance of ZooKeeper on AWS. This is not sufficient for a production, use, however, where a ZooKeeper ensemble of at least three nodes is recommended. See the section <<setting-up-an-external-zookeeper-ensemble.adoc#setting-up-an-external-zookeeper-ensemble,Setting Up an External ZooKeeper Ensemble>> for information about how to change this single-instance into an ensemble.
. Download a stable version of ZooKeeper. In this example we're using ZooKeeper v3.4.6. On the node you're using to host ZooKeeper (`zookeeper-node`), download the package and untar it:
. Download a stable version of ZooKeeper. In this example we're using ZooKeeper v{ivy-zookeeper-version}. On the node you're using to host ZooKeeper (`zookeeper-node`), download the package and untar it:
+
[source,bash]
[source,bash,subs="attributes"]
----
# download stable version of ZooKeeper, here 3.4.6
$ wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
# download stable version of ZooKeeper, here {ivy-zookeeper-version}
$ wget https://archive.apache.org/dist/zookeeper/zookeeper-{ivy-zookeeper-version}/zookeeper-{ivy-zookeeper-version}.tar.gz
# untar
$ tar -zxvf zookeeper-3.4.6.tar.gz
$ tar -zxvf zookeeper-{ivy-zookeeper-version}.tar.gz
----
+
Add an environment variable for ZooKeeper's home directory (`ZOO_HOME`) to the `.bashrc` for the user who will be running the process. The rest of the instructions assume you have set this variable. Correct the path to the ZooKeeper installation as appropriate if where you put it does not match the below.
+
[source,bash]
[source,bash,subs="attributes"]
----
$ export ZOO_HOME=$PWD/zookeeper-3.4.6
$ export ZOO_HOME=$PWD/zookeeper-{ivy-zookeeper-version}
# put the env variable in .bashrc
# vim ~/.bashrc
export ZOO_HOME=/home/ec2-user/zookeeper-3.4.6
export ZOO_HOME=/home/ec2-user/zookeeper-{ivy-zookeeper-version}
----
. Change directories to `ZOO_HOME`, and create the ZooKeeper configuration by using the template provided by ZooKeeper.
+

View File

@ -84,7 +84,7 @@ bin/solr start -f -q
== Permanent Logging Settings
Solr uses http://logging.apache.org/log4j/1.2/[Log4J version 1.2] for logging which is configured using `server/resources/log4j.properties`. Take a moment to inspect the contents of the `log4j.properties` file so that you are familiar with its structure. By default, Solr log messages will be written to `SOLR_LOGS_DIR/solr.log`.
Solr uses http://logging.apache.org/log4j/{ivy-log4j-version}/[Log4J version {ivy-log4j-version}] for logging which is configured using `server/resources/log4j.properties`. Take a moment to inspect the contents of the `log4j.properties` file so that you are familiar with its structure. By default, Solr log messages will be written to `SOLR_LOGS_DIR/solr.log`.
When you're ready to deploy Solr in production, set the variable `SOLR_LOGS_DIR` to the location where you want Solr to write log files, such as `/var/solr/logs`. You may also want to tweak `log4j.properties`. Note that if you installed Solr as a service using the instructions provided in <<taking-solr-to-production.adoc#taking-solr-to-production,Taking Solr to Production>>, then see `/var/solr/log4j.properties` instead of the default `server/resources` version.

View File

@ -20,9 +20,9 @@ Solr can identify languages and map text to language-specific fields during inde
Solr supports three implementations of this feature:
* Tika's language detection feature: https://tika.apache.org/1.17/detection.html
* Tika's language detection feature: https://tika.apache.org/{ivy-tika-version}/detection.html
* LangDetect language detection: https://github.com/shuyo/language-detection
* OpenNLP language detection: http://opennlp.apache.org/docs/1.8.4/manual/opennlp.html#tools.langdetect
* OpenNLP language detection: http://opennlp.apache.org/docs/{ivy-opennlp-version}/manual/opennlp.html#tools.langdetect
You can see a comparison between the Tika and LangDetect implementations here: http://blog.mikemccandless.com/2011/10/accuracy-and-performance-of-googles.html. In general, the LangDetect implementation supports more languages with higher performance.
@ -80,7 +80,7 @@ Here is an example of a minimal OpenNLP `langid` configuration in `solrconfig.xm
==== OpenNLP-specific Parameters
`langid.model`::
An OpenNLP language detection model. The OpenNLP project provides a pre-trained 103 language model on the http://opennlp.apache.org/models.html[OpenNLP site's model dowload page]. Model training instructions are provided on the http://opennlp.apache.org/docs/1.8.4/manual/opennlp.html#tools.langdetect[OpenNLP website]. This parameter is required.
An OpenNLP language detection model. The OpenNLP project provides a pre-trained 103 language model on the http://opennlp.apache.org/models.html[OpenNLP site's model dowload page]. Model training instructions are provided on the http://opennlp.apache.org/docs/{ivy-opennlp-version}/manual/opennlp.html#tools.langdetect[OpenNLP website]. This parameter is required.
==== OpenNLP Language Codes

View File

@ -201,7 +201,7 @@ Implements the Daitch-Mokotoff Soundex algorithm, which allows identification of
== Double Metaphone Filter
This filter creates tokens using the http://commons.apache.org/codec/apidocs/org/apache/commons/codec/language/DoubleMetaphone.html[`DoubleMetaphone`] encoding algorithm from commons-codec. For more information, see the <<phonetic-matching.adoc#phonetic-matching,Phonetic Matching>> section.
This filter creates tokens using the http://commons.apache.org/proper/commons-codec/archives/{ivy-commons-codec-version}/apidocs/org/apache/commons/codec/language/DoubleMetaphone.html[`DoubleMetaphone`] encoding algorithm from commons-codec. For more information, see the <<phonetic-matching.adoc#phonetic-matching,Phonetic Matching>> section.
*Factory class:* `solr.DoubleMetaphoneFilterFactory`
@ -1026,7 +1026,7 @@ This filter creates tokens using one of the phonetic encoding algorithms in the
*Arguments:*
`encoder`:: (required) The name of the encoder to use. The encoder name must be one of the following (case insensitive): `http://commons.apache.org/codec/apidocs/org/apache/commons/codec/language/DoubleMetaphone.html[DoubleMetaphone]`, `http://commons.apache.org/codec/apidocs/org/apache/commons/codec/language/Metaphone.html[Metaphone]`, `http://commons.apache.org/codec/apidocs/org/apache/commons/codec/language/Soundex.html[Soundex]`, `http://commons.apache.org/codec/apidocs/org/apache/commons/codec/language/RefinedSoundex.html[RefinedSoundex]`, `http://commons.apache.org/codec/apidocs/org/apache/commons/codec/language/Caverphone.html[Caverphone]` (v2.0), `http://commons.apache.org/codec/apidocs/org/apache/commons/codec/language/ColognePhonetic.html[ColognePhonetic]`, or `http://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/language/Nysiis.html[Nysiis]`.
`encoder`:: (required) The name of the encoder to use. The encoder name must be one of the following (case insensitive): `http://commons.apache.org/proper/commons-codec/archives/{ivy-commons-codec-version}/apidocs/org/apache/commons/codec/language/DoubleMetaphone.html[DoubleMetaphone]`, `http://commons.apache.org/proper/commons-codec/archives/{ivy-commons-codec-version}/apidocs/org/apache/commons/codec/language/Metaphone.html[Metaphone]`, `http://commons.apache.org/proper/commons-codec/archives/{ivy-commons-codec-version}/apidocs/org/apache/commons/codec/language/Soundex.html[Soundex]`, `http://commons.apache.org/proper/commons-codec/archives/{ivy-commons-codec-version}/apidocs/org/apache/commons/codec/language/RefinedSoundex.html[RefinedSoundex]`, `http://commons.apache.org/proper/commons-codec/archives/{ivy-commons-codec-version}/apidocs/org/apache/commons/codec/language/Caverphone.html[Caverphone]` (v2.0), `http://commons.apache.org/proper/commons-codec/archives/{ivy-commons-codec-version}/apidocs/org/apache/commons/codec/language/ColognePhonetic.html[ColognePhonetic]`, or `http://commons.apache.org/proper/commons-codec/archives/{ivy-commons-codec-version}/apidocs/org/apache/commons/codec/language/Nysiis.html[Nysiis]`.
`inject`:: (true/false) If true (the default), then new phonetic tokens are added to the stream. Otherwise, tokens are replaced with the phonetic equivalent. Setting this to false will enable phonetic matching, but the exact spelling of the target word may not match.

View File

@ -124,7 +124,7 @@ Implementation of a custom metrics supplier is beyond the scope of this guide, b
* `cpu`, which uses the current thread's CPU time
<histogram>:: This element defines the implementation of a `Histogram` supplier. This element also supports the `clock` parameter shown above with the `meter` element, and also:
`<str name="reservoir">`::: The fully-qualified class name of the `Reservoir` implementation to use. The default is `com.codahale.metrics.ExponentiallyDecayingReservoir` but there are other options available with the http://metrics.dropwizard.io/3.1.0/manual/core/#histograms[Codahale Metrics library] that Solr uses. The following parameters are supported, within the mentioned limitations:
`<str name="reservoir">`::: The fully-qualified class name of the `Reservoir` implementation to use. The default is `com.codahale.metrics.ExponentiallyDecayingReservoir` but there are other options available with the http://metrics.dropwizard.io/{ivy-dropwizard-version}/manual/core.html#histograms[Codahale Metrics library] that Solr uses. The following parameters are supported, within the mentioned limitations:
* `size`, the reservoir size. The default is 1028.
* `alpha`, the decay parameter. The default is 0.015. This is only valid for the `ExponentiallyDecayingReservoir`.
* `window`, the window size, in seconds, and only valid for the `SlidingTimeWindowReservoir`. The default is 300 (5 minutes).

View File

@ -23,7 +23,7 @@ Shutting down a redundant Solr instance will also shut down its ZooKeeper server
The solution to this problem is to set up an external ZooKeeper ensemble. Fortunately, while this process can seem intimidating due to the number of powerful options, setting up a simple ensemble is actually quite straightforward, as described below.
.How Many ZooKeepers?
[quote,ZooKeeper Administrator's Guide,http://zookeeper.apache.org/doc/r3.4.11/zookeeperAdmin.html]
[quote,ZooKeeper Administrator's Guide,http://zookeeper.apache.org/doc/r{ivy-zookeeper-version}/zookeeperAdmin.html]
____
"For a ZooKeeper service to be active, there must be a majority of non-failing machines that can communicate with each other. *To create a deployment that can tolerate the failure of F machines, you should count on deploying 2xF+1 machines*. Thus, a deployment that consists of three machines can handle one failure, and a deployment of five machines can handle two failures. Note that a deployment of six machines can only handle two failures since three machines is not a majority.
@ -36,7 +36,7 @@ It is generally recommended to have an odd number of ZooKeeper servers in your e
For example, if you only have two ZooKeeper nodes and one goes down, 50% of available servers is not a majority, so ZooKeeper will no longer serve requests. However, if you have three ZooKeeper nodes and one goes down, you have 66% of available servers available, and ZooKeeper will continue normally while you repair the one down node. If you have 5 nodes, you could continue operating with two down nodes if necessary.
More information on ZooKeeper clusters is available from the ZooKeeper documentation at http://zookeeper.apache.org/doc/r3.4.11/zookeeperAdmin.html#sc_zkMulitServerSetup.
More information on ZooKeeper clusters is available from the ZooKeeper documentation at http://zookeeper.apache.org/doc/r{ivy-zookeeper-version}/zookeeperAdmin.html#sc_zkMulitServerSetup.
== Download Apache ZooKeeper
@ -46,7 +46,7 @@ The first step in setting up Apache ZooKeeper is, of course, to download the sof
====
When using stand-alone ZooKeeper, you need to take care to keep your version of ZooKeeper updated with the latest version distributed with Solr. Since you are using it as a stand-alone application, it does not get upgraded when you upgrade Solr.
Solr currently uses Apache ZooKeeper v3.4.11.
Solr currently uses Apache ZooKeeper v{ivy-zookeeper-version}.
====
== Setting Up a Single ZooKeeper
@ -78,7 +78,7 @@ Once this file is in place, you're ready to start the ZooKeeper instance.
To run the instance, you can simply use the `ZOOKEEPER_HOME/bin/zkServer.sh` script provided, as with this command: `zkServer.sh start`
Again, ZooKeeper provides a great deal of power through additional configurations, but delving into them is beyond the scope of this tutorial. For more information, see the ZooKeeper http://zookeeper.apache.org/doc/r3.4.5/zookeeperStarted.html[Getting Started] page. For this example, however, the defaults are fine.
Again, ZooKeeper provides a great deal of power through additional configurations, but delving into them is beyond the scope of this tutorial. For more information, see the ZooKeeper http://zookeeper.apache.org/doc/r{ivy-zookeeper-version}/zookeeperStarted.html[Getting Started] page. For this example, however, the defaults are fine.
=== Point Solr at the Instance
@ -184,4 +184,4 @@ You may also want to secure the communication between ZooKeeper and Solr.
To setup ACL protection of znodes, see <<zookeeper-access-control.adoc#zookeeper-access-control,ZooKeeper Access Control>>.
For more information on getting the most power from your ZooKeeper installation, check out the http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html[ZooKeeper Administrator's Guide].
For more information on getting the most power from your ZooKeeper installation, check out the http://zookeeper.apache.org/doc/r{ivy-zookeeper-version}/zookeeperAdmin.html[ZooKeeper Administrator's Guide].

View File

@ -29,7 +29,7 @@ When using the Solr Cell framework, it is helpful to keep the following in mind:
* Tika will automatically attempt to determine the input document type (Word, PDF, HTML) and extract the content appropriately. If you like, you can explicitly specify a MIME type for Tika with the `stream.type` parameter.
* Tika works by producing an XHTML stream that it feeds to a SAX ContentHandler. SAX is a common interface implemented for many different XML parsers. For more information, see http://www.saxproject.org/quickstart.html.
* Solr then responds to Tika's SAX events and creates the fields to index.
* Tika produces metadata such as Title, Subject, and Author according to specifications such as the DublinCore. See http://tika.apache.org/1.16/formats.html for the file types supported.
* Tika produces metadata such as Title, Subject, and Author according to specifications such as the DublinCore. See http://tika.apache.org/{ivy-tika-version}/formats.html for the file types supported.
* Tika adds all the extracted text to the `content` field.
* You can map Tika's metadata fields to Solr fields.
* You can pass in literals for field values. Literals will override Tika-parsed values, including fields in the Tika metadata object, the Tika content field, and any "captured content" fields.
@ -147,7 +147,7 @@ Defines a file path and name to a customized Tika configuration file. This is on
Prefixes all fields that are not defined in the schema with the given prefix. This is very useful when combined with dynamic field definitions. Example: `uprefix=ignored_` would effectively ignore all unknown fields generated by Tika given the example schema contains `<dynamicField name="ignored_*" type="ignored"/>`
`xpath`::
When extracting, only return Tika XHTML content that satisfies the given XPath expression. See http://tika.apache.org/1.16/index.html for details on the format of Tika XHTML. See also http://wiki.apache.org/solr/TikaExtractOnlyExampleOutput.
When extracting, only return Tika XHTML content that satisfies the given XPath expression. See http://tika.apache.org/{ivy-tika-version}/ for details on the format of Tika XHTML. See also http://wiki.apache.org/solr/TikaExtractOnlyExampleOutput.
== Order of Operations

View File

@ -107,14 +107,14 @@ Resource bundles can be added by providing a JAR file visible by the SolrResourc
|Context Reference |Description
|`request` |{solr-javadocs}solr-core/org/apache/solr/request/SolrQueryRequest.html[SolrQueryRequest] javadocs
|`response` |{solr-javadocs}solr-core/org/apache/solr/response/SolrQueryResponse.html[QueryResponse] most of the time, but in some cases where QueryResponse doesn't like the request handler's output (https://wiki.apache.org/solr/AnalysisRequestHandler[AnalysisRequestHandler], for example, causes a ClassCastException parsing "response"), the response will be a SolrResponseBase object.
|`esc` |A Velocity http://velocity.apache.org/tools/2.0/tools-summary.html#EscapeTool[EscapeTool] instance
|`date` |A Velocity http://velocity.apache.org/tools/2.0/tools-summary.html#ComparisonDateTool[ComparisonDateTool] instance
|`list` |A Velocity http://velocity.apache.org/tools/2.0/apidocs/org/apache/velocity/tools/generic/ListTool.html[ListTool] instance
|`math` |A Velocity http://velocity.apache.org/tools/2.0/tools-summary.html#MathTool[MathTool] instance
|`number` |A Velocity http://velocity.apache.org/tools/2.0/tools-summary.html#NumberTool[NumberTool] instance
|`sort` |A Velocity http://velocity.apache.org/tools/2.0/tools-summary.html#SortTool[SortTool] instance
|`display` |A Velocity http://velocity.apache.org/tools/2.0/tools-summary.html#DisplayTool[DisplayTool] instance
|`resource` |A Velocity http://velocity.apache.org/tools/2.0/tools-summary.html#ResourceTool[ResourceTool] instance
|`esc` |A Velocity http://velocity.apache.org/tools/{ivy-velocity-tools-version}/tools-summary.html#EscapeTool[EscapeTool] instance
|`date` |A Velocity http://velocity.apache.org/tools/{ivy-velocity-tools-version}/tools-summary.html#ComparisonDateTool[ComparisonDateTool] instance
|`list` |A Velocity http://velocity.apache.org/tools/{ivy-velocity-tools-version}/apidocs/org/apache/velocity/tools/generic/ListTool.html[ListTool] instance
|`math` |A Velocity http://velocity.apache.org/tools/{ivy-velocity-tools-version}/tools-summary.html#MathTool[MathTool] instance
|`number` |A Velocity http://velocity.apache.org/tools/{ivy-velocity-tools-version}/tools-summary.html#NumberTool[NumberTool] instance
|`sort` |A Velocity http://velocity.apache.org/tools/{ivy-velocity-tools-version}/tools-summary.html#SortTool[SortTool] instance
|`display` |A Velocity http://velocity.apache.org/tools/{ivy-velocity-tools-version}/tools-summary.html#DisplayTool[DisplayTool] instance
|`resource` |A Velocity http://velocity.apache.org/tools/{ivy-velocity-tools-version}/tools-summary.html#ResourceTool[ResourceTool] instance
|`engine` |The current VelocityEngine instance
|`page` |An instance of Solr's PageTool (only included if the response is a QueryResponse where paging makes sense)
|`debug` |A shortcut to the debug part of the response, or null if debug is not on. This is handy for having debug-only sections in a template using `#if($debug)...#end`

View File

@ -16,7 +16,7 @@
// specific language governing permissions and limitations
// under the License.
This section describes using ZooKeeper access control lists (ACLs) with Solr. For information about ZooKeeper ACLs, see the ZooKeeper documentation at http://zookeeper.apache.org/doc/r3.4.11/zookeeperProgrammers.html#sc_ZooKeeperAccessControl.
This section describes using ZooKeeper access control lists (ACLs) with Solr. For information about ZooKeeper ACLs, see the ZooKeeper documentation at http://zookeeper.apache.org/doc/r{ivy-zookeeper-version}/zookeeperProgrammers.html#sc_ZooKeeperAccessControl.
== About ZooKeeper ACLs