Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Revert "SOLR-14067: v3 Create /contrib/scripting module with ScriptingUpdateProcessor (#2215)" 

This reverts commit cf5db8d651.
This commit is contained in:
epugh@opensourceconnections.com 2021-01-25 07:22:21 -05:00
parent f942b2dd8a
commit ce1bba6d66
42 changed files with 209 additions and 892 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
gradle/maven/defaults-maven.gradle
View File

@ -66,7 +66,6 @@ configure(rootProject) {
":solr:contrib:langid",
":solr:contrib:jaegertracer-configurator",
":solr:contrib:prometheus-exporter",
":solr:contrib:scripting",
":solr:test-framework",
]

1
settings.gradle
View File

@ -65,7 +65,6 @@ include "solr:contrib:extraction"
include "solr:contrib:langid"
include "solr:contrib:jaegertracer-configurator"
include "solr:contrib:prometheus-exporter"
include "solr:contrib:scripting"
include "solr:contrib:ltr"
include "solr:webapp"
include "solr:test-framework"

3
solr/CHANGES.txt
View File

@ -188,9 +188,6 @@ Other Changes
* SOLR-14297: Replace commons-codec Base64 with JDK8 Base64 (Andras Salamon via Houston Putman)
* SOLR-14067: StatelessScriptUpdateProcessorFactory moved to it's own /contrib/scripting/ package instead
of shipping as part of Solr due to security concerns. Renamed to ScriptUpdateProcessorFactory for simpler name. (Eric Pugh)
Bug Fixes
---------------------
* SOLR-14546: Fix for a relatively hard to hit issue in OverseerTaskProcessor that could lead to out of order execution

14
solr/contrib/scripting/README.md
View File

@ -1,14 +0,0 @@
Welcome to Apache Solr Scripting!
===============================
# Introduction
The Scripting contrib module pulls together various scripting related functions.
Today, the ScriptUpdateProcessorFactory allows Java scripting engines to support scripts written in languages such as JavaScript, Ruby, Python, and Groovy to be used during Solr document update processing, allowing dramatic flexibility in expressing custom document processing before being indexed. It exposes hooks for commit, delete, etc, but add is the most common usage. It is implemented as an UpdateProcessor to be placed in an UpdateChain.
## Getting Started
For information on how to get started please see:
* [Solr Reference Guide's section on Update Request Processors](https://lucene.apache.org/solr/guide/update-request-processors.html)
* [Solr Reference Guide's section on ScriptUpdateProcessorFactory](https://lucene.apache.org/solr/guide/script-update-processor.html)

25
solr/contrib/scripting/build.gradle
View File

@ -1,25 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
apply plugin: 'java-library'
description = 'Scripting Package'
dependencies {
implementation project(':solr:core')
testImplementation project(':solr:test-framework')
}

21
solr/contrib/scripting/src/java/org/apache/solr/scripting/update/package-info.java
View File

@ -1,21 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* Support for scripting during document updates.
*/
package org.apache.solr.scripting.update;

26
solr/contrib/scripting/src/java/overview.html
View File

@ -1,26 +0,0 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<body>
Apache Solr Search Server: Scripting contrib
<p>
This package provides an Update Processor that allows for Java scripting engines
to be used during the Solr document update processing.
</p>
</body>
</html>

27
solr/contrib/scripting/src/test-files/solr/collection1/conf/invalid.script.xml
View File

@ -1,27 +0,0 @@
<?xml version="1.0" ?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- This file is designed to test that loading a .xml file into
a script engine that is configured for JavaScript properly raises
an exception.
-->
<foo version="1.0">
<bar/>
</foo>

73
solr/contrib/scripting/src/test-files/solr/collection1/conf/schema.xml
View File

@ -1,73 +0,0 @@
<?xml version="1.0" ?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- The Solr schema file. This file should be named "schema.xml" and
should be located where the classloader for the Solr webapp can find it.
This schema is used for testing, and as such has everything and the
kitchen sink thrown in. See example/solr/conf/schema.xml for a
more concise example.
-->
<schema name="test" version="1.6">
<fieldType name="int" class="${solr.tests.IntegerFieldType}" docValues="${solr.tests.numeric.dv}" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
<fieldType name="double" class="${solr.tests.DoubleFieldType}" docValues="${solr.tests.numeric.dv}" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
<fieldType name="string" class="solr.StrField" sortMissingLast="true"/>
<!-- solr.TextField allows the specification of custom
text analyzers specified as a tokenizer and a list
of token filters.
-->
<fieldType name="text" class="solr.TextField">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
</fieldType>
<fieldType name="nametext" class="solr.TextField">
<analyzer class="org.apache.lucene.analysis.core.WhitespaceAnalyzer"/>
</fieldType>
<field name="id" type="string" indexed="true" stored="true" multiValued="false" required="false"/>
<field name="name" type="nametext" indexed="true" stored="true"/>
<field name="subject" type="text" indexed="true" stored="true"/>
<!-- Dynamic field definitions. If a field name is not found, dynamicFields
will be used if the name matches any of the patterns.
RESTRICTION: the glob-like pattern in the name attribute must have
a "*" only at the start or the end.
EXAMPLE: name="*_i" will match any field ending in _i (like myid_i, z_i)
Longer patterns will be matched first. if equal size patterns
both match, the first appearing in the schema will be used.
-->
<dynamicField name="*_i" type="int" indexed="true" stored="true"/>
<dynamicField name="*_d" type="double" indexed="true" stored="true"/>
<dynamicField name="*_s" type="string" indexed="true" stored="true"/>
<dynamicField name="*_sm" type="string" indexed="true" stored="true" multiValued="true"/>
<uniqueKey>id</uniqueKey>
</schema>

51
solr/contrib/scripting/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
View File

@ -1,51 +0,0 @@
<?xml version="1.0" ?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!--
A solrconfig.xml snippet containing indexConfig settings for randomized testing.
-->
<indexConfig>
<!-- this sys property is not set by SolrTestCaseJ4 because we ideally want to use
the RandomMergePolicy in all tests - but some tests expect very specific
Merge behavior, so those tests can set it as needed.
-->
<mergePolicyFactory class="${solr.tests.mergePolicyFactory:org.apache.solr.util.RandomMergePolicyFactory}" />
<useCompoundFile>${useCompoundFile:false}</useCompoundFile>
<maxBufferedDocs>${solr.tests.maxBufferedDocs}</maxBufferedDocs>
<ramBufferSizeMB>${solr.tests.ramBufferSizeMB}</ramBufferSizeMB>
<maxCommitMergeWaitTime>${solr.tests.maxCommitMergeWaitTime:-1}</maxCommitMergeWaitTime>
<ramPerThreadHardLimitMB>${solr.tests.ramPerThreadHardLimitMB}</ramPerThreadHardLimitMB>
<mergeScheduler class="${solr.tests.mergeScheduler}" />
<writeLockTimeout>1000</writeLockTimeout>
<commitLockTimeout>10000</commitLockTimeout>
<!-- this sys property is not set by SolrTestCaseJ4 because almost all tests should
use the single process lockType for speed - but tests that explicitly need
to vary the lockType canset it as needed.
-->
<lockType>${solr.tests.lockType:single}</lockType>
<infoStream>${solr.tests.infostream:false}</infoStream>
</indexConfig>

126
solr/contrib/scripting/src/test-files/solr/collection1/conf/stateless-solrconfig-script-updateprocessor.xml
View File

@ -1,126 +0,0 @@
<?xml version="1.0" ?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!--
Test Config for ScriptUpdateProcessorFactory
-->
<config>
<luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
<xi:include href="solrconfig.snippet.randomindexconfig.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
<requestHandler name="/select" class="solr.SearchHandler"></requestHandler>
<directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.RAMDirectoryFactory}"/>
<schemaFactory class="ClassicIndexSchemaFactory"/>
<updateRequestProcessorChain name="force-script-engine" default="true">
<processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="engine">javascript</str>
<str name="script">missleading.extension.updateprocessor.js.txt</str>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
<updateRequestProcessorChain name="run-no-scripts">
<!-- for bypassing all scripts -->
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
<updateRequestProcessorChain name="single-script">
<processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">trivial.updateprocessor0.js</str>
<lst name="params">
<bool name="boolValue">true</bool>
<int name="intValue">1</int>
</lst>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
<updateRequestProcessorChain name="dual-scripts-arr">
<processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<arr name="script">
<str>trivial.updateprocessor0.js</str>
<str>trivial.updateprocessor1.js</str>
</arr>
<lst name="params">
<bool name="boolValue">true</bool>
<int name="intValue">1</int>
</lst>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
<updateRequestProcessorChain name="dual-scripts-strs">
<processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">trivial.updateprocessor0.js</str>
<str name="script">trivial.updateprocessor1.js</str>
<lst name="params">
<bool name="boolValue">true</bool>
<int name="intValue">1</int>
</lst>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
<updateRequestProcessorChain name="conditional-scripts">
<!-- multiple scripts,
test that the first one can conditionally stop execution -->
<processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">conditional.updateprocessor.js</str>
<str name="script">addfields.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="conditional-script">
<!-- single script, followed by another processor
(that happens to be a script).
test that the first one can conditionally stop execution -->
<processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">conditional.updateprocessor.js</str>
</processor>
<processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">addfields.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="error-on-add">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">throw.error.on.add.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="missing-functions">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">missing.functions.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="javascript-compatibility">
<processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">cross-compatible.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="evil">
<processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">evil.js</str>
</processor>
</updateRequestProcessorChain>
</config>

49
solr/contrib/scripting/src/test/org/apache/solr/scripting/update/TestBadScriptingUpdateProcessorConfig.java
View File

@ -1,49 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.solr.scripting.update;
import javax.script.ScriptEngineManager;
import org.apache.solr.core.AbstractBadConfigTestBase;
import org.junit.Assume;
public class TestBadScriptingUpdateProcessorConfig extends AbstractBadConfigTestBase {
public void testBogusScriptEngine() throws Exception {
// sanity check
Assume.assumeTrue(null == (new ScriptEngineManager()).getEngineByName("giberish"));
assertConfigs("bad-solrconfig-bogus-scriptengine-name.xml",
"schema.xml","giberish");
}
public void testMissingScriptFile() throws Exception {
// sanity check
Assume.assumeNotNull((new ScriptEngineManager()).getEngineByExtension("js"));
assertConfigs("bad-solrconfig-missing-scriptfile.xml",
"schema.xml","a-file-name-that-does-not-exist.js");
}
public void testInvalidScriptFile() throws Exception {
// sanity check
Assume.assumeNotNull((new ScriptEngineManager()).getEngineByName("javascript"));
assertConfigs("bad-solrconfig-invalid-scriptfile.xml",
"schema.xml","invalid.script.xml");
}
}

2
solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptEngineCustomizer.java → solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java
View File

@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.solr.scripting.update;
package org.apache.solr.update.processor;
import javax.script.ScriptEngine;

21
solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.java → solr/core/src/java/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactory.java
View File

@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.solr.scripting.update;
package org.apache.solr.update.processor;
import org.apache.solr.common.SolrException;
import org.apache.solr.common.SolrException.ErrorCode;
@ -26,8 +26,6 @@ import org.apache.solr.request.SolrQueryRequest;
import org.apache.solr.request.LocalSolrQueryRequest;
import org.apache.solr.response.SolrQueryResponse;
import org.apache.solr.update.*;
import org.apache.solr.update.processor.UpdateRequestProcessor;
import org.apache.solr.update.processor.UpdateRequestProcessorFactory;
import org.apache.solr.util.plugin.SolrCoreAware;
import org.apache.commons.io.IOUtils;
import org.apache.commons.io.FilenameUtils;
@ -61,8 +59,9 @@ import org.slf4j.LoggerFactory;
/**
* <p>
* An update request processor factory that enables the use of update
* processors implemented as scripts which can be loaded from the
* configSet. Previously known as the StatelessScriptUpdateProcessorFactory.
* processors implemented as scripts which can be loaded by the
* {@link SolrResourceLoader} (usually via the <code>conf</code> dir for
* the SolrCore).
* </p>
* <p>
* This factory requires at least one configuration parameter named
@ -101,7 +100,7 @@ import org.slf4j.LoggerFactory;
* Internally this update processor uses JDK 6 scripting engine support,
* and any {@link Invocable} implementations of <code>ScriptEngine</code>
* that can be loaded using the Solr Plugin ClassLoader may be used.
* By default, the engine used for each script is determined by the file
* By default, the engine used for each script is determined by the filed
* extension (ie: a *.js file will be treated as a JavaScript script) but
* this can be overridden by specifying an explicit "engine" name init
* param for the factory, which identifies a registered name of a
@ -121,7 +120,7 @@ import org.slf4j.LoggerFactory;
* A simple example...
* </p>
* <pre class="prettyprint">
* &lt;processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory"&gt;
* &lt;processor class="solr.StatelessScriptUpdateProcessorFactory"&gt;
* &lt;str name="script"&gt;updateProcessor.js&lt;/str&gt;
* &lt;/processor&gt;
* </pre>
@ -131,7 +130,7 @@ import org.slf4j.LoggerFactory;
* scope of each script...
* </p>
* <pre class="prettyprint">
* &lt;processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory"&gt;
* &lt;processor class="solr.StatelessScriptUpdateProcessorFactory"&gt;
* &lt;arr name="script"&gt;
* &lt;str name="script"&gt;first-processor.js&lt;/str&gt;
* &lt;str name="script"&gt;second-processor.py&lt;/str&gt;
@ -147,7 +146,7 @@ import org.slf4j.LoggerFactory;
* explicit script engine is used....
* </p>
* <pre class="prettyprint">
* &lt;processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory"&gt;
* &lt;processor class="solr.StatelessScriptUpdateProcessorFactory"&gt;
* &lt;arr name="script"&gt;
* &lt;str name="script"&gt;first-processor.txt&lt;/str&gt;
* &lt;str name="script"&gt;second-processor.txt&lt;/str&gt;
@ -158,7 +157,7 @@ import org.slf4j.LoggerFactory;
*
* @since 4.0.0
*/
public class ScriptUpdateProcessorFactory extends UpdateRequestProcessorFactory implements SolrCoreAware {
public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcessorFactory implements SolrCoreAware {
private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
@ -184,7 +183,7 @@ public class ScriptUpdateProcessorFactory extends UpdateRequestProcessorFactory
args.removeConfigArgs(SCRIPT_ARG);
if (scripts.isEmpty()) {
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"ScriptUpdateProcessorFactory must be " +
"StatelessScriptUpdateProcessorFactory must be " +
"initialized with at least one " + SCRIPT_ARG);
}
scriptFiles = new ArrayList<>();

0
solr/contrib/scripting/src/test-files/solr/collection1/conf/addfields.updateprocessor.js → solr/core/src/test-files/solr/collection1/conf/addfields.updateprocessor.js
View File

2
solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-bogus-scriptengine-name.xml → solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-bogus-scriptengine-name.xml
View File

@ -22,7 +22,7 @@
<luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
<updateRequestProcessorChain name="force-script-engine" default="true">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="engine">giberish</str>
<str name="script">missleading.extension.updateprocessor.js.txt</str>
</processor>

4
solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-invalid-scriptfile.xml → solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-invalid-scriptfile.xml
View File

@ -22,10 +22,10 @@
<luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
<updateRequestProcessorChain name="force-script-engine" default="true">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="engine">javascript</str>
<!-- not parsable as javascript -->
<str name="script">invalid.script.xml</str>
<str name="script">currency.xml</str>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>

2
solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml → solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
View File

@ -22,7 +22,7 @@
<luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
<updateRequestProcessorChain name="force-script-engine" default="true">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">a-file-name-that-does-not-exist.js</str>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />

0
solr/contrib/scripting/src/test-files/solr/collection1/conf/conditional.updateprocessor.js → solr/core/src/test-files/solr/collection1/conf/conditional.updateprocessor.js
View File

0
solr/contrib/scripting/src/test-files/solr/collection1/conf/cross-compatible.js → solr/core/src/test-files/solr/collection1/conf/cross-compatible.js
View File

0
solr/contrib/scripting/src/test-files/solr/collection1/conf/evil.js → solr/core/src/test-files/solr/collection1/conf/evil.js
View File

0
solr/contrib/scripting/src/test-files/solr/collection1/conf/missing.functions.updateprocessor.js → solr/core/src/test-files/solr/collection1/conf/missing.functions.updateprocessor.js
View File

0
solr/contrib/scripting/src/test-files/solr/collection1/conf/missleading.extension.updateprocessor.js.txt → solr/core/src/test-files/solr/collection1/conf/missleading.extension.updateprocessor.js.txt
View File

24
solr/contrib/scripting/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml → solr/core/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml
View File

@ -18,7 +18,7 @@
-->
<!--
Test Config for ScriptUpdateProcessorFactory
Test Config that for ScriptUpdateProcessor
-->
<config>
@ -29,7 +29,7 @@
<schemaFactory class="ClassicIndexSchemaFactory"/>
<updateRequestProcessorChain name="force-script-engine" default="true">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="engine">javascript</str>
<str name="script">missleading.extension.updateprocessor.js.txt</str>
</processor>
@ -42,7 +42,7 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="single-script">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">trivial.updateprocessor0.js</str>
<lst name="params">
<bool name="boolValue">true</bool>
@ -53,7 +53,7 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="dual-scripts-arr">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<arr name="script">
<str>trivial.updateprocessor0.js</str>
<str>trivial.updateprocessor1.js</str>
@ -67,7 +67,7 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="dual-scripts-strs">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">trivial.updateprocessor0.js</str>
<str name="script">trivial.updateprocessor1.js</str>
<lst name="params">
@ -81,7 +81,7 @@
<updateRequestProcessorChain name="conditional-scripts">
<!-- multiple scripts,
test that the first one can conditionally stop execution -->
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">conditional.updateprocessor.js</str>
<str name="script">addfields.updateprocessor.js</str>
</processor>
@ -91,34 +91,34 @@
<!-- single script, followed by another processor
(that happens to be a script).
test that the first one can conditionally stop execution -->
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">conditional.updateprocessor.js</str>
</processor>
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">addfields.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="error-on-add">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">throw.error.on.add.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="missing-functions">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">missing.functions.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="javascript-compatibility">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">cross-compatible.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="evil">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">evil.js</str>
</processor>
</updateRequestProcessorChain>

0
solr/contrib/scripting/src/test-files/solr/collection1/conf/throw.error.on.add.updateprocessor.js → solr/core/src/test-files/solr/collection1/conf/throw.error.on.add.updateprocessor.js
View File

0
solr/contrib/scripting/src/test-files/solr/collection1/conf/trivial.updateprocessor0.js → solr/core/src/test-files/solr/collection1/conf/trivial.updateprocessor0.js
View File

0
solr/contrib/scripting/src/test-files/solr/collection1/conf/trivial.updateprocessor1.js → solr/core/src/test-files/solr/collection1/conf/trivial.updateprocessor1.js
View File

23
solr/core/src/test-files/solr/configsets/upload/with-script-processor/missleading.extension.updateprocessor.js.txt Normal file
View File

@ -0,0 +1,23 @@
function processAdd(cmd) {
// Integer.valueOf is needed here to get a tru java object, because
// all javascript numbers are floating point (ie: java.lang.Double)
cmd.getSolrInputDocument().addField("script_added_i",
java.lang.Integer.valueOf(42));
cmd.getSolrInputDocument().addField("script_added_d", 42.3);
}
function processDelete() {
// NOOP
}
function processCommit() {
// NOOP
}
function processRollback() {
// NOOP
}
function processMergeIndexes() {
// NOOP
}
function finish() {
// NOOP
}

10
solr/core/src/test-files/solr/configsets/upload/with-script-processor/solrconfig.xml
View File

@ -37,9 +37,12 @@
<luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
<lib dir="${solr.install.dir:../../../../../}/contrib/scripting/build/libs/" regex="solr-scripting-\d.*\.jar" />
<updateHandler class="solr.DirectUpdateHandler2">
<commitWithin>
<softCommit>${solr.commitwithin.softcommit:true}</softCommit>
</commitWithin>
</updateHandler>
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
@ -52,10 +55,11 @@
<updateRequestProcessorChain name="force-script-engine" default="true">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="engine">javascript</str>
<str name="script">trivial.updateprocessor.js</str>
<str name="script">missleading.extension.updateprocessor.js.txt</str>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
<requestHandler name="/update" class="solr.UpdateRequestHandler" />
</config>

22
solr/core/src/test-files/solr/configsets/upload/with-script-processor/trivial.updateprocessor.js
View File

@ -1,22 +0,0 @@
function processAdd(cmd) {
// Integer.valueOf is needed here to get a tru java object, because
// all javascript numbers are floating point (ie: java.lang.Double)
cmd.getSolrInputDocument().addField("script_added_i",
java.lang.Integer.valueOf(42));
}
function processDelete() {
// NOOP
}
function processCommit() {
// NOOP
}
function processRollback() {
// NOOP
}
function processMergeIndexes() {
// NOOP
}
function finish() {
// NOOP
}

26
solr/core/src/test/org/apache/solr/core/TestBadConfig.java
View File

@ -16,6 +16,10 @@
*/
package org.apache.solr.core;
import javax.script.ScriptEngineManager;
import org.junit.Assume;
public class TestBadConfig extends AbstractBadConfigTestBase {
public void testUnsetSysProperty() throws Exception {
@ -49,6 +53,28 @@ public class TestBadConfig extends AbstractBadConfigTestBase {
}
}
public void testBogusScriptEngine() throws Exception {
// sanity check
Assume.assumeTrue(null == (new ScriptEngineManager()).getEngineByName("giberish"));
assertConfigs("bad-solrconfig-bogus-scriptengine-name.xml",
"schema.xml","giberish");
}
public void testMissingScriptFile() throws Exception {
// sanity check
Assume.assumeNotNull((new ScriptEngineManager()).getEngineByExtension("js"));
assertConfigs("bad-solrconfig-missing-scriptfile.xml",
"schema.xml","a-file-name-that-does-not-exist.js");
}
public void testInvalidScriptFile() throws Exception {
// sanity check
Assume.assumeNotNull((new ScriptEngineManager()).getEngineByName("javascript"));
assertConfigs("bad-solrconfig-invalid-scriptfile.xml",
"schema.xml","currency.xml");
}
public void testBogusMergePolicy() throws Exception {
assertConfigs("bad-mpf-solrconfig.xml", "schema-minimal.xml",
"DummyMergePolicyFactory");

2
solr/contrib/scripting/src/test/org/apache/solr/scripting/update/ScriptEngineTest.java → solr/core/src/test/org/apache/solr/update/processor/ScriptEngineTest.java
View File

@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.solr.scripting.update;
package org.apache.solr.update.processor;
import org.apache.lucene.util.Constants;

18
solr/contrib/scripting/src/test/org/apache/solr/scripting/update/ScriptUpdateProcessorFactoryTest.java → solr/core/src/test/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactoryTest.java
View File

@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.solr.scripting.update;
package org.apache.solr.update.processor;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
@ -25,22 +25,20 @@ import org.apache.solr.common.SolrException;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.core.SolrCore;
import org.apache.solr.update.processor.UpdateProcessorTestBase;
import org.apache.solr.update.processor.UpdateRequestProcessorChain;
import org.junit.Assume;
import org.junit.BeforeClass;
/**
* Tests {@link ScriptUpdateProcessorFactory}.
* Tests {@link StatelessScriptUpdateProcessorFactory}.
*
* TODO: This test, to run from an IDE, requires a working directory of &lt;path-to&gt;/solr/contrib/scripting/src/test-files. Fix!
* TODO: This test, to run from an IDE, requires a working directory of &lt;path-to&gt;/solr/core/src/test-files. Fix!
*/
public class ScriptUpdateProcessorFactoryTest extends UpdateProcessorTestBase {
public class StatelessScriptUpdateProcessorFactoryTest extends UpdateProcessorTestBase {
@BeforeClass
public static void beforeClass() throws Exception {
Assume.assumeNotNull((new ScriptEngineManager()).getEngineByExtension("js"));
initCore("solrconfig-script-updateprocessor.xml", "schema.xml");
initCore("solrconfig-script-updateprocessor.xml", "schema12.xml");
}
/**
@ -70,7 +68,7 @@ public class ScriptUpdateProcessorFactoryTest extends UpdateProcessorTestBase {
public void testSingleScript() throws Exception {
SolrCore core = h.getCore();
UpdateRequestProcessorChain chained = core.getUpdateProcessingChain("single-script");
final ScriptUpdateProcessorFactory factory = ((ScriptUpdateProcessorFactory) chained.getProcessors().get(0));
final StatelessScriptUpdateProcessorFactory factory = ((StatelessScriptUpdateProcessorFactory) chained.getProcessors().get(0));
final List<String> functionMessages = new ArrayList<>();
factory.setScriptEngineCustomizer(new ScriptEngineCustomizer() {
@Override
@ -114,8 +112,8 @@ public class ScriptUpdateProcessorFactoryTest extends UpdateProcessorTestBase {
"dual-scripts-strs"}) {
UpdateRequestProcessorChain chained = core.getUpdateProcessingChain(chain);
final ScriptUpdateProcessorFactory factory =
((ScriptUpdateProcessorFactory) chained.getProcessors().get(0));
final StatelessScriptUpdateProcessorFactory factory =
((StatelessScriptUpdateProcessorFactory) chained.getProcessors().get(0));
final List<String> functionMessages = new ArrayList<>();
ScriptEngineCustomizer customizer = new ScriptEngineCustomizer() {
@Override

1
solr/packaging/build.gradle
View File

@ -52,7 +52,6 @@ dependencies {
":solr:contrib:langid",
":solr:contrib:ltr",
":solr:contrib:prometheus-exporter",
":solr:contrib:scripting"
].each { contribName ->
distSolr project(contribName)
contrib project(path: contribName, configuration: "packaging")

17
solr/server/solr/configsets/sample_techproducts_configs/conf/solrconfig.xml
View File

@ -83,8 +83,6 @@
<lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-ltr-\d.*\.jar" />
<lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-scripting-\d.*\.jar" />
<!-- an exact 'path' can be used instead of a 'dir' to specify a
specific jar file. This will cause a serious error to be logged
if it can't be loaded.
@ -658,9 +656,6 @@
enableRemoteStreaming - enables use of the stream.file
and stream.url parameters for specifying remote streams.
enableStreamBody - This attribute controls whether streaming
content from the HTTP parameter stream.body is allowed.
multipartUploadLimitInKB - specifies the max size (in KiB) of
Multipart File Uploads that Solr will allow in a Request.
@ -679,12 +674,12 @@
*** WARNING ***
Before enabling remote streaming, you should make sure your
system has authentication enabled.
-->
<requestParsers enableRemoteStreaming="true"
enableStreamBody="true"
<requestParsers enableRemoteStreaming="false"
multipartUploadLimitInKB="-1"
formdataUploadLimitInKB="-1"
addHttpRequestToContext="false"/>
-->
<!-- HTTP Caching
@ -1278,10 +1273,11 @@
This example hooks in an update processor implemented using JavaScript.
See more about script update processor at https://lucene.apache.org/solr/guide/script-update-processor.html
See more about the script update processor at http://wiki.apache.org/solr/ScriptUpdateProcessor
-->
<!--
<updateRequestProcessorChain name="script">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="script">update-script.js</str>
<lst name="params">
<str name="config_param">example config parameter</str>
@ -1289,6 +1285,7 @@
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
-->
<!-- Response Writers

4
solr/server/solr/configsets/sample_techproducts_configs/conf/update-script.js
View File

@ -4,14 +4,14 @@
In order for this to be executed, it must be properly wired into solrconfig.xml; by default it is commented out in
the example solrconfig.xml and must be uncommented to be enabled.
See https://lucene.apache.org/solr/guide/script-update-processor.html for more details.
See http://wiki.apache.org/solr/ScriptUpdateProcessor for more details.
*/
function processAdd(cmd) {
doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument
id = doc.getFieldValue("id");
logger.warn("update-script#processAdd: id=" + id); // WARN level messages will show up in Solr Admin Logging UI
logger.info("update-script#processAdd: id=" + id);
// Set a field value:
// doc.setField("foo_s", "whatever");

2
solr/solr-ref-guide/src/configsets-api.adoc
View File

@ -88,7 +88,7 @@ This functionality is enabled by default, but can be disabled via a runtime para
A configset is uploaded in a "trusted" mode if authentication is enabled and the upload operation is performed as an authenticated request. Without authentication, a configset is uploaded in an "untrusted" mode. Upon creation of a collection using an "untrusted" configset, the following functionality will not work:
* The XSLT transformer (`tr` parameter) cannot be used at request processing time.
* If specified in the configset, the ScriptUpdateProcessorFactory will not initialize.
* If specified in the configset, the StatelessScriptUpdateProcessor will not initialize.
* Collections won't initialize if <lib> directives are used in the configset. (Note: Libraries added to Solr's classpath don't need the <lib> directive)
If you use any of these parameters or features, you must have enabled security features in your Solr installation and you must upload the configset as an authenticated user.

1
solr/solr-ref-guide/src/configuring-solrconfig-xml.adoc
View File

@ -8,7 +8,6 @@
query-settings-in-solrconfig, \
requestdispatcher-in-solrconfig, \
update-request-processors, \
script-update-processor, \
codec-factory
// Licensed to the Apache Software Foundation (ASF) under one

3
solr/solr-ref-guide/src/major-changes-in-solr-9.adoc
View File

@ -144,9 +144,6 @@ _(raw; not yet edited)_
* SOLR-14972: The default port of prometheus exporter has changed from 9983 to 8989, so you may need to adjust your configuration after upgrade.
* SOLR-14067: StatelessScriptUpdateProcessorFactory moved to it's own /contrib/scripting/ package instead
of shipping as part of Solr due to security concerns. Renamed to ScriptUpdateProcessorFactory for simpler name.
=== Upgrade Prerequisites in Solr 9
* Upgrade all collections in stateFormat=1 to stateFormat=2 *before* upgrading to Solr 9, as Solr 9 does not support the

286
solr/solr-ref-guide/src/script-update-processor.adoc
View File

@ -1,286 +0,0 @@
= Script Update Processor
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
The {solr-javadocs}/contrib/scripting/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.html[ScriptUpdateProcessorFactory] allows Java scripting engines to be used
during Solr document update processing, allowing dramatic flexibility in
expressing custom document processing logic before being indexed. It has hooks to the
commit, delete, rollback, etc indexing actions, however add is the most common usage.
It is implemented as an UpdateProcessor to be placed in an UpdateChain.
TIP: This used to be known as the _StatelessScriptingUpdateProcessor_ and was renamed to clarify the key aspect of this update processor is it enables scripting.
The script can be written in any scripting language supported by your JVM (such
as JavaScript), and executed dynamically so no pre-compilation is necessary.
WARNING: Being able to run a script of your choice as part of the indexing pipeline is a really powerful tool, that I sometimes call the
_Get out of jail free_ card because you can solve some problems this way that you can't in any other way. However, you are introducing some
potential security vulnerabilities.
== Installing the ScriptingUpdateProcessor and Scripting Engines
The scripting update processor lives in the contrib module `/contrib/scripting`, and you need to explicitly add it to your Solr setup.
Java 11 and previous versions come with a JavaScript engine called Nashorn, but Java 12 will require you to add your own JavaScript engine. Other supported scripting engines like
JRuby, Jython, Groovy, all require you to add JAR files.
Learn more about adding the `dist/solr-scripting-*.jar` file, and any other needed JAR files (depending on your scripting engine) into Solr's <<libs.adoc#lib-directories,Lib Directories>>.
== Configuration
[source,xml]
----
<updateRequestProcessorChain name="script">
<processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">update-script.js</str>
</processor>
<!-- optional parameters passed to script
<lst name="params">
<str name="config_param">example config parameter</str>
</lst>
-->
<processor class="solr.LogUpdateProcessorFactory" />
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
----
NOTE: The processor supports the defaults/appends/invariants concept for its config.
However, it is also possible to skip this level and configure the parameters directly underneath the `<processor>` tag.
Below follows a list of each configuration parameters and their meaning:
`script`::
The script file name. The script file must be placed in the `conf/ directory.
There can be one or more "script" parameters specified; multiple scripts are executed in the order specified.
`engine`::
Optionally specifies the scripting engine to use. This is only needed if the extension
of the script file is not a standard mapping to the scripting engine. For example, if your
script file was coded in JavaScript but the file name was called `update-script.foo`,
use "javascript" as the engine name.
`params`::
Optional parameters that are passed into the script execution context. This is
specified as a named list (`<lst>`) structure with nested typed parameters. If
specified, the script context will get a "params" object, otherwise there will be no "params" object available.
== Script execution context
Every script has some variables provided to it.
`logger`::
Logger (org.slf4j.Logger) instance. This is useful for logging information from the script.
`req`::
{solr-javadocs}/core/org/apache/solr/request/SolrQueryRequest.html[SolrQueryRequest] instance.
`rsp`::
{solr-javadocs}/core/org/apache/solr/response/SolrQueryResponse.html[SolrQueryResponse] instance.
`params`::
The "params" object, if any specified, from the configuration.
== Examples
The `processAdd()` and the other script methods can return false to skip further
processing of the document. All methods must be defined, though generally the
`processAdd()` method is where the action is.
Here's a URL that works with the techproducts example setup demonstrating specifying
the "script" update chain: `http://localhost:8983/solr/techproducts/update?commit=true&stream.contentType=text/csv&fieldnames=id,description&stream.body=1,foo&update.chain=script`
which logs the following:
[source,text]
----
INFO: update-script#processAdd: id=1
----
You can see the message recorded in the Solr logging UI.
=== Javascript
Note: There is a JavaScript example `update-script.js` as part of the `techproducts` configset.
Check `solrconfig.xml` and uncomment the update request processor definition to enable this feature.
[source,javascript]
----
function processAdd(cmd) {
doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument
id = doc.getFieldValue("id");
logger.info("update-script#processAdd: id=" + id);
// Set a field value:
// doc.setField("foo_s", "whatever");
// Get a configuration parameter:
// config_param = params.get('config_param'); // "params" only exists if processor configured with <lst name="params">
// Get a request parameter:
// some_param = req.getParams().get("some_param")
// Add a field of field names that match a pattern:
// - Potentially useful to determine the fields/attributes represented in a result set, via faceting on field_name_ss
// field_names = doc.getFieldNames().toArray();
// for(i=0; i < field_names.length; i++) {
// field_name = field_names[i];
// if (/attr_.*/.test(field_name)) { doc.addField("attribute_ss", field_names[i]); }
// }
}
function processDelete(cmd) {
// no-op
}
function processMergeIndexes(cmd) {
// no-op
}
function processCommit(cmd) {
// no-op
}
function processRollback(cmd) {
// no-op
}
function finish() {
// no-op
}
----
=== Ruby
Ruby support is implemented via the https://www.jruby.org/[JRuby] project.
To use JRuby as the scripting engine, add `jruby.jar` to Solr.
Here's an example of a JRuby update processing script (note that all variables passed in require prefixing with `$`, such as `$logger`):
[source,ruby]
----
def processAdd(cmd)
doc = cmd.solrDoc # org.apache.solr.common.SolrInputDocument
id = doc.getFieldValue('id')
$logger.info "update-script#processAdd: id=#{id}"
doc.setField('source_s', 'ruby')
$logger.info "update-script#processAdd: config_param=#{$params.get('config_param')}"
end
def processDelete(cmd)
# no-op
end
def processMergeIndexes(cmd)
# no-op
end
def processCommit(cmd)
# no-op
end
def processRollback(cmd)
# no-op
end
def finish()
# no-op
end
----
==== Known issues
The following in JRuby does not work as expected for some reason, though it does work properly in JavaScript:
[source,ruby]
----
# $logger.info "update-script#processAdd: request_param=#{$req.params.get('request_param')}"
# $rsp.add('script_processed',id)
----
=== Groovy
Add JARs from a Groovy distro's `lib/` directory to Solr. All JARs from
Groovy's distro probably aren't required, but more than just the main `groovy.jar`
file is needed (at least when this was tested using Groovy 2.0.6)
[source,groovy]
----
def processAdd(cmd) {
doc = cmd.solrDoc // org.apache.solr.common.SolrInputDocument
id = doc.getFieldValue('id')
logger.info "update-script#processAdd: id=" + id
doc.setField('source_s', 'groovy')
logger.info "update-script#processAdd: config_param=" + params.get('config_param')
logger.info "update-script#processAdd: request_param=" + req.params.get('request_param')
rsp.add('script_processed',id)
}
def processDelete(cmd) {
// no-op
}
def processMergeIndexes(cmd) {
// no-op
}
def processCommit(cmd) {
// no-op
}
def processRollback(cmd) {
// no-op
}
def finish() {
// no-op
}
----
=== Python
Python support is implemented via the https://www.jython.org/[Jython] project.
Add the *standalone* `jython.jar` (the JAR that contains all the dependencies) into Solr.
[source,python]
----
def processAdd(cmd):
doc = cmd.solrDoc
id = doc.getFieldValue("id")
logger.info("update-script#processAdd: id=" + id)
def processDelete(cmd):
logger.info("update-script#processDelete")
def processMergeIndexes(cmd):
logger.info("update-script#processMergeIndexes")
def processCommit(cmd):
logger.info("update-script#processCommit")
def processRollback(cmd):
logger.info("update-script#processRollback")
def finish():
logger.info("update-script#finish")
----

2
solr/solr-ref-guide/src/update-request-processors.adoc
View File

@ -283,7 +283,7 @@ What follows are brief descriptions of the currently available update request pr
{solr-javadocs}/core/org/apache/solr/update/processor/SignatureUpdateProcessorFactory.html[SignatureUpdateProcessorFactory]:: Uses a defined set of fields to generate a hash "signature" for the document. Useful for only indexing one copy of "similar" documents.
{solr-javadocs}/contrib/scripting/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.html[ScriptUpdateProcessorFactory]:: An processor that enables the use of update processors implemented as scripts. Learn more at the <<script-update-processor.adoc#script-update-processor,script update processor>> page.
{solr-javadocs}/core/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactory.html[StatelessScriptUpdateProcessorFactory]:: An update request processor factory that enables the use of update processors implemented as scripts.
{solr-javadocs}/core/org/apache/solr/update/processor/TemplateUpdateProcessorFactory.html[TemplateUpdateProcessorFactory]:: Allows adding new fields to documents based on a template pattern. This update processor can also be used at runtime (without defining it in `solrconfig.xml`), see the section <<templateupdateprocessorfactory>> below.

4
solr/test-framework/src/java/org/apache/solr/update/processor/UpdateProcessorTestBase.java
View File

@ -140,7 +140,7 @@ public class UpdateProcessorTestBase extends SolrTestCaseJ4 {
/**
* Convenience method for building up SolrInputDocuments
*/
protected final SolrInputDocument doc(SolrInputField... fields) {
final SolrInputDocument doc(SolrInputField... fields) {
SolrInputDocument d = new SolrInputDocument();
for (SolrInputField f : fields) {
d.put(f.getName(), f);
@ -162,7 +162,7 @@ public class UpdateProcessorTestBase extends SolrTestCaseJ4 {
/**
* Convenience method for building up SolrInputFields with default boost
*/
protected final SolrInputField f(String name, Object... values) {
final SolrInputField f(String name, Object... values) {
return field(name, values);
}
}