The recent addition of support for a "readonly" mode for collections
opens the door to restoring to already-existing collections.
This commit adds a codepath to allow this. Any compatible existing
collection may be used for restoration, including the collection that
was the original source of the backup.
* relocate xslt related classes into scripting contrib
* relocating files to scripting and seperating out unit tests
* relocate files under test-files/scripting/solr, similar to how we do it in other contribs. deals with some issues in finding files
* Reformatting using the Google Java Format...
* use actual param name, not the variable to properly test api!
* Clean up references to paths, and deal with the mish mash of Xslt and XSLT in class names.
* Move XSLT processing out of XMLLoader
* Move TransformerProvider.Dedupe getTransformer logic.
Co-authored-by: epugh@opensourceconnections.com <>
Co-authored-by: David Smiley <dsmiley@apache.org>
SOLR-13608 introduces a new "incremental" backup format, which allows
storage of multiple backup "points" in the same location. This
development introduces a need for APIs to manage these potentially
plural backups.
This commit introduces /admin/collections?action=LISTBACKUPS and
/admin/collections?action=DELETEBACKUP to handle these backups.
In Solr 8.6.3, minCompetitiveScore of WANDScorer resets to zero for each index segment and remain zero until maxScore is updated.
There are two causes of this problem:
* MaxScoreCollector does not set minCompetitiveScore of MinCompetitiveScoreAwareScorable newly generated for another index segment.
* MaxScoreCollector updates minCompetitiveScore only if maxScore is updated. This behavior is correct considering the purpose of MaxScoreCollector.
For details, see the attached pdf https://issues.apache.org/jira/secure/attachment/13019548/wand.pdf.
The initial incremental-backup commit introduced several test failures
on Windows test runs that I neglected to catch before committing. Most
of these failures were the result of bad 'location' path handling in the
test logic itself, though there were a few tweaks made to Solr code
itself to better handle Windows paths as well.
Solr supports two different ways to write v2 APIs: a JSON spec based
approach, and one based on annotated POJOs. The POJO method is now
preferred.
This commit switches the /v2/collections APIs over to the
annotation-based approach. Since V2RequestSupport only works with
jsonspec-based APIs, this commit also changes CollectionAdminRequest
to no longer implement that interface.
* Make all Tool option descriptions follow the same general pattern for describing them.
* Figure out a switch to determine level of either cluster or collections(s)
* better wording on what cluster versus collection params mean
Co-authored-by: epugh@opensourceconnections.com <>
This commit introduces a new way for Solr to do backups (with a new
underlying file structure). This new "incremental" backup process
improves over the existing backup mechanism in several ways:
- multiple backups "points" can now be stored at a given backup
location/name, allowing users to choose which point in time they want
to restore
- subsequent backups skip over uploading files that were uploaded by
previous backups, saving time and network time.
- files are checksumed as they're uploaded, ensuring that corrupted
indices aren't persisted and accidentally restored later.
Incremental backups are now the default, and traditional backups
should now be considered 'deprecated' but can still be created by
passing an `incremental=false` parameter on backup requests.
* Creating Scripting contrib module to centralize the less secure code related to scripts.
* tweak the changelog and update notice to explain why the name changed and the security posture thinking
* the test script happens to be a currency.xml, which made me think we were doing something specific to currency types, but instead any xml formatted file will suffice for the test.
* Update solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.java
* Update solr/contrib/scripting/src/java/org/apache/solr/scripting/update/package-info.java
* drop the ing, and be more specific on the name of the ref guide page
* comment out the script update chain.
The sample techproducts configSet is used by many of the solr unit tests, and by default doesn't have access to the jar file in the contrib module. This is commented out, similar to how the lang contrib is.
* using a Mock for the script processor in order to keep the trusted configSets tests all together.
* tweak since we are using a mock script processor
Co-authored-by: David Smiley <dsmiley@apache.org>
When we are creating a new thread we should give it a descriptive name and enforce this via ForbiddenAPIs. This doesn't apply to Runnable or Callable objects that we pass to an executor, since those should be getting named by the executor itself.
We don't require this in tests because the tests should be more self contained and there is less benefit in descriptive names. If somebody is already profiling a test, then they likely have the context to understand what the unnamed threads are doing, whereas a thread dump from a running Solr instance should have good thread names for everything. This is especially helpful when doing profiling, otherwise we end up with a bunch of Thread-# that are hard to tell apart and search on.
Same fix as the #2210 PR commit earlier but this time not extending SystemInfoHandlerTest and also not adding a static SystemInfoHandler.getSecurityInfo variant for test use.
* Creating Scripting contrib module to centralize the less secure code related to scripts.
* tweak the changelog and update notice to explain why the name changed and the security posture thinking
* the test script happens to be a currency.xml, which made me think we were doing something specific to currency types, but instead any xml formatted file will suffice for the test.
* drop the ing, and be more specific on the name of the ref guide page
* use the same name everywhere
Co-authored-by: David Smiley <dsmiley@apache.org>
Ilan Ginzburg
Tobias Kaessmann
David Smiley
Jason Gerlowski
Chris Hostetter
Timothy Potter
Eric Pugh
Ishan Chattopadhyaya
epugh@opensourceconnections.com
Naoto MINAMI
Jason Gerlowski
Joel Bernstein
David Smiley
Mike Drob
Nazerke Seidan
Andrzej Bialecki
Tim Owen
Christine Poerschke
András Salamon