Same fix as the #2210 PR commit earlier but this time not extending SystemInfoHandlerTest and also not adding a static SystemInfoHandler.getSecurityInfo variant for test use.
* Creating Scripting contrib module to centralize the less secure code related to scripts.
* tweak the changelog and update notice to explain why the name changed and the security posture thinking
* the test script happens to be a currency.xml, which made me think we were doing something specific to currency types, but instead any xml formatted file will suffice for the test.
* drop the ing, and be more specific on the name of the ref guide page
* use the same name everywhere
Co-authored-by: David Smiley <dsmiley@apache.org>
Today we force indices that were created with N-2 and older versions of Lucene
to fail on open. This check doesn't even check if the codecs are available. In order
to allow users to open older indices and for us to support N-2 versions this change
adds an API on DirectoryReader to specify a minimum index version on a per reader basis.
This doesn't apply for the IndexWriter which will fail on opening older indices.
* add test case for SOLR-15071
* add temporary @Ignore to be removed when the fix is committed
Co-authored-by: Florin Babes <florin.babes@emag.ro>
Co-authored-by: Christine Poerschke <cpoerschke@apache.org>
* introduce jattach check if jstack is missing. jattach ships in the Solr docker image instead of jstack.
* get the full path to the jattach command
Co-authored-by: Christine Poerschke <cpoerschke@apache.org>
Prior to this commit, SuggestComponent used a HashMap as part of the
response it built on the server side. This class is serialized/
deserialized differently depending on the SolrJ ResponseParser used:
a LinkedHashMap when javabin was used, and a SimpleOrderedMap when XML
was used. This discrepancy led to ClassCastException's in downstream
SolrJ code.
This commit fixes the issue by changing SuggestComponent to avoid these
types that are serialized differently. "suggest" response sections now
deserialize as a NamedList in SolrJ, and the SuggesterResponse POJO has
been updated accordingly.
This ensures all derefernced fields are not parsed into actual valuesource
but parsed into a placeholder value. This works for 1-level of dereferencing
* When the schema defines _root_, and you want to do atomic/partial updates...
** _root_ needn't be stored or have docValues any more
** _nest_path_ field isn't needed for this any more
** Simplified internal logic
* Allow (and recommend, eventually insist) that the _root_ field be passed for atomic/partial updates to child docs.
** In the absence of _root_, assume the _route_ param is equivalent to ameliorate back-compat scope. This is a temporary hack; remove in SOLR-15064.
** One of the two is required; you'll get an exception if the assumption is false. THIS IS A BACK-COMPAT CHANGE
* Ensure that the update log contains the _root_ field if it's defined in the schema; in some cases it wasn't. It's important for robustness of atomic/partial updates to child docs. Caveat: the buffer replay scenario is not tested with child docs.
* Limited the cases when a realtime searcher is re-opened. It was being applied to any update that included child docs but now only some narrow subset: only for atomic/partial updates, and when the update log contains an in-place update for the same nest because it's complicated to resolve those log entries.
* Internal improvements to RealTimeGetComponent to aid clarity & robustness & probably performance...
** Use SolrDocumentFetcher.solrDoc(docID, ReturnFields) instead of more manual loading. Will do more with this in another PR.
** Clarify when only root doc IDs are expected.
** Use Resolution enum more, add PARTIAL, remove DOC_WITH_CHILDREN; enhance docs.
** When have ReturnFields, a Set of "onlyTheseFields" becomes redundant. Add a child doc resolution via a transformer when needed.
** Clarified where copy-field targets are removed
* NestPathField should default to single valued, instead of inheriting the schema default, which for ancient schemas was multi-valued.
* AddUpdateCommand.getLuceneDocument(s) methods are very internal; made package visible and refactored a bit for clarity
* DocumentBuilder: when in-place update, skip id and _root_ here, thus also simplifying further logic
* NestedShardedAtomicUpdateTest no longer extends AbstractFullDistribZkTestBase because it wasn't really leveraging the "control client" checking, and it added too much complexity to debug failures.
missing, allBuckets, and numBuckets is not supported with stream method.
So, avoiding picking stream method when any one of them is enabled even if
facet sort is 'index asc'
CopyFields are regenerated in case of replace-field or replace-field-type.
While regenerating, source and destionation are checked against fields but source/dest
could match dynamic rule too.
For example,
<copyField source="something_s" dest="spellcheck"/>
<dynamicField name="*_s" type="string"/>
here, something_s is not present in schema but matches the dynamic rule.
To handle the above case, need to check dynamicFieldCache too while regenerating the
copyFields
Return proper error code on invalid value with in-place update.
Handle invalid value for inc op with the in-place update, uses toNativeType to convert increment value instead of direct parsing. Also, return an error when inc operation is specified for the non-numeric field
SOLR-13027 Use TestInjection so that we always have a Tragic Event
When we encounter a tragic error in the index writer, we can trigger a
leader election instead of queing up a delete and re-add of the node in
question. This should result in a more graceful transition, and the
previous leader will eventually be put into recovery by a new leader.
closes#2120
Prior to this change, it was not feasible to write a custom aggregate function in plugin code
because the FacetContext class did not expose its fields for access outside its package. Also
some of the useful abstract classes for SlotAcc were package-private too
Adds two metrics to the SolrCloud Overseer: solr_metrics_overseer_stateUpdateQueueSize and solr_metrics_overseer_collectionWorkQueueSize with corresponding entries in the the Prometheus exporter's default/stock configuration.
Co-authored-by: Saatchi Bhalla <s.bhalla@salesforce.com>
This addresses review feedback from David Smiley on Jira. It moves the request registration to the ContentStreamHandlerBase class instead of doing a hack-ish instanceof check inside HttpSolrCall.
* Reduced dependencies from Solr server down to just SolrJ. Don't add WEB-INF/lib.
* Was missing some dependencies in lib/; now has all except SolrJ & logging.
* Can run via gradle, "gradlew run"
* Has own log4j2.xml now
Has own CHANGES.md now.
The Dockerfile should not initialize /var/solr's contents because this is confusing and redundant with init_var_solr.sh.
No need for init_var_solr.sh to echo what it does; VERBOSE can be used to accomplish that.
Separate CHANGES.md for Docker and contrib modules.
This adds a switch to `Lucene80DocValuesFormat` which allows to
configure whether to prioritize retrieval speed over compression ratio
or the other way around. When prioritizing retrieval speed, binary doc
values are written using the exact same format as before more aggressive
compression got introduced.
Docs fetched from the update log via RTG look different than docs
fetched from commits in the index: the types of
field-values may be different between the two, etc.
This is a problem for atomic add/remove of field values, where matching
existing values has historically been done by object equals() calls (via
Collection operations). This relies on equality checks which don't have
flexible enough semantics to match values across these different types.
(For example, `new Long(1).equals(new Integer(1))` returns `false`).
This was causing some add-distinct and remove operations on
uncommitted values to silently fail to remove field values.
This commit patches over this by converting between types in the more
common cases before using the fallback behavior.
* Added ability to remove SolrCore.closeHooks
* Keep references to CloseHooks in ReplicationHandler and remove them on ReplicationHandler.shutdown()
closes#1997
The shutdown process waits for all replicas/cores to be closed before removing the election node of the leader. This can take some time due to index flush or merge activities on the leader cores and delays new leaders from being elected. Moreover, jetty stops accepting new requests on receiving SIGTERM which means that even though a leader technically exists, no new indexing requests can be processed by the node. This commit waits for all in-flight indexing requests to complete, removes election nodes (thus triggering leader election) and then closes all replicas.
Co-authored-by: Cao Manh Dat <datcm@apache.org>
Andrzej Bialecki
epugh@opensourceconnections.com
Chris Hostetter
Christine Poerschke
Eric Pugh
András Salamon
Cassandra Targett
David Smiley
Joel Bernstein
Patrick Marty
Simon Willnauer
Noble Paul
Florin Babes
Mike Drob
Timothy Potter
Jason Gerlowski
Houston Putman
Munendra S N
Tomas Fernandez Lobbe
Florin Babes
Bruno Roustant
noblepaul
Tim Dillon
Dawid Weiss
John Gallagher
Pieter van Boxtel
Kevin Risden
Mike Drob
Tim Owen
David Smiley
saatchibhalla
Jan Høydahl
Shalin Shekhar Mangar
Thomas Mortagne
Ilan Ginzburg
Alessandro Benedetti
Erick Erickson
Marcus
Nazerke Seidan
Adrien Grand
Tomoko Uchida
Michael Aleythe
Anver Sotnikov
Michael Sokolov