Commit Graph

8267 Commits

Author SHA1 Message Date
Andrzej Bialecki 68e4304453 SOLR-14347: Autoscaling placement wrong when concurrent replica placements are calculated. 2020-03-23 18:21:40 +01:00
Munendra S N 7f460faffb fix typos in subquery doc transformer 2020-03-23 19:57:46 +05:30
Munendra S N 06fd70fc0f SOLR-14348: split TestJsonFacets to multiple test classes
* TestJsonFacet split into 3 classes, TestsJsonFacets, TestJsonFacetErrors
  and TestJsonRangeFacet
* TestJsonFacets contains mainly terms faceting and stats
* range facet covers distributed cases too
2020-03-23 19:50:58 +05:30
Erick Erickson 5fd55d77e9 SOLR-12353: SolrDispatchFilter expensive non-conditional debug line degrades performance 2020-03-19 15:26:44 -04:00
Munendra S N 78e670f19e SOLR-14012: return long from unique and hll even for standalone 2020-03-19 19:21:57 +05:30
Munendra S N e9d6c24fb7 SOLR-11725: use corrected sample formula to calc stdDev in JSON facets
* Both stdDev and variance uses corrected sample formula to compute
  the values. This is similar to StatsComponent
2020-03-19 19:21:57 +05:30
David Smiley 4fd96bedc2
SOLR-14256: replaced EMPTY with empty() to fix deadlock 2020-03-18 08:31:29 -04:00
Munendra S N 7f37a55a8c SOLR-13944: remove redundant checks in SpellCheckCollator
* After SOLR-14073, these checks become redundant and add
  additional tests for spellcheck with collapse
2020-03-18 10:51:50 +05:30
Munendra S N 0b063fd2b7 SOLR-10157: improve error message in case of unknown aggregations 2020-03-18 10:46:57 +05:30
Munendra S N 6a59d443bc LUCENE-8908: return def val from objectVal when exists returns false
* This behavior is similar to floatVal in QueryValueSource
2020-03-18 10:41:13 +05:30
Munendra S N 2b327e5deb SOLR-13199: Fix NPE in ChildDocTransformer when parenFilter is invalid
* Return proper error message and error code
2020-03-18 10:32:28 +05:30
ameliahenderson fda935482b
SOLR-8306: Optimize expand.rows=0 to compute only total hits (#1334)
* When expand.rows=0, expand documents are not returned. So, computing them could be avoided and only total hits need to be computed
2020-03-18 09:23:04 +05:30
Mike Drob adb829cf35 [logging] log actual size of transient core cache 2020-03-13 15:45:52 -05:00
Mike 74721fa4c6
SOLR-14289 Skip ZkChroot check when not necessary (#1298) 2020-03-13 13:56:17 -05:00
Andrzej Bialecki 0f10b5f042 SOLR-13264: IndexSizeTrigger aboveOp / belowOp properties not in valid properties. 2020-03-13 12:30:17 +01:00
David Smiley 732348ec7f SOLR-14197: SolrResourceLoader refactorings to reduce API
* Remove SRL.listConfigDir (unused)
* Remove SRL.getDataDir
* Remove SRL.getCoreName
* Remove SRL.getCoreProperties
 XmlConfigFile needs to be passed in the substitutableProperties
 IndexSchema needs to be passed in the substitutableProperties
 Remove redundant Properties from CoreContainer constructors
* Remove SRL.newAdminHandlerInstance (unused)
* Remove SRL.openSchema and openConfig
* Avoid SRL.getConfigDir
 Also harmonized similar initialization logic between DIH Tika processor & ExtractingRequestHandler.
* Ensure SRL.addToClassLoader and reloadLuceneSPI are called at most once
 Don't auto-load "lib" in constructor; wrong place for this logic.
* Avoid SRL.getInstancePath
 Added SolrCore.getInstancePath instead
 Use CoreContainer.getSolrHome instead
 NodeConfig should track solrHome separate from SolrResourceLoader
* Simplify some SolrCore constructors
* Move locateSolrHome to new SolrPaths
* Move "User Files" stuff to SolrPaths
2020-03-11 09:20:28 -04:00
Joel Bernstein 01688cd8f1 SOLR-14073: Fix segment look ahead NPE in CollapsingQParserPlugin 2020-03-08 20:26:04 -04:00
noble 4cf37ade35 Revert "SOLR-13942: /api/cluster/zk/* to fetch raw ZK data"
This reverts commit bc6fa3b650.
2020-03-06 23:08:34 +11:00
Yannick Welsch 8a88dd02c6 Remove SimpleFSDirectory in favor of NIOFSDirectory 2020-03-06 00:04:25 +01:00
Tomas Fernandez Lobbe 2cabeb924d
Add try-with-resources to CloudManager in tests (#1315) 2020-03-04 13:32:44 -08:00
Mike Drob dd9b9f2f7f SOLR-14223 Fix tests for windows 2020-03-04 10:50:00 -08:00
Tomas Fernandez Lobbe 4d2d052b7d
Fix resource leak in TestPolicyCloud (#1312) 2020-03-03 11:42:20 -08:00
Noble Paul bc6fa3b650
SOLR-13942: /api/cluster/zk/* to fetch raw ZK data 2020-03-03 22:36:17 +11:00
Pushkar Raste 17c576a36f SOLR-14299 IndexFetcher doesn't reset count to 0 after the last packet is received 2020-03-02 16:26:25 -08:00
Shalin Shekhar Mangar 4897a64713 SOLR-13996: Rename LegacyReplicaSource to StandaloneReplicaSource 2020-03-01 07:00:51 +05:30
Jan Høydahl 62f5bd50cd
SOLR-14281: Make sharedLib configurable through SysProp (#1288) 2020-02-27 14:16:33 +01:00
Andrzej Bialecki c6bf8b6cec SOLR-14252: NullPointerException in AggregateMetric. 2020-02-27 13:00:43 +01:00
Mikhail Khludnev 3befb8be94 SOLR-13411: reject incremental update for route.field, uniqueKey and
_version_.
2020-02-27 14:51:34 +03:00
noble edefcf156b SOLR-14270: fix incorrect date format 2020-02-27 17:28:31 +11:00
Christine Poerschke fc24fa9506 SOLR-14267: complete <query><HashDocSet> solrconfig.xml removal 2020-02-26 17:47:25 +00:00
Eric Pugh b4f3efbf94 SOLR-13965: In GraphHandler, support <expressible> configuration and deprecate <streamFunctions> configuration.
(Eric Pugh via Christine Poerschke)

Closes #1033 pull request.
2020-02-26 17:36:33 +00:00
Alan Woodward 98dafe2e10
LUCENE-9207: Don't build span queries in QueryBuilder (#1239)
QueryBuilder currently has special logic for graph phrase queries with no slop,
constructing a spanquery that attempts to follow all paths using a combination of
OR and NEAR queries. However, this type of query has known bugs(LUCENE-7398).
This commit removes this logic and just builds a disjunction of phrase queries, one 
phrase per path.
2020-02-26 14:32:34 +00:00
Chris Hostetter 852f02b4b7 Some additional testing to prove 'defered function computation' in a syntax variant i was recently asked about 2020-02-25 13:21:13 -07:00
David Smiley 50a7075862
SOLR-14256: Remove HashDocSet; add DocSet.getBits.
* DocSet is now fixed at two implementations, read-only and doc ordered.
* DocSetBase removed; not needed anymore.  DocSet is now an abstract class.
2020-02-25 09:48:20 -05:00
David Smiley 25892271e8
SOLR-14258: DocList should not extend DocSet 2020-02-25 09:39:43 -05:00
Mike 1770797387
SOLR-14223 Create RSAKeyPair from disk (#1217)
* Create properties for PublicKeyHandler to read existing keys from disk
* Move pregenerated keys from core/test-files to test-framework
* Update tests to use existing keys instead of new keys each run
2020-02-24 12:07:10 -08:00
Anshum Gupta 7ba9d4d756
SOLR-14272: Remove autoReplicaFailoverBadNodeExpiration and autoReplicaFailoverWorkLoopDelay for 9.0 as it was deprecated in 7.1 (#1269)
* SOLR-14272: Remove autoReplicaFailoverBadNodeExpiration and autoReplicaFailoverWorkLoopDelay for 9.0 as it was deprecated in 7.1
2020-02-24 09:46:05 -08:00
Alessandro Benedetti 663611c99c
[SOLR-12238] Synonym Queries boost (#357)
SOLR-12238: Handle boosts in QueryBuilder

QueryBuilder now detects per-term boosts supplied by a BoostAttribute when
building queries using a TokenStream.  This commit also adds a DelimitedBoostTokenFilter
that parses boosts from tokens using a delimiter token, and exposes this in Solr
2020-02-24 10:29:41 +00:00
Jan Høydahl 57c7139ea3
SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in production (#1273) 2020-02-24 09:59:33 +01:00
Anshum Gupta bc41f25cc9
SOLR-14271: Delete CollectionsAPIAsyncDistributedZkTest.testAsyncIdBackCompat as that is no longer required (#1275) 2020-02-21 09:53:10 -08:00
Erick Erickson 292bed0c42 SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload 2020-02-21 10:57:16 -05:00
Jan Høydahl 89b13377a1
SOLR-14250: Do not log error when trying to consume non-existing input stream due to Expect: 100-continue (#1250) 2020-02-21 10:30:10 +01:00
Noble Paul 9f3f7244ac
SOLR-14270 export command to have an option to write to a zip file (#1266) 2020-02-21 13:41:50 +11:00
Mike Drob 79966132fc SOLR-14264 Set minimum gzip size for responses 2020-02-20 17:14:21 -06:00
Anshum Gupta cea4226367
SOLR-14271: Remove duplicate async id check meant for pre Solr 8 versions (#1268)
* SOLR-14271: Remove duplicate async id check meant for pre Solr 8 versions
2020-02-20 15:13:05 -08:00
Eric Pugh f23def6b72 SOLR-13965: s/StreamHandler/GraphHandler fix GraphHandler.getDescription()
(Eric Pugh via Christine Poerschke)
2020-02-18 14:32:52 +00:00
Eric Pugh 5d32c04096 SOLR-13965: StreamHandler class-level javadoc edits
(Eric Pugh via Christine Poerschke)
2020-02-18 14:31:44 +00:00
Chris Hostetter f549ee3535 SOLR-13794: Replace redundent test only copy of '_default' configset with SolrTestCase logic to correctly set 'solr.default.confdir' system property
This change allows us to remove kludgy test only code from ZkController
2020-02-14 11:36:53 -07:00
Erick Erickson 0767a9d4d7 Code comment only change 2020-02-11 19:32:54 -05:00
yonik c3e44e1fec SOLR-14058: fix peersync bounds check iterating over versions 2020-02-11 10:43:21 -08:00
David Smiley 9a4f7661e9
SOLR-14194: Highlighters now supports docValues for the uniqueKey
and the original highlighter can highlight docValues.
2020-02-11 02:18:08 -05:00
Mike 71b869381e
SOLR-14247 Remove unneeded sleeps (#1244) 2020-02-10 21:13:56 -06:00
Shalin Shekhar Mangar c65b97665c
SOLR-13996: Refactor HttpShardHandler.prepDistributed method (#1220)
SOLR-13996: Refactor HttpShardHandler.prepDistributed method into smaller pieces

This commit introduces an interface named ReplicaSource which is marked as experimental. It has two sub-classes named CloudReplicaSource (for solr cloud) and LegacyReplicaSource for non-cloud clusters. The prepDistributed method now calls out to these sub-classes depending on whether the cluster is running on cloud mode or not.
2020-02-10 19:57:05 +05:30
Kevin Risden c4a8a77d23
SOLR-14209: Upgrade JQuery to 3.4.1
* JQuery 2.1.3 to 3.4.1
* jstree 1.0-rc1 to v3.3.8

Closes #1209

Signed-off-by: Kevin Risden <krisden@apache.org>
2020-02-08 11:57:56 -06:00
Robert Muir f41eabdc5f
LUCENE-8279: fix javadocs wrong header levels and accessibility issues
Java 13 adds a new doclint check under "accessibility" that the html
header nesting level isn't crazy.

Many are incorrect because the html4-style javadocs had horrible
font-sizes, so developers used the wrong header level to work around it.
This is no issue in trunk (always html5).

Java recommends against using such structured tags at all in javadocs,
but that is a more involved change: this just "shifts" header levels
in documents to be correct.
2020-02-08 10:00:00 -05:00
Shalin Shekhar Mangar f5c132be6d SOLR-14248: Improve ClusterStateMockUtil and make its methods public 2020-02-08 11:59:27 +05:30
Nicholas Knize 206a70e7b7 LUCENE-9149: Increase data dimension limit in BKD 2020-02-07 16:08:14 -06:00
Andrzej Bialecki 9a19093586 SOLR-14245: Validate Replica / ReplicaInfo on creation. 2020-02-07 17:56:39 +01:00
Robert Muir 0d339043e3
LUCENE-9209: fix javadocs to be html5, enable doclint html checks, remove jtidy
Current javadocs declare an HTML5 doctype: !DOCTYPE HTML. Some HTML5
features are used, but unfortunately also some constructs that do not
exist in HTML5 are used as well.

Because of this, we have no checking of any html syntax. jtidy is
disabled because it works with html4. doclint is disabled because it
works with html5. our docs are neither.

javadoc "doclint" feature can efficiently check that the html isn't
crazy. we just have to fix really ancient removed/deprecated stuff
(such as use of tt tag).

This enables the html checking in both ant and gradle. The docs are
fixed via straightforward transformations.

One exception is table cellpadding, for this some helper CSS classes
were added to make the transition easier (since it must apply padding
to inner th/td, not possible inline). I added TODOs, we should clean
this up. Most problems look like they may have been generated from a
GUI or similar and not a human.
2020-02-06 22:30:52 -05:00
Adrien Grand fe349ddcf2
SOLR-14242: HdfsDirectory#createTempOutput. (#1240) 2020-02-05 16:38:53 +01:00
Adrien Grand 2d8428ec2e
SOLR-14238: Fix HdfsDirectory to no longer overwrite existing files. (#1237) 2020-02-04 19:35:15 +01:00
Tomas Fernandez Lobbe bb90569f1d
SOLR-14219: Revert changes in OverseerSolrRespose and move serialization (#1227)
SOLR-14095 Introduced an issue for rolling restarts (Incompatible Java serialization). This change fixes the compatibility issue while keeping the functionality in SOLR-14095
2020-02-04 10:26:57 -08:00
Munendra S N c91dd9d0e4 SOLR-14090: fix delete-copy-field when source is dynamic field 2020-02-04 21:33:31 +05:30
Munendra S N 4eff9c9b5e SOLR-10567: add support for DateRangeField in JSON facet range 2020-02-04 21:26:40 +05:30
Erick Erickson b0bb299dc4
LUCENE-9134: Port ant-regenerate tasks to Gradle build (#1230)
LUCENE-9134: Port ant-regenerate tasks to Gradle build (Solr javacc)
2020-02-04 09:16:38 -05:00
Erick Erickson d3ac1329a3
LUCENE-8656: Deprecations in FuzzyQuery (#1229)
LUCENE-8656: Deprecations in FuzzyQuery

Closes #1229
2020-02-03 08:52:33 -05:00
Mikhail Khludnev d8bc9bcfcf SOLR-12325: uniqueBlock(\{!v=foo:bar}) 2020-02-02 15:15:35 +03:00
Jason Gerlowski 719b38c8d8
SOLR-13892: Add 'top-level' docValues Join implementation (#1171) 2020-01-31 11:21:01 -05:00
Adrien Grand 7941d109bd SOLR-13897: Fix precommit. 2020-01-28 20:11:47 +01:00
Robert Muir 975df9ddd3
LUCENE-9182: add apache license headers to all .gradle files and enforce in rat task 2020-01-27 12:05:34 -05:00
Shalin Shekhar Mangar 776631254f SOLR-13897: Fix unsafe publication of Terms object in ZkShardTerms that can cause visibility issues and race conditions under contention 2020-01-27 12:08:20 +05:30
andywebb1975 efd0e8f3e8 SOLR-14189 switch from String.trim() to StringUtils.isBlank() (#1172) 2020-01-26 12:03:39 +01:00
Mike eaa3dbe440
SOLR-14162 TestInjection can leak Timer objects (#1137) 2020-01-24 14:04:22 -06:00
Kevin Risden b150202c85
SOLR-11554: Support handling OPTIONS request for Hadoop authentication filter
Signed-off-by: Kevin Risden <krisden@apache.org>
2020-01-23 13:45:54 -05:00
Andrzej Bialecki 434f90265b SOLR-14211: Fix a bug introduced in SOLR-14192. 2020-01-23 19:33:12 +01:00
Anshum Gupta 3c0146196a
SOLR-14206: Annotate HttpSolrCall as thread-safe (#1203)
* SOLR-14206: Annotate HttpSolrCall and V2HttpCall as thread-safe
2020-01-23 08:37:03 -08:00
Jan Høydahl e744f7977e
SOLR-14196: AdminUI login not working for JWTAuth when blockUnknown=false (#1190) 2020-01-23 14:51:36 +01:00
Shalin Shekhar Mangar 84270dc6cf SOLR-14172: Collection metadata remains in zookeeper if too many shards are requested.
This also fixes a bug where an inability to assign a node based on existing autoscaling policy resulted in a server error instead of a bad request.

This closes #1152.
2020-01-23 15:19:53 +05:30
Shalin Shekhar Mangar 04193d5252 SOLR-14207: Fix logging statements with less or more arguments than placeholders 2020-01-23 14:00:08 +05:30
Houston Putman ffba54a827 SOLR-11746: Adding existence queries for PointFields
* DocValuesFieldExistsQuery and NormsFieldExistsQuery are used for existence queries when possible.
* Added documentation on the difference between field:* and field:[* TO *]
2020-01-22 18:00:55 -05:00
Chris Hostetter 6b3e7feba1 SOLR-14159: Eliminate some 'spin loops' in tests that may be contributing factors to odd test failures 2020-01-22 14:44:56 -07:00
Jan Høydahl 75c64089a1
SOLR-14198: Nullpointer exception in AuditEvent with AuthorizationContext (#1192) 2020-01-22 21:39:43 +01:00
David Smiley f1db918e20
SOLR-14040: restore legacy Collection auto-creation 2020-01-22 15:26:37 -05:00
Chris Hostetter 95dfddc7d4 SOLR-12859: Fixed DocExpirationUpdateProcessorFactory to work with BasicAuth and other auth plugins that delegate to PKI for server initiated node-to-node communication. 2020-01-22 09:32:25 -07:00
Christine Poerschke 1af171e47f SOLR-13965: Factor out public static StreamHandler.addExpressiblePlugins method. 2020-01-21 18:33:26 +00:00
David Smiley ab924fd4ea
SOLR-14040: shareSchema support for SolrCloud
* Use Caffeine impl and weak values (to the schema). Previously the cache never evicted!
* now populating the configSet name from ZK into CloudDescriptor when CloudDescriptor is loaded
* actual schema name needs to be deterministic now; fallback from non-existent managed-schema to schema.xml will thwart this cache
* a test conf/core.properties wasn't actually used and became a problem in it's weird location after I refactored some logic
2020-01-21 13:18:37 -05:00
Andrzej Bialecki 6244b7150e SOLR-14192: Race condition between SchemaManager and ZkIndexSchemaReader. 2020-01-20 17:11:47 +01:00
Mike 338d386ae0
LUCENE-9145 First pass addressing static analysis (#1181)
Fixed a bunch of the smaller warnings found by error-prone compiler
plugin, while ignoring a lot of the bigger ones.
2020-01-17 13:30:39 -06:00
Chris Hostetter 5f2d7c4855 SOLR-14184: Internal 'test' variable DirectUpdateHandler2.commitOnClose has been removed and replaced with TestInjection.skipIndexWriterCommitOnClose 2020-01-16 11:47:06 -07:00
Christine Poerschke 94ee211366 Fix 'Restar[t]ing' typo in RecoveryStrategy INFO log message. 2020-01-16 18:13:47 +00:00
Joel Bernstein 35d8e3de6d SOLR-14130: Continue to improve log parsing logic 2020-01-16 11:36:11 -05:00
Jason Gerlowski 424ace6f5d
SOLR-14186: Enforce CRLF in Windows files with .gitattributes (#1163) 2020-01-16 08:30:39 -05:00
Alan Woodward 7ea7ed72ac LUCENE-9068: Solr query handling code catches FuzzyTermsException 2020-01-16 09:52:18 +00:00
Andrzej Bialecki 543505470c SOLR-14128: Tentative fix: put replicas on other nodes than overseer, wait for
all replicas to complete the reload.
2020-01-16 10:42:26 +01:00
Joel Bernstein 4c473db99d SOLR-14130: Add parsing instructions for different types of query records 2020-01-15 15:01:28 -05:00
Dawid Weiss 3008dd9526 Merge remote-tracking branch 'origin/master' into gradle-master 2020-01-13 17:55:53 +01:00
0xflotus 5a73ad0178 Two minor Javadoc cleanups (#1002) 2020-01-13 09:22:04 -05:00
Jason Gerlowski 6e4756fd48
SOLR-13890: Add "top-level" DV "terms" implementation (#1151)
{!terms} queries have a docValues-based implementation that uses per-segment DV structures.  This does well with a small to moderate (a few hundred) number of query terms, but doesn't well scale beyond that due to repetitive seeks done on each segment.

This commit introduces an implementation that uses a "top-level" docValues structure, which scales much better to very large {!terms} queries (many hundreds, thousands of terms).
2020-01-13 06:43:21 -05:00
Dawid Weiss f9dde4de52 Merge remote-tracking branch 'origin/master' into gradle-master 2020-01-13 08:37:15 +01:00
Chris Hostetter 9a2497f637 SOLR-13486: Fix trivial test bug in TestTlogReplayVsRecovery
Add TODOs for future test improvements once underlying race condition is fixed in core code
2020-01-11 16:48:05 -07:00