Commit Graph

904 Commits

Author SHA1 Message Date
Munendra S N 36b280bd0a SOLR-11775: return long val for facet count in json facet
* Long value is returned for any count related to json facets
  irrespective of number of shards
2020-04-10 19:30:20 +05:30
Andy Throgmorton d1601f6fdf SOLR-14260: SolrJ pluggable ConnectionSocketFactory in HttpClientUtil
see SocketFactoryRegistryProvider
Fixes #1261
2020-03-26 22:15:50 -04:00
Mike Drob a31ecd2eb8 SOLR-14322 Improve AbstractFullDistribZkTestBase.waitForThingsToLevelOut 2020-03-26 17:38:05 -05:00
Uwe Schindler 2c7a710945
LUCENE-9281: Retire SPIClassIterator from master because Java 9+ uses different mechanism to load services when module system is used (#1360)
LUCENE-9281: Use java.util.ServiceLoader to load codec components and analysis factories to be compatible with Java Module System
2020-03-25 18:03:36 +01:00
David Smiley 732348ec7f SOLR-14197: SolrResourceLoader refactorings to reduce API
* Remove SRL.listConfigDir (unused)
* Remove SRL.getDataDir
* Remove SRL.getCoreName
* Remove SRL.getCoreProperties
 XmlConfigFile needs to be passed in the substitutableProperties
 IndexSchema needs to be passed in the substitutableProperties
 Remove redundant Properties from CoreContainer constructors
* Remove SRL.newAdminHandlerInstance (unused)
* Remove SRL.openSchema and openConfig
* Avoid SRL.getConfigDir
 Also harmonized similar initialization logic between DIH Tika processor & ExtractingRequestHandler.
* Ensure SRL.addToClassLoader and reloadLuceneSPI are called at most once
 Don't auto-load "lib" in constructor; wrong place for this logic.
* Avoid SRL.getInstancePath
 Added SolrCore.getInstancePath instead
 Use CoreContainer.getSolrHome instead
 NodeConfig should track solrHome separate from SolrResourceLoader
* Simplify some SolrCore constructors
* Move locateSolrHome to new SolrPaths
* Move "User Files" stuff to SolrPaths
2020-03-11 09:20:28 -04:00
Mike 1770797387
SOLR-14223 Create RSAKeyPair from disk (#1217)
* Create properties for PublicKeyHandler to read existing keys from disk
* Move pregenerated keys from core/test-files to test-framework
* Update tests to use existing keys instead of new keys each run
2020-02-24 12:07:10 -08:00
Erick Erickson 292bed0c42 SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload 2020-02-21 10:57:16 -05:00
Chris Hostetter f549ee3535 SOLR-13794: Replace redundent test only copy of '_default' configset with SolrTestCase logic to correctly set 'solr.default.confdir' system property
This change allows us to remove kludgy test only code from ZkController
2020-02-14 11:36:53 -07:00
Chris Hostetter f1fc3e7ba2 SOLR-14247: Revert SolrTestCase Logger removal 2020-02-12 11:51:36 -07:00
Mike 71b869381e
SOLR-14247 Remove unneeded sleeps (#1244) 2020-02-10 21:13:56 -06:00
Shalin Shekhar Mangar c65b97665c
SOLR-13996: Refactor HttpShardHandler.prepDistributed method (#1220)
SOLR-13996: Refactor HttpShardHandler.prepDistributed method into smaller pieces

This commit introduces an interface named ReplicaSource which is marked as experimental. It has two sub-classes named CloudReplicaSource (for solr cloud) and LegacyReplicaSource for non-cloud clusters. The prepDistributed method now calls out to these sub-classes depending on whether the cluster is running on cloud mode or not.
2020-02-10 19:57:05 +05:30
Robert Muir 975df9ddd3
LUCENE-9182: add apache license headers to all .gradle files and enforce in rat task 2020-01-27 12:05:34 -05:00
Robert Muir 5f964eeef2
SOLR-14217: tests respect tests.workDir correctly (prevent SSD destruction) 2020-01-27 06:07:48 -05:00
Shalin Shekhar Mangar 04193d5252 SOLR-14207: Fix logging statements with less or more arguments than placeholders 2020-01-23 14:00:08 +05:30
David Smiley ab924fd4ea
SOLR-14040: shareSchema support for SolrCloud
* Use Caffeine impl and weak values (to the schema). Previously the cache never evicted!
* now populating the configSet name from ZK into CloudDescriptor when CloudDescriptor is loaded
* actual schema name needs to be deterministic now; fallback from non-existent managed-schema to schema.xml will thwart this cache
* a test conf/core.properties wasn't actually used and became a problem in it's weird location after I refactored some logic
2020-01-21 13:18:37 -05:00
Mike 338d386ae0
LUCENE-9145 First pass addressing static analysis (#1181)
Fixed a bunch of the smaller warnings found by error-prone compiler
plugin, while ignoring a lot of the bigger ones.
2020-01-17 13:30:39 -06:00
Chris Hostetter 5f2d7c4855 SOLR-14184: Internal 'test' variable DirectUpdateHandler2.commitOnClose has been removed and replaced with TestInjection.skipIndexWriterCommitOnClose 2020-01-16 11:47:06 -07:00
Dawid Weiss 5897b78572 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-20 17:35:40 +01:00
Kevin Risden aab3c5faa3
SOLR-14106: Cleanup Jetty SslContextFactory usage
Jetty 9.4.16.v20190411 and up introduced separate
client and server SslContextFactory implementations.
This split requires the proper use of of
SslContextFactory in clients and server configs.

This fixes the following
* SSL with SOLR_SSL_NEED_CLIENT_AUTH not working since v8.2.0
* Http2SolrClient SSL not working in branch_8x

Signed-off-by: Kevin Risden <krisden@apache.org>
2019-12-19 23:05:47 -05:00
Robert Muir 7490bfd828 SOLR-14085: remove solr fork of lucene test securitymanager 2019-12-18 23:06:31 -05:00
Chris Hostetter d30f90e349 SOLR-14099: expanded comment on static final variable based on followup questions in Jira from Dawid 2019-12-18 16:36:34 -07:00
Dawid Weiss 28b19c2af2 Merge with master. 2019-12-18 09:32:35 +01:00
Chris Hostetter cc4262a8ec SOLR-14099: Fixed @LogLevel annotation in test-framework to correctly 'unset' Loggers after test 2019-12-17 14:58:45 -07:00
Dawid Weiss 6094d4dd13 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-12 14:16:48 +01:00
Ishan Chattopadhyaya fa1ad006b6 SOLR-14071: Untrusted configsets shouldn't be allowed to use <lib> directive 2019-12-12 15:00:09 +05:30
Robert Muir dc031ea382 SOLR-14050: clean up tests use of network addresses
Solr tests now have a similar policy to Lucene, loopback use only. If a
test tries to resolve or connect to the internet, it will get SecurityException.

Some solr tests explicitly try to talk to dead nodes with real
networking. This is not good and asking for trouble, but use low loopback port numbers instead of
multicast addresses. The idea is that it fails faster. Move these to
constants so that stuff isn't copy-pasted everywhere, in case we have to
do something different later.
2019-12-11 12:51:45 -05:00
Dawid Weiss 27d5509644 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-11 08:57:18 +01:00
Kevin Risden 48775ea18e
SOLR-14033: Fix Hadoop tests with security manager
This removes the Solr security manager hacks
for Hadoop. It does so by:
* Using a fake group mapping class instead of ShellGroupMapping
* Copies a few Hadoop classes and modifies them for tests with no Shell
* Nulls out some of the static variables in the tests

The Hadoop files were copied from Apache Hadoop 3.2.0
and copied to the test package to be only picked up
during tests. They were modified to remove the need to
shell out for access. The assumption is that these
HDFS integration tests only run on Unix based systems
and therefore Windows compatibility was removed in some
of the modified classes. The long term goal is to remove
these custom Hadoop classes. All the copied classes are
in the org.apache.hadoop package.

Signed-off-by: Kevin Risden <krisden@apache.org>
2019-12-10 16:15:24 -05:00
Dawid Weiss 20eaaa012d Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-10 14:52:30 +01:00
Houston Putman 1c78d2c637
SOLR-12217: Support shards.preference in SolrJ for individual shard requests (#984) 2019-12-09 17:11:58 -05:00
Dawid Weiss 0d18581225 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-06 19:02:26 +01:00
Dawid Weiss a7444f7af4 LUCENE-9077: Add support for running under gradle test runner. 2019-12-06 19:00:50 +01:00
Dawid Weiss f371df327f Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-06 11:57:29 +01:00
Robert Muir 33ca971d2b SOLR-14020: move hadoop hacks out of lucene TestSecurityManager into a solr one 2019-12-05 14:53:23 -05:00
Dawid Weiss d4a9842375 Initial gradle build layer. 2019-12-02 15:34:57 +01:00
Dawid Weiss 063c82ebd6 SOLR-13952: reverting Erick's commit (with permission). 2019-11-25 17:56:20 +01:00
Erick Erickson 4b34d726ab SOLR-13952: Separate out Gradle-specific code from other (mostly test) changes and commit separately 2019-11-24 13:24:40 -05:00
Chris Hostetter 3a5d67b87b SOLR-13869: remove deprecated SolrTestCaseJ4.getNextAvailablePort() from master 2019-11-12 14:03:39 -07:00
Chris Hostetter 603be023fe SOLR-13864: SolrTestCaseJ4.getNextAvailablePort() has been deprecated 2019-11-12 13:41:46 -07:00
Koen De Groote e7e6cfaecf LUCENE-8994: Code Cleanup - Pass values to list constructor instead of empty constructor followed by addAll(). (#919) 2019-10-14 18:45:47 +02:00
Noble Paul e3bd5a7da2
SOLR-13821: Package Store for storing package artifacts (#929)
SOLR-13821: Package Store for storing package artifacts
2019-10-08 14:24:33 +11:00
Ishan Chattopadhyaya 515af6d3ec SOLR-13661: Reverting all half-baked stuff from SOLR-13707, SOLR-13659, SOLR-13565, SOLR-13650, SOLR-13710, SOLR-13721, SOLR-13637
All half baked package management and hot-classloading code reverted to allow for a fresh start.
2019-10-06 18:00:40 +05:30
Christine Poerschke b51013a10b SOLR-13812: Add javadocs, uneven rejection and basic test coverage for the SolrTestCaseJ4.params method.
(Diego Ceccarelli, Christine Poerschke, Munendra S N)
2019-10-04 11:18:33 +01:00
Chris Hostetter 0ec7986fc0 Minor logging improvements 2019-09-30 10:00:33 -07:00
Erick Erickson 4f89987141 :SOLR-13454: Investigate ReindexCollectionTest failures, added more safeguards in bandaid code 2019-09-29 12:25:43 -04:00
Chris Hostetter ec9780c8aa SOLR-13747: New TestSSLTestConfig.testFailIfUserRunsTestsWithJVMThatHasKnownSSLBugs() to give people running tests more visibility if/when they use a known-buggy JVM causing most SSL tests to silently SKIP 2019-09-26 14:12:20 -07:00
Jason Gerlowski 319cb005d3 SOLR-13539: Introduce EmbeddedSolrServerTestBase
This groundwork commit allows tests to randomize request content-type
more flexibly.  This will be taken advantage of by subsequent commits.

Co-Authored-By: Thomas Woeckinger
Closes: #755
2019-08-29 09:52:46 -04:00
Noble Paul cf21340294
SOLR-13723: JettySolrRunner should support /api/* (the v2 end point) 2019-08-29 14:01:52 +10:00
noble 54bd950819 formatting 2019-08-28 14:34:35 +10:00
Chris Hostetter 2fdc882bc4 Additional logging in test framework methods that 'waitFor' something to better trace order of operations when failures occur 2019-08-21 13:13:40 -07:00
Noble Paul a4b0ab530f
SOLR-13650: Support for named global classloaders 2019-08-20 05:00:38 +10:00
Chris Hostetter f5856ef404 SOLR-13701: Fixed JWTAuthPlugin to update metrics prior to continuing w/other filters or returning error 2019-08-18 12:20:51 -07:00
Chris Hostetter 0291db44bc SOLR-13464: fix javadoc typo that precommit somehow missed? 2019-08-12 14:55:17 -07:00
Chris Hostetter c7822c393e SOLR-13464: Test work arounds
* Refactor existing work around in BasicAuthIntegrationTest up into SolrCloudAuthTestCase for re-use in JWTAuthPluginIntegrationTest

 * Simplify BasicAuthOnSingleNodeTest and PKIAuthenticationIntegrationTest to use their existing (static) security settings on creation of MiniSolrCloud.  Since they no longer modify security.json once the nodes are alive, the issue no longer affects them
2019-08-12 14:03:54 -07:00
Tomoko Uchida 9b986d268f SOLR-13593: Allow to look up analyzer components by their SPI names in field type configuration. 2019-08-11 16:35:25 +09:00
Munendra S N 8c4fde94fe SOLR-12555: use expectThrows() to verify the ex thrown in tests 2019-08-03 13:00:49 +05:30
Chris Hostetter ab470a6564 SOLR-13664: Fixed SolrTestCaseJ4.deleteCore() to properly reset the dataDir used by initCore() 2019-08-01 09:15:18 -07:00
Chris Hostetter 6dea203d11 SOLR-13660: Fixed AbstractFullDistribZkTestBase.waitForActiveReplicaCount() to ensure replicas are active. 2019-07-30 10:14:38 -07:00
Munendra S N 96d400f511 SOLR-12870: use StandardCharsets instead of String values
Fixes #469
2019-07-20 10:10:32 +05:30
Noble Paul bd30c3701e
SOLR-13565, SOLR-13553 (#774)
* SOLR-13565: initial commit

* SOLR-13565: updated with testcase

* SOLR-13565: removed unused methods

* SOLR-13565: better logging

* SOLR-13565: disable SSL

* SOLR-13565: more tests

* SOLR-13565: syncing with master

* SOLR-13565: fixing tests

* SOLR-13565: fixing tests

* SOLR-13534: Fix test

Remove buggy 'port roulette' code that can easily fail if OS gives the selected port to a different process just before creating the server

Use jetty's built in support for listining on an OS selected port instead

Also increase timeouts to better account for slow/heavily loaded (ie:jenkins) VMs where SolrCore reloading may take longer then 10 seconds

* SOLR-13565: set proper permission name

* SOLR-13565: syncing with master

* SOLR-13565: syncing with master

* SOLR-13565: removed accidental change

* SOLR-13565: removed accidental change

* SOLR-13565: removed accidental change

* SOLR-13565: more tests

* SOLR-13565: Tests with key signing tests

* SOLR-13565: fixing concurrency issues in tests

* SOLR-13565: add tests with 512 bit RSA

* SOLR-13565: fixing concurrency issues

* SOLR-13565: remove unused code
2019-07-20 14:20:32 +10:00
Munendra S N 0e877aac34 SOLR-12554: Expose IndexWriterConfig's ramPerThreadHardLimitMB
* When ramPerThreadHardLimitMB is not specified, then Lucene's
  default value 1945 is used. The specified value should be
  greater than 0 and less than 2048MB
2019-07-01 22:42:19 +05:30
Chris Hostetter aaf20aefa4 SOLR-12988: SSLTestConfig has been changed to throw AssumptionViolatedException when tests/seeds request SSL but the JVM appears to be an OpenJDK version known to have SSL bugs
SOLR-13574: Add CHANGES entry that was overlooked
2019-06-27 15:51:20 -07:00
Chris Hostetter 7e57d3a9d9 SOLR-13574: Fix many test and test-framework classes to not fail on After/AfterClass cleanup if assumptions fail in Before/BeforeClass setup 2019-06-27 10:28:06 -07:00
Chris Hostetter 689fa583a0 Revert "SOLR-12988: SSLTestConfig has been changed to throw AssumptionViolatedException when tests/seeds request SSL but the JVM appears to be an OpenJDK version known to have SSL bugs"
This reverts commit 6d6f14d391.

Reason for revert: after doing more testing I realized there are tests I overlooked which can (with randomized SSL usage) trigger NullPointerException
(or other related failures) in After/AfterClass due assumptions about cleanup that isn't actaully neccessary due to the AssumptionFailure
that may occur during Before/BeforeClass
2019-06-24 15:42:23 -07:00
Chris Hostetter 6d6f14d391 SOLR-12988: SSLTestConfig has been changed to throw AssumptionViolatedException when tests/seeds request SSL but the JVM appears to be an OpenJDK version known to have SSL bugs 2019-06-24 15:02:18 -07:00
Chris Hostetter c8c2f2f25b SOLR-12988: Fix Revert
an (erroneous) assigment to useSsl was left in the constructor after the (intended) conditional assigment, rendering the conditional logic useless
2019-06-20 16:04:49 -07:00
Cao Manh Dat e3752e87d0 SOLR-12988: Revert changes 2019-06-19 21:07:38 +01:00
Cao Manh Dat 91944a468e SOLR-12988: Skip running tests with SSL on Java 11 to 11.0.2 2019-06-19 10:43:17 +01:00
Cao Manh Dat c8382890ae SOLR-12988: Avoid using TLSv1.3 for HttpClient 2019-06-18 12:49:51 +01:00
Mikhail Khludnev 4f6314c59b SOLR-7530: /terms responds per field arrays in JSON by default 2019-06-18 08:38:02 +02:00
Chris Hostetter 5a974860fa SOLR-13490: Fix CollectionStateWatcher/CollectionStatePredicate based APIs in ZkStateReader and CloudSolrClient to be triggered on liveNode changes.
Also add Predicate<DocCollection> equivilents for callers that don't care about liveNodes.
2019-06-17 09:59:43 -07:00
Cao Manh Dat 0c24aa6c75 SOLR-13541: Upgrade Jetty to 9.4.19.v20190610 2019-06-14 15:46:19 +01:00
Erick Erickson bfb5b41144 SOLR-12013: collections API CUSTERSTATUS command fails when configset missing 2019-06-11 15:56:02 -07:00
Koen De Groote 67104dd615 LUCENE-8847: Code Cleanup: Rewrite StringBuilder.append with concatted strings (#707)
This specific commit affects all points in the casebase where the argument of a StringBuilder.append() call is itself a regular String concatenation.
This defeats the purpose of using StringBuilder and also introduces an extra alloction.
These changes should avoid that.

ant tests have run, succeeded on local machine.

Removing test files from the changes.

Another suggested rework.
2019-06-10 18:07:43 +02:00
Cao Manh Dat 301ea0e462
SOLR-13434: OpenTracing support for Solr (#685) 2019-06-04 20:04:11 +01:00
Erick Erickson 7ebeab71f4 SOLR-8346: Upgrade Zookeeper to version 3.5.5 2019-06-03 17:50:35 -07:00
Erick Erickson 0aaf543208 SOLR-13454: Investigate ReindexCollectionTest failures. I missed a place where I should have called the bandaid code 2019-05-10 13:56:04 -07:00
Erick Erickson 577be08bf2 SOLR-13454: Investigate ReindexCollectionTest failures 2019-05-09 12:45:47 -07:00
noble 733b071564 SOLR-13320 : add an update param failOnVersionConflicts=false to updates not fail when there is a version conflict 2019-05-07 15:20:02 +10:00
Mikhail Khludnev 5ca0602d28 SOLR-12291: fixing premature completion of async tasks
* extract async tracking methods from OverseerCollectionMessageHandler into the separate class
* replacing hashmap to named list to avoid entry loss
2019-04-30 09:57:42 +03:00
Ishan Chattopadhyaya dd9899b1c1 SOLR-5970: Return correct status upon collection creation failure 2019-04-28 23:16:29 +05:30
Ishan Chattopadhyaya 4b49bd99ca SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields 2019-04-28 23:01:58 +05:30
Chris Hostetter 1c3d23e58a SOLR-13336: fix CloudInspectUtil to use filter to eliminate risk of TooManyClausesException 2019-04-19 10:59:53 -07:00
Erick Erickson 2a3bd8e864 SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload bandaid for DocValuesNotIndexedTest 2019-04-18 21:38:01 -07:00
Uwe Schindler faaee86efb LUCENE-8738: Move to Java 11 as minimum Java version (merged branch: jira/LUCENE-8738)
Co-authored-by: Adrien Grand <jpountz@apache.org>
2019-04-16 14:00:09 +02:00
Moshe 8527ec11af SOLR-12638: Partial/Atomic updates of nested docs.
and [child] now works in RTG.
2019-04-10 03:02:59 -04:00
Jan Høydahl 361502da8a SOLR-12120: New AuditLoggerPlugin type allowing custom Audit logger plugins 2019-04-04 15:48:51 +02:00
Kevin Risden 6596ed1c16
SOLR-13075: Harden SaslZkACLProviderTest (Kevin Risden, Hrishikesh Gadre, Peter Cseh)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-04-03 10:14:49 -04:00
Gus Heck 2d690885e5 SOLR-13322 - let forbidden apis check for sysout in solr core 2019-04-02 22:45:33 -04:00
Bar Rotstein 5b7866b085 SOLR-12955: Refactored DistributedUpdateProcessor to put SolrCloud specifics into a subclass
Closes #528
2019-03-19 13:26:31 -04:00
erick 9272c29539 SOLR-13268: Clean up any test failures resulting from defaulting to async logging 2019-03-10 11:05:00 -07:00
Mikhail Khludnev b8d569aff0 SOLR-9882: reporting timeAllowed breach as partialResults instead of 500 error 2019-03-04 17:41:57 +03:00
Cao Manh Dat e99934b240 SOLR-13276: Adding Http2 equivalent classes of CloudSolrClient and HttpClusterStateProvider 2019-03-04 09:41:45 +00:00
Erick 0de3905ce7 SOLR-12055: Enable async logging by default SOLR-12753: Async logging ring buffer and OOM error 2019-02-22 12:27:02 -08:00
Tomas Fernandez Lobbe bcd90f1561
SOLR-12708: CREATE collection request doesn't fail or cleanup when the request fails (#568)
When performing an async CREATE requests, responses to internal requests are also included inside the "success" or "failed" elements of the general response. This will make the operation cleanup in case of failure, the same way as we do when the request is synchronous.
2019-02-20 13:24:46 -08:00
shalinmangar 7e2d40197c SOLR-13248: Autoscaling based replica placement is broken out of the box.
Solr 7.5 enabled autoscaling based replica placement by default but in the absence of default cluster policies, autoscaling can place more than 1 replica of the  same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not respected. Due to these reasons,  this issue reverts the default replica placement policy to the 'legacy' assignment policy that was the default until Solr 7.4.
2019-02-18 13:46:05 -08:00
markrmiller 4a3ddc94d8
SOLR-9515: Update to Hadoop 3 (Mark Miller, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-01 15:04:56 -05:00
Kevin Risden e4f202c1e3
Revert "SOLR-9515: Update to Hadoop 3"
This reverts commit 6bb24673f4.
2019-01-31 19:46:02 -05:00
markrmiller 6bb24673f4
SOLR-9515: Update to Hadoop 3
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-01-31 13:00:20 -05:00
Jan Høydahl d09c724a0e SOLR-12121: Fix test fails in JWTAuthPluginIntegrationTest.testMetrics 2019-01-24 12:43:27 +01:00