Commit Graph

7751 Commits

Author SHA1 Message Date
Christine Poerschke e9feb1146b SOLR-12941: remove unused import in IndexSizeTriggerTest 2019-05-28 16:26:11 +01:00
Andrzej Bialecki 04d781db8b SOLR-12941: Fix IndexSizeTrigger to correctly work with "aboveBytes" and "splitMethod=link" parameters. 2019-05-28 11:34:36 +02:00
Noble Paul c445ad4c75 SOLR-13493: /autoscaling/suggestions to be able to filter by type (#686)
* SOLR-13493: /autoscaling/suggestions to be able to filter by type
2019-05-28 07:58:39 +10:00
Erick Erickson 909910a9be SOLR-12562: Reverting Clean up RealTimeGetComponent.toSolrDoc (sha 97e7d8a3d7 and 15aa9dfb3d)
(cherry picked from commit 57b64f2684)
2019-05-27 08:38:33 -07:00
Erick Erickson 15aa9dfb3d SOLR-12652: Clean up RealTimeGetComponent.toSolrDoc
(cherry picked from commit 97e7d8a3d7)
2019-05-26 08:31:58 -07:00
Chris Hostetter 9465763628 Fix TestCloudSearcherWarming to work around SOLR-13490
Also clean up some crufty System.out/System.err pollution

(cherry picked from commit af4e1d324a)
2019-05-25 08:40:20 -07:00
Noble Paul e1fa16c324 SOLR-13484: autoscaling/diagnostics APIshould be able to give diagnostics output from config pasted as a payload 2019-05-24 08:06:21 +10:00
noble 93ce10e705 SOLR-13484:
autoscaling/diagnostics APIshould be able to give diagnostics output from config pasted as a payload
2019-05-24 07:54:46 +10:00
Erick Erickson 75f8ae55d0 SOLR-13454: Investigate ReindexCollectionTest failures
(cherry picked from commit 379e6f64b8)
2019-05-23 08:44:28 -07:00
Chris Hostetter 0ff6a59396 Fix TestStressCloudBlindAtomicUpdates to play nice with tests.dups
(cherry picked from commit 3ed31cfbc7)
2019-05-22 11:23:04 -07:00
Chris Hostetter 4dd3186c0f SOLR-13474: Fix "Search is temporarily disabled" logic to be consistent for entire request
SOLR-13469: Fix rejected requests during full disk full + IndexFetch to use 503 not 403
(cherry picked from commit 6cc67e433e)

Conflicts:
	solr/CHANGES.txt
2019-05-20 10:29:00 -07:00
Andrzej Bialecki 4591f67390 SOLR-13475: Clean up empty alias lists. Fix the ordering or operations in the test. 2019-05-20 14:17:08 +02:00
Andrzej Bialecki 97d0d46db6 SOLR-13475: Null Pointer Exception when querying collection through collection alias. 2019-05-20 13:35:00 +02:00
Jan Høydahl 889cc4fc5c SOLR-13479: Harden OverseerTaskQueueTest
(cherry picked from commit 18cb42ee80)
2019-05-17 20:57:19 +02:00
Alessandro Benedetti 6f83dad301 SOLR-12304: MLT component now supports mlt.interestingTerms
(cherry picked from commit b9db118ed3)
2019-05-16 23:13:03 -04:00
Noble Paul 92451cf99c SOLR-13468: unused imports 2019-05-17 04:10:04 +10:00
David Smiley 6a3d32728b SOLR-13467: Include the S2 Geometry lib to make it simpler to use prefixTree="s2" on a Geo3D spatial field.
* Improved documentation on Geo3D.
* Better testing for Geo3D.

(cherry picked from commit 3a88ab616c)
2019-05-15 22:18:01 -04:00
Noble Paul 0f67df40b6 SOLR-13437: fork noggit code into Solr (#666)
* SOLR-13437: fork noggit code into Solr
2019-05-16 12:02:50 +10:00
Noble Paul 92ff728372 SOLR-13468: autoscaling/suggestions should be able to give suggestions from config sent as a payload (#678)
* SOLR-13468: autoscaling/suggestions should be able to give suggestions from config sent as a payload
2019-05-16 07:41:28 +10:00
Chris Hostetter 4585ce0868 SOLR-13440: fix precommit failures
(cherry picked from commit 7ef5d5fe5f)
2019-05-14 13:37:06 -07:00
Chris Hostetter 371d645e6a SOLR-12999: Harden TestReplicationHandlerDiskOverFlow against sporadic timing failures
- ensure IndexFetcher injection is reset in @After method
 - replace System.out with Logger
 - Log and fail on any exceptions in any callbacks/threads
 - use CyclicBarrier (instead of CountdownLatch) to ensure the Query Thread loop doesn't monopolize
   CPU preventing IndexFetcher callback from ever being run

(Some of these improvements directly address jenkins failures we've been seeing)

(cherry picked from commit bf8c6ea435)
2019-05-14 13:37:05 -07:00
Andrzej Bialecki f6774a2b6e SOLR-13440: Fix occasional test failures by providing assert helpers aware of simulation
artifacts.
2019-05-14 19:04:43 +02:00
Andrzej Bialecki 65d9b4979f SOLR-13407: Make sure we always have the latest version of aliases in cache before asserting. 2019-05-14 13:43:24 +02:00
Andrzej Bialecki 87ebe6edd6 SOLR-13440: Fix a precommit issue. 2019-05-13 19:55:08 +02:00
Andrzej Bialecki 33b4b6c14b SOLR-13440: Support saving/restoring autoscaling state for repeatable simulations. 2019-05-13 19:20:12 +02:00
Erick Erickson 41d216cce0 SOLR-13454: Investigate ReindexCollectionTest failures. moving bandaid calls
(cherry picked from commit cd8508408a)
2019-05-12 10:37:20 -07:00
Erick Erickson 4c10edc3e4 SOLR-13454: Investigate ReindexCollectionTest failures. I missed a place where I should have called the bandaid code
(cherry picked from commit 0aaf543208)
2019-05-10 13:56:33 -07:00
Cao Manh Dat 3e300e1a94 SOLR-13445: Hardness the test 2019-05-10 15:27:49 +01:00
Erick Erickson 8bac8a70a1 SOLR-13454: Investigate ReindexCollectionTest failures
(cherry picked from commit 577be08bf2)
2019-05-09 12:46:25 -07:00
Chris Hostetter 20e261c483 Merge remote-tracking branch 'refs/remotes/origin/branch_8x' into branch_8x 2019-05-09 11:40:54 -07:00
Chris Hostetter 786f5fc359 minimize randomization of IndexWriterConfig in TestDocTermOrdsUninvertLimit to reduce risk of OOM in nightly tests
(cherry picked from commit 5e40000e27)
2019-05-09 11:35:02 -07:00
Jan Høydahl 03cca62af3 SOLR-13453: Adjust auth metrics asserts in tests after SOLR-13449 (#668)
(cherry picked from commit 5b772f7c9d)
2019-05-09 20:16:11 +02:00
Chris Hostetter eed96570aa SOLR-12833: prevent NPE in DistributedUpdateProcessorTest AfterClass when mockito assumption fails in BeforeClass
(cherry picked from commit cde00b9a84)
2019-05-08 11:35:28 -07:00
Cao Manh Dat 2ec14f0323 SOLR-13445: Fix precommit 2019-05-08 19:17:17 +01:00
Cao Manh Dat 8a1b966165 SOLR-13445: Preferred replicas on nodes with same system properties as the query master 2019-05-08 17:59:33 +01:00
Erick Erickson 212b2b97dc SOLr-13428: Take the WARN message out of the logs when optimizing.
(cherry picked from commit f4399a495e)
2019-05-08 06:15:28 -07:00
Cao Manh Dat d9fbcc6b85 SOLR-13453: Marking mentioned tests as AwaitsFix 2019-05-08 11:18:34 +01:00
noble 8cde22ba01 SOLR-13320 : add an update param failOnVersionConflicts=false to updates not fail when there is a version conflict 2019-05-07 15:22:57 +10:00
Ishan Chattopadhyaya d92582fa21 SOLR-13410: Making the test more resilient 2019-05-07 00:58:54 +05:30
Andrzej Bialecki b2337262b1 SOLR-12833: This test needs the update log - use "enable.update.log" property to turn it on. 2019-05-06 12:07:08 +02:00
Ishan Chattopadhyaya 3d9d72ba69 SOLR-13410: Fix for test failure of the newly added test 2019-05-04 20:33:01 +05:30
Ishan Chattopadhyaya 400a918071 SOLR-13410: Designated overseer wasn't able to rejoin election queue upon restart 2019-05-04 00:29:49 +05:30
Jan Høydahl 7c5a02efd0 SOLR-12120: Fix bug in draining queue before shutdown
(cherry picked from commit cdd130ccb6)
2019-05-03 00:30:22 +02:00
noble 9952c2a24f SOLR-13441: Refactor code to minimize direct use of noggit 2019-05-03 07:22:13 +10:00
Ishan Chattopadhyaya 2faed4cada SOLR-13348: Speed up collapsing by avoiding scoring of ineligible documents 2019-05-03 02:46:03 +05:30
Ishan Chattopadhyaya f8caca54ac SOLR-12833: Fix PeerSyncTest and TestInPlaceUpdatesDistrib failures 2019-05-03 02:37:39 +05:30
David Smiley 9bd9f1b296 SOLR-12833: DistributedUpdateProcessorTest assumeWorkingMockito() 2019-05-02 18:52:03 +02:00
Andrzej Bialecki 2d3e50c56d SOLR-13427: Delete duplicate package description. 2019-05-02 18:40:42 +02:00
David Smiley 688c8d285f SOLR-12833: Test should use ExecutorUtil
(cherry picked from commit 1f91932ba4)
2019-05-01 14:34:21 -04:00
Ishan Chattopadhyaya 8ea32d61fe Adding 8.2 version 2019-05-01 14:51:55 +05:30
Andrzej Bialecki dbc9a3c767 SOLR-12833: Avoid unnecessary memory cost when DistributedUpdateProcessor timed-out lock is not used. 2019-04-30 22:32:09 +02:00
Andrzej Bialecki 81ab1cd442 SOLR-13427: Add missing package description. 2019-04-30 19:20:13 +02:00
Mikhail Khludnev 39ff3052c3 SOLR-12291: fixing premature completion of async tasks
* extract async tracking methods from OverseerCollectionMessageHandler into the separate class
* replacing hashmap to named list to avoid entry loss
2019-04-30 10:03:34 +03:00
Andrzej Bialecki 44efae15a9 SOLR-13427: Support simulating the execution of autoscaling suggestions. 2019-04-30 00:23:03 +02:00
Andrzej Bialecki 72230b69b3 SOLR-13407: Fix NPE and be consistent about returning empty instead of null properties. 2019-04-29 12:20:52 +02:00
Shalin Shekhar Mangar 793c974a59 SOLR-13432: Add .toString methods to BitDocSet and SortedIntDocSet so that enabling "showItems" on the filter caches shows some useful information about the values in the cache
(cherry picked from commit f77c56dbc6)
2019-04-29 12:34:58 +05:30
Ishan Chattopadhyaya 980472aa47 SOLR-5970: Fix precommit 2019-04-28 23:36:20 +05:30
Ishan Chattopadhyaya 855c7d6bb5 SOLR-5970: Return correct status upon collection creation failure 2019-04-28 23:20:21 +05:30
Ishan Chattopadhyaya a877632aaf SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields 2019-04-28 23:04:37 +05:30
Erick Erickson 16203c2668 SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload. More fixes, bad test. 2019-04-28 10:27:13 -04:00
Kevin Risden 0e7e470ada
SOLR-13414: SolrSchema - Avoid NPE if Luke returns field with no type defined
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-04-26 09:42:13 -04:00
Mikhail Khludnev efa9d9571f SOLR-13081: catching solrj exception as well in the negative test 2019-04-25 13:50:15 +03:00
Mikhail Khludnev 13be0b1af2 SOLR-13081: Let in-place update work with route.field 2019-04-25 10:01:13 +03:00
Erick Erickson f08ddbc713 SOLR-13268: Clean up any test failures resulting from defaulting to async logging
(cherry picked from commit 48dc020dda)
2019-04-24 10:54:31 -07:00
Andrzej Bialecki 60af5dfcfc SOLR-13423: Upgrade RRD4j to version 3.5. 2019-04-24 13:10:08 +02:00
Ishan Chattopadhyaya a2d499e32a SOLR-12167: Throw an exception, instead of just a warning, upon unknown atomic update 2019-04-24 16:22:34 +05:30
Jan Høydahl e02f25d975 SOLR-12120: Harden AuditLoggerIntegrationTest
(cherry picked from commit 2261b4e8e1)
2019-04-23 15:20:00 +02:00
Mikhail Khludnev 075c123cb1 SOLR-13337: faster /terms response in distributed mode. 2019-04-22 23:28:57 +03:00
erick 033c20f108 SOLR-13400: Replace Observable pattern in TransientSolrCoreCache 2019-04-19 17:14:37 -07:00
Chris Hostetter 37857b0fd7 Reduce the upper bounds on some randomized variables in FullSolrCloudDistribCmdsTest to reduce risk of OOM
(cherry picked from commit dbf90a6e7b)
2019-04-19 14:48:55 -07:00
Tomas Fernandez Lobbe 92f5d8298c SOLR-13281: Fix NPE in DocExpirationUpdateProcessor 2019-04-18 22:23:34 -07:00
Erick Erickson 249a03d4f5 SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload bandaid for DocValuesNotIndexedTest
(cherry picked from commit 2a3bd8e864)
2019-04-18 21:38:56 -07:00
Andrzej Bialecki 7fd49c3228 SOLR-12461: Upgrade Dropwizard Metrics to 4.0.5 release. 2019-04-18 19:09:06 +02:00
Andrzej Bialecki 667ccbc16f SOLR-13407: Reject update requests sent to non-routed multi collection aliases. 2019-04-18 14:55:00 +02:00
erick ca1cc248e0 SOLR-13408: Cannot start/stop DaemonStream repeatedly, other API improvements
(cherry picked from commit a9771a5849)
2019-04-17 16:08:39 -07:00
Simon Willnauer 651f41e21b LUCENE-8671: Introduce Reader attributes (#640)
Reader attributes allows a per IndexReader configuration of codec internals.
For instance this allows a per reader configuration if FSTs are loaded into memory or are left
on disk.
2019-04-15 22:24:31 +02:00
Jan Høydahl 7f51292156 SOLR-12371: Editing authorization config via REST API now works in standalone mode
(cherry picked from commit 9707bb6fa9)
2019-04-15 21:34:28 +02:00
Chris Hostetter 4c9311e8bf Merge remote-tracking branch 'refs/remotes/origin/branch_8x' into branch_8x
Conflicts:
	solr/CHANGES.txt
2019-04-15 11:40:42 -07:00
markrmiller fe37e18d10 SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node exists after an exists check and the Overseer work loop should not allow free spinning the loop when it hits a KeeperException. 2019-04-15 12:54:35 -05:00
Chris Hostetter 59a3c45d9c SOLR-13336: add maxBooleanClauses (default to 1024) setting to solr.xml, reverting previous effective value of Integer.MAX_VALUE-1, to restrict risk of pathalogical query expansion.
(cherry picked from commit d90034f0d6)
2019-04-15 10:27:29 -07:00
Noble Paul d7e3953ada SOLR-13395: make DebugComponent.getRequestId() public 2019-04-15 11:14:56 +10:00
Jan Høydahl d05e7b3df4 SOLR-13398: Move log "Processing SSL Credential Provider chain" from INFO to DEBUG to prevent leaking into bin/solr printout
(cherry picked from commit 03f5a5e7a1)
2019-04-12 11:55:22 +02:00
Jan Høydahl de432003a7 SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in background thread
(cherry picked from commit f2c59db273)
2019-04-12 10:23:08 +02:00
Jan Høydahl 43f5e6170c SOLR-12121: Refresh JWK from IdP on invalid sig. Some logging improvements. Minor test fixes
(cherry picked from commit 3b3879d880)
2019-04-12 09:51:21 +02:00
Cao Manh Dat 5691f87ecc SOLR-13339: Prevent recovery, fetching index being kicked off after SolrCores already closed 2019-04-11 09:22:35 +01:00
Andrzej Bialecki f83098752c SOLR-13262: Add collection RENAME command and support using aliases in most collection admin commands. 2019-04-10 18:57:16 +02:00
Uwe Schindler 4fe1f410f4 SOLR-13388: Fix FileExchangeRateProvider to be a public class, as it appears in schema.xml 2019-04-10 18:55:42 +02:00
Andrzej Bialecki a81d2c4a45 Release static solrClient after the test is done (occasionally caused failures on jenkins). 2019-04-10 12:26:33 +02:00
Moshe d01b13c8ed SOLR-12638: Partial/Atomic updates of nested docs.
and [child] now works in RTG.

(cherry picked from commit 8527ec11af)
2019-04-10 03:03:58 -04:00
Jan Høydahl 77a4604c39 SOLR-12120: Do not fail the main request if synchronous auditing fails, log ERROR
Document that sub classes should call super.close() or a new waitForQueueToDrain() before closing itself

(cherry picked from commit 3e628b562c)
2019-04-09 15:39:19 +02:00
Chris Hostetter 90ae5c1956 SOLR-13369: disable TriLevelCompositeIdRoutingTest until someone who actually understands how the /bits option is *suppose* to work can assess it to determine if the test is flawed or it there is a bug in the underlying CompositeIdRouter
(cherry picked from commit 2533fd1ede)
2019-04-08 18:17:39 -07:00
Chris Hostetter deb7987272 SOLR-13376: Disable test until it can be re-written to reflect actual expected behavior of how/when node markers will be cleaned up
(cherry picked from commit ab55b6386b)
2019-04-08 18:06:21 -07:00
Chris Hostetter ea2a91ef1f BlockCacheTest uses BlockCache which uses the common ForkJoinPool so we need to ensure the SolrIgnoredThreadsFilter is used
(cherry picked from commit 6bc360ee67)
2019-04-08 10:57:23 -07:00
Andrzej Bialecki f3cdd326bd SOLR-13368: Tentative fix for a race condition in managed schema initialization. 2019-04-08 11:30:36 +02:00
Chris Hostetter afe1df1c22 set the lastmod of the files to be deleted to be 1 day ago so that '-remove_old_solr_logs 0' doesn't run afoul of coarse granularity timestamps on windows jenkins machines if the test starts and runs quickly
(cherry picked from commit 0b7421eca6)
2019-04-05 16:46:01 -07:00
Chris Hostetter fe1a994fe2 Use FSDirectory for TestDocTermOrdsUninvertLimit to reduce the amount of RAM the test needs.
This lowers the risk of OOM, w/o needing to sacrifice testing of how the randomized merging/segments might impact the uninversion

(cherry picked from commit 37166ce4e9)
2019-04-05 15:07:23 -07:00
Erick Erickson 18dc7a5c96 SOLR-13075: Harden SaslZkACLProviderTest. Put AwaitsFix back
(cherry picked from commit 03d76c611b)
2019-04-05 08:49:43 -07:00
Jan Høydahl 4d95746b01 SOLR-12120: Harden AuditLoggerIntegrationTest, timing issues
(cherry picked from commit aadc94a6e3)
2019-04-05 11:35:26 +02:00
Jan Høydahl b0f4b5dc75 SOLR-12120: New AuditLoggerPlugin type allowing custom Audit logger plugins
(cherry picked from commit 361502da8a)
2019-04-05 11:34:03 +02:00
Simon Willnauer 28e8a30b53 LUCENE-8671: Expose FST off/on-heap options on Lucene50PostingsFormat (#613)
Before we can expose options to configure this postings format
on a per-reader basis we need to expose the option to load the terms
index FST off or on heap on the postings format. This already allows to
change the default in a per-field posting format if an expert user
wants to change the defaults. This essentially provides the ability to change
defaults globally while still involving some glue code.
2019-04-04 17:00:32 +02:00
Jan Høydahl 9dd51915fa SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin
(cherry picked from commit 96ef35cd2c)
2019-04-04 16:26:19 +02:00
Jason Gerlowski 4b86a39680 SOLR-13331: Fix AtomicUpdate 'remove' ops in SolrJ
The racent change introducing ByteArrayUtf8CharSequence altered the
NamedLists produced by atomic-update requests so that they include
instances of this class for requests coming in as javabin.  This is a
problem for 'remove' atomic-updates, which need to be able to compare
these ByteArrayUtf8CharSequence instances with existing field values
represented as Strings.  equals() would always return false, and
'remove' operations would have no effect.

This commit converts items as necessary to allow atomic-update
operations to work as expected.
2019-04-04 06:53:01 -04:00
Chris Hostetter de1f62e52f Fix test to use solr.disable.shardsWhitelist=true rather then trying to whitelist the set of servers
The previous version of this test had a chicken/egg problem (needed to init the servers w/the whitelist but didn't know the port nums until after init)

that caused it to require 'restarting' the servers -- leading to the possibility of 'Address already in use' errors from jenkins machines if the OS reclaimed the port between the stop/start of the jetty instance

(cherry picked from commit 3fe4d0a30a)
2019-04-03 15:15:36 -07:00
Kevin Risden 9f2e04b3f2
SOLR-13075: Harden SaslZkACLProviderTest (Kevin Risden, Hrishikesh Gadre, Peter Cseh)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-04-03 11:09:24 -04:00
Jason Gerlowski d5b9fbee37 SOLR-13355: Obey 'ALL' for handlers with other predefined perms
Prior to this commit, RuleBasedAuthorizationPlugin would check for the
predefined 'ALL' permission only when the endpoint being hit wasn't
associated with another predefined-permission.

This resulted in some very unintuitive behavior. For example, the
permission {name:all, role:admin} would correctly prevent a
role:foo user from accessing /admin/info/properties, but would allow
write access to /admin/authorization because of the SECURITY_EDIT
predefined perm associated with that endpoint.

This commit fixes this bug so that the 'all' permission is always
consulted whether or not the endpoint is associated with other predefined
permissions.
2019-04-03 07:34:49 -04:00
Jason Gerlowski 68918206f5 SOLR-13355: Small refactors to RuleBasedAuthorizationPlugin 2019-04-03 07:34:37 -04:00
erick a248bc209e SOLR-13075: Harden SaslZkACLProviderTest. Testing new ZK version
(cherry picked from commit 3810d5e)
2019-04-02 21:46:08 -07:00
Chris Hostetter c18da4c6bc SOLR-13352: Remove risk of deadlock/threadleak when shutting down an Overseer(TriggerThread)
(cherry picked from commit 1071d09336)
2019-04-01 10:35:12 -07:00
Erick Erickson d420382d04 SOLR-12028: BadApple and AwaitsFix annotations usage
(cherry picked from commit e04a28e140)
2019-04-01 09:28:25 -07:00
Kevin Risden 1ad668e0bf
SOLR-13359: Make UpdateHandler support other prefixes (besides hdfs:/)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-31 10:47:17 -04:00
Kevin Risden ae95487db6
SOLR-11473: Make HDFSDirectoryFactory support other prefixes (besides hdfs:/)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-30 14:27:32 -04:00
erick 03a3562f78 SOLR-13349:High CPU usage in Solr due to Java 8 bug
(cherry picked from commit b2941ff)
2019-03-28 15:51:14 -07:00
Kevin Risden e99fd063b0
SOLR-13353: Add SolrCli AuthTool test
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-28 18:36:33 -04:00
Joel Bernstein 843763db06 SOLR-13294: Change test file name 2019-03-27 15:50:29 -04:00
Joel Bernstein 168e409d6d SOLR-13294: TestSQLHandler failures on windows jenkins machines 2019-03-27 15:50:08 -04:00
Ishan Chattopadhyaya b08566cacc SOLR-7414: CSVResponseWriter & XLSXResponseWriter return empty field when fl alias is combined with * selector 2019-03-27 16:29:25 +05:30
Henning Andersen 251d64a557 LUCENE-8735: Avoid FileAlreadyExistsException on windows. (#619)
FilterDirectory.getPendingDeletions() did not delegate the call, which
resulted in a new IndexWriter on same directory not considering pending
delete files. This could in turn result in a FileAlreadyExistsException
when running windows.
2019-03-26 15:04:52 +01:00
Kevin Risden 862a755d67
SOLR-13342: Remove dom4j from Solr
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-25 08:31:26 -04:00
Gus Heck cbaa68e9c0 sysout checking SOLR-13322 2019-03-24 21:29:16 -04:00
Gus Heck 33edce88f1 SOLR-13323 - remove unused csv related classes/tests 2019-03-24 20:35:15 -04:00
Andrzej Bialecki 62c6e8d5d9 SOLR-13322: Properly log exceptions. 2019-03-21 22:22:34 +01:00
Kevin Risden 96c293ec1b
SOLR-9079: Remove commons-lang as a dependency
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-20 19:51:34 -04:00
David Smiley 8f25ded16f SOLR-13253: avoid using IndexSchema.getResourceLoader for non-schema things.
Furthermore it's reference to SolrConfig was removed.

(cherry picked from commit 85a702cdff)
2019-03-19 13:52:38 -04:00
Bar Rotstein de58717183 SOLR-12955: Refactored DistributedUpdateProcessor to put SolrCloud specifics into a subclass
Closes #528

(cherry picked from commit 5b7866b085)
2019-03-19 13:28:27 -04:00
Andrzej Bialecki b778417054 SOLR-11127: REINDEXCOLLECTION command for re-indexing of existing collections. 2019-03-19 13:47:44 +01:00
Kevin Risden 105979fb4c
SOLR-8033: Remove debug if branch in HdfsTransactionLog
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-19 06:20:47 -04:00
Kevin Risden 6064b03ac6
SOLR-13330: Improve HDFS tests
Related JIRAs:
* SOLR-11010
* SOLR-11381
* SOLR-12040
* SOLR-13297

Changes:
* Consolidate hdfs configuration into HdfsTestUtil
* Ensure socketTimeout long enough for HDFS tests
* Ensure HdfsTestUtil.getClientConfiguration used in tests
* Replace deprecated HDFS calls
* Use try-with-resources to ensure closing of HDFS resources

Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-19 05:27:46 -04:00
erick 0894e39ceb SOLR-13268: Patch that flushes when shutting down 2019-03-18 11:53:54 -07:00
Chris Hostetter 695bfa3c90 SOLR-12923: Mea culpa: Remove useless import of java.lang... that breaks precommit
(cherry picked from commit 5c143022e7)
2019-03-15 14:39:42 -07:00
Chris Hostetter 87ec0c3600 SOLR-12923: fix SimClusterStateProvider to use lock.lockInterruptibly() exclusively, and make SimCloudManager's Callable checks tollerant of Callables that may have failed related to interrupts w/o explicitly throwing InterruptedException
(cherry picked from commit 1a54c6b19d)
2019-03-15 14:24:07 -07:00
Chris Hostetter 0ac45c1668 SOLR-12923: tweak the randomization in testCreateLargeSimCollections to reduce the max possible totalCores
also decrease the number of iters while increase the cluster shape wait time to reduce the risk of spurious failures on machines under heavy contention w/o making the the test any slower on average

(cherry picked from commit c79aeee5f9)
2019-03-15 14:24:07 -07:00
Chris Hostetter 653ba8d245 SOLR-12923: Fix some issues w/concurrency and exception swallowing in SimClusterStateProvider/SimCloudManager
There are 3 tightly related bug fixes in these changes:

1) ConcurrentModificationExceptions were being thrown by some SimClusterStateProvider methods when
   creating collections/replicas due to the use of ArrayLists nodeReplicaMap. These ArrayLists were changed
   to use synchronizedList wrappers.
2) The Exceptions from #1 were being swallowed/hidden by code using SimCloudManager.submit() w/o checking
   the result of the resulting Future object. (As a result, tests waiting for a particular ClusterShape
   would timeout regardless of how long they waited.)   To protect against "silent" failures like this,
   this SimCloudManager.submit() has been updated to wrap all input Callables such that any uncaught errors
   will be logged and "counted."  SimSolrCloudTestCase will ensure a suite level failure if any such failures
   are counted.
3) The changes in #2 exposed additional concurrency problems with the Callables involved in leader election:
   These would frequently throw IllegalStateExceptions due to assumptions about the state/existence of
   replicas when the Callables were created vs when they were later run -- notably a Callable may have been
   created that held a reference to a Slice, but by the time that Callable was run the collection (or a
   node, etc...) refered to by that Slice may have been deleted.  While fixing this, the leader election
   logic was also cleaned up such that adding a replica only triggers leader election for that shard, not
   every shard in the collection.

While auditing this code, cleanup was also done to ensure all usage of SimClusterStateProvider.lock was
also cleaned up to remove all risky points where an exception may have been possible after aquiring the
lock but before the try/finally that ensured it would be unlocked.

(cherry picked from commit 76babf876a)
2019-03-15 14:24:07 -07:00
Andrzej Bialecki f7e41ef868 SOLR-13292: Fix a logic bug when point types are present, add more details to the unit test. 2019-03-15 11:54:13 +01:00
Alan Woodward ecf86ecab7 LUCENE-3041: QueryVisitor (#581)
This commit adds an introspection API to Query, allowing users to traverse
the nested structure of a query and examine its leaves.  It replaces the existing
`extractTerms` method on Weight, and alters some highlighting code to use
the new API
2019-03-15 08:29:32 +00:00
Jan Høydahl e68d16f32c SOLR-12121: Remove a System.out.println in JWTAuthPlugin
(cherry picked from commit a18aa2118b)
2019-03-14 10:52:32 +01:00
Gus Heck 1d85f13eeb SOLR-13131 Category Routed Aliases
(cherry picked from commit d8f2a02fdb)
2019-03-13 01:36:40 -04:00
Gus Heck 470813143d SOLR-12891 MacroExpander will no longer will expand URL parameters by
default inside of the 'expr' parameter, add InjectionDefense class
for safer handling of untrusted data in streaming expressions and add
-DStreamingExpressionMacros system property to revert to legacy behavior

(cherry picked from commit 9edc557f45)
2019-03-12 11:56:28 -04:00
Erick Erickson a87ba09e11 SOLR-13268: Clean up any test failures resulting from defaulting to async logging. Put TestXmlQParser back
(cherry picked from commit b893548d97)
2019-03-11 21:42:37 -07:00
Erick Erickson 6f6fbf0be2 SOLR-12732: TestLogWatcher failure on Jenkins. Deal with superclass logging bleeding into first watcher in the test 2019-03-11 13:00:26 -07:00
Chris Hostetter 44ff1c0ef9 SOLR-13300: DistributedFacetExistsSmallTest should not attempt to compare results of a query that is known to differ in behavior in distributed mode
(cherry picked from commit 7eb728a611)
2019-03-11 10:03:07 -07:00
Kevin Risden b4d9a98b32
SOLR-13307: Ensure HDFS tests clear System properties they set (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-11 11:22:22 -04:00
Mikhail Khludnev 5a5a39ee18 SOLR-13284: fallback to json instead of NPE and 500 code, when wt is omitted or wrong. 2019-03-10 23:12:22 +03:00
erick 277251c202 SOLR-13268: Clean up any test failures resulting from defaulting to async logging
(cherry picked from commit 9272c29)
2019-03-10 11:35:51 -07:00
Andrzej Bialecki f6f5f995ef SOLR-13292: Provide extended per-segment status of a collection. 2019-03-09 14:09:34 +01:00
Erick Erickson 83ab355772 SOLR-12732: TestLogWatcher failure on Jenkins. Added more logging
(cherry picked from commit 8c6e305365)
2019-03-08 21:43:08 -08:00
Chris Hostetter de5f0947e6 Delete OpenCloseCoreStressTest
This is a fairly old test that isn't particularlay valuable, all it really does is stress test indexing, but w/o actually validating the results.

In it's current state, it can suffer from exceeding file handle limits depending on merge settings, and isn't reliably reproducible due to it's
approach of "do (a variable) amount of work until a fixed amount of wall clock time has elapsed"

(cherry picked from commit 4335ad97af)
2019-03-08 14:53:10 -07:00
Chris Hostetter 27aeb11a37 SOLR-12923: harden TestSimLargeCluster
- added logging
- ensure start/finish trigger action counters are incremented before latches are released
- replace arbitrary sleep calls with a trigger listener countdown latch
- increase all await() times: This means that 'real' failures (which should be rare and hopefully
  reproducible) will be 'slow', but the trade off will be less hard to reproduce 'false failures'
  due to thread contention on slow or heavily loaded (ie: jenkins) machines

(cherry picked from commit 20de3d2ee0)
2019-03-07 15:41:50 -07:00
Erick Erickson fc76b70bd5 SOLR-13261: Make SortableTextField work with export/streaming, now requires useDocValuesAsStored='true'
(cherry picked from commit 1e09268e78)
2019-03-07 11:00:01 -08:00
Cao Manh Dat 747832907b SOLR-13302: Adding HttpPartitionTest and ForceLeaderTest for TLOG replicas 2019-03-07 12:18:58 +00:00
Cao Manh Dat b40efc883c SOLR-12313: Fix precommit 2019-03-07 12:01:09 +00:00