Commit Graph

7841 Commits

Author SHA1 Message Date
Cao Manh Dat c8382890ae SOLR-12988: Avoid using TLSv1.3 for HttpClient 2019-06-18 12:49:51 +01:00
Cao Manh Dat 6d5453d508 Fix precommit 2019-06-18 12:36:43 +01:00
Daniel Collins 342b558b01 SOLR-13434: Fix opentracing mock dependency for Solr core tests 2019-06-18 11:37:31 +01:00
Mikhail Khludnev 4f6314c59b SOLR-7530: /terms responds per field arrays in JSON by default 2019-06-18 08:38:02 +02:00
Chris Hostetter 3030ea9d94 eliminate CPU hogging spin-loops in OverseerTest
this test already uses waitForState (frequently via verifyReplicaStatus) so there is no reason to include CPU/network/ZK intensive infinite loop checks looking for udpated cluster state
2019-06-17 15:42:14 -07:00
Chris Hostetter 7eb8703df6 SOLR-13490: fix TestWaitForStateWithJettyShutdowns to use correct (randomized) JettyConfig 2019-06-17 15:34:14 -07:00
Chris Hostetter 5a974860fa SOLR-13490: Fix CollectionStateWatcher/CollectionStatePredicate based APIs in ZkStateReader and CloudSolrClient to be triggered on liveNode changes.
Also add Predicate<DocCollection> equivilents for callers that don't care about liveNodes.
2019-06-17 09:59:43 -07:00
Noble Paul d514beab94
SOLR-13534: Dynamic loading of jars from a remote url (#712)
* SOLR-13534 : Dynamic loading to support loading jars from a URL
2019-06-17 16:49:31 +10:00
Gus Heck 5d550a34a9 SOLR-13420 Routed Aliases now use collection properties instead of core properties 2019-06-14 13:51:16 -04:00
Mikhail Khludnev 2ee0b16bf1 SOLR-13333: make terms.ttf work without terms.list in standalone mode 2019-06-14 15:42:22 +02:00
Christine Poerschke 335aaf801e SOLR-13515: remove SolrPluginUtils.IdentityRegenerator in favour of NoOpRegenerator 2019-06-14 13:10:24 +01:00
Christine Poerschke 43fc05c1ed SOLR-13511: Add SearchHandler.newResponseBuilder method to facilitate custom plugins' maintenance of per-request state in a custom ResponseBuilder. (Ramsey Haddad, Christine Poerschke) 2019-06-14 13:10:24 +01:00
Benoit Vanalderweireldt 2c10b1de19 SOLR-8754: add tests for org.apache.solr.util.hll.NumberUtilTest (#15) 2019-06-13 12:57:13 +02:00
Thomas Wöckinger 828960927f SOLR-13347: Add support for reading/writing UUID from/to TransactionLog (#681)
SOLR-13347: Transaction log to natively support UUID types
2019-06-13 08:56:41 +10:00
Mikhail Khludnev 5ba6c0c7a2 SOLR-13509: add omitHeader=false for shards requests to avoid NPE on partialResuls check 2019-06-12 18:09:29 +02:00
Erick Erickson bfb5b41144 SOLR-12013: collections API CUSTERSTATUS command fails when configset missing 2019-06-11 15:56:02 -07:00
Gus Heck abeecfacd4 SOLR-13439 - Adds ability to locally cache collection properties for a specified duration. 2019-06-11 10:12:52 -04:00
Andrzej Bialecki e2832b37d7 Improve this test - instead of waiting a fixed amount of time wait until an update arrives.
Also, deal with partial updates.
2019-06-11 14:10:30 +02:00
Andrzej Bialecki 6527e74ed8 Improve the test by actually checking that the version of Aliases is
more recent, instead of simply waiting.
2019-06-11 12:13:54 +02:00
Jan Høydahl 1942f8cb69 Revert "SOLR-13526: @AwaitsFix failing tests"
This reverts commit 85abdbeb
2019-06-11 11:03:00 +02:00
Andrzej Bialecki 010466ec04 SOLR-13512: Raw index data analysis tool (extension of COLSTATUS collection command). 2019-06-10 19:21:46 +02:00
Koen De Groote 67104dd615 LUCENE-8847: Code Cleanup: Rewrite StringBuilder.append with concatted strings (#707)
This specific commit affects all points in the casebase where the argument of a StringBuilder.append() call is itself a regular String concatenation.
This defeats the purpose of using StringBuilder and also introduces an extra alloction.
These changes should avoid that.

ant tests have run, succeeded on local machine.

Removing test files from the changes.

Another suggested rework.
2019-06-10 18:07:43 +02:00
Tim Underwood 97ca9df7ef LUCENE-8834: Cache the SortedNumericDocValues.docValueCount() value whenever it is used in a loop (#698) 2019-06-10 08:56:21 +02:00
Christine Poerschke 85f4029c5c SOLR-13496: Fix distributed grouping related NullPointerException in JSONWriter.writeSolrDocument 2019-06-07 16:05:10 +01:00
Jan Høydahl 85abdbebf5 SOLR-13526: @AwaitsFix failing tests 2019-06-07 08:34:17 +02:00
Cao Manh Dat 3364753661 SOLR-13434: Using back Java 9 type reference 2019-06-06 12:11:37 +01:00
Christine Poerschke 757e4548c7 SOLR-13518: extra assertNotNull info for SolrInfoBeanTest 2019-06-05 20:39:01 +01:00
Cao Manh Dat da832d4f3a SOLR-13434: Fixes problem on Java 8 build 2019-06-05 15:35:05 +01:00
Chris Hostetter 92cf9d8758 Fix TestSort.testRandomFieldNameSorts typo that ccaused very sporadic but reproducible failures 2019-06-04 13:49:57 -07:00
Cao Manh Dat 301ea0e462
SOLR-13434: OpenTracing support for Solr (#685) 2019-06-04 20:04:11 +01:00
erick 7fb5b7ed35 SOLR-12249: Better error message when grouping on a tokenized (non SortableText) field in SolrCloud 2019-06-04 10:27:06 -07:00
Cao Manh Dat cee4ed783e SOLR-13510: Intermittent 401's for internode requests with basicauth enabled 2019-06-04 16:12:49 +01:00
Erick Erickson 7ebeab71f4 SOLR-8346: Upgrade Zookeeper to version 3.5.5 2019-06-03 17:50:35 -07:00
Noble Paul 0a41163d27
SOLR-13504: In autoscaling policies, use an explicit 'put : on-each' to specify the the rules is applied on each node (#694)
SOLR-13504: In autoscaling policies, use an explicit 'put : on-each' to specify the the rules is applied on each node
2019-06-03 18:13:18 +10:00
markrmiller b09d462ee4 SOLR-13489: Stop the leader from trying to rejoin the election on session expiration and harden our zk reconnect code path. 2019-06-02 22:56:56 -05:00
Chris Hostetter c2b92f3015 Fix all solr tests to pass w/ -Dtests.dups=2 -Dtests.jvms=1 2019-05-29 10:27:15 -07:00
Christine Poerschke 90e0d1eba1 SOLR-12941: remove unused import in IndexSizeTriggerTest 2019-05-28 16:11:51 +01:00
Andrzej Bialecki 32ad372433 SOLR-12941: Fix IndexSizeTrigger to correctly work with "aboveBytes" and "splitMethod=link" parameters. 2019-05-28 11:34:57 +02:00
Noble Paul 6ac31c6db3
SOLR-13493: /autoscaling/suggestions to be able to filter by type (#686)
* SOLR-13493: /autoscaling/suggestions to be able to filter by type
2019-05-28 07:20:12 +10:00
Erick Erickson 57b64f2684 SOLR-12562: Reverting Clean up RealTimeGetComponent.toSolrDoc (sha 97e7d8a3d7 and 15aa9dfb3d) 2019-05-27 08:34:48 -07:00
Erick Erickson 97e7d8a3d7 SOLR-12652: Clean up RealTimeGetComponent.toSolrDoc 2019-05-26 08:23:53 -07:00
Chris Hostetter af4e1d324a Fix TestCloudSearcherWarming to work around SOLR-13490
Also clean up some crufty System.out/System.err pollution
2019-05-25 08:40:04 -07:00
Erick Erickson 379e6f64b8 SOLR-13454: Investigate ReindexCollectionTest failures 2019-05-23 08:43:34 -07:00
Noble Paul 975a4f8f55 Merge remote-tracking branch 'origin/master' into jira/SOLR-13484 2019-05-23 16:59:14 +10:00
Noble Paul a1c9f8f7db SOLR-13484: refactored code 2019-05-23 12:50:09 +10:00
Chris Hostetter 3ed31cfbc7 Fix TestStressCloudBlindAtomicUpdates to play nice with tests.dups 2019-05-22 11:22:52 -07:00
noble 750a5fdd02 SOLR-13484:
autoscaling/diagnostics APIshould be able to give diagnostics output from config pasted as a payload
2019-05-22 16:22:20 +10:00
Namgyu Kim 5a694ea26f LUCENE-8805: Parameter changes for stringField() in StoredFieldVisitor
Signed-off-by: Namgyu Kim <kng0828@gmail.com>
Signed-off-by: Adrien Grand <jpountz@gmail.com>
2019-05-21 20:18:42 +02:00
Chris Hostetter 6cc67e433e SOLR-13474: Fix "Search is temporarily disabled" logic to be consistent for entire request
SOLR-13469: Fix rejected requests during full disk full + IndexFetch to use 503 not 403
2019-05-20 10:24:02 -07:00
Andrzej Bialecki 93e57e63cd SOLR-13475: Null Pointer Exception when querying collection through collection alias. 2019-05-20 15:19:35 +02:00
Jan Høydahl 18cb42ee80 SOLR-13479: Harden OverseerTaskQueueTest 2019-05-17 20:54:46 +02:00
Alessandro Benedetti b9db118ed3 SOLR-12304: MLT component now supports mlt.interestingTerms 2019-05-16 23:11:43 -04:00
Noble Paul 1581d798b4 SOLR-13468: unused imports 2019-05-17 04:12:42 +10:00
David Smiley 3a88ab616c SOLR-13467: Include the S2 Geometry lib to make it simpler to use prefixTree="s2" on a Geo3D spatial field.
* Improved documentation on Geo3D.
* Better testing for Geo3D.
2019-05-15 22:15:13 -04:00
Noble Paul bd64ed6d2a
SOLR-13437: fork noggit code into Solr (#666)
* SOLR-13437: fork noggit code into Solr
2019-05-16 11:10:27 +10:00
Noble Paul c464d8a719
SOLR-13468: autoscaling/suggestions should be able to give suggestions from config sent as a payload (#678)
* SOLR-13468: autoscaling/suggestions should be able to give suggestions from config sent as a payload
2019-05-16 07:31:08 +10:00
Chris Hostetter 7ef5d5fe5f SOLR-13440: fix precommit failures 2019-05-14 13:35:59 -07:00
Chris Hostetter bf8c6ea435 SOLR-12999: Harden TestReplicationHandlerDiskOverFlow against sporadic timing failures
- ensure IndexFetcher injection is reset in @After method
 - replace System.out with Logger
 - Log and fail on any exceptions in any callbacks/threads
 - use CyclicBarrier (instead of CountdownLatch) to ensure the Query Thread loop doesn't monopolize
   CPU preventing IndexFetcher callback from ever being run

(Some of these improvements directly address jenkins failures we've been seeing)
2019-05-14 13:35:19 -07:00
Andrzej Bialecki de29d0d948 SOLR-13440: Fix occasional test failures by providing assert helpers aware of simulation
artifacts.
2019-05-14 19:03:02 +02:00
Andrzej Bialecki 13d1c113c1 SOLR-13407: Make sure we always have the latest version of aliases in cache before asserting. 2019-05-14 13:42:00 +02:00
Andrzej Bialecki 2315c6d1b8 SOLR-13440: Fix a precommit issue. 2019-05-13 19:55:39 +02:00
Andrzej Bialecki f2c18bacf2 SOLR-13440: Support saving/restoring autoscaling state for repeatable simulations. 2019-05-13 18:44:26 +02:00
Erick Erickson cd8508408a SOLR-13454: Investigate ReindexCollectionTest failures. moving bandaid calls 2019-05-12 10:36:46 -07:00
Erick Erickson 0aaf543208 SOLR-13454: Investigate ReindexCollectionTest failures. I missed a place where I should have called the bandaid code 2019-05-10 13:56:04 -07:00
Cao Manh Dat 6a06bcd470 SOLR-13445: Hardness the test 2019-05-10 15:25:54 +01:00
Erick Erickson 577be08bf2 SOLR-13454: Investigate ReindexCollectionTest failures 2019-05-09 12:45:47 -07:00
Chris Hostetter 5e40000e27 minimize randomization of IndexWriterConfig in TestDocTermOrdsUninvertLimit to reduce risk of OOM in nightly tests 2019-05-09 11:34:43 -07:00
Jan Høydahl 5b772f7c9d
SOLR-13453: Adjust auth metrics asserts in tests after SOLR-13449 (#668) 2019-05-09 20:13:46 +02:00
Chris Hostetter cde00b9a84 SOLR-12833: prevent NPE in DistributedUpdateProcessorTest AfterClass when mockito assumption fails in BeforeClass 2019-05-08 11:35:14 -07:00
Cao Manh Dat 81cfbcd009 SOLR-13445: Fix precommit 2019-05-08 19:16:54 +01:00
Cao Manh Dat 6b5b74bc9c SOLR-13445: Preferred replicas on nodes with same system properties as the query master 2019-05-08 17:55:49 +01:00
Gus Heck 76b854cb4f SOLR-13306 Add a request parameter to execute a streaming expression locally 2019-05-08 12:13:07 -04:00
Erick Erickson f4399a495e SOLr-13428: Take the WARN message out of the logs when optimizing. 2019-05-08 06:13:40 -07:00
Cao Manh Dat 5a35ba41f2 SOLR-13453: Marking mentioned tests as AwaitsFix 2019-05-08 11:17:54 +01:00
noble 733b071564 SOLR-13320 : add an update param failOnVersionConflicts=false to updates not fail when there is a version conflict 2019-05-07 15:20:02 +10:00
Ishan Chattopadhyaya ca29340d8b SOLR-13410: Making the test more resilient 2019-05-07 00:59:23 +05:30
Andrzej Bialecki ef3b6a1aa3 SOLR-12833: This test needs the update log - use "enable.update.log" property to turn it on. 2019-05-06 12:08:13 +02:00
Ishan Chattopadhyaya 67e4b0c517 SOLR-13410: Fix for test failure of the newly added test 2019-05-04 20:32:20 +05:30
Ishan Chattopadhyaya 1882a17115 SOLR-13410: Designated overseer wasn't able to rejoin election queue upon restart 2019-05-04 00:29:07 +05:30
Jan Høydahl cdd130ccb6 SOLR-12120: Fix bug in draining queue before shutdown 2019-05-02 23:26:16 +02:00
noble 8908205b42 SOLR-13441: Refactor code to minimize direct use of noggit 2019-05-03 07:20:05 +10:00
Ishan Chattopadhyaya 29a98825d0 SOLR-13348: Speed up collapsing by avoiding scoring of ineligible documents 2019-05-03 02:45:15 +05:30
Ishan Chattopadhyaya ceeac409a5 SOLR-12833: Fix PeerSyncTest and TestInPlaceUpdatesDistrib failures 2019-05-03 02:36:52 +05:30
Andrzej Bialecki e54610ce33 SOLR-13427: Delete duplicate package description. 2019-05-02 18:39:21 +02:00
David Smiley 1f91932ba4 SOLR-12833: Test should use ExecutorUtil 2019-05-01 14:30:35 -04:00
David Smiley a32decde6a SOLR-12833: DistributedUpdateProcessorTest assumeWorkingMockito() 2019-05-01 00:30:04 -04:00
Andrzej Bialecki 70e090717d SOLR-12833: Avoid unnecessary memory cost when DistributedUpdateProcessor timed-out lock is not used. 2019-04-30 22:31:44 +02:00
Andrzej Bialecki 710e15cec5 SOLR-13427: Add missing package description. 2019-04-30 19:19:49 +02:00
Mikhail Khludnev 5ca0602d28 SOLR-12291: fixing premature completion of async tasks
* extract async tracking methods from OverseerCollectionMessageHandler into the separate class
* replacing hashmap to named list to avoid entry loss
2019-04-30 09:57:42 +03:00
Andrzej Bialecki 6eccf2bf53 SOLR-13427: Support simulating the execution of autoscaling suggestions. 2019-04-29 23:14:38 +02:00
Andrzej Bialecki ced0243a3e SOLR-13407: Fix NPE and be consistent about returning empty instead of null properties. 2019-04-29 12:19:29 +02:00
Shalin Shekhar Mangar f77c56dbc6 SOLR-13432: Add .toString methods to BitDocSet and SortedIntDocSet so that enabling "showItems" on the filter caches shows some useful information about the values in the cache 2019-04-29 12:34:03 +05:30
Ishan Chattopadhyaya 8dd22bc056 SOLR-5970: Fix precommit 2019-04-28 23:29:11 +05:30
Ishan Chattopadhyaya dd9899b1c1 SOLR-5970: Return correct status upon collection creation failure 2019-04-28 23:16:29 +05:30
Ishan Chattopadhyaya 4b49bd99ca SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields 2019-04-28 23:01:58 +05:30
Erick Erickson a6262af842 SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload. More fixes, bad test. 2019-04-28 10:26:01 -04:00
Kevin Risden 35aeb7f623
SOLR-13414: SolrSchema - Avoid NPE if Luke returns field with no type defined
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-04-26 09:27:47 -04:00
Mikhail Khludnev 6d94631538 SOLR-13081: Let in-place update work with route.field 2019-04-25 14:05:42 +03:00
Erick Erickson 48dc020dda SOLR-13268: Clean up any test failures resulting from defaulting to async logging 2019-04-24 09:23:11 -07:00
Ishan Chattopadhyaya 33c94562a6 SOLR-12167: Throw an exception, instead of just a warning, upon unknown atomic update 2019-04-24 16:20:10 +05:30
Andrzej Bialecki 170f5fb7a3 SOLR-13423: Upgrade RRD4j to version 3.5. 2019-04-24 12:33:48 +02:00
Jan Høydahl 2261b4e8e1 SOLR-12120: Harden AuditLoggerIntegrationTest 2019-04-23 15:18:16 +02:00
Mikhail Khludnev a95e68e13b SOLR-13337: faster /terms response in distributed mode. 2019-04-22 23:26:12 +03:00
Cao Manh Dat c533988874 SOLR-13413: Adding debug log for HttpConnection 2019-04-22 10:39:05 +01:00
Erick Erickson d87196c141 SOLR-13400: Replace Observable pattern in TransientSolrCoreCache 2019-04-19 17:00:41 -07:00
Chris Hostetter dbf90a6e7b Reduce the upper bounds on some randomized variables in FullSolrCloudDistribCmdsTest to reduce risk of OOM 2019-04-19 14:48:37 -07:00
Tomas Fernandez Lobbe 9f644b1c69 SOLR-13281: Fix NPE in DocExpirationUpdateProcessor 2019-04-18 22:22:26 -07:00
Erick Erickson 2a3bd8e864 SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload bandaid for DocValuesNotIndexedTest 2019-04-18 21:38:01 -07:00
Andrzej Bialecki bd8905150d SOLR-12461: Upgrade Dropwizard Metrics to 4.0.5 release. 2019-04-18 19:08:20 +02:00
Andrzej Bialecki f46ba5227e SOLR-13407: Reject update requests sent to non-routed multi collection aliases. 2019-04-18 14:53:06 +02:00
erick a9771a5849 SOLR-13408: Cannot start/stop DaemonStream repeatedly, other API improvements 2019-04-17 16:02:06 -07:00
Uwe Schindler faaee86efb LUCENE-8738: Move to Java 11 as minimum Java version (merged branch: jira/LUCENE-8738)
Co-authored-by: Adrien Grand <jpountz@apache.org>
2019-04-16 14:00:09 +02:00
Jan Høydahl 9707bb6fa9 SOLR-12371: Editing authorization config via REST API now works in standalone mode 2019-04-15 21:09:30 +02:00
Simon Willnauer a302be381e
LUCENE-8671: Introduce Reader attributes (#640)
Reader attributes allows a per IndexReader configuration of codec internals.
For instance this allows a per reader configuration if FSTs are loaded into memory or are left
on disk.
2019-04-15 20:39:36 +02:00
markrmiller ecbf7daf56 SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node exists after an exists check and the Overseer work loop should not allow free spinning the loop when it hits a KeeperException. 2019-04-15 12:53:36 -05:00
Chris Hostetter d90034f0d6 SOLR-13336: add maxBooleanClauses (default to 1024) setting to solr.xml, reverting previous effective value of Integer.MAX_VALUE-1, to restrict risk of pathalogical query expansion. 2019-04-15 10:27:08 -07:00
Noble Paul e783207535 SOLR-13395: make DebugComponent.getRequestId() public 2019-04-15 11:10:06 +10:00
Jan Høydahl 03f5a5e7a1 SOLR-13398: Move log "Processing SSL Credential Provider chain" from INFO to DEBUG to prevent leaking into bin/solr printout 2019-04-12 11:54:40 +02:00
Jan Høydahl f2c59db273
SOLR-12860: MetricsHistoryHandler now always uses PKI Auth (#642)
* SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in background thread
2019-04-12 10:21:13 +02:00
Jan Høydahl 3b3879d880 SOLR-12121: Refresh JWK from IdP on invalid sig. Some logging improvements. Minor test fixes 2019-04-12 09:49:11 +02:00
Cao Manh Dat a67a941e19 SOLR-13339: Prevent recovery, fetching index being kicked off after SolrCores already closed 2019-04-11 09:21:14 +01:00
Andrzej Bialecki 02c4503f8c SOLR-13262: Add collection RENAME command and support using aliases in most collection admin commands. 2019-04-10 18:56:39 +02:00
Uwe Schindler eafe42f090 SOLR-13388: Fix FileExchangeRateProvider to be a public class, as it appears in schema.xml 2019-04-10 18:53:51 +02:00
Andrzej Bialecki d40a6d4e95 Release static solrClient after the test is done (occasionally caused failures on jenkins). 2019-04-10 12:26:02 +02:00
Moshe 8527ec11af SOLR-12638: Partial/Atomic updates of nested docs.
and [child] now works in RTG.
2019-04-10 03:02:59 -04:00
Gus Heck 4a93199803 SOLR-13370 - Trying reduction of cluster size, but not clear that
should help from local tests/profile, but maybe it helps in more memory
constrained build servers, this and prior version both beasted success
10 rounds of 5 on a machine with lots of memory.
2019-04-09 10:02:16 -04:00
Jan Høydahl 3e628b562c SOLR-12120: Do not fail the main request if synchronous auditing fails, log ERROR
Document that sub classes should call super.close() or a new waitForQueueToDrain() before closing itself
2019-04-09 14:25:52 +02:00
Chris Hostetter 2533fd1ede SOLR-13369: disable TriLevelCompositeIdRoutingTest until someone who actually understands how the /bits option is *suppose* to work can assess it to determine if the test is flawed or it there is a bug in the underlying CompositeIdRouter 2019-04-08 18:17:29 -07:00
Chris Hostetter ab55b6386b SOLR-13376: Disable test until it can be re-written to reflect actual expected behavior of how/when node markers will be cleaned up 2019-04-08 18:06:09 -07:00
Chris Hostetter 6bc360ee67 BlockCacheTest uses BlockCache which uses the common ForkJoinPool so we need to ensure the SolrIgnoredThreadsFilter is used 2019-04-08 10:57:02 -07:00
Andrzej Bialecki 0859be134d SOLR-13368: Tentative fix for a race condition in managed schema initialization. 2019-04-08 10:47:29 +02:00
Chris Hostetter 0b7421eca6 set the lastmod of the files to be deleted to be 1 day ago so that '-remove_old_solr_logs 0' doesn't run afoul of coarse granularity timestamps on windows jenkins machines if the test starts and runs quickly 2019-04-05 16:45:41 -07:00
Chris Hostetter 37166ce4e9 Use FSDirectory for TestDocTermOrdsUninvertLimit to reduce the amount of RAM the test needs.
This lowers the risk of OOM, w/o needing to sacrifice testing of how the randomized merging/segments might impact the uninversion
2019-04-05 15:06:50 -07:00
Erick Erickson 03d76c611b SOLR-13075: Harden SaslZkACLProviderTest. Put AwaitsFix back 2019-04-05 08:47:36 -07:00
Jan Høydahl aadc94a6e3 SOLR-12120: Harden AuditLoggerIntegrationTest, timing issues 2019-04-05 10:35:34 +02:00
Simon Willnauer 1ec229b604
LUCENE-8671: Expose FST off/on-heap options on Lucene50PostingsFormat (#613)
Before we can expose options to configure this postings format
on a per-reader basis we need to expose the option to load the terms
index FST off or on heap on the postings format. This already allows to
change the default in a per-field posting format if an expert user
wants to change the defaults. This essentially provides the ability to change
defaults globally while still involving some glue code.
2019-04-04 16:59:37 +02:00
Jan Høydahl 96ef35cd2c SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin 2019-04-04 16:07:23 +02:00
Jan Høydahl 361502da8a SOLR-12120: New AuditLoggerPlugin type allowing custom Audit logger plugins 2019-04-04 15:48:51 +02:00
Jason Gerlowski 9b44fba0d4 SOLR-13331: Fix AtomicUpdate 'remove' ops in SolrJ
The racent change introducing ByteArrayUtf8CharSequence altered the
NamedLists produced by atomic-update requests so that they include
instances of this class for requests coming in as javabin.  This is a
problem for 'remove' atomic-updates, which need to be able to compare
these ByteArrayUtf8CharSequence instances with existing field values
represented as Strings.  equals() would always return false, and
'remove' operations would have no effect.

This commit converts items as necessary to allow atomic-update
operations to work as expected.
2019-04-04 06:51:28 -04:00
Chris Hostetter 3fe4d0a30a Fix test to use solr.disable.shardsWhitelist=true rather then trying to whitelist the set of servers
The previous version of this test had a chicken/egg problem (needed to init the servers w/the whitelist but didn't know the port nums until after init)

that caused it to require 'restarting' the servers -- leading to the possibility of 'Address already in use' errors from jenkins machines if the OS reclaimed the port between the stop/start of the jetty instance
2019-04-03 15:15:03 -07:00
Kevin Risden 6596ed1c16
SOLR-13075: Harden SaslZkACLProviderTest (Kevin Risden, Hrishikesh Gadre, Peter Cseh)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-04-03 10:14:49 -04:00
erick 3810d5e7ac SOLR-13075: Harden SaslZkACLProviderTest. Testing new ZK version 2019-04-02 21:45:30 -07:00
Gus Heck 2d690885e5 SOLR-13322 - let forbidden apis check for sysout in solr core 2019-04-02 22:45:33 -04:00
Chris Hostetter 1071d09336 SOLR-13352: Remove risk of deadlock/threadleak when shutting down an Overseer(TriggerThread) 2019-04-01 10:34:51 -07:00
Erick Erickson e04a28e140 SOLR-12028: BadApple and AwaitsFix annotations usage 2019-04-01 09:27:33 -07:00
Kevin Risden a99cec0402
SOLR-13359: Make UpdateHandler support other prefixes (besides hdfs:/)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-31 10:45:57 -04:00
Kevin Risden 42f8203fc3
SOLR-11473: Make HDFSDirectoryFactory support other prefixes (besides hdfs:/)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-30 14:26:00 -04:00
Jason Gerlowski 07b37ff26b SOLR-13355: Obey 'ALL' for handlers with other predefined perms
Prior to this commit, RuleBasedAuthorizationPlugin would check for the
predefined 'ALL' permission only when the endpoint being hit wasn't
associated with another predefined-permission.

This resulted in some very unintuitive behavior. For example, the
permission {name:all, role:admin} would correctly prevent a
role:foo user from accessing /admin/info/properties, but would allow
write access to /admin/authorization because of the SECURITY_EDIT
predefined perm associated with that endpoint.

This commit fixes this bug so that the 'all' permission is always
consulted whether or not the endpoint is associated with other predefined
permissions.
2019-03-29 22:33:12 -04:00
Jason Gerlowski 5fe03bcd01 SOLR-13355: Small refactors to RuleBasedAuthorizationPlugin 2019-03-29 22:33:00 -04:00
erick b2941ff0da SOLR-13349:High CPU usage in Solr due to Java 8 bug 2019-03-28 15:49:54 -07:00
Kevin Risden 8d658a8cfb
SOLR-13353: Add SolrCli AuthTool test
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-28 14:08:07 -04:00
Joel Bernstein 98cadcf9f6 SOLR-13294: Change test file name 2019-03-27 15:36:47 -04:00
Joel Bernstein d407d556cc SOLR-13294: TestSQLHandler failures on windows jenkins machines 2019-03-27 15:36:47 -04:00
Ishan Chattopadhyaya e7939d5907 SOLR-7414: CSVResponseWriter & XLSXResponseWriter return empty field when fl alias is combined with * selector 2019-03-27 14:38:36 +05:30
Henning Andersen 04afdb6442 LUCENE-8735: Avoid FileAlreadyExistsException on windows. (#619)
FilterDirectory.getPendingDeletions() did not delegate the call, which
resulted in a new IndexWriter on same directory not considering pending
delete files. This could in turn result in a FileAlreadyExistsException
when running windows.
2019-03-26 14:56:45 +01:00
Kevin Risden 0231c1ecb6
SOLR-13342: Remove dom4j from Solr
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-25 08:15:01 -04:00
Gus Heck dad414ca22 SOLR-13323 - remove unused csv related classes/tests 2019-03-24 20:01:46 -04:00
Gus Heck c60685f9e4 SOLR-13324 - Don't swallow/print exception in URLClassifyProcessor anymore 2019-03-24 19:07:26 -04:00
Andrzej Bialecki 863d34bff5 SOLR-13322: Properly log exceptions. 2019-03-21 22:22:03 +01:00
Kevin Risden 6222abf448
SOLR-9079: Remove commons-lang as a dependency
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-20 19:37:19 -04:00
David Smiley 85a702cdff SOLR-13253: avoid using IndexSchema.getResourceLoader for non-schema things.
Furthermore it's reference to SolrConfig was removed.
2019-03-19 13:51:44 -04:00
Bar Rotstein 5b7866b085 SOLR-12955: Refactored DistributedUpdateProcessor to put SolrCloud specifics into a subclass
Closes #528
2019-03-19 13:26:31 -04:00
Andrzej Bialecki 6f2b7bf5c0 SOLR-11127: REINDEXCOLLECTION command for re-indexing of existing collections. 2019-03-19 13:42:59 +01:00
Kevin Risden 9fea3060b9
SOLR-8033: Remove debug if branch in HdfsTransactionLog
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-19 06:20:09 -04:00
Kevin Risden cf828163bd
SOLR-13330: Improve HDFS tests
Related JIRAs:
* SOLR-11010
* SOLR-11381
* SOLR-12040
* SOLR-13297

Changes:
* Consolidate hdfs configuration into HdfsTestUtil
* Ensure socketTimeout long enough for HDFS tests
* Ensure HdfsTestUtil.getClientConfiguration used in tests
* Replace deprecated HDFS calls
* Use try-with-resources to ensure closing of HDFS resources

Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-18 15:25:36 -04:00
erick 3ac07b8dfb SOLR-13268: Patch that flushes when shutting down 2019-03-18 11:52:39 -07:00
Chris Hostetter 5c143022e7 SOLR-12923: Mea culpa: Remove useless import of java.lang... that breaks precommit 2019-03-15 14:39:19 -07:00
Chris Hostetter 1a54c6b19d SOLR-12923: fix SimClusterStateProvider to use lock.lockInterruptibly() exclusively, and make SimCloudManager's Callable checks tollerant of Callables that may have failed related to interrupts w/o explicitly throwing InterruptedException 2019-03-15 12:06:53 -07:00
Chris Hostetter c79aeee5f9 SOLR-12923: tweak the randomization in testCreateLargeSimCollections to reduce the max possible totalCores
also decrease the number of iters while increase the cluster shape wait time to reduce the risk of spurious failures on machines under heavy contention w/o making the the test any slower on average
2019-03-15 10:43:28 -07:00
Andrzej Bialecki 571b307266 SOLR-13292: Fix a logic bug when point types are present, add more details to the unit test. 2019-03-15 11:52:37 +01:00
Chris Hostetter 76babf876a SOLR-12923: Fix some issues w/concurrency and exception swallowing in SimClusterStateProvider/SimCloudManager
There are 3 tightly related bug fixes in these changes:

1) ConcurrentModificationExceptions were being thrown by some SimClusterStateProvider methods when
   creating collections/replicas due to the use of ArrayLists nodeReplicaMap. These ArrayLists were changed
   to use synchronizedList wrappers.
2) The Exceptions from #1 were being swallowed/hidden by code using SimCloudManager.submit() w/o checking
   the result of the resulting Future object. (As a result, tests waiting for a particular ClusterShape
   would timeout regardless of how long they waited.)   To protect against "silent" failures like this,
   this SimCloudManager.submit() has been updated to wrap all input Callables such that any uncaught errors
   will be logged and "counted."  SimSolrCloudTestCase will ensure a suite level failure if any such failures
   are counted.
3) The changes in #2 exposed additional concurrency problems with the Callables involved in leader election:
   These would frequently throw IllegalStateExceptions due to assumptions about the state/existence of
   replicas when the Callables were created vs when they were later run -- notably a Callable may have been
   created that held a reference to a Slice, but by the time that Callable was run the collection (or a
   node, etc...) refered to by that Slice may have been deleted.  While fixing this, the leader election
   logic was also cleaned up such that adding a replica only triggers leader election for that shard, not
   every shard in the collection.

While auditing this code, cleanup was also done to ensure all usage of SimClusterStateProvider.lock was
also cleaned up to remove all risky points where an exception may have been possible after aquiring the
lock but before the try/finally that ensured it would be unlocked.
2019-03-14 22:27:48 -07:00
Alan Woodward fbd05167f4
LUCENE-3041: QueryVisitor (#581)
This commit adds an introspection API to Query, allowing users to traverse
the nested structure of a query and examine its leaves.  It replaces the existing
`extractTerms` method on Weight, and alters some highlighting code to use
the new API
2019-03-14 15:04:33 +00:00
Jan Høydahl a18aa2118b SOLR-12121: Remove a System.out.println in JWTAuthPlugin 2019-03-14 10:48:09 +01:00
Gus Heck d8f2a02fdb SOLR-13131 Category Routed Aliases 2019-03-13 01:32:23 -04:00
Gus Heck 9edc557f45 SOLR-12891 MacroExpander will no longer will expand URL parameters by
default inside of the 'expr' parameter, add InjectionDefense class
for safer handling of untrusted data in streaming expressions and add
-DStreamingExpressionMacros system property to revert to legacy behavior
2019-03-12 10:46:30 -04:00
Erick Erickson b893548d97 SOLR-13268: Clean up any test failures resulting from defaulting to async logging. Put TestXmlQParser back 2019-03-11 21:40:23 -07:00
Erick Erickson 28c8171fe6 SOLR-12732: TestLogWatcher failure on Jenkins. Deal with superclass logging bleeding into first watcher in the test 2019-03-11 12:55:08 -07:00
Chris Hostetter 7eb728a611 SOLR-13300: DistributedFacetExistsSmallTest should not attempt to compare results of a query that is known to differ in behavior in distributed mode 2019-03-11 10:02:46 -07:00
Kevin Risden 6777af074f
SOLR-13307: Ensure HDFS tests clear System properties they set (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-11 11:13:37 -04:00
Mikhail Khludnev e353dcfe7d SOLR-13284: fallback to json instead of NPE and 500 code, when wt is omitted or wrong. 2019-03-10 23:10:38 +03:00
erick 9272c29539 SOLR-13268: Clean up any test failures resulting from defaulting to async logging 2019-03-10 11:05:00 -07:00
Andrzej Bialecki 27df824a4b SOLR-13292: Provide extended per-segment status of a collection. 2019-03-09 14:08:01 +01:00
Erick Erickson 8c6e305365 SOLR-12732: TestLogWatcher failure on Jenkins. Added more logging 2019-03-08 21:40:06 -08:00
Chris Hostetter 4335ad97af Delete OpenCloseCoreStressTest
This is a fairly old test that isn't particularlay valuable, all it really does is stress test indexing, but w/o actually validating the results.

In it's current state, it can suffer from exceeding file handle limits depending on merge settings, and isn't reliably reproducible due to it's
approach of "do (a variable) amount of work until a fixed amount of wall clock time has elapsed"
2019-03-08 14:52:33 -07:00
Chris Hostetter 20de3d2ee0 SOLR-12923: harden TestSimLargeCluster
- added logging
- ensure start/finish trigger action counters are incremented before latches are released
- replace arbitrary sleep calls with a trigger listener countdown latch
- increase all await() times: This means that 'real' failures (which should be rare and hopefully
  reproducible) will be 'slow', but the trade off will be less hard to reproduce 'false failures'
  due to thread contention on slow or heavily loaded (ie: jenkins) machines
2019-03-07 15:40:57 -07:00
Erick Erickson 1e09268e78 SOLR-13261: Make SortableTextField work with export/streaming, now requires useDocValuesAsStored='true' 2019-03-07 10:59:20 -08:00
Cao Manh Dat 002a4aac3e SOLR-13302: Adding HttpPartitionTest and ForceLeaderTest for TLOG replicas 2019-03-07 12:18:41 +00:00
Cao Manh Dat 583ae97067 SOLR-12313: Fix precommit 2019-03-07 12:00:33 +00:00
Cao Manh Dat 4eb1c6f3c6 SOLR-12313: Removing TestInjection#waitForInSyncWithLeader 2019-03-07 11:25:32 +00:00
Chris Hostetter e827dc2234 SOLR-13237: AwaitsFix LeaderTragicEventTest until we can make the underlying corruption detection reliable 2019-03-06 13:29:29 -07:00
erick 5f91670a49 Clean up any test failures resulting from defaulting to async logging, reenable shutdown hook 2019-03-05 14:10:36 -08:00
Erick Erickson 558c515bf3 SOLR-13295: Reproducible failure in TestDistributedGrouping 2019-03-05 13:56:40 -08:00
Mikhail Khludnev de030178b5 SOLR-9882: picking results for mergeIds even after partial ones occurred 2019-03-05 23:26:47 +03:00
Andrzej Bialecki 0bc26d07ad SOLR-13271: Read-only mode for SolrCloud collections. 2019-03-05 17:16:25 +01:00
Chris Hostetter 7bfe7b265a SOLR-13294: refactor test to include more loging to help diagnose some windows jenkins failures 2019-03-04 14:16:02 -07:00
Chris Hostetter 7f7357696f SOLR-12923: harden testEventQueue by replacing the arbitrary sleep call with a countdown latch 2019-03-04 09:54:23 -07:00
Chris Hostetter 235b15acfc SOLR-12923: increase all await() times in TriggerIntegrationTest
This means that 'real' failures (which should be rare and hopefully reproducile) will be 'slow', but the trade off will be less hard to reproduce 'false failures' due to thread contention on slow or heavily loaded (ie: jenkins) machines
2019-03-04 09:54:23 -07:00
Mikhail Khludnev b8d569aff0 SOLR-9882: reporting timeAllowed breach as partialResults instead of 500 error 2019-03-04 17:41:57 +03:00
Chris Hostetter fed80599a6 SOLR-12923: increase all await() times in TestSimTriggerIntegration
This means that 'real' failures (which should be rare and hopefully reproducile) will be 'slow', but the trade off will be less hard to reproduce 'false failures' due to thread contention on slow or heavily loaded (ie: jenkins) machines
2019-03-01 11:38:18 -07:00
Chris Hostetter 5e28e94a93 fix AfterClass in OverseerTest to not fail with NPE if assumeWorkingMockito() call short circuited the BeforeClass 2019-02-28 15:17:12 -07:00