Commit Graph

7845 Commits

Author SHA1 Message Date
Kevin Risden 42f8203fc3
SOLR-11473: Make HDFSDirectoryFactory support other prefixes (besides hdfs:/)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-30 14:26:00 -04:00
Jason Gerlowski 07b37ff26b SOLR-13355: Obey 'ALL' for handlers with other predefined perms
Prior to this commit, RuleBasedAuthorizationPlugin would check for the
predefined 'ALL' permission only when the endpoint being hit wasn't
associated with another predefined-permission.

This resulted in some very unintuitive behavior. For example, the
permission {name:all, role:admin} would correctly prevent a
role:foo user from accessing /admin/info/properties, but would allow
write access to /admin/authorization because of the SECURITY_EDIT
predefined perm associated with that endpoint.

This commit fixes this bug so that the 'all' permission is always
consulted whether or not the endpoint is associated with other predefined
permissions.
2019-03-29 22:33:12 -04:00
Jason Gerlowski 5fe03bcd01 SOLR-13355: Small refactors to RuleBasedAuthorizationPlugin 2019-03-29 22:33:00 -04:00
erick b2941ff0da SOLR-13349:High CPU usage in Solr due to Java 8 bug 2019-03-28 15:49:54 -07:00
Kevin Risden 8d658a8cfb
SOLR-13353: Add SolrCli AuthTool test
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-28 14:08:07 -04:00
Joel Bernstein 98cadcf9f6 SOLR-13294: Change test file name 2019-03-27 15:36:47 -04:00
Joel Bernstein d407d556cc SOLR-13294: TestSQLHandler failures on windows jenkins machines 2019-03-27 15:36:47 -04:00
Ishan Chattopadhyaya e7939d5907 SOLR-7414: CSVResponseWriter & XLSXResponseWriter return empty field when fl alias is combined with * selector 2019-03-27 14:38:36 +05:30
Henning Andersen 04afdb6442 LUCENE-8735: Avoid FileAlreadyExistsException on windows. (#619)
FilterDirectory.getPendingDeletions() did not delegate the call, which
resulted in a new IndexWriter on same directory not considering pending
delete files. This could in turn result in a FileAlreadyExistsException
when running windows.
2019-03-26 14:56:45 +01:00
Kevin Risden 0231c1ecb6
SOLR-13342: Remove dom4j from Solr
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-25 08:15:01 -04:00
Gus Heck dad414ca22 SOLR-13323 - remove unused csv related classes/tests 2019-03-24 20:01:46 -04:00
Gus Heck c60685f9e4 SOLR-13324 - Don't swallow/print exception in URLClassifyProcessor anymore 2019-03-24 19:07:26 -04:00
Andrzej Bialecki 863d34bff5 SOLR-13322: Properly log exceptions. 2019-03-21 22:22:03 +01:00
Kevin Risden 6222abf448
SOLR-9079: Remove commons-lang as a dependency
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-20 19:37:19 -04:00
David Smiley 85a702cdff SOLR-13253: avoid using IndexSchema.getResourceLoader for non-schema things.
Furthermore it's reference to SolrConfig was removed.
2019-03-19 13:51:44 -04:00
Bar Rotstein 5b7866b085 SOLR-12955: Refactored DistributedUpdateProcessor to put SolrCloud specifics into a subclass
Closes #528
2019-03-19 13:26:31 -04:00
Andrzej Bialecki 6f2b7bf5c0 SOLR-11127: REINDEXCOLLECTION command for re-indexing of existing collections. 2019-03-19 13:42:59 +01:00
Kevin Risden 9fea3060b9
SOLR-8033: Remove debug if branch in HdfsTransactionLog
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-19 06:20:09 -04:00
Kevin Risden cf828163bd
SOLR-13330: Improve HDFS tests
Related JIRAs:
* SOLR-11010
* SOLR-11381
* SOLR-12040
* SOLR-13297

Changes:
* Consolidate hdfs configuration into HdfsTestUtil
* Ensure socketTimeout long enough for HDFS tests
* Ensure HdfsTestUtil.getClientConfiguration used in tests
* Replace deprecated HDFS calls
* Use try-with-resources to ensure closing of HDFS resources

Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-18 15:25:36 -04:00
erick 3ac07b8dfb SOLR-13268: Patch that flushes when shutting down 2019-03-18 11:52:39 -07:00
Chris Hostetter 5c143022e7 SOLR-12923: Mea culpa: Remove useless import of java.lang... that breaks precommit 2019-03-15 14:39:19 -07:00
Chris Hostetter 1a54c6b19d SOLR-12923: fix SimClusterStateProvider to use lock.lockInterruptibly() exclusively, and make SimCloudManager's Callable checks tollerant of Callables that may have failed related to interrupts w/o explicitly throwing InterruptedException 2019-03-15 12:06:53 -07:00
Chris Hostetter c79aeee5f9 SOLR-12923: tweak the randomization in testCreateLargeSimCollections to reduce the max possible totalCores
also decrease the number of iters while increase the cluster shape wait time to reduce the risk of spurious failures on machines under heavy contention w/o making the the test any slower on average
2019-03-15 10:43:28 -07:00
Andrzej Bialecki 571b307266 SOLR-13292: Fix a logic bug when point types are present, add more details to the unit test. 2019-03-15 11:52:37 +01:00
Chris Hostetter 76babf876a SOLR-12923: Fix some issues w/concurrency and exception swallowing in SimClusterStateProvider/SimCloudManager
There are 3 tightly related bug fixes in these changes:

1) ConcurrentModificationExceptions were being thrown by some SimClusterStateProvider methods when
   creating collections/replicas due to the use of ArrayLists nodeReplicaMap. These ArrayLists were changed
   to use synchronizedList wrappers.
2) The Exceptions from #1 were being swallowed/hidden by code using SimCloudManager.submit() w/o checking
   the result of the resulting Future object. (As a result, tests waiting for a particular ClusterShape
   would timeout regardless of how long they waited.)   To protect against "silent" failures like this,
   this SimCloudManager.submit() has been updated to wrap all input Callables such that any uncaught errors
   will be logged and "counted."  SimSolrCloudTestCase will ensure a suite level failure if any such failures
   are counted.
3) The changes in #2 exposed additional concurrency problems with the Callables involved in leader election:
   These would frequently throw IllegalStateExceptions due to assumptions about the state/existence of
   replicas when the Callables were created vs when they were later run -- notably a Callable may have been
   created that held a reference to a Slice, but by the time that Callable was run the collection (or a
   node, etc...) refered to by that Slice may have been deleted.  While fixing this, the leader election
   logic was also cleaned up such that adding a replica only triggers leader election for that shard, not
   every shard in the collection.

While auditing this code, cleanup was also done to ensure all usage of SimClusterStateProvider.lock was
also cleaned up to remove all risky points where an exception may have been possible after aquiring the
lock but before the try/finally that ensured it would be unlocked.
2019-03-14 22:27:48 -07:00
Alan Woodward fbd05167f4
LUCENE-3041: QueryVisitor (#581)
This commit adds an introspection API to Query, allowing users to traverse
the nested structure of a query and examine its leaves.  It replaces the existing
`extractTerms` method on Weight, and alters some highlighting code to use
the new API
2019-03-14 15:04:33 +00:00
Jan Høydahl a18aa2118b SOLR-12121: Remove a System.out.println in JWTAuthPlugin 2019-03-14 10:48:09 +01:00
Gus Heck d8f2a02fdb SOLR-13131 Category Routed Aliases 2019-03-13 01:32:23 -04:00
Gus Heck 9edc557f45 SOLR-12891 MacroExpander will no longer will expand URL parameters by
default inside of the 'expr' parameter, add InjectionDefense class
for safer handling of untrusted data in streaming expressions and add
-DStreamingExpressionMacros system property to revert to legacy behavior
2019-03-12 10:46:30 -04:00
Erick Erickson b893548d97 SOLR-13268: Clean up any test failures resulting from defaulting to async logging. Put TestXmlQParser back 2019-03-11 21:40:23 -07:00
Erick Erickson 28c8171fe6 SOLR-12732: TestLogWatcher failure on Jenkins. Deal with superclass logging bleeding into first watcher in the test 2019-03-11 12:55:08 -07:00
Chris Hostetter 7eb728a611 SOLR-13300: DistributedFacetExistsSmallTest should not attempt to compare results of a query that is known to differ in behavior in distributed mode 2019-03-11 10:02:46 -07:00
Kevin Risden 6777af074f
SOLR-13307: Ensure HDFS tests clear System properties they set (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-11 11:13:37 -04:00
Mikhail Khludnev e353dcfe7d SOLR-13284: fallback to json instead of NPE and 500 code, when wt is omitted or wrong. 2019-03-10 23:10:38 +03:00
erick 9272c29539 SOLR-13268: Clean up any test failures resulting from defaulting to async logging 2019-03-10 11:05:00 -07:00
Andrzej Bialecki 27df824a4b SOLR-13292: Provide extended per-segment status of a collection. 2019-03-09 14:08:01 +01:00
Erick Erickson 8c6e305365 SOLR-12732: TestLogWatcher failure on Jenkins. Added more logging 2019-03-08 21:40:06 -08:00
Chris Hostetter 4335ad97af Delete OpenCloseCoreStressTest
This is a fairly old test that isn't particularlay valuable, all it really does is stress test indexing, but w/o actually validating the results.

In it's current state, it can suffer from exceeding file handle limits depending on merge settings, and isn't reliably reproducible due to it's
approach of "do (a variable) amount of work until a fixed amount of wall clock time has elapsed"
2019-03-08 14:52:33 -07:00
Chris Hostetter 20de3d2ee0 SOLR-12923: harden TestSimLargeCluster
- added logging
- ensure start/finish trigger action counters are incremented before latches are released
- replace arbitrary sleep calls with a trigger listener countdown latch
- increase all await() times: This means that 'real' failures (which should be rare and hopefully
  reproducible) will be 'slow', but the trade off will be less hard to reproduce 'false failures'
  due to thread contention on slow or heavily loaded (ie: jenkins) machines
2019-03-07 15:40:57 -07:00
Erick Erickson 1e09268e78 SOLR-13261: Make SortableTextField work with export/streaming, now requires useDocValuesAsStored='true' 2019-03-07 10:59:20 -08:00
Cao Manh Dat 002a4aac3e SOLR-13302: Adding HttpPartitionTest and ForceLeaderTest for TLOG replicas 2019-03-07 12:18:41 +00:00
Cao Manh Dat 583ae97067 SOLR-12313: Fix precommit 2019-03-07 12:00:33 +00:00
Cao Manh Dat 4eb1c6f3c6 SOLR-12313: Removing TestInjection#waitForInSyncWithLeader 2019-03-07 11:25:32 +00:00
Chris Hostetter e827dc2234 SOLR-13237: AwaitsFix LeaderTragicEventTest until we can make the underlying corruption detection reliable 2019-03-06 13:29:29 -07:00
erick 5f91670a49 Clean up any test failures resulting from defaulting to async logging, reenable shutdown hook 2019-03-05 14:10:36 -08:00
Erick Erickson 558c515bf3 SOLR-13295: Reproducible failure in TestDistributedGrouping 2019-03-05 13:56:40 -08:00
Mikhail Khludnev de030178b5 SOLR-9882: picking results for mergeIds even after partial ones occurred 2019-03-05 23:26:47 +03:00
Andrzej Bialecki 0bc26d07ad SOLR-13271: Read-only mode for SolrCloud collections. 2019-03-05 17:16:25 +01:00
Chris Hostetter 7bfe7b265a SOLR-13294: refactor test to include more loging to help diagnose some windows jenkins failures 2019-03-04 14:16:02 -07:00
Chris Hostetter 7f7357696f SOLR-12923: harden testEventQueue by replacing the arbitrary sleep call with a countdown latch 2019-03-04 09:54:23 -07:00
Chris Hostetter 235b15acfc SOLR-12923: increase all await() times in TriggerIntegrationTest
This means that 'real' failures (which should be rare and hopefully reproducile) will be 'slow', but the trade off will be less hard to reproduce 'false failures' due to thread contention on slow or heavily loaded (ie: jenkins) machines
2019-03-04 09:54:23 -07:00
Mikhail Khludnev b8d569aff0 SOLR-9882: reporting timeAllowed breach as partialResults instead of 500 error 2019-03-04 17:41:57 +03:00
Chris Hostetter fed80599a6 SOLR-12923: increase all await() times in TestSimTriggerIntegration
This means that 'real' failures (which should be rare and hopefully reproducile) will be 'slow', but the trade off will be less hard to reproduce 'false failures' due to thread contention on slow or heavily loaded (ie: jenkins) machines
2019-03-01 11:38:18 -07:00
Chris Hostetter 5e28e94a93 fix AfterClass in OverseerTest to not fail with NPE if assumeWorkingMockito() call short circuited the BeforeClass 2019-02-28 15:17:12 -07:00
Kevin Risden 15f3c3b0e6
SOLR-7321: Remove reflection in FSHDFSUtils.java (Mike Drob, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-27 13:57:40 -05:00
Mikhail Khludnev 58beaeb74d SOLR-13227: don't throw exceptions for counting facet.range.other. 2019-02-26 23:38:51 +03:00
Kevin Risden 8f78c0591c
SOLR-9762: Remove the workaround implemented for HADOOP-13346 (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-26 14:52:32 -05:00
Kevin Risden 9753e00294
SOLR-13074: MoveReplicaHDFSTest leaks threads, falls into an endless loop, logging like crazy (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-26 14:48:47 -05:00
Kevin Risden 6a886b274d
SOLR-13060: Improve HdfsAutoAddReplicasIntegrationTest and HdfsCollectionsAPIDistributedZkTest
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-25 11:14:37 -05:00
Kevin Risden 9e7ec87f23
SOLR-9763: Remove the workaround implemented for HADOOP-12767 (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-24 14:23:27 -05:00
erick fe5a96a284 SOLR-13268: Clean up any test failures resulting from SOLR-12055 (async logging). Kevin's upgrades 2019-02-24 09:50:04 -08:00
Mikhail Khludnev 52097627f8 SOLR-11883: reporting syntax errors as Bad Request instead of 500. 2019-02-23 23:55:55 +03:00
Erick 0de3905ce7 SOLR-12055: Enable async logging by default SOLR-12753: Async logging ring buffer and OOM error 2019-02-22 12:27:02 -08:00
Erick Erickson 6b4e90617d SOLR-13261: Make SortableTextField work with export/streaming 2019-02-22 11:04:31 -08:00
Tomas Fernandez Lobbe 906a08838b Fix TestTlogReplica
At some point got broken and nobody noticed since it's not running
2019-02-21 21:33:49 -08:00
Jason Gerlowski 5ab5ba773a Minor cleanup to ExportWriter 2019-02-21 16:05:48 -05:00
Tomas Fernandez Lobbe 6e81def01c Fix TestPullReplica.testKillLeader
Somewhere along the way, this was broken and since the test is being excluded, nobody noticed
2019-02-20 21:15:29 -08:00
Tomas Fernandez Lobbe bcd90f1561
SOLR-12708: CREATE collection request doesn't fail or cleanup when the request fails (#568)
When performing an async CREATE requests, responses to internal requests are also included inside the "success" or "failed" elements of the general response. This will make the operation cleanup in case of failure, the same way as we do when the request is synchronous.
2019-02-20 13:24:46 -08:00
Kevin Risden 86fe24cbef
SOLR-13222: Improve logging in StreamingSolrClients (Peter Cseh via Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-20 10:29:22 -05:00
Noble Paul a89b065c1f SOLR-13255 : ClasscastException when URPs try to read a String field which returns a ByteArrayUTF8CHarSequence . This is a regression in release 7.7 2019-02-20 22:19:18 +11:00
Kevin Risden af3ff118ef
SOLR-11763: Upgrade Guava to 25.1-jre (Markus Jelsma, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-19 17:27:58 -05:00
shalinmangar 7e2d40197c SOLR-13248: Autoscaling based replica placement is broken out of the box.
Solr 7.5 enabled autoscaling based replica placement by default but in the absence of default cluster policies, autoscaling can place more than 1 replica of the  same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not respected. Due to these reasons,  this issue reverts the default replica placement policy to the 'legacy' assignment policy that was the default until Solr 7.4.
2019-02-18 13:46:05 -08:00
Ishan Chattopadhyaya 6a0f7b251d SOLR-11876: In-place updates fail during resolution if required fields are present 2019-02-18 14:33:28 +05:30
Erick Erickson 3235c61e77 SOLR-12028: BadApple and AwaitsFix annotations usage. 2019-02-17 10:40:58 -08:00
Tomas Fernandez Lobbe 8ac34c2d6d SOLR-13229: Cleanup replicasMetTragicEvent after all exceptions 2019-02-15 14:51:35 -08:00
Simon Willnauer 4a513fa99f
LUCENE-8292: Make TermsEnum fully abstract (#574) 2019-02-15 17:32:55 +01:00
Alan Woodward 10acaf02f8 SOLR-13126: Correctly combine multiplicative query boosts 2019-02-15 10:09:26 +00:00
Andrzej Bialecki b4b9c39392 SOLR-13252: Fix an NPE when setting a "policy" property for an existing collection. 2019-02-14 21:38:22 +01:00
Andrzej Bialecki abcda8282c SOLR-13155: Minor addition to the "stats" section to make it easier to
estimate the distribution of cores across nodes.
2019-02-14 11:35:53 +01:00
Noble Paul cfefdade5f SOLR-13171 : A true streaming parser for javabin payload/stream without creating any objects 2019-02-13 18:23:43 +11:00
Bruno P. Kinoshita 5310152450 Fix some spell check issues 2019-02-12 21:35:30 -08:00
Andrzej Bialecki 242ff88e02 SOLR-13155: Add command-line option for testing autoscaling configurations. 2019-02-12 11:53:15 +01:00
Mikhail Khludnev bf69a40d16 SOLR-12330: rethow NPEs and other json.facet syntax errors properly as 400 2019-02-09 23:57:30 +03:00
yyuan2 a3a4ecd80b LUCENE-8662: Change TermsEnum.seekExact(BytesRef) to abstract 2019-02-08 15:10:38 -08:00
Tomas Fernandez Lobbe f2b84577a2 Set types in ZkController callables 2019-02-07 22:26:39 -08:00
Chris Hostetter 956772b7ef disable TestInjection in RestartWhileUpdatingTest
work around for SOLR-13189 and SOLR-13212
2019-02-07 18:27:32 -07:00
Kevin Risden 475f2fd36b
SOLR-9515 - Remove pom dependency kerby-kerb
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-07 11:38:18 -05:00
Chris Hostetter 87ad59f826 SOLR-13210: Fix TriLevelCompositeIdRoutingTest to actually make sense 2019-02-06 14:42:30 -07:00
Chris Hostetter ea2956fda3 Harden OrderedExecutorTest to use concurrent latches/barriers for testing parallelism instead of making assumpions about how milliseconds something should take in another thread 2019-02-06 14:32:12 -07:00
Tomas Fernandez Lobbe 0add3bb0ac Removed some unused variables from DistributedUpdateProcessor 2019-02-06 11:31:55 -08:00
Noble Paul b061947e91 SOLR-12999: Index replication could delete segments before downloading segments from master if there is not enough disk space 2019-02-06 23:15:07 +11:00
Kevin Risden 1a23ab005b
SOLR-10199: Solr's Kerberos functionality does not work in Java9 due to dependency on hadoop's AuthenticationFilter which attempt access to JVM protected classes
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-02 12:03:20 -05:00
Kevin Risden e8a24bc1c9
SOLR-8182: Update TestSolrCloudWithKerberosAlt to work with JDK9+
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-02 12:03:00 -05:00
Kevin Risden 429481bc82
Fix precommit - unused import
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-01 15:32:07 -05:00
markrmiller 4a3ddc94d8
SOLR-9515: Update to Hadoop 3 (Mark Miller, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-01 15:04:56 -05:00
Kevin Risden e4f202c1e3
Revert "SOLR-9515: Update to Hadoop 3"
This reverts commit 6bb24673f4.
2019-01-31 19:46:02 -05:00
markrmiller 6bb24673f4
SOLR-9515: Update to Hadoop 3
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-01-31 13:00:20 -05:00
Chris Hostetter 806744b0f1 reduce number of nodes/shards used in TriLevelCompositeIdRoutingTest to prevent (incredibly frequent) OOM in nightly 2019-01-31 10:45:12 -07:00
Dawid Weiss 4fc218e74e SOLR-13193: LeaderTragicEventTest doesn't catch NoSuchFileException when corrupting files 2019-01-31 11:55:05 +01:00
Chris Hostetter 0a01b9e127 disable TestInjection in TestStressCloudBlindAtomicUpdates
work around for SOLR-13189
2019-01-29 21:23:04 -07:00
Andrzej Bialecki 1cfbd3e1c8 SOLR-13072: Make sure the new overseer leader is present. 2019-01-29 11:22:28 +01:00
Chris Hostetter 8bee03f490 Add logging and impoved cleanup to TestStressCloudBlindAtomicUpdates setup/teardown codepaths that occasionally cause suite level failures in jenkins
The use of closeQuietly should hopefully prevent failures closing one HttpClient from resulting in other client objects being leaked

The setup changes are unlikey to improve test reliability, but will hopefully help diagnose where/how NPEs are coming from that currently cause some suite failures.
2019-01-28 16:22:28 -07:00
David Smiley 381a30b26c SOLR-12768: added _nest_path_ to the default schema (thereby enabling nested docs)
* new NestPathField encapsulating details for how _nest_path_ is indexed
** tweaked the analysis to index 1 token instead of variable
* TokenizerChain has new CustomAnalyzer copy-constructor
2019-01-28 13:25:06 -05:00
Dawid Weiss d7dc53ff7c LUCENE-8474: Remove deprecated RAMDirectory. 2019-01-28 13:49:03 +01:00
Mikhail Khludnev 43f2723213 SOLR-13156: support facet.sort for facet.field={!terms=foo,bar}field 2019-01-28 15:27:09 +03:00
Noble Paul 250d21da94 SOLR-13130: during the ResponseBuilder.STAGE_GET_FIELDS directly copy string bytes and avoid creating String Objects 2019-01-27 16:02:11 +11:00
Chris Hostetter ec68359065 SOLR-13168: Fixed a bug in TestInjection that caused test only code to be invoked when TLOG replicas recieved commits if java assertions were enabled
(see also: SOLR-12313)
2019-01-25 15:50:46 -07:00
Tomas Fernandez Lobbe ef81dadc7d SOLR-12373: Remove deprecated constructor 2019-01-25 13:48:40 -08:00
Tomas Fernandez Lobbe 0bd1911db6 SOLR-12373: Let DocBasedVersionConstraintsProcessor define fields to use in tombstones
A new config option, "tombstoneConfig" allows the DocBasedVersionConstraintsProcessor
to add extra fields to the tombstone generated when a document is deleted. This can
be useful when the schema has required fields.
2019-01-25 13:38:18 -08:00
Chris Hostetter e2b8b0e5b1 SOLR-12801: completely prevent tlog replicas from being used.
This follows the spirit of the change Mark intended in his previous commit to this test, but his solution wasn't covering all cases on backcompat to branch_7x

(see also: SOLR-12313)
2019-01-25 10:18:55 -07:00
Mikhail Khludnev c317119654 SOLR-13029: configure buffer size in HdfsBackupRepository. 2019-01-24 18:06:09 +03:00
Jan Høydahl d09c724a0e SOLR-12121: Fix test fails in JWTAuthPluginIntegrationTest.testMetrics 2019-01-24 12:43:27 +01:00
Andrzej Bialecki 84819c8379 SOLR-13072: Fix precommit. 2019-01-24 10:47:44 +01:00
Andrzej Bialecki 72a99e9c5c SOLR-13072: Enable this test again. 2019-01-23 15:46:25 +01:00
Jan Høydahl ea2c8ba38e SOLR-12121: JWT Token authentication plugin with OpenID Connect implicit flow login through Admin UI 2019-01-23 13:03:01 +01:00
Chris Hostetter 15e5ca999f SOLR-13140: harden SearchRateTriggerIntegrationTest by using more absolute rate thresholds and latches to track when all events have been processed so we don't need to 'guess' about sleep calls
This commit also disables testDeleteNode pending an AwaitsFix on SOLR-13163
2019-01-22 11:47:11 -07:00
Erick Erickson 60aef389cf SOLR-11998:RebalanceLeaders API broken response format with wt=JSON 2019-01-20 23:13:19 -06:00
Jimmy Casey 8b1c7296b7 Fixed Spelling. 2019-01-20 11:50:16 -08:00
Erick Erickson a692d05a90 SOLR-13091: REBALANCELEADERS is broken 2019-01-19 19:20:39 -08:00
Toke Eskildsen c13645bd4c LUCENE-8585: Create jump-tables for DocValues at index-time 2019-01-18 22:42:04 +01:00
fabric8-cd 63268000fb SOLR-13053: Adding test for assert that times nodes were removed and added was actually restored 2019-01-18 11:00:15 +00:00
Tomas Fernandez Lobbe 8b54b20fc4 SOLR-12770: Make it possible to configure a host whitelist for distributed search 2019-01-15 11:44:57 -08:00
Jan Høydahl b7f99fe55a SOLR-13137: NPE when /admin/zookeeper/status endpoint hit in standalone mode 2019-01-15 16:17:10 +01:00
Chris Hostetter d970375cd2 SOLR-12923: add a latch to TestTriggerListener to harden test that use it so they can deterministically know when all events have been proceeded
This hardens several flakey tests, and allows the removal of a several arbitrary sleep calls
2019-01-14 18:16:40 -07:00
Gus Heck dcc9ffe186 SOLR-13051 improve TRA update processor test
- remove some timeouts
  - better async mechanism linked to SolrCore lifecycle
  - add some additional tests to be a bit more thorough
2019-01-11 14:47:57 -05:00
Noble Paul 0635c657c1 SOLR-13016: precommit errors 2019-01-10 11:58:53 +11:00
Noble Paul 3ffc9099bf SOLR-13016: test errors fixed 2019-01-10 11:57:58 +11:00
David Smiley df119573db SOLR-12888: Run URP now auto-registers NestedUpdateProcessor before it. 2019-01-08 23:23:02 -05:00
Noble Paul d814d862b0 SOLR-12983: Create DocValues fields directly from byte[] 2019-01-09 13:44:41 +11:00
Andrzej Bialecki 7db4121b45 SOLR-13072: Use the same wait in other simulated tests where the same race condition may occur. 2019-01-08 19:12:38 +01:00
Andrzej Bialecki a37e2c609c SOLR-13072: Wait for autoscaling config refresh to finish before modifying the cluster
and enable the tests for now.
2019-01-08 16:16:55 +01:00
Noble Paul 825e63c00b SOLR-12983: tests don't need to use the optimization 2019-01-08 17:26:19 +11:00
Noble Paul 3932a4cc6c SOLR-12983: JavabinLoader should avoid creating String Objects and create UTF8CharSequence fields from byte[] 2019-01-08 15:27:48 +11:00
Chris Hostetter 5a513fab83 SOLR-13118: Fix various nodeAdded/nodeLost trigger (integration) tests related to restoriung state
This includes some cleanup and refactoring of unrelated test methods in the same classes to use new helper methods
2019-01-07 17:53:04 -07:00
Andrzej Bialecki 6e745bd250 SOLR-12730: Implement staggered SPLITSHARD requests in IndexSizeTrigger. 2019-01-07 19:14:54 +01:00
Alan Woodward 906f94a8cc Remove unnecessary back compatibility 2019-01-07 13:20:01 +00:00
Alan Woodward 04f48dba8b Move to version 9.0.0 2019-01-07 13:04:01 +00:00
Jan Høydahl 0b6ea3f108 SOLR-7896: Avoid browser basicAuth dialogue when blockUnknown=false. Always show Dashboard menu. Clarify refGuide 2019-01-07 13:09:26 +01:00
Shalin Shekhar Mangar 2bd6f246b0 SOLR-11126: New Node-level health check handler at /admin/info/healthcheck and /node/health paths that checks if the node is live, connected to zookeeper and not shutdown 2019-01-06 12:41:49 +05:30
David Smiley 84264c7410 SOLR-12365: moved parseLuceneVersionString to SolrConfig class 2019-01-05 17:14:37 -05:00
David Smiley ff19a3a261 SOLR-12365: Renamed class Config to XmlConfigFile 2019-01-05 10:08:36 -05:00
David Smiley ca23040baa SOLR-12365: Renamed class Config to XmlConfigFile 2019-01-05 09:51:02 -05:00
Noble Paul 000d0c1364 SOLR-12514: FIxed the test and another bug 2019-01-05 19:31:28 +11:00
noble 669ca8f283 SOLR-12514: Adding a positive testcase also 2019-01-04 14:56:17 +11:00
noble 663fa0e17f SOLR-12514: Adding a positive testcase also 2019-01-04 14:50:52 +11:00
Mike McCandless 63dfba4c7d LUCENE-8601: attributes added to IndexableFieldType during indexing will now be preserved in the index and accessible at search time via FieldInfo attributes 2019-01-03 18:44:41 -05:00
David Smiley 6342ec699e SOLR-12633: remove anonChildDocs update parameter used in nested docs in JSON. 2019-01-03 10:58:59 -05:00
Andrzej Bialecki 0a7a478c1f SOLR-12514: Properly close a SolrClient to fix the test teardown. 2019-01-03 14:59:31 +01:00
Andrzej Bialecki e5fda5d6f1 SOLR-13050: Fix the test so that .system events are collected again. 2019-01-02 22:11:57 +01:00
Jason Gerlowski c34f29a4b4 SOLR-13090: Add sysprop override for maxBooleanClauses 2019-01-02 10:58:32 -05:00
Andrzej Bialecki b9457b78d5 SOLR-13050: Fix another test that could accidentally kill the .system leader node.
Improve fallback in SystemLogListener when target collection is not present.
2019-01-02 16:55:16 +01:00
Noble Paul 302184dd7f SOLR-12514: Rule-base Authorization plugin skips authorization if querying node does not have collection replica 2019-01-02 19:49:40 +11:00
Noble Paul 619b38a19b SOLR-12514: Rule-base Authorization plugin skips authorization if querying node does not have collection replica 2019-01-02 19:44:03 +11:00
Shalin Shekhar Mangar 5016959ce8 SOLR-13082: A trigger that creates trigger events more frequently than the cool down period can starve other triggers.
This is mitigated to some extent by randomly choosing the trigger to resume after cool down. It is recommended that scheduled triggers not be used for very frequent operations to avoid this problem.
2019-01-02 11:59:00 +05:30
Christine Poerschke 6a2de771bf SOLR-13096: rename TestRankQueryPlugin to RankQueryTestPlugin 2018-12-31 11:05:31 +00:00
Erick Erickson 24ae0d16cd SOLR-12028: Catching up with annotations after recent Solr test work 2018-12-27 18:03:50 -08:00
David Smiley 106d300052 SOLR-12535: index time boosts in JSON are no longer accepted 2018-12-26 09:39:42 -05:00
David Smiley 1d0a086217 SOLR-13080: TermsQParserPlugin automaton method should (must?) sort input 2018-12-21 13:26:03 -05:00
Jason Gerlowski 207b3f4453 SOLR-13045: Sim node versioning should start at 0
Prior to this commit, new ZK nodes being simulated by the sim framework
were started with a version of -1.  This causes problems, since -1 is
also coincidentally the flag value used to ignore optimistic concurrency
locking and force overwrite values.
2018-12-21 08:04:44 -05:00
Jason Gerlowski 272178eff5 SOLR-13045: Allow SimDistribStateManager to create top-level data nodes
While working on a related issue in SimDistribStateManager, I noticed
that `createData()` only worked successfully on nodes nested more than
one level under root.  (i.e. `createData("/foo", someData, mode)` would
fail, while the same with "/foo/bar" wouldn't).  This was due to an edge
case in SimDistribStateManager's path building logic.  This commit fixes
this issue.
2018-12-20 14:09:05 -05:00
Andrzej Bialecki 846dfbef39 SOLR-13072: Fix an api change. 2018-12-20 15:49:59 +01:00
Andrzej Bialecki 1f0e875db6 SOLR-13072: Management of markers for nodeLost / nodeAdded events is broken. 2018-12-20 13:16:28 +01:00
Dawid Weiss 71f024ac8f SOLR-13074: clean up static variables properly, avoiding NPEs. 2018-12-18 23:20:55 +01:00
Chris Hostetter 73299f0f22 SOLR-13079: refactor and harden common 'suspend-trigger' patern in autoscaling test setup 2018-12-18 11:28:31 -07:00
Andrzej Bialecki f5479383b1 SOLR-13076: TimeOut breaks the simulation framework. 2018-12-17 16:49:27 +01:00
markrmiller 5e305e2f00 SOLR-13075: Add @AwaitsFix annotation to SaslZkACLProviderTest. 2018-12-16 14:49:07 -06:00
Cao Manh Dat f80e8e1167 Merge jira/http2 branch to master 2018-12-16 16:58:20 +00:00
Steve Rowe a62b5941b0 SOLR-13060: Annotate never-terminating HDFS Nightly tests with AwaitsFix 2018-12-15 12:24:57 -05:00
Dawid Weiss e916f1fb86 LUCENE-8611: Update randomizedtesting to 2.7.2, JUnit to 4.12, add hamcrest-core dependency. 2018-12-15 09:49:36 +01:00
Simon Willnauer e974311d91 LUCENE-8609: Allow getting consistent docstats from IndexWriter
Today we have #numDocs() and #maxDoc() on IndexWriter. This is enough
to get all stats for the current index but it's subject to concurrency
and might return numbers that are not consistent ie. some cases can
return maxDoc < numDocs which is undesirable. This change adds a getDocStats()
method to index writer to allow fetching consistent numbers for these stats.

This change also deprecates IndexWriter#numDocs() and IndexWriter#maxDoc()
and replaces all their usages wiht IndexWriter#getDocStats()
2018-12-14 19:36:25 +01:00
Jason Gerlowski 1d601ed646 SOLR-13065: Harden TestSimExecutePlanAction 2018-12-14 12:22:09 -05:00
Jason Gerlowski 655e9ecf7c Fix active replica count reporting in SimClusterStateProvider
One codepath assumed all replicas were active, instead of checking the
state of each individually.
2018-12-14 12:21:47 -05:00
Jason Gerlowski dd910d1a06 Indicate collection in `clusterShape` log messages
Many of Solr's tests use CloudTestUtils' `waitForState` and
`clusterShape` methods to wait until a SolrCloud cluster matches a
particular expected shape.  The code periodically checks on the cluster
state, and logs a message if a collection doesn't match the state we
expect of it.  Prior to this commit, these log messages omitted the
collection name though, which makes things a little confusing when a
test is checking on the state of multiple collections simultaneously (as
can happen when Triggers fire in the background).
2018-12-14 12:21:19 -05:00
Chris Hostetter f844461357 SOLR-13072: disable flawed test of flawed functionality 2018-12-13 15:32:09 -07:00
Chris Hostetter 46516b7077 SOLR-13068: Update many cloud/autoscaling tests to use TimeSource.NANO_TIME (instead of CURRENT_TIME) for comparisons
as much as possible, the tests actual use the time source directly from the cluster, so that even if the cluster is changed to use an alternate time source, the tests will be consistent
2018-12-13 11:02:24 -07:00
Jason Gerlowski d7ad2f46c3 SOLR-13037: Harden TestSimGenericDistributedQueue 2018-12-13 06:27:34 -05:00
Jan Høydahl 81dbad54e0 SOLR-12799: Allow Authentication Plugins to intercept internode requests on a per-request basis
Add 'forwardCredentials' parameter to BasicAuth which will then skip using PKI on sub requests
2018-12-13 10:51:56 +01:00
markrmiller 42f13731b3 SOLR-12801: Harden SimSolrCloudTests. 2018-12-12 18:50:23 -06:00
markrmiller 44b51cd041 SOLR-13067: Harden BasicAuthIntegrationTest. 2018-12-12 18:50:23 -06:00
markrmiller 7de72c9bc7 SOLR-13066: A failure while reloading a SolrCore can result in the SolrCore not being closed. 2018-12-12 18:49:06 -06:00
Jan Høydahl 9728dbc167 SOLR-12791, SOLR-13067: Fix test failure for BasicAuthIntegrationTest
Make PkiAuthenticationIntegrationTest beast-able
2018-12-13 00:14:37 +01:00
Joel Bernstein 1e68726831 SOLR-12801: Suppress SSL and older codecs 2018-12-12 15:38:40 -05:00
Jan Høydahl ef2f0cd88c SOLR-12791: Add Metrics reporting for AuthenticationPlugin 2018-12-12 11:33:32 +01:00
Jan Høydahl 280f67927e SOLR-7896: Add a login page to Admin UI, with initial support for Basic Auth 2018-12-12 10:37:23 +01:00
Jan Høydahl 5affe7421f SOLR-13025: SchemaSimilarityFactory fallback to LegacyBM25Similarity for luceneMatchVersion < 8.0 2018-12-12 10:33:50 +01:00
markrmiller bcdc6dadbb SOLR-13040: Fix TestSQLHandler, remove delete core calls. 2018-12-12 02:06:26 -06:00
Steve Rowe ec1bd0da2f SOLR-13060: set suite timeout on non-terminating HDFS Nightly tests to one hour 2018-12-11 18:49:06 -08:00
Gus Heck a7859ff62a SOLR-13058 Fix synchronized block 2018-12-11 18:45:42 -05:00
Chris Hostetter ade7546ad2 Merge remote-tracking branch 'refs/remotes/origin/master' 2018-12-11 13:21:52 -07:00
Chris Hostetter de65b45fca change NodeLostTrigger's initial run() log from INFO to DEBUG 2018-12-11 13:18:32 -07:00
markrmiller c852418148 SOLR-12801: Harden TestSimLargeCluster. 2018-12-11 14:17:10 -06:00
Chris Hostetter 3147c131e0 SOLR-13054: rewrite TriggerSetPropertiesIntegrationTest
test no longer depends on changing static non-final non-volatile variables used by multiple threads

test also no longer depends on arbitrary sleep calls, instead threads await/poll on concurrent signaling objects/queues
2018-12-11 10:39:00 -07:00
markrmiller 274a6eba90 SOLR-12801: Try to harden CdcrBootstrapTest. 2018-12-11 08:58:42 -06:00
markrmiller 19d968dc3d SOLR-12801: Harden OverseerTaskQueueTest and DistributedQueueTest. 2018-12-11 08:58:42 -06:00
markrmiller b9278d7c99 SOLR-12923: Harden TestSimTriggerIntegration. 2018-12-11 08:58:42 -06:00
markrmiller eb1735ece9 SOLR-12923: Harden TestSimLargeCluster. 2018-12-11 08:58:42 -06:00
markrmiller a25ddcd4f3 SOLR-12801: Harden InfixSuggestersTest#testShutdownDuringBuild. 2018-12-11 08:58:42 -06:00
markrmiller ab891fa472 SOLR-12801: Harden TestCloudRecovery. 2018-12-11 05:45:34 -06:00
markrmiller 496c627882 SOLR-12801: Raise this test socket timeout to 30 seconds. 2018-12-11 05:45:28 -06:00
markrmiller 3205491beb SOLR-12923: Harden TestSimTriggerIntegration. 2018-12-11 05:45:19 -06:00
markrmiller 7605427481 SOLR-12801: Harden BasicDistributedZk2Test. 2018-12-11 05:45:12 -06:00
Cao Manh Dat 40502b9ba8 SOLR-13028: Remove AwaitsFix for AutoAddReplicasPlanActionTest.testSimple to get log failure 2018-12-11 10:31:59 +00:00
Cao Manh Dat d26c237e35 Remove BadApple for TestSimTriggerIntegration.testListeners to get log failure 2018-12-11 09:53:58 +00:00
Noble Paul c651cbfd9a SOLR-12885: BinaryResponseWriter (javabin format) should directly copy from BytesRef to output 2018-12-11 18:35:57 +11:00
Chris Hostetter a2199c72d4 SOLR-13050: add workaround for issue to SystemLogListenerTest
make sure the node we kill isn't the .system collection leader
2018-12-10 19:06:42 -07:00
Jason Gerlowski f89f109ec1 SOLR-13045: Harden TestSimPolicyCloud
This commit fixes a race condition in SimClusterStateProvider, fixing
several fails in TestSimPolicyCloud.
2018-12-10 17:39:12 -05:00
Cao Manh Dat af6e15e838 SOLR-13053: NodeAddedTrigger and NodeLostTrigger do not reserve added/removed time populated by restoreState 2018-12-10 20:08:07 +00:00
markrmiller d96475a0bb SOLR-12801: Fix TestSolrCloudWithKerberosAlt. 2018-12-10 11:46:34 -06:00
markrmiller 202dad8c88 SOLR-12923: Harden TestSimLargeCluster. 2018-12-10 10:02:58 -06:00
markrmiller a1657616bb SOLR-12923: Harden TestSimTriggerIntegration. 2018-12-10 10:02:58 -06:00
markrmiller 18356de837 SOLR-12923: Harden TestSimTriggerIntegration. 2018-12-09 22:55:32 -06:00
markrmiller 5405b2f1c3 SOLR-12801: Try to harden TestReplicatinHandler#doTestIndexFetchOnMasterRestart. 2018-12-09 22:55:25 -06:00
markrmiller f47976a219 SOLR-12801: Try to harden SolrRrdBackendFactoryTest. 2018-12-09 22:55:19 -06:00
markrmiller 6f8d598781 SOLR-12801: Harden CollectionsAPISolrJTest#testCreateCollWithDefaultClusterPropertiesNewFormat. 2018-12-09 20:52:03 -06:00
markrmiller 665517d3bf SOLR-12801: Harden CollectionsAPISolrJTest#testCreateCollWithDefaultClusterPropertiesOldFormat. 2018-12-09 18:50:47 -06:00
markrmiller 1d5ae9857f SOLR-12801: Raise timeout in CollectionsAPIAsyncDistributedZkTest. 2018-12-09 18:31:28 -06:00
markrmiller 5d529802c4 SOLR-12801: harden DocValuesNotIndexedTest 2018-12-09 18:31:28 -06:00
markrmiller 0824f7b045 SOLR-12801: use waitForActiveCollection call 2018-12-09 18:31:28 -06:00
markrmiller f6063a1461 SOLR-12801: boost wait to try and harden 2018-12-09 18:31:28 -06:00
markrmiller 4133ce4cc1 SOLR-12923: Harden TestSimTriggerIntegration.testNodeMarkersRegistration. 2018-12-09 18:31:28 -06:00
markrmiller 3eb2612e67 SOLR-12988: Disable SSL on Java 11/12 for now. 2018-12-09 18:31:28 -06:00
markrmiller 0aef561245 SOLR-12923: Harden TestSimNodeLostTrigger#testTrigger. 2018-12-09 18:31:28 -06:00
markrmiller 3f1b75ab82 SOLR-12801: harden MultiThreadedOCPTest 2018-12-09 18:31:28 -06:00
markrmiller a9a5cdc34d SOLR-12801: raise jetty idle socket timeout 2018-12-09 18:31:28 -06:00
markrmiller 40432cdfbc SOLR-12801: tweak shard count specificiation 2018-12-09 18:31:28 -06:00
markrmiller 906e32af13 SOLR-12801: harden CollectionsAPIDistributedZkTest 2018-12-09 18:31:28 -06:00
markrmiller 65bfb1639c SOLR-12801: harden CollectionsAPISolrJTest 2018-12-09 18:31:28 -06:00
markrmiller 1184fbed63 SOLR-12923: Harden AutoScalingHandlerTest. 2018-12-09 18:31:28 -06:00
markrmiller b94911b260 SOLR-12801: don't do single index check here on windows. 2018-12-09 18:31:28 -06:00
markrmiller e25d6850c4 SOLR-13037: Add @AwaitsFix annotation to TestSimGenericDistributedQueue. 2018-12-09 18:31:28 -06:00
Erick Erickson f690302e05 SOLR-12727: Upgrade ZooKeeper dependency to 3.4.13 2018-12-09 08:50:51 -08:00
markrmiller 46d7994045 SOLR-13028: @AwaitsFix AutoAddReplicasPlanActionTest#testSimple. 2018-12-06 08:56:03 -06:00
markrmiller b9a966e5f7 SOLR-12833: Add configurable timeout to VersionBucket lock. 2018-12-06 08:36:56 -06:00
markrmiller 38cfd0e259 SOLR-13040: Add AwaitsFix annotation to TestSQLHandler and improve exception information related to that test. 2018-12-05 19:34:13 -06:00
Chris Hostetter 868e005760 SOLR-13032: harden MaxSizeAutoCommitTest to eliminate race conditions and eliminate use of arbitrary sleep 2018-12-04 10:44:05 -07:00
Gus Heck 59919b4ac0 SOLR-13036 Fix retry logic in JettySolrRunner 2018-12-03 21:54:34 -05:00
markrmiller e14432efd8 SOLR-12801: Wait for thread on close and add AwaitsFix annotation - most of these sim tests can fail to elect shard leaders. 2018-12-03 17:40:51 -06:00
markrmiller e3cbdd0505 SOLR-13032: Add @AwaitsFix annotation to test. 2018-12-03 11:07:40 -06:00
markrmiller 1408f5255f SOLR-12801: Disable TimeRoutedAliasUpdateProcessorTest because the feature leaks threads. 2018-12-03 09:07:27 -06:00
Gus Heck b6b9f9554e SOLR-12801 fix cut/paste oops in comment 2018-12-02 16:41:55 -05:00
Gus Heck cf4d749410 SOLR-12801 Revert breaking commit a3ec5b5fdf and improve test
to be more complete. Also, add comments to make it easier for
others to understand TimeRoutedAliasUpdateProcessorTest
2018-12-02 16:34:40 -05:00
Jason Gerlowski c951578fca SOLR-6117: Unify ReplicationHandler error handling
Prior to this commit, ReplicationHandler had a few inconsistencies
in how it reported errors:
  - Sometimes the 'status' field was used as an enum (e.g. 'success'
    v. 'failure'.  Elsewhere it is used to hold error messages.
  - Sometimes an explicit 'message' field was used, but often not.
  - Sometimes a stack trace was provided in place of an error message.

This commit tweaks the various error cases in ReplicationHandler to
report errors consistently.  'status' is always an enum-type value.  A
'message' field is provided for all errors, with an optional 'exception'
field.
2018-12-02 16:23:04 -05:00
Jason Gerlowski 6e4c9b3b58 SOLR-12555: Use `expectThrows` for expected exceptions
This commit replaces the `try { doX(); fail(); }` pattern with the
`expectThrows` test helper, which was created for this purpose.

This commit makes these changes in the core package:
`o.a.solr.core`.
2018-12-01 17:19:34 -05:00
markrmiller e5281ef3d5 SOLR-12801: Don't possibly block forever in this test in remaining spots. 2018-12-01 13:02:58 -06:00
markrmiller b294d8460f SOLR-12801: Don't possibly block forever in this test in another spot. 2018-12-01 12:55:05 -06:00
markrmiller b5a9cca168 SOLR-12801: Raise THREAD_POOL_MAX_IDLE_TIME_MS. 2018-12-01 12:42:28 -06:00
markrmiller cbe9f82202 SOLR-12801: Don't possibly block forever in this test. 2018-12-01 08:58:13 -06:00
markrmiller 755044b58f SOLR-12801: Wait for collection creation in test. 2018-12-01 08:09:03 -06:00
markrmiller 9b0b9032e2 SOLR-12801: Fix thread leak in test. 2018-12-01 00:53:12 -06:00
markrmiller 7f88bfa112 SOLR-12801: Wait for collections properly. 2018-12-01 00:26:03 -06:00
markrmiller a3ec5b5fdf SOLR-12801: Wait for executor to finish shutdown. 2018-12-01 00:26:03 -06:00
markrmiller d8f482f5fb SOLR-13030: Update executor usage to work correctly with Java 11 and update Mockito & HttpComponents to work with Java 11 and fix get remote info retry to work across jvms better. 2018-12-01 00:26:02 -06:00
markrmiller a01d0d9ef9 SOLR-13030: Close MetricsHistoryHandler inline. 2018-11-30 18:13:26 -06:00
Chris Hostetter 5dc988f5ee SOLR-12839: JSON 'terms' Faceting now supports a 'prelim_sort' option to use when initially selecting the top ranking buckets, prior to the final 'sort' option used after refinement. 2018-11-30 15:49:06 -07:00
javanna cf016f8987 LUCENE-8563: Remove k1+1 constant factor from BM25 formula numerator.
Signed-off-by: Adrien Grand <jpountz@gmail.com>
2018-11-30 09:42:49 +01:00
markrmiller 75b1831967 SOLR-12801: Make massive improvements to the tests.
SOLR-12804: Remove static modifier from Overseer queue access.

SOLR-12896: Introduce more checks for shutdown and closed to improve clean close and shutdown. (Partial)

SOLR-12897: Introduce AlreadyClosedException to clean up silly close / shutdown logging. (Partial)

SOLR-12898: Replace cluster state polling with ZkStateReader#waitFor. (Partial)

SOLR-12923: The new AutoScaling tests are way too flaky and need special attention. (Partial)

SOLR-12932: ant test (without badapples=false) should pass easily for developers. (Partial)

SOLR-12933: Fix SolrCloud distributed commit.
2018-11-29 11:58:51 -06:00
David Smiley 1534bbe4ae SOLR-5211: fix test 2018-11-27 07:50:01 -08:00
David Smiley dc134be449 SOLR-5211: ignore temporarily pending moshe fixing 2018-11-26 10:11:21 -05:00
Moshe a346ba0d3c SOLR-5211: Always populate _root_ (if defined).
And, small refactor: Clarified how _version_ is transferred from root to children.
2018-11-25 21:47:49 -05:00
Mikhail Khludnev d7b878e90c SOLR-12546: Let csv response writer to handle docValues fields by default. 2018-11-25 11:26:39 +03:00
Alan Woodward 2459072286 SOLR-12775: Add deprecated versions of LowerCaseTokenizer and LowerCaseTokenizerFactory 2018-11-22 13:54:39 +00:00
Noble Paul 492c3440de SOLR-13006: ZkNodeProps to be able to load from both javabin and JSON 2018-11-21 18:20:03 +11:00
Cao Manh Dat 05167ed01d SOLR-12986: HttpPartitionTest reliably failing seed 2018-11-20 15:01:19 +00:00
Noble Paul 25bca6f165 SOLR-12992: When using binary format, ExportWriter to directly copy BytesRef instead of creating new String 2018-11-20 14:58:32 +11:00
Christine Poerschke eb8010405d SOLR-12972: remove deprecated, unused SolrIndexConfig.luceneVersion 2018-11-19 14:51:13 +00:00
Christine Poerschke 7abb25eff5 SOLR-12972: deprecate unused SolrIndexConfig.luceneVersion 2018-11-19 14:06:09 +00:00
Alan Woodward 65486442c4 LUCENE-8497: Replace MultiTermAwareComponent with normalize() method 2018-11-19 10:09:37 +00:00
Jim Ferenczi 759af0127c LUCENE-8464: Fix legacy.TestMultiValuedNumericRangeQuery to count the number of hits accurately 2018-11-19 09:42:48 +01:00
Jim Ferenczi dc2c7784e6 Fix legacy.TestNumericRangeQuery to count the number of hits accurately 2018-11-17 21:20:27 +01:00
Christophe Bismuth b4449c73e4 LUCENE-8464: ConstantScoreScorer now implements setMinCompetitveScore in order to early terminate the iterator if the minimum score is greater than the constant score.
Signed-off-by: Jim Ferenczi <jimczi@apache.org>
2018-11-16 15:14:11 +01:00
Adrien Grand 3302ed2ac6 LUCENE-8470: Remove legacy doc values APIs from lucene/core. 2018-11-16 14:24:34 +01:00
Erick Erickson 763e64260f SOLR-12881: Remove unneeded import statements 2018-11-14 17:48:15 -08:00
Cao Manh Dat f357c06276 SOLR-12969: Inconsistency with leader when PeerSync return ALREADY_IN_SYNC 2018-11-12 10:10:22 +00:00
Cao Manh Dat 397b88aefa SOLR-12313: No need to wait for in-sync with leader in RecoveryAfterSoftCommitTest since we only care about recovery 2018-11-10 06:01:19 +00:00
Chris Hostetter 77a4bfaa90 SOLR-12962: Added a new 'uninvertible' option for fields and fieldtypes. This defaults to 'true' for backcompat allowing a FieldCache to be built for indexed fields as needed, but users are encouraged to set this to false (using docValues as needed) to reduce the risk of large fluxuations in heap size due to unexpected attempts to sort/facet/function on non-docValue fields. 2018-11-09 08:30:04 -07:00
Tim Underwood fbb987282e SOLR-12880: Show the FacetProcessor class name instead of the
FacetRequest in the JSON Facets debug-trace output
2018-11-08 23:32:26 -05:00
Tim Underwood 243a8a668a SOLR-12964: Use DocValuesIterator.advanceExact() instead of the advance()/docID() pattern 2018-11-08 14:34:17 -05:00
Gus Heck 53482e510c SOLR-12938 - fix test case for handling of bogus collection names
that was failing when HttpClusterStateProvider is used instead of
ZkClusterStateProvider
2018-11-07 20:05:32 -05:00
David Smiley d0cd4245bd LUCENE-8557: LeafReader.getFieldInfos should always return the same instance
MemoryIndex: compute/cache up-front
Solr Collapse/Expand with top_fc: compute/cache up-front
Json Facets numerics / hash DV: use the cached fieldInfos on SolrIndexSearcher
SolrIndexSearcher: move the cached FieldInfos to SlowCompositeReaderWrapper
2018-11-06 14:45:32 -05:00
Alexandre Rafalovitch 0ddbc4bf95 SOLR-12966: Add Javadoc @since tag to URP classes 2018-11-06 08:05:07 -05:00
Alexandre Rafalovitch c07df19666 SOLR-12956: Add Javadoc @since tag to Analyzer component classes 2018-11-06 07:38:20 -05:00
Cao Manh Dat 13a83564bb SOLR-12313: Make the test finish quicker by lower down intervals 2018-11-06 11:19:13 +00:00
Steve Rowe 01808eee93 SOLR-12243: Edismax missing phrase queries when phrases contain multiterm synonyms 2018-11-05 10:32:49 -05:00
Gus Heck 45b772f4a9 SOLR-12938 - Cluster status now returns results for aliases instead of
throwing an exception
2018-11-04 12:01:51 -05:00
Jason Gerlowski f669a1fb0e SOLR-12555: Use `expectThrows` for expected exceptions
This commit replaces the `try { doX(); fail(); }` pattern with the
`expectThrows` test helper, which was created for this purpose.

This commit makes these changes in the core package: `o.a.solr.search`.

Closes #464
2018-11-03 10:48:16 -04:00
Chris Hostetter c5ff4a4444 SOLR-12954: fix facet.pivot refinement bugs when using facet.sort=index and facet.mincount>1 2018-11-02 20:32:32 -07:00
Chris Hostetter 1e0e8d45e5 SOLR-7804: remove problematic assertions on lossy stats.field values that aren't fundemental to the purpose of the test anyway 2018-11-01 22:03:20 -07:00
Gus Heck 23595e5cc4 Supress an IDE warning with an annotation, and verify my commit karma 2018-11-01 21:27:16 -04:00
Tim Underwood cf445ba549 SOLR-12882: Eliminate excessive lambda allocation in json facet FacetFieldProcessorByHashDV.collectValFirstPhase 2018-11-01 15:05:16 -04:00
Chris Hostetter 05f72a77ef SOLR-12946: tighten up request/expectations in test
This should prevent some non-reproducible failures where (i think) the problem was inconsistnet background merges between collections being compared, resulting in facet counts (of 0) for terms that weren't in equivilent (merged) indexes.

since the purpose of the test has nothing to do with mincount, force facet.mincount=0 and be more exact about how many terms we expect (in case the root problem lies somewhere else, be more assertive about finding it)
2018-10-31 15:33:39 -07:00
Andrzej Bialecki 3ed64375ef SOLR-12942: Add an option in IndexSizeTrigger to select the split shard method. 2018-10-31 18:47:34 +01:00
Mikhail Khludnev 5aa8ad5b14 SOLR-12875: AIOOBE fix when unique()/uniqueBlock() is used with DVHASH method in json.facet 2018-10-30 10:56:34 +03:00
Steve Rowe d362439e27 SOLR-12861: Add Solr factory for new ByteBuffersDirectory 2018-10-29 08:23:14 -04:00
Noble Paul f33be7a172 SOLR-12868: Request forwarding for v2 API is broken 2018-10-26 12:50:45 +11:00
Tommaso Teofili 26e14986af SOLR-12879 - added missing test for min_hash qp to QueryEqualityTest 2018-10-25 09:57:16 +02:00
Varun Thacker 71988c756b SOLR-12793: Move TestCloudJSONFacetJoinDomain amd TestCloudJSONFacetSKG to the facet test package 2018-10-24 13:57:35 -07:00
Christine Poerschke ab14cc9566 SOLR-9425: fix NullPointerException in TestSolrConfigHandlerConcurrent 2018-10-24 19:30:44 +01:00
Christine Poerschke c277674f0e SOLR-12873: Replace few remaining occurrences of LUCENE_CURRENT with LATEST for luceneMatchVersion. 2018-10-24 19:28:12 +01:00
Cao Manh Dat e083b1501e Add log.warn when a replica become leader after timeout 2018-10-24 08:56:01 +07:00
David Smiley 3e89b7a771 SOLR-12754: New hl.weightMatches for UnifiedHighlighter WEIGHT_MATCHES
(defaults to true in master/8)
2018-10-23 13:28:10 -04:00
Tommaso Teofili 2e757f6c25 SOLR-12879 - registered MinHashQParserPlugin to QParserPlugin as min_hash 2018-10-23 09:12:46 +02:00
Cao Manh Dat 7512cd9425 SOLR-11812: Remove LIROnShardRestartTest since the transition from old lir to new lir is no longer supported 2018-10-23 09:58:59 +07:00
Anshum Gupta d799fd53c7 SOLR-5004: Allow a shard to be split into 'n' sub-shards using the collections API 2018-10-22 15:01:24 -07:00
Noble Paul c31a95d261 SOLR-12846: Added support for "host" variable in autoscaling policy rules 2018-10-23 00:03:59 +11:00
Andrzej Bialecki 8c70811f3a SOLR-12729: Unlock the shard on error. 2018-10-22 11:28:09 +02:00
Noble Paul 5de6332209 SOLR-12892: MapWriter to use CharSequence instead of String 2018-10-22 11:14:41 +11:00
Tommaso Teofili a7c9c9d8ce SOLR-12879 - MinHash query parser 2018-10-20 08:28:02 +02:00
David Smiley fd9164801e LUCENE-7875: Moved MultiFields static methods to MultiTerms, FieldInfos and MultiBits.
MultiBits is now public and has getLiveDocs.
2018-10-18 19:49:14 -04:00
Simon Willnauer 5a4fd86cce LUCENE-8535: Drop out of the box Block-Join highlight support
Highlighter doesn't support ToParent and ToChildBlockJoinQuery out of the
box anymore. In oder to highlight on Block-Join Queries a custom WeightedSpanTermExtractor
should be used.
2018-10-18 14:53:01 +02:00
Christine Poerschke d2f7272b0e SOLR-12877: avoid NPE in TestUtilizeNode.getReplicaList 2018-10-16 12:54:00 -04:00
Christine Poerschke 15002eba2f SOLR-12877: avoid NPE in TestTlogReplica.testRealTimeGet 2018-10-16 12:52:51 -04:00
Christine Poerschke 1ccd555862 Fix couple of typos. 2018-10-15 15:08:17 -04:00
Erick Erickson 6c5df58b5a SOLR-12028: BadApple and AwaitsFix annotations usage 2018-10-14 21:13:33 -04:00
Shalin Shekhar Mangar 971a0e3f4a SOLR-12739: Use legacy assignment in AutoAddReplicasPlanActionTest 2018-10-11 08:58:47 +05:30
Shalin Shekhar Mangar 095707d547 SOLR-12739: Remove unused import 2018-10-10 18:18:29 +05:30
Shalin Shekhar Mangar 50d1c7b481 SOLR-12739: Release the policy session as soon as we're done with the computation.
This fixes the CollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes test failures. Due to the various tests for exceptional conditions, there were times where the session was not released causing stale data to remain in the policy session cache.
2018-10-10 17:12:50 +05:30
Shalin Shekhar Mangar 940a7303ee SOLR-12739: Use cluster instead of collection as the key for using legacy assignment. 2018-10-10 16:19:17 +05:30
Shalin Shekhar Mangar 9f34a7c776 SOLR-12739: Fix failures in AutoAddReplicasIntegrationTest and its sub-class.
This test too makes assumptions about how replicas are placed. In the legacy assignment strategy, the replica of a given collection are spread equally across all nodes but with the new policy based strategy, all cores across collections are spread out. Therefore the assumptions in this test were wrong. I've changed this test to use the legacy assignment policy because testing the autoAddReplicas feature doesn't have to depend on new replica assignment strategies. This change also fixes a bug in Assign which used "collection" key instead of "cluster" to figure out which strategy to use.
2018-10-10 15:38:52 +05:30
Steve Rowe 41e3d07369 SOLR-11812: fix precommit 2018-10-09 17:34:16 -04:00
Steve Rowe 50478ea72d SOLR-11812: fix precommit 2018-10-09 17:27:21 -04:00
Andrzej Bialecki 23e22e6460 SOLR-12729: SplitShardCmd should lock the parent shard to prevent parallel splitting requests. 2018-10-09 21:30:10 +02:00
Cao Manh Dat a37a213975 SOLR-11812: Remove backward compatibility of old LIR implementation in 8.0 2018-10-09 16:20:49 +07:00
Shalin Shekhar Mangar 184ed88ecb SOLR-12845: Disable the test until we can add a default policy equivalent to the hard coded legacy assignment rules 2018-10-09 13:54:46 +05:30
Noble Paul b4d9b25f44 SOLR-12843: Implement a MultiContentWriter in SolrJ to post multiple files/payload at once 2018-10-09 17:44:40 +11:00
Shalin Shekhar Mangar dbed8bafe6 SOLR-12739: Autoscaling policy framework is now used as the default strategy to select the nodes on which new replicas or replicas of new collections are created.
Previously, the maxShardsPerNode parameter was not allowed on collections when autoscaling policy was configured. Also if an autoscaling policy was configured then the default was to set an unlimited maxShardsPerNode automatically. Now the maxShardsPerNode parameter is always allowed during collection creation and maxShardsPerNode should be set correctly (if required) regardless of whether autoscaling policies are in effect or not. The default value of maxShardsPerNode continues to be 1 as before. It can be set to -1 during collection creation to fall back to the old behavior of unlimited maxShardsPerNode when using autoscaling policy. This patch also fixes PolicyHelper to find the free disk space requirements of a new replica from the leader only if said leader node is alive.
2018-10-09 12:10:28 +05:30
Nicholas Knize 1118299c33 LUCENE-8496: Selective indexing - modify BKDReader/BKDWriter to allow users to select a fewer number of dimensions to be used for creating the index than the total number of dimensions used for field encoding. i.e., dimensions 0 to N may be used to determine how to split the inner nodes, and dimensions N+1 to D are ignored and stored as data dimensions at the leaves. 2018-10-08 18:51:03 -05:00
David Smiley e2b8beccb0 SOLR-12782: UninvertingReader avoids FieldInfo creation.
Wrapping is now a bit more lightweight: Does not create FieldInfo for fields that
can't be uninverted (saves mem) and can avoid wrapping the reader altogether if there's nothing to uninvert.
IndexSchema.getUninversionMap refactored to getUninversionMapper and no longer merges FieldInfos.
2018-10-05 20:40:39 -07:00
Shalin Shekhar Mangar eb47099ee2 SOLR-12836: ZkController creates a cloud solr client with no connection or read timeouts.
This changes ZkController to use the http client created by the update shard handler instead of creating a custom one.
2018-10-05 15:18:11 +05:30
Noble Paul 14e6eb2c24 use MapWriter._get() 2018-10-05 08:32:00 +10:00
Andrzej Bialecki 452c2dabf0 SOLR-12815: Implement maxOps limit for IndexSizeTrigger. 2018-10-04 12:48:25 +02:00
Shalin Shekhar Mangar 152fd966a7 SOLR-12827: Migrate cluster wide defaults syntax in cluster properties to a nested structure
The cluster wide defaults structure has changed from {collectionDefaults: {nrtReplicas : 2}} to {defaults : {collection : {nrtReplicas : 2}}}.  The old format continues to be supported and can be read from ZK as well as written using the V2 set-obj-property syntax but it is deprecated and will be removed in Solr 9. We recommend that users change their API calls to use the new format going forward.
2018-10-04 16:12:55 +05:30
Shalin Shekhar Mangar 05949a32c9 SOLR-12648: Use the information whether custom preferences were specified in the CloudUtil.usePolicyFramework method instead of comparing directly against default preferences. Comparing against default preferences was wrong because it ignores the case where the user explicitly added the same preferences as the default ones. 2018-10-04 13:26:51 +05:30
Shalin Shekhar Mangar 554ac64666 SOLR-12524: Marking the test as AwaitsFix 2018-10-04 06:53:10 +05:30
Tomas Fernandez Lobbe 46f753d7c6 SOLR-12767: Always include the achieved rf in the response
This commit deprecates the min_rf parameter. Solr now always includes the achieved replication
factor in the update requests (as if min_rf was always specified). Also, reverts the changes
introduced in SOLR-8034, replicas that don't ack an update will have to recover to prevent
inconsistent shards.
2018-10-03 09:40:15 -07:00
Andrzej Bialecki eba7bedadf SOLR-12725: ParseDateFieldUpdateProcessorFactory should reuse ParsePosition. 2018-10-03 13:59:02 +02:00
Shalin Shekhar Mangar 65105aa81b SOLR-12648: Autoscaling framework based replica placement is not used unless a policy is specified or non-empty cluster policy exists 2018-10-03 15:52:51 +05:30
Andrzej Bialecki e687748405 SOLR-12080: Disable this test until it's fixed. 2018-10-03 07:56:26 +02:00
Andrzej Bialecki 2201b65266 SOLR-12709: Several fixes to the simulator and its .system collection auto-creation. 2018-10-02 17:35:45 +02:00
Noble Paul 4ca885ac9e SOLR-12824: fixed test failure 2018-10-02 21:17:22 +10:00
Noble Paul 961d565955 SOLR-12824: NamedList to implement MapWriter interface 2018-10-02 17:34:58 +10:00
David Smiley fe844c739b LUCENE-8513: Remove MultiFields.getFields
SlowCompositeReaderWrapper now works with MultiTerms directly
2018-10-01 10:39:12 -04:00
Noble Paul 86e00405b0 SOLR-12792: extract out test data 2018-10-01 22:00:09 +10:00
Cao Manh Dat 93e2ae3fc4 SOLR-12774: IndexOutOfBoundsException on logging startupVersions 2018-10-01 15:25:30 +07:00
David Smiley 964cc88cee SOLR-12593: remove date parsing from extract contrib
* added "ignored_*" to the default configSet
* Updated Ref Guide info on Solr Cell to demonstrate usage without using the techproducts configSet

Closes #438
2018-09-28 16:50:11 -04:00
Andrzej Bialecki 918b11c7ce For simulated queries count also buffered updates. 2018-09-28 20:24:29 +02:00
Charles Sanders 9481c1f623 SOLR-5163: edismax now throws an exception when qf refers to a nonexistent field 2018-09-27 15:53:26 -04:00
Peter Somogyi 044bc2a485 SOLR-12652: Remove SolrMetricManager.overridableRegistryName() 2018-09-27 15:39:55 -04:00
Andrzej Bialecki 2369c89634 SOLR-12709: Add TestSimExtremeIndexing for testing simulated large indexing jobs.
Several important improvements to the simulator.
2018-09-27 14:49:43 +02:00
Shalin Shekhar Mangar c587410f99 SOLR-12756: Refactor Assign and extract replica placement strategies out of it.
Now, assignment is done with the help of a builder class instead of calling a method with large number of arguments. The number of special cases that had to be handled have been cut down as well.
2018-09-27 16:15:38 +05:30
Cao Manh Dat 667b8299e6 SOLR-12805: Store previous term (generation) of replica when start recovery process 2018-09-26 10:35:44 +07:00