Commit Graph

13086 Commits

Author SHA1 Message Date
markrmiller fe37e18d10 SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node exists after an exists check and the Overseer work loop should not allow free spinning the loop when it hits a KeeperException. 2019-04-15 12:54:35 -05:00
Chris Hostetter 59a3c45d9c SOLR-13336: add maxBooleanClauses (default to 1024) setting to solr.xml, reverting previous effective value of Integer.MAX_VALUE-1, to restrict risk of pathalogical query expansion.
(cherry picked from commit d90034f0d6)
2019-04-15 10:27:29 -07:00
Noble Paul d7e3953ada SOLR-13395: make DebugComponent.getRequestId() public 2019-04-15 11:14:56 +10:00
Christine Poerschke cae323629e SOLR-13366: Clarify 'Invalid stage name' warning logging in AutoScalingConfig 2019-04-12 16:15:34 +01:00
Joel Bernstein 9d6c4cb986 SOLR-13391: Update CHANGES.txt 2019-04-12 10:07:31 -04:00
Joel Bernstein 4f6e78282f SOLR-13391: Add variance and standard deviation stream evaluators
Squashed commit of the following:

commit 406d4b959a42e4830ac1c52836ccbcfc1b614b46
Author: Nazerke <seinaz1997@gmail.com>
Date:   Fri Apr 12 14:03:34 2019 +0200

    added missing package

commit 32c239687c39c5da3e4f2d0f25df73127331fa99
Author: Nazerke <seinaz1997@gmail.com>
Date:   Fri Apr 12 14:03:14 2019 +0200

    added package

commit 7b3f9bd415002969a4ec5d87a9ffbfd6fcff6e92
Author: Nazerke <seinaz1997@gmail.com>
Date:   Fri Apr 12 14:02:28 2019 +0200

    added var and stddev functions

commit 77c4f9fdd9f111862a55b645aad960457291414c
Author: Nazerke <seinaz1997@gmail.com>
Date:   Fri Apr 12 14:00:59 2019 +0200

    added test for the variance and standard deviation stream evaluators

commit 2d9692c178590b65e46cfd9e04ca0384c7d39ec5
Author: naz <nazerke.seidan@cloudera.com>
Date:   Wed Apr 10 19:50:30 2019 +0200

    added var and stddev new evaluators

commit d265225747bce9a0eabd713994ddd4990dbbbfa2
Author: naz <nazerke.seidan@cloudera.com>
Date:   Wed Apr 10 19:49:23 2019 +0200

    variance streaming evaluator

commit a3330064bb62b5723b9125334ef1d61fc3b098d3
Author: naz <nazerke.seidan@cloudera.com>
Date:   Wed Apr 10 19:49:02 2019 +0200

    standard deviation streaming evaluator
2019-04-12 09:58:16 -04:00
Jan Høydahl d05e7b3df4 SOLR-13398: Move log "Processing SSL Credential Provider chain" from INFO to DEBUG to prevent leaking into bin/solr printout
(cherry picked from commit 03f5a5e7a1)
2019-04-12 11:55:22 +02:00
Jan Høydahl de432003a7 SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in background thread
(cherry picked from commit f2c59db273)
2019-04-12 10:23:08 +02:00
Jan Høydahl 43f5e6170c SOLR-12121: Refresh JWK from IdP on invalid sig. Some logging improvements. Minor test fixes
(cherry picked from commit 3b3879d880)
2019-04-12 09:51:21 +02:00
Chris Hostetter c58787d045 SOLR-13393: Fixed ZkClientClusterStateProvider to prevent risk of leaking ZkStateReader/threads when processing concurrent requests during shutdown.
This primarily affected tests, but may have also caused odd errors/delays when restart/shutting down solr nodes.

(cherry picked from commit 980fd7d5c5)
2019-04-11 14:23:14 -07:00
Cao Manh Dat 5691f87ecc SOLR-13339: Prevent recovery, fetching index being kicked off after SolrCores already closed 2019-04-11 09:22:35 +01:00
Andrzej Bialecki f83098752c SOLR-13262: Add collection RENAME command and support using aliases in most collection admin commands. 2019-04-10 18:57:16 +02:00
Uwe Schindler 4fe1f410f4 SOLR-13388: Fix FileExchangeRateProvider to be a public class, as it appears in schema.xml 2019-04-10 18:55:42 +02:00
Andrzej Bialecki a81d2c4a45 Release static solrClient after the test is done (occasionally caused failures on jenkins). 2019-04-10 12:26:33 +02:00
Moshe d01b13c8ed SOLR-12638: Partial/Atomic updates of nested docs.
and [child] now works in RTG.

(cherry picked from commit 8527ec11af)
2019-04-10 03:03:58 -04:00
Jan Høydahl 77a4604c39 SOLR-12120: Do not fail the main request if synchronous auditing fails, log ERROR
Document that sub classes should call super.close() or a new waitForQueueToDrain() before closing itself

(cherry picked from commit 3e628b562c)
2019-04-09 15:39:19 +02:00
Chris Hostetter 90ae5c1956 SOLR-13369: disable TriLevelCompositeIdRoutingTest until someone who actually understands how the /bits option is *suppose* to work can assess it to determine if the test is flawed or it there is a bug in the underlying CompositeIdRouter
(cherry picked from commit 2533fd1ede)
2019-04-08 18:17:39 -07:00
Chris Hostetter deb7987272 SOLR-13376: Disable test until it can be re-written to reflect actual expected behavior of how/when node markers will be cleaned up
(cherry picked from commit ab55b6386b)
2019-04-08 18:06:21 -07:00
Chris Hostetter ea2a91ef1f BlockCacheTest uses BlockCache which uses the common ForkJoinPool so we need to ensure the SolrIgnoredThreadsFilter is used
(cherry picked from commit 6bc360ee67)
2019-04-08 10:57:23 -07:00
Andrzej Bialecki f3cdd326bd SOLR-13368: Tentative fix for a race condition in managed schema initialization. 2019-04-08 11:30:36 +02:00
Cassandra Targett cef83b3f86 Ref Guide: fix numbered list to resolve build warnings 2019-04-06 08:11:15 -05:00
Chris Hostetter afe1df1c22 set the lastmod of the files to be deleted to be 1 day ago so that '-remove_old_solr_logs 0' doesn't run afoul of coarse granularity timestamps on windows jenkins machines if the test starts and runs quickly
(cherry picked from commit 0b7421eca6)
2019-04-05 16:46:01 -07:00
Chris Hostetter fe1a994fe2 Use FSDirectory for TestDocTermOrdsUninvertLimit to reduce the amount of RAM the test needs.
This lowers the risk of OOM, w/o needing to sacrifice testing of how the randomized merging/segments might impact the uninversion

(cherry picked from commit 37166ce4e9)
2019-04-05 15:07:23 -07:00
Erick Erickson 37863e2d09 SOLR-12809: Document recommended Java/Solr combinations
(cherry picked from commit 7602f3c78e)
2019-04-05 10:22:44 -07:00
Erick Erickson 18dc7a5c96 SOLR-13075: Harden SaslZkACLProviderTest. Put AwaitsFix back
(cherry picked from commit 03d76c611b)
2019-04-05 08:49:43 -07:00
Joel Bernstein bb596caecb SOLR-13374: Update CHANGES.txt 2019-04-05 09:09:11 -04:00
Joel Bernstein e87514909b SOLR-13374: Add fetchSize parameter to the jdbc Streaming Expression 2019-04-05 08:54:00 -04:00
Jan Høydahl 4d95746b01 SOLR-12120: Harden AuditLoggerIntegrationTest, timing issues
(cherry picked from commit aadc94a6e3)
2019-04-05 11:35:26 +02:00
Jan Høydahl b0f4b5dc75 SOLR-12120: New AuditLoggerPlugin type allowing custom Audit logger plugins
(cherry picked from commit 361502da8a)
2019-04-05 11:34:03 +02:00
Simon Willnauer 28e8a30b53 LUCENE-8671: Expose FST off/on-heap options on Lucene50PostingsFormat (#613)
Before we can expose options to configure this postings format
on a per-reader basis we need to expose the option to load the terms
index FST off or on heap on the postings format. This already allows to
change the default in a per-field posting format if an expert user
wants to change the defaults. This essentially provides the ability to change
defaults globally while still involving some glue code.
2019-04-04 17:00:32 +02:00
Jan Høydahl 9dd51915fa SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin
(cherry picked from commit 96ef35cd2c)
2019-04-04 16:26:19 +02:00
Jason Gerlowski 4b86a39680 SOLR-13331: Fix AtomicUpdate 'remove' ops in SolrJ
The racent change introducing ByteArrayUtf8CharSequence altered the
NamedLists produced by atomic-update requests so that they include
instances of this class for requests coming in as javabin.  This is a
problem for 'remove' atomic-updates, which need to be able to compare
these ByteArrayUtf8CharSequence instances with existing field values
represented as Strings.  equals() would always return false, and
'remove' operations would have no effect.

This commit converts items as necessary to allow atomic-update
operations to work as expected.
2019-04-04 06:53:01 -04:00
Chris Hostetter de1f62e52f Fix test to use solr.disable.shardsWhitelist=true rather then trying to whitelist the set of servers
The previous version of this test had a chicken/egg problem (needed to init the servers w/the whitelist but didn't know the port nums until after init)

that caused it to require 'restarting' the servers -- leading to the possibility of 'Address already in use' errors from jenkins machines if the OS reclaimed the port between the stop/start of the jetty instance

(cherry picked from commit 3fe4d0a30a)
2019-04-03 15:15:36 -07:00
Kevin Risden 9f2e04b3f2
SOLR-13075: Harden SaslZkACLProviderTest (Kevin Risden, Hrishikesh Gadre, Peter Cseh)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-04-03 11:09:24 -04:00
Jason Gerlowski fdeedce21f SOLR-13355: Add missing CHANGES.txt entry 2019-04-03 07:41:28 -04:00
Jason Gerlowski d5b9fbee37 SOLR-13355: Obey 'ALL' for handlers with other predefined perms
Prior to this commit, RuleBasedAuthorizationPlugin would check for the
predefined 'ALL' permission only when the endpoint being hit wasn't
associated with another predefined-permission.

This resulted in some very unintuitive behavior. For example, the
permission {name:all, role:admin} would correctly prevent a
role:foo user from accessing /admin/info/properties, but would allow
write access to /admin/authorization because of the SECURITY_EDIT
predefined perm associated with that endpoint.

This commit fixes this bug so that the 'all' permission is always
consulted whether or not the endpoint is associated with other predefined
permissions.
2019-04-03 07:34:49 -04:00
Jason Gerlowski 68918206f5 SOLR-13355: Small refactors to RuleBasedAuthorizationPlugin 2019-04-03 07:34:37 -04:00
erick a248bc209e SOLR-13075: Harden SaslZkACLProviderTest. Testing new ZK version
(cherry picked from commit 3810d5e)
2019-04-02 21:46:08 -07:00
erick 312e61df5f SOLR-13363: Upgrade to ZooKeeper 3.4.14
(cherry picked from commit 5837687)
2019-04-02 20:22:17 -07:00
Christine Poerschke 4a2ea61c95 minor stream-decorator-reference.adoc tweak w.r.t. parallel function's partitionKeys 2019-04-02 19:55:12 +01:00
Christine Poerschke 9cab6266ef s/reduce/rollup wording correction in stream-decorator-reference.adoc 2019-04-02 19:55:01 +01:00
Jason Gerlowski 7c0caeacfa SOLR-13362: Add 'includeIndexFieldFlags' in LukeRequest 2019-04-01 20:40:49 -04:00
Chris Hostetter c18da4c6bc SOLR-13352: Remove risk of deadlock/threadleak when shutting down an Overseer(TriggerThread)
(cherry picked from commit 1071d09336)
2019-04-01 10:35:12 -07:00
Erick Erickson d420382d04 SOLR-12028: BadApple and AwaitsFix annotations usage
(cherry picked from commit e04a28e140)
2019-04-01 09:28:25 -07:00
Cassandra Targett 71cdf793ac SOLR-13259: clarify language around *must* reindex vs *should* reindex 2019-04-01 10:06:31 -05:00
Kevin Risden 1ad668e0bf
SOLR-13359: Make UpdateHandler support other prefixes (besides hdfs:/)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-31 10:47:17 -04:00
Jason Gerlowski 09b41ce578 Improve docs on using basic-auth in SolrJ 2019-03-31 10:38:16 -04:00
Kevin Risden ae95487db6
SOLR-11473: Make HDFSDirectoryFactory support other prefixes (besides hdfs:/)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-30 14:27:32 -04:00
erick 03a3562f78 SOLR-13349:High CPU usage in Solr due to Java 8 bug
(cherry picked from commit b2941ff)
2019-03-28 15:51:14 -07:00
Kevin Risden e99fd063b0
SOLR-13353: Add SolrCli AuthTool test
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-28 18:36:33 -04:00
Kevin Risden 543ea46afa
SOLR-13351: Workaround for VELOCITY-908
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-28 09:26:02 -04:00
Joel Bernstein 843763db06 SOLR-13294: Change test file name 2019-03-27 15:50:29 -04:00
Joel Bernstein 168e409d6d SOLR-13294: TestSQLHandler failures on windows jenkins machines 2019-03-27 15:50:08 -04:00
Ishan Chattopadhyaya b08566cacc SOLR-7414: CSVResponseWriter & XLSXResponseWriter return empty field when fl alias is combined with * selector 2019-03-27 16:29:25 +05:30
Henning Andersen 251d64a557 LUCENE-8735: Avoid FileAlreadyExistsException on windows. (#619)
FilterDirectory.getPendingDeletions() did not delegate the call, which
resulted in a new IndexWriter on same directory not considering pending
delete files. This could in turn result in a FileAlreadyExistsException
when running windows.
2019-03-26 15:04:52 +01:00
Cassandra Targett 3d317ffbe2 SOLR-11959: Add docs on CDCR limitation with Basic Auth 2019-03-26 08:01:08 -05:00
Kevin Risden efbb494c8c
SOLR-13112: Upgrade jackson to 2.9.8
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-25 12:19:39 -04:00
Kevin Risden f545e64061
SOLR-13335: Upgrade to velocity 2.0 and velocity-tools 3.0
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-25 10:02:23 -04:00
Kevin Risden 862a755d67
SOLR-13342: Remove dom4j from Solr
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-25 08:31:26 -04:00
Gus Heck cbaa68e9c0 sysout checking SOLR-13322 2019-03-24 21:29:16 -04:00
Gus Heck 33edce88f1 SOLR-13323 - remove unused csv related classes/tests 2019-03-24 20:35:15 -04:00
Shalin Shekhar Mangar f220a511b9 SOLR-11126: Move change log entry under 8.0 2019-03-22 07:31:35 +05:30
Andrzej Bialecki 62c6e8d5d9 SOLR-13322: Properly log exceptions. 2019-03-21 22:22:34 +01:00
Kevin Risden 96c293ec1b
SOLR-9079: Remove commons-lang as a dependency
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-20 19:51:34 -04:00
David Smiley 8f25ded16f SOLR-13253: avoid using IndexSchema.getResourceLoader for non-schema things.
Furthermore it's reference to SolrConfig was removed.

(cherry picked from commit 85a702cdff)
2019-03-19 13:52:38 -04:00
Bar Rotstein de58717183 SOLR-12955: Refactored DistributedUpdateProcessor to put SolrCloud specifics into a subclass
Closes #528

(cherry picked from commit 5b7866b085)
2019-03-19 13:28:27 -04:00
Andrzej Bialecki b778417054 SOLR-11127: REINDEXCOLLECTION command for re-indexing of existing collections. 2019-03-19 13:47:44 +01:00
Kevin Risden 105979fb4c
SOLR-8033: Remove debug if branch in HdfsTransactionLog
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-19 06:20:47 -04:00
Kevin Risden 6064b03ac6
SOLR-13330: Improve HDFS tests
Related JIRAs:
* SOLR-11010
* SOLR-11381
* SOLR-12040
* SOLR-13297

Changes:
* Consolidate hdfs configuration into HdfsTestUtil
* Ensure socketTimeout long enough for HDFS tests
* Ensure HdfsTestUtil.getClientConfiguration used in tests
* Replace deprecated HDFS calls
* Use try-with-resources to ensure closing of HDFS resources

Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-19 05:27:46 -04:00
erick 0894e39ceb SOLR-13268: Patch that flushes when shutting down 2019-03-18 11:53:54 -07:00
Chris Hostetter 695bfa3c90 SOLR-12923: Mea culpa: Remove useless import of java.lang... that breaks precommit
(cherry picked from commit 5c143022e7)
2019-03-15 14:39:42 -07:00
Chris Hostetter 87ec0c3600 SOLR-12923: fix SimClusterStateProvider to use lock.lockInterruptibly() exclusively, and make SimCloudManager's Callable checks tollerant of Callables that may have failed related to interrupts w/o explicitly throwing InterruptedException
(cherry picked from commit 1a54c6b19d)
2019-03-15 14:24:07 -07:00
Chris Hostetter 0ac45c1668 SOLR-12923: tweak the randomization in testCreateLargeSimCollections to reduce the max possible totalCores
also decrease the number of iters while increase the cluster shape wait time to reduce the risk of spurious failures on machines under heavy contention w/o making the the test any slower on average

(cherry picked from commit c79aeee5f9)
2019-03-15 14:24:07 -07:00
Chris Hostetter 653ba8d245 SOLR-12923: Fix some issues w/concurrency and exception swallowing in SimClusterStateProvider/SimCloudManager
There are 3 tightly related bug fixes in these changes:

1) ConcurrentModificationExceptions were being thrown by some SimClusterStateProvider methods when
   creating collections/replicas due to the use of ArrayLists nodeReplicaMap. These ArrayLists were changed
   to use synchronizedList wrappers.
2) The Exceptions from #1 were being swallowed/hidden by code using SimCloudManager.submit() w/o checking
   the result of the resulting Future object. (As a result, tests waiting for a particular ClusterShape
   would timeout regardless of how long they waited.)   To protect against "silent" failures like this,
   this SimCloudManager.submit() has been updated to wrap all input Callables such that any uncaught errors
   will be logged and "counted."  SimSolrCloudTestCase will ensure a suite level failure if any such failures
   are counted.
3) The changes in #2 exposed additional concurrency problems with the Callables involved in leader election:
   These would frequently throw IllegalStateExceptions due to assumptions about the state/existence of
   replicas when the Callables were created vs when they were later run -- notably a Callable may have been
   created that held a reference to a Slice, but by the time that Callable was run the collection (or a
   node, etc...) refered to by that Slice may have been deleted.  While fixing this, the leader election
   logic was also cleaned up such that adding a replica only triggers leader election for that shard, not
   every shard in the collection.

While auditing this code, cleanup was also done to ensure all usage of SimClusterStateProvider.lock was
also cleaned up to remove all risky points where an exception may have been possible after aquiring the
lock but before the try/finally that ensured it would be unlocked.

(cherry picked from commit 76babf876a)
2019-03-15 14:24:07 -07:00
Jan Høydahl 4540fa427a SOLR-13244: Nodes view fails when a node is temporarily down
(cherry picked from commit 8f29d1eaad)
2019-03-15 13:31:34 +01:00
Andrzej Bialecki f7e41ef868 SOLR-13292: Fix a logic bug when point types are present, add more details to the unit test. 2019-03-15 11:54:13 +01:00
Alan Woodward ecf86ecab7 LUCENE-3041: QueryVisitor (#581)
This commit adds an introspection API to Query, allowing users to traverse
the nested structure of a query and examine its leaves.  It replaces the existing
`extractTerms` method on Weight, and alters some highlighting code to use
the new API
2019-03-15 08:29:32 +00:00
Moshe 7961e17e7e SOLR-13129: nested docs: add more/better documentation in Solr ref-guide
(cherry picked from commit c2a6772f1e)
2019-03-14 09:12:41 -04:00
Jan Høydahl e68d16f32c SOLR-12121: Remove a System.out.println in JWTAuthPlugin
(cherry picked from commit a18aa2118b)
2019-03-14 10:52:32 +01:00
Shalin Shekhar Mangar cedff86aaa SOLR-13234: Fix for turkish locales
WhenSolrExporterIntegrationTest.jvmMetrics ran on a JVM with the Turkish locale, (test seed: 62880F3B9F140C89). The JVM metric for terminated thread-count has a dotless-i e.g. termınated.
This causes the check for matching metrics to fail.

We could normalize the text in this case, however I think it's better to ensure we have the correct total number of JVM thread metrics rather than looking at Prometheus labels which maybe localized.

This closes #605.

(cherry picked from commit 6d0386c901)
2019-03-14 11:12:35 +05:30
Gus Heck 1d85f13eeb SOLR-13131 Category Routed Aliases
(cherry picked from commit d8f2a02fdb)
2019-03-13 01:36:40 -04:00
Gus Heck 39adb09fc9 Remove inadvertently included println 2019-03-12 16:07:02 -04:00
Gus Heck 470813143d SOLR-12891 MacroExpander will no longer will expand URL parameters by
default inside of the 'expr' parameter, add InjectionDefense class
for safer handling of untrusted data in streaming expressions and add
-DStreamingExpressionMacros system property to revert to legacy behavior

(cherry picked from commit 9edc557f45)
2019-03-12 11:56:28 -04:00
Erick Erickson a87ba09e11 SOLR-13268: Clean up any test failures resulting from defaulting to async logging. Put TestXmlQParser back
(cherry picked from commit b893548d97)
2019-03-11 21:42:37 -07:00
Erick Erickson 6f6fbf0be2 SOLR-12732: TestLogWatcher failure on Jenkins. Deal with superclass logging bleeding into first watcher in the test 2019-03-11 13:00:26 -07:00
Chris Hostetter 44ff1c0ef9 SOLR-13300: DistributedFacetExistsSmallTest should not attempt to compare results of a query that is known to differ in behavior in distributed mode
(cherry picked from commit 7eb728a611)
2019-03-11 10:03:07 -07:00
Kevin Risden b4d9a98b32
SOLR-13307: Ensure HDFS tests clear System properties they set (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-11 11:22:22 -04:00
Mikhail Khludnev 5a5a39ee18 SOLR-13284: fallback to json instead of NPE and 500 code, when wt is omitted or wrong. 2019-03-10 23:12:22 +03:00
erick 277251c202 SOLR-13268: Clean up any test failures resulting from defaulting to async logging
(cherry picked from commit 9272c29)
2019-03-10 11:35:51 -07:00
Andrzej Bialecki f6f5f995ef SOLR-13292: Provide extended per-segment status of a collection. 2019-03-09 14:09:34 +01:00
Erick Erickson 83ab355772 SOLR-12732: TestLogWatcher failure on Jenkins. Added more logging
(cherry picked from commit 8c6e305365)
2019-03-08 21:43:08 -08:00
Chris Hostetter 7ebd7f2fa5 Merge remote-tracking branch 'refs/remotes/origin/branch_8x' into branch_8x 2019-03-08 14:55:05 -07:00
Chris Hostetter de5f0947e6 Delete OpenCloseCoreStressTest
This is a fairly old test that isn't particularlay valuable, all it really does is stress test indexing, but w/o actually validating the results.

In it's current state, it can suffer from exceeding file handle limits depending on merge settings, and isn't reliably reproducible due to it's
approach of "do (a variable) amount of work until a fixed amount of wall clock time has elapsed"

(cherry picked from commit 4335ad97af)
2019-03-08 14:53:10 -07:00
Jan Høydahl 4dfe945ae7 SOLR-12121: Move CHANGES entry from Improvements to New Features section
(cherry picked from commit 9eabaf46a2)
2019-03-08 22:11:54 +01:00
Chris Hostetter 27aeb11a37 SOLR-12923: harden TestSimLargeCluster
- added logging
- ensure start/finish trigger action counters are incremented before latches are released
- replace arbitrary sleep calls with a trigger listener countdown latch
- increase all await() times: This means that 'real' failures (which should be rare and hopefully
  reproducible) will be 'slow', but the trade off will be less hard to reproduce 'false failures'
  due to thread contention on slow or heavily loaded (ie: jenkins) machines

(cherry picked from commit 20de3d2ee0)
2019-03-07 15:41:50 -07:00
Christine Poerschke c55415cda8 SOLR-13254: Correct message that is logged in solrj's ConnectionManager when an exception occurred while reconnecting to ZooKeeper. (hu xiaodong via Christine Poerschke) 2019-03-07 21:09:43 +00:00
Erick Erickson fc76b70bd5 SOLR-13261: Make SortableTextField work with export/streaming, now requires useDocValuesAsStored='true'
(cherry picked from commit 1e09268e78)
2019-03-07 11:00:01 -08:00
Uwe Schindler dac4169b01 Fix Windows startup script to disable HTTP/2 if TLS is enabled on Java 8. 2019-03-07 14:37:17 +01:00
Gus Heck f772e4143e SOLR-13296 fix doc example so that it can be accepted by Solr 2019-03-07 08:07:12 -05:00
Cao Manh Dat 747832907b SOLR-13302: Adding HttpPartitionTest and ForceLeaderTest for TLOG replicas 2019-03-07 12:18:58 +00:00
Cao Manh Dat b40efc883c SOLR-12313: Fix precommit 2019-03-07 12:01:09 +00:00
Cao Manh Dat d6ee413acd SOLR-12313: Removing TestInjection#waitForInSyncWithLeader 2019-03-07 11:26:12 +00:00
Chris Hostetter bb5b7f08b1 SOLR-13237: AwaitsFix LeaderTragicEventTest until we can make the underlying corruption detection reliable
(cherry picked from commit e827dc2234)
2019-03-06 13:29:42 -07:00
Joel Bernstein 9456d25b13 SOLR-13287: Remove System.out debugging 2019-03-06 13:57:05 -05:00
erick e4e4ea3771 Clean up any test failures resulting from defaulting to async logging, reenable shutdown hook
(cherry picked from commit 5f91670)
2019-03-05 14:11:27 -08:00
Erick Erickson bbf4cbc17e SOLR-13295: Reproducible failure in TestDistributedGrouping 2019-03-05 14:02:42 -08:00
Mikhail Khludnev 76df3ac75d SOLR-9882: picking results for mergeIds even after partial ones occurred 2019-03-05 23:28:47 +03:00
Andrzej Bialecki 5f605daf92 SOLR-13271: Read-only mode for SolrCloud collections. 2019-03-05 17:17:00 +01:00
Joel Bernstein 357254b642 SOLR-13287: Update CHANGES.txt 2019-03-05 10:12:52 -05:00
Joel Bernstein 6ccd03e1fd SOLR-13287: Allow zplot to visualize probability distributions in Apache Zeppelin 2019-03-05 09:44:25 -05:00
Adrien Grand b15dfbc7a9 SOLR-13234: Remove 9.0 from Solr changes. 2019-03-05 09:56:29 +01:00
Chris Hostetter 30f7562eb4 SOLR-13294: refactor test to include more loging to help diagnose some windows jenkins failures 2019-03-04 14:16:20 -07:00
Cassandra Targett 68adeab46a SOLR-13259: Add new section on Reindexing in Solr (#594)
Add new reindexing.adoc page; standardize on "reindex" vs "re-index"
2019-03-04 12:25:35 -06:00
Chris Hostetter b8b51389ae SOLR-12923: harden testEventQueue by replacing the arbitrary sleep call with a countdown latch
(cherry picked from commit 7f7357696f)
2019-03-04 10:00:11 -07:00
Chris Hostetter 666e83d84a SOLR-12923: increase all await() times in TriggerIntegrationTest
This means that 'real' failures (which should be rare and hopefully reproducile) will be 'slow', but the trade off will be less hard to reproduce 'false failures' due to thread contention on slow or heavily loaded (ie: jenkins) machines

(cherry picked from commit 235b15acfc)
2019-03-04 10:00:03 -07:00
Cao Manh Dat 0acc1f0584 SOLR-13276: Move change entry to 8.1 2019-03-04 15:52:18 +00:00
Jason Gerlowski a545592c15 SOLR-13256: Add ref-guide upgrade notes for 7.7 2019-03-04 10:31:42 -05:00
Mikhail Khludnev 17c2634217 SOLR-9882: reporting timeAllowed breach as partialResults instead of 500 error 2019-03-04 17:43:58 +03:00
Adrien Grand 1c1fd08a2c Add 7.7.1 version and bwc indices. 2019-03-04 10:51:47 +01:00
Cao Manh Dat 3064b9bb5f SOLR-13276: Adding Http2 equivalent classes of CloudSolrClient and HttpClusterStateProvider 2019-03-04 09:43:35 +00:00
Shalin Shekhar Mangar 02eb9d3440 SOLR-13234: Prometheus Metric Exporter not threadsafe.
This changes the prometheus exporter to collect metrics from Solr on a fixed interval controlled by this tool and prevents concurrent collections. This change also improves performance slightly by using the cluster state instead of sending multiple HTTP requests to each node to lookup all the cores.

This closes #571.

(cherry picked from commit 1f9c767aac)
2019-03-04 11:04:33 +05:30
Noble Paul b14748e61f SOLR-13285: Updates with enum fields and javabin cause ClassCastException 2019-03-04 10:26:09 +11:00
Chris Hostetter 677a97edc7 SOLR-12923: increase all await() times in TestSimTriggerIntegration
This means that 'real' failures (which should be rare and hopefully reproducile) will be 'slow', but the trade off will be less hard to reproduce 'false failures' due to thread contention on slow or heavily loaded (ie: jenkins) machines

(cherry picked from commit fed80599a6)
2019-03-01 11:38:31 -07:00
Noble Paul 3fcf7fa54e SOLR-13282: JDBCStreamTest should use assertEquals(double, double, delta) instead of '==' to compare 2 double values 2019-03-01 10:14:11 +11:00
Chris Hostetter 62d7385a0e fix AfterClass in OverseerTest to not fail with NPE if assumeWorkingMockito() call short circuited the BeforeClass
(cherry picked from commit 5e28e94a93)
2019-02-28 15:17:31 -07:00
Joel Bernstein d4158f0106 SOLR-10436: Update CHANGES.txt 2019-02-28 09:33:16 -05:00
Joel Bernstein b516d67c93 SOLR-10436: Add hashRollup Streaming Expression 2019-02-28 09:22:21 -05:00
Kevin Risden ba70919dd6
SOLR-7321: Remove reflection in FSHDFSUtils.java (Mike Drob, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-27 14:07:49 -05:00
Cao Manh Dat feff459a1b SOLR-12297: Remove debugging System.out line 2019-02-27 09:54:35 +00:00
Mikhail Khludnev 34757670d9 SOLR-13227: don't throw exceptions for counting facet.range.other. 2019-02-26 23:43:31 +03:00
Kevin Risden 71d8aefc0c
SOLR-9762: Remove the workaround implemented for HADOOP-13346 (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-26 14:53:15 -05:00
Kevin Risden 2234e9ba67
SOLR-13074: MoveReplicaHDFSTest leaks threads, falls into an endless loop, logging like crazy (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-26 14:49:52 -05:00
Kevin Risden 129de3f5e9
SOLR-13060: Improve HdfsAutoAddReplicasIntegrationTest and HdfsCollectionsAPIDistributedZkTest
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-25 11:29:38 -05:00
Kevin Risden e0231fddba
SOLR-9763: Remove the workaround implemented for HADOOP-12767 (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-24 14:24:21 -05:00
erick a108b4f730 SOLR-13268: Clean up any test failures resulting from SOLR-12055 (async logging). Kevin's upgrades
(cherry picked from commit fe5a96a)
2019-02-24 10:05:57 -08:00
Mikhail Khludnev d68e992d03 SOLR-11883: reporting syntax errors as Bad Request instead of 500. 2019-02-24 16:39:40 +03:00
erick 4aa0645ea6 SOLR-12055: SOLR-12753: Missed mentioning SOLR-12753 in 8.1 CHANGES.txt 2019-02-22 13:14:21 -08:00
erick 4530c8041f SOLR-12055: Enable async logging by default SOLR-12753: Async logging ring buffer and OOM error 2019-02-22 12:49:31 -08:00
erick 07cc2d98ef SOLR-12055: Enable async logging by default SOLR-12753: Async logging ring buffer and OOM error
(cherry picked from commit 0de3905)
2019-02-22 12:47:29 -08:00
Erick Erickson cf1e3c4603 SOLR-13261: Make SortableTextField work with export/streaming
(cherry picked from commit 6b4e90617d)
2019-02-22 11:06:04 -08:00
Tomas Fernandez Lobbe 7e7047a225 Fix TestTlogReplica
At some point got broken and nobody noticed since it's not running
2019-02-21 21:39:44 -08:00
Jason Gerlowski 97d8fac87b Minor cleanup to ExportWriter 2019-02-21 16:06:20 -05:00
Tomas Fernandez Lobbe 4c0ecedf2b Fix TestPullReplica.testKillLeader
Somewhere along the way, this was broken and since the test is being excluded, nobody noticed
2019-02-20 21:16:39 -08:00
Tomas Fernandez Lobbe 128da043df SOLR-12708: CREATE collection request doesn't fail or cleanup when the request fails (#568)
When performing an async CREATE requests, responses to internal requests are also included inside the "success" or "failed" elements of the general response. This will make the operation cleanup in case of failure, the same way as we do when the request is synchronous.
2019-02-20 14:06:10 -08:00
Christine Poerschke 1c55df083c Fix typo in 'Index Size Trigger' example JSON. 2019-02-20 20:24:29 +00:00
Kevin Risden 5d71e90b17
SOLR-13222: Improve logging in StreamingSolrClients (Peter Cseh via Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-20 11:10:01 -05:00
Noble Paul e636c47bf6 SOLR-13255 : ClasscastException when URPs try to read a String field which returns a ByteArrayUTF8CHarSequence . This is a regression in release 7.7 2019-02-20 22:21:39 +11:00
Noble Paul 7a33ba3323 SOLR-13255 : ClasscastException when URPs try to read a String field which returns a ByteArrayUTF8CHarSequence . This is a regression in release 7.7 2019-02-20 22:20:35 +11:00
Kevin Risden cdabcb88d3
SOLR-11763: Upgrade Guava to 25.1-jre (Markus Jelsma, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-19 17:29:06 -05:00
shalinmangar d697b4a6ad SOLR-13248: Adding upgrade notes which explain the problem and the mitigation as well as steps to revert to the old behavior
(cherry picked from commit 97875af3f9)
2019-02-18 14:26:28 -08:00
shalinmangar 7ede4e2b43 SOLR-13248: Autoscaling based replica placement is broken out of the box.
Solr 7.5 enabled autoscaling based replica placement by default but in the absence of default cluster policies, autoscaling can place more than 1 replica of the  same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not respected. Due to these reasons,  this issue reverts the default replica placement policy to the 'legacy' assignment policy that was the default until Solr 7.4.

(cherry picked from commit 7e2d40197c)
2019-02-18 14:26:20 -08:00
Ishan Chattopadhyaya a1efe97931 SOLR-11876: In-place updates fail during resolution if required fields are present 2019-02-18 14:34:49 +05:30
Erick Erickson 4428f53c91 SOLR-12028: BadApple and AwaitsFix annotations usage.
(cherry picked from commit 3235c61e77)
2019-02-17 10:41:36 -08:00
Tomas Fernandez Lobbe 3f0dd8a70f SOLR-13229: Cleanup replicasMetTragicEvent after all exceptions 2019-02-15 14:52:06 -08:00
Simon Willnauer 8dfbbec892 LUCENE-8292: Make TermsEnum fully abstract (#574) 2019-02-15 17:33:44 +01:00
Alan Woodward 249dfdae01 SOLR-13126: Correctly combine multiplicative query boosts 2019-02-15 11:20:35 +00:00
Cassandra Targett b57cacbb71 Ref Guide: fix "cloud mode" to "SolrCloud mode" 2019-02-14 15:36:02 -06:00
Cassandra Targett 6633d4665c SOLR-12770: make docs on shards param a little more clear, fix a couple typos 2019-02-14 14:56:55 -06:00
Andrzej Bialecki 18a48934b5 SOLR-13252: Fix an NPE when setting a "policy" property for an existing collection. 2019-02-14 21:38:58 +01:00
Christine Poerschke 4c81d4dd94 further solr-ref-guide grammar fixes (s/rather then/rather than/g) 2019-02-14 19:01:04 +00:00
Ken Geis 6285b3ba15 fix grammar in streaming-expressions.adoc
(Ken Geis via Christine Poerschke)

This closes #572 pull request, thank you.
2019-02-14 18:49:23 +00:00
Andrzej Bialecki b6ebc748c9 SOLR-13155: Minor addition to the "stats" section to make it easier to
estimate the distribution of cores across nodes.
2019-02-14 11:39:41 +01:00
Jason Gerlowski a6f9f53ce5 SOLR-13241: Add 'autoscaling' tool to solr.cmd
Prior to this commit, SOLR-13155 added support for an 'autoscaling' tool
to bin/solr, but not to the Windows equivalent bin/solr.cmd.  This
commit adds the necessary plumbing to the Windows version of this
script.  It also removes some dead help-text from the bin/solr script.
2019-02-13 10:51:45 -05:00
Noble Paul 93288d7e28 SOLR-13171 : A true streaming parser for javabin payload/stream without creating any objects 2019-02-13 18:25:09 +11:00
Bruno P. Kinoshita a1b2913049 Fix some spell check issues 2019-02-12 21:36:48 -08:00
Andrzej Bialecki 98cea5440f SOLR-13155: Add command-line option for testing autoscaling configurations. 2019-02-12 11:53:57 +01:00
Alan Woodward 83f8d14226 Add next minor release version 8.1.0
This reverts commit b20afd322d.
2019-02-12 09:15:58 +00:00
Mikhail Khludnev 8777d3ff6b SOLR-12330: clarifying domain.(filter,query) for json.facet in the ref guide. 2019-02-10 23:39:00 +03:00
Mikhail Khludnev b63b8756b7 SOLR-12330: rethow NPEs and other json.facet syntax errors properly as 400 2019-02-10 00:02:06 +03:00
yyuan2 d60b1e4ee0 LUCENE-8662: Change TermsEnum.seekExact(BytesRef) to abstract 2019-02-08 15:16:50 -08:00
Cassandra Targett 416c9f88b8 Ref Guide: fix typos & abbreviated words 2019-02-08 13:11:10 -06:00
Alan Woodward 2655d44e0f Fix escaping in Solr Reference Guide 2019-02-08 13:45:48 +00:00
Tomas Fernandez Lobbe e8b7700ba9 Set types in ZkController callables 2019-02-07 22:27:20 -08:00
Konstantin Perikov cfceff87c4 Update language-analysis.adoc 2019-02-07 22:12:58 -08:00
Chris Hostetter 2d48bde21b disable TestInjection in RestartWhileUpdatingTest
work around for SOLR-13189 and SOLR-13212

(cherry picked from commit 956772b7ef)
2019-02-07 18:27:47 -07:00
Kevin Risden fd061bddf2
SOLR-9515 - Remove pom dependency kerby-kerb
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-07 11:47:42 -05:00
Jason Gerlowski 199725d5d5 SOLR-13042: Miscellaneous JSON Faceting ref-guide improvements 2019-02-06 21:17:32 -05:00
Chris Hostetter 69b5a04a4d SOLR-13210: Fix TriLevelCompositeIdRoutingTest to actually make sense
(cherry picked from commit 87ad59f826)
2019-02-06 14:45:07 -07:00
Chris Hostetter a38336285d Harden OrderedExecutorTest to use concurrent latches/barriers for testing parallelism instead of making assumpions about how milliseconds something should take in another thread
(cherry picked from commit ea2956fda3)
2019-02-06 14:45:02 -07:00
Tomas Fernandez Lobbe 9bafc4912f Removed some unused variables from DistributedUpdateProcessor 2019-02-06 11:32:51 -08:00
Jan Høydahl d28ec13b9a SOLR-12121: Use a different JSON parser for JWTAuthPluginTest after upgrade to Hadoop3 2019-02-06 09:02:24 +01:00
Jan Høydahl 0b3ebdd290 SOLR-12121: Fix test fails in JWTAuthPluginIntegrationTest.testMetrics
(cherry picked from commit d09c724a0e)
2019-02-06 08:39:27 +01:00
Jan Høydahl ceb73803a6 SOLR-12121: JWT Token authentication plugin with OpenID Connect implicit flow login through Admin UI
(cherry picked from commit ea2c8ba38e)
2019-02-06 08:39:03 +01:00
Dawid Weiss a552be9a1e SOLR-13193: LeaderTragicEventTest doesn't catch NoSuchFileException when corrupting files 2019-02-05 11:27:16 +01:00
Joel Bernstein 14665dac6f SOLR-13146: Update CHANGES.txt 2019-02-03 18:27:18 -05:00
Kevin Risden dfa3ea349a
SOLR-10199: Solr's Kerberos functionality does not work in Java9 due to dependency on hadoop's AuthenticationFilter which attempt access to JVM protected classes
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-02 12:04:22 -05:00
Kevin Risden 52b1c580ac
SOLR-8182: Update TestSolrCloudWithKerberosAlt to work with JDK9+
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-02 12:04:13 -05:00
markrmiller f20d2d0154
SOLR-9515: Update to Hadoop 3 (Mark Miller, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-02 09:33:36 -05:00
David Smiley 10930fd83a SOLR-5211: Document that delete-by-id (and updates) don't affect child/nested docs
(cherry picked from commit 372d68f7f6)
2019-02-01 15:24:53 -05:00
Noble Paul 34da61e863 SOLR-12999: Index replication could delete segments before downloading segments from master if there is not enough disk space 2019-02-01 08:17:05 +11:00
Noble Paul d3c686aa24 SOLR-12999: Index replication could delete segments before downloading segments from master if there is not enough disk space 2019-02-01 08:15:22 +11:00
Chris Hostetter 2c6d666eda reduce number of nodes/shards used in TriLevelCompositeIdRoutingTest to prevent (incredibly frequent) OOM in nightly
(cherry picked from commit 806744b0f1)
2019-01-31 10:46:07 -07:00
Joel Bernstein 4457115cf0 SOLR-13147: Update CHANGES.txt 2019-01-30 14:51:19 -05:00
Chris Hostetter 73cfa810c7 disable TestInjection in TestStressCloudBlindAtomicUpdates
work around for SOLR-13189

(cherry picked from commit 0a01b9e127)
2019-01-29 21:23:21 -07:00
Joel Bernstein b3c9082f77 SOLR-13104: Update CHANGES.txt 2019-01-29 22:01:43 -05:00
Joel Bernstein 768a62702a SOLR-13134: Update CHANGES.txt 2019-01-29 21:56:15 -05:00
Joel Bernstein d4342c4d78 SOLR-13088: Update CHANGES.txt 2019-01-29 21:49:35 -05:00
Joel Bernstein a1595a18aa SOLR-12984: Update CHANGES.txt 2019-01-29 21:39:42 -05:00
jimczi b20afd322d Revert "Add next minor release version 8.1.0"
This reverts commit df4fabb3df.
2019-01-29 17:02:40 +01:00
jimczi ed49c5c14e Add next release version 7.8.0 2019-01-29 14:28:13 +01:00