Commit Graph

8103 Commits

Author SHA1 Message Date
Kevin Risden 7a9a6ef79e
SOLR-14077: Hadoop shouldn't need to look for metrics config in user home
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-12-13 22:08:48 -05:00
yonik 5f8e65c58f SOLR-14079: fix SPLITSHARD splitByPrefix in async mode 2019-12-13 17:55:05 -05:00
yonik db65c82c39 SOLR-13884: detect multiple replicas on single node 2019-12-13 17:37:44 -05:00
yonik 73c535261c SOLR-13884: use policies, preferences 2019-12-13 17:37:44 -05:00
yonik fae6aebc74 SOLR-13884: add ConcurrentCreateCollectionTest test 2019-12-13 17:37:44 -05:00
erick 8278886966 SOLR-14026: Upgrade Jetty to 9.4.24.v20191120 and dropwizard to 4.1.2 2019-12-13 10:01:37 -05:00
Dawid Weiss 16466d5256 Add dependency on log4j-1.2 api for solr core tests. 2019-12-13 15:32:59 +01:00
Dawid Weiss 4500f0e327 Consolidating versions between gradle and ant. 2019-12-13 13:31:23 +01:00
Dawid Weiss 035cede6b8 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-13 10:51:14 +01:00
Robert Muir f083f40b28 SOLR-14076: clean up static fields leak in nightly-only test 2019-12-13 04:06:07 -05:00
Kevin Risden 3ba005465a
SOLR-14048: Improve Hadoop test sanity checks
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-12-12 21:27:38 -05:00
Dawid Weiss 4cee6c68c7 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-12 17:27:28 +01:00
Kevin Risden 322ed8ce4d
SOLR-14047: Make sure tests don't pickup other Hadoop installs
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-12-12 10:23:29 -05:00
Dawid Weiss 6094d4dd13 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-12 14:16:48 +01:00
noble b35f1debe3 SOLR-14013: javabin performance regressions 2019-12-12 23:26:37 +11:00
Ishan Chattopadhyaya 7097e9c41a SOLR-13978: Removed LTR from default configset 2019-12-12 15:09:21 +05:30
Ishan Chattopadhyaya fa1ad006b6 SOLR-14071: Untrusted configsets shouldn't be allowed to use <lib> directive 2019-12-12 15:00:09 +05:30
Ishan Chattopadhyaya dce36c10e9 SOLR-13978: Remove non-essential components from default configset
The default configset no longer has the following:
   - Library inclusions (<lib ../>) for extraction, solr-cell libs, clustering, velocity and language identifier
   - /browse, /tvrh and /update/extract handlers
   - TermVector component (if someone wants it, can be added using config APIs)
   - XSLT response writer
   - Velocity response writer
If you want to use them in your collections, please add them to your configset manually or through the Config APIs.
2019-12-12 10:43:31 +05:30
Munendra S N 88b5b5a7cd SOLR-13970: disallow using collapse/expand with grouping
* Using collapse with grouping would cause inconsistent behavior.
  This is because grouping calls the same postfilter twice without
  resetting the internal state of the DocValues cache
* Using expand with grouping would cause NPE
2019-12-12 10:21:20 +05:30
Ishan Chattopadhyaya f681d433ae SOLR-14062: Split IndexSizeTriggerTest into three 2019-12-12 10:12:17 +05:30
Ishan Chattopadhyaya 8db8ab3be2 SOLR-13945: SPLITSHARD can cause data loss due to rollback when final commit fails 2019-12-12 08:58:34 +05:30
Kevin Risden 861c77cb17
SOLR-14053: Fix HDFS tests to be ignored if Native I/O isn't available (remove tests.disableHdfs)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-12-11 22:02:07 -05:00
Mikhail Khludnev f01b3e97d1 SOLR-13904: Make Analytics component sensitive to timeAllowed. 2019-12-11 23:48:17 +03:00
Robert Muir dc031ea382 SOLR-14050: clean up tests use of network addresses
Solr tests now have a similar policy to Lucene, loopback use only. If a
test tries to resolve or connect to the internet, it will get SecurityException.

Some solr tests explicitly try to talk to dead nodes with real
networking. This is not good and asking for trouble, but use low loopback port numbers instead of
multicast addresses. The idea is that it fails faster. Move these to
constants so that stuff isn't copy-pasted everywhere, in case we have to
do something different later.
2019-12-11 12:51:45 -05:00
Dawid Weiss 5cbbdae793 SOLR-14055: LegacyCloudClusterPropTest fails on Windows (requires SuppressSSL). 2019-12-11 16:40:31 +01:00
Dawid Weiss 77a5dd6926 SOLR-14055: LegacyCloudClusterPropTest fails on Windows (requires SuppressSSL). 2019-12-11 16:39:54 +01:00
Dawid Weiss 16019344cb Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-11 15:03:46 +01:00
Dawid Weiss 9e236a4835
SOLR-14053: Don't start hdfs when native libraries can't be loaded. Remove tests.disableHdfs. (#1072) 2019-12-11 15:01:24 +01:00
Jason Gerlowski d8aa04575f
SOLR-13972: Warn about insecure settings on startup (#1058) 2019-12-11 07:16:09 -05:00
Dawid Weiss 87104a155a Correct commons-compress dependency. 2019-12-11 09:44:16 +01:00
Dawid Weiss 9fad7b67b0 Follow-up to changes on master. 2019-12-11 09:01:37 +01:00
Dawid Weiss 27d5509644 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-11 08:57:18 +01:00
Ishan Chattopadhyaya 34f11c9f57 SOLR-13822: Fix Eclipse project 2019-12-11 05:55:30 +05:30
Thomas Wöckinger 9e84da95a2 SOLR-13966: forgot to test empty list 2019-12-10 16:29:19 -05:00
Kevin Risden 48775ea18e
SOLR-14033: Fix Hadoop tests with security manager
This removes the Solr security manager hacks
for Hadoop. It does so by:
* Using a fake group mapping class instead of ShellGroupMapping
* Copies a few Hadoop classes and modifies them for tests with no Shell
* Nulls out some of the static variables in the tests

The Hadoop files were copied from Apache Hadoop 3.2.0
and copied to the test package to be only picked up
during tests. They were modified to remove the need to
shell out for access. The assumption is that these
HDFS integration tests only run on Unix based systems
and therefore Windows compatibility was removed in some
of the modified classes. The long term goal is to remove
these custom Hadoop classes. All the copied classes are
in the org.apache.hadoop package.

Signed-off-by: Kevin Risden <krisden@apache.org>
2019-12-10 16:15:24 -05:00
Andrzej Bialecki c4f0c33638 SOLR-13975, SOLR-13896: ConcurrentUpdateSolrClient connection stall prevention. 2019-12-10 21:45:43 +01:00
Dawid Weiss 20eaaa012d Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-10 14:52:30 +01:00
Andrzej Bialecki fed199df7b SOLR-13563: SPLITSHARD using LINK method fails on disk usage checks. 2019-12-10 14:21:37 +01:00
Christine Poerschke b97afd79ca Fix 5 javac warnings: 'yield' may become a restricted identifier in a future release. 2019-12-10 12:32:34 +00:00
Thomas Wöckinger e250f3e82c SOLR-13966: LatLonPointSpatialField fix for RealTimeGetComponent
used in atomic/partial updates.
Closes #1035
2019-12-09 22:51:18 -05:00
Houston Putman 1c78d2c637
SOLR-12217: Support shards.preference in SolrJ for individual shard requests (#984) 2019-12-09 17:11:58 -05:00
Dawid Weiss f853d994ec Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-09 16:48:21 +01:00
Andrzej Bialecki 86cab79730 SOLR-13979: Expose separate metrics for distributed and non-distributed requests. 2019-12-09 13:46:33 +01:00
Dawid Weiss a6d7017f2e Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-09 10:55:54 +01:00
Munendra S N df508ffe01 SOLR-11706: fix for multivalued trie date in min/max and more tests
* selecting single value for multivalued trie date field is not
  supported.
* add additional tests for variance, unique, hll aggregations
2019-12-08 10:49:37 +05:30
Michael Gibney 8e9876f516 SOLR-7798: robust support for expand when used w/o collapsing (#325)
There are applications of ExpandComponent that intentionally do not
involve prior collapsing of results on the expand field, which can lead
to an NPE in expand component when expand.field (for matched docs) has
fewer unique values than the number of matched docs.

This commit refines the approach taken in SOLR-13877, which addressed
the same underlying issue.
2019-12-08 09:59:33 +05:30
Kevin Risden 7c8635d600
SOLR-14028: Fix test permissions for TestSolrCLIRunExample
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-12-07 16:32:11 -05:00
Dawid Weiss c3bb81f032 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-07 10:53:34 +01:00
Dawid Weiss 78575f0a2c Revert "Adding woodstox temporarily so that tests pass."
This reverts commit de8a37ee72.
2019-12-07 10:51:00 +01:00
Munendra S N a1e51cd777 SOLR-11706: minor refactors
* rename stdDev, variance methods to reflect the functionality
* add util functions to compute corrected stdDev and variance
* use DocValuesIterator#advanceExact to check if values exists for the doc
2019-12-07 11:16:25 +05:30
Erick Erickson 7d5d44a3c4 SOLR-13988: Harden CreateCollectionCleanupTest. Incorporate Dawid Weiss' suggestion for windows 2019-12-06 19:10:27 -05:00
Mike 155ab116a2
Revert "SOLR-13990: Switch out woodstox-core-asl with aalto-xml and upgrade woodstox stax-2 API (#1050)" (#1063)
This reverts commit 2387bb9d60.
2019-12-06 17:08:40 -06:00
Dawid Weiss 0d18581225 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-06 19:02:26 +01:00
Erick Erickson 106b9d6866 SOLR-13988: Harden CreateCollectionCleanupTest 2019-12-06 11:59:53 -05:00
Dawid Weiss de8a37ee72 Adding woodstox temporarily so that tests pass. 2019-12-06 13:37:48 +01:00
Dawid Weiss f371df327f Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-06 11:57:29 +01:00
Andrzej Bialecki d2b01ef28f SOLR-13831: Context property _loop_iter_ should be a string in order to support
variable expansion.
2019-12-05 18:27:12 +01:00
Anshum Gupta 2387bb9d60
SOLR-13990: Switch out woodstox-core-asl with aalto-xml and upgrade woodstox stax-2 API (#1050) 2019-12-05 18:37:53 +05:30
Dawid Weiss 1a24ccb4ee Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-05 11:17:34 +01:00
Munendra S N 12e8cca644 SOLR-11706: add support for aggregation on multivalued fields
* min, max, sum, sumsq, avg, stddev, variance, percentile aggregations
  in JSON facets now supports multivalued fields
2019-12-05 10:48:22 +05:30
Robert Muir c4126ef858 SOLR-14015: remove blanket filesystem read access from solr-tests.policy
Restrict this to only minimal paths like lucene. It is the defense for directory traversal attacks.
It will also help find bad bugs where things are reading filesystem in the wrong locations.
2019-12-04 23:16:19 -05:00
Robert Muir becc16fb28 SOLR-14002: fix another static leak in test 2019-12-03 22:43:11 -05:00
Robert Muir 165529767b SOLR-14000: clean up more static field leaks in tests
On windows, these objects can't be inspected due to security restrictions. So the test runner fails the tests since it does not know how big the leak is.
2019-12-03 18:51:00 -05:00
Dawid Weiss 7c26c6de02 Merge remote-tracking branch 'origin/master' into gradle-master 2019-12-03 18:45:12 +01:00
Dawid Weiss 49bab132b1 Applying two forbidden api check violations (applied to master). 2019-12-03 14:26:02 +01:00
Dawid Weiss 0f61aa9516 Forbidden APIs: add missing root locale. 2019-12-03 13:07:23 +01:00
noble a51c7b89f2 SOLR-13995: Move ZkShardTerms.Terms to SolrJ 2019-12-03 15:16:34 +11:00
Noble Paul d8f9f47ca0
SOLR-13992: Refactor code to have collection, shard name in Replica,Slice (#1051)
* SOLR-13992: Refactor code to have collection name shard name in Replica,Slice
2019-12-03 13:10:19 +11:00
Dawid Weiss d4a9842375 Initial gradle build layer. 2019-12-02 15:34:57 +01:00
Shalin Shekhar Mangar 5a697344ed SOLR-13805: NPE when calling /solr/admin/info/health on standalone solr 2019-11-30 10:04:00 +05:30
David Smiley 6a72b81ed3 SOLR-13971: Revert changes to the default configset.
* clarified these are Java system properties
* trivial dead code change; Boolean.getBoolean returns a primitive
2019-11-28 10:45:58 -05:00
Bruno Roustant b4cc3c6ad2
SOLR-13968: Support postingsFormat and docValuesFormat in schema fields.
Closes #1039
2019-11-28 16:13:10 +01:00
Jan Høydahl 9f78482295
SOLR-13969: Clean up and document AuditEvent API (#1041) 2019-11-28 09:55:11 +01:00
Ishan Chattopadhyaya 50e8cea918 SOLR-13971: Removing velocity from _default and disabling custom template support by default 2019-11-28 07:52:43 +05:30
noble 6c7a095f33 SOLR-13963: JavaBinCodec has concurrent modification of CharArr resulting in corrupt internode updates 2019-11-26 08:12:23 +11:00
Dawid Weiss 063c82ebd6 SOLR-13952: reverting Erick's commit (with permission). 2019-11-25 17:56:20 +01:00
Thomas Wöckinger b5fd6d7b22 SOLR-13961: Allow null/empty for removal of child doc in atomic update
Closes #1030
2019-11-25 09:57:07 -05:00
Erick Erickson 4b34d726ab SOLR-13952: Separate out Gradle-specific code from other (mostly test) changes and commit separately 2019-11-24 13:24:40 -05:00
Jan Høydahl 592ea19eff SOLR-12193: reverting one line back to trace 2019-11-24 00:06:07 +01:00
Jan Høydahl d809bc27f1 SOLR-12193: Move some log messages to TRACE level, remove some dead code 2019-11-23 23:57:59 +01:00
Jan Høydahl 312431b182
SOLR-13465 CoreContainer.auditloggerPlugin should be volatile (#672) 2019-11-23 01:21:47 +01:00
Jan Høydahl e45c5ce9b9
SOLR-13905 Make findRequestType in AuditEvent more robust (#1014) 2019-11-22 23:34:20 +01:00
Eric Pugh 537862d5bb Document how to load your own streaming plugins (#1025) 2019-11-22 11:16:57 -08:00
Munendra S N 2740d90dee SOLR-13912: add support for countvals aggregation in JSON facet module
* This aggregation is equivalent to StatsComponent's count
2019-11-22 18:16:20 +05:30
Chris Hostetter 8f30f6436c SOLR-13946: Fix SpellCheckCollatorTest.testEstimatedHitCounts lower/upper bounds to match computable values based on random merge policies 2019-11-21 11:46:35 -07:00
noble f98555854c SOLR-13822: Bug fixs and tests for URP loading 2019-11-21 11:25:16 +11:00
Jan Høydahl f00bcd5609
SOLR-13941: Configure JettySolrRunner same as in web.xml (#1018) 2019-11-20 10:20:54 +01:00
noble a593d3050c Revert "SOLR-13942: /api/cluster/zk/* to fetch raw ZK data"
This reverts commit 935a2987f8.

this was accidentally pushed to master.
2019-11-20 06:08:06 +11:00
noble 935a2987f8 SOLR-13942: /api/cluster/zk/* to fetch raw ZK data 2019-11-19 18:05:18 +11:00
Chris Hostetter 59465c20c4 SOLR-13943: AwaitsFix TimeRoutedAliasUpdateProcessorTest.testDateMathInStart 2019-11-18 16:32:21 -07:00
Mikhail Khludnev 51b1c5a023 LUCENE-9036: ExitableDirectoryReader checks timeout on DocValues access. 2019-11-18 23:34:43 +03:00
Chris Hostetter 1411aaee94 HdfsRecoveryZkTest & HdfsNNFailoverTest: Remove @BadApple anotation
These tests were originally anotated @BadApple in early 2018 as pat of SOLR-12028.

Neither test has failed since 2018-12-28.

Since we no longer have logs from those older jenkins builds, it's hard to be certain how/why this
test was failing, or why exactly it stopped failing – but it's possible the underlying issues were
addressed by general hardening of SolrCloud and the associated base test classes around the same time.
2019-11-18 12:22:12 -07:00
Adrien Grand 7755cdf03f
LUCENE-9027: Use SIMD instructions to decode postings. (#973) 2019-11-18 19:08:32 +01:00
Chris Hostetter f9076d85cf SOLR-13924: AwaitsFix: MoveReplicaHDFSTest 2019-11-18 09:58:00 -07:00
Ishan Chattopadhyaya 0857bb60d3 SOLR-13662, SOLR-13822: Fixing bug with refresh API, un-ignoring the test and new reference guide on Package Management 2019-11-18 04:00:00 +05:30
Ishan Chattopadhyaya 5200ee0ef9 SOLR-13662: Ignoring the PackageManagerCLITest until further investigation of failures 2019-11-17 06:06:02 +05:30
Munendra S N 0602956a44 SOLR-13911: add support for missing aggregation in JSON facet module
* This missing aggregation is equivalent to StatsComponent's missing
2019-11-16 12:12:11 +05:30
Chris Hostetter 3fb39b2960 TestDistributedStatsComponentCardinality: Remove @BadApple anotation
This test was originally anotated @BadApple ~2018-10-14 as pat of SOLR-12028.

So far in 2019, it has failed in 62 jenkins-BadApple builds, but all prior to 2019-06-14.

Since we no longer have logs from those older jenkins builds, it's hard to be certain how/why
this test was failing, or why exactly it *stopped* failing -- but a likely suspect is that
the root cause of the problem may have been fixed as part of the Jetty upgrade in SOLR-13541.
2019-11-15 22:25:39 -07:00
Andrzej Bialecki 66e78ba7c5 SOLR-13898: Allow inexact double comparison. Close resources. 2019-11-15 20:38:20 +01:00
Chris Hostetter 805305c410 SOLR-13909: ReplicationHandler testing: Replace the completely broken CheckBackupStatus with a new BackupStatusChecker helper class 2019-11-15 10:17:26 -07:00