This removes the Solr security manager hacks
for Hadoop. It does so by:
* Using a fake group mapping class instead of ShellGroupMapping
* Copies a few Hadoop classes and modifies them for tests with no Shell
* Nulls out some of the static variables in the tests
The Hadoop files were copied from Apache Hadoop 3.2.0
and copied to the test package to be only picked up
during tests. They were modified to remove the need to
shell out for access. The assumption is that these
HDFS integration tests only run on Unix based systems
and therefore Windows compatibility was removed in some
of the modified classes. The long term goal is to remove
these custom Hadoop classes. All the copied classes are
in the org.apache.hadoop package.
Signed-off-by: Kevin Risden <krisden@apache.org>
There are applications of ExpandComponent that intentionally do not
involve prior collapsing of results on the expand field, which can lead
to an NPE in expand component when expand.field (for matched docs) has
fewer unique values than the number of matched docs.
This commit refines the approach taken in SOLR-13877, which addressed
the same underlying issue.
* rename stdDev, variance methods to reflect the functionality
* add util functions to compute corrected stdDev and variance
* use DocValuesIterator#advanceExact to check if values exists for the doc
Restrict this to only minimal paths like lucene. It is the defense for directory traversal attacks.
It will also help find bad bugs where things are reading filesystem in the wrong locations.
On windows, these objects can't be inspected due to security restrictions. So the test runner fails the tests since it does not know how big the leak is.
These tests were originally anotated @BadApple in early 2018 as pat of SOLR-12028.
Neither test has failed since 2018-12-28.
Since we no longer have logs from those older jenkins builds, it's hard to be certain how/why this
test was failing, or why exactly it stopped failing – but it's possible the underlying issues were
addressed by general hardening of SolrCloud and the associated base test classes around the same time.
(cherry picked from commit 1411aaee94d49f26c55272f3876a4261357467c8)
This test was originally anotated @BadApple ~2018-10-14 as pat of SOLR-12028.
So far in 2019, it has failed in 62 jenkins-BadApple builds, but all prior to 2019-06-14.
Since we no longer have logs from those older jenkins builds, it's hard to be certain how/why
this test was failing, or why exactly it *stopped* failing -- but a likely suspect is that
the root cause of the problem may have been fixed as part of the Jetty upgrade in SOLR-13541.
(cherry picked from commit 3fb39b2960ae3b04d8f97a683a8c9daaef3f4696)
