previously, entire classpath of ant (ant itself, plugins, ivy, etc) were
polluting the unit tests classpath. it leads to non-reproducible build
issues because tests classpath is different depending on things outside
of source code control.
for example, solr tests launching hadoop, hadoop launching jetty, jetty
scanning classpath -> boom
Solr tests now have a similar policy to Lucene, loopback use only. If a
test tries to resolve or connect to the internet, it will get SecurityException.
Some solr tests explicitly try to talk to dead nodes with real
networking. This is not good and asking for trouble, but use low loopback port numbers instead of
multicast addresses. The idea is that it fails faster. Move these to
constants so that stuff isn't copy-pasted everywhere, in case we have to
do something different later.
This removes the Solr security manager hacks
for Hadoop. It does so by:
* Using a fake group mapping class instead of ShellGroupMapping
* Copies a few Hadoop classes and modifies them for tests with no Shell
* Nulls out some of the static variables in the tests
The Hadoop files were copied from Apache Hadoop 3.2.0
and copied to the test package to be only picked up
during tests. They were modified to remove the need to
shell out for access. The assumption is that these
HDFS integration tests only run on Unix based systems
and therefore Windows compatibility was removed in some
of the modified classes. The long term goal is to remove
these custom Hadoop classes. All the copied classes are
in the org.apache.hadoop package.
Signed-off-by: Kevin Risden <krisden@apache.org>
(Julien Massenet, Diego Ceccarelli, Munendra S N, Christine Poerschke)
Resolved Conflicts:
lucene/grouping/src/java/org/apache/lucene/search/grouping/TopGroups.java
There are applications of ExpandComponent that intentionally do not
involve prior collapsing of results on the expand field, which can lead
to an NPE in expand component when expand.field (for matched docs) has
fewer unique values than the number of matched docs.
This commit refines the approach taken in SOLR-13877, which addressed
the same underlying issue.
* rename stdDev, variance methods to reflect the functionality
* add util functions to compute corrected stdDev and variance
* use DocValuesIterator#advanceExact to check if values exists for the doc
whoami displays a warning if the effective-uid is not in /etc/password.
This can happen in certain situations when running in a docker
container. This replaces the 'whoami' usage with a safer check.
The solr permissions are weak sauce due to the huge number of features, third-party dependencies, etc.
Hence they have access to do many things. For "scripting" such as velocity we have to look at a more aggressive stance:
Step 1: Can we wrap a sandbox around the whole goddamn thing and call it a day?
Step 2: Let's separate the "engine" from "untrusted code" and only be an asshole to the latter.
Step 3: Java's security is shit, Lets contain that classloader and whitelist access.
Ishan Chattopadhyaya
Kevin Risden
Dawid Weiss
Robert Muir
Joel Bernstein
Mikhail Khludnev
Erik Hatcher
Jason Gerlowski
Andrzej Bialecki
Ignacio Vera
Erick Erickson
Thomas Wöckinger
Vergil Cheynov
Christine Poerschke
Alessandro Benedetti
Houston Putman
Chris Hostetter
Adrien Grand
ngomx
Jan Høydahl
Daiki Tsuzuku
Munendra S N
Michael Gibney
Jan Høydahl