Commit Graph

8588 Commits

Author SHA1 Message Date
noblepaul 26bb6415d1 SOLR-14894: ASL header 2020-09-24 12:40:05 +10:00
noblepaul 565c5b1ac4 SOLR-14894: Use annotations to implement V2 collection APIs 2020-09-24 12:37:12 +10:00
noblepaul 1c9c1509fa SOLR-14890: syncing with 8x 2020-09-24 10:58:40 +10:00
Noble Paul fd0c08615d
SOLR-14890: Refactor code to use annotations for configset API (#1911) 2020-09-23 21:55:51 +10:00
David Smiley 7b53671920
SOLR-14768: Fix multipart POST to Solr. (#1838)
Regression from 8.6
Multipart POST would fail due to a NoClassDefFoundError of Jetty MultiPart.  Solr cannot access many Jetty classes, which is not noticeable in our tests.
2020-09-22 17:42:18 -04:00
David Smiley 2197776be6
SOLR-13181: param macro expansion could throw (#1877)
...StringIndexOutOfBoundsException on bad syntax
* failOnMissingParams: should have been returning null (failing) on bad syntax cases

Co-authored-by: Christine Poerschke <cpoerschke@apache.org>
2020-09-22 15:46:59 -04:00
Tomas Fernandez Lobbe 7b7a5a16ce
SOLR-10391: Add overwrite option to UPLOAD ConfigSet action (#1861)
When set to true, Solr will overwrite an existing configset in ZooKeeper in an UPLOAD.
A new cleanup parameter can also be passed to let Solr know what to do with the files that existed in the old configset, but no longer exist in the new configset (remove or keep)
2020-09-22 10:36:59 -07:00
Alexandre Rafalovitch 1611586417
SOLR-14880: Support coreRootDirectory setting when create new cores from command line, in standalone mode
Update to match latest code with shorter name
2020-09-22 09:02:14 -04:00
Alexandre Rafalovitch 91b6223f38
SOLR-14878: Expose coreRootDirectory via API (Correction to make name shorter and always return value) 2020-09-21 22:56:08 -04:00
noblepaul 4087958d31 SOLR-14884: TestContainerPlugin.testApiFromPackage jenkins failures 2020-09-22 11:20:49 +10:00
Tomas Fernandez Lobbe f1921b99b4
Improve TestConfigSetsAPI (#1892)
* Replace Auth plugin with mocks

* Remove unused password param

* Start cluster only once

* Use SolrCloudTestCase

* Use MiniSolrCloudCluster's methods to remove collections and configsets
2020-09-21 10:03:47 -07:00
Jason Gerlowski 9a6ca2dcd0 SOLR-14859: Set fieldType defaults for DateRangeField
AbstractSpatialPrefixTreeFieldType (and its children) create index
fields based on a prototype with options frozen in PrefixTreeStrategy,
regardless of options specified in the schema.  This works fine most of
the time, but causes problems when QParsers or other query optimization
logic makes decisions based on these options (which are potentially out
of sync with the underlying index data).  Most commonly this causes
issues with "exists" (e.g. [* TO *]) queries.

This commit enforces fieldType defaults that line up with the 'hardcoded'
FieldType used by PrefixTreeStrategy.  Options on either the fieldType
or the field itself which contradict these defaults will result in
exceptions at schema load/modification time.
2020-09-21 12:57:08 -04:00
Alexandre Rafalovitch da2714cb86
SOLR-9607: Finalize move of Terms component and request handler into the implicit definitions (#1897)
* Updated implicit definition with terms=true, distrib=false
* Commented out terms handler with notice, as this is the config used in tests
* Remove spurious mentions cluttering other test configs
* Remove implicit terms=true param
* Remove definitions from shipped configsets
* Improve documentation
* Add CHANGES record
2020-09-21 09:05:37 -04:00
noblepaul 73c06e35e8 SOLR-14881: ManagedSchemaRoundRobinCloudTest test failures 2020-09-21 15:08:23 +10:00
noblepaul 64ca559042 SOLR-14879: address TestPackages test failures 2020-09-21 14:55:54 +10:00
Tom Edge fa756b1c31
SOLR-14802: geodist: Support most spatial field types as an arg
FunctionQParser: overload parseValueSourceList with flags
2020-09-20 23:19:48 -04:00
Noble Paul 3664182d24
SOLR-14879: address TestPackages test failures (#1895) 2020-09-20 13:01:47 +10:00
Alexandre Rafalovitch f6e8bc8c31
SOLR-14878: Expose solr.xml's coreRootDirectory property via the System Settings API (#1894) 2020-09-19 22:07:03 -04:00
Dawid Weiss 5ec2bac91c
LUCENE-9531: Consolidate duplicated generated classes CharStream and FastCharStream (#1886) 2020-09-18 08:53:30 +02:00
Noble Paul ee0a374bb8
SOLR-14875: Make SolrEventListeners load from packages (#1887) 2020-09-18 12:07:29 +10:00
Ilan Ginzburg dbba48b3e5
SOLR-14613: use set-placement-plugin for both setting and unsetting plugin config 2020-09-17 15:01:19 +02:00
Dawid Weiss 6c9d7adf79
LUCENE-9527: upgrade javacc to 7.0.4 (#1884) 2020-09-17 13:29:18 +02:00
noblepaul 515608a087 SOLR-14151: fixed the classloading issue 2020-09-17 19:10:04 +10:00
Dawid Weiss 4f344cb0d4
LUCENE-9530: cleaned up javacc gradle generation scripts. (#1883)
* LUCENE-9530: cleaned up gradle javacc generation/ tweaks script so that it's consistent across runs. Removed ant remnants.
2020-09-17 10:53:02 +02:00
noblepaul cbb1659640 Revert "Revert "SOLR-14151: Bug fixes (#1815)""
This reverts commit 27a14fe481.

Undoing accidental commit
2020-09-17 11:19:21 +10:00
noblepaul e5d3e4d567 Merge branch 'master' of github.com:apache/lucene-solr 2020-09-17 09:06:48 +10:00
Ilan Ginzburg c7d234cafd
SOLR-14613: Autoscaling replacement using placement plugins
Allow using placement plugins to compute replica placement on the cluster for Collection API calls.
This is the first code drop for the replacement of the Autoscaling feature.
Javadoc of sample plugin org.apache.solr.cluster.placement.plugins.SamplePluginAffinityReplicaPlacement details how to enable this replica placement strategy.
PR's #1684 then #1845
2020-09-17 00:10:31 +02:00
Erik Hatcher 22022463d7 SOLR-14799: JWT authentication plugin only requires sub claim when principalClaim=sub 2020-09-16 13:41:19 -04:00
Erik Hatcher c63684f93b Revert "SOLR-14799: JWT authentication plugin only requires sub claim when principalClaim=sub"
This reverts commit bc0c9ffee3.
2020-09-16 12:45:03 -04:00
Erik Hatcher bc0c9ffee3 SOLR-14799: JWT authentication plugin only requires sub claim when principalClaim=sub 2020-09-16 12:27:55 -04:00
noblepaul 6b87cfb88c Merge branch 'master' of github.com:apache/lucene-solr 2020-09-16 19:19:45 +10:00
Noble Paul 7b8e72e553
SOLR-14871 Use Annotations for v2 APIs in/cluster path (#1878) 2020-09-16 18:06:43 +10:00
noblepaul a8099d6367 Merge branch 'master' of github.com:apache/lucene-solr 2020-09-14 18:47:20 +10:00
noblepaul cc31e23341 SOLR-14151: Fixing TestBulkSchemaConcurrent failures 2020-09-14 15:42:11 +10:00
noblepaul 27a14fe481 Revert "SOLR-14151: Bug fixes (#1815)"
This reverts commit 95ab98c920.

revert the previous change
2020-09-14 12:22:58 +10:00
Dawid Weiss ed930f4b29
SOLR-14417: Gradle build sometimes fails RE BlockPoolSlice (#1854)
* SOLR-14417: workaround the compiler scope problem.

* Make the list modifiable.
2020-09-10 22:25:33 +02:00
Tomas Fernandez Lobbe 0250978869
SOLR-14510: Remove deprecations added with BlockMax WAND support (#1823) 2020-09-10 10:33:09 -07:00
Tomas Fernandez Lobbe 2f86255b94 SOLR-14849: Add explicit sort order in testMinExactCount 2020-09-09 14:03:37 -07:00
Mike Drob c902837bb2
SOLR-14846 Clean up Optional use (#1843)
* Remove Optional.ofNullable.orElse pattern
* Remove use of Optional as method parameter
2020-09-09 09:19:48 -05:00
Christine Poerschke f46515ee6b
Rename TestDirectoryFactory to DirectoryFactoriesTest (which is different from DirectoryFactoryTest). (#1790) 2020-09-09 13:08:20 +01:00
Mike Drob 984466f31b
SOLR-14770 Avoid reregistering JVM Guage as well (#1469) 2020-09-08 10:18:24 -05:00
Noble Paul 95ab98c920
SOLR-14151: Bug fixes (#1815) 2020-09-08 09:58:42 +10:00
Jason Gerlowski ecb8ca2fef SOLR-14821: {!terms} dVTFTL supports single-valued strings
Prior to this commit the docValuesTermsFilterTopLevel method of the
{!terms} query parser would return zero results when run against a
single-valued String.  This commit fixes this by wrapping the
single-valued 'SortedDocValues' in a 'SortedSetDocValues' object.
2020-09-05 13:54:27 -04:00
Erick Erickson f71ba62d4a SOLR-14579: Comment SolrJ 'Utils' generic map functions 2020-09-05 09:19:45 -04:00
Andy Vuong 2544df8f6d
SOLR-14658: SolrJ collectionStatus(col) should only fetch one status (#1687)
An optimization or a perf-bug depending on point of view
2020-09-03 14:25:36 -04:00
Chris Hostetter a70a47d053 SOLR-14383: Fix nested indexing ref-guide documentation and corrisponding search examples
Squashed commit of branch jira/SOLR-14383 as of...

commit 34da84c238
Merge: 5d293276a3 8bd79ec301
Author: Chris Hostetter <hossman@apache.org>
Date:   Tue Sep 1 16:39:03 2020 -0700

    Merge branch 'master' into jira/SOLR-14383
2020-09-02 14:16:23 -07:00
Robert Muir 784ede4eda
LUCENE-9215: replace checkJavaDocs.py with doclet (#1802)
This has the same logic as the previous python, but no longer relies
upon parsing HTML output, instead using java's doclet processor.

The errors are reported like "normal" javadoc errors with source file
name and line number and happen when running "gradlew javadoc"

Although the "rules" are the same as the previous python, the python had
some bugs where the checker didn't quite do exactly what we wanted, so
some fixes were applied throughout.

Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
Co-authored-by: Uwe Schindler <uschindler@apache.org>
2020-09-02 08:29:17 -04:00
Ilan Ginzburg 825c18c59b
SOLR-14794: revert commit passing CloudConfig everywhere (a3b3ba10e3) and remove tab in solr/bin/solr.cmd 2020-09-01 17:21:12 +02:00
noblepaul 4a68cf18b6 SOLR-14151: move the SolrClassLoader interface to SolrJ 2020-09-01 10:23:15 +10:00
David Smiley 06903a40ee
LUCENE-9456: fix DirectUpdateHandlerTest#testPrepareCommit (#1803)
Check for specific files being present or not or changing.  Don't make assumptions about file count.
2020-08-30 11:32:56 -04:00
Ilan Ginzburg a3b3ba10e3
SOLR-14794: pass configuration to Collection API commands (#1801)
Pass CloudConfig instance representing the solrcloud section of solr.xml configuration from Overseer to the Collection and Config Set API commands it executes.
2020-08-30 11:36:28 +02:00
Alexandre Rafalovitch a57ba25400
SOLR-14783: Remove DIH from 9.0 (#1794)
* Remove DIH example directory

* Remove contrib code directories

* Remove contrib package related configurations for build tools

* Remove mention of DIH example

* remove dih as build dependencies and no-longer needed version pins

* Remove README references to DIH

* Remove dih mention from the script that probably does need to exist at all

* More build artifact references

* More removed dependencies leftovers (licenses/versions)

* No need to smoke exclude DIH anymore

* Remove Admin UI's DIH integration

* Remove DIH from shortname package list

* Remove unused DIH (related? not?) dataset
Unclear what is happening here, but there is no reference to that directory anywhere else
The other parallel directories ARE referenced in a TestConfigSetsAPI.java

* Hidden Idea files references

* No DIH to ignore anymore

* Remove last Derby DB references

* Remove DIH from documentation
Add the information in Major Changes document with the link to the external repo

* Added/updated a mention to CHANGES

* Fix leftover library mentions

* Fix Spellings
2020-08-29 10:52:04 -04:00
noblepaul ec27b75232 use proper version String 2020-08-29 18:18:41 +10:00
Tomas Fernandez Lobbe 59d087f0b3
SOLR-14774: Create HealthCheckHandler in CoreContainer (#1774)
This commit does two things:
* Allow users to plug-in different implementations of the handler (they must extend HealthCheckHandler)
* Remove the HealthCheckHandler from the implicit SolrCore plugins
2020-08-28 15:59:56 -07:00
Ishan Chattopadhyaya d84977eb5c SOLR-14616: Remove CDCR 2020-08-29 02:13:13 +05:30
Erick Erickson 69fa5a00fb LUCENE-9433: Remove Ant support from trunk 2020-08-28 09:31:16 -04:00
Atri Sharma b381595f55 Fix NPE in CB Config Resolution 2020-08-27 20:52:12 +05:30
Atri Sharma 6a7da3cd50
Update Circuit Breaker configured as a standard plugin (#1785) 2020-08-27 14:06:22 +05:30
Adrien Grand 741251b1c3 LUCENE-9447: Fix test failure. 2020-08-26 12:55:46 +02:00
Adrien Grand 913976dbf7
LUCENE-9447: Make BEST_COMPRESSION better with highly compressible data. (#1762)
This makes BEST_COMPRESSION split blocks into sub blocks and use preset
dictionaries to improve compression ratios.
2020-08-26 11:04:34 +02:00
Philipp Trulson f4c4fbcafc SOLR-14752: Fix error in Zookeeper status when Prometheus plugin is enabled in ZK
Signed-off-by: Jan Høydahl <janhoy@apache.org>
2020-08-26 10:03:42 +02:00
Simon Willnauer 8294e1ae20
LUCENE-8962: Merge segments on getReader (#1623)
Add IndexWriter merge-on-refresh feature to selectively merge 
small segments on getReader, subject to a configurable timeout, 
to improve search performance by reducing the number of small 
segments for searching.

Co-authored-by: Mike McCandless <mikemccand@apache.org>
2020-08-24 20:19:08 +02:00
Atri Sharma 96a853b200
SOLR-14772: Fix TestConfigOverlay (#1778) 2020-08-24 22:34:23 +05:30
noblepaul 1c8f4c988a SOLR-14750: use core id to avoid duplicate core reloads 2020-08-22 17:53:55 +10:00
Erick Erickson c9c75810c2 Revert "LUCENE-9433: Remove Ant support from trunk"
This reverts commit 37cd17dc
2020-08-21 16:57:58 -04:00
Anderson Dorow 66b6ce2cd0
SOLR-13438: on collection delete, also delete .AUTOCREATED config set (#1759)
Co-authored-by: Anderson Dorow <Anderson.Dorow@zooplus.com>
2020-08-21 14:43:16 -05:00
Erick Erickson 37cd17dcf5 LUCENE-9433: Remove Ant support from trunk 2020-08-21 15:19:52 -04:00
Atri Sharma 2f37f40171
SOLR-14615: Implement CPU Utilization Based Circuit Breaker (#1737)
This commit introduces CPU based circuit breaker. This circuit breaker
tracks the average CPU load per minute and triggers if the value exceeds
a configurable value.

This commit also adds a specific control flag for Memory Circuit Breaker
to allow enabling/disabling the same.
2020-08-20 13:21:26 +05:30
Noble Paul 8caf57d50b
SOLR-14750: TestBulkSchemaConcurrent fails often(#1760) 2020-08-19 10:52:12 +10:00
Ishan Chattopadhyaya 77a4d495cc SOLR-14731: Rename @SolrSingleThreaded to @SolrThreadUnsafe, mark DistribPackageStore with the annotation
Co-authored-by: Marcus <marcuseagan@gmail.com>
2020-08-18 16:21:38 +05:30
Atri Sharma 4fed9ff8d3 Move TestRequestRateLimiter to Nightly 2020-08-18 13:43:49 +05:30
Andrzej Bialecki d9fb14dec1 SOLR-13858: Remove no longer used field and getter. 2020-08-17 14:47:38 +02:00
David Smiley 0ce2d610c2
SOLR-14722: timeAllowed should track from req creation (#1726)
* set(long) instead of set(Long).
* Fix javadocs CommonParams.TIME_ALLOWED
2020-08-15 00:41:07 -04:00
Yuriy Koval 44c4e6ef31
SOLR-14703 Edismax parser replaces whitespace characters with spaces (#1713) 2020-08-14 14:12:53 -04:00
Noble Paul d517361bb1
SOLR-14680: Provide an implementation for the new SolrCluster API (#1730) 2020-08-13 13:34:24 +10:00
Jason Gerlowski a6515ca38f SOLR-14748: Correct condition on startup auth/ssl logging 2020-08-12 18:44:02 -04:00
andywebb1975 e72a0d66c6
SOLR-13751: add BooleanSimilarityFactory (#867) 2020-08-12 15:35:50 +01:00
Andrzej Bialecki a5543dfb51 SOLR-14470: Fix test failures by reducing the randomness of test data. 2020-08-12 13:45:25 +02:00
Atri Sharma 1d2749295b
Harden RequestRateLimiter Tests (#1736)
* Harden RequestRateLimiter Tests

This commit adds higher data size and load in the test path. Also improves
the asserts that are performed.
2020-08-11 22:42:53 +05:30
Jason Gerlowski 5887032e95
SOLR-14692: Allow 'method' specification on JSON Facet join domain transforms (#1707) 2020-08-11 08:21:07 -04:00
Cao Manh Dat 57b0160659
SOLR-14641: PeerSync, remove canHandleVersionRanges check (#1663) 2020-08-10 15:04:34 +07:00
Tomas Fernandez Lobbe bd1dd6bdfb SOLR-14582: Fix TestConfig 2020-08-07 16:01:11 -07:00
Gus Heck f827286a2c
SOLR-14582 - Broken test needs fix (#1727)
AwaitsFix for ConfigTest#testDefaults()
2020-08-07 18:49:04 -04:00
Atri Sharma a074418da0
SOLR-13528: Implement Request Rate Limiters (#1686)
This commit introduces two functionalities: request rate limiting and ability to identify requests based on type (indexing, search, admin). The default rate limiter rate limits query requests based on configurable parameters which can be set in web.xml. Note that this rate limiting works at a JVM level, not a core/collection level.
2020-08-08 01:36:06 +05:30
Tomas Fernandez Lobbe e6275d9970
SOLR-14582: Expose IWC.setMaxCommitMergeWaitMillis in Solr's index config (#1602) 2020-08-07 10:54:36 -07:00
noblepaul 321c4bbe1f SOLR-14654: clean up ref guide, CHANGES.txt 2020-08-06 16:46:12 +10:00
Noble Paul 6f2f1c3de2
SOLR-14654 Remove plugin loading from .system collection (for 9.0) (#1677) 2020-08-06 16:32:32 +10:00
Tomas Fernandez Lobbe 6752111be8 SOLR-14702: doFetch reads legacy parameters too 2020-08-05 14:53:50 -07:00
Marcus 7110118ad4
SOLR-14702: Remove oppressive language (part1) (#1711)
Replace references to "master" and "slave" with "leader" and "follower"
2020-08-05 14:52:48 -07:00
Ishan Chattopadhyaya bd21da6eca SOLR-14604: Add the ability to uninstall a package from with the Package CLI (#1710)
Co-authored-by: Marcus <marcuseagan@gmail.com>
2020-08-04 14:53:25 +05:30
Chris Hostetter a6c058a260 SOLR-14657: Improve error handling in IndexReader realted metrics that were causing scary ERROR logging if metrics were requested while Solr was in the process of closing/re-opening a new IndexReader 2020-08-03 09:58:16 -07:00
Munendra S N 07a89e7bf6 SOLR-14516: fix NPE is resp writer while writing docvalue only field
This issue occurs only while fetching uncommitted doc through /get.
Instead of directly calling stringValue() on IndexableField use
FieldType's toExtern() or toObject() to get the writable value for the field.
2020-07-31 20:05:41 +05:30
Marcus f443edebd2
SOLR-14681: Introduce ability to delete .jar stored in the Package Store (#1702) 2020-07-31 18:23:18 +10:00
Christine Poerschke 2f155aa0c3 SOLR-14489: remove ReplicationHandler's deprecated "master.commitReserveDuration" use (in favour of "commitReserveDuration") 2020-07-29 10:23:17 +01:00
Jan Høydahl ebb5219b1b
SOLR-14671: Parsing dynamic ZK config sometimes cause NuberFormatException (#1701) 2020-07-29 10:33:02 +02:00
David Smiley a3624029ad
SOLR-14651: Metrics History could disable better (#1672)
* SolrRrdBackendFactory should not be created if history is disabled
* Disable MetricsHistoryHandler by default in tests
* Await shutdown of all executors
2020-07-28 16:46:27 -04:00
Erick Erickson 6bf5f4a87f SOLR-11868: CloudSolrClient.setIdField is confusing, it's really the routing field. Should be deprecated. 2020-07-26 18:18:05 -04:00
David Smiley 5295007022
SOLR-14652: SolrCore should hold its own CoreDescriptor (#1675)
(minor refactoring)
Also:
* SolrCore's constructors don't need a "name" since it's guaranteed to always be the name in the coreDescriptor.  I checked.
* SolrCore's constructor shouldn't call coreContainer.solrCores.addCoreDescriptor(cd); because it's the container's responsibility to manage such things.  I made SolrCores.putCore ensure the descriptor is added, and this is called by CoreContainer.registerCore which is called after new SolrCore instances are created.
* solrCore.setName should only be called when we expect the name to change.  Furthermore that shouldn't ever happen in SolrCloud so I added checks.
* solrCore.setName calls coreMetricManager.afterCoreSetName() which is something that is really only related to a rename, not name initialization (from the constructor).  I renamed that method and further only call it if the name did change from non-null.
2020-07-25 01:08:23 -04:00
Erick Erickson 4b2e90b3aa SOLR-11656: TLOG replication doesn't work properly after rebalancing leaders. 2020-07-24 13:10:35 -04:00
Atri Sharma ec007c4bf2
Use FileSystem.newInstance instead of FileSystem.get in HDFSBackupRepository (#1682)
FileSystem.get can cause FileSystem closed exceptions, especially with S3.
2020-07-24 09:57:58 +05:30