Commit Graph

8062 Commits

Author SHA1 Message Date
yonik c0da27173c SOLR-13399: annotate flaky test 2019-07-24 21:58:50 -04:00
noble 85814e262c SOLR-13637: addressing test failures take 2 2019-07-23 12:17:49 -04:00
Andrzej Bialecki d92159b896 SOLR-13558: Allow dynamic resizing of SolrCache-s. 2019-07-23 12:21:21 +02:00
noble 7f04604d56 SOLR-13637: reject duplicate jar updates 2019-07-22 16:23:44 -04:00
noble 2ef0ba3dce SOLR-13637: addressing test failures 2019-07-22 08:33:02 -07:00
Noble Paul 631edee1cb
SOLR-13637 Enable loading of plugins from the corecontainer memclassloader 2019-07-21 12:21:56 -07:00
noble 7d716f1107 SOLR-13565: Precommit errors 2019-07-22 03:31:39 +10:00
Mikhail Khludnev c173c2e863 SOLR-11556: fixing multiple backup repository support. 2019-07-21 13:04:48 +03:00
Munendra S N 96d400f511 SOLR-12870: use StandardCharsets instead of String values
Fixes #469
2019-07-20 10:10:32 +05:30
Noble Paul bd30c3701e
SOLR-13565, SOLR-13553 (#774)
* SOLR-13565: initial commit

* SOLR-13565: updated with testcase

* SOLR-13565: removed unused methods

* SOLR-13565: better logging

* SOLR-13565: disable SSL

* SOLR-13565: more tests

* SOLR-13565: syncing with master

* SOLR-13565: fixing tests

* SOLR-13565: fixing tests

* SOLR-13534: Fix test

Remove buggy 'port roulette' code that can easily fail if OS gives the selected port to a different process just before creating the server

Use jetty's built in support for listining on an OS selected port instead

Also increase timeouts to better account for slow/heavily loaded (ie:jenkins) VMs where SolrCore reloading may take longer then 10 seconds

* SOLR-13565: set proper permission name

* SOLR-13565: syncing with master

* SOLR-13565: syncing with master

* SOLR-13565: removed accidental change

* SOLR-13565: removed accidental change

* SOLR-13565: removed accidental change

* SOLR-13565: more tests

* SOLR-13565: Tests with key signing tests

* SOLR-13565: fixing concurrency issues in tests

* SOLR-13565: add tests with 512 bit RSA

* SOLR-13565: fixing concurrency issues

* SOLR-13565: remove unused code
2019-07-20 14:20:32 +10:00
yonik c5cabf1e03 SOLR-13399: SPLITSHARD splitByPrefix for compositeId 2019-07-19 11:35:01 -04:00
Gus Heck a7435d9e6e SOLR-13375 - fix test. Can't shuffle the to random collections at
this point in the test since the collections available are changing due
to deletions and we might try to communicate with a collection
that was (correctly) deleted.
2019-07-19 10:00:53 -04:00
Christine Poerschke 5e3947091a SOLR-13585: Factor out SearchGroupsResultTransformer.[de]serializeOneSearchGroup methods. (Christine Poerschke, Diego Ceccarelli) 2019-07-18 14:23:32 +01:00
Munendra S N 1fc416404c SOLR-13206: Fix AIOOBE when group.facet is specified with group.query
group.facet is supported only for group.field. When group.facet is
used with group.query, then return proper error code
2019-07-18 11:08:17 +05:30
Munendra S N 4b75776f5a SOLR-13634:move ResponseBuilderTest to same package as ResponseBuilder 2019-07-17 22:38:35 +05:30
Munendra S N 1ecd02deb5 SOLR-12368: inplace update for field that doesn't yet exist in any doc
If the field is non-stored, non-indexed and docvalue enabled numeric field
then inplace update can be done. previously, lucene didn't support
docvalue update for field that is not yet present in indexWriter but
LUCENE-8316 added support for this.
This adds support to update field which satisfies inplace conditions
but which doesn't yet exist in any docs
2019-07-17 21:45:15 +05:30
Chris Hostetter 19c78ddf98 SOLR-13534: Fix test
Remove buggy 'port roulette' code that can easily fail if OS gives the selected port to a different process just before creating the server

Use jetty's built in support for listining on an OS selected port instead

Also increase timeouts to better account for slow/heavily loaded (ie:jenkins) VMs where SolrCore reloading may take longer then 10 seconds
2019-07-16 14:34:23 -07:00
noble 2d357c960c SOLR-13534: Make the test more robust 2019-07-16 11:53:13 +10:00
Gus Heck c97551cc36 SOLR-13375 - 2 dimensional routed aliases 2019-07-15 09:57:13 -04:00
Ishan Chattopadhyaya 26ede632e6 SOLR-13619: Kerberos plugin to forward original user principal 2019-07-15 15:10:07 +05:30
Ishan Chattopadhyaya 7e0af71c1e SOLR-13472: Forwarded requests should skip authorization on receiving nodes 2019-07-15 14:43:41 +05:30
Chris Hostetter 7ddba3b712 Harden DeleteReplicaTest
* tighten assertions related to type of watcher that should be removed

* use waitForActiveCollection before deleting collections to work around SOLR-13616 and/or SOLR-13627
2019-07-13 08:02:49 -07:00
Chris Hostetter f85a78c441 SOLR-13532: Fix http timeout and error logging bugs in RecoveryStrategy 2019-07-11 14:04:46 -07:00
Chris Hostetter 8a277cab7d Harden TestPolicyCloud
- ensure all collections/replicas are active

- tighten assertions around expected replica locations

- eliminate some redundent code

These changes should also help ensure we don't get (more) spurious failures due to SOLR-13616
2019-07-10 17:16:59 -07:00
Munendra S N 8322ea55f4 SOLR-11263: add test for payload func with undefined field
Payload func with undefined used to throw NPE. In SOLR-11610, this
was fixed to return proper error but there are no tests to verify
changed the behavior.
This add simple test to verify error code and error message
2019-07-10 10:17:55 +05:30
Christine Poerschke bf9a7e2626 Fix two 'against' typos. 2019-07-09 10:56:49 +01:00
Andrzej Bialecki dd4813d5b8 SOLR-13583: Return 400 Bad Request instead of 500 Server Error when a complex
alias is found but a simple alias was expected.
2019-07-05 09:19:32 +02:00
S N Munendra 5f1d342a96 SOLR-13538: Fix classcastEx in TrieDateField for atomic updates (#764)
SOLR-13538: toNativeType () TrieDate & EnumField do not handle CharSequence properly
2019-07-05 14:43:12 +10:00
Anshum Gupta b7090d9c25
SOLR-13507: Remove support for addr parameter from the /solr/admin/zookeeper endpoint. (#759) 2019-07-03 10:50:01 -07:00
Munendra S N 96860eb181 SOLR-13596: remove deprecated groupSpec methods 2019-07-03 20:55:56 +05:30
Tomoko Uchida 2e3e6c5944 SOLR-13602: Update _default schema for unittests. 2019-07-04 00:08:13 +09:00
Andrzej Bialecki 5897787291 SOLR-13003: Query Result Cache does not honour maxRamBytes parameter. 2019-07-03 17:03:43 +02:00
Munendra S N e3d247f288 SOLR-13603: remove deprecated groupSpec methods usage
This is precursor to removing deprecated methods
2019-07-03 19:33:08 +05:30
Andrzej Bialecki 717afb11fc SOLR-13583: Impossible to delete a collection with the same name as an existing alias. 2019-07-03 15:52:36 +02:00
David Smiley 325824cd39 SOLR-13158: DIH: Add System property toggle for use of dataConfig param 2019-07-02 23:59:39 -04:00
Chris Hostetter b4a602f6b2 SOLR-13599: additional 'checkpoint' logging to try and help diagnose strange failures 2019-07-02 14:52:13 -07:00
Atri Sharma f686167e71 LUCENE-8857: Introduce Custom Tiebreakers in TopDocs#merge (#757)
This commit introduces custom tiebreakers which allows users to
specify custom tiebreakers when ordering hits to return. A
default tiebreaker is introduced for tie breaking on shard index
first and then docID.
2019-07-02 20:35:18 +02:00
Munendra S N 48b026d5ee SOLR-13404: Fix NPE when group=true and no group.field is present
* This was introduced in SOLR-12249
2019-07-02 10:27:54 +05:30
Munendra S N cfd22cd493 SOLR-13404: support group.query in multishard env with group.main=true
group.query after execution forms QueryCommandResult. In case of
group.main=true or group.format=simple, QueryCommandResult was not
consumed in EndResultTransformer. Also, MainEndResultTransformer assumed
that always group.field would be specified. When group.field not specified
it failed with AIOOBE. After adding suppport for QueryCommandResult in
EndResultTransformers and handling AIOOBE, group.query started giving results

Working on tests exposed few other issues. Results differed b/w standalone
& distributed mode.
* One of the reason is that TopGroupShardResponseProcessor doesn't consider correct
  limit and offset when group format is simple. In case of simple, start and rows should be used
  as limit and offset instead of group.limit and group.offset.
* Secondly, In distributed second phase grouping, computing docsToCollect didn't consider
  group response format. This issue is again similar to above issue
* offset(group.offset or start) not being considered during TopDocs#merge caused
  different results. The fix was to use to offset in merge process
* group.offset doesn't support negative values but there is no checks on the value.
  In case of negative values AIOOBE. Now, checks are added for negative values and
  returns proper error message(this change is for both standalone and distrbuted).
  Validation is done only in case of group.format=grouped as that is only case when
  group.offset is consumed.

Fixing above issues resolved the differences b/w standalone and distributed mode.
2019-07-02 08:11:13 +05:30
Munendra S N 0e877aac34 SOLR-12554: Expose IndexWriterConfig's ramPerThreadHardLimitMB
* When ramPerThreadHardLimitMB is not specified, then Lucene's
  default value 1945 is used. The specified value should be
  greater than 0 and less than 2048MB
2019-07-01 22:42:19 +05:30
Jason Gerlowski 8242e6ce1d SOLR-13539: Fix mv update of UUID, enum, bool and binary fields
Co-Authored-By: Thomas Wockinger
2019-07-01 08:53:54 -04:00
Mikhail Khludnev 2fdb4dd7d4 SOLR-13577: wait jetty to stop as @hossman advised. 2019-06-30 09:45:35 +03:00
Munendra S N cfd6e1adfa SOLR-12364: add test cases for edismax boost
* This adds tests for bf and boosts
* Use expectThrows in edismax tests to verify exception
2019-06-29 22:18:02 +05:30
Munendra S N 9228cefbc8 SOLR-9409: improve error message on unsupported types in collapsing
* Improve error message when collapsing is not supported on given
  fieldtype
* Return 400 error code when unsupported value are passed for max,min
  or in case of syntax error
2019-06-29 21:37:09 +05:30
Chris Hostetter 8b72e91df7 SOLR-13580: update test to account for different versions of java using different locale specific numeric formatting characters 2019-06-28 12:10:54 -07:00
Christine Poerschke a49ddbaf11 SOLR-13576: Factor out a TopGroupsShardResponseProcessor.fillResultIds method. (Christine Poerschke, Diego Ceccarelli) 2019-06-28 18:38:41 +01:00
Christine Poerschke 5d2569eab1 SOLR-13280: Strengthen ScheduledTrigger's preferredOperation parameter validation. 2019-06-28 18:35:34 +01:00
Christine Poerschke 993c051a0e SOLR-13279: Clarify ScheduledTrigger's "every parameter missing" error response. 2019-06-28 18:32:25 +01:00
Mikhail Khludnev d54555c757 SOLR-13577: spin until slave got a replication failure while master is down. 2019-06-28 17:28:48 +03:00
Chris Hostetter aaf20aefa4 SOLR-12988: SSLTestConfig has been changed to throw AssumptionViolatedException when tests/seeds request SSL but the JVM appears to be an OpenJDK version known to have SSL bugs
SOLR-13574: Add CHANGES entry that was overlooked
2019-06-27 15:51:20 -07:00
Chris Hostetter 7e57d3a9d9 SOLR-13574: Fix many test and test-framework classes to not fail on After/AfterClass cleanup if assumptions fail in Before/BeforeClass setup 2019-06-27 10:28:06 -07:00
Mikhail Khludnev 48fb45e614 SOLR-13577: retrying TestReplicationHandler.doTestIndexFetchOnMasterRestart. 2019-06-26 14:16:22 +03:00
Jan Høydahl b54126169b SOLR-13569: AdminUI visual indication of prod/test/dev environment 2019-06-26 12:09:02 +02:00
Alan Woodward 53f56fb7ad LUCENE-8811: Move max clause checks to IndexSearcher 2019-06-26 10:55:55 +01:00
Chris Hostetter 583c219183 SOLR-13580: add assumeThat calls to ParsingFieldUpdateProcessorsTest to skip test methods impacted by java 13-ea bug in NumberFormat.parse() 2019-06-25 16:26:03 -07:00
Andrzej Bialecki 4d1058db6e SOLR-13566: REINDEXCOLLECTION does not work with (basic) authentication. 2019-06-25 16:54:27 +02:00
Noble Paul 742c80550c reformatting 2019-06-25 17:02:17 +10:00
Mikhail Khludnev 4589bbe47b SOLR-12866: Turn TestHdfsCloudBackupRestore ON. No changes yet. 2019-06-25 08:49:26 +03:00
Munendra S N e0e5296abc SOLR-12979: fail fast when collapse field is non-docValued & non-uninvertible
* Improve error message when collapse field is non-docValued & non-uninvertible.
  Return error code 400 instead of 500 in the above case
2019-06-25 09:50:41 +05:30
David Smiley 85ec39d931 SOLR-13367: Range queries will now highlight in hl.method=unified mode.
Lucene MatchesUtils.disjunction method for disjunction over
 BytesRefIterator terms.
2019-06-25 00:10:08 -04:00
Chris Hostetter 689fa583a0 Revert "SOLR-12988: SSLTestConfig has been changed to throw AssumptionViolatedException when tests/seeds request SSL but the JVM appears to be an OpenJDK version known to have SSL bugs"
This reverts commit 6d6f14d391.

Reason for revert: after doing more testing I realized there are tests I overlooked which can (with randomized SSL usage) trigger NullPointerException
(or other related failures) in After/AfterClass due assumptions about cleanup that isn't actaully neccessary due to the AssumptionFailure
that may occur during Before/BeforeClass
2019-06-24 15:42:23 -07:00
Chris Hostetter 6d6f14d391 SOLR-12988: SSLTestConfig has been changed to throw AssumptionViolatedException when tests/seeds request SSL but the JVM appears to be an OpenJDK version known to have SSL bugs 2019-06-24 15:02:18 -07:00
Munendra S N 9cfba4a728 SOLR-13187: Fix NPE when invalid qParser is specified
* When non-existent qParser is specified return 400 error code
* SOLR-13197: Fix NPE when createQParser is called in StatsField
2019-06-24 22:38:39 +05:30
Munendra S N 54aff4ac7d SOLR-12127: set op with null or empty list val should be atomic update
* Inplace update supports set and inc operation but when null or
  empty list is specified with set op, then it should always be treated
  as atomic update since this case is equivalent to removing field
  from the document
2019-06-24 14:50:51 +05:30
Tomoko Uchida 98c85a0e1a LUCENE-8778: Define analyzer SPI names as static final fields and document the names in all analysis components. This also changes SPI loader to detect service names via the static NAME fields instead of class names. 2019-06-22 10:46:37 +09:00
David Smiley 54cc70127b LUCENE-8848 LUCENE-7757 LUCENE-8492: UnifiedHighlighter.hasUnrecognizedQuery
The UH now detects that parts of the query are not understood by it.
When found, it highlights more safely/reliably.
Fixes compatibility with complex and surround query parsers.
2019-06-21 17:05:56 -04:00
David Smiley 54c5b8a7f1 SOLR-13523: Fix Atomic Updates when _nest_path_ is declared.
Change the most common test schema to include this field so we better
test our code paths.
2019-06-20 11:59:22 -04:00
Gus Heck 5d47875184 SOLR-13419 - Better infix for TRA collection names 2019-06-20 09:47:52 -04:00
Cao Manh Dat e3752e87d0 SOLR-12988: Revert changes 2019-06-19 21:07:38 +01:00
Cao Manh Dat 91944a468e SOLR-12988: Skip running tests with SSL on Java 11 to 11.0.2 2019-06-19 10:43:17 +01:00
Noble Paul ed42d36c7b SOLR-13329: changed the put:on-each to put: on-each-node 2019-06-19 00:16:57 +10:00
Cao Manh Dat c8382890ae SOLR-12988: Avoid using TLSv1.3 for HttpClient 2019-06-18 12:49:51 +01:00
Cao Manh Dat 6d5453d508 Fix precommit 2019-06-18 12:36:43 +01:00
Daniel Collins 342b558b01 SOLR-13434: Fix opentracing mock dependency for Solr core tests 2019-06-18 11:37:31 +01:00
Mikhail Khludnev 4f6314c59b SOLR-7530: /terms responds per field arrays in JSON by default 2019-06-18 08:38:02 +02:00
Chris Hostetter 3030ea9d94 eliminate CPU hogging spin-loops in OverseerTest
this test already uses waitForState (frequently via verifyReplicaStatus) so there is no reason to include CPU/network/ZK intensive infinite loop checks looking for udpated cluster state
2019-06-17 15:42:14 -07:00
Chris Hostetter 7eb8703df6 SOLR-13490: fix TestWaitForStateWithJettyShutdowns to use correct (randomized) JettyConfig 2019-06-17 15:34:14 -07:00
Chris Hostetter 5a974860fa SOLR-13490: Fix CollectionStateWatcher/CollectionStatePredicate based APIs in ZkStateReader and CloudSolrClient to be triggered on liveNode changes.
Also add Predicate<DocCollection> equivilents for callers that don't care about liveNodes.
2019-06-17 09:59:43 -07:00
Noble Paul d514beab94
SOLR-13534: Dynamic loading of jars from a remote url (#712)
* SOLR-13534 : Dynamic loading to support loading jars from a URL
2019-06-17 16:49:31 +10:00
Gus Heck 5d550a34a9 SOLR-13420 Routed Aliases now use collection properties instead of core properties 2019-06-14 13:51:16 -04:00
Mikhail Khludnev 2ee0b16bf1 SOLR-13333: make terms.ttf work without terms.list in standalone mode 2019-06-14 15:42:22 +02:00
Christine Poerschke 335aaf801e SOLR-13515: remove SolrPluginUtils.IdentityRegenerator in favour of NoOpRegenerator 2019-06-14 13:10:24 +01:00
Christine Poerschke 43fc05c1ed SOLR-13511: Add SearchHandler.newResponseBuilder method to facilitate custom plugins' maintenance of per-request state in a custom ResponseBuilder. (Ramsey Haddad, Christine Poerschke) 2019-06-14 13:10:24 +01:00
Benoit Vanalderweireldt 2c10b1de19 SOLR-8754: add tests for org.apache.solr.util.hll.NumberUtilTest (#15) 2019-06-13 12:57:13 +02:00
Thomas Wöckinger 828960927f SOLR-13347: Add support for reading/writing UUID from/to TransactionLog (#681)
SOLR-13347: Transaction log to natively support UUID types
2019-06-13 08:56:41 +10:00
Mikhail Khludnev 5ba6c0c7a2 SOLR-13509: add omitHeader=false for shards requests to avoid NPE on partialResuls check 2019-06-12 18:09:29 +02:00
Erick Erickson bfb5b41144 SOLR-12013: collections API CUSTERSTATUS command fails when configset missing 2019-06-11 15:56:02 -07:00
Gus Heck abeecfacd4 SOLR-13439 - Adds ability to locally cache collection properties for a specified duration. 2019-06-11 10:12:52 -04:00
Andrzej Bialecki e2832b37d7 Improve this test - instead of waiting a fixed amount of time wait until an update arrives.
Also, deal with partial updates.
2019-06-11 14:10:30 +02:00
Andrzej Bialecki 6527e74ed8 Improve the test by actually checking that the version of Aliases is
more recent, instead of simply waiting.
2019-06-11 12:13:54 +02:00
Jan Høydahl 1942f8cb69 Revert "SOLR-13526: @AwaitsFix failing tests"
This reverts commit 85abdbeb
2019-06-11 11:03:00 +02:00
Andrzej Bialecki 010466ec04 SOLR-13512: Raw index data analysis tool (extension of COLSTATUS collection command). 2019-06-10 19:21:46 +02:00
Koen De Groote 67104dd615 LUCENE-8847: Code Cleanup: Rewrite StringBuilder.append with concatted strings (#707)
This specific commit affects all points in the casebase where the argument of a StringBuilder.append() call is itself a regular String concatenation.
This defeats the purpose of using StringBuilder and also introduces an extra alloction.
These changes should avoid that.

ant tests have run, succeeded on local machine.

Removing test files from the changes.

Another suggested rework.
2019-06-10 18:07:43 +02:00
Tim Underwood 97ca9df7ef LUCENE-8834: Cache the SortedNumericDocValues.docValueCount() value whenever it is used in a loop (#698) 2019-06-10 08:56:21 +02:00
Christine Poerschke 85f4029c5c SOLR-13496: Fix distributed grouping related NullPointerException in JSONWriter.writeSolrDocument 2019-06-07 16:05:10 +01:00
Jan Høydahl 85abdbebf5 SOLR-13526: @AwaitsFix failing tests 2019-06-07 08:34:17 +02:00
Cao Manh Dat 3364753661 SOLR-13434: Using back Java 9 type reference 2019-06-06 12:11:37 +01:00
Christine Poerschke 757e4548c7 SOLR-13518: extra assertNotNull info for SolrInfoBeanTest 2019-06-05 20:39:01 +01:00
Cao Manh Dat da832d4f3a SOLR-13434: Fixes problem on Java 8 build 2019-06-05 15:35:05 +01:00
Chris Hostetter 92cf9d8758 Fix TestSort.testRandomFieldNameSorts typo that ccaused very sporadic but reproducible failures 2019-06-04 13:49:57 -07:00
Cao Manh Dat 301ea0e462
SOLR-13434: OpenTracing support for Solr (#685) 2019-06-04 20:04:11 +01:00
erick 7fb5b7ed35 SOLR-12249: Better error message when grouping on a tokenized (non SortableText) field in SolrCloud 2019-06-04 10:27:06 -07:00
Cao Manh Dat cee4ed783e SOLR-13510: Intermittent 401's for internode requests with basicauth enabled 2019-06-04 16:12:49 +01:00
Erick Erickson 7ebeab71f4 SOLR-8346: Upgrade Zookeeper to version 3.5.5 2019-06-03 17:50:35 -07:00
Noble Paul 0a41163d27
SOLR-13504: In autoscaling policies, use an explicit 'put : on-each' to specify the the rules is applied on each node (#694)
SOLR-13504: In autoscaling policies, use an explicit 'put : on-each' to specify the the rules is applied on each node
2019-06-03 18:13:18 +10:00
markrmiller b09d462ee4 SOLR-13489: Stop the leader from trying to rejoin the election on session expiration and harden our zk reconnect code path. 2019-06-02 22:56:56 -05:00
Chris Hostetter c2b92f3015 Fix all solr tests to pass w/ -Dtests.dups=2 -Dtests.jvms=1 2019-05-29 10:27:15 -07:00
Christine Poerschke 90e0d1eba1 SOLR-12941: remove unused import in IndexSizeTriggerTest 2019-05-28 16:11:51 +01:00
Andrzej Bialecki 32ad372433 SOLR-12941: Fix IndexSizeTrigger to correctly work with "aboveBytes" and "splitMethod=link" parameters. 2019-05-28 11:34:57 +02:00
Noble Paul 6ac31c6db3
SOLR-13493: /autoscaling/suggestions to be able to filter by type (#686)
* SOLR-13493: /autoscaling/suggestions to be able to filter by type
2019-05-28 07:20:12 +10:00
Erick Erickson 57b64f2684 SOLR-12562: Reverting Clean up RealTimeGetComponent.toSolrDoc (sha 97e7d8a3d7 and 15aa9dfb3d) 2019-05-27 08:34:48 -07:00
Erick Erickson 97e7d8a3d7 SOLR-12652: Clean up RealTimeGetComponent.toSolrDoc 2019-05-26 08:23:53 -07:00
Chris Hostetter af4e1d324a Fix TestCloudSearcherWarming to work around SOLR-13490
Also clean up some crufty System.out/System.err pollution
2019-05-25 08:40:04 -07:00
Erick Erickson 379e6f64b8 SOLR-13454: Investigate ReindexCollectionTest failures 2019-05-23 08:43:34 -07:00
Noble Paul 975a4f8f55 Merge remote-tracking branch 'origin/master' into jira/SOLR-13484 2019-05-23 16:59:14 +10:00
Noble Paul a1c9f8f7db SOLR-13484: refactored code 2019-05-23 12:50:09 +10:00
Chris Hostetter 3ed31cfbc7 Fix TestStressCloudBlindAtomicUpdates to play nice with tests.dups 2019-05-22 11:22:52 -07:00
noble 750a5fdd02 SOLR-13484:
autoscaling/diagnostics APIshould be able to give diagnostics output from config pasted as a payload
2019-05-22 16:22:20 +10:00
Namgyu Kim 5a694ea26f LUCENE-8805: Parameter changes for stringField() in StoredFieldVisitor
Signed-off-by: Namgyu Kim <kng0828@gmail.com>
Signed-off-by: Adrien Grand <jpountz@gmail.com>
2019-05-21 20:18:42 +02:00
Chris Hostetter 6cc67e433e SOLR-13474: Fix "Search is temporarily disabled" logic to be consistent for entire request
SOLR-13469: Fix rejected requests during full disk full + IndexFetch to use 503 not 403
2019-05-20 10:24:02 -07:00
Andrzej Bialecki 93e57e63cd SOLR-13475: Null Pointer Exception when querying collection through collection alias. 2019-05-20 15:19:35 +02:00
Jan Høydahl 18cb42ee80 SOLR-13479: Harden OverseerTaskQueueTest 2019-05-17 20:54:46 +02:00
Alessandro Benedetti b9db118ed3 SOLR-12304: MLT component now supports mlt.interestingTerms 2019-05-16 23:11:43 -04:00
Noble Paul 1581d798b4 SOLR-13468: unused imports 2019-05-17 04:12:42 +10:00
David Smiley 3a88ab616c SOLR-13467: Include the S2 Geometry lib to make it simpler to use prefixTree="s2" on a Geo3D spatial field.
* Improved documentation on Geo3D.
* Better testing for Geo3D.
2019-05-15 22:15:13 -04:00
Noble Paul bd64ed6d2a
SOLR-13437: fork noggit code into Solr (#666)
* SOLR-13437: fork noggit code into Solr
2019-05-16 11:10:27 +10:00
Noble Paul c464d8a719
SOLR-13468: autoscaling/suggestions should be able to give suggestions from config sent as a payload (#678)
* SOLR-13468: autoscaling/suggestions should be able to give suggestions from config sent as a payload
2019-05-16 07:31:08 +10:00
Chris Hostetter 7ef5d5fe5f SOLR-13440: fix precommit failures 2019-05-14 13:35:59 -07:00
Chris Hostetter bf8c6ea435 SOLR-12999: Harden TestReplicationHandlerDiskOverFlow against sporadic timing failures
- ensure IndexFetcher injection is reset in @After method
 - replace System.out with Logger
 - Log and fail on any exceptions in any callbacks/threads
 - use CyclicBarrier (instead of CountdownLatch) to ensure the Query Thread loop doesn't monopolize
   CPU preventing IndexFetcher callback from ever being run

(Some of these improvements directly address jenkins failures we've been seeing)
2019-05-14 13:35:19 -07:00
Andrzej Bialecki de29d0d948 SOLR-13440: Fix occasional test failures by providing assert helpers aware of simulation
artifacts.
2019-05-14 19:03:02 +02:00
Andrzej Bialecki 13d1c113c1 SOLR-13407: Make sure we always have the latest version of aliases in cache before asserting. 2019-05-14 13:42:00 +02:00
Andrzej Bialecki 2315c6d1b8 SOLR-13440: Fix a precommit issue. 2019-05-13 19:55:39 +02:00
Andrzej Bialecki f2c18bacf2 SOLR-13440: Support saving/restoring autoscaling state for repeatable simulations. 2019-05-13 18:44:26 +02:00
Erick Erickson cd8508408a SOLR-13454: Investigate ReindexCollectionTest failures. moving bandaid calls 2019-05-12 10:36:46 -07:00
Erick Erickson 0aaf543208 SOLR-13454: Investigate ReindexCollectionTest failures. I missed a place where I should have called the bandaid code 2019-05-10 13:56:04 -07:00
Cao Manh Dat 6a06bcd470 SOLR-13445: Hardness the test 2019-05-10 15:25:54 +01:00
Erick Erickson 577be08bf2 SOLR-13454: Investigate ReindexCollectionTest failures 2019-05-09 12:45:47 -07:00
Chris Hostetter 5e40000e27 minimize randomization of IndexWriterConfig in TestDocTermOrdsUninvertLimit to reduce risk of OOM in nightly tests 2019-05-09 11:34:43 -07:00
Jan Høydahl 5b772f7c9d
SOLR-13453: Adjust auth metrics asserts in tests after SOLR-13449 (#668) 2019-05-09 20:13:46 +02:00
Chris Hostetter cde00b9a84 SOLR-12833: prevent NPE in DistributedUpdateProcessorTest AfterClass when mockito assumption fails in BeforeClass 2019-05-08 11:35:14 -07:00
Cao Manh Dat 81cfbcd009 SOLR-13445: Fix precommit 2019-05-08 19:16:54 +01:00
Cao Manh Dat 6b5b74bc9c SOLR-13445: Preferred replicas on nodes with same system properties as the query master 2019-05-08 17:55:49 +01:00
Gus Heck 76b854cb4f SOLR-13306 Add a request parameter to execute a streaming expression locally 2019-05-08 12:13:07 -04:00
Erick Erickson f4399a495e SOLr-13428: Take the WARN message out of the logs when optimizing. 2019-05-08 06:13:40 -07:00
Cao Manh Dat 5a35ba41f2 SOLR-13453: Marking mentioned tests as AwaitsFix 2019-05-08 11:17:54 +01:00
noble 733b071564 SOLR-13320 : add an update param failOnVersionConflicts=false to updates not fail when there is a version conflict 2019-05-07 15:20:02 +10:00
Ishan Chattopadhyaya ca29340d8b SOLR-13410: Making the test more resilient 2019-05-07 00:59:23 +05:30
Andrzej Bialecki ef3b6a1aa3 SOLR-12833: This test needs the update log - use "enable.update.log" property to turn it on. 2019-05-06 12:08:13 +02:00
Ishan Chattopadhyaya 67e4b0c517 SOLR-13410: Fix for test failure of the newly added test 2019-05-04 20:32:20 +05:30
Ishan Chattopadhyaya 1882a17115 SOLR-13410: Designated overseer wasn't able to rejoin election queue upon restart 2019-05-04 00:29:07 +05:30
Jan Høydahl cdd130ccb6 SOLR-12120: Fix bug in draining queue before shutdown 2019-05-02 23:26:16 +02:00
noble 8908205b42 SOLR-13441: Refactor code to minimize direct use of noggit 2019-05-03 07:20:05 +10:00
Ishan Chattopadhyaya 29a98825d0 SOLR-13348: Speed up collapsing by avoiding scoring of ineligible documents 2019-05-03 02:45:15 +05:30
Ishan Chattopadhyaya ceeac409a5 SOLR-12833: Fix PeerSyncTest and TestInPlaceUpdatesDistrib failures 2019-05-03 02:36:52 +05:30
Andrzej Bialecki e54610ce33 SOLR-13427: Delete duplicate package description. 2019-05-02 18:39:21 +02:00
David Smiley 1f91932ba4 SOLR-12833: Test should use ExecutorUtil 2019-05-01 14:30:35 -04:00
David Smiley a32decde6a SOLR-12833: DistributedUpdateProcessorTest assumeWorkingMockito() 2019-05-01 00:30:04 -04:00
Andrzej Bialecki 70e090717d SOLR-12833: Avoid unnecessary memory cost when DistributedUpdateProcessor timed-out lock is not used. 2019-04-30 22:31:44 +02:00
Andrzej Bialecki 710e15cec5 SOLR-13427: Add missing package description. 2019-04-30 19:19:49 +02:00
Mikhail Khludnev 5ca0602d28 SOLR-12291: fixing premature completion of async tasks
* extract async tracking methods from OverseerCollectionMessageHandler into the separate class
* replacing hashmap to named list to avoid entry loss
2019-04-30 09:57:42 +03:00
Andrzej Bialecki 6eccf2bf53 SOLR-13427: Support simulating the execution of autoscaling suggestions. 2019-04-29 23:14:38 +02:00
Andrzej Bialecki ced0243a3e SOLR-13407: Fix NPE and be consistent about returning empty instead of null properties. 2019-04-29 12:19:29 +02:00
Shalin Shekhar Mangar f77c56dbc6 SOLR-13432: Add .toString methods to BitDocSet and SortedIntDocSet so that enabling "showItems" on the filter caches shows some useful information about the values in the cache 2019-04-29 12:34:03 +05:30
Ishan Chattopadhyaya 8dd22bc056 SOLR-5970: Fix precommit 2019-04-28 23:29:11 +05:30
Ishan Chattopadhyaya dd9899b1c1 SOLR-5970: Return correct status upon collection creation failure 2019-04-28 23:16:29 +05:30
Ishan Chattopadhyaya 4b49bd99ca SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields 2019-04-28 23:01:58 +05:30
Erick Erickson a6262af842 SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload. More fixes, bad test. 2019-04-28 10:26:01 -04:00
Kevin Risden 35aeb7f623
SOLR-13414: SolrSchema - Avoid NPE if Luke returns field with no type defined
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-04-26 09:27:47 -04:00
Mikhail Khludnev 6d94631538 SOLR-13081: Let in-place update work with route.field 2019-04-25 14:05:42 +03:00
Erick Erickson 48dc020dda SOLR-13268: Clean up any test failures resulting from defaulting to async logging 2019-04-24 09:23:11 -07:00
Ishan Chattopadhyaya 33c94562a6 SOLR-12167: Throw an exception, instead of just a warning, upon unknown atomic update 2019-04-24 16:20:10 +05:30
Andrzej Bialecki 170f5fb7a3 SOLR-13423: Upgrade RRD4j to version 3.5. 2019-04-24 12:33:48 +02:00
Jan Høydahl 2261b4e8e1 SOLR-12120: Harden AuditLoggerIntegrationTest 2019-04-23 15:18:16 +02:00
Mikhail Khludnev a95e68e13b SOLR-13337: faster /terms response in distributed mode. 2019-04-22 23:26:12 +03:00
Cao Manh Dat c533988874 SOLR-13413: Adding debug log for HttpConnection 2019-04-22 10:39:05 +01:00
Erick Erickson d87196c141 SOLR-13400: Replace Observable pattern in TransientSolrCoreCache 2019-04-19 17:00:41 -07:00
Chris Hostetter dbf90a6e7b Reduce the upper bounds on some randomized variables in FullSolrCloudDistribCmdsTest to reduce risk of OOM 2019-04-19 14:48:37 -07:00
Tomas Fernandez Lobbe 9f644b1c69 SOLR-13281: Fix NPE in DocExpirationUpdateProcessor 2019-04-18 22:22:26 -07:00
Erick Erickson 2a3bd8e864 SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload bandaid for DocValuesNotIndexedTest 2019-04-18 21:38:01 -07:00
Andrzej Bialecki bd8905150d SOLR-12461: Upgrade Dropwizard Metrics to 4.0.5 release. 2019-04-18 19:08:20 +02:00
Andrzej Bialecki f46ba5227e SOLR-13407: Reject update requests sent to non-routed multi collection aliases. 2019-04-18 14:53:06 +02:00
erick a9771a5849 SOLR-13408: Cannot start/stop DaemonStream repeatedly, other API improvements 2019-04-17 16:02:06 -07:00
Uwe Schindler faaee86efb LUCENE-8738: Move to Java 11 as minimum Java version (merged branch: jira/LUCENE-8738)
Co-authored-by: Adrien Grand <jpountz@apache.org>
2019-04-16 14:00:09 +02:00
Jan Høydahl 9707bb6fa9 SOLR-12371: Editing authorization config via REST API now works in standalone mode 2019-04-15 21:09:30 +02:00
Simon Willnauer a302be381e
LUCENE-8671: Introduce Reader attributes (#640)
Reader attributes allows a per IndexReader configuration of codec internals.
For instance this allows a per reader configuration if FSTs are loaded into memory or are left
on disk.
2019-04-15 20:39:36 +02:00
markrmiller ecbf7daf56 SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node exists after an exists check and the Overseer work loop should not allow free spinning the loop when it hits a KeeperException. 2019-04-15 12:53:36 -05:00
Chris Hostetter d90034f0d6 SOLR-13336: add maxBooleanClauses (default to 1024) setting to solr.xml, reverting previous effective value of Integer.MAX_VALUE-1, to restrict risk of pathalogical query expansion. 2019-04-15 10:27:08 -07:00
Noble Paul e783207535 SOLR-13395: make DebugComponent.getRequestId() public 2019-04-15 11:10:06 +10:00
Jan Høydahl 03f5a5e7a1 SOLR-13398: Move log "Processing SSL Credential Provider chain" from INFO to DEBUG to prevent leaking into bin/solr printout 2019-04-12 11:54:40 +02:00
Jan Høydahl f2c59db273
SOLR-12860: MetricsHistoryHandler now always uses PKI Auth (#642)
* SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in background thread
2019-04-12 10:21:13 +02:00
Jan Høydahl 3b3879d880 SOLR-12121: Refresh JWK from IdP on invalid sig. Some logging improvements. Minor test fixes 2019-04-12 09:49:11 +02:00
Cao Manh Dat a67a941e19 SOLR-13339: Prevent recovery, fetching index being kicked off after SolrCores already closed 2019-04-11 09:21:14 +01:00
Andrzej Bialecki 02c4503f8c SOLR-13262: Add collection RENAME command and support using aliases in most collection admin commands. 2019-04-10 18:56:39 +02:00
Uwe Schindler eafe42f090 SOLR-13388: Fix FileExchangeRateProvider to be a public class, as it appears in schema.xml 2019-04-10 18:53:51 +02:00
Andrzej Bialecki d40a6d4e95 Release static solrClient after the test is done (occasionally caused failures on jenkins). 2019-04-10 12:26:02 +02:00
Moshe 8527ec11af SOLR-12638: Partial/Atomic updates of nested docs.
and [child] now works in RTG.
2019-04-10 03:02:59 -04:00
Gus Heck 4a93199803 SOLR-13370 - Trying reduction of cluster size, but not clear that
should help from local tests/profile, but maybe it helps in more memory
constrained build servers, this and prior version both beasted success
10 rounds of 5 on a machine with lots of memory.
2019-04-09 10:02:16 -04:00
Jan Høydahl 3e628b562c SOLR-12120: Do not fail the main request if synchronous auditing fails, log ERROR
Document that sub classes should call super.close() or a new waitForQueueToDrain() before closing itself
2019-04-09 14:25:52 +02:00
Chris Hostetter 2533fd1ede SOLR-13369: disable TriLevelCompositeIdRoutingTest until someone who actually understands how the /bits option is *suppose* to work can assess it to determine if the test is flawed or it there is a bug in the underlying CompositeIdRouter 2019-04-08 18:17:29 -07:00
Chris Hostetter ab55b6386b SOLR-13376: Disable test until it can be re-written to reflect actual expected behavior of how/when node markers will be cleaned up 2019-04-08 18:06:09 -07:00
Chris Hostetter 6bc360ee67 BlockCacheTest uses BlockCache which uses the common ForkJoinPool so we need to ensure the SolrIgnoredThreadsFilter is used 2019-04-08 10:57:02 -07:00
Andrzej Bialecki 0859be134d SOLR-13368: Tentative fix for a race condition in managed schema initialization. 2019-04-08 10:47:29 +02:00
Chris Hostetter 0b7421eca6 set the lastmod of the files to be deleted to be 1 day ago so that '-remove_old_solr_logs 0' doesn't run afoul of coarse granularity timestamps on windows jenkins machines if the test starts and runs quickly 2019-04-05 16:45:41 -07:00
Chris Hostetter 37166ce4e9 Use FSDirectory for TestDocTermOrdsUninvertLimit to reduce the amount of RAM the test needs.
This lowers the risk of OOM, w/o needing to sacrifice testing of how the randomized merging/segments might impact the uninversion
2019-04-05 15:06:50 -07:00
Erick Erickson 03d76c611b SOLR-13075: Harden SaslZkACLProviderTest. Put AwaitsFix back 2019-04-05 08:47:36 -07:00
Jan Høydahl aadc94a6e3 SOLR-12120: Harden AuditLoggerIntegrationTest, timing issues 2019-04-05 10:35:34 +02:00
Simon Willnauer 1ec229b604
LUCENE-8671: Expose FST off/on-heap options on Lucene50PostingsFormat (#613)
Before we can expose options to configure this postings format
on a per-reader basis we need to expose the option to load the terms
index FST off or on heap on the postings format. This already allows to
change the default in a per-field posting format if an expert user
wants to change the defaults. This essentially provides the ability to change
defaults globally while still involving some glue code.
2019-04-04 16:59:37 +02:00
Jan Høydahl 96ef35cd2c SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin 2019-04-04 16:07:23 +02:00
Jan Høydahl 361502da8a SOLR-12120: New AuditLoggerPlugin type allowing custom Audit logger plugins 2019-04-04 15:48:51 +02:00
Jason Gerlowski 9b44fba0d4 SOLR-13331: Fix AtomicUpdate 'remove' ops in SolrJ
The racent change introducing ByteArrayUtf8CharSequence altered the
NamedLists produced by atomic-update requests so that they include
instances of this class for requests coming in as javabin.  This is a
problem for 'remove' atomic-updates, which need to be able to compare
these ByteArrayUtf8CharSequence instances with existing field values
represented as Strings.  equals() would always return false, and
'remove' operations would have no effect.

This commit converts items as necessary to allow atomic-update
operations to work as expected.
2019-04-04 06:51:28 -04:00
Chris Hostetter 3fe4d0a30a Fix test to use solr.disable.shardsWhitelist=true rather then trying to whitelist the set of servers
The previous version of this test had a chicken/egg problem (needed to init the servers w/the whitelist but didn't know the port nums until after init)

that caused it to require 'restarting' the servers -- leading to the possibility of 'Address already in use' errors from jenkins machines if the OS reclaimed the port between the stop/start of the jetty instance
2019-04-03 15:15:03 -07:00
Kevin Risden 6596ed1c16
SOLR-13075: Harden SaslZkACLProviderTest (Kevin Risden, Hrishikesh Gadre, Peter Cseh)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-04-03 10:14:49 -04:00
erick 3810d5e7ac SOLR-13075: Harden SaslZkACLProviderTest. Testing new ZK version 2019-04-02 21:45:30 -07:00
Gus Heck 2d690885e5 SOLR-13322 - let forbidden apis check for sysout in solr core 2019-04-02 22:45:33 -04:00
Chris Hostetter 1071d09336 SOLR-13352: Remove risk of deadlock/threadleak when shutting down an Overseer(TriggerThread) 2019-04-01 10:34:51 -07:00
Erick Erickson e04a28e140 SOLR-12028: BadApple and AwaitsFix annotations usage 2019-04-01 09:27:33 -07:00
Kevin Risden a99cec0402
SOLR-13359: Make UpdateHandler support other prefixes (besides hdfs:/)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-31 10:45:57 -04:00
Kevin Risden 42f8203fc3
SOLR-11473: Make HDFSDirectoryFactory support other prefixes (besides hdfs:/)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-30 14:26:00 -04:00
Jason Gerlowski 07b37ff26b SOLR-13355: Obey 'ALL' for handlers with other predefined perms
Prior to this commit, RuleBasedAuthorizationPlugin would check for the
predefined 'ALL' permission only when the endpoint being hit wasn't
associated with another predefined-permission.

This resulted in some very unintuitive behavior. For example, the
permission {name:all, role:admin} would correctly prevent a
role:foo user from accessing /admin/info/properties, but would allow
write access to /admin/authorization because of the SECURITY_EDIT
predefined perm associated with that endpoint.

This commit fixes this bug so that the 'all' permission is always
consulted whether or not the endpoint is associated with other predefined
permissions.
2019-03-29 22:33:12 -04:00
Jason Gerlowski 5fe03bcd01 SOLR-13355: Small refactors to RuleBasedAuthorizationPlugin 2019-03-29 22:33:00 -04:00
erick b2941ff0da SOLR-13349:High CPU usage in Solr due to Java 8 bug 2019-03-28 15:49:54 -07:00
Kevin Risden 8d658a8cfb
SOLR-13353: Add SolrCli AuthTool test
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-28 14:08:07 -04:00
Joel Bernstein 98cadcf9f6 SOLR-13294: Change test file name 2019-03-27 15:36:47 -04:00
Joel Bernstein d407d556cc SOLR-13294: TestSQLHandler failures on windows jenkins machines 2019-03-27 15:36:47 -04:00
Ishan Chattopadhyaya e7939d5907 SOLR-7414: CSVResponseWriter & XLSXResponseWriter return empty field when fl alias is combined with * selector 2019-03-27 14:38:36 +05:30
Henning Andersen 04afdb6442 LUCENE-8735: Avoid FileAlreadyExistsException on windows. (#619)
FilterDirectory.getPendingDeletions() did not delegate the call, which
resulted in a new IndexWriter on same directory not considering pending
delete files. This could in turn result in a FileAlreadyExistsException
when running windows.
2019-03-26 14:56:45 +01:00
Kevin Risden 0231c1ecb6
SOLR-13342: Remove dom4j from Solr
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-25 08:15:01 -04:00
Gus Heck dad414ca22 SOLR-13323 - remove unused csv related classes/tests 2019-03-24 20:01:46 -04:00
Gus Heck c60685f9e4 SOLR-13324 - Don't swallow/print exception in URLClassifyProcessor anymore 2019-03-24 19:07:26 -04:00
Andrzej Bialecki 863d34bff5 SOLR-13322: Properly log exceptions. 2019-03-21 22:22:03 +01:00
Kevin Risden 6222abf448
SOLR-9079: Remove commons-lang as a dependency
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-20 19:37:19 -04:00
David Smiley 85a702cdff SOLR-13253: avoid using IndexSchema.getResourceLoader for non-schema things.
Furthermore it's reference to SolrConfig was removed.
2019-03-19 13:51:44 -04:00
Bar Rotstein 5b7866b085 SOLR-12955: Refactored DistributedUpdateProcessor to put SolrCloud specifics into a subclass
Closes #528
2019-03-19 13:26:31 -04:00
Andrzej Bialecki 6f2b7bf5c0 SOLR-11127: REINDEXCOLLECTION command for re-indexing of existing collections. 2019-03-19 13:42:59 +01:00
Kevin Risden 9fea3060b9
SOLR-8033: Remove debug if branch in HdfsTransactionLog
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-19 06:20:09 -04:00
Kevin Risden cf828163bd
SOLR-13330: Improve HDFS tests
Related JIRAs:
* SOLR-11010
* SOLR-11381
* SOLR-12040
* SOLR-13297

Changes:
* Consolidate hdfs configuration into HdfsTestUtil
* Ensure socketTimeout long enough for HDFS tests
* Ensure HdfsTestUtil.getClientConfiguration used in tests
* Replace deprecated HDFS calls
* Use try-with-resources to ensure closing of HDFS resources

Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-18 15:25:36 -04:00
erick 3ac07b8dfb SOLR-13268: Patch that flushes when shutting down 2019-03-18 11:52:39 -07:00
Chris Hostetter 5c143022e7 SOLR-12923: Mea culpa: Remove useless import of java.lang... that breaks precommit 2019-03-15 14:39:19 -07:00
Chris Hostetter 1a54c6b19d SOLR-12923: fix SimClusterStateProvider to use lock.lockInterruptibly() exclusively, and make SimCloudManager's Callable checks tollerant of Callables that may have failed related to interrupts w/o explicitly throwing InterruptedException 2019-03-15 12:06:53 -07:00
Chris Hostetter c79aeee5f9 SOLR-12923: tweak the randomization in testCreateLargeSimCollections to reduce the max possible totalCores
also decrease the number of iters while increase the cluster shape wait time to reduce the risk of spurious failures on machines under heavy contention w/o making the the test any slower on average
2019-03-15 10:43:28 -07:00
Andrzej Bialecki 571b307266 SOLR-13292: Fix a logic bug when point types are present, add more details to the unit test. 2019-03-15 11:52:37 +01:00
Chris Hostetter 76babf876a SOLR-12923: Fix some issues w/concurrency and exception swallowing in SimClusterStateProvider/SimCloudManager
There are 3 tightly related bug fixes in these changes:

1) ConcurrentModificationExceptions were being thrown by some SimClusterStateProvider methods when
   creating collections/replicas due to the use of ArrayLists nodeReplicaMap. These ArrayLists were changed
   to use synchronizedList wrappers.
2) The Exceptions from #1 were being swallowed/hidden by code using SimCloudManager.submit() w/o checking
   the result of the resulting Future object. (As a result, tests waiting for a particular ClusterShape
   would timeout regardless of how long they waited.)   To protect against "silent" failures like this,
   this SimCloudManager.submit() has been updated to wrap all input Callables such that any uncaught errors
   will be logged and "counted."  SimSolrCloudTestCase will ensure a suite level failure if any such failures
   are counted.
3) The changes in #2 exposed additional concurrency problems with the Callables involved in leader election:
   These would frequently throw IllegalStateExceptions due to assumptions about the state/existence of
   replicas when the Callables were created vs when they were later run -- notably a Callable may have been
   created that held a reference to a Slice, but by the time that Callable was run the collection (or a
   node, etc...) refered to by that Slice may have been deleted.  While fixing this, the leader election
   logic was also cleaned up such that adding a replica only triggers leader election for that shard, not
   every shard in the collection.

While auditing this code, cleanup was also done to ensure all usage of SimClusterStateProvider.lock was
also cleaned up to remove all risky points where an exception may have been possible after aquiring the
lock but before the try/finally that ensured it would be unlocked.
2019-03-14 22:27:48 -07:00
Alan Woodward fbd05167f4
LUCENE-3041: QueryVisitor (#581)
This commit adds an introspection API to Query, allowing users to traverse
the nested structure of a query and examine its leaves.  It replaces the existing
`extractTerms` method on Weight, and alters some highlighting code to use
the new API
2019-03-14 15:04:33 +00:00
Jan Høydahl a18aa2118b SOLR-12121: Remove a System.out.println in JWTAuthPlugin 2019-03-14 10:48:09 +01:00
Gus Heck d8f2a02fdb SOLR-13131 Category Routed Aliases 2019-03-13 01:32:23 -04:00
Gus Heck 9edc557f45 SOLR-12891 MacroExpander will no longer will expand URL parameters by
default inside of the 'expr' parameter, add InjectionDefense class
for safer handling of untrusted data in streaming expressions and add
-DStreamingExpressionMacros system property to revert to legacy behavior
2019-03-12 10:46:30 -04:00
Erick Erickson b893548d97 SOLR-13268: Clean up any test failures resulting from defaulting to async logging. Put TestXmlQParser back 2019-03-11 21:40:23 -07:00
Erick Erickson 28c8171fe6 SOLR-12732: TestLogWatcher failure on Jenkins. Deal with superclass logging bleeding into first watcher in the test 2019-03-11 12:55:08 -07:00
Chris Hostetter 7eb728a611 SOLR-13300: DistributedFacetExistsSmallTest should not attempt to compare results of a query that is known to differ in behavior in distributed mode 2019-03-11 10:02:46 -07:00
Kevin Risden 6777af074f
SOLR-13307: Ensure HDFS tests clear System properties they set (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-03-11 11:13:37 -04:00
Mikhail Khludnev e353dcfe7d SOLR-13284: fallback to json instead of NPE and 500 code, when wt is omitted or wrong. 2019-03-10 23:10:38 +03:00
erick 9272c29539 SOLR-13268: Clean up any test failures resulting from defaulting to async logging 2019-03-10 11:05:00 -07:00
Andrzej Bialecki 27df824a4b SOLR-13292: Provide extended per-segment status of a collection. 2019-03-09 14:08:01 +01:00
Erick Erickson 8c6e305365 SOLR-12732: TestLogWatcher failure on Jenkins. Added more logging 2019-03-08 21:40:06 -08:00
Chris Hostetter 4335ad97af Delete OpenCloseCoreStressTest
This is a fairly old test that isn't particularlay valuable, all it really does is stress test indexing, but w/o actually validating the results.

In it's current state, it can suffer from exceeding file handle limits depending on merge settings, and isn't reliably reproducible due to it's
approach of "do (a variable) amount of work until a fixed amount of wall clock time has elapsed"
2019-03-08 14:52:33 -07:00
Chris Hostetter 20de3d2ee0 SOLR-12923: harden TestSimLargeCluster
- added logging
- ensure start/finish trigger action counters are incremented before latches are released
- replace arbitrary sleep calls with a trigger listener countdown latch
- increase all await() times: This means that 'real' failures (which should be rare and hopefully
  reproducible) will be 'slow', but the trade off will be less hard to reproduce 'false failures'
  due to thread contention on slow or heavily loaded (ie: jenkins) machines
2019-03-07 15:40:57 -07:00
Erick Erickson 1e09268e78 SOLR-13261: Make SortableTextField work with export/streaming, now requires useDocValuesAsStored='true' 2019-03-07 10:59:20 -08:00
Cao Manh Dat 002a4aac3e SOLR-13302: Adding HttpPartitionTest and ForceLeaderTest for TLOG replicas 2019-03-07 12:18:41 +00:00
Cao Manh Dat 583ae97067 SOLR-12313: Fix precommit 2019-03-07 12:00:33 +00:00
Cao Manh Dat 4eb1c6f3c6 SOLR-12313: Removing TestInjection#waitForInSyncWithLeader 2019-03-07 11:25:32 +00:00
Chris Hostetter e827dc2234 SOLR-13237: AwaitsFix LeaderTragicEventTest until we can make the underlying corruption detection reliable 2019-03-06 13:29:29 -07:00
erick 5f91670a49 Clean up any test failures resulting from defaulting to async logging, reenable shutdown hook 2019-03-05 14:10:36 -08:00
Erick Erickson 558c515bf3 SOLR-13295: Reproducible failure in TestDistributedGrouping 2019-03-05 13:56:40 -08:00
Mikhail Khludnev de030178b5 SOLR-9882: picking results for mergeIds even after partial ones occurred 2019-03-05 23:26:47 +03:00
Andrzej Bialecki 0bc26d07ad SOLR-13271: Read-only mode for SolrCloud collections. 2019-03-05 17:16:25 +01:00
Chris Hostetter 7bfe7b265a SOLR-13294: refactor test to include more loging to help diagnose some windows jenkins failures 2019-03-04 14:16:02 -07:00
Chris Hostetter 7f7357696f SOLR-12923: harden testEventQueue by replacing the arbitrary sleep call with a countdown latch 2019-03-04 09:54:23 -07:00
Chris Hostetter 235b15acfc SOLR-12923: increase all await() times in TriggerIntegrationTest
This means that 'real' failures (which should be rare and hopefully reproducile) will be 'slow', but the trade off will be less hard to reproduce 'false failures' due to thread contention on slow or heavily loaded (ie: jenkins) machines
2019-03-04 09:54:23 -07:00
Mikhail Khludnev b8d569aff0 SOLR-9882: reporting timeAllowed breach as partialResults instead of 500 error 2019-03-04 17:41:57 +03:00
Chris Hostetter fed80599a6 SOLR-12923: increase all await() times in TestSimTriggerIntegration
This means that 'real' failures (which should be rare and hopefully reproducile) will be 'slow', but the trade off will be less hard to reproduce 'false failures' due to thread contention on slow or heavily loaded (ie: jenkins) machines
2019-03-01 11:38:18 -07:00
Chris Hostetter 5e28e94a93 fix AfterClass in OverseerTest to not fail with NPE if assumeWorkingMockito() call short circuited the BeforeClass 2019-02-28 15:17:12 -07:00
Kevin Risden 15f3c3b0e6
SOLR-7321: Remove reflection in FSHDFSUtils.java (Mike Drob, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-27 13:57:40 -05:00
Mikhail Khludnev 58beaeb74d SOLR-13227: don't throw exceptions for counting facet.range.other. 2019-02-26 23:38:51 +03:00
Kevin Risden 8f78c0591c
SOLR-9762: Remove the workaround implemented for HADOOP-13346 (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-26 14:52:32 -05:00
Kevin Risden 9753e00294
SOLR-13074: MoveReplicaHDFSTest leaks threads, falls into an endless loop, logging like crazy (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-26 14:48:47 -05:00
Kevin Risden 6a886b274d
SOLR-13060: Improve HdfsAutoAddReplicasIntegrationTest and HdfsCollectionsAPIDistributedZkTest
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-25 11:14:37 -05:00
Kevin Risden 9e7ec87f23
SOLR-9763: Remove the workaround implemented for HADOOP-12767 (Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-24 14:23:27 -05:00
erick fe5a96a284 SOLR-13268: Clean up any test failures resulting from SOLR-12055 (async logging). Kevin's upgrades 2019-02-24 09:50:04 -08:00
Mikhail Khludnev 52097627f8 SOLR-11883: reporting syntax errors as Bad Request instead of 500. 2019-02-23 23:55:55 +03:00
Erick 0de3905ce7 SOLR-12055: Enable async logging by default SOLR-12753: Async logging ring buffer and OOM error 2019-02-22 12:27:02 -08:00
Erick Erickson 6b4e90617d SOLR-13261: Make SortableTextField work with export/streaming 2019-02-22 11:04:31 -08:00
Tomas Fernandez Lobbe 906a08838b Fix TestTlogReplica
At some point got broken and nobody noticed since it's not running
2019-02-21 21:33:49 -08:00
Jason Gerlowski 5ab5ba773a Minor cleanup to ExportWriter 2019-02-21 16:05:48 -05:00
Tomas Fernandez Lobbe 6e81def01c Fix TestPullReplica.testKillLeader
Somewhere along the way, this was broken and since the test is being excluded, nobody noticed
2019-02-20 21:15:29 -08:00
Tomas Fernandez Lobbe bcd90f1561
SOLR-12708: CREATE collection request doesn't fail or cleanup when the request fails (#568)
When performing an async CREATE requests, responses to internal requests are also included inside the "success" or "failed" elements of the general response. This will make the operation cleanup in case of failure, the same way as we do when the request is synchronous.
2019-02-20 13:24:46 -08:00
Kevin Risden 86fe24cbef
SOLR-13222: Improve logging in StreamingSolrClients (Peter Cseh via Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-20 10:29:22 -05:00
Noble Paul a89b065c1f SOLR-13255 : ClasscastException when URPs try to read a String field which returns a ByteArrayUTF8CHarSequence . This is a regression in release 7.7 2019-02-20 22:19:18 +11:00
Kevin Risden af3ff118ef
SOLR-11763: Upgrade Guava to 25.1-jre (Markus Jelsma, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-19 17:27:58 -05:00
shalinmangar 7e2d40197c SOLR-13248: Autoscaling based replica placement is broken out of the box.
Solr 7.5 enabled autoscaling based replica placement by default but in the absence of default cluster policies, autoscaling can place more than 1 replica of the  same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not respected. Due to these reasons,  this issue reverts the default replica placement policy to the 'legacy' assignment policy that was the default until Solr 7.4.
2019-02-18 13:46:05 -08:00
Ishan Chattopadhyaya 6a0f7b251d SOLR-11876: In-place updates fail during resolution if required fields are present 2019-02-18 14:33:28 +05:30
Erick Erickson 3235c61e77 SOLR-12028: BadApple and AwaitsFix annotations usage. 2019-02-17 10:40:58 -08:00
Tomas Fernandez Lobbe 8ac34c2d6d SOLR-13229: Cleanup replicasMetTragicEvent after all exceptions 2019-02-15 14:51:35 -08:00
Simon Willnauer 4a513fa99f
LUCENE-8292: Make TermsEnum fully abstract (#574) 2019-02-15 17:32:55 +01:00
Alan Woodward 10acaf02f8 SOLR-13126: Correctly combine multiplicative query boosts 2019-02-15 10:09:26 +00:00
Andrzej Bialecki b4b9c39392 SOLR-13252: Fix an NPE when setting a "policy" property for an existing collection. 2019-02-14 21:38:22 +01:00
Andrzej Bialecki abcda8282c SOLR-13155: Minor addition to the "stats" section to make it easier to
estimate the distribution of cores across nodes.
2019-02-14 11:35:53 +01:00
Noble Paul cfefdade5f SOLR-13171 : A true streaming parser for javabin payload/stream without creating any objects 2019-02-13 18:23:43 +11:00
Bruno P. Kinoshita 5310152450 Fix some spell check issues 2019-02-12 21:35:30 -08:00
Andrzej Bialecki 242ff88e02 SOLR-13155: Add command-line option for testing autoscaling configurations. 2019-02-12 11:53:15 +01:00
Mikhail Khludnev bf69a40d16 SOLR-12330: rethow NPEs and other json.facet syntax errors properly as 400 2019-02-09 23:57:30 +03:00
yyuan2 a3a4ecd80b LUCENE-8662: Change TermsEnum.seekExact(BytesRef) to abstract 2019-02-08 15:10:38 -08:00
Tomas Fernandez Lobbe f2b84577a2 Set types in ZkController callables 2019-02-07 22:26:39 -08:00
Chris Hostetter 956772b7ef disable TestInjection in RestartWhileUpdatingTest
work around for SOLR-13189 and SOLR-13212
2019-02-07 18:27:32 -07:00
Kevin Risden 475f2fd36b
SOLR-9515 - Remove pom dependency kerby-kerb
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-07 11:38:18 -05:00
Chris Hostetter 87ad59f826 SOLR-13210: Fix TriLevelCompositeIdRoutingTest to actually make sense 2019-02-06 14:42:30 -07:00
Chris Hostetter ea2956fda3 Harden OrderedExecutorTest to use concurrent latches/barriers for testing parallelism instead of making assumpions about how milliseconds something should take in another thread 2019-02-06 14:32:12 -07:00
Tomas Fernandez Lobbe 0add3bb0ac Removed some unused variables from DistributedUpdateProcessor 2019-02-06 11:31:55 -08:00
Noble Paul b061947e91 SOLR-12999: Index replication could delete segments before downloading segments from master if there is not enough disk space 2019-02-06 23:15:07 +11:00
Kevin Risden 1a23ab005b
SOLR-10199: Solr's Kerberos functionality does not work in Java9 due to dependency on hadoop's AuthenticationFilter which attempt access to JVM protected classes
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-02 12:03:20 -05:00
Kevin Risden e8a24bc1c9
SOLR-8182: Update TestSolrCloudWithKerberosAlt to work with JDK9+
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-02 12:03:00 -05:00
Kevin Risden 429481bc82
Fix precommit - unused import
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-01 15:32:07 -05:00
markrmiller 4a3ddc94d8
SOLR-9515: Update to Hadoop 3 (Mark Miller, Kevin Risden)
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-02-01 15:04:56 -05:00
Kevin Risden e4f202c1e3
Revert "SOLR-9515: Update to Hadoop 3"
This reverts commit 6bb24673f4.
2019-01-31 19:46:02 -05:00
markrmiller 6bb24673f4
SOLR-9515: Update to Hadoop 3
Signed-off-by: Kevin Risden <krisden@apache.org>
2019-01-31 13:00:20 -05:00
Chris Hostetter 806744b0f1 reduce number of nodes/shards used in TriLevelCompositeIdRoutingTest to prevent (incredibly frequent) OOM in nightly 2019-01-31 10:45:12 -07:00
Dawid Weiss 4fc218e74e SOLR-13193: LeaderTragicEventTest doesn't catch NoSuchFileException when corrupting files 2019-01-31 11:55:05 +01:00
Chris Hostetter 0a01b9e127 disable TestInjection in TestStressCloudBlindAtomicUpdates
work around for SOLR-13189
2019-01-29 21:23:04 -07:00
Andrzej Bialecki 1cfbd3e1c8 SOLR-13072: Make sure the new overseer leader is present. 2019-01-29 11:22:28 +01:00
Chris Hostetter 8bee03f490 Add logging and impoved cleanup to TestStressCloudBlindAtomicUpdates setup/teardown codepaths that occasionally cause suite level failures in jenkins
The use of closeQuietly should hopefully prevent failures closing one HttpClient from resulting in other client objects being leaked

The setup changes are unlikey to improve test reliability, but will hopefully help diagnose where/how NPEs are coming from that currently cause some suite failures.
2019-01-28 16:22:28 -07:00
David Smiley 381a30b26c SOLR-12768: added _nest_path_ to the default schema (thereby enabling nested docs)
* new NestPathField encapsulating details for how _nest_path_ is indexed
** tweaked the analysis to index 1 token instead of variable
* TokenizerChain has new CustomAnalyzer copy-constructor
2019-01-28 13:25:06 -05:00
Dawid Weiss d7dc53ff7c LUCENE-8474: Remove deprecated RAMDirectory. 2019-01-28 13:49:03 +01:00
Mikhail Khludnev 43f2723213 SOLR-13156: support facet.sort for facet.field={!terms=foo,bar}field 2019-01-28 15:27:09 +03:00
Noble Paul 250d21da94 SOLR-13130: during the ResponseBuilder.STAGE_GET_FIELDS directly copy string bytes and avoid creating String Objects 2019-01-27 16:02:11 +11:00
Chris Hostetter ec68359065 SOLR-13168: Fixed a bug in TestInjection that caused test only code to be invoked when TLOG replicas recieved commits if java assertions were enabled
(see also: SOLR-12313)
2019-01-25 15:50:46 -07:00
Tomas Fernandez Lobbe ef81dadc7d SOLR-12373: Remove deprecated constructor 2019-01-25 13:48:40 -08:00
Tomas Fernandez Lobbe 0bd1911db6 SOLR-12373: Let DocBasedVersionConstraintsProcessor define fields to use in tombstones
A new config option, "tombstoneConfig" allows the DocBasedVersionConstraintsProcessor
to add extra fields to the tombstone generated when a document is deleted. This can
be useful when the schema has required fields.
2019-01-25 13:38:18 -08:00
Chris Hostetter e2b8b0e5b1 SOLR-12801: completely prevent tlog replicas from being used.
This follows the spirit of the change Mark intended in his previous commit to this test, but his solution wasn't covering all cases on backcompat to branch_7x

(see also: SOLR-12313)
2019-01-25 10:18:55 -07:00
Mikhail Khludnev c317119654 SOLR-13029: configure buffer size in HdfsBackupRepository. 2019-01-24 18:06:09 +03:00
Jan Høydahl d09c724a0e SOLR-12121: Fix test fails in JWTAuthPluginIntegrationTest.testMetrics 2019-01-24 12:43:27 +01:00
Andrzej Bialecki 84819c8379 SOLR-13072: Fix precommit. 2019-01-24 10:47:44 +01:00
Andrzej Bialecki 72a99e9c5c SOLR-13072: Enable this test again. 2019-01-23 15:46:25 +01:00
Jan Høydahl ea2c8ba38e SOLR-12121: JWT Token authentication plugin with OpenID Connect implicit flow login through Admin UI 2019-01-23 13:03:01 +01:00
Chris Hostetter 15e5ca999f SOLR-13140: harden SearchRateTriggerIntegrationTest by using more absolute rate thresholds and latches to track when all events have been processed so we don't need to 'guess' about sleep calls
This commit also disables testDeleteNode pending an AwaitsFix on SOLR-13163
2019-01-22 11:47:11 -07:00
Erick Erickson 60aef389cf SOLR-11998:RebalanceLeaders API broken response format with wt=JSON 2019-01-20 23:13:19 -06:00
Jimmy Casey 8b1c7296b7 Fixed Spelling. 2019-01-20 11:50:16 -08:00
Erick Erickson a692d05a90 SOLR-13091: REBALANCELEADERS is broken 2019-01-19 19:20:39 -08:00
Toke Eskildsen c13645bd4c LUCENE-8585: Create jump-tables for DocValues at index-time 2019-01-18 22:42:04 +01:00
fabric8-cd 63268000fb SOLR-13053: Adding test for assert that times nodes were removed and added was actually restored 2019-01-18 11:00:15 +00:00
Tomas Fernandez Lobbe 8b54b20fc4 SOLR-12770: Make it possible to configure a host whitelist for distributed search 2019-01-15 11:44:57 -08:00
Jan Høydahl b7f99fe55a SOLR-13137: NPE when /admin/zookeeper/status endpoint hit in standalone mode 2019-01-15 16:17:10 +01:00
Chris Hostetter d970375cd2 SOLR-12923: add a latch to TestTriggerListener to harden test that use it so they can deterministically know when all events have been proceeded
This hardens several flakey tests, and allows the removal of a several arbitrary sleep calls
2019-01-14 18:16:40 -07:00
Gus Heck dcc9ffe186 SOLR-13051 improve TRA update processor test
- remove some timeouts
  - better async mechanism linked to SolrCore lifecycle
  - add some additional tests to be a bit more thorough
2019-01-11 14:47:57 -05:00
Noble Paul 0635c657c1 SOLR-13016: precommit errors 2019-01-10 11:58:53 +11:00
Noble Paul 3ffc9099bf SOLR-13016: test errors fixed 2019-01-10 11:57:58 +11:00
David Smiley df119573db SOLR-12888: Run URP now auto-registers NestedUpdateProcessor before it. 2019-01-08 23:23:02 -05:00
Noble Paul d814d862b0 SOLR-12983: Create DocValues fields directly from byte[] 2019-01-09 13:44:41 +11:00
Andrzej Bialecki 7db4121b45 SOLR-13072: Use the same wait in other simulated tests where the same race condition may occur. 2019-01-08 19:12:38 +01:00
Andrzej Bialecki a37e2c609c SOLR-13072: Wait for autoscaling config refresh to finish before modifying the cluster
and enable the tests for now.
2019-01-08 16:16:55 +01:00
Noble Paul 825e63c00b SOLR-12983: tests don't need to use the optimization 2019-01-08 17:26:19 +11:00
Noble Paul 3932a4cc6c SOLR-12983: JavabinLoader should avoid creating String Objects and create UTF8CharSequence fields from byte[] 2019-01-08 15:27:48 +11:00