Commit Graph

533 Commits

Author SHA1 Message Date
Uwe Schindler b45c21f9db
Fix errorprone with alternative runtime (#12808) 2023-11-14 22:56:55 +01:00
Robert Muir c28d174cd7
script to run microbenchmarks across different ec2 instance types (#12787) 2023-11-10 12:31:10 -05:00
Jakub Slowinski 8ae598bae5
Remove patching for doc blocks. (#12741)
* Change Postings back to using FOR in Lucene99PostingsFormat

We are still keeping PFOR for positions only.
This is a partial revert of https://github.com/apache/lucene/pull/69 which brings back ForDeltaUtil.

* fix merge commit

* Add forgotten forDeltaUtil calls to reader

* Addressing comments: adding Lucene90RWPostingsFormat + more

Also:
* Change to Changes.txt
* Removal of dead code which was only used in unit tests
* Removal of test code from PForUtil

* Changes.txt edit in right place now

* Apply suggestions from code review: `90 -> 99 refactoring`

Co-authored-by: gf2121 <52390227+gf2121@users.noreply.github.com>

* Remove decodeTo32 from ForUtil and regenerate

---------

Co-authored-by: gf2121 <52390227+gf2121@users.noreply.github.com>
2023-11-06 10:46:03 -05:00
Dawid Weiss d6836d3d0e
tests.multiplier could be omitted in failed test reproduce line (#12752)
The default tests.multiplier passed from gradle was 1, but
LuceneTestCase tried to compute its default value from TESTS_NIGHTLY.
This could lead to subtle errors: nightly mode failures would not report
tests.multipler=1 and when started from the IDE, the tests.multiplier
would be set to 2 (leading to different randomness).
2023-11-03 17:05:17 +01:00
Dawid Weiss 8400f89a91
Fix javac task inputs so that they include modular dependencies #12742 (#12745)
Fix javac task inputs so that they include modular dependencies #12742
2023-11-02 08:49:41 +01:00
Nelson Osacky e5b55761d0
Build: build scans on ge.apache.org to benefit from deep build insights (#12293)
Apache committers who opt-in (via authentication) can have their local build scans be submitted to ge.apache.org.

Co-authored-by: Clay Johnson <cjohnson@gradle.com>
2023-10-24 12:32:18 -04:00
Dawid Weiss 2482f7688b
Add createClassLoader to replicator permissions (block specific to jacoco). (#12684) 2023-10-16 09:11:57 +02:00
Robert Muir 94619e00e0
fix Panama provider to only have static final constants, but allow sysprops for testing.
Set those sysprops by the build when running tests: force integer vectors on always, and randomize vector size
2023-10-14 19:51:59 -04:00
Robert Muir a4ff129de8
migrate all vectorbench methods to lucene (#12667)
* migrate all vectorbench methods to lucene

Co-authored-by: Uwe Schindler <uschindler@apache.org>
2023-10-14 11:36:42 -04:00
Dawid Weiss 06341ffe1d GITHUB#12655: upgrade jacoco aggregation plugin (failed with gradle 8.x), change html output property. 2023-10-12 20:42:17 +02:00
Dawid Weiss 603cd41ca1
Added JMH micro-benchmarks submodule (#12663) 2023-10-12 20:25:34 +02:00
Uwe Schindler fad6653495
Fix Gradle toolchain download with Gradle 8.4 (#12655) (#12662) 2023-10-12 13:22:48 +02:00
Dawid Weiss 2e12a35c87 GITHUB#12655: javac options via -J only when the compiler is in forked mode. 2023-10-12 09:05:03 +02:00
Kevin Risden 2c42b8941a
GITHUB#12655: Update googleJavaFormat plugin 2023-10-11 16:12:01 -04:00
Kevin Risden 30d3eba933
GITHUB#12655: Upgrade to Gradle 8.4
See https://github.com/apache/lucene/pull/12650 for details

Closes GITHUB#12655
2023-10-11 16:11:53 -04:00
Kevin Risden 75da33836b
avoid-circular-jar-checks (#12618) 2023-10-03 16:44:55 -04:00
Tyler Bertrand e43bea4fa4
Resolve CompileJava task cache miss (#12577)
Resolve CompileJava task cache miss. Implement CommandLineArgumentProvider to give relative path sensitivity to apiJar file
2023-09-21 10:42:20 +02:00
Dawid Weiss ebfbc831ab
Omit -Ptests.haltonfailure=false in failed test repro line #12565 (#12566) 2023-09-18 14:30:53 +02:00
Houston Putman 1c70c40082
Enable search for site javadocs (#12430) 2023-07-24 10:38:19 -04:00
Uwe Schindler f668cfd1cd
Fix forUtil.gradle to actually execute python script and also fix type error in script (#12411) 2023-07-03 16:22:03 +02:00
Uwe Schindler e503805758
Remove usage and add some legacy java.util classes to forbiddenapis (Stack, Hashtable, Vector) (#12404) 2023-06-29 16:56:41 +02:00
Chris Hegarty 1090928c14
Implement VectorUtilProvider with Java 21 Project Pamana Vector API (#12363)
This commit enables the Panama Vector API for Java 21. The version of
VectorUtilPanamaProvider for Java 21 is identical to that of Java 20.
As such, there is no specific 21 version - the Java 20 version will be
loaded from the MRJAR.
2023-06-13 09:44:58 +01:00
Uwe Schindler c8e05c8cd6
Implement MMapDirectory with Java 21 Project Panama Preview API (#12294) 2023-06-12 21:07:04 +02:00
Uwe Schindler c188d47a8b
Handle jdk.internal classes mentioned in vector superclass or interfaces during extraction (#12329) 2023-05-25 17:21:03 +02:00
Uwe Schindler cf7245e38e Refactor loop to not addAll set to itsself on initial round (followup of #12311) 2023-05-25 09:06:45 +02:00
Chris Hegarty f756f90644
Integrate the Incubating Panama Vector API (#12311)
Leverage accelerated vector hardware instructions in Vector Search.

Lucene already has a mechanism that enables the use of non-final JDK APIs, currently used for the Previewing Pamana Foreign API. This change expands this mechanism to include the Incubating Pamana Vector API. When the jdk.incubator.vector module is present at run time the Panamaized version of the low-level primitives used by Vector Search is enabled. If not present, the default scalar version of these low-level primitives is used (as it was previously).

Currently, we're only targeting support for JDK 20. A subsequent PR should evaluate JDK 21.
---------

Co-authored-by: Uwe Schindler <uschindler@apache.org>
Co-authored-by: Robert Muir <rmuir@apache.org>
2023-05-25 07:59:50 +01:00
Uwe Schindler 84e2e3afc3
Make sure APIJAR reproduces with different timezone (unfortunately java encodes the date using local timezone) (#12315) 2023-05-19 18:42:55 +02:00
Uwe Schindler e4d8a5c5cb
Implement MMapDirectory with Java 20 Project Panama Preview API (#12188) 2023-03-09 21:27:31 +01:00
Tyler Bertrand c514089d66
Gradle optimizations (#12150)
* Define inputs and outputs for task validateJarLicenses
  * Lazily configure validateJarLicenses
* Move functionality from copyTestResources task into processTestResources task
  * Lazily configure processTestResources
  * Altered TestCustomAnalyzer.testStopWordsFromFile() to find resources in updated location
* Resolve "overlapping output" issue preventing processTestResources from being cached
* Provide system properties from CommandLineArgumentProviders
  * Configure certain system properties as inputs to take advantage of UP-TO-DATE checking
  * Applies the correct pathing strategies to take full advantage of caching even if builds are executed from different locations on disk
* Make validateSourcePatterns task cacheable by removing .gradle directory from its input
2023-03-06 19:17:37 +01:00
Uwe Schindler 8564da434d
Generate gradle.properties from gradlew (#12131)
* SOLR-16641 - Generate gradle.properties from gradlew (#1320)
* Adapt for Lucene
* Remove localSettings from smoker; thanks @colvinco
* Print properties at end for debugging
* Add CHANGES.txt entry

---------

Co-authored-by: Colvin Cowie <colvin.cowie.dev@gmail.com>
Co-authored-by: Colvin Cowie <51863265+colvinco@users.noreply.github.com>
2023-02-06 19:47:15 +01:00
Michael Gibney d4006c0362
remove username from MANIFEST.MF in build artifacts (#12096) 2023-01-23 16:45:38 -05:00
Robert Muir 4f33aa8515
Upgrade to errorprone 2.18 (#12086) 2023-01-14 14:39:23 -05:00
Robert Muir e06f8c2e8b
Update to error-prone 2.17 (#12056) 2023-01-14 11:38:39 -05:00
Uwe Schindler c9401bf064
Patch class files for Java 19 code to no longer have the "preview" flag (this enables Java 19 memory segments by default) (#12033) 2022-12-26 10:07:44 +01:00
Robert Muir 3ac71adbdf
Ban use of Math.fma across the entire codebase (#12014)
When FMA is not supported by the hardware, these methods fall back to
BigDecimal usage which causes them to be 2500x slower.

While most hardware in the last 10 years may have the support, out of
box both VirtualBox and QEMU don't pass thru FMA support (for the latter
at least you can tweak it with e.g. -cpu host or similar to fix this).

This creates a terrible undocumented performance trap. Prevent it from
sneaking into our codebase.
2022-12-17 08:01:22 -05:00
Dawid Weiss 486003833f
Run spotless after javac (#12012) (#12015) 2022-12-13 08:42:04 +01:00
Robert Muir 06f9179295
Enable LongDoubleConversion error-prone check (#12010) 2022-12-12 20:55:39 -05:00
Dawid Weiss 1f741ff63c
Upgrade gradle to 7.6. (#11993) 2022-12-02 09:18:38 +01:00
Robert Muir fad3108b27
fix wrong serialization by ShapeDocValues (#11974)
Closes #11973
2022-12-01 20:32:42 -05:00
Robert Muir 0a9bb6e2ac
Disable useless error-prone checks (libraries/frameworks we do not use) (#11971)
These are easy/obvious ones to disable since we don't use the
functionality at all: the checks are literally useless.

This gives some performance boost to the error-prone, although it is
still pretty slow.

triage most of the previously disabled checks into TODO, noisy, etc
2022-12-01 08:46:23 -05:00
Robert Muir 4e93f29318
fix bad shift amounts and enable check (#11979) 2022-11-25 11:47:25 -05:00
Robert Muir 545c93a394
fix use of wrong array toString() method in test, enable check (#11978) 2022-11-25 11:47:04 -05:00
Robert Muir 4885b5f856
fix use of wrong array equals() method in test, enable check (#11977) 2022-11-25 11:46:48 -05:00
Robert Muir f4286493d1
fix variable assigned to itself in test and enable check (#11980) 2022-11-25 11:45:45 -05:00
Robert Muir d3fe435be6
Invert error-prone configuration to be allow-list vs deny-list (#11970)
This does not change the semantics or performance of our setup.

Instead, it explicitly enables checks that we want vs disabling checks
that we don't want.

Also reordered checks to match the error-prone website list of checks
for easier maintenance.

It is now clear that many useless checks are enabled, we can disable
some of them and try to get the performance reasonable.
2022-11-23 14:53:27 -05:00
Dawid Weiss 3f6410b738
Implement source code regeneration for test-framework perl scripts (#11952) 2022-11-19 23:40:45 +01:00
Dawid Weiss 82e57d1d6b Remove obsolete badapples tests switch (#468) 2022-11-18 16:57:24 +01:00
Robert Muir 99aa6dd0eb
Lower gradle heap: 3GB is unnecessary (#11936) 2022-11-18 10:55:32 -05:00
Dawid Weiss 2f21a866c1
Add star import check/validation (#11949)
* Remove some old cruft that only slows down checks. Add star import check
* Expand wildcard imports to comply with the rule.
2022-11-18 16:42:59 +01:00
Dawid Weiss a5e2525946 An alternative workaround for gradle -J javac arg passing bug (#11937) 2022-11-16 09:46:24 +01:00