Leverage accelerated vector hardware instructions in Vector Search.
Lucene already has a mechanism that enables the use of non-final JDK APIs, currently used for the Previewing Pamana Foreign API. This change expands this mechanism to include the Incubating Pamana Vector API. When the jdk.incubator.vector module is present at run time the Panamaized version of the low-level primitives used by Vector Search is enabled. If not present, the default scalar version of these low-level primitives is used (as it was previously).
Currently, we're only targeting support for JDK 20. A subsequent PR should evaluate JDK 21.
---------
Co-authored-by: Uwe Schindler <uschindler@apache.org>
Co-authored-by: Robert Muir <rmuir@apache.org>
* Define inputs and outputs for task validateJarLicenses
* Lazily configure validateJarLicenses
* Move functionality from copyTestResources task into processTestResources task
* Lazily configure processTestResources
* Altered TestCustomAnalyzer.testStopWordsFromFile() to find resources in updated location
* Resolve "overlapping output" issue preventing processTestResources from being cached
* Provide system properties from CommandLineArgumentProviders
* Configure certain system properties as inputs to take advantage of UP-TO-DATE checking
* Applies the correct pathing strategies to take full advantage of caching even if builds are executed from different locations on disk
* Make validateSourcePatterns task cacheable by removing .gradle directory from its input
When FMA is not supported by the hardware, these methods fall back to
BigDecimal usage which causes them to be 2500x slower.
While most hardware in the last 10 years may have the support, out of
box both VirtualBox and QEMU don't pass thru FMA support (for the latter
at least you can tweak it with e.g. -cpu host or similar to fix this).
This creates a terrible undocumented performance trap. Prevent it from
sneaking into our codebase.
These are easy/obvious ones to disable since we don't use the
functionality at all: the checks are literally useless.
This gives some performance boost to the error-prone, although it is
still pretty slow.
triage most of the previously disabled checks into TODO, noisy, etc
This does not change the semantics or performance of our setup.
Instead, it explicitly enables checks that we want vs disabling checks
that we don't want.
Also reordered checks to match the error-prone website list of checks
for easier maintenance.
It is now clear that many useless checks are enabled, we can disable
some of them and try to get the performance reasonable.
* pass jvm args to javac #11925
* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging
* speed up the pure javac case too
It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.
Instead it is best to tune org.gradle.jvmargs.
* use the flags consistently everywhere (tests and doc)
* handle the different possible alt toolchain cases
The difference is invoking 'java' versus invoking 'javac', so the args must be fed differently.
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
* pass jvm args to javac #11925
* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging
* speed up the pure javac case too
It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.
Instead it is best to tune org.gradle.jvmargs.
* use the flags consistently everywhere (tests and doc)
Co-authored-by: Dawid Weiss <dweiss@apache.org>
* pass jvm args to javac #11925
* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging
* speed up the pure javac case too
It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.
Instead it is best to tune org.gradle.jvmargs.
* use the flags consistently everywhere (tests and doc)
Co-authored-by: Robert Muir <rmuir@apache.org>
* jacoco/ coverage shouldn't trigger all test tasks as dependencies - instead, it should run after those test tasks that you choose to run. removed java plugin from top-level.
* Make coverage depend on the default test task.
* Update jacoco log plugin so that it doesn't make hard dependencies on test tasks.
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
Currently, this task is too silent and just writes HTML reports. It is a
nice improvement to print the summary to the console.
Before:
```
> Task :lucene:analysis:icu:jacocoTestReport
Code coverage report at: /home/rmuir/workspace/lucene/lucene/analysis/icu/build/reports/jacoco/test/html.
```
After:
```
> Task :lucene:analysis:icu:jacocoTestReport
Code coverage report at: /home/rmuir/workspace/lucene/lucene/analysis/icu/build/reports/jacoco/test/html.
> Task :lucene:analysis:icu:jacocoLogTestCoverage
Test Coverage:
- Class Coverage: 100%
- Method Coverage: 87.9%
- Branch Coverage: 82.7%
- Line Coverage: 92.8%
- Instruction Coverage: 92.7%
- Complexity Coverage: 78.8%
```
As long as soureSets are named "mainXX", with XX a feature version, we check everything automatically:
- ECJ is disabled (we can't do a check without forking ECJ as a separate process using toolkit, we may support this later)
- forbiddenapis (we disable checks for missing classes)
- errorprone is disabled (errorprone does not work correctly at moment with forked compiler)
- by default, Lucene will only generate a config for Java 17 (or 11 in 9.x), without the MR-JAR sourceSets
- if passed -Peclipse.javaVersion=19, it will include matching sourcesets and set compiler version to given version in classpath
This uses Gradle's auto-provisioning to compile Java 19 classes and build a multi-release JAR from them. Please make sure to regenerate gradle.properties (delete it) or change "org.gradle.java.installations.auto-download" to "true"
* Upgrade several build dependencies.
* Update error prone rules (those are off but they do trigger warnings/ errors)
* A few corrections I made before I turned off new warnings. Let's do nother issue to fix them.
* Remove usages of System.currentTimeMillis() from tests
- Use Random from `RandomizedRunner` to be able to use a Seed to
reproduce tests, instead of a seed coming from wall clock.
- Replace time based tests, using wall clock to determine periods
with counter of repetitions, to have a consistent reproduction.
Closes: #11459
* address comments
* tune iterations
* tune iterations for nightly
* add Comment on Lev & pretty the toDot
* use auto generate scripts to add comment
* update checksum
* update checksum
* restore toDot
* add removeDeadStates in levAutomata
Co-authored-by: tangdonghai <tangdonghai@meituan.com>
* Try to fix the gradle compilation in idea
* Try to detect sync and build phases within intellij and act accordingly to support both modes of compilation (gradle and intellij).
The current error isn't helpful as it suggests a per-module command. If
the user has modified multiple modules, they will be running gradle
commands to try to fix each one of them, when it would be easier to just
run 'gradlew tidy' a single time and fix everything.
Previously -Xlint:text-blocks and -Xlint:text-blocks were enabled
conditionally, if the user had at least java 15 or java 16,
respectively. Enable them always.
Add new options so that the warnings list is fully configured:
* -Xlint:module (new in java 17)
* -Xlint:strictfp (new in java 17)
Disable "path" with -Xlint:-path rather than commenting it out, for
consistency.
Disable "missing-explicit-ctor" (new in java 17), as it is unlikely to
succeed right now.
Alphasort the flags and doc how to get the updated list, this makes it
easy to compare and keep up to date.
Uwe Schindler
Chris Hegarty
Tyler Bertrand
Michael Gibney
Robert Muir
Dawid Weiss
Patrick Zhai
Marios Trivyzas
Tomoko Uchida
tang donghai
Mike Drob
Adrien Grand