Commit Graph

533 Commits

Author SHA1 Message Date
Robert Muir b292151f21
reland: 11925 javac options (#11937)
* pass jvm args to javac  #11925

* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging

* speed up the pure javac case too

It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.

Instead it is best to tune org.gradle.jvmargs.

* use the flags consistently everywhere (tests and doc)

* handle the different possible alt toolchain cases

The difference is invoking 'java' versus invoking 'javac', so the args must be fed differently.

Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2022-11-15 19:28:52 -05:00
Robert Muir 0753f48a80
Revert "speed up javac and error-prone tasks by using less resources (#11927)"
This reverts commit deab699188.

Having a shitty day: git lost changes :(
2022-11-15 19:08:43 -05:00
Robert Muir deab699188
speed up javac and error-prone tasks by using less resources (#11927)
* pass jvm args to javac  #11925
* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging
* speed up the pure javac case too

It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.

Instead it is best to tune org.gradle.jvmargs.

* use the flags consistently everywhere (tests and doc)

Co-authored-by: Dawid Weiss <dweiss@apache.org>
2022-11-15 19:04:17 -05:00
Robert Muir a02db83235
Revert "speed up javac and error-prone tasks by using less resources (#11927)"
This reverts commit b9c51d9a59.
2022-11-15 18:19:09 -05:00
Dawid Weiss b9c51d9a59
speed up javac and error-prone tasks by using less resources (#11927)
* pass jvm args to javac  #11925
* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging
* speed up the pure javac case too

It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.

Instead it is best to tune org.gradle.jvmargs.

* use the flags consistently everywhere (tests and doc)

Co-authored-by: Robert Muir <rmuir@apache.org>
2022-11-15 16:06:38 -05:00
Robert Muir 1b9d98d6ec
enable error-prone "narrow calculation" check (#11923)
This check finds bugs such as https://github.com/apache/lucene/pull/11905.

See https://errorprone.info/bugpattern/NarrowCalculation
2022-11-15 06:42:41 -05:00
Uwe Schindler e9ef61ba39 Fix bug with set of strings since upgrade of Gradle -> explicit cast from GString to String 2022-11-10 17:18:30 +01:00
Patrick Zhai 26ec0dd44c
add gradle aggregated coverage console log and html location (#11882)
* jacoco/ coverage shouldn't trigger all test tasks as dependencies - instead, it should run after those test tasks that you choose to run. removed java plugin from top-level.

* Make coverage depend on the default test task.

* Update jacoco log plugin so that it doesn't make hard dependencies on test tasks.

Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2022-10-28 23:33:37 -07:00
Dawid Weiss 5c7edd7f38
Upgrade to gradle 7.5.1 (excluding launch scripts, which we have customized) (#11886) 2022-10-28 08:49:36 +02:00
Robert Muir 4c434b7089
make 'gradle coverage' print test coverage summaries. (#11837)
Currently, this task is too silent and just writes HTML reports. It is a
nice improvement to print the summary to the console.

Before:

```
> Task :lucene:analysis:icu:jacocoTestReport
Code coverage report at: /home/rmuir/workspace/lucene/lucene/analysis/icu/build/reports/jacoco/test/html.
```

After:

```
> Task :lucene:analysis:icu:jacocoTestReport
Code coverage report at: /home/rmuir/workspace/lucene/lucene/analysis/icu/build/reports/jacoco/test/html.

> Task :lucene:analysis:icu:jacocoLogTestCoverage
Test Coverage:
    - Class Coverage: 100%
    - Method Coverage: 87.9%
    - Branch Coverage: 82.7%
    - Line Coverage: 92.8%
    - Instruction Coverage: 92.7%
    - Complexity Coverage: 78.8%
```
2022-10-05 21:46:20 -04:00
Uwe Schindler f54fddc89f
GH-11819: Exclude MR-JAR sourceSet and folders from Idea Sync (#11836) 2022-10-04 11:49:39 +02:00
Uwe Schindler df94e6c005
Clean up MR-JAR build, so we do not have hardcoded "19" everywhere in validation tasks (#11835)
As long as soureSets are named "mainXX", with XX a feature version, we check everything automatically:
- ECJ is disabled (we can't do a check without forking ECJ as a separate process using toolkit, we may support this later)
- forbiddenapis (we disable checks for missing classes)
- errorprone is disabled (errorprone does not work correctly at moment with forked compiler)
2022-10-02 20:41:46 +02:00
Uwe Schindler e5a226ec7c For now only use bundled signatures from minJavaVersion (#11834)
# Conflicts:
#	gradle/validation/forbidden-apis.gradle
2022-10-02 17:54:11 +02:00
Uwe Schindler aae293437f
Upgrade forbiddenapis to 3.4 (#11834) 2022-10-02 16:42:36 +02:00
Uwe Schindler 1f30800cb5
GH-11819: Fix the Eclipse part to support deveopment of the MR-JAR: (#11823)
- by default, Lucene will only generate a config for Java 17 (or 11 in 9.x), without the MR-JAR sourceSets
- if passed -Peclipse.javaVersion=19, it will include matching sourcesets and set compiler version to given version in classpath
2022-09-27 11:11:49 +02:00
Uwe Schindler 3b9c728ab5
MR-JAR rewrite of MMapDirectory with JDK-19 preview Panama APIs (>= JDK-19-ea+23) (#912)
This uses Gradle's auto-provisioning to compile Java 19 classes and build a multi-release JAR from them. Please make sure to regenerate gradle.properties (delete it) or change "org.gradle.java.installations.auto-download" to "true"
2022-09-26 15:22:04 +02:00
Dawid Weiss 6b82be5f11
Regenerate sources after dependency updates. (#11817) 2022-09-25 18:09:30 +02:00
Dawid Weiss 5d121ce44c
Upgrade several build dependencies. (#11812)
* Upgrade several build dependencies.

* Update error prone rules (those are off but they do trigger warnings/ errors)

* A few corrections I made before I turned off new warnings. Let's do nother issue to fix them.
2022-09-25 17:10:22 +02:00
Dawid Weiss 54fba99cb1
Upgrade google java format and apply tidy (#11811) 2022-09-24 15:40:27 +02:00
Dawid Weiss 9acc653995
GH-11172: remove WindowsDirectory and native subproject. (#11774) 2022-09-15 16:22:46 +02:00
Marios Trivyzas dbffe3472b
LUCENE-10423: Remove usages of System.currentTimeMillis() from tests (#11749)
* Remove usages of System.currentTimeMillis() from tests

- Use Random from `RandomizedRunner` to be able to use a Seed to
  reproduce tests, instead of a seed coming from wall clock.
- Replace time based tests, using wall clock to determine periods
  with counter of repetitions, to have a consistent reproduction.

Closes: #11459

* address comments

* tune iterations

* tune iterations for nightly
2022-09-06 17:55:01 -04:00
Tomoko Uchida e61958e4fd links to github should be '/issues' 2022-08-27 11:54:20 +09:00
tang donghai b08e34722d
LUCENE-10646: Add some comment on LevenshteinAutomata (#1016)
* add Comment on Lev & pretty the toDot

* use auto generate scripts to add comment

* update checksum

* update checksum

* restore toDot

* add removeDeadStates in levAutomata

Co-authored-by: tangdonghai <tangdonghai@meituan.com>
2022-08-07 10:01:30 -04:00
Dawid Weiss f93e52e5bb
LUCENE-10669: The build should be more helpful when generated resources are touched (#1053) 2022-07-30 20:45:32 +02:00
Uwe Schindler eafc6420f3
Exclude Lucene's own JAR files from classpath entries in Eclipse config (#976) 2022-06-24 19:42:42 +02:00
Uwe Schindler 831ae2465b
Remove the deflater hack introduced because of JDK-8252739 (#977)
This bug was fixed in JDK-16 so it does not apply to main branch (Java 17) anymore.
2022-06-24 14:23:10 +02:00
Dawid Weiss 54c67db10d
Try to fix the gradle compilation in idea (#945)
* Try to fix the gradle compilation in idea
* Try to detect sync and build phases within intellij and act accordingly to support both modes of compilation (gradle and intellij).
2022-06-09 11:14:10 +02:00
Dawid Weiss e8c1720956 Revert "Try to detect sync and build phases within intellij and act accordingly to support both modes of compilation (gradle and intellij)."
This reverts commit 13e10eaae2.
2022-06-03 20:19:21 +02:00
Dawid Weiss 13e10eaae2 Try to detect sync and build phases within intellij and act accordingly to support both modes of compilation (gradle and intellij). 2022-06-03 20:18:04 +02:00
Dawid Weiss 5b92002fed LUCENE-10370: recreate temporary location in case it's wiped by a clean. 2022-05-23 10:51:02 +02:00
Dawid Weiss 63b66e06cd
LUCENE-10370: pass proper classpath/module arguments for forking jvms from within tests. (#909) 2022-05-21 00:30:02 +02:00
Tomoko Uchida b911d1d47c
LUCENE-10531: Add @RequiresGUI test group for GUI tests (#893)
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2022-05-18 09:26:06 +09:00
Uwe Schindler 7a8071c9d4
Detect CI builds and enable errorprone by default for those CI builds (#890) 2022-05-14 20:49:50 +02:00
Robert Muir 3edfeb5eb2
LUCENE-10532: remove @Slow annotation (#832)
Remove `@Slow` annotation, for more consistency with CI and local jobs. All tests can be fast!
2022-05-09 23:03:55 -04:00
Tomoko Uchida 5f48469837
Allow to link to github PR from changes (#854) 2022-05-02 23:06:39 +09:00
Dawid Weiss 75aadb9589
gradle 7.3.3 quick upgrade (#856) 2022-04-29 21:02:19 +02:00
Dawid Weiss a53d05b9f9
Upgrade spotless and use runToFixMessage for 'gradlew tidy' hint. (#834) 2022-04-25 14:51:14 +02:00
Robert Muir d6461eab0b
improve spotless error to suggest running 'gradlew tidy' (#817)
The current error isn't helpful as it suggests a per-module command. If
the user has modified multiple modules, they will be running gradle
commands to try to fix each one of them, when it would be easier to just
run 'gradlew tidy' a single time and fix everything.
2022-04-21 08:30:10 -04:00
Dawid Weiss ba1062620c
LUCENE-10510: Check module access prior to running gjf/spotless tasks (#802) 2022-04-10 20:35:45 +02:00
Tomoko Uchida 2a3e5ca07f
LUCENE-10475: Merge o.a.l.a.[ja|ko].util into o.a.l.a.[ja|ko].dict (#772) 2022-03-29 21:09:26 +09:00
Uwe Schindler ff263f0aa4
Upgrade to forbiddenapis 3.3 (#768) 2022-03-26 17:09:42 +01:00
Mike Drob b3906e96ea
LUCENE-9651 Update benchmark module docs (#759) 2022-03-23 14:51:28 -05:00
Tomoko Uchida 76c9fd4e38 LUCENE-10416: Update Korean Dictionary to mecab-ko-dic-2.1.1-20180720 for Nori 2022-02-20 21:39:03 +09:00
Dawid Weiss 8aa4763070 LUCENE-10419: fix rat thread safety bug. 2022-02-13 18:43:13 +01:00
Dawid Weiss a861ff8df2 LUCENE-10419: revert debugging changes. 2022-02-13 18:34:57 +01:00
Dawid Weiss 50b7e2970f LUCENE-10419: more debugging code. The message from AbstractStringBuilder suggests a concurrency issue somewhere, but I just can't see it! 2022-02-12 20:22:49 +01:00
Dawid Weiss 1f1da12c89 LUCENE-10419: add debugging code. 2022-02-10 12:03:54 +01:00
Dawid Weiss 9a28c91a5a LUCENE-10283: bump the minimum source/release in javadoc settings. 2022-02-02 17:25:50 +01:00
Dawid Weiss 87bba4152c LUCENE-10283: bump the minimum source/release in ecj linter settings. 2022-02-02 17:25:41 +01:00
Robert Muir 61edacee5d
update javac flags for java 17 (#628)
Previously -Xlint:text-blocks and -Xlint:text-blocks were enabled
conditionally, if the user had at least java 15 or java 16,
respectively. Enable them always.

Add new options so that the warnings list is fully configured:
* -Xlint:module (new in java 17)
* -Xlint:strictfp (new in java 17)

Disable "path" with -Xlint:-path rather than commenting it out, for
consistency.

Disable "missing-explicit-ctor" (new in java 17), as it is unlikely to
succeed right now.

Alphasort the flags and doc how to get the updated list, this makes it
easy to compare and keep up to date.
2022-01-28 05:48:58 -05:00
Adrien Grand 2ebc57a465
LUCENE-10283: Bump minimum required Java version to 17. (#579)
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2022-01-10 15:42:15 +01:00
Uwe Schindler 42fe2d5620
LUCENE-10364: Prepare and update errorprone plugin for Java 17 (#590) 2022-01-07 19:19:46 +01:00
Dawid Weiss ff547e7bbd
LUCENE-10328: Module path for compiling and running tests is wrong (#571) 2022-01-05 20:42:02 +01:00
Uwe Schindler 475fbd0bdd
LUCENE-10352: Convert TestAllAnalyzersHaveFactories and TestRandomChains to a global integration test and discover classes to check from module system (#582)
Co-authored-by: Robert Muir <rmuir@apache.org>
2022-01-05 15:35:02 +01:00
Dawid Weiss 0f0d06ca28
LUCENE-10347: add a helper task 'collectRuntimeJars' that assembles binary artifacts under each module's build 'runtimeJars' folder. (#576) 2022-01-03 21:11:35 +01:00
Uwe Schindler 0b517573a4
LUCENE-10342: Add logging to static initializers to warn users if unmapping or object size calculation does not work (#572)
Co-authored-by: Tomoko Uchida <tomoko.uchida.1111@gmail.com>
2021-12-29 18:18:21 +01:00
Dawid Weiss d928c47d6e
LUCENE-10338: Scan for tests only by convention file name pattern (#565) 2021-12-26 18:18:24 +01:00
Dawid Weiss a94fbb79ac LUCENE-10301: make the test-framework a proper module by moving all test
classes to org.apache.lucene.tests.*. Also changes distribution layout
(all modules are now under modules/).
2021-12-21 20:30:45 +01:00
Dawid Weiss e790125504
LUCENE-10331: don't emit the contents of the inputs file until we're actually running the task (#554) 2021-12-20 19:27:37 +01:00
Dawid Weiss 5512786dd9 LUCENE-10327: workaround for gradle emitting empty sourcepath. 2021-12-19 08:52:01 +01:00
Dawid Weiss d2c98912eb This reverts commit a7b50f723d. 2021-12-19 08:51:13 +01:00
Dawid Weiss a7b50f723d Reverting back to b48cac02. 2021-12-18 23:36:30 +01:00
Dawid Weiss 2a44ff532e LUCENE-10308: sort input files for ecj so that module-info.java comes first. 2021-12-18 21:17:56 +01:00
Dawid Weiss d42db56bab LUCENE-10255: initial support for Java Modules. 2021-12-18 20:45:51 +01:00
Dawid Weiss b48cac0206
LUCENE-10285: try to force ordering of internal tasks, in spite of making top-level wrapper dependencies. (#549) 2021-12-17 19:12:09 +01:00
Dawid Weiss 1a429c621e Render javadocs for all projects, even if they're not part of site. 2021-12-17 17:57:38 +01:00
Uwe Schindler db9dff225c after reading code, correct the argument file to comply with ECJ's parser 2021-12-17 12:18:28 +01:00
Uwe Schindler 6941701c6d Escape the options in ECJ's options file 2021-12-17 11:37:27 +01:00
Dawid Weiss ae92e96481 Address Uwe's remarks. 2021-12-16 20:05:05 +01:00
Dawid Weiss e0745c7b24 LUCENE-10255: re-add utilities for debugging packages and services. These are not included by default to avoid unnecessary compilation overhead. 2021-12-16 17:59:54 +01:00
Dawid Weiss 5b3b75efd8 LUCENE-10308: Make ecj and javadoc run with modular paths 2021-12-16 17:51:01 +01:00
Dawid Weiss 1bcdc600b3 LUCENE-10304: exclude module-info.java from all sourcesets for Eclipse, otherwise things break (predictably). 2021-12-10 19:56:55 +01:00
Dawid Weiss 458c0486c0 LUCENE-10304: a workaround for intellij's problem with runtime scopes on dependencies. 2021-12-10 17:16:19 +01:00
Dawid Weiss 600d8345f8 LUCENE-10306: set up module configurations to consume full JARs for test projects. 2021-12-10 17:16:19 +01:00
Dawid Weiss 328b3cc55f LUCENE-10255: add support for .tests subprojects which contain module tests. 2021-12-10 17:16:19 +01:00
Dawid Weiss 6d83c2e08e LUCENE-10255: add gradle compilation and module descriptor support for the java module system. Adds module descriptors to all Lucene subprojects. 2021-12-10 17:16:19 +01:00
Dawid Weiss b9c22fdb49 LUCENE-9871: minor cleanups of extra semicolons and solr build remnants. 2021-12-10 10:29:35 +01:00
Dawid Weiss 8367f700c7 LUCENE-10294: Avoid compiling javadocs twice in 'gradlew check'. 2021-12-09 09:56:11 +01:00
Robert Muir c8f5b9127d
LUCENE-10243: increase unicode versions of tokenizers to 12.1 (#465)
* Bump %unicode 9 -> %unicode 12.1 for the 3 unicode grammars
* regenerate emoji conformance tests for unicode 12.1
* modify wordbreak conformance tests to use emoji data (which replaces old crazy E_base etc properties)
* regenerate wordbreak conformance tests
* Simplify grammar files and word-break conformance test generator, now that full-width numbers are WordBreak=Numeric
* Use jflex emoji properties rather than ICU-generated ones
2021-12-03 20:20:57 -05:00
Dawid Weiss 20cb6817db
LUCENE-10234: Change module prefix to org.apache.* (#487) 2021-11-30 22:03:33 +01:00
Robert Muir 453168ec76
support tables in generated html documentation (#489)
Tables can be used in markdown (e.g. MIGRATE.md) and will become html tables in our generated HTML docs on the website.
2021-11-29 17:38:14 -05:00
Robert Muir 63c89f678d
Speed up ECJ tasks by avoiding --release (#484)
LUCENE-10185 caused a large performance regression in ECJ tasks by using the --release flag.

Instead of using --release, we can just disable "terminal deprecation", and leave this check to `javac`. The --release flag makes this tool run 50% slower.
2021-11-28 15:10:32 -05:00
Dawid Weiss 1029651d12 Don't log warnings from ant (different class loader, I guess). Makes Alan happier. 2021-11-26 11:39:55 +01:00
Robert Muir af831d2810
LUCENE-10239: upgrade jflex (1.7.0 -> 1.8.2) (#452)
Upgrade jflex.

Change doesn't alter the behavior of any of the analyzers (unicode
version or grammar refactorings), just the minimal to get new tooling
working.
2021-11-19 09:24:27 -05:00
Dawid Weiss bae095ae48
LUCENE-10240: gradle regenerate fails on java 17 (#449) 2021-11-17 18:36:34 +01:00
Dawid Weiss 0eeba8d37c
LUCENE-10238: Update icu4j to 70.1. (#447) 2021-11-17 18:13:40 +01:00
Dawid Weiss f5e5cf008a
LUCENE-10234: Add automatic module name to JAR manifests. (#440) 2021-11-15 17:02:40 +01:00
Dawid Weiss f725b27e12
LUCENE-10223: interval support in standard syntax parser (#429) 2021-11-11 08:54:59 +01:00
Dawid Weiss 263765a9b0 LUCENE-10226: test target creates a weird folder (lazy property). 2021-11-09 08:34:04 +01:00
Uwe Schindler 1ae6b2a6b9
UCENE-10218: Extend validateSourcePatterns task to scan for LTR/RTL unicode to catch "Trojan Source" source code attacks (#425)
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2021-11-03 17:19:24 +01:00
Dawid Weiss d14ee2d01e Use a dumber offlineLinks map but expose more structured information to gradle. 2021-11-02 12:18:52 +01:00
Dawid Weiss d819a2eec4 Cherry pick the validateSourcePatterns. Correct exclusion patterns (.gradle and .idea are only at the root project level). 2021-11-02 12:18:51 +01:00
Jerome Prinet d44e03b8b0 Make ratTask cacheable 2021-11-02 12:18:51 +01:00
Jerome Prinet b3e59232f5 Make checkBrokenLinks cacheable 2021-11-02 12:18:51 +01:00
Dawid Weiss 1a38cac68e LUCENE-10195: add commented-out org.gradle.caching=true to the generated local settings. 2021-11-02 12:18:51 +01:00
Dawid Weiss 0544819b78
LUCENE-10200: store git revision in the release folder and read it back from buildAndPushRelease (#419) 2021-11-01 09:29:06 +01:00
Dawid Weiss 6d8ea58ccd LUCENE-10200: Rename pddl-10.txt to reference glove. 2021-10-31 10:41:16 +01:00
Mike Drob 23256a30fa
Replace deprecated Gradle 7.2 properties (#417) 2021-10-29 09:59:47 -05:00
Dawid Weiss 62eb9a809e LUCENE-10200: remove unused dangling license exclusions. Add references to the remaining ones. 2021-10-27 20:40:39 +02:00
Dawid Weiss 780846a732
LUCENE-10198: Allow external JAVA_OPTS in gradlew scripts; use sane defaults (heap, stack and system proxies) (#405)
Co-authored-by: balmukundblr <balmukund.mandal@intel.com>
2021-10-26 09:15:55 +02:00