Commit Graph

77 Commits

Author SHA1 Message Date
Robert Muir e2110e0b8e
Add regeneration workaround to the docs 2024-03-30 09:52:21 -04:00
sabi0 f6b2006195
Fix typo in help/formatting.txt (#12960) 2023-12-21 19:58:53 +01:00
Mike McCandless ee3d60ff92 fix silly typo 2023-12-12 13:04:45 -05:00
zhouhui 27c8e3f13f
Add downloading binutils instructions for the macos. (#12804) 2023-11-14 05:51:51 -05:00
Rob Muir 90f8bac9f7 slight correction 2023-10-21 02:05:34 -04:00
Rob Muir c09fb560f7 add hsdis instructions for the macos 2023-10-21 02:02:24 -04:00
Michael McCandless afb2a60751
Bound the RAM used by the NodeHash (sharing common suffixes) during FST compilation (#12633)
* tweak comments; change if to switch

* remove old SOPs, minor comment styling, fixed silly performance bug on rehash using the wrong bitsRequired (count vs node)

* first raw cut; some nocommits added; some tests fail

* tests pass!

* fix silly fallback hash bug

* remove SOPs; add some temporary debugging metrics

* add temporary tool to test FST performance across differing NodeHash sizes

* remove (now deleted) shouldShareNonSingletonNodes call from Lucene90BlockTreeTermsWriter

* add simple tool to render results table to GitHub MD

* add simple temporary tool to iterate all terms from a provided luceneutil wikipedia index and build an FST from them

* first cut at using packed ints for hash t able again

* add some nocommits; tweak test_all_sizes.py to new RAM usage approach; when half of the double barrel is full, allocate new primary hash at full size to save cost of continuously rehashing for a large FST

* switch to limit suffix hash by RAM usage not count (more intuitive for users); clean up some stale nocommits

* switch to more intuitive approximate RAM (mb) limit for allowed size of NodeHash

* nuke a few nocommits; a few more remain

* remove DO_PRINT_HASH_RAM

* no more FST pruning

* remove final nocommit: randomly change allowed NodeHash suffix RAM size in TestFSTs.testRealTerms

* remove SOP

* tidy

* delete temp utility tools

* remove dead (FST pruning) code

* add CHANGES entry; fix one missed fst.addNode -> fstCompiler.addNode during merge conflict resolution

* remove a mal-formed nocommit

* fold PR feedback

* fold feedback

* add gradle help test details on how to specify heap size for the test JVM; fix bogus assert (uncovered by Test2BFST); add TODO to Test2BFST anticipating building massive FSTs in small bounded RAM

* suppress sysout checks for Test2BFSTs; add helpful comment showing how to run it directly

* tidy
2023-10-20 11:52:55 -04:00
Robert Muir a4ff129de8
migrate all vectorbench methods to lucene (#12667)
* migrate all vectorbench methods to lucene

Co-authored-by: Uwe Schindler <uschindler@apache.org>
2023-10-14 11:36:42 -04:00
Dawid Weiss 603cd41ca1
Added JMH micro-benchmarks submodule (#12663) 2023-10-12 20:25:34 +02:00
Uwe Schindler 8564da434d
Generate gradle.properties from gradlew (#12131)
* SOLR-16641 - Generate gradle.properties from gradlew (#1320)
* Adapt for Lucene
* Remove localSettings from smoker; thanks @colvinco
* Print properties at end for debugging
* Add CHANGES.txt entry

---------

Co-authored-by: Colvin Cowie <colvin.cowie.dev@gmail.com>
Co-authored-by: Colvin Cowie <51863265+colvinco@users.noreply.github.com>
2023-02-06 19:47:15 +01:00
Dawid Weiss 326142c485
Add a note about gradle checks being possibly a subset of all validation checks. (#11966) 2022-11-23 12:19:28 +01:00
Robert Muir b292151f21
reland: 11925 javac options (#11937)
* pass jvm args to javac  #11925

* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging

* speed up the pure javac case too

It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.

Instead it is best to tune org.gradle.jvmargs.

* use the flags consistently everywhere (tests and doc)

* handle the different possible alt toolchain cases

The difference is invoking 'java' versus invoking 'javac', so the args must be fed differently.

Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2022-11-15 19:28:52 -05:00
Robert Muir 0753f48a80
Revert "speed up javac and error-prone tasks by using less resources (#11927)"
This reverts commit deab699188.

Having a shitty day: git lost changes :(
2022-11-15 19:08:43 -05:00
Robert Muir deab699188
speed up javac and error-prone tasks by using less resources (#11927)
* pass jvm args to javac  #11925
* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging
* speed up the pure javac case too

It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.

Instead it is best to tune org.gradle.jvmargs.

* use the flags consistently everywhere (tests and doc)

Co-authored-by: Dawid Weiss <dweiss@apache.org>
2022-11-15 19:04:17 -05:00
Robert Muir a02db83235
Revert "speed up javac and error-prone tasks by using less resources (#11927)"
This reverts commit b9c51d9a59.
2022-11-15 18:19:09 -05:00
Dawid Weiss b9c51d9a59
speed up javac and error-prone tasks by using less resources (#11927)
* pass jvm args to javac  #11925
* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging
* speed up the pure javac case too

It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.

Instead it is best to tune org.gradle.jvmargs.

* use the flags consistently everywhere (tests and doc)

Co-authored-by: Robert Muir <rmuir@apache.org>
2022-11-15 16:06:38 -05:00
Robert Muir 3edfeb5eb2
LUCENE-10532: remove @Slow annotation (#832)
Remove `@Slow` annotation, for more consistency with CI and local jobs. All tests can be fast!
2022-05-09 23:03:55 -04:00
Robert Muir 1089b482fc
LUCENE-10528: use Xvfb in test to avoid messing up user's desktop (#828)
Co-authored-by: Tomoko Uchida <tomoko.uchida.1111@gmail.com>
2022-04-23 08:00:33 -04:00
Tomoko Uchida 872349cef9
Add some basic tasks to help/workflow (#811) 2022-04-18 11:34:28 +09:00
Rich Bowen e9789afb39
LUCENE-10513: Run `gradlew tidy` first (#808)
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2022-04-12 21:22:55 +09:00
Dawid Weiss ba1062620c
LUCENE-10510: Check module access prior to running gjf/spotless tasks (#802) 2022-04-10 20:35:45 +02:00
Dawid Weiss 486141f0eb LUCENE-9660: correct help/tests.txt. 2021-10-26 08:45:58 +02:00
Mike Drob 95759d299e
Fix typo 2021-10-14 13:28:10 -05:00
Chris Hostetter f64c81c3f8 LUCENE-10173: remove max-worker restriction added by LUCENE-9488 when 'useGpg' in effect
Also update docs to remove the point of confusion that lead to thinking that restriction was useful
2021-10-14 10:50:16 -07:00
Dawid Weiss 8bcc3dc430
LUCENE-9488: rewrite distribution assembly, signing and checksum generation (#372) 2021-10-13 11:50:58 +02:00
Nhat Nguyen 9a17d67658
LUCENE-9935: Enable bulk merge for stored fields with index sort (#134)
This commit enables bulk-merges (i.e., raw chunk copying) for stored 
fields when index sort is enabled
2021-05-12 21:00:18 -04:00
Chris Hostetter a6cf46dada LUCENE-9936: Add gpg signing of the tgz & zip distribution files 2021-05-04 10:20:59 -07:00
Robert Muir 06907a2c12
LUCENE-9188: Add jacoco code coverage support to gradle (#119)
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
Co-authored-by: Uwe Schindler <uschindler@apache.org>
2021-05-02 16:24:06 +02:00
Dawid Weiss bd8f182b13
LUCENE-9933: Add non-file properties to wrapped regenerate checksums (#95) 2021-04-19 13:37:47 +02:00
Dawid Weiss beafd113de
LUCENE-9931: Rename checksummed regen. tasks FooInternal and generated wrappers Foo (#88) 2021-04-16 22:35:51 +02:00
Robert Muir 9d15435b15
LUCENE-9916: add a simple regeneration help doc (#73)
Add a simple regeneration help doc

Improve task help and checksum failure message (include corresponding regeneration task). Sorry for being verbose. Maybe somebody will read it. :)

Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2021-04-11 11:28:41 -04:00
Dawid Weiss 246c4beb22
LUCENE-9854: Clean up utilities to download and extract test/ benchmark data sets. (#27) 2021-03-22 12:22:39 +01:00
Dawid Weiss 1679076bde Nuke more unused/ obsolete refs. 2021-03-19 13:11:37 +01:00
Dawid Weiss f1299bca9f Nuke the obsolete ant.txt help. 2021-03-19 13:09:42 +01:00
Houston Putman 2b8d7bcd6a
SOLR-15075: Solr docker gradle improvements (#2197)
* Removed docker plugin from gradle builds.
* Removed package docker image.
* Tasks now have correct inputs/outputs/dependencies.
* Move gradle help text to docker folder.
* Reduce duplicated Docker layer by doing file removal and chmod in another stage.

Co-authored-by: David Smiley <dsmiley@apache.org>
2021-01-26 10:22:50 -05:00
Dawid Weiss a827aaac2b LUCENE-9570: add parial reformatting commit reference to .git-blame-ignore-revs [thanks rmuir] and add a note to git.txt aiding in making this a persistent local option. 2020-12-18 13:10:47 +01:00
Dawid Weiss c94b035df8
LUCENE-9564: add spotless and gjf (automatic code formatter) 2020-12-17 13:11:54 +01:00
Houston Putman d65041359e
SOLR-14949: Adding githubUrl option for docker build. (#2074) 2020-11-10 13:31:33 -05:00
Houston Putman 212b0f8657
SOLR-14949: Ability to customize Solr Docker build (#2020)
Also added a gradlew helpDocker page.
2020-11-10 10:42:38 -05:00
Erick Erickson f6c4f8a755 SOLR-14910: Use in-line tags for logger declarations in Gradle ValidateLogCalls that are non-standard, change //logok to //nowarn 2020-10-03 09:47:37 -04:00
Mike Drob 14e4edc1e9
SOLR-14847: Create Solr Server TGZ (#1844)
Create new targets for building solr binary release artifacts:

gradlew -p solr/packaging distTar distZip

Co-authored-by: Houston Putman <houston@apache.org>
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
Co-authored-by: Mike Drob <mdrob@apache.org>
2020-09-10 10:55:48 -05:00
Christine Poerschke ccb299a2de Fix typo in help/ant.txt file. 2020-09-09 13:06:45 +01:00
Dawid Weiss b1e2d0c890
LUCENE-9438: Eclipse IDE support with gradle build system (#1761) 2020-08-21 21:47:11 +02:00
Dawid Weiss 83ed210fd0
LUCENE-9465: 'beast' task from within gradle (#1757) 2020-08-18 09:28:50 +02:00
Dawid Weiss 8ebf2d0b21 LUCENE-9312: Allow builds against arbitrary JVMs (squashed
jira/LUCENE-9312)
2020-07-21 09:19:38 +02:00
Jason Gerlowski 0ea0358624 SOLR-14532: Add *.iml files to gitignore
Also clarifies our docs on importing the project into IntelliJ.
2020-06-17 20:22:17 -04:00
Erick Erickson 217c2faa2c LUCENE-7788: fail precommit on unparameterised log messages and examine for wasted work/objects 2020-05-01 13:06:57 -04:00
Erick Erickson 9ae05e9b4f LUCENE-7788: fail precommit on unparameterised log messages and examine for wasted work/objects 2020-04-30 19:50:31 -04:00
Erick Erickson 6e96d01efc LUCENE-7788: fail precommit on unparameterised log messages and examine for wasted work/objects 2020-04-29 10:56:54 -04:00
Erick Erickson 960610a615 LUCENE-7788: fail precommit on unparameterised log messages and examine for wasted work/objects 2020-04-27 20:45:57 -04:00