FileSwitchDirectory splits file actions between 2 directories based
on file extensions. The extensions are respected on write operations
like delete or create but ignored when we list the content of the
directories. Until now we only deduplicated the contents on
Directory#listAll which can cause inconsistencies and hard to debug
errors due to double deletions in IndexWriter is a file is pending
delete in one of the directories but still shows up in the directory
listing form the other directory. This case can happen if both
directories point to the same underlying FS directory which is a
common use-case to split between mmap and NIOFS.
This change filters out files from directories depending on their
file extension to make sure files that are deleted in one directory
are not returned form another if they point to the same FS directory.
Today in the method IOUtils#fsync we ignore IOExceptions when fsyncing a
directory. However, the catch block here is too broad, for example it
would be ignoring IOExceptions when we try to open a non-existent
file. This commit addresses that by scoping the ignored exceptions only
to the invocation of FileChannel#force. This prevents us from
suppressing an exception in case we run into an unexpected issue when
opening the file.
However, fsyncing directories on Windows is not possible. We always
suppressed this by allowing that an AccessDeniedException is thrown when
attemping to open the directory for reading. Yet, per the above, this
suppression also allowed other IOExceptions to be suppressed, and that
should be considered a bug (e.g., not only the directory not existing,
but any filesystem error and other reasons that we might get an access
denied there, like genuine permissions issues). Rather than relying on
exceptions for flow control and continuing to suppress there, we simply
return early if attempting to fsync a directory on Windows (we should
not put this burden on the caller).
* LUCENE-8837: smokeTestRelease.py option --download-only
Move download() and check_and() functions to scriptutil
Add cwd param to run() function in scriptutil
Move the check_ant function from buildAndPushRelease into scriptutil.py, and let it return the version.
This specific commit affects all points in the casebase where the argument of a StringBuilder.append() call is itself a regular String concatenation.
This defeats the purpose of using StringBuilder and also introduces an extra alloction.
These changes should avoid that.
ant tests have run, succeeded on local machine.
Removing test files from the changes.
Another suggested rework.
Noble Paul
Noble Paul
Ishan Chattopadhyaya
Joel Bernstein
Gus Heck
Cao Manh Dat
Mikhail Khludnev
Charlie Yan
Christine Poerschke
Jan Høydahl
Erick Erickson
Cassandra Targett
Alan Woodward
Jan Høydahl
Benoit Vanalderweireldt
Dag Wieers
Thomas Wöckinger
Simon Willnauer
Andrzej Bialecki
Adrien Grand
Jason Tedor
Ignacio Vera
Koen De Groote