FileSwitchDirectory fails if the tmp file are not in the same directory
as the file it's renamed to. This is correct behavior but breaks with
tmp files used with index sorting. This change tries best effort to find
the right extension directory if the file ends with `.tmp`
this test already uses waitForState (frequently via verifyReplicaStatus) so there is no reason to include CPU/network/ZK intensive infinite loop checks looking for udpated cluster state
(cherry picked from commit 3030ea9d9418626ffb3c3bbd174aee344a778b25)
Also add Predicate<DocCollection> equivilents for callers that don't care about liveNodes.
(cherry picked from commit 5a974860fa83408a86ca64b417f3111b037da7eb)
FileSwitchDirectory splits file actions between 2 directories based
on file extensions. The extensions are respected on write operations
like delete or create but ignored when we list the content of the
directories. Until now we only deduplicated the contents on
Directory#listAll which can cause inconsistencies and hard to debug
errors due to double deletions in IndexWriter is a file is pending
delete in one of the directories but still shows up in the directory
listing form the other directory. This case can happen if both
directories point to the same underlying FS directory which is a
common use-case to split between mmap and NIOFS.
This change filters out files from directories depending on their
file extension to make sure files that are deleted in one directory
are not returned form another if they point to the same FS directory.
Today in the method IOUtils#fsync we ignore IOExceptions when fsyncing a
directory. However, the catch block here is too broad, for example it
would be ignoring IOExceptions when we try to open a non-existent
file. This commit addresses that by scoping the ignored exceptions only
to the invocation of FileChannel#force. This prevents us from
suppressing an exception in case we run into an unexpected issue when
opening the file.
However, fsyncing directories on Windows is not possible. We always
suppressed this by allowing that an AccessDeniedException is thrown when
attemping to open the directory for reading. Yet, per the above, this
suppression also allowed other IOExceptions to be suppressed, and that
should be considered a bug (e.g., not only the directory not existing,
but any filesystem error and other reasons that we might get an access
denied there, like genuine permissions issues). Rather than relying on
exceptions for flow control and continuing to suppress there, we simply
return early if attempting to fsync a directory on Windows (we should
not put this burden on the caller).
* LUCENE-8837: smokeTestRelease.py option --download-only
Move download() and check_and() functions to scriptutil
Add cwd param to run() function in scriptutil
Move the check_ant function from buildAndPushRelease into scriptutil.py, and let it return the version.
(cherry picked from commit 44287d420624fe669a319f95f4712486bd57ae38)
