Commit Graph

95 Commits

Author SHA1 Message Date
Mark Payne c97510aa66
NIFI-13862: Ensure that when FlowFile Repository is checkpointed, we always call the onGlobalSync() method to ensure that any transient claims that may have been provided (such as those from Stateless NiFi) get dealt with appropriately.
This closes #9375

Signed-off-by: Joseph Witt <joewitt@apache.org>
2024-10-10 13:41:53 -07:00
dan-s1 f64e68b23c
NIFI-13625 Removed or Replaced System.out.println in Tests (#9150)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-05 20:35:40 -05:00
dan-s1 b3fac8c80b
NIFI-13403 Removed unnecessary toString in logging statements
This closes #8984

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-06-25 20:13:08 -05:00
dan-s1 7deac6afac
NIFI-13266 Removed String concatenation in logging messages (#8940)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-06-11 23:26:58 -04:00
dan-s1 86d36d2327
NIFI-13265 Removed instantiation of Object arrays for log arguments
This closes #8896

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-05-30 17:17:33 -05:00
Pierre Villard ee179f4acf
NIFI-13238 Added Checkstyle rules for whitespace
This closes #8832

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-05-15 06:41:19 -05:00
Joseph Witt 42a4385c82
NIFI-12998 Restructured framework and extension modules
Simplified dependency management into Bill of Material dependencies as parents that align to classloader structures.

- Moved nifi-framework-bundle to root level
- Moved nifi-nar-bundle modules to root nifi-extension-bundles
- Added nifi-extension-bom for internal and external extensions
- Added nifi-framework-nar-bom and nifi-server-nar-bom for managing framework dependencies
- Renamed nifi-jetty-bundle to nifi-jetty-nar

This closes #8677

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-04-26 12:12:48 -05:00
exceptionfactory 98cebc6687
NIFI-12185 Added nifi-bom Bill of Materials (#7852)
- Imported nifi-bom to nifi-commons and nifi-nar-bundles parent modules
- Removed explicit version and scope properties for dependencies declared in nifi-bom
2023-10-10 13:55:22 -04:00
Pierre Villard c06bb97e85
NIFI-11833 - remove deprecated classes in nifi-commons (#7503)
Signed-off-by: Otto Fowler <ottobackwards@gmail.com>
2023-07-20 11:58:51 -04:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
Joe Witt c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt 109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
Joe Witt 8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt 8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Mike Thomsen 8d513c5ed3
NIFI-9080 Converted nifi-commons to use JUnit 5
This closes #5332

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-25 21:07:54 -05:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
Joe Witt 88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT 2021-02-15 12:09:32 -07:00
Joe Witt 4afb2ba743
NIFI-7873-RC4 prepare for next development iteration 2021-02-15 12:09:31 -07:00
Joe Witt 487280bee9
NIFI-7873-RC4 prepare release nifi-1.13.0-RC4 2021-02-15 12:09:30 -07:00
Mark Payne fd068fe978
NIFI-7557: uses a canonical representation of strings when recovering data from FlowFile Repository in order to avoid using huge amounts of heap when not necessary
- Fixed some problems with unit/integration tests

This closes #4507.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-09-03 10:21:50 -04:00
Joe Witt 8baa5c9940
NIFI-7692 updating for next dev release 1.13.0 2020-08-18 14:48:02 -07:00
Joe Witt fb57bcbc11
NIFI-7692-RC1 prepare for next development iteration 2020-08-13 09:20:39 -07:00
Joe Witt 303d6c59ba
NIFI-7692-RC1 prepare release nifi-1.12.0-RC1 2020-08-13 09:20:36 -07:00
Mark Payne d68720920f
NIFI-7242: When a Parameter is changed, any property referencing that parameter should have its #onPropertyModified method called. Also renamed Accumulo tests to integration tests because they start embedded servers and connect to them, which caused failures in my environment. Also fixed a bug in TestLengthDelimitedJournal because it was resulting in failures when building locally as well.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4134.
2020-03-11 21:00:43 +01:00
Joe Witt 3de77ebacc
NIFI-7021-RC3 prepare for next development iteration 2020-01-19 14:14:40 -05:00
Joe Witt 633408bce7
NIFI-7021-RC3 prepare release nifi-1.11.0-RC3 2020-01-19 14:14:38 -05:00
Andy LoPresto 2cc467eb58
NIFI-3833 Added encrypted flowfile repository implementation.
Added EncryptedSchemaRepositoryRecordSerde.
Refactored CryptoUtils utility methods for repository encryption configuration validation checks to RepositoryEncryptorUtils.
Added FlowFile repo encryption config container.
Added more logging in cryptographic and serialization operations.
Generalized log messages in shared encryption services.
Added encrypted serde factory.
Added marker impl for encrypted WAL.
Moved validation of FF repo encryption config earlier in startup process.
Refactored duplicate property lookup code in NiFiProperties.
Added title case string helper.
Added validation and warning around misformatted encryption repo properties.
Added unit tests.
Added documentation to User Guide & Admin Guide.
Added screenshot for docs.
Added links to relevant sections of NiFi In-Depth doc to User Guide.
Added flowfile & content repository encryption configuration properties to default nifi.properties.

Signed-off-by: Joe Witt <joewitt@apache.org>
Signed-off-by: Mark Payne <markap14@hotmail.com>

This closes #3968.
2020-01-10 10:44:59 -08:00
Joe Witt f8c3d877cf
NIFI-6733 updating to next release version for master branch 2019-11-04 13:31:39 -05:00
Joe Witt 418179f5b2
NIFI-6733-RC3 prepare for next development iteration 2019-10-28 15:13:13 -07:00
Joe Witt b217ae20ad
NIFI-6733-RC3 prepare release nifi-1.10.0-RC3 2019-10-28 15:12:57 -07:00
Mark Payne eb6085a31d
NIFI-6658: Implement new bin/nifi.sh diagnostics command that is responsible for obtaining diagnostic information about many different parts of nifi, the operating system, etc.
This closes #3727.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-09-13 10:45:53 -04:00
Mark Payne 6b17c4b134
NIFI-6613: If LengthDelimitedJournal gets poisoned, log the reason and hold onto it so that it can be included as the cause of subsequent Exceptions that are thrown
This closes #3704.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-09-13 09:02:45 +09:00
Mark Payne 40dcd1577b
NIFI-6410: Addressed race condition in LengthDelimitedJournal in which a Thread could throw an Exception, then another Thread could update the Journal before the first thread closes it. Added unit test to replicate.
This closes #3561
2019-07-02 10:15:52 -04:00
Andy LoPresto e6c843f465
NIFI-6323 Changed URLs for repositories, project description, and mailing lists to use HTTPS.
NIFI-6323 Changed URLs for splunk.artifactoryonline.com to use HTTPS (certificate validity warning in browsers, but command-line connection using openssl s_client is successful).
NIFI-6323 Changed URLs for XMLNS schema locations to use HTTPS (the XMLNS and schema identifier remain http:// because they are not designed to be resolvable).
NIFI-6323 Fixed Maven XML schema descriptor URLs.

This closes #3497
2019-05-29 14:36:40 -04:00
Mark Payne 58a25cfa5a
NIFI-6110: Updated StandardProcessSession such that if we fail to update the FlowFile Repository, we do not decrement claimant counts for any FlowFiles that were removed. Doing so can cause an issue where a FlowFile is removed, then the FlowFileRepo update fails, resulting in the flowfile being rolled back, but after its claimant count is decremented. It will then be processed again, which can result in the same thing, and we'll end up decrementing the claimant count repeatedly. Also updated LengthDelimitedJournal so that if the overflow directory already exists, it does not fail when trying to create the directory and instead just moves on. Updated unit tests to test both of these new fixes and updated DummyRecordSerde to be thread-safe because the TestLengthDelimitedJournal now needs it to be thread safe due to the new unit test that was added.
This closes #3358.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-08 10:32:38 -05:00
joewitt 0e204f3576
NIFI-6029-RC2 prepare for next development iteration 2019-02-16 21:50:35 -05:00
joewitt 45bb53d2aa
NIFI-6029-RC2 prepare release nifi-1.9.0-RC2 2019-02-16 21:50:15 -05:00
Mark Payne 83ac191736
NIFI-5997: If we swap out data, ensure that we do not increment the size of the queue by the size of the data that we failed to swap out. Also, if the FlowFile Repo does not know about a given swap file, do not restore it on restart
This closes #3290.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-04 14:29:17 -05:00
Jeff Storck c0182294ed NIFI-5720-RC3 prepare for next development iteration 2018-10-22 22:16:43 -04:00
Jeff Storck 98aabf2c50 NIFI-5720-RC3 prepare release nifi-1.8.0-RC3 2018-10-22 22:16:23 -04:00
Mark Payne c425bd2880 NIFI-5533: Be more efficient with heap utilization
- Updated FlowFile Repo / Write Ahead Log so that any update that writes more than 1 MB of data is written to a file inside the FlowFile Repo rather than being buffered in memory
 - Update SplitText so that it does not hold FlowFiles that are not the latest version in heap. Doing them from being garbage collected, so while the Process Session is holding the latest version of the FlowFile, SplitText is holding an older version, and this results in two copies of the same FlowFile object

NIFI-5533: Checkpoint

NIFI-5533: Bug Fixes

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2974
2018-10-09 09:18:02 -04:00
Mark Payne 5872eb3c4a NIFI-5331: When checkpointing SequentialAccessWriteAheadLog, if the journal is not healthy, ensure that we roll it over and ensure that if an Exception is thrown when attempting to fsync() or close() the journal, we continue creating a new one.
This closes #2952.
Signed-off-by: Brandon Devries <devriesb@apache.org>
2018-10-04 15:25:53 -04:00
Andy LoPresto f60585a9b6
NIFI-5376 Removed deprecation warnings.
Updated Javadoc for SiteToSiteClient#createTransaction() and HttpClient implementation.
Reverted exception listing in method contract for SiteToSiteClient#createTransaction and HttpClient tion of same.
Reverted import ordering in TestSiteToSiteClient.
Reverted exception listing in TestGetHDFSFileInfo, TestListHDFS, and StandardHttpFlowFileServerProtocol.
Restored @SuppressWarnings annotation and removed unnecessary "public static" keywords from inner classes in SiteToSiteClient.

This closes #2841.

Signed-off-by: Joe Witt <joewitt@apache.org>
2018-07-09 20:45:34 -07:00