Commit Graph

5400 Commits

Author SHA1 Message Date
Matt Gilman 8b23929638
NIFI-6786:
- Upgrading nimbus-jose-jwt.

This closes #3830.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-21 11:41:29 -04:00
Andrew Lim e9717c76ac
NIFI-6553 Updated Admin Guide with a section that recommends antivirus scan exclusions
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3793.
2019-10-19 16:41:02 +02:00
Andrew Lim a30eb3a08a NIFI-6457 Add Upgrading NiFi section to Admin Guide (#3818)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-10-18 13:25:48 -07:00
samhjelmfelt ea1becac4f
NIFI-6703: Add Stateless NiFi to CLI
NIFI-6703: Fixed extracted nar directory and marked api as experimental
NIFI-6703: Moving nifi-stateless into nifi-framework
NIFI-6703: Refactored to fix jetty/spring issues
NIFI-6703: checkstyle fix
NIFI-6703: updated to mirror traditional NiFi's bootstrap process and java11 dependency management
NIFI-6703: minor changes
NIFI-6703: Documentation fixes

This closes #3795.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-17 15:46:32 -04:00
Mark Payne b004f1f94c
NIFI-6779: When resolving node identifier, if NodeIdentifier has a different value for hostname/port for web api or cluster protocol, assume that node is correct about itself instead of assuming that Cluster Coordinator knows best about the other node
NIFI-6779: Remove any conflicting Node Identifiers when a new Node ID is encountered

This closes #3819
2019-10-17 11:21:17 -04:00
Andrew Rodriguez cd90b1d3e1
NIFI-6773 This closes #3820. Adding lz4-framed option for compression/decompression
NIFI-6773 Adding two unit tests for lz4-framed compression/decompression
NIFI-6773 Adding compressed lz4 file for decompression test

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-16 17:45:32 -04:00
Mark Bean c4ccd5f505
NIFI-6750: This closes #3810. add FETCH provenance event to FetchFile
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-15 14:52:39 -04:00
Bryan Bende d570fe8154
NIFI-4820 This closes #3813. Improving security configuration for Kafka 2.0 processors
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-15 14:05:25 -04:00
Matthew Burgess 77e9bda0ff
NIFI-6764: Recreate GroovyShell in ExecuteGroovyScript only when necessary
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3803.
2019-10-15 11:55:52 +02:00
Bahlul Haider 266a59d05a
NIFI-6755 Fix broken link to Quartz documentation
Update the links to the quartz documentation in the User Guide section
for cron trigger scheduling.

This closes #3802.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2019-10-14 15:54:03 -04:00
Mark Payne 9dd0dda688
NIFI-6772: Improved the information that is emitted about Garbage Collection when a Diagnostics Dump is performed
This closes #3809
2019-10-14 11:32:47 -04:00
JF Beauvais 08fe648b1c
[NIFI-6374] ConsumeEWS fails when email attachment has no content type - ensure non null content type
This closes #3596.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-10-11 15:10:11 +09:00
Bryan Bende af81afce40
NIFI-5753 Make use of keyPassword optional and only used when keystore is used, bump registry client version
This closes #3126.
2019-10-10 15:09:28 -04:00
Grzegorz Kołakowski 5d65e6aba4
NIFI-5753 Add SSL support to HortonworksSchemaRegistry service
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-10 15:09:16 -04:00
Joe Witt a273ff10f9
NIFI-6758 fixing checkstyle issues 2019-10-10 15:04:16 -04:00
Matt Gilman 7db6ba841f
NIFI-6768:
- Ensuring the parameter context resources are available via the /resources endpoint.

This closes #3808.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-10 11:49:39 -04:00
Endre Zoltan Kovacs 5238dc20b1
NIFI-6758: allow pre-unpacked nar's to be recorded in ExtensionMapping
- fixing resource leak

This closes #3806.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-10 11:36:29 -04:00
Matt Gilman ed1e843609
NIFI-6769:
- Ensuring the users policy listing can handle when the specified policy is unknown.

This closes #3807.
2019-10-10 11:30:14 -04:00
markap14 d64f4fa942 NIFI-6760: When writing/reading the length of a DataFrame, do so usin… (#3801)
* NIFI-6760: When writing/reading the length of a DataFrame, do so using a 4-byte integer instead of a 2-byte short. When using uncompressed data, we know that the length of the DataFrame will be no more than 64 KB so a 2-byte short is sufficient. However, if data is compresed, there's a chance that the compressed form of the data will be larger than the uncompressed form (for example, with random binary data or with encrypted data). In this situation, we can end up overflowing the 2-byte short, which causes the length that is written to be wrong. Using a 4-byte integer avoids this situation.

* NIFI-6760: Fixed unit tests
2019-10-10 10:19:21 -04:00
Matt Gilman 4e8dd6557f
NIFI-6756:
- Fixing issue opening a users policy listing when there is a policy for a specific parameter context.

This closes #3805.

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-10 00:55:47 -04:00
Matt Gilman 99e9010b32
NIFI-6766:
- Ensuring policy label is properly escaped when populating the user's access policy listing.
This closes #3804.

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-10 00:19:22 -04:00
Mark Payne 5414cd5016 NIFI-6759: When encountering Exception during load-balanced connection communications, after closing socket, return from method rather than continuing loop
This closes #3800.
2019-10-09 16:05:44 -04:00
Matt Gilman 9a496fe9d2
NIFI-6751:
- Fixing the identifier on the user table. In a previous task, this was changed to utilize the URI but that does not work with other code interacting with this table.

This closes #3798.
2019-10-09 12:00:22 -04:00
Mark Payne 47f45abdf9
NIFI-6748: Fixed bug in Parameter Contexts' affected components where if a Controller Service referenced a Parameter, any component that references that service should also be considered an affected component but wasn't. Also fixed a bug in how we handled stopping a Processor that was in the STARTING phase.
This closes #3794.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-09 10:20:45 -04:00
Mark Payne e394f6683a
NIFI-6753: Fixed bug where all values being provided to the CSV Writer were String objects, which resulted in the CSV Writer improperly quoting numeric values when the schema indicates that the value is a number. Now, we will only convert the value to a String if the value is not a Number and/or the schema does not indicate a numeric ty type
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3797
2019-10-09 09:54:45 -04:00
Kevin Doran 2d90145842
NIFI-6749 This closes #3792. Fix Swagger for ControllerServiceReferencingComponentDTO
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-08 11:48:10 -04:00
Joe Witt 0f02de6002
NIFI-6733 updating key apache commons dependencies and apache base dependency for build
This closes #3791.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2019-10-07 22:20:48 -04:00
Matthew Burgess b588073cbb
NIFI-6741: Add RecordPath support to scripting components
This closes #3790.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-10-04 17:37:53 -07:00
Peter Turcsanyi ba141690c5
NIFI-6734: Fixed S3 multipart upload in case of SSE S3 and CSE* encryptions.
Removed unnecessary code from S3 CSE* encryptions.
S3 Encryption Service documentation fixes and improvements.
Renamed region property of StandardS3EncryptionService to kms-region.
Renamed Client-side Customer Master Key in StandardS3EncryptionService.
Use Client-side Customer Key on the GUI / documentation (similar to
Server-side Customer Key).
Use C suffix in constants and class names (similar to SSE_C).
Fixed / extended StandardS3EncryptionService validation.
FetchS3Object encryption strategy changes.
Disable SSE S3 and SSE KMS for FetchS3Object. In case of fetching the
S3 object, these strategies are handled implicitly / automatically.
Set the encryption strategy on the fetched FF that was used to store
the S3 object, instead of the one that is used to read the object (eg.
non-encrypted or SSE S3 encrypted objects can be fetched with a CSE client).
Typo fix.

This closes #3787.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-10-04 15:57:06 -07:00
Rob Fellows df90c65246 NIFI-6735 - validate components before restarting processors following parameter context update.
This closes #3782.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-10-04 15:48:09 -04:00
Peter Wicks 555004cdde
NIFI-6684 - Fixing checkstyle violation (#3788)
signed-off by: Peter Wicks <patricker@gmail.com>
2019-10-04 08:41:56 -06:00
Mark Payne 03473b922b NIFI-6001: When adding Controller Services from a Versioned Flow, make a second iteration over any newly added Controller Services and update them again. This is done so that any linkage between the Controller Services happens properly, once all services have been created. 2019-10-03 16:41:55 -04:00
Kotaro Terada d7ca37d065
NIFI-6723: Enrich processor-related and JVM GC metrics in Prometheus Reporting Task
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3771
2019-10-03 13:17:42 -04:00
Mike 02d3b7e92b NIFI-6656 Added a default visibility expression configuration item to… (#3723)
* NIFI-6656 Added a default visibility expression configuration item to the HBase map cache client services.

* NIFI-6656 Fixed validation bug.

* NIFI-6656 Added stylecheck fix.

* NIFI-6656 Deleted shared interface and moved property descriptor into both services.

* NIFI-6656 Removed dependency from hbase api.
2019-10-03 12:53:53 -04:00
Bryan Bende 9c9f9c10a9 NIFI-5910 Retain external service reference when already set to an existing external service
This closes #3783.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-10-03 12:32:46 -04:00
James Cheng f1d35f46ce NIFI-6684 Add more property to Hive3ConnectionPool (#3763)
* NiFi-6684 Add more property to Hive3ConnectionPool

signed-off by: Peter Wicks <patricker@gmail.com>
2019-10-03 07:24:29 -06:00
Pierre Villard b12a9ad446
NIFI-6159 - Add BigQuery processor using the Streaming API
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3394.
2019-10-03 11:37:00 +02:00
Kotaro Terada a0a66839c4
NIFI-6715: Metrics of removed/renamed components continues to remain in PrometheusReportingTask
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3766.
2019-10-03 11:33:49 +02:00
Kotaro Terada 261a395992
NIFI-6716: Prometheus Metrics Server returns 500 when a Remote Process Group used
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3767.
2019-10-03 11:07:58 +02:00
Mark Payne 99cf87c330
NIFI-6736: Create thread on demand to handle incoming request from client for load balancing. This allows us to avoid situations where we don't have enough threads and we block on the server side, waiting for data, when clients are trying to send data in another connection
This closes #3784.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-02 14:29:00 -04:00
Mark Payne 2f6f852915 NIFI-6589: Addressed NPE 2019-10-02 10:55:55 -04:00
Jeff Storck 8d748223ff
NIFI-6275 ListHDFS now ignores scheme and authority when uses "Full Path" filter mode
Updated description for "Full Path" filter mode to state that it will ignore scheme and authority
Added tests to TestListHDFS for listing an empty and nonexistent dirs
Updated TestListHDFS' mock file system to track state properly when FileStatus instances are added, and updated listStatus to work properly with the underlying Map that contains FileStatus instances
Updated ListHDFS' additional details to document "Full Path" filter mode ignoring scheme and authority, with an example
Updated TestRunners, StandardProcessorTestRunner, MockProcessorInitializationContext to support passing in a logger.

NIFI-6275 Updated the "Full Path" filter mode to check the full path of a file with and without its scheme and authority against the filter regex
Added additional documentation for how ListHDFS handles scheme and authority when "Full Path" filter mode is used
Added test case for "Full Path" filter mode with a regular expression that includes scheme and authority

This closes #3483.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-10-02 17:08:37 +09:00
Alessandro D'Armiento 39a258dc38
NIFI-6727 Fixed the MockProcessContext decrypt bug and added unit test.
This closes #3773.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-10-01 12:53:44 -07:00
Joe Witt 8abf330328
Revert NIFI-6703 as it creates startup issues or logs a confusing message to the user. Details on JIRA
This reverts commit 61baa41e7b.
2019-10-01 15:33:07 -04:00
Andy LoPresto c2746fac5f
NIFI-4573 This closes #3460. Refactored logic handling flow XML encryption migration.
Added unit tests.

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-01 13:26:05 -04:00
Troy Melhase c0de26b8d6
NIFI-4573 Catch exceptions during flow content migration. 2019-10-01 13:06:45 -04:00
Koji Kawamura 6541eac625
NIFI-6598 Storing peers into managed-state
- Fixed checkstyle errors.
- Added PeerPersistence interface.
- Expose RemoteProcessGroup state via REST API
- Made stateManager transient.

This closes #3677.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-01 09:56:10 -04:00
Jeff Storck 2f3ed5c40c NIFI-6694 Removed erroneous inclusion of the Hive 3 component on the JDK 8 ja_JP locale build stage on Travis
This closes #3781
2019-10-01 00:06:58 -04:00
samhjelmfelt 61baa41e7b
NIFI-6703: This closes #3762. Add Stateless NiFi to CLI
NIFI-6703: Fixed extracted nar directory and marked api as experimental

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-09-30 23:23:15 -04:00
Mark Payne fbd6200ab3
NIFI-6589: This closes #3670. Cache results from zookeeper when determining the leader
NIFI-6589: Updated CuratorLeaderElectionManager to cache results for no more than 5 seconds per review feedback

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-09-30 16:47:27 -04:00