Commit Graph

5754 Commits

Author SHA1 Message Date
nagasivanath 425ba63e66
NIFI-6968 Create Connection Model Allows Multiple Adds
This closes #4013.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-02-06 10:32:59 -05:00
Pierre Villard 7503e5f43f
NIFI-7108 - Upgrade com.puppycrawl.tools:checkstyle (#4041)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-02-05 14:13:00 -05:00
Matt Gilman b1b06bf5e1 NIFI-6958:
- Addressing issue causing errors preventing version changes when there is a processor to enable or disable in a sub process group.
2020-02-05 13:49:06 -05:00
Bryan Bende 5d851e6a13
NIFI-7067 Allow a user and group with the same name/identity to exist
This closes #4019
2020-02-05 09:19:18 -05:00
Christian Zügner 98f9b7c033
NIFI-7095: ResetSetRecordSet: handle java.sql.Array Types in normalizeValue method
Some jdbc drivers e.g. Oracle returns java.sql.Array objects for array types, not just Lists.
This commit also handles these cases, and extracts the primitive java arrays out of this jdbc holder class.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4034.
2020-02-04 17:07:44 -05:00
Matthew Burgess dd6bcc7485
NIFI-7054: Add RecordSinkServiceLookup for dynamic sink routing (#4020)
* NIFI-7054: Add RecordSinkServiceLookup for dynamic sink routing

* NIFI-7054: Incorporated review comments
2020-02-03 09:45:08 -05:00
Andrew Lim d80875e6ba
NIFI-7053 Update Toolkit Guide with macOS 10.15 requirements for trus… (#4018)
* NIFI-7053 Update Toolkit Guide with macOS 10.15 requirements for trusted certificates

* Simplified note about trusted certs in macOS 10.15

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-01-31 10:18:05 -08:00
Andy LoPresto d7c29f4637 NIFI-7079 Cleaned up formatting and comments. 2020-01-31 08:58:51 -05:00
Andy LoPresto 95746d346c NIFI-7079 Added SecureHasher interface and Argon2SecureHasher implementation.
Added unit tests.
Fixed JwtService charset references.
Changed FingerprintFactory to mask sensitive values via Argon2 hashing.
Resolved RAT checks.
2020-01-31 08:58:51 -05:00
Aldrin Piri c8af8eaa76
NIFI-7080
* Ensure script files in Docker builds are treated as executable.
* Updating git permissions to reflect executable for both secure.sh scripts.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
Signed-off-by: Kevin Doran <kdoran@apache.org>

This closes #4027.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2020-01-31 01:03:32 -05:00
Matthew Burgess 76e8c51e11
NIFI-7073: This closes #4025. Route to failure when error on PutHDFS file system close
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-30 13:06:46 -05:00
mdayakar 850869c6d2
NIFI-7049 : SFTP processors shouldn't silently try to access known hosts file of the user
Signed-off-by: Arpad Boda <aboda@apache.org>

This closes #4014
2020-01-30 15:15:43 +01:00
Matthew Burgess 04fae9cb5f
NIFI-7069: Add groovy-dateutil module to scripting bundles
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4021.
2020-01-30 14:49:41 +07:00
Mark Payne 37ef10e6fb
NIFI-7076: This closes #4024. Revert "NIFI-6787 - Before: When checking if a load balanced connection queue is full, we compare the totalSize.get() and getMaxQueueSize()."
This reverts commit 7731609582.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-29 15:50:55 -05:00
Andrew Lim aedccb8297 NIFI-7066 Update nifi.analytics.query.interval documented default value in Admin Guide to match nifi.properties value (#4022) 2020-01-27 20:07:21 -05:00
Mark Payne be34767c8a
NIFI-7059: This closes #4007. Fixed bug that results in priorities not properly being set in the SocketLoadBalancedFlowFileQueue. Even though the queue's setPriorities method was called, the underlying may not have contained the localPartition. As a result, when setPriorities() was called, it did not properly delegate that call to . As a result, the queue knew that the Prioritizers were set but the local queue did not apply them. This happened due to a race condition between queue creation and NodeClusterCoordinator learning the Local Node Identifier. Additionally, updated NodeClusterCoordinator so that it will persist its state when it does learn the Local Node Identifier, as that was not being persisted. In testing this, also encounterd a deadlock in a particular Integration Test that was run, around AbstractFlowFileQueue and its synchronization. Because the 'synchronized' keyword synchronizes on 'this' and the concrete implementation also uses 'synchronized' and the result is the same 'this', a deadlock can occur that is difficult to understand. As a result, refactored AbstractFlowFileQueue to instead use read/write locks.
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-22 23:30:41 -05:00
Joe Witt 379c3fbe51
NIFI-7021 wrap up post nifi 1.11.0 release 2020-01-22 22:09:29 -05:00
Joe Witt 535cafdb83
Merge branch 'NIFI-7021-RC3' bringing the Apache NiFi 1.11.0 to master 2020-01-22 21:15:12 -05:00
Bryan Bende 24b846d08a
NIFI-7051 Protect against empty group membership in ShellUserGroupProvider, and add differentiator to id seeding
NIFI-7051 Fixing issue where identity was being used instead of identifier, making a flag to control legacy id behavior, increasing timeout of shell command runner, and changing the NSS system check command to return less info

NIFI-7051 Updating command for getSystemCheck in NSS impl to use getent --version to improve performance

This closes #4003.
2020-01-22 15:51:42 -05:00
nagasivanath 24ef8ba4cb
Update nf-processor-details.js
NIFI-7032:
- Processor Details no longer appears when clicking 'View Processor Details'
- handling the review comments

This closes #3990
2020-01-21 13:54:12 -05:00
Joe Witt 3de77ebacc
NIFI-7021-RC3 prepare for next development iteration 2020-01-19 14:14:40 -05:00
Joe Witt 633408bce7
NIFI-7021-RC3 prepare release nifi-1.11.0-RC3 2020-01-19 14:14:38 -05:00
Joe Witt ed324058c7
NIFI-7021 cacheFile not reliable on builds with latest puppycrawl checkstyle 2020-01-19 13:49:56 -05:00
Joe Witt 67c36f3c3e
NIFI-7021 apache parent pom and common dep maintenance for build 2020-01-19 13:32:37 -05:00
Gardella Juan Pablo cddaac591b
NIFI-7034 Thise closes #4002. Connection leak with JMSConsumer and JMSPublisher
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 12:08:22 -05:00
Kotaro Terada b045c4ce0b
NIFI-6827 This closes #3862. Mismatch in "lib" directories of rpm and non-rpm build
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 12:06:04 -05:00
mans2singh ddaaded08f
NIFI-6861 - This closes #3882. Added missing display name property
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 12:02:03 -05:00
Drew Kerrigan 3015ee39da
NIFI-6944 This closes #3928. fixed NiFiClientUtil.getUrl and modified NiFiClientUtilSpec to test non-default WEB_HTTP_PORT
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 11:45:21 -05:00
Marc Parisi de2a286a7a
NIFI-818: This closes #3926. Initial implementation of NiFi-Accumulo ( https://github.com/phrocker/nifi-accumulo ) with connectors to Apache Accumulo 2.x
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 11:43:19 -05:00
Dirk Ernst van den Biggelaar ab0113664d
NIFI-6997 This closes #3971. connection closing fixed
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 11:09:52 -05:00
Gal Shinder 4fec54debf
NIFI-6996 This closes #3981. Fixed a stackoverflow that occurs when a subscription fails.
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 11:06:37 -05:00
Tamas Palfy b35ad7cd20
NIFI-7039 - This closes #3995. Fix: PublishJMS outgoing flowfile attribute cleanup could lead to ConcurrentModificationException
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 11:04:39 -05:00
Joe Gresock e51ab8c9d6
NIFI-7036: This closes #3993. Adding 'Append to Attributes' to QueryElasticsearchHttp
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 10:58:40 -05:00
Gardella Juan Pablo c1301e196c
NIFI-6915 This closes #3961. Jms Durable non shared subscription is broken
Revert NIFI-4834 enhancement for durable non shared consumers only.

Updated also AbstractJMSProcessor class to be public. The testing are not working
without that change, as org.apache.nifi.jms.processors.PublishJMSIT and
org.apache.nifi.jms.processors.ConsumeJMSIT are not working, as @OnSchedule
method is not called (because it is not public).
The method org.apache.nifi.util.StandardProcessorTestRunner.run(int iterations, boolean stopOnFinish, boolean initialize, long runWait) uses ReflectionUtils.invokeMethodsWithAnnotation which does not call non public
methods.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 10:04:39 -05:00
Joe Witt b205b99668
NIFI-7041 This closes #4000. Ensure that if the permissions arent set by the flowfile or processor property that we dont attempt to set perms on the remote host
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 08:47:58 -05:00
Mark Payne 3d99d02f93
NIFI-7043: This closes #3999. When a Record incorporates its 'inactive fields' the schema should not change if there are no inactive fields (i.e., if the record has not been modified in such a way that any new fields were added to its schema)
NIFI-7043: Account for case where MapRecord.incorporateInactiveFields is called, and there are no inactive fields, but there are updated fields (i.e., fields whose type has changed from the schema).

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-18 21:07:31 -05:00
Joe Witt 0789ec3a6b
NIFI-6882 from plain patch file in JIRA contributed by 'Josh'
Removed tests which created dirs outside target location and which do not appear unit testable.
2020-01-18 21:02:50 -05:00
Matthew Burgess 71226ce077
NIFI-7044: Fixed 'InputStream not closed' issue in PutElasticsearchRecord and DeleteHBaseCells
This closes #3997

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-01-18 10:39:47 -05:00
Nathan Gough 4ec9155cbc
NIFI-6770 - Set validator to Validator.VALID to allow empty password for truststores.
Added no-password keystore for tests
System NiFi truststore now allows a passwordless truststore. Added a unit test to prove this.
Forgot no-password-truststore.jks file for the unit test.
Refactored utility method from CertificateUtils to KeyStoreUtils.
Added utility methods to verify keystore and key passwords.
Added unit tests.
Implemented different keystore and truststore validation logic.
Refactored internal custom validation in StandardSSLContextService.
Added unit test resource for keystore with different key and keystore passwords.
Added unit test to generate passwordless truststore for https://nifi.apache.org for live testing.
Resolved NPE in SSLContext generation in StandardSSLContextService
Added unit test to generate passwordless truststore for localhost for InvokeHTTP testing.
Resolved TrustManagerFactoryImpl initialization error.
Fixed unit test without proper cleanup which caused RAT failures.

Co-authored-by: Andy LoPresto <alopresto@apache.org>

This closes #3823.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-01-17 20:01:06 -08:00
bahlulh f8057d0c2f
NIFI-7042: This closes #3996.Azure Reporting Tasking throwing NullReferenceException
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-17 15:29:40 -05:00
Alex Voitau 3ef62dd118
NIFI-7038 - Update toolkite-guide.doc. Fix typo.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3994.
2020-01-17 18:03:02 +07:00
Andy LoPresto 34f2a592df
NIFI-7023 This closes #3991. Removed SLF4J and Log4J transitive dependencies from Zookeeper so tests log correctly.
Changed template handling.
Added unit tests.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-16 13:00:08 -05:00
Peter Turcsanyi ce050f4ecb
NIFI-7009: Atlas reporting task retrieves only the active flow components
Filter out the deleted components before querying them, instead of retrieving
all the components before filtering.

This closes #3979
2020-01-16 11:31:18 -05:00
Matthew Burgess 91e9e65a5c
NIFI-6543: Added unit tests for lifecycle methods
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3974.
2020-01-16 19:11:35 +07:00
wagggz 48d4e6d14c
Add support for onUnscheduled method
Add support for onUnscheduled method1

Add support for onUnscheduled method

Add support for onUnscheduled method
2020-01-16 19:11:26 +07:00
Joe Witt 23c8234586
NIFI-7031 updating copyright year on NOTICES 2020-01-15 16:10:31 -05:00
Shawn Weeks aa24664801
NIFI-7022 - This closes #3989. Disable Zookeeper Admin Server for Embedded Zookeeper
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-15 14:43:21 -05:00
Andrew Lim cdbcc4725c NIFI-6987 Remove Claim Management section from Admin Guide
This closes #3964

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2020-01-15 09:24:45 -05:00
Matthew Burgess bb699e7497
NIFI-7017: This closes #3988. Fixed PrometheusReportingTask for nested PG status
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-15 08:27:49 -05:00
Tamas Palfy 103325354b NIFI-7015 - ConsumeJMS now supports ObjectMessage, MapMessage and StreamMessage types as well. Added optional ERROR_QUEUE property. Result flowfiles get a 'jms.messagetype' attribute that contains the incoming message type (TextMessage, BytesMessage, ObjectMessage, MapMessage or StreamMessage). 2020-01-14 17:07:44 -05:00