Apache/nifi
1
0
Fork 0
mirror of https://github.com/apache/nifi.git synced 2025-02-12 04:55:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,791 Commits 148 Branches 203 Tags
Commit Graph

3823 Commits

Author SHA1 Message Date
Joe Witt
992d990dd2
NIFI-7291 updated enforcement of dependency rules and build command 
Reviewed by markap14
This closes #4166.

Signed-off-by: Joe Witt <joewitt@apache.org>
 2020-03-27 16:34:59 -04:00
Marc Parisi
1343b00e0b NIFI-7290: omit transitive that is causing the build to fail and not needed for the test scope 2020-03-27 13:12:13 -04:00
Matthew Burgess
be2929b8b0
NIFI-7287: Move services-api dependency from Prometheus reporting task to its NAR 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4162
 2020-03-27 08:48:41 -04:00
Tamas Palfy
c79473baf5 NIFI-7286 ListenTCPRecord cleanup changed from @OnStopped to @OnUnscheduled 2020-03-26 11:45:09 -04:00
Troy Melhase
483f23a8aa
NIFI-7153 Adds ContentLengthFilter to enforce configurable maximum length on incoming HTTP requests. 
Adds DoSFilter to enforce configurable maximum on incoming HTTP requests per second.
Redirected log messages for ContentLengthFilter to nifi-app.log in logback.xml.

This closes #4125.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2020-03-25 17:23:22 -07:00
Bryan Bende
975f2bdc4f
NIFI-7279 This closes #4160. Protect against NPE in RedisDistributedMapCacheClientService when value is null 
Signed-off-by: Joe Witt <joewitt@apache.org>
 2020-03-25 17:40:20 -04:00
Bryan Bende
5784888082
NIFI-7281 This closes #4159. Use BufferedInputStream in StandardSocketChannelRecordReader in order to support mark/reset 
Signed-off-by: Joe Witt <joewitt@apache.org>
 2020-03-25 17:39:44 -04:00
Michael Hogue
aeaf953e06
NIFI-5925: Added controller services to set of components that are searched 
NIFI-5925: cleanup, add negative test

NIFI-5925: fixed checkstyle

This closes #4105

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2020-03-25 08:55:44 -04:00
Bryan Bende
5fd25d6235
NIFI-7278 Adding support for SCRAM-SHA-512 to Kafka 2.0 processors 2020-03-24 16:53:22 -04:00
Mubashir Kazia
9b4292024b
NIFI-7269 - Upgrade solrj version to 7 in nifi-solr-processors 
Remove unused imports

Use the latest solrj version(8.4.1)

Setup default schemaFactory for tests
The default schemaFactory ManagedIndexSchemaFactory creates
additional files in test's resources directory. Change it to
ClassicIndexSchemaFactory for classic behavior.

This closes #4152.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2020-03-24 14:55:51 -04:00
Bryan Bende
6412097eb2
NIFI-7257 Added HadoopDBCPConnectionPool 
- Updated InstanceClassLoader to resolve files that are in the instance urls or additional urls
- Updated nifi-mock to support KerberosContext and removeProperty for ControllerServices
- Added unit test for HadoopDBCPConnectionPool
- Addressing review feedback

This closes #4149.
 2020-03-24 11:05:18 -04:00
Bryan Bende
7f32aa56db
NIFI-7271 Make command timeout configurable for ShellUserGroupProvider 
- Changing ShellRunner to use a separate thread for reading the output of the process
- Removing unused member variable
- Addressing review feedback

This closes #4154.
 2020-03-23 20:43:06 -04:00
Joe Witt
254b20fa5f
NIFI-7274 add time for test conditions to be met 2020-03-22 09:54:43 -04:00
Bryan Bende
2feeb57159
NIFI-7221 Support v2 and v3 protocol version for Hortonworks Schema Registry 
- Update nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/serialization/SchemaRegistryRecordSetWriter.java
- Addressing review feedback

This closes #4120.
 2020-03-20 10:56:47 -04:00
Pierre Villard
1fe79021b5
NIFI-7221 Initial work 2020-03-20 10:56:32 -04:00
Joe Witt
f694e6464f NIFI-7187 adding missing version strings from accumulo bundle pom 
- Removed Cat X JSON.org dep inclusion which seems to not be necessary
- Updated a ton of easier/safer looking deps
- Updated tika due to CVE

This closes #4086

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2020-03-20 10:07:56 -04:00
Eduardo Fontes
943de310ad
NIFI-6293 
Add support to Mongo Extended JSON v2
Add org.json lib
Replace evil json
Replace evil json for alternative
Include testExtendedJsonSupport
Style adjustment
Remove unecessary new JSON parser
Fix query in testExtendedJsonSupport
Parse with Jackson and BSON
Back to default MONGO_URI

This closes #4068

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2020-03-18 18:36:09 -04:00
M Tien
f91d6c420d
NIFI-7268 Removed org.mindrot.jBcrypt library and replaced with at.fa… (#4151) 
* NIFI-7268 Removed org.mindrot.jBcrypt library and replaced with at.favre.lib.bcrypt library.
Updated LICENSE and NOTICE files to reflect changes.
Updated unit tests.

Co-authored-by: Andy LoPresto <alopresto@apache.org>

* NIFI-7268 Fixed typo in Javadoc.

Co-authored-by: Andy LoPresto <alopresto@apache.org>
 2020-03-17 19:49:15 -07:00
Pierre Villard
7105bf36a8
NIFI-7267 - Upgrade spring-data-redis in Redis bundle (#4150) 
Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2020-03-17 15:51:28 -07:00
Pierre Villard
71bf12976f
NIFI-7258 - fix overflow in PutAzureEventHub when not configured correctly 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4146.
 2020-03-16 21:28:12 +01:00
Andy LoPresto
3feb85a030
NIFI-7223 Resolved merge conflicts from additional test case for null key password. 
This closes #4145.

Signed-off-by: Joe Witt <joewitt@apache.org>
 2020-03-16 12:47:41 -04:00
Nathan Gough
69b6c231fd
NIFI-7223 - Added another test for when keyPasswd is not present. 2020-03-16 12:28:18 -04:00
Andy LoPresto
f9d75056fa
NIFI-7223 [WIP] Resolved compilation issues in unit test on OpenJDK 11 by removing Sun security class references. 
Added OkHttpReplicationClient#isTLSConfigured() method.
Added unit test.
NIFI-7223 Fixed remaining unit tests for TLS regression.
Renamed tests for clarity.
 2020-03-16 12:28:01 -04:00
Nathan Gough
7374361b5c
NIFI-7223 - Fixed a minor issue where the OkHttpReplicationClient class loaded blank properties as empty string instead of an expected null value. Added a isNotBlank check. Added unit tests for replication client and HTTPNotificationService. 
NIFI-7223 - Renamed some variables and methods.
NIFI-7223 - Removed unused dependency. Corrected security properties in administration-guide.
 2020-03-16 12:26:50 -04:00
Matthew Burgess
ace5754524 NIFI-7208: Restore default timezone in JdbcCommon 2020-03-13 15:00:24 -04:00
Mark Payne
0ad58200af
NIFI-7256: This closes #4142. Fixed thresholds in unit test. Instead of assuming that multiple runs of the processor will occur within 100 milliseconds, allowed the multiple runs to occur within 3 mins of one another. 
Signed-off-by: Joe Witt <joewitt@apache.org>
 2020-03-13 14:05:40 -04:00
Joe Witt
561be89a21
NIFI-7250 activate user.timezone appropriate to each region 
NIFI-7250 fix a test which appears brittle at least on windows builds on slow environments
NIFI-7250 activated a timezone run for AU Australia/Melbourne which exposed a poor magic number and needless assertion but interesting results worth keeping

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4140.
 2020-03-13 17:30:19 +01:00
Matthew Burgess
798a8eeb50
NIFI-7249: Force String keys in maps in DataTypeUtils.inferDataType() 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4139.
 2020-03-13 17:20:45 +01:00
Adam Taft
decb5d062e
NIFI-6742 Use JUnit TemporaryFolder when creating test databases 
- Add @Rule for TemporaryFolder
- Replace use of previous target/db with TemporaryFolder/db
- Remove use of ~/test db (in home directory)
- Remove System.out lines

Signed-off-by: Marc Parisi <phrocker@apache.org>

This closes #4137.
 2020-03-12 20:48:52 -04:00
Joe Witt
97e250cdaa
NIFI-7244 Updated all tests which dont run well on windows to either work or be ignored on windows 
Also dealt with unreliable tests which depend on timing by ignoring them or converting to IT.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4132.
 2020-03-12 19:13:59 +01:00
Peter Turcsanyi
dfaef38805
NIFI-7248: Atlas reporting task handles PutHive3Streaming 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4138.
 2020-03-12 10:44:48 +01:00
Mark Payne
d4a2afc25c
NIFI-7241: When updating Process Group to match VersionedProcessGroup, remove any connections before recursing into child groups. This ensures that if a Port exists in child group A and is connected to a port in child group B, if the VersionedProcessGroup indicates to remove the port, that connection will be removed before attempting to remove the port. Updating and adding connections must still be done last, after all components have been added. But missing connections can be removed earlier. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4136.
 2020-03-12 10:25:04 +01:00
Mark Payne
d68720920f
NIFI-7242: When a Parameter is changed, any property referencing that parameter should have its #onPropertyModified method called. Also renamed Accumulo tests to integration tests because they start embedded servers and connect to them, which caused failures in my environment. Also fixed a bug in TestLengthDelimitedJournal because it was resulting in failures when building locally as well. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4134.
 2020-03-11 21:00:43 +01:00
Mark Payne
67676ba5b4 Fixed bug in JsonRowRecordReader when reading a 'raw' record with a schema that indicates that a field should be a Map. Also updated unit test to explicitly define schema, since schema inference will never return a Map but rather a Record 2020-03-11 14:29:44 -04:00
Pierre Villard
cd83e70b91 unit test reproducing the issue 2020-03-11 14:29:44 -04:00
Peter Turcsanyi
3de3ad4029
NIFI-7245: JMS processors yield when connection factory initialisation failed 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4133.
 2020-03-11 19:24:10 +01:00
Gardella Juan Pablo
89d8b877f9 NIFI-7050 ConsumeJMS is not yielded in case of exception 
This closes #4004.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
 2020-03-11 10:54:20 +01:00
Peter Turcsanyi
9cb85a4fd0
NIFI-7239: Upgrade the Hive 3 bundle to use Apache Hive 3.1.2 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4129.
 2020-03-11 10:26:10 +01:00
Pierre Villard
a679e88b6f
NIFI-4970 - Add a property to deal with empty GZIP HTTP response 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4127.
 2020-03-11 10:08:51 +01:00
Mark Payne
afad982e91
NIFI-7200: Revert "NIFI-6530 - HTTP SiteToSite server returns 201 in case no data is available" 
This reverts commit f01668e66ad2e45197915769e966a4be27e1592e.

Signed-off-by: Joe Witt <joewitt@apache.org>
 2020-03-10 14:32:43 -04:00
Pierre Villard
12c8402ac3
NIFI-7210 - added PG path in bulletins for S2S Bulletin RT 
Added group path to BULLETIN table for QueryNiFiReportingTask

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4100
 2020-03-09 16:29:22 -04:00
Mike
a33a5e35b3
Merge pull request #4119 from eduardofontes/patch-4 
NIFI-7233 - Fix Mongo Processors to avoid cycle in provenance graph
 2020-03-08 13:18:12 -04:00
Pierre Villard
0f775f3a57
NIFI-7195 - Catch MongoException to route flow files to failure 
This closes #4089

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2020-03-08 08:10:54 -04:00
Pierre Villard
578430c9d9 NIFI-7197 - In-place replacement in LookupRecord processor 
This closes #4088

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2020-03-06 15:35:59 -05:00
Mark Bean
f4b65afb64 NIFI-7231: move controller service validation out of synchronized block for enabling 
This closes #4118.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2020-03-06 15:01:19 -05:00
Eduardo Fontes
57947cd2a2
Fiz writeBatch to avoid cycle in provenance graph 2020-03-06 16:06:21 -03:00
Mark Payne
f283c1191c
Fixed unit test failed with NIFI-7232 
This closes #4117.

Signed-off-by: Joe Witt <joewitt@apache.org>
 2020-03-06 12:46:17 -05:00
Joe Witt
20dda05f26
NIFI-7232 if users do not supply a remote path PutSFTP with conflict resolution will fail with an NPE 2020-03-06 12:46:02 -05:00
Peter Turcsanyi
7c57e75da4 NIFI-7226: Add Connection Factory configuration properties to PublishJMS and ConsumeJMS processors 
Some JMS client libraries may not work with the existing controller services due to incompatible
classloader handling between the 3rd party library and NiFi.
Via configuring the Connection Factory on the processor itself, only the processor's and its
children's classloaders will be used which eliminates the mentioned incompatibility.

This closes #4110.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2020-03-06 09:51:37 -05:00
Joe Witt
040c8a0af9
NIFI-7222 Cleaned up API for FTP/SFTP remote file retrieval and ensure we close remote file resources for SFTP pulls in particular 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4115.
 2020-03-06 12:48:15 +01:00