Commit Graph

236 Commits

Author SHA1 Message Date
Mark Bathori b375f4b138
NIFI-11124: Add hadoop.file.url attribute to HDFS processors
This closes #6916.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-02-07 23:10:16 +01:00
Peter Gyori 4d3fcb6843
NIFI-11000 Add compression example to CreateHadoopSequenceFile documentation
This closes #6801

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-25 10:33:29 -06:00
Peter Turcsanyi cb77a96059
NIFI-11088: This closes #6878. updated copyright year to 2023 in NOTICE files
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-23 09:42:44 -07:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
Lehel Boér feb3148b68
NIFI-10628: Added Default Schedule Time and Run Duration to processors
This closes #6512

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-21 14:57:46 -05:00
Joe Witt c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt 109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
exceptionfactory 04c0df1723
NIFI-10471 Documented and applied standard deprecation logging
- Updated Admin Guide with section on Logging Configuration and Deprecation Logging
- Updated Developer Guide with section on Deprecating Components and Features
- Added Deprecation Logging when adding components marked with DeprecationNotice to the Flow Configuration
- Added Deprecation Logging on deprecated methods in standard Controller Service implementations
- Removed integration tests for deprecated PersistentProvenanceRepository
- Updated logging and added documentation on minor upgrade steps
- Updated logging for HDFSNarProvider and RocksDB Repository

This closes #6390
Signed-off-by: Paul Grey <greyp@apache.org>
2022-09-13 19:40:03 -04:00
Joe Witt 8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt 8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
exceptionfactory b3b13a4ee0
NIFI-10145 Upgraded Hadoop from 3.3.2 to 3.3.3
- Excluded slf4j-reload4j implementation of Log4j 1
- Updated nifi-hive3-processors to leverage shared Hadoop version from 3.1.1
- Updated nifi-accumulo-bundle to leverage shared Hadoop version from 3.1.1
- Updated nifi-atlas-bundle to leverage shared Hadoop version from 3.3.2
- Updated nifi-spark-receiver to leverage shared Hadoop version from 3.3.2

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6139.
2022-06-28 16:19:38 +02:00
exceptionfactory 0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00
Mike Thomsen 7271e8cea7
NIFI-7234 Standardized on Avro 1.11.0
NIFI-7234 Replaced Jackson 1.X references with Jackson 2.X references in various classes.
NIFI-7234 Added jackson-annotations to nifi-hdfs-processors.
NIFI-7234 Various updates to bring our test cases into better alignment with the Avro specification as of 1.11.
Fixed a checkstyle issue.

NIFI-7234 Made changes requested in a review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5900
2022-04-29 17:48:59 -04:00
exceptionfactory 4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes #5992

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
Peter Turcsanyi 47f75651bb
NIFI-9911: Fixed checking HDFS URI in ListHDFS' Directory property
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5957
2022-04-12 08:50:49 -04:00
Bence Simon 9c71d83c5c NIFI-9615 Extending capabilities of NAR provider with restraining, conflict resolution strategy and refactors to make it more flexible 2022-04-06 11:54:31 -04:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Mike Thomsen 589a1c24d1
NIFI-9124 Refactored nifi-jms-bundle to use JUnit 5
- NIFI-9122 Refactored nifi-influxdb-bundle to use JUnit 5
- NIFI-9121 Refactored nifi-ignite-bundle to use JUnit 5
- NIFI-9120 Refactored nifi-html-bundle to use JUnit 5
- NIFI-9119 Refactored nifi-hl7-bundle to use JUnit 5
- NIFI-9118 Refactored nifi-hive-bundle to use JUnit 5
- NIFI-9117 Refactored nifi-hbase-bundle to use JUnit 5
- NIFI-9116 Refactored nifi-hazelcast-bundle to use JUnit 5
- NIFI-9114 Refactored nifi-hadoop-bundle to use JUnit 5
- Added DisabledOnOs(OS.WINDOWS) to Hive tests with @TempDir from JUnit 5 due to Windows-specific problems in JUnit 5

This closes #5791

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-28 10:39:47 -06:00
exceptionfactory fe1139b8bb
NIFI-9692 Upgraded Apache Commons Lang3 to 3.12.0
- Moved commons-lang3 version management to root Maven configuration
- Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3
- Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5773
2022-02-16 09:28:21 -05:00
exceptionfactory a2f6420f43
NIFI-9638 Refactored Google Guava usage in extensions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5742
2022-02-09 16:39:53 -05:00
exceptionfactory e2243500d3
NIFI-9618 Upgraded Checkstyle to 9.2.1
- Corrected several missing return statements
- Increased Maven heap size to 3 GB for ci-workflow

This closes #5700

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 20:08:16 -06:00
Joe Witt 2f95eed8cb
NIFI-9575 This closes #5667. updating copyright year to 2022
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-01-19 09:14:09 -07:00
exceptionfactory 12ba579b8f NIFI-9539 Upgraded Netty from 4.1.69 to 4.1.72
- Added Netty 4 BOM to root Maven configuration
- Removed individual Netty 4 dependency overrides
- Removed commented dependencies and unnecessary declarations from MiNiFi

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5635.
2022-01-05 18:11:03 -05:00
exceptionfactory 7c6bdcb035
NIFI-9491 Excluded commons-logging and added jcl-over-slf4j references
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5608
2021-12-17 07:21:03 -05:00
markap14 839fbf7d19
NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which H… (#5533)
* NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which Hadoop related processors (and potentially others) can indicate that they need full classloaders to be cloned but can share with other instances in certain circumstances
- Added system tests

* NIFI-9382: Renamed interface based on review feedback

* NIFI-9382: Removed ReentrantKerberosUser.
2021-11-22 14:55:19 -05:00
Joe Witt 9e845d87ff
Merge branch 'NIFI-9344-RC3' 2021-11-07 16:23:10 -07:00
Bence Simon 9b724df6d9 NIFI-9260 Making the 'write and rename' behaviour optional for PutHDFS
This closes #5423.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-11-04 20:19:26 +01:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Paul Grey 362a243e0f
NIFI-9360 Update PutHDFS to handle filesystems which do not support getAclStatus()
This closes #5505

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-02 21:57:03 -05:00
Bence Simon 75d1d7f6e7
NIFI-9265 Fixing path handling for HDFS processors when there are multiplied separators in the path
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5437.
2021-10-28 10:47:53 +02:00
exceptionfactory 60d6d469bf
NIFI-9342 Upgraded to Netty 3.10.6 and 4.1.69
- Replaced Netty 3.6.9 and 3.7.1 with 3.10.6
- Replaced Netty 4.1 with 4.1.69

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5490.
2021-10-28 09:34:31 +02:00
exceptionfactory 4bcd03024a
NIFI-9283 Excluded Log4j 1.2 and upgraded Log4j2 to 2.14.1
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5440.
2021-10-06 11:17:36 -04:00
Paul Grey d6805abf9b
NIFI-9235 - Log conflicts between umask and ACL in PutHDFS
This closes #5409

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-05 13:04:33 -05:00
hondawei 30cf49db7e
NIFI-4542 - add target.dir.created to indicate if the target directory created
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5397.
2021-09-26 11:48:18 +02:00
exceptionfactory f5a421ccb1
NIFI-9067 Upgraded JAXB Runtime from 2.3.0 to 2.3.5
- Replaced old com.sun.xml.bind:jaxb-impl and jaxb-core with current org.glassfish.jaxb:jaxb-runtime
- Replaced old javax.xml.bind:jaxb-api with current jakarta.xml.bind-api
- Removed unnecessary dependency references to javax.activation-api

This closes #5320

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-07 13:27:53 -04:00
Peter Gyori 73062146c6 NIFI-9076: hdfs operations in MoveHDFS wrapped in UGI.doAs()
This closes #5334.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-08-25 19:18:48 +02:00
Bryan Bende 21c2fb95d3
NIFI-8973 Implement KerberosUserService API and keytab, password, and ticket cache implementations
NIFI-8974 Integrate KerberosUserService with HDFS processors

NIFI-8980 Integrate KerberosUserService with Kafka 2.6 processors
- Introduced SelfContainerKerberosUserService to restrict which impls can be used with Kafka
- Add variations of KerberosUser doAs that allow setting the context ClassLoader
- Add additional unit tests for configurations

This closes #5277

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-18 19:54:56 -05:00
Bence Simon c3ecf2fea8 NIFI-9032 Refactoring HDFS processors in order to increase flexibility
This closes #5295.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-08-17 16:57:27 +02:00
Peter Gyori 4477921665
NIFI-8787: Wrapped hdfs.exists() call in UGI.doAs() in GetHDFS processor (#5217)
* NIFI-8787: Wrapped hdfs.exists() call in UGI.doAs() in GetHDFS processor

* NIFI-8787: Unit tests updated
2021-07-16 11:24:14 -04:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
Peter Turcsanyi 39ffa4a8ac
NIFI-8737: Fixed incorrect provenance data in HDFS processors when Directory property is inconsistent with core-site.xml 2021-06-25 14:19:57 +02:00
Bence Simon a3f54fa578
NIFI-8736 Adding capability to override HDFS location for NAR autoloading
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5185.
2021-06-24 19:37:19 +02:00
simonbence 73363d9aea
NIFI-8717 Refactoring PutHDFS (#5175)
NIFI-8717 Refactoring PutHDFS
2021-06-22 13:13:14 -04:00
exceptionfactory f2698a4e1e
NIFI-8723 Upgraded Jackson to 2.12.3
- Updated MiNiFi to reference root jackson.version
- Updated Registry assembly to reference root jackson.version
- Added dependency management in NAR modules to override transitive versions

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5174
2021-06-22 11:21:45 -04:00
exceptionfactory 2efc2b6b8b
NIFI-8718 Upgraded Apache Commons IO to 2.10.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5169
2021-06-21 11:29:27 -04:00
Mark Payne ecacfdaa4c NIFI-8469: Introduced ProcessSession.commitAsync and updated processors to use it. Deprecated ProcessSession.commit()
- Updated Mock Framework to now fail tests that use ProcessSession.commit() unless they first call TestRunner.setAllowSynchronousSessionCommits(true)
- Updated stateless nifi in order to make use of async session commits
- Fixed bug that caused stateless to not properly handle Additional Classpath URLs and bug that caused warnings about validation to get generated when a flow that used controller services was initialized. While this is not really in scope of NIFI-8469, it was found when testing and blocked further progress so addresssed here.
- If Processor fails to progress when run from stateless, trigger from start of flow until that is no longer the case
- Introduced notion of TransactionThresholds that can limit the amount of data that a flow will bring in for a given invocation of stateless dataflow
- Several new system-level tests
2021-05-19 08:51:16 -04:00