Apache/nifi
1
0
Fork 0
mirror of https://github.com/apache/nifi.git synced 2025-02-18 16:14:45 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,984 Commits 147 Branches 203 Tags
Commit Graph

2300 Commits

Author SHA1 Message Date
Mark Payne
b603cb955d NIFI-4060: Initial implementation of MergeRecord 
NIFI-4060: Addressed threading issue with RecordBin being updated after it is completed; fixed issue that caused mime.type attribute not to be written properly if all incoming flowfiles already have a different value for that attribute

NIFI-4060: Bug fixes; improved documentation; added a lot of debug information; updated StandardProcessSession to produce more accurate logs in case of a session being committed/rolled back with open input/output streams
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1958
 2017-07-12 16:36:48 -04:00
Matt Gilman
eefad29167 NIFI-4127: 
- Introducing composite ConfigurableUserGroupProvider and UserGroupProvider.
- Adding appropriate unit tests.
- Updating object model to support per resource (user/group/policy) configuration.
- Updating UI to support per resource (user/group/policy) configuration.
- Adding necessary documentation.
- Updating documentation to clarify integrity checks.
- Providing an example of configuring a composite implementation.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1978.
 2017-07-11 18:13:04 +02:00
Joe Skora
325fe53fa1 NIFI-3677 - ExtractMediaMetadata should close TikaInputStream 
* Added finally block to close TikaInputStream.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1993.
 2017-07-11 18:04:50 +02:00
Jeff Storck
58fc5f24b8
NIFI-1586 Removed check for distributed ZK quorum before starting the DatadirCleanupMananger to enable autopurge during standalone ZK server usage. This closes #2001 2017-07-11 10:04:08 -04:00
mans2singh
0fd51b4d12 NIFI-4145 Added writes attributes annotation 
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1998.
 2017-07-10 21:25:26 -07:00
m-hogue
78fbb8f2ee
NIFI-3939: Reviewed and corrected all incorrect nifi-web-api resource response types. This closes #1999 2017-07-10 14:40:13 -04:00
Mark Payne
87e062ff55
NIFI-4167: StandardResourceClaimManager should not synchronize on a ResourceClaim in order to determine the claim count. This closes #1996 2017-07-10 14:33:07 -04:00
Tony Kurc
2dc45a4dd7 NIFI-4154 Fixing line endings in .java,.html files 
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1982.
 2017-07-09 12:13:15 -07:00
mans2singh
affc88e599 NIFI-4118 RethinkDB bundle with put processor 
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1942.
 2017-07-09 11:10:23 -07:00
Mark Payne
9e296830ab
NIFI-4151: Ensure that we properly call invalidateValidationContext() when properties change; ensure that in the controller service provider we don't replace a controller service with a new node if the ID's match, as we won't be able to actually add the new one to the flow. This closes #1995 2017-07-07 16:41:32 -04:00
Yolanda M. Davis
6df97bbc88
NIFI-4135 - added hadoop-client and enhanced Authorizers entity to support classpath for resources entry 
NIFI-4135 - classpath under class

This closes #1956.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2017-07-07 10:08:48 -04:00
kwydler
e89512e744 NIFI-4161: Adding expression evaluation to AWS PROXY_HOST and PROXY_HOST_PORT property usage 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1990.
 2017-07-07 13:28:33 +02:00
Koji Kawamura
e84f9a2416 NIFI-4160: SFTPTransfer connection timeout for opening channel. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1991.
 2017-07-07 12:31:50 +02:00
Koji Kawamura
50c364a793 NIFI-4162: PutSQL batch update error message should include the cause 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1992.
 2017-07-07 12:25:10 +02:00
Pierre Villard
45f82dc855 NIFI-4111 - NiFi shutdown 
Fixed threads shutdown so that NiFi can shutdown gracefully

NIFI-4111 - Review - Handling SocketRemoteSiteListener (RAW S2S)

This closes #1963.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2017-07-07 14:23:18 +09:00
Mark Payne
3906d4e1d2 NIFI-1763: Initial implementation of ConfluentSchemaRegistry. 
NIFI-1763: Fixed bug where the Confluent Schema Registry Schema Access Writer was not being created

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1938
 2017-07-06 15:52:07 -04:00
Pierre Villard
a3b72f1bb7
NIFI-4143 - externalize MAX_CONCURRENT_REQUESTS. This closes #1962 2017-07-06 15:38:55 -04:00
Mark Payne
ba56774fa1
NIFI-4151: Updated UpdateAttribute to only create JAXB Context once; Minor performance tweaks to standard validators and StatusMerge.prettyPrint; updated AbstractConfiguredComponent to not create a new ValidationContext each time that validate is called but only when needed; updated FlowController, StandardControllerServiceProvider, and StandardProcessGroup so that component lookups can be performed using a ConcurrentMap at FlowController level instead of having to perform a depth-first search through all ProcessGroups when calling findProcessor(), findProcessGroup(), findXYZ() 
This closes #1979
 2017-07-06 13:43:56 -04:00
Mark Payne
cff81c0cd2
NIFI-4153: Use a LinkedBlockingQueue instead of a SynchronousQueue for Request Replicator's thread pool so that requests will queue when all threads are active, instead of throwing an Exception. This closes #1980 2017-07-06 10:11:48 -04:00
m-hogue
e6b166a3a2 NIFI-552: added regex properties for include and ignore filters in LogAttribute 
This closes #1981

Signed-off-by: Tony Kurc <tkurc@apache.org>
 2017-07-05 21:21:50 -04:00
m-hogue
47eece5798 NIFI-3193: added ability to authenticate using cert common names 
This closes #1971.

Signed-off-by: Tony Kurc <tkurc@apache.org>
Also reviewed by Pierre Villard <pierre.villard.fr@gmail.com>
 2017-07-05 15:28:54 -04:00
Pierre Villard
7843b885ee NIFI-4144 - added min/max age to ListHDFS processor 
This closes #1966.

Signed-off-by: Tony Kurc <tkurc@apache.org>
 2017-07-04 21:18:43 -04:00
Pierre Villard
8b5342dec0 NIFI-3974 - Added charset in LogAttribute 
This closes #1967

Signed-off-by: Tony Kurc <tkurc@apache.org>
 2017-07-02 12:03:07 -04:00
Pierre Villard
902b6e205d NIFI-4131 - GFT warning message when badly setting relationships 
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1954
 2017-06-30 15:12:10 -04:00
Mark Payne
7f4cfd51ea NIFI-3897: If swapping data into queue and an unexpected exception/error is thrown, do not lose track of the swap file 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1960.
 2017-06-30 13:53:05 +02:00
Mark Payne
cdc154f7c8 NIFI-4046: If we are unable to parse out any records from a Kafka Mesaage with ConsumeKafkaRecord, then we should route all of the bytes received to 'parse.failure' 
NIFI-4046: Addressed issue of Record Writer failing with ConsumeKafkaRecord

This closes #1906.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2017-06-30 17:56:14 +09:00
m-hogue
58a623dfa2 NIFI-4037 added InvokeGRPC processor, with proto service IDL 
NIFI-4038 added ListenGRPC processor

This closes #1947

Signed-off-by: Tony Kurc <tkurc@apache.org>
 2017-06-29 22:01:22 -04:00
yuri1969
3bf1d12706
NIFI-4125 Added secure transform feature and configuration to TransformXML processor to mitigate XXE file system leaks. 
This closes #1946.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2017-06-29 10:40:33 -07:00
Mike Thomsen
5172797448 NIFI-4122 Added the ability to combine multiple Mongo result documents into a single output JSON array. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1948.
 2017-06-29 12:43:52 +02:00
Sreeram Garlapati
82ef671953 NIFI-4007 - Update azure EventHubs client to latest version 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1880.
 2017-06-29 09:30:39 +02:00
Michael Andre Pearce
bdfd710692 NIFI-3983 - Support ability to make JMS 2.0 durable subscriptions on Topic 
Add logic in Consumer adding support for all topic consumer combinations, non-durable, durable, shared, durable-shared.
Add new optional config option to supply subscription name.
Add new optional config option to supply clientId.

This closes #1863.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2017-06-29 10:23:15 +09:00
Matt Burgess
9bfa7469cb NIFI-4137: Add support for UTF-8 in GetFTP and PutFTP 
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1957
 2017-06-28 16:41:14 -04:00
Matt Gilman
c99c036c20 NIFI-4045: 
- Addressing issues causing the eventId to not be relayed when submitting a lineage request under certain conditions.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1903.
 2017-06-27 21:48:55 +02:00
Andre F de Miranda
1e7eceee84 NIFI-4126 - Add SupportBatching, SideEffectFree and EventDriven annotations to ExtractGrok 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1950.
 2017-06-27 20:56:56 +02:00
yuri1969
202eb5ccbe
NIFI-4119 - Improve UX of canvas label configuration by providing immediate focus to value field 
This closes #1940.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2017-06-27 10:38:35 -07:00
kwydler
c37aed4de2 NIFI-4128 Added "All" MessageAttributeNames to GetSQS processor ReceiveMessageRequest 
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1949.
 2017-06-26 22:25:16 -07:00
Scott Aslan
24e2981012 [NIFI-4117] update connection config prioritizers to be contained within their parent element 
This closes #1944.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
 2017-06-26 20:25:50 +02:00
Steve Champagne
68b42c9e54 NIFI-4055: Add a compression option to AvroRecordSetWriter 
This closes #1909.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
 2017-06-26 19:16:31 +02:00
Koji Kawamura
91ed96f8c4 NIFI-4040: Continue if SFTP failed getting user home directory 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1898.
 2017-06-26 19:09:22 +02:00
Koji Kawamura
69613f29c9
NIFI-4050: Fixed NPE at AbstractConfiguredComponent.validate 
This closes #1905.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2017-06-23 13:58:22 -04:00
Pierre Villard
47165afc0c
NIFI-4073 - fix duplicated stack trace 
This closes #1916.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2017-06-23 11:20:07 -04:00
Mark Payne
8878d732dd NIFI-4103: Return an empty Optional if an IP address cannot be found 
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1933
 2017-06-23 11:14:57 -04:00
Andy LoPresto
ff65b42e88 NIFI-4110 Added Thread.sleep() to ensure provenance event indexing occurs before retrieval and unit test passes. 
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1935
 2017-06-23 08:48:34 -04:00
patricker
7c27d4c588 NIFI-3992: Added Original Headers to Include/Exclude 
Some headers can cause problems with message parsing, specifically the 'Content-Type' header.
If an email contains attachments, ConsumeEWS may generate emails where the attachments cannot be extracted.

This closes #1867.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2017-06-23 17:38:19 +09:00
Andy LoPresto
253ea2e73b NIFI-4095 Changed minimum capture group count in ExtractText from 1 to 0. 
Added unit test and removed obsolete test.
Added custom validation to enforce capture group if "include capture group 0" is false.
 2017-06-22 23:31:35 +02:00
Koji Kawamura
5c755c006b
NIFI-4017: Emit provenance event from Notify. 
This closes #1890.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2017-06-22 16:49:31 -04:00
Pierre Villard
f54e146561
NIFI-4085 Add exception in log message when (S)FTP listing fails 
This closes #1929.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2017-06-22 16:06:18 -04:00
Niels Zeilemaker
5b82b735ab
NIFI-4106 Increase log level of user authentication rejection to warn. 
This closes #1934.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2017-06-22 14:03:25 -04:00
Jeff Storck
668a64cd56 NIFI-1586 Implements autopurging of transaction log and snapshot files 
When NiFi is clustered, and autopurge.purgeInterval is greater than 1, the DatadirCleanupManager will be started in order to automatically purge transaction log and snapshot files based on the autopurge settings in zookeeper.properties

This closes #1928.
 2017-06-22 09:30:38 -04:00
Jeff Storck
c99100c934
NIFI-4010 Enables EL on Fetch/List/PutSFTP and List/Fetch/Put/DeleteHDFS processor properties 
FetchSFTP/ListSFTP/PutSFTP: Private Key Path
ListHDFS/FetchHDFS/PutHDFS/DeleteHDFS: Hadoop Configuration Resources, Kerberos Principal, Kerberos Keytab, Kerberos Relogin Period

This closes #1148
This closes #1930.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2017-06-21 17:14:49 -04:00