Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,133 Commits 143 Branches 197 Tags 390 MiB
Commit Graph

115 Commits

Author SHA1 Message Date
Matt Burgess b7dc21bd95 NIFI-2547: Fixed checkstyle errors and updated to reflect KerberosProperties changes 2016-08-21 10:26:40 -04:00
ricky 26d362b144 NIFI-2547: Add DeleteHDFS Processor 
This processor adds the capability to delete files or
directories inside of HDFS.

Paths supports both static and expression language values,
as well as glob support (e.g. /data/for/2016/07/*).

This processor may be used standalone, as well as part of a
downstream connection.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

Add Glob Matcher with Tests

Also set displayName on properties.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #850
 2016-08-21 10:10:21 -04:00
Bryan Bende e0e4b3407a NIFI-2574 This clsoes #887. Fixing NPE when using kerberos keytab location from contexts, and cleaning up hadoop/hbase/hive kerberos variables 2016-08-18 12:19:46 -04:00
joewitt 7d7401add4 NIFI-2574 Changed NiFiProperties to avoid static initializer and updated all references to it. 2016-08-17 00:10:07 -07:00
Bryan Bende fd0dd51ff5 NIFI-2553 Fixing handling of Paths in HDFS processors 
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #843
 2016-08-15 08:59:42 -04:00
Bryan Rosander 2dc094765d NIFI-2516 - Extract version info into parent pom, upgrade to commons-io 2.5 
NIFI-2516 - Removing slf4j-api, version from slf4j-log4j12

This closes #809.
 2016-08-09 22:43:29 +02:00
Mark Payne 0e0166cec5
NIFI-2483: Removed 'passthrough' relationship 
This closes #789.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-08-04 16:12:21 -04:00
Bryan Bende aca2a92d6b
Fixing contrib-check in GetHDFSSequenceFileTest 2016-08-04 15:08:08 -04:00
Mike Moser b7b1dc2fe6 NIFI-2471 fix Hadoop configuration resources when talking to multiple Hadoop clusters 
This closes #779.
 2016-08-04 14:51:03 -04:00
Mark Payne bc6b22389b NIFI-2407: Fixed NPE 2016-08-04 14:50:30 -04:00
Jeff Storck 1ea49ad8cf NIFI-2407 Implements EL support on some properties of processors/services 
- Added EL support to "Directory" property of ListHDFS and GetHDFS processors
 - Added EL support to "Destination" property for ConsumeJMS and PublishJMS processors
 - Added EL support to "MQ ConnectionFactory Implementation", "MQ client library path", "Broker URI" properties for the JMS Connection Factory Provider
 - Added EL support to "Database Connection URL", "Database Driver Class Name", "DB Driver jar url", "DB username", and "DB password" properties for the DBCP Connection Pool
 - Removed NotificationServiceManagerSpec.groovy, previous PR from Joe Witt covers the needed testing for Variable Registry/Value Lookup at this time
This closes #778.
 2016-08-04 14:23:29 -04:00
Bryan Rosander 0d730c5cc1
NIFI-2435 - Wrapping reader call in doAs if necessary 
This closes #780.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-08-04 14:01:05 -04:00
Pierre Villard 8c011d91e1 NIFI-2060 - fixed unit tests to be compatible with 2.7.x 
This closes #547
 2016-08-01 12:46:51 -04:00
Bryan Bende c27763a12f NIFI-1733 Initial commit for nifi-ranger-bundle 
- Adding Authorizer implementation for Ranger
- Adding build profile and assembly that controls the inclusion of Ranger in the final assembly
- Add properties to specify ranger admin identity and a flag to indicate if ranger is using kerberos, plugin is updated to perform a UGI login if ranger is using kerberos
- Adding LICENSE and NOTICE for Ranger NAR, and some other licensing clean up
- Adding tests for kerberos properties, adding test for RangerBasePluginWithPolicies, cleaning up code to use Java 8 features

This closes #574
 2016-07-26 10:15:22 -04:00
joewitt f987b21609 NIFI-1157 searched for and resolved all remaining references to deprecated items that were clearly addressable. 2016-07-14 09:32:35 -04:00
JJ 2ce785766d NIFI-1037 Created a processor for HDFS' inotify events. This processor gets notifications for a select path. 
This closes #493.
 2016-06-17 00:02:28 +02:00
Matt Burgess b32c70c419 NIFI-1929: Improvements for PutHDFS attribute handling 
This closes #486.
 2016-06-06 11:58:55 +02:00
Pierre Villard 372ffb8aa0 NIFI-1811 Removed ProcessorLog and updated dependent interfaces 
This closes #403.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-05-19 13:08:09 -04:00
Pierre Villard 293a3042cd NIFI-1788 Fixed listed allowable values 
NIFI-1788 Added UT

NIFI-1788 modified test name
This closes #368
 2016-04-20 10:48:54 -04:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Bryan Bende 04c6830937 NIFI-1698 Improving customValidate in AbstractHadoopProcessor and HBaseClient service to not reload Configuration unless it changed. This closes #313 2016-04-04 10:14:05 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Bryan Bende 8773ec3d3e NIFI-1488 Refactoring HBase Kerberos support 
- Storing UGI so we can support multiple HBaseClientServices with different configs
- Creating nifi-hadoop-utils to hold utility code shared between HDFS and HBase processors
- Incorporating KerberosProperties into existing hadoop processors

This closes #281
 2016-03-17 17:12:57 -04:00
Mark Payne 56f79e1e85 NIFI-1588: Reworked how ListHDFS store state so that only 2 timestamps are stored and no paths 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-14 15:59:24 -04:00
Mark Payne 8c488d7e8e NIFI-1587: Always poll state from State Manager when running ListHDFS instead of relying on local state over the cluster state 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 14:36:36 -04:00
Oleg Zhurakousky a87230358d NIFI-1536 This closes #241. ensured that UserGroupInformation is re-initialized on each call to 'AbstractHadoopProcessor.resetHDFSResources(..)' 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-22 17:23:53 -05:00
joewitt 86ab4428f0 NIFI-1520 by default skip javadoc and source jar generation in nars and wars 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #234
 2016-02-18 18:45:26 -05:00
Pierre Villard 502aebc66a NIFI-1063 Fixed exception logging 
Fixed logging of an exception in PutHDFS

Reviewed by Tony Kurc (tkurc@apache.org). This closes #226
 2016-02-17 20:59:31 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Tony Kurc 5be83166ee NIFI-1485: Updated 'header' section of NiFi NOTICE files 2016-02-07 13:52:00 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Mark Payne eba25ecaca NIFI-259, NIFI-1339: Added OnConfigurationRestored annotation, always invoke onPropertyModified even on restart when properties are changed from defaults, as was done previously 2016-01-17 14:40:27 -05:00
Mark Payne 0151b1eed1 NIFI-259: Fixed checkstyle violations 2016-01-13 13:57:47 -05:00
Mark Payne bbce596d74 NIFI-259: Updated GetHBase to use new State Management; bug fixes; updated docs 2016-01-13 12:47:08 -05:00
Mark Payne 774c29a4da NIFI-259: Added Stateful annotation as described on ticket 2016-01-12 15:28:35 -05:00
Mark Payne e559c68766 NIFI-259: Began updating processors to use new state management 2016-01-12 10:21:47 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
joewitt d755e43ec8 NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration 2015-12-08 13:00:10 -05:00
joewitt b66c029090 NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2 2015-12-08 12:59:59 -05:00
joewitt ba2719836b NIFI-1267 addressing unused imports due to contrib-check fail 2015-12-07 22:18:31 -05:00
joewitt 10803c81d9 NIFI-1267 needed to remove permissions change 2015-12-07 21:28:07 -05:00
joewitt 3ecccf02de NIFI-1267 removed previous technique to force exception in test 2015-12-07 20:44:21 -05:00
Mark Payne fee9f381a7 NIFI-1267: Extend processor and force an Exception to be thrown from changeOwner instead of relying on permissions to cause failure, since root user has super permissions and as a result flowfiles are not routed to failure 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-07 17:09:17 -05:00
Tony Kurc 3a7ddc6a35 NIFI-1054: Fixed DOS line endings in xml, java and js source files 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-01 22:49:51 -05:00
Joe Percivall fb335ea282 NIFI-1165: Fix for tests TestRouteText and PutHDFS which did not succeed on Windows 
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-11-19 01:01:34 -05:00
Tony Kurc 528dab78d6 NIFI-1073: Fixing coverity discovered errors. Resource leaks, and statics 
Reviewed by Bryan Bende (bbende@apache.org)
 2015-11-14 18:56:43 -05:00
Bryan Bende 33ef59c5ba NIFI-1127 Adding Kerberos properties to FetchHDFS and ListHDFS. 
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-11-12 22:14:12 -05:00
Oleg Zhurakousky a65a29921a NIFI-1124, NIFI-1062 addressed more PR comments 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-11-11 10:12:30 -05:00
Oleg Zhurakousky 70cc61bb53 NIFI-1124, NIFI-1062 addressed PR comments 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-11-11 10:12:30 -05:00
Oleg Zhurakousky 62e3cfc629 NIFI-1024 NIFI-1062 Fixed PutHDFS processor to properly route failures. Ensured that during put failures the FlowFile is routed to 'failure' relationship. Added validation test Re-enabled previously ignored test. 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-11-11 10:12:30 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
Mark Payne 1c1738670c NIFI-10: Updated FetchHDFS, FetchFileTransfer to use new FETCH provenance event 2015-10-26 17:14:29 -04:00
Mark Payne 8a80060851 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-10-25 11:11:49 -04:00
ricky f2c4f2d2a1 [PATCH] NIFI-997 Periodically attempt a kerberos relogin in AbstractHadoopProcessor 
- attempt a relogin based on an interval specified in the processor configuration
 - use hadoop's UserGroupInformation.checkTGTAndReloginFromKeytab to determine if a relogin is necessary based on the ticket and do so if needed
 - improve code readability with HdfsResources object in AbstractHadoopProcessor

Reviewed and Amended by Tony Kurc (tkurc@apache.org). This closes #97
 2015-10-25 00:49:58 -04:00
Mark Payne 4afd8f88f8 NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree 2015-10-07 17:26:14 -04:00
Mark Payne f884086fc2 NIFI-713: Fixed checkstyle issue 2015-09-23 15:20:25 -04:00
Mark Payne da56b49b88 Merge branch 'NIFI-713' of https://github.com/rickysaltzer/nifi into NIFI-713 2015-09-23 12:06:50 -04:00
Matt Gilman ded74ec94c NIFI-876 prepare for next development iteration 2015-09-14 21:48:11 -04:00
Matt Gilman 2ec735e350 NIFI-876 prepare release nifi-0.3.0-RC1 2015-09-14 21:48:00 -04:00
ricky c72fb201d5 NIFI-713: Infer Hadoop Compression Automatically 
- Three main types of compression options:
	NONE 		: no compression
	AUTOMATIC 	: infers codec by extension
	SPECIFIED	: specified codec (e.g. snappy, gzip, bzip, or lz4)
 2015-08-31 14:38:15 -04:00
ricky 7fb6e884a7 NIFI-866: Add Kerberos Support for Hadoop 
- Add krb5.conf to nifi.properties

	nifi.kerberos.krb5.file | path to krb5.conf

- Connections to secure Hadoop clusters will be determined by their config,
  that is, hadoop.security.authentication should be set to kerberos.

- Added two optional arguments to AbstractHadoopProcessor (principal and keytab),
  these are only required if the cluster you're connecting to is secured. Both of
  these options require the krb5.conf to be present in nifi.properties.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-08-24 15:36:22 -04:00
joewitt aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00