Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,043 Commits 143 Branches 197 Tags 390 MiB
Commit Graph

4043 Commits

Author SHA1 Message Date
Bryan Bende ff7283d65d NIFI-4506 Adding toDate and format functions to record path. This closes #2221. 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-01-03 11:01:33 -05:00
Koji Kawamura 1959586389 NIFI-4729: This closes #2366. ConvertJSONToSQL converted boolean values in reverse 
Signed-off-by: joewitt <joewitt@apache.org>
 2018-01-02 22:01:40 -05:00
U-WOODMARK\johannes.peter 9f919b9b65 NIFI-4583 Restructure nifi-solr-processors 
Make all methods static

NIFI-4583 Restructure nifi-solr-processors R2

This closes #2285.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-01-03 10:10:42 +09:00
Koji Kawamura 84cecfbeea NIFI-4707: Fixed ProcessGroup tree 
- Removed duplicated creation of a ParentProcessGroupSearchNode for the
root ProcessGroup.
- Removed duplicated creation of a ParentProcessGroupSearchNode for each
component inside a ProcessGroup.
- Fixed ProcessGroup id hierarchy.
- Fixed filtering logic.
- Added unit tests for filtering by ProcessGroupId and Remote
Input/Output ports.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2351
 2018-01-02 14:49:00 -05:00
Matthew Burgess 97dc20e2d9 NIFI-4707: Changed process group parent stack to tree 2018-01-02 14:46:48 -05:00
Koji Kawamura d65e6b2563 NIFI-4707: Improved S2SProvenanceReportingTask 
- Simplified consumeEvents method signature
- Refactored ComponentMapHolder methods visibility
- Renamed componentMap to componentNameMap
- Map more metadata from ConnectionStatus for Remote Input/Output Ports
- Support Process Group hierachy filtering
- Throw an exception when the reporting task fails to send provenance
data to keep current provenance event index so that events can be
consumed again
 2018-01-02 14:46:42 -05:00
Matthew Burgess 1f793923a4 NIFI-4707: Build full component map for ID -> Name association in provenance reporting" 
NIFI-4707: Add process group ID/name to S2SProvReportingTask records

NIFI-4707: Added support for filtering provenance on process group ID

NIFI-4707: Fixed support for provenance in Atlas reporting task

NIFI-4707: Refactored common code into reporting-utils, fixed filtering
 2018-01-02 14:46:36 -05:00
Matt Gilman b7c9c88f9f
NIFI-4444: 
- Ensure the /nifi-api/controller redirection filter executes before matching.

This closes # 2358.

Signed-off-by: Andy LoPresto <alopresto.apache@gmail.com>
 2018-01-02 10:01:03 -05:00
Kevin Doran 482f371958
NIFI-4701 Add authorizers.xml support to toolkit. 
Adds authorizers.xml to the files understood by the encrypt-config
tool in the NiFi Toolkit. If enabled, then the sensitive properties
for LdapUserGroupProvider in authorizers.xml will be encrypted.
Also fixes a bug wherein encrypt-config replaces multiple XML nodes
in login-indentity-providers.xml when LdapProvider is not the first
provider listed in the file.
Enable properties in authorizers.xml to be encrypted by the master key.

This closes #2350.

Signed-off-by: Andy LoPresto <alopresto.apache@gmail.com>
 2017-12-31 17:41:04 -05:00
Mark Payne c91d99884a NIFI-4717: Several minor bug fixes and performance improvements around record-oriented processors 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2359
 2017-12-29 10:43:21 -05:00
Mike Moser c59a967623 NIFI-4504, NIFI-4505 added removeAndGet, removeByPatternAndGet, and keySet methods to MapCache API 
cleaned up some warnings on deprecated nifi.stream.io classes

This closes #2284.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2017-12-29 00:23:09 +09:00
Marco Gaido 2fbe922a2b NIFI-2169: This closes #2343. Cache compiled regexp for RouteText 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-26 18:02:02 -05:00
Marco Gaido 353fcdda9c NIFI-3660: This closes #2356. Support schema containing a map with an array value in ConvertAvroToORC 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-26 17:46:35 -05:00
Aldrin Piri 49285d325f NIFI-4719 - Incrementing Template encoding version to 1.2 to reflect changes incorporated in NIFI-3155 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2357.
 2017-12-23 18:15:49 +01:00
Matthew Burgess 14d2291db8 NIFI-4496: Added JacksonCSVRecordReader to allow choice of CSV parser. This closes #2245. 2017-12-22 08:56:29 -05:00
Koji Kawamura 62e388aa4f NIFI-4709 - Fixed ListAzureBlobStorage timestamp precision handling. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2354.
 2017-12-21 15:15:09 +01:00
Koji Kawamura e8f6ff440a NIFI-4714 - Use lowercase hostname for validation. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2355.
 2017-12-21 14:38:36 +01:00
Matthew Burgess febb119fac NIFI-4696 - Add Flowfile attribute EL support and per-table concurrency to PutHiveStreaming 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2342.
 2017-12-20 12:11:38 +01:00
Koji Kawamura 1ee8d16a21
NIFI-4005: Azure Blob Storage SAS support, incorporating review comments. This closes #2353 
- Renamed Azure to AzureStorageUtils.
- Fixed whitespacing in property description.
- Renamed SAS String to SAS Token.
 2017-12-19 10:34:54 -05:00
Andrew Grande 17ddaf6be0
NIFI-4005 Add support for Azure Shared Access Signature (SAS) Tokens. Upgraded the client library from 5.0.0 to 5.2.0 2017-12-19 10:30:45 -05:00
Norito Agetsuma f04ddcf442 NIFI-4539 - ExtractGrok - Add support returning only named captures 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2233.
 2017-12-19 15:49:14 +01:00
Matthew Burgess 612675e428 NIFI-4684 - Added SQL Parameter Attribute Prefix property to ConvertJSONToSQL 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2333.
 2017-12-19 10:32:52 +01:00
Matt Gilman 17718940d7 NIFI-4689 - Ensuring all provenance properties are represented in nifi.properties. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2352.
 2017-12-19 09:44:25 +01:00
Mark Payne 9750cf2fcd NIFI-3709: Added XMLENC to L&N file. Moved inclusion of nifi-atlas-nar to a profile named include-atlas. This closes #2335. 2017-12-18 12:24:09 -05:00
Koji Kawamura fc73c60924 NIFI-3709: Export NiFi flow dataset lineage to Apache Atlas 2017-12-18 12:23:58 -05:00
trixpan 44a3ac9eff NIFI-4706 - This closes #2348. Bump ParCEFone version 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-18 10:13:34 -05:00
Mark Payne 406db4867a NIFI-4702: When we check the next line for matches in Grok Reader, store the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2349
 2017-12-18 09:41:05 -05:00
Josh Anderton f50a07ec88 NIFI-4655: This closes #2347. Fixed NPE in the InvokeHTTP processor caused when an SSLContextService was assigned without keystore properties 
- Added check for keystore properties and only initialized keystore when necessary.
- Added TestInvokeHttpTwoWaySSL test class to test with two-way SSL
- Modified TestInvokeHttpSSL to test with one-way SSL

Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-18 09:11:53 -05:00
Marco Gaido 9217e2fc63 NIFI-4619: Enable expression language on AWSCredentialsProviderControllerService properties 
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2344.
 2017-12-17 12:51:34 -08:00
Marco Gaido 463dcd8812 NIFI-4629: This closes #2345. Put flowfiles without the grouping attribute in the default group 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-15 10:35:13 -05:00
Mike Thomsen 15eeb22116 NIFI-4479 Added DeleteMongo processor. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2295
 2017-12-14 20:18:20 -05:00
joewitt 5b2e2afc17 NIFI-4678 also fixing a few asciidoc warnings 2017-12-14 17:09:12 -05:00
edwarzjl 68016ddbe8 NIFI-4678 This closes #2321. change all the old fasioned headings in docs to modern ones 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-14 17:08:13 -05:00
joewitt a12abc24e5 NIFI-4683 This closes #2339. added a few missing license/notice entries for spark controller service and processor nars, removed dead code, cleaned up logging. This sign off is relevant to commit 52684c3ec3b7c0ee28c6cf64cfd178976996615a 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-14 16:23:41 -05:00
Matthew Burgess 2192138b06 NIFI-4683: Add ability to execute Spark jobs via Livy 2017-12-14 16:23:41 -05:00
jknulst d543cfde25 NIFI-4675 Lifted restriction on demarcator and kafka.key usage together. This closes #2326. 2017-12-14 15:15:13 -05:00
Joe Percivall 48ae4be015 NIFI-4475 Changing the get(batchSize) method in StandardProcessSession so that it checks all connections before returning nothing. This closes #2337. 2017-12-14 15:08:55 -05:00
Koji Kawamura 57947d64cd NIFI-4694: Make PutSFTP Disable Directory Listing a normal property. This closes #2338. 2017-12-14 15:05:40 -05:00
Koji Kawamura b8375a681a NIFI-3377: NiFi RPG errors when switching between transport protocols. This closes #2340. 2017-12-14 15:02:40 -05:00
Kevin Doran 2608351113
NIFI-4667 Fix LDAP Sync Interval 
Corrects time unit conversion for the Sync Interval config property
for LdapUserGroupProvider in authorizers.xml.

Also enforces a minimum value of 10 secs for the Sync Interval to help
catch unintentional misconfigurations, for example users upgrading
from previous versions, where tiny Sync Interval values could be set
as a workaround for NIFI-4667.

This closes #2341
 2017-12-14 15:01:52 -05:00
Mike Thomsen d7347a2dc3 NIFI-4644 Fixed LookupService API to allow for more than String/String lookup pairs. This closes #2304. 2017-12-13 16:04:23 -05:00
Mike Thomsen bfe92b9000 NIFI-3970 added changes from a code review. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2334
 2017-12-11 19:16:34 -05:00
Matthew Burgess 63840377dd NIFI-3970: Add CSVRecordLookupService 2017-12-11 19:14:27 -05:00
Mike Thomsen 59d3c64195 NIFI-4636 Updated GetMongo to support expression language on limit, results per flowfile and batch fields. 
NIFI-4636: Added unit tests
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2289
 2017-12-11 12:49:42 -05:00
Mark Payne f772f2f093 NIFI-4671: This closes #2328. Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-11 11:46:15 -05:00
Jeff Storck 600586d6be NIFI-3366 This closes #2332. Added parent/child flowfile relationship between the incoming flowfile and the files that are moved from the input directory to the output directory. 
Updated to allow tests to check for evaluation of properties that support expression language.
Fixed bug with changeOwner attempting to operate on original file rather than the moved/copied file.
Added license header to MoveHDFSTest.java
Added test for moving a directory of files that contains a subdir, ensuring non-recursive behavior
Added to the description of the processor that it is non-recursive when a directory is used as input.
Added RAT exclude for test resource .dotfile to pom.xml

Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-11 08:41:36 -05:00
Gray Gwizdz 3731fbee88 NIFI-3366 MoveHDFS processor supports expressions language for input and copy operations 
Signed-off-by: Jeff Storck <jtswork@gmail.com>
 2017-12-11 08:41:25 -05:00
Mark Payne c138987bb4 NIFI-4656, NIFI-4680: This closes #2330. Fix error handling in consume/publish kafka processors. Address issue with HortonworksSchemaRegistry throwing RuntimeException when it should be IOException. Fixed bug in ConsumeerLease/ConsumKafkaRecord that caused it to report too many records received 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-08 16:01:14 -05:00
Mark Payne 1fc1d38fd8 NIFI-4562: This closes #2331. If an Exception is thrown when merging FlowFiles, ensure that we remove the 'bundle' flowfile before exiting the method 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-08 15:35:36 -05:00
Mark Payne 3b74d2ddad NIFI-4633: This closes #2327. Ensure that everywhere that a FlowFile is passed into ProcessSession that we used the most up-to-date version of it 
Ensure that when ProcessSession.clone(FlowFile) is called, we obtain the most recent version of the FlowFile before attempting to obtain FlowFile size.

Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-08 15:30:01 -05:00