Commit Graph

4048 Commits

Author SHA1 Message Date
Matthew Burgess e439cfef10 NIFI-4389: This closes #2368. Add Ivy to scripting NARs for Grape/Grab in Groovy
Signed-off-by: joewitt <joewitt@apache.org>
2018-01-04 10:20:47 -05:00
Matt Gilman a2a8fcb31c NIFI-4092: This closes #2365.
- Removing direct dependency on jaxb.

Signed-off-by: joewitt <joewitt@apache.org>
2018-01-04 09:38:35 -05:00
joewitt 6ee6b5e57c NIFI-4734 refactored couchbase bundle, jms bundle, removed jms test-lib usage, fixed broken SeeAlso and service links causing docs warns. Corrected L&N in each bundle. This closes #2369.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-01-04 09:16:17 -05:00
Jeff Storck 42a1ee011b NIFI-4323 This closes #2360. Wrapped Get/ListHDFS hadoop operations in ugi.doAs calls
NIFI-3472 NIFI-4350 Removed explicit relogin code from HDFS/Hive/HBase components and updated SecurityUtils.loginKerberos to use UGI.loginUserFromKeytab. This brings those components in line with daemon-process-style usage, made possible by NiFi's InstanceClassloader isolation.  Relogin (on ticket expiry/connection failure) can now be properly handled by hadoop-client code implicitly.
NIFI-3472 Added default value (true) for javax.security.auth.useSubjectCredsOnly to bootstrap.conf
NIFI-3472 Added javadoc explaining the removal of explicit relogin threads and usage of UGI.loginUserFromKeytab
Readded Relogin Period property to AbstractHadoopProcessor, and updated its documentation to indicate that it is now a deprecated property
Additional cleanup of code that referenced relogin periods
Marked KerberosTicketRenewer is deprecated

NIFI-3472 Cleaned up imports in TestPutHiveStreaming
2018-01-03 11:31:47 -05:00
joewitt 7467bb7b0f
NIFI-4730 restructure of nifi-gcp-bundle
This closes #2367.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-01-03 11:21:52 -05:00
Bryan Bende ff7283d65d NIFI-4506 Adding toDate and format functions to record path. This closes #2221.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-01-03 11:01:33 -05:00
Koji Kawamura 1959586389 NIFI-4729: This closes #2366. ConvertJSONToSQL converted boolean values in reverse
Signed-off-by: joewitt <joewitt@apache.org>
2018-01-02 22:01:40 -05:00
U-WOODMARK\johannes.peter 9f919b9b65 NIFI-4583 Restructure nifi-solr-processors
Make all methods static

NIFI-4583 Restructure nifi-solr-processors R2

This closes #2285.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-01-03 10:10:42 +09:00
Koji Kawamura 84cecfbeea NIFI-4707: Fixed ProcessGroup tree
- Removed duplicated creation of a ParentProcessGroupSearchNode for the
root ProcessGroup.
- Removed duplicated creation of a ParentProcessGroupSearchNode for each
component inside a ProcessGroup.
- Fixed ProcessGroup id hierarchy.
- Fixed filtering logic.
- Added unit tests for filtering by ProcessGroupId and Remote
Input/Output ports.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2351
2018-01-02 14:49:00 -05:00
Matthew Burgess 97dc20e2d9 NIFI-4707: Changed process group parent stack to tree 2018-01-02 14:46:48 -05:00
Koji Kawamura d65e6b2563 NIFI-4707: Improved S2SProvenanceReportingTask
- Simplified consumeEvents method signature
- Refactored ComponentMapHolder methods visibility
- Renamed componentMap to componentNameMap
- Map more metadata from ConnectionStatus for Remote Input/Output Ports
- Support Process Group hierachy filtering
- Throw an exception when the reporting task fails to send provenance
data to keep current provenance event index so that events can be
consumed again
2018-01-02 14:46:42 -05:00
Matthew Burgess 1f793923a4 NIFI-4707: Build full component map for ID -> Name association in provenance reporting"
NIFI-4707: Add process group ID/name to S2SProvReportingTask records

NIFI-4707: Added support for filtering provenance on process group ID

NIFI-4707: Fixed support for provenance in Atlas reporting task

NIFI-4707: Refactored common code into reporting-utils, fixed filtering
2018-01-02 14:46:36 -05:00
Matt Gilman b7c9c88f9f
NIFI-4444:
- Ensure the /nifi-api/controller redirection filter executes before matching.

This closes # 2358.

Signed-off-by: Andy LoPresto <alopresto.apache@gmail.com>
2018-01-02 10:01:03 -05:00
Kevin Doran 482f371958
NIFI-4701 Add authorizers.xml support to toolkit.
Adds authorizers.xml to the files understood by the encrypt-config
tool in the NiFi Toolkit. If enabled, then the sensitive properties
for LdapUserGroupProvider in authorizers.xml will be encrypted.
Also fixes a bug wherein encrypt-config replaces multiple XML nodes
in login-indentity-providers.xml when LdapProvider is not the first
provider listed in the file.
Enable properties in authorizers.xml to be encrypted by the master key.

This closes #2350.

Signed-off-by: Andy LoPresto <alopresto.apache@gmail.com>
2017-12-31 17:41:04 -05:00
Mark Payne c91d99884a NIFI-4717: Several minor bug fixes and performance improvements around record-oriented processors
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2359
2017-12-29 10:43:21 -05:00
Mike Moser c59a967623 NIFI-4504, NIFI-4505 added removeAndGet, removeByPatternAndGet, and keySet methods to MapCache API
cleaned up some warnings on deprecated nifi.stream.io classes

This closes #2284.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-12-29 00:23:09 +09:00
Marco Gaido 2fbe922a2b NIFI-2169: This closes #2343. Cache compiled regexp for RouteText
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-26 18:02:02 -05:00
Marco Gaido 353fcdda9c NIFI-3660: This closes #2356. Support schema containing a map with an array value in ConvertAvroToORC
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-26 17:46:35 -05:00
Aldrin Piri 49285d325f NIFI-4719 - Incrementing Template encoding version to 1.2 to reflect changes incorporated in NIFI-3155
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2357.
2017-12-23 18:15:49 +01:00
Matthew Burgess 14d2291db8 NIFI-4496: Added JacksonCSVRecordReader to allow choice of CSV parser. This closes #2245. 2017-12-22 08:56:29 -05:00
Koji Kawamura 62e388aa4f NIFI-4709 - Fixed ListAzureBlobStorage timestamp precision handling.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2354.
2017-12-21 15:15:09 +01:00
Koji Kawamura e8f6ff440a NIFI-4714 - Use lowercase hostname for validation.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2355.
2017-12-21 14:38:36 +01:00
Matthew Burgess febb119fac NIFI-4696 - Add Flowfile attribute EL support and per-table concurrency to PutHiveStreaming
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2342.
2017-12-20 12:11:38 +01:00
Koji Kawamura 1ee8d16a21
NIFI-4005: Azure Blob Storage SAS support, incorporating review comments. This closes #2353
- Renamed Azure to AzureStorageUtils.
- Fixed whitespacing in property description.
- Renamed SAS String to SAS Token.
2017-12-19 10:34:54 -05:00
Andrew Grande 17ddaf6be0
NIFI-4005 Add support for Azure Shared Access Signature (SAS) Tokens. Upgraded the client library from 5.0.0 to 5.2.0 2017-12-19 10:30:45 -05:00
Norito Agetsuma f04ddcf442 NIFI-4539 - ExtractGrok - Add support returning only named captures
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2233.
2017-12-19 15:49:14 +01:00
Matthew Burgess 612675e428 NIFI-4684 - Added SQL Parameter Attribute Prefix property to ConvertJSONToSQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2333.
2017-12-19 10:32:52 +01:00
Matt Gilman 17718940d7 NIFI-4689 - Ensuring all provenance properties are represented in nifi.properties.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2352.
2017-12-19 09:44:25 +01:00
Mark Payne 9750cf2fcd NIFI-3709: Added XMLENC to L&N file. Moved inclusion of nifi-atlas-nar to a profile named include-atlas. This closes #2335. 2017-12-18 12:24:09 -05:00
Koji Kawamura fc73c60924 NIFI-3709: Export NiFi flow dataset lineage to Apache Atlas 2017-12-18 12:23:58 -05:00
trixpan 44a3ac9eff NIFI-4706 - This closes #2348. Bump ParCEFone version
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-18 10:13:34 -05:00
Mark Payne 406db4867a NIFI-4702: When we check the next line for matches in Grok Reader, store the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2349
2017-12-18 09:41:05 -05:00
Josh Anderton f50a07ec88 NIFI-4655: This closes #2347. Fixed NPE in the InvokeHTTP processor caused when an SSLContextService was assigned without keystore properties
- Added check for keystore properties and only initialized keystore when necessary.
- Added TestInvokeHttpTwoWaySSL test class to test with two-way SSL
- Modified TestInvokeHttpSSL to test with one-way SSL

Signed-off-by: joewitt <joewitt@apache.org>
2017-12-18 09:11:53 -05:00
Marco Gaido 9217e2fc63 NIFI-4619: Enable expression language on AWSCredentialsProviderControllerService properties
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2344.
2017-12-17 12:51:34 -08:00
Marco Gaido 463dcd8812 NIFI-4629: This closes #2345. Put flowfiles without the grouping attribute in the default group
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-15 10:35:13 -05:00
Mike Thomsen 15eeb22116 NIFI-4479 Added DeleteMongo processor.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2295
2017-12-14 20:18:20 -05:00
joewitt 5b2e2afc17 NIFI-4678 also fixing a few asciidoc warnings 2017-12-14 17:09:12 -05:00
edwarzjl 68016ddbe8 NIFI-4678 This closes #2321. change all the old fasioned headings in docs to modern ones
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-14 17:08:13 -05:00
joewitt a12abc24e5 NIFI-4683 This closes #2339. added a few missing license/notice entries for spark controller service and processor nars, removed dead code, cleaned up logging. This sign off is relevant to commit 52684c3ec3b7c0ee28c6cf64cfd178976996615a
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-14 16:23:41 -05:00
Matthew Burgess 2192138b06 NIFI-4683: Add ability to execute Spark jobs via Livy 2017-12-14 16:23:41 -05:00
jknulst d543cfde25 NIFI-4675 Lifted restriction on demarcator and kafka.key usage together. This closes #2326. 2017-12-14 15:15:13 -05:00
Joe Percivall 48ae4be015 NIFI-4475 Changing the get(batchSize) method in StandardProcessSession so that it checks all connections before returning nothing. This closes #2337. 2017-12-14 15:08:55 -05:00
Koji Kawamura 57947d64cd NIFI-4694: Make PutSFTP Disable Directory Listing a normal property. This closes #2338. 2017-12-14 15:05:40 -05:00
Koji Kawamura b8375a681a NIFI-3377: NiFi RPG errors when switching between transport protocols. This closes #2340. 2017-12-14 15:02:40 -05:00
Kevin Doran 2608351113
NIFI-4667 Fix LDAP Sync Interval
Corrects time unit conversion for the Sync Interval config property
for LdapUserGroupProvider in authorizers.xml.

Also enforces a minimum value of 10 secs for the Sync Interval to help
catch unintentional misconfigurations, for example users upgrading
from previous versions, where tiny Sync Interval values could be set
as a workaround for NIFI-4667.

This closes #2341
2017-12-14 15:01:52 -05:00
Mike Thomsen d7347a2dc3 NIFI-4644 Fixed LookupService API to allow for more than String/String lookup pairs. This closes #2304. 2017-12-13 16:04:23 -05:00
Mike Thomsen bfe92b9000 NIFI-3970 added changes from a code review.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2334
2017-12-11 19:16:34 -05:00
Matthew Burgess 63840377dd NIFI-3970: Add CSVRecordLookupService 2017-12-11 19:14:27 -05:00
Mike Thomsen 59d3c64195 NIFI-4636 Updated GetMongo to support expression language on limit, results per flowfile and batch fields.
NIFI-4636: Added unit tests
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2289
2017-12-11 12:49:42 -05:00
Mark Payne f772f2f093 NIFI-4671: This closes #2328. Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-11 11:46:15 -05:00