Commit Graph

5903 Commits

Author SHA1 Message Date
Pierre Villard 958d9c4cb6
NIFI-7493 - fix checkstyle issue 2020-07-17 12:22:47 +02:00
Vasily Makarov 950437b2d7
NIFI-6603 Add cancel button to the variables.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4366.
2020-07-17 11:54:57 +02:00
Mike Thomsen b8d51333d5
NIFI-7582 Added profile builds for kafka 0.9 and 0.10 so they can be removed from the convenience binary.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4364.
2020-07-17 11:38:42 +02:00
Mark Payne 7e09e0db33
NIFI-7493: When inferring schema for XML data, if we find a text element that also has attributes, infer it as a Record type, in order to match how the data will be read when using the XML Reader
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4375.
2020-07-17 11:35:10 +02:00
Peter Turcsanyi 21c3085d15
NIFI-7654: Deprecated Client Auth property on AMQP processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4412.
2020-07-17 11:29:49 +02:00
Bryan Bende f6c2824f74
NIFI-7608 Add CLI command to replace contents of a process group
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4392.
2020-07-17 11:21:28 +02:00
Mark Payne 44fc4d9f27
NIFI-7552: When Process Group is configured to transfer data in batch, add an attribute to each outbound FlowFile that indicates how many FlowFiles went to each port. Updated user guide to explain the new attributes.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4345.
2020-07-17 10:33:59 +02:00
Joe Witt 77078a85d9
NIFI-7304 resolving a contrib check issue with line length 2020-07-14 13:30:53 -07:00
markap14 f8a730568f
NIFI-7627: Fixed typo that showed the wrong case for Run Statuses in the ProcessorRunStatusDetailsDTO (#4400) 2020-07-14 15:44:52 -04:00
Andy LoPresto dbee774c5b
NIFI-7304 Removed default value for nifi.web.max.content.size.
Added Bundle#toString() method.
Refactored implementation of filter addition logic.
Added logging.
Added unit tests to check for filter enablement.
Introduced content-length exception handling in StandardPublicPort.
Added filter bypass functionality for framework requests in ContentLengthFilter.
Updated property documentation in Admin Guide.
Renamed methods & added Javadoc to clarify purpose of filters in JettyServer.
Cleaned up conditional logic in StandardPublicPort.
Moved ContentLengthFilterTest to correct module.
Refactored unit tests for accuracy and clarity.
Fixed remaining merge conflict due to method renaming.

Signed-off-by: Joe Witt <joe.witt@gmail.com>
2020-07-14 10:42:00 -07:00
Andy LoPresto 0fa8776f4d
NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and documentation.
Fixed unit test failures introduced from static imports during refactoring.

Signed-off-by: Joe Witt <joe.witt@gmail.com>
2020-07-14 10:39:28 -07:00
Peter Gyori 4f11e36260 NIFI-7635: StandardConfigurationContext.getProperty() gets the property descriptor and its default value from the component itself
This closes #4408.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-07-14 13:20:59 -04:00
Tamás Bunth b1e77019ac NIFI-6128 UnpackContent: Store unpacked file data
Tar format allows us to archive files with their original permission,
owner, group name and last modification time.

When unpacking with Tar unpacker, these information are stored in new
attributes with names: "file.inner.*". This way, it preserves backward
compatibility when using parallel with GetFile processor (which stores
information in "file.*").

NIFI-6128 Tar unpackContent: assert date of last modification of content is a valid date format.

NIFI-6128 UnpackContent: use original attributes

In case of tar format:
- Use "file.*" attributes instead of "file.inner.*" (which eventually
  lead to overwrite if tar had been fetched with GetFile)
- Store file permission in "rwx" format, instead of integer
  representation.
- Also replace SimpleDateFormat with DateTimeFormatter.

Replace generic error with IllegalArgumentException
Also refactor: move permission string to top of file as static constant.

Update test

Remove 'file systems' text

Unpackcontent: Fill file.creationTime attribute
which always holds the very same value as file.lastModifiedTime.

This closes #4370.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-14 18:04:56 +02:00
Tamás Bunth 72666a9c7e NIFI-7589 Fix path value when unpacking tar
Refactor: remove variable reassignment

This closes #4371.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-14 17:32:51 +02:00
Mike bb5e9fb00b
NIFI-7497 Removed a few style check bugs that crept up in the last commit. (#4402) 2020-07-13 09:34:24 -04:00
Cuong Nguyen c7a235f854 NIFI-7629: Make property PROJECT_ID evaluate EL on processors ConsumeGCPubSub & PublishGCPubSub 2020-07-13 08:54:41 -04:00
Tamas Palfy 19a84dd3dd NIFI-7581 Added ControllerService-based credential settings support for ADLS processors
NIFI-7581 Separated Controller Service for providing Azure credentials for ADLS (ADLSCredentialsControllerService) form the one that does the same for Blob storages (AzureStorageCredentialsDetails). (This was done due to the considerable difference in the APIs of the libraries used to connect to both.)
NIFI-7581 Fix: Register controller service in META-INF. Minor fixes.
NIFI-7581 Minor changes (documentation, type etc.)
NIFI-7581 Updated integration tests.
NIFI-7581 Minor changes (renaming).

This closes #4369.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-12 11:56:31 +02:00
Mike Thomsen 8d53f5d0c9
NIFI-7497 Removed a few style check bugs that crept up in the last commit. 2020-07-10 17:12:27 -04:00
Peter Turcsanyi ae877b9908
NIFI-7591: Allow PutS3Object to post to AWS Snowball
Added properties to enable/disable chunked encoding and path-style access
for endpoints that do not support chunked encoding / only support path-style access.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4386.
2020-07-10 18:24:44 +02:00
Mark Payne 1fc25db47c
NIFI-7469: Updated RepositoryRecord to include flag indicating whether or not the content of a FlowFile was modified. This allows us to explicitly keep track of this state rather than implying it (potentially incorrectly).
This closes #4399.
2020-07-10 10:44:51 -04:00
neptunesalt ee91341ec3
NIFI-7497 Adding support for AWS Credentials Assume Role to be able to set the STS Endpoint
NIFI-7497 Updating property description per comment

This closes #4309

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-10 09:19:58 -04:00
Chad Zobrisky b0494603da
NIFI-7610 Updated dockerhub readme to latest nifi version
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4394
2020-07-09 12:17:32 -04:00
Tamas Palfy ad95287e78
NIFI-6934 In PutDatabaseRecord added DatabaseAdapter-based UPSERT support for Postgres (9.5+)
NIFI-6934 Added more documentation and unit tests.

NIFI-6934 Added missing license for new test class.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4350
2020-07-09 11:26:41 -04:00
Andy LoPresto ca384b3f3b
NIFI-7620 Fixed failing unit test due to timeout. (#4395) 2020-07-09 09:14:34 -04:00
Mark Payne 6cece9cad7
NIFI-7572: Added ScriptedTransformRecord processor. Addressed a couple of minor bugs in mock classes that were encountered during testing.
NIFI-7572: Addressed review feedback
NIFI-7572: Fixed bug that resulted in constantly recompiling Jython script. Updated documentation showing performance difference. Fixed problematic unit tests for TestResizeImage

This closes #4374

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-08 15:52:06 -04:00
markap14 e17db80514
NIFI-7607: Fixed bug that caused the wrong Controller Service to be de-referenced when a component that used to reference a Controller Service is changed to reference a different Controller Service (#4391) 2020-07-07 16:38:42 -04:00
mtien 43fb57e7bb
NIFI-7332 Added method to log available claim names from the ID provider response when the OIDC Identifying User claim is not found. Revised log message to print available claims.
Added new StandardOidcIdentityProviderGroovyTest file.
Updated deprecated methods in StandardOidcIdentityProvider. Changed log output to print all available claim names from JWTClaimsSet. Added unit test.
Added comments in getAvailableClaims() method.
Fixed typos in NiFi Docs Admin Guide.
Added license to Groovy test.
Fixed a checkstyle error.
Refactor exchangeAuthorizationCode method.
Added unit tests.
Verified all unit tests added so far are passing.
Refactored code. Added unit tests.
Refactored OIDC provider to decouple constructor & network-dependent initialization.
Added unit tests.
Added unit tests.
Refactored OIDC provider to separately authorize the client. Added unit tests.
Added unit tests.

NIFI-7332 Refactored exchangeAuthorizationCode method to separately retrieve the NiFi JWT.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4344.
2020-07-07 15:54:32 -04:00
Matthew Burgess f040c6aadb
NIFI-5213: Allow AvroReader to process files w embedded schema even when the access strategy is explicit schema
NIFI-5213: Incorporated review comments

This closes #2718

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 14:43:57 -04:00
mtien f43dc8063f
NIFI-7542 Upgrade additional jackson-databind versions.
This close #4362

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 08:26:13 -04:00
Joe Witt f8e5be3c09
NIFI-7602 added verify phase so contrib-check can actually execute
This closes #4388

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 07:40:42 -04:00
KovalevIV ea02788de6
NIFI-7603 fix doc
This closes #4387

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 07:35:48 -04:00
mtien 74b4d5ac5f
NIFI-7595 Upgrade log4j-core dependency.
This closes #4382

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 07:28:45 -04:00
Tamas Palfy c396927299 NIFI-7594 In HandleHttpRequest deleting multipart file resources after processing.
This closes #4379.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-06 14:15:43 +02:00
Gardella Juan Pablo e1584207d1 NIFI-7563: Optimize the usage of JMS sessions and message producers
The introduced changes prevent creating unnecesary sessions and producers
in some scenarios.

This closes #4378.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-07-03 16:43:29 -05:00
VKadam 239a2e884c
NIFI-7513 Added custom DNS resolution steps to walkthrough (#4359) 2020-07-01 10:52:22 -07:00
Tamas Palfy 197df577ac NIFI-7590 In 'CassandraSessionProvider.onDisabled' setting Cassandra-related references properly to null after closing them so that they can be renewed in 'onEnabled' (which creates them only if set to 'null', leaving them closed otherwise).
NIFI-7590 Removed 'CassandraSessionProvider.onStopped'.

This closes #4373.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-06-30 21:56:14 +02:00
Andy LoPresto 045027892a
NIFI-7587 This closes #4372. Increased tolerance for non-deterministic unit test.
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-06-30 09:27:43 -07:00
Javi Roman c221e4934d
NIFI-7578 nifi-toolkit CLI Process Group Create command
- Remove unused imports
- Fix checkstyle errors

This closes #4358.
2020-06-30 09:48:45 -04:00
Tamas Palfy c2f46c44ca NIFI-7586 In CassandraSesionProvider added properties to set socket-level read timeout and connect timeout.
In QueryCassandra when writing flowfile to the sesion it's done on the raw OutputStream.
Wrapped it in a BufferedOutputStream for better performance.

This closes #4368.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-06-30 15:21:03 +02:00
mtien 87ec8558a4 NIFI-7577 Update jquery usages.
This closes #4357

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2020-06-26 15:01:30 -04:00
mtien 7df2421fe7 NIFI-7577 Upgrade angular version. 2020-06-26 14:39:47 -04:00
Makarov Vasily a2c88bc98f
NIFI-6163 Reporting task cannot be set to running when in INVALID state
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4334.
2020-06-26 16:48:55 +02:00
Tamás Bunth 91cd74d725 NIFI-7576 ListenHTTP: Honor primary node only
ListenHTTP processor now binds port and creates a HTTP connection only
if one of the following conditions apply:
- Primary node execution is 'false'
- Primary node execution is 'true' and node is elected as primary node.

Changes:
- Connection is established in 'onTrigger' annotated method instead of
  'onSchedule'. (This is similar to how handleHTTPRequest processor
  handles connections.)
- 'onPrimaryNodeStateChange' annotated method is introduced to tear down
  server on reelection of primary node

This closes #4356.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-06-23 14:26:45 -04:00
Peter Turcsanyi 327d73f5cd
NIFI-7523: Use SSL Context Service for Atlas HTTPS connection in Atla… (#4348)
* NIFI-7523: Use SSL Context Service for Atlas HTTPS connection in Atlas reporting task

Also fixing ControllerServiceDisabledException-s when validating the Kerberos config

* NIFI-7523: Fixed test failure on Windows

* NIFI-7523: Added license headers.

* NIFI-7523: Fixed another test failure on Windows

* NIFI-7523: Review changes
2020-06-23 14:22:50 -04:00
mtien 005d05f20b
NIFI-7542 Override jackson-databind version.
NIFI-7542 Override additional jackson-databind versions.
NIFI-7542 Upgrade jackson-databind dependency to 2.9.10.5 in the root pom.xml.

This closes #4343

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-06-22 17:37:57 -04:00
Andy LoPresto 94c98c019f
NIFI-7558 Fixed CatchAllFilter init logic by calling super.init().
Renamed legacy terms.
Updated documentation.

This closes #4351.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-06-22 12:20:28 -07:00
dennisjaheruddin 704260755a
NIFI-7501
Update nf-context-menu.js for an intuitive road to parameters
When rightclicking a process group the variables are shown, but parameters are not. This makes sense as they have a prerequisite, in the form of a parameter context. This change gives a more consistent experience for finding the functionality regarding parameters by ensuring the contextmenu shows the possibility to configure a parameter context. Once the paramater context has been created for a process group, the parameters text shows, so this is no longer visible. People would then need to click configure to change the context, just as they would be required to do now.

Added generateflowfile load tag and description
Added GenerateFlowFile load tag to be consistent with DuplicateFlowFile and updated description to refer to DuplicateFlowFile.

Revert "Update nf-context-menu.js for an intuitive road to parameters"

This reverts commit 3c44b1661f.

This closes #4333

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-06-22 11:12:22 -04:00
Matthew Burgess 32fa9ae51b
NIFI-7516: Catch and log SingularMatrixExceptions in OrdinaryLeastSquares model (#4323) 2020-06-22 09:15:41 -04:00
Mike Thomsen 64b544d5af NIFI-7529 Removed OS and Java information from InvokeHttp's UserAgent field so that it's removed regardless of whether or not this field is kept.
This closes #4332
2020-06-21 12:40:53 -04:00
Mark Payne 57c7883f64
NIFI-7566: Avoid using Thread.sleep() to wait for Site-to-Site connection to be handled. Instead, use TimeUnit.timedWait and use Object.notifyAll when setting the beingServiced flag. This significantly reduces latency and improves throughput for small-batch site-to-site communications
This closes #4353.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-06-19 15:37:02 -07:00