- Removing needClientAuth property since cluster comms now requires two way ssl. Jetty client auth settings are based on configured features.
- Removing dead code.
- Updating documentation.
- Removing references to needClientAuth property in all test resources.
- Removing overloaded util method with strict parameter.
This closes#3102.
- Updated FlowFile Repo / Write Ahead Log so that any update that writes more than 1 MB of data is written to a file inside the FlowFile Repo rather than being buffered in memory
- Update SplitText so that it does not hold FlowFiles that are not the latest version in heap. Doing them from being garbage collected, so while the Process Session is holding the latest version of the FlowFile, SplitText is holding an older version, and this results in two copies of the same FlowFile object
NIFI-5533: Checkpoint
NIFI-5533: Bug Fixes
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2974
Refactoring StandardFlowFileQueue to have an AbstractFlowFileQueue
Refactored more into AbstractFlowFileQueue
Added documentation, cleaned up code some
Refactored FlowFileQueue so that there is SwappablePriorityQueue
Several unit tests written
Added REST API Endpoint to allow PUT to update connection to use load balancing or not. When enabling load balancing, though, I saw the queue size go from 9 to 18. Then was only able to process 9 FlowFiles.
Bug fixes
Code refactoring
Added integration tests, bug fixes
Refactored clients to use NIO
Bug fixes. Appears to finally be working with NIO Client!!!!!
NIFI-5516: Refactored some code from NioAsyncLoadBalanceClient to LoadBalanceSession
Bug fixes and allowed load balancing socket connections to be reused
Implemented ability to compress Nothing, Attributes, or Content + Attributes when performing load-balancing
Added flag to ConnectionDTO to indicate Load Balance Status
Updated Diagnostics DTO for connections
Store state about cluster topology in NodeClusterCoordinator so that the state is known upon restart
Code cleanup
Fixed checkstyle and unit tests
NIFI-5516: Updating logic for Cluster Node Firewall so that the node's identity comes from its certificate, not from whatever it says it is.
NIFI-5516: FIxed missing License headers
NIFI-5516: Some minor code cleanup
NIFI-5516: Adddressed review feedback; Bug fixes; some code cleanup. Changed dependency on nifi-registry from SNAPSHOT to official 0.3.0 release
NIFI-5516: Take backpressure configuration into account
NIFI-5516: Fixed ConnectionDiagnosticsSnapshot to include node identifier
NIFI-5516: Addressed review feedback
This closes#2947
Refactored some test code to be clearer.
Renamed some resources to be consistent across modules.
Changed passwords to meet new minimum length requirements.
This closes#3018
NIFI-5540 Added failing unit test and ignored regression test to demonstrate missing default sensitive properties key in flow election encryptor creation.
NIFI-5540 Added equality logic to StringEncryptor and utility equality methods to CryptoUtils.
NIFI-5540 Added default sensitive properties key population logic and log warning to StringEncryptor.
NIFI-5540 Cleaned up formatting.
NIFI-5540 Cleaned up boolean logic.
NIFI-5540 Added Javadoc to StringEncryptor.
NIFI-5540 Added unit test for StringEncryptor#equals().
NIFI-5540 Added performance benchmarking unit tests for constantTimeEquals methods for String, byte[], and char[].
NIFI-5540 Fixed checkstyle issue.
NIFI-5540 Fixed unit tests for default key population.
This closes#2959.
Signed-off-by: Bryan Bende <bbende@apache.org>
Removed NiFiHostnameVerifier. Removed NiFi WebUtils usage of NiFiHostnameVerifier.
Added unit tests for the DefaultHostnameVerifier to WebUtils.java
Added groovy-eclipse-compiler definition to nifi-web-utils/pom.xml to execute Groovy unit tests.
This closes#2919.
Co-authored-by: Andy LoPresto <alopresto@apache.org>
Signed-off-by: Andy LoPresto <alopresto@apache.org>
NIFI-5442 Populate request contextPath attribute during AccessResource before displaying on message-page.jsp.
Refactored shared code from CatchAllFilter to WebUtils.
NIFI-5442 Refactored filter and context path code to shared parent filter and subclass.
NIFI-5442 Removed unnecessary initParams from nifi-web-ui web.xml.
NIFI-5442 Added explicit dispatchers to nifi-web-ui web.xml and removed unnecessary code from AccessResource.
This closes#2908
Updated Javadoc for SiteToSiteClient#createTransaction() and HttpClient implementation.
Reverted exception listing in method contract for SiteToSiteClient#createTransaction and HttpClient tion of same.
Reverted import ordering in TestSiteToSiteClient.
Reverted exception listing in TestGetHDFSFileInfo, TestListHDFS, and StandardHttpFlowFileServerProtocol.
Restored @SuppressWarnings annotation and removed unnecessary "public static" keywords from inner classes in SiteToSiteClient.
This closes#2841.
Signed-off-by: Joe Witt <joewitt@apache.org>
NIFI-5141: Loosened regex for floating-point numbers to account for decimal place followed by 0 digits, such as '13.' and also added unit tests
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2679
- Created resuable KeytabUser and KeytabConfiguration in nifi-security-utils
- Refactored Solr processors to use a KeytabControllerService and no longer rely on JAAS system property
- Wrapped all calls in SolrProcessor onTrigger in a doAs when kerberos is enabled
- Added IT tests against MiniKDC
- This closes#2674
NIFI-950: Still seeing some slow response times when instantiating a large template in cluster mode so making some minor tweaks based on the results of CPU profiling
NIFI-5112: Refactored FlowSerializer so that it creates the desired intermediate data model that can be serialized, separate from serializing. This allows us to hold the FlowController's Read Lock only while creating the data model, not while actually serializing the data. Configured Jersey Client in ThreadPoolRequestReplicator not to look for features using the Service Loader for every request. Updated Template object to hold a DOM Node that represents the template contents instead of having to serialize the DTO, then parse the serialized form as a DOM object each time that it needs to be serialized.
NIFI-5112: Change ThreadPoolRequestReplicator to use OkHttp client instead of Jersey Client
NIFI-5111: Ensure that if a node is no longer cluster coordinator, that it clears any stale heartbeats.
NIFI-5110: Notify StandardProcessScheduler when a component is removed so that it will clean up any resource related to component lifecycle.
NIFI-950: Avoid gathering the Status objects for entire flow when we don't need them; removed unnecessary code
NIFI-950: Bug fixes
NIFI-950: Bug fix; added validation status to ProcessorDTO, ControllerServiceDTO, ReportingTaskDTO; updated DebugFlow to allow for pause time to be set in the customValidate method for testing functionality
NIFI-950: Addressing test failures
NIFI-950: Bug fixes
NIFI-950: Addressing review feedback
NIFI-950: Fixed validation logic in mock framework
This closes#2693
NIFI-4942 [WIP] More unit tests passing.
NIFI-4942 All unit tests pass and test artifacts are cleaned up.
NIFI-4942 Added RAT exclusions.
NIFI-4942 Added Scrypt hash format checker. Added unit tests.
NIFI-4942 Added NiFi hash format checker. Added unit tests.
NIFI-4942 Added check for simultaneous use of -z/-y. Added logic to check hashed password/key. Added logic to retrieve secure hash from file to compare. Added unit tests (125/125).
NIFI-4942 Added new ExitCode. Added logic to return current hash params in JSON for Ambari to consume. Fixed typos in error messages. Added unit tests (129/129).
NIFI-4942 Added Scrypt hash format verification for hash check. Added unit tests.
NIFI-4942 Fixed RAT checks.
Signed-off-by: Yolanda Davis <ymdavis@apache.org>
This closes#2628
- Added configurable character set encoding for JMS TextMessages
- Improved PublishJMS/ConsumeJMS documentation
- Validate character set in property validator instead of OnScheduled
Added failing unit test for #available() at various states (initial, during read, after read).
Implemented #available() delegation.
All tests pass.
This closes#2461.
Signed-off-by: Kevin Doran <kdoran@apache.org>
Added HostHeaderHandler regression unit tests before adding new functionality.
Added logic for parsing nifi.web.proxy.host property.
Added default property in nifi.properties and pom.xml.
Added logic for IPv6 and custom default hostnames.
Improved error messaging.
Added HostHeaderHandler unit tests.
Disabled HostHeaderSanitizationCustomizer in HTTP mode.
Fixed HTML escaping in error message.
Improved error message.
Added failing unit test for parsing custom hostnames.
Fixed custom hostname parsing.
Fixed unit tests.
Added TODO for IPv6 custom hostname parsing and unit test.
Added IPv6 custom hostname parsing and unit tests.
Fixed checkstyle issues.
Removed empty element in host list when no value defined.
Improved error message formatting.
Added unit tests.
Removed HostHeaderSanitizationCustomizer.
Removed InvalidPropertiesFormatException from NiFiProperties.
Removed InvalidPropertiesFormatException from HostHeaderHandler.
This closes#2415
using per-record flowfiles
NIFI-4428: Added initial L&N, fixed dependency hierarchy
NIFI-4428: Exposed additional ZK properties
NIFI-4428: More work to reduce dependencies in Druid artifacts
NIFI-4428: Removed inaccessible attribute
NIFI-4428: Fixed typo in property description
NIFI-4428: Updates to Druid NAR L&N
NIFI-4428: Updated LICENSE to refer to ICU License for icu4j
- Upgrading to Jersey 2.x.
- Updating NOTICE files where necessary.
- Fixing checkstyle issues.
This closes#2206.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
- Upgraded immediately actionable dependency versions from Meterian report.
- Upgraded jackson-core test dependencies for HBase and Elasticsearch modules.
- Only 3 instances of jackson-core < 2.8.6 (Google Cloud Platform and Spark Receiver modules).
- Upgraded version of poi dependency in nifi-email-processors to 3.16.
- Resolving dependency issues after rebasing against 1.5.0-SNAPSHOT.
- Removed jackson-databind from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.version} in all referenced artifacts.
- Removed jackson-mapper-asl from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.old.version} in all referenced artifacts.
- Removed Jasypt from <dependencyManagement> and added explicit version in test dependency for legacy compatibility.
- This closes#2084
- Adding support for KnoxSSO.
- Updated the docs for nifi.security.user.knox.audiences.
- The KnoxSSO cookie is removed prior to request replication.
This closes#2177
Added external compatibility regression test for StringEncryptor to ensure continued functionality during removal of Jasypt.
Documents custom salt lengths and iteration counts for each encryption method.
Added (ignored) failing tests for keyed encryption (Jasypt does not support keyed encryption).
Changed StringEncryptor to non-final class and added protected default constructor.
Added failing test for initialization status.
Added utility methods in CipherUtility.
Moved PBE cipher providers (and tests) from nifi-standard-processors to nifi-security-utils module.
Implemented PBE and keyed encryption/decryption logic.
Moved Scrypt unit test back into scrypt package.
Resolved test failures in limited strength cryptographic environment.
Implemented keyed encryption/decryption and enabled unit tests.
Removed Jasypt dependency from production scope (kept in test scope for backward compatibility tests).
Signed-off-by: joewitt <joewitt@apache.org>
- Added XmlUtils class.
- Added unit test.
- Added XXE test resource.
- Refactored JAXB unmarshalling globally to prevent XXE attacks.
- Refactored duplicated/legacy code.
- Cleaned up commented code.
- Switched from FileInputStream back to StreamSource in AuthorizerFactoryBean.
- This closes#2134
NIFI-4377: Updated RecordPath Guide to include the new fieldName() function and improved unit test for the function
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#2147.
- Initial implementation of Process Group level Variable Registry
- Updated to incorporate PR Feedback
- Changed log message because slf4j-simple apparently has a memory leak; passing a String instead of passing in the Controller Service object as an argument addresses this.
- This closes#2051
- Introducing support for OpenId Connect.
- Updating REST API and UI to support the authorization code flow.
- Adding/fixing documentation.
- Implementing time constant equality checks where appropriate.
- Corrected error handling during startup and throughout the OIDC login sequence.
- Redacting the token values from the user log.
- Defaulting to RS256 when not preferred algorithm is specified.
- Marking the OIDC endpoints as non-guaranteed in to allow for minor adjustments if/when additional SSO techniques are introduced.
This closes#2047.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
- Moved key provider interface and implementations from nifi-data-provenance-utils module to nifi-security-utils module.
- Refactored duplicate byte[] concatenation methods from utility classes and removed deprecation warnings from CipherUtility.
- Created KeyProviderFactory to encapsulate key provider instantiation logic.
- Added logic to handle legacy package configuration values for key providers.
- Added unit tests.
- Added resource files for un/limited strength cryptography scenarios.
- Added ASL to test resources.
- Moved legacy FQCN handling logic to CryptUtils.
- Added unit tests to ensure application startup logic handles legacy FQCNs.
- Moved master key extraction/provision out of FBKP.
- Removed nifi-security-utils dependency on nifi-properties-loader module.
- Added unit tests.
NiFi can now parse an Avro schema of a record that references an already defined record, including itself.
Signed-off-by: James Wing <jvwing@gmail.com>
This closes#2034.
- Introducing the LdapUserGroupProvider.
- Updating documentation accordingly.
- Moving the IdentityMapping utilities so they were accessible.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#1923.
Jeff Storck
Matt Gilman
Carl Gieringer
Mark Payne
Matthew Burgess
Andy LoPresto
joewitt
Pierre Villard
thenatog
Jan Hentschel
Koji Kawamura
patricker
zenfenan
Mike Thomsen
Bryan Bende
greencee
Mike Moser
Derek Straka
Yolanda M. Davis
Wesley-Lawrence
James Wing
m-hogue