Commit Graph

59 Commits

Author SHA1 Message Date
Bryan Bende 5c3ca9d537
NIFI-10981 Ensure NarAutoLoader starts after provider retrieves NARs,… (#6785)
- NIFI-10981 Ensure NarAutoLoader starts after provider retrieves NARs, and ensure the auto-loader attempts to process any NARs available before starting to watch for new files
- Add system test with NAR Provider that copies NARs from a local directory and verifies the NARs were loaded and linked correctly
- Exclude new assembly from CI test-compile
2022-12-16 11:30:37 -05:00
exceptionfactory 4725ca36ba
NIFI-10849 Changed Maven compile to test-compile in GitHub ci-workflow
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6691.
2022-11-22 13:23:55 +01:00
exceptionfactory 9e3987d04e
NIFI-10826 Separated Maven Compile and Verify in ci-workflow
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6666
2022-11-16 12:56:33 -05:00
exceptionfactory 4569c597f2
NIFI-10749 Corrected Windows NPM Cache for GitHub Workflow
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6616
2022-11-03 11:13:24 -04:00
exceptionfactory 7d6dc2cace
NIFI-10658 Upgraded GitHub Actions to current versions
- Upgraded stale from 3 to 6
- Upgraded checkout from 2 to 3 in system-tests
- Upgraded setup-java from 2 to 3 in system-tests
- Upgraded upload-artifact from 2 to 3 in system-tests

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6542.
2022-10-18 09:28:30 -04:00
Ashish Kurmi c9ade4802c
NIFI-10575 Added minimum GitHub token permissions for workflows
This closes #6469

Signed-off-by: David Handermann <exceptionfactory@apache.org>
Co-authored-by: Ashish Kurmi <akurmi@stepsecurity.io>
Co-authored-by: Alex <aleksandrosansan@gmail.com>
2022-10-03 10:02:36 -05:00
Joe Witt adf8b0d359
NIFI-10272 taking off parallel build as it has been unstable 2022-07-25 12:27:05 -07:00
exceptionfactory 8185ad0598
NIFI-10249 Enabled Parallel Builds in GitHub Workflow
- Upgraded maven-enforcer-plugin from 3.0.0 to 3.1.0
- Upgraded maven-dependency-plugin from 3.2.0 to 3.3.0
- Upgraded checkstyle from 9.2.1 to 9.3
- Upgraded buildnumber-maven-plugin from 1.4 to 3.0.0

This closes #6222.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-07-19 09:51:31 -04:00
exceptionfactory eb91eb12b0
NIFI-10224 Moved GitHub Workflow Analysis to Separate Job
- Upgraded multiple GitHub Actions from Version 2 to 3
- Replaced Maven Cache step with cache feature in Setup Java Action
- Moved checkstyle and license checks from verify to validate phase in Maven execution
- Removed unnecessary contrib-check profile from nifi-registry

This closes #6202
Signed-off-by: Paul Grey <greyp@apache.org>
2022-07-18 12:46:30 -04:00
exceptionfactory c9c7d5f5a3
NIFI-10038 Added Java 17 to system-tests workflow (#6060) 2022-05-23 16:31:42 -04:00
exceptionfactory 8f3445d48f
NIFI-10000 Updated Pull Request Template Summary
- Removed italicized formatting for example summary paragraph and updated example issue number

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #6025.
2022-05-06 18:36:27 -04:00
exceptionfactory fbfdcdca9f
NIFI-9970 Restructured Pull Request Template
- Added Summary, Tracking, and Verification section headings
- Reworded checklist to declarative statements instead of questions
- Added JDK 17 to list of build platforms

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6002.
2022-04-28 10:04:11 +02:00
Paul Grey c88c404dd8
NIFI-9826 Moved build locale settings from Maven to GitHub Workflow
- Adjusted Java 17 build locale settings
- Added test to verify match between System properties and build environment variables

This closes #5980

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-04-20 14:19:00 -05:00
exceptionfactory 105890ef37
NIFI-9773 Refactored nifi-system-tests using JUnit 5
- Enabled system-tests workflow on push and pull requests

This closes #5901

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-26 07:22:07 -04:00
exceptionfactory 412e21be97
NIFI-9281 Enabled building on Java 17
- Added Ubuntu Zulu JDK 17 GitHub build
- Adjusted MiNiFi C2 FileSystemConfigurationCache test to avoid using environment variables
- Adjusted MiNiFi StatusLogger and StatusLoggerTest to avoid overriding private logger
- Adjusted failure reason attribute check in TestGetIgniteCache
- Adjusted TestRangerAuthorizer and TestRangerNiFiAuthorizer to avoid checking nested exceptions
- Adjusted encrypt-config TestUtil to avoid unnecessary comparison of different types
- Disabled Javascript tests on Java 15 and higher
- Disabled several Hive 3 tests on Java 17 for StringInternUtils illegal access
- Refactored nifi-enrich-processors tests to use Mockito without Powermock
- Refactored nifi-toolkit-tls tests to avoid illegal reflective access
- Removed deprecated X509Certificate test in CertificateUtilsTest
- Removed kryo serialization from nifi-site-to-site-client test
- Updated TestHashContent to use SHA-1 instead of SHA for hash algorithm
- Upgraded maven-war-plugin from 2.5 to 3.3.2
- Upgraded nifi-graph-bundle dependencies from Groovy 2.5.14 to 3.0.8
- Upgraded QuestDB from 4.2.1 to 6.2.1 in nifi-framework-core

This closes #5870

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-17 14:08:38 -04:00
Paul Grey a0fc7f5e61
NIFI-9799 Enabled style checking for nifi-system-tests in ci-workflow
This closes #5869

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-16 10:53:35 -05:00
Paul Grey 6bc85155f9
NIFI-9749 - Capture additional logging for system-test workflow runs
This closes #5827

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-02 18:52:43 -06:00
Scott Aslan b8ec6d38dc [NIFI-9554] upgrade node and npm
update lintin
update font awesome path
bump nifi-fds version
update nifi deps
update frontend maven plugin
change CI npm cache restore keys so that we will not use cached directories for older versions of node

This closes #5751.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-14 14:24:02 -05:00
exceptionfactory e2243500d3
NIFI-9618 Upgraded Checkstyle to 9.2.1
- Corrected several missing return statements
- Increased Maven heap size to 3 GB for ci-workflow

This closes #5700

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 20:08:16 -06:00
Paul Grey 73de3a0fe7
NIFI-9586 Excluded additional assembly modules from ci-workflow
This closes #5690

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 13:24:42 -06:00
exceptionfactory d987351031
NIFI-9608 Disabled system-tests workflow for pull requests
This closes #5689

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-20 10:42:25 -06:00
Paul Grey 70aa719cbf
NIFI-9586 Removed Surefire ForkNodeFactory configuration
- Falling back to default LegacyForkNodeFactory due to SUREFIRE-1800
- Added upload of Surefire results on automated build failures

This closes #5675

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-19 13:50:05 -06:00
Bryan Bende 0f02dac3fa
NIFI-9452 Generate a RuntimeManifest for NiFi at build time
- Write additional fields to extnesion-manifest.xml for processors
- Update C2 model classes to support new fields for processors, properties, and scheduling
- Create converter between NiFi model and C2 model
- Create generator and execute during the build

Add profile to nifi-assembly that skips the binary assembly, update github workflow to enable this profile instead of excluding nifi-assembly

Add additional documentation on new fields in processor definition and reporting task definition

Set charset to UTF-8 on the OutputStreamWriter

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5612
2021-12-21 16:35:15 -05:00
Mark Payne 20392d5ef4
NIFI-9503: Fixed issue in system-tests
- Fixed system-tests workflow so that troubleshooting info is always uploaded for failures

This closes #5614

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-18 19:59:54 -06:00
Mark Payne 90b39b593a
NIFI-9069 Changed framework dataflow serialization to support JSON
- Changed framework so that it serializes the dataflow into a VersionedDataflow using JSON as well as XML, and prefers the JSON representation on load, if it's available. This also results in the need for the cluster protocol to exchange its representation of the dataflow to using JSON. Rather than re-implementing all of the complex logic of Flow Fingerprinting, updated to just inherit the cluster's flow.
- Moved logic to synchronize Process Group with Versioned Process Group into a new ProcessGroupSynchronizer class instead of having all of the logic within StandardProcessGroup
- Reworked versioned components to use an instance id.
- Renamed StandardFlowSynchronizer to XmlFlowSynchronizer; introduced new StandardFlowSynchronizer that delegates to the appropriate (Xml or Versioned)FlowSynchronzer
- Updated to allow import of VersionedProcessGroup even if not all bundles are available - will now use ghost components
- Introduced a VersionedDataflow object to hold controller-level services, reporting tasks, parameter contexts, templates, etc.
- Allow mutable requests to be made while nodes are disconnected. Also fixed issue in AbstractPolicyBasedAuthorizer that caused ClassNotFoundException / NoClassDefFoundError if the authorizations were changed and then a node attempts to rejoin the cluster. The Authorizer was attempting to use XmlUtils, which is in nifi-security-utils and so so by madking nifi-security-utils a provided dependency of nifi-framework-api, but this doesn't work, because nifi-framework-api is loaded by a higher-level classloader, so the classloader that loads AbstractPolicyBasedAuthorizer will never have the appropriate classloader to provide nifi-security-utils. Addressed this by copying the code for creating a safe document builder from XmlUtils to AbstractPolicyBasedAuthorizer.
- Fixed bug that occurred when importing a Process Group that has 2 parameter contexts, one inheriting from another, where neither is pre-defined in the existing flow
- Fixed bug that was encountered when Updating a Versioned Process Group where one version had a disabled processor and the other had the processor running.
- Increased system-tests workflow timeout to 120 minutes
- Added additional exception handling to system tests

This closes #5514

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-02 15:10:38 -06:00
Chris Sampson 3a63bd579e
NIFI-8002:
- Creation of a PaginatedJsonQueryElasticsearch and ConsumeElasticsearch processors
- Integration Tests for ES Client Service with version and flavour-specific tests for applicable functionality

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5193.
2021-10-04 19:27:12 -04:00
exceptionfactory 9dfcba24cd
NIFI-9179 Enabled system-tests workflow for selected modules
- Increased system test NiFi heap memory to 512 MB

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5389.
2021-09-18 15:02:54 +02:00
exceptionfactory c2c3727dd5
NIFI-9040 Add GitHub Workflow for System Tests (#5310)
- Added GitHub Workflow for System Tests
- Added system-tests badge to README.md
- Added nifi-stateless-system-test-suite to system-tests workflow
2021-08-19 09:52:16 -04:00
Paul Grey fc1fa3f32c
NIFI-9057 Manage concurrency for GitHub Workflow Runs
This closes #5315

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-18 18:29:06 -05:00
exceptionfactory b34b5b8eef
NIFI-9037 Simplified GitHub Workflow Configuration
- Created environment variables for shared Maven commands and projects
- Combined multiple Maven Cache actions
- Added node_modules to NPM Cache action configuration
- Disabled NiFi Registry integration tests in the default configuration
- Removed JUnit Results artifacts from workflow
- Removed parallel build argument from GitHub workflow Maven command
- Upgraded Surefire Plugin to 3.0.0-M5
- Upgraded Enforcer Plugin to 3.0.0
- Added package-lock.json for nifi-jolt-transform-json-ui

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5298
2021-08-09 19:27:46 -04:00
exceptionfactory 54624bc26d
NIFI-8758 Increased GitHub build timeout to 120 minutes
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5199.
2021-07-06 11:29:53 +02:00
tpalfy dfa683af0e
NIFI-8528 Migrate NiFi Registry into NiFi codebase (#5065)
NIFI-8528 Migrate NiFi Registry fully codebase into NiFi as a module. No changes except certain dependency scopes to preserve the NiFi Registry original by overriding the new parent (nifi).
- Version adjustments. Removed distinct checkstye rules form nifi-registry. (Using nifi's instead.)
- Made some tests Windows-compatible.
- Consolidated LICENSE, NOTICE and README.md.
- Fixed CryptoKeyLoaderGroovyTest.groovy.
- Disable frontend-maven-plugin on Windows.
- Skipping all goals of the frontend-maven-plugin on Windows.
- Registry integration tests not to run in github jobs (same as the original settings). Skip all registry tests (build and run) on Windows.
- Removed Husky from registry.
2021-05-26 09:34:55 -04:00
Paul Grey cc554a6b11
NIFI-8478 Addressed i18n test issues; CI supplies locale to surefire
This closes #5040

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-03 13:40:01 -05:00
Joe Witt 8207c9db20
NIFI-8456 This closes #5017. Ensure we have only three builds and we maximally cover zulu vs adopt, linux vs windows vs osx, and java 8 vs 11 and EN vs FR vs JP
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-04-21 14:30:24 -07:00
exceptionfactory bbe75e81dd
NIFI-8441 Added timeout-minutes for GitHub workflow on Windows
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5010.
2021-04-20 09:23:17 +02:00
Joe Witt 8c66049d1d
NIFI-8411 update to use latest java action and specific distribution so we can validate azul and zulu
This closes #4995

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-04-14 08:43:43 -04:00
Joe Witt f0611f31da
NIFI-8422 ensure stale workflow for github ci is there and update asf license header missing from a few files
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4997.
2021-04-14 09:26:30 +02:00
Joe Witt 4189a1c1eb
NIFI-8350 This closes 4921. updated worfklow
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-03-23 07:09:59 -07:00
Joe Witt 434c995185
NIFI-7873 relaxing github ci test speeds to help restore reliability
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-01 15:13:19 -07:00
Joe Witt d08a0babb7
NIFI-7873 increasing test parallelism to use all cores available 2021-01-27 09:26:41 -07:00
Joe Witt d1bfa67927
NIFI-7939 improving version ranges to fix usage of now deprecated github actions commands
self merging given nature of change

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-10-22 07:16:31 -07:00
Andy LoPresto 0fa8776f4d
NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and documentation.
Fixed unit test failures introduced from static imports during refactoring.

Signed-off-by: Joe Witt <joe.witt@gmail.com>
2020-07-14 10:39:28 -07:00
Joe Witt f8e5be3c09
NIFI-7602 added verify phase so contrib-check can actually execute
This closes #4388

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 07:40:42 -04:00
Andy LoPresto 45e626f6a8
Updated pull request template to separate JDK 8 and 11 questions 2020-05-22 12:19:58 -07:00
Peter Turcsanyi bce82a5e6f
NIFI-7416: Update travis-ci to GitHub Actions in the PR template
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4247.
2020-05-01 16:17:07 +02:00
Joe Witt a9bed56f68
NIFI-7311 adding additional cleanup and output to github actions yml and not running nifi-system-test module
Self merging wo review as part of github ci actions stability.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-04-02 12:54:49 -04:00
Joe Witt 992d990dd2
NIFI-7291 updated enforcement of dependency rules and build command
Reviewed by markap14
This closes #4166.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-27 16:34:59 -04:00
Joe Witt 271de92046
NIFI-7238 printing maven version info in same build command 2020-03-25 15:21:25 -04:00
Joe Witt 7d83622f05
NIFI-7238 remove no longer needed delete command 2020-03-25 12:01:45 -04:00
Joe Witt efe2bca4cb
NIFI-7238 Continue to improve Github Actions CI stability 2020-03-25 11:44:20 -04:00