Apache/nifi
1
0
Fork 0
mirror of https://github.com/apache/nifi.git synced 2025-02-12 13:05:14 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,235 Commits 148 Branches 203 Tags
Commit Graph

2517 Commits

Author SHA1 Message Date
Mike Thomsen
bfe92b9000 NIFI-3970 added changes from a code review. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2334
 2017-12-11 19:16:34 -05:00
Matthew Burgess
63840377dd NIFI-3970: Add CSVRecordLookupService 2017-12-11 19:14:27 -05:00
Mike Thomsen
59d3c64195 NIFI-4636 Updated GetMongo to support expression language on limit, results per flowfile and batch fields. 
NIFI-4636: Added unit tests
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2289
 2017-12-11 12:49:42 -05:00
Mark Payne
f772f2f093 NIFI-4671: This closes #2328. Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-11 11:46:15 -05:00
Jeff Storck
600586d6be NIFI-3366 This closes #2332. Added parent/child flowfile relationship between the incoming flowfile and the files that are moved from the input directory to the output directory. 
Updated to allow tests to check for evaluation of properties that support expression language.
Fixed bug with changeOwner attempting to operate on original file rather than the moved/copied file.
Added license header to MoveHDFSTest.java
Added test for moving a directory of files that contains a subdir, ensuring non-recursive behavior
Added to the description of the processor that it is non-recursive when a directory is used as input.
Added RAT exclude for test resource .dotfile to pom.xml

Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-11 08:41:36 -05:00
Gray Gwizdz
3731fbee88 NIFI-3366 MoveHDFS processor supports expressions language for input and copy operations 
Signed-off-by: Jeff Storck <jtswork@gmail.com>
 2017-12-11 08:41:25 -05:00
Mark Payne
c138987bb4 NIFI-4656, NIFI-4680: This closes #2330. Fix error handling in consume/publish kafka processors. Address issue with HortonworksSchemaRegistry throwing RuntimeException when it should be IOException. Fixed bug in ConsumeerLease/ConsumKafkaRecord that caused it to report too many records received 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-08 16:01:14 -05:00
Mark Payne
1fc1d38fd8 NIFI-4562: This closes #2331. If an Exception is thrown when merging FlowFiles, ensure that we remove the 'bundle' flowfile before exiting the method 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-08 15:35:36 -05:00
Mark Payne
3b74d2ddad NIFI-4633: This closes #2327. Ensure that everywhere that a FlowFile is passed into ProcessSession that we used the most up-to-date version of it 
Ensure that when ProcessSession.clone(FlowFile) is called, we obtain the most recent version of the FlowFile before attempting to obtain FlowFile size.

Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-08 15:30:01 -05:00
Mark Payne
113ad5ecfa NIFI-4639: Updated PublishKafka 1.0 processor to use a fresh writer for each output record as well. This closes #2292. 2017-12-08 09:14:17 -05:00
matthew-silverman
c9cc76b5c8 NIFI-4639: fresh writer for each output record 2017-12-08 08:39:22 -05:00
Mark Payne
8e6649ba15 NIFI-2776: This closes #2315. When joining a cluster, if a processor is stopping but cluster indicates that processor should be running, cause processor to start when its last thread finishes 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-07 15:45:18 -05:00
joewitt
a774f1df69 NIFI-4673 changed offending tests to be integration tests and fixed travis config to run the build only once and during appropriate script phase instead of install. Reviewed by Bende. 2017-12-06 16:13:42 -05:00
joewitt
7e61c6333c
NIFI-2375 updated to latest Tika across codebase. Fixed 
tests and updated L&N for Tika parsers.  Turned unstable grpc test to IT
 test. Updated deps due to Tika testing findings. This closes #2320
 2017-12-06 11:53:15 -05:00
joewitt
cdc1facf39
NIFI-4664, NIFI-4662, NIFI-4660, NIFI-4659 moved tests which are timing/threading/network dependent and brittle to integration tests and un-ignored tests that are IT. Updated travis to reduce impact on infra and appveyor now skips test runs so is just to prove build works on windows. This closes #2319 
squash
 2017-12-06 10:53:09 -05:00
joewitt
a0f568b69c
NIFI-4661 no longer building and including the gprc elements by default. This closes #2318 2017-12-05 14:26:05 -05:00
William Griffiths
9736cb9d33 NIFI-4663 This closes #2317. Change log for document not found from warn to debug as we have a NOT_FOUND relationship 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-05 12:44:54 -05:00
Mark Payne
00b11e82b7 NIFI-4600: This closes #2312. Added nifi-kafka-1-0-nar and nifi-kafka-1-0-processors modules 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-04 16:51:59 -05:00
joewitt
2c68d0e098
NIFI-4652 updating to latest apache version and updating key maven plugins, fixing versions for grpc to avoid extranous build dir, correcting checkstyle issues, resolving build problem with incorrect usage of activeByDefault in maven. Key change is now for the 'dir-only' profile it is now activated via a system property -Ddir-only instead of profile. No other activeByDefault profiles exist except to minifi the javascript but that usage is acceptible as-is. This closes #2313 2017-12-04 09:59:23 -05:00
Mark Payne
596a329dc7 NIFI-4653: Fixed bug where Process Group DTO was not containing variables when created with the #createConciseProcessGroup method. This closes #2314. 2017-12-04 09:31:54 -05:00
Joey Frazee
a458a139b3 NIFI-4650 This closes #2308. Added independent hadoop.http.core.version property 
This closes #2308

Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-02 12:58:29 -05:00
joewitt
9fdab10c7c
NIFI-4645 now being very specific to include if not ppc. This closes #2309 2017-12-01 12:47:40 -05:00
aburkard
103a345e85 NIFI-4628 Add support for ListS3Version2 API 
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2300.
 2017-11-30 21:53:05 -08:00
joewitt
dd981e87dd NIFI-4642 updated tests to be more tolerant/variable to different system speeds. Many of these should be integration tests and not unit tests. This closes #2303. 2017-11-30 14:59:58 -05:00
Mark Payne
45df23b1e0 NIFI-4607, NIFI-3975, NIFI-4602, NIFI-4606: This closes #2272. Fixed bug in TailFile that caused new Primary Node to not pull current Clustered State when File Location was set to Remote. Fixed bug that caused TailFile to occasionally become 'stuck' when the file it is tailing is renamed and a new file is created with the same name. Removed the 'Rolling Strategy' property from TailFile because it is not actually used in the processor anymore. Deleted MonitorMemoryTest because the unit test was testing the behavior of FlowController more than the behavior of the reporting task itself and in order to do so had a dependency in the pom.xml on nifi-framework-core, which means that it no longer compiles when FlowController is modified. 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-11-30 11:10:46 -05:00
Matthew Burgess
3b15ed855c NIFI-4559: Add non-zero status relationship to ExecuteStreamCommand 
NIFI-4559: Removed Penalize Non-zero Status property and updated doc per review comments
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2246
 2017-11-30 10:12:00 -05:00
sbouchex
33281300cd NIFI-4597 This closes #2283. correcting imports, checkstyle issues, and adding validator 
Add a property to override HTTP return code in ListenHTTP processor

Signed-off-by: joewitt <joewitt@apache.org>
 2017-11-30 10:02:32 -05:00
Matt Gilman
3df3ff6c61 NIFI-4646: This closes #2306. 
- Ensuring the reason behind the SchemaNotFoundException is reported when interacting with the configured SchemaRegistry.

Signed-off-by: joewitt <joewitt@apache.org>
 2017-11-30 09:02:21 -05:00
joewitt
9ea9996b36
NIFI-4645 make sure the grpc bundle is only built and nar included IF not building on powerpc arch. This closes #2305 2017-11-29 13:33:17 -05:00
Joey Frazee
73fa0429f0
NIFI-4640 Sub source w/ . in nifi.sh to be POSIX compliant 
This closes #2302

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2017-11-28 20:02:58 -05:00
Mark Payne
cec2764140
NIFI-4632: Add the local hostname to the list of validated host headers 
This closes #2288.

Signed-off-by: Andy LoPresto <alopresto.apache@gmail.com>
 2017-11-27 19:08:29 -08:00
joewitt
ecb9c5e929 NIFI-4445 on fast machines the engine ran for 0 millis so the test needs to allow this case 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2296
 2017-11-27 11:47:54 -05:00
Mark Payne
9766558cab
NIFI-4526: Allow Target URI's of Remote Process Groups to be changed. This closes #2298 2017-11-27 10:50:32 -05:00
Matthew Burgess
16a23f5a0f NIFI-4589: Allow multiple keys in FetchDistributedMapCache, add subMap operation to API 
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #2260.
 2017-11-21 20:13:35 +00:00
Mark Payne
412b3fbbe2 NIFI-4459: This closes #2240. Catch Exception if trying to iterate over many confluent schemas and unable to load one; in this case log a WARNING and continue on; also updated Jersey client to newest 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-11-21 12:52:05 -05:00
m-hogue
73702004b9 NIFI-3515: This closes #2257. Added EventDriven annotation to PutFile 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-11-21 12:40:13 -05:00
Janosch Woschitz
e8b2387cb2 NIFI-4623: This closes #2281. Removed obsolete instability warning in documentation of newer (>= 0_10) Kafka processors 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-11-21 12:16:54 -05:00
m-hogue
020f9d0596 NIFI-4584: Added @EventDriven to InvokeGRPC 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2265.
 2017-11-21 11:29:42 +01:00
Matthew Burgess
8f2501ffac NIFI-4612: Allow AvroSchemaRegistry to disable name validation 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2275.
 2017-11-21 11:14:19 +01:00
Matt Gilman
ff5325b923
NIFI-4614: - Updating the types of resources that are filtered out for viewing purposes. Updates include resources with no values and resources that contain wildcards. 
This closes #2277.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2017-11-20 10:20:01 -05:00
Matt Gilman
439e13a8d5
NIFI-4567: 
- Adding new properties to allow the referenced attribute of a user/group to be configurable when detecting group membership.
- Expanding on documentation regarding the new properties.

This closes #2274.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2017-11-20 10:09:53 -05:00
Matt Gilman
6b75921725 NIFI-4617: 
- Ensuring a charset is specified when convert a string into bytes.
- Ensuring the aggregate snapshot is non null before attempting to promote the runStatus.

This closes #2278.
 2017-11-20 09:39:23 -05:00
Andy LoPresto
5d643edfab
NIFI-4501 
- Changed request header handling logic.
- Removed unnecessary Maven dependency.
- This closes #2279
 2017-11-17 20:36:55 -05:00
sbouchex
c89d793364
NIFI-4615 
- processor status is set to the ProcessorStatusDTO object
- This closes #2276
 2017-11-17 15:44:09 -05:00
Matt Gilman
2da72f1524
NIFI-4593: - Ensuring the necessary jackson dependencies are bundled. 
This closes #2263.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2017-11-16 15:44:37 -05:00
Matthew Burgess
53de8c93ba NIFI-4603 - Externalize Max HTTP Header Size to nifi.properties 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2271.
 2017-11-15 10:12:39 +01:00
Andy LoPresto
7f8007f22a NIFI-4608 Removed unnecessary NiFiProperties mocking given enhancements to StringEncryptor. 
Fixed checkstyle issues.
Fixed test failure in FingerprintFactoryTest.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2273.
 2017-11-15 10:09:18 +01:00
Pierre Villard
b9c53083c4 NIFI-4576 - rewording processors / cores 
NIFI-4576: Rewording processor load average to core load average

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2267
 2017-11-14 13:08:53 -05:00
Mike Thomsen
ad2d12a20c NIFI-4578 Added strategy for dealing with nullable fields in PutHBaseRecord. 
NIFI-4578 Added changes from code review.

This closes #2256.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2017-11-14 13:59:30 +09:00
Pierre Villard
9e9c129c21 NIFI-4577 - ValidateCsv - add attributes to indicate number of valid/invalid lines 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2268
 2017-11-13 21:48:30 -05:00