Martijn van Groningen
0fe1b4eab1
PipelineStore no longer is a lifecycle component
...
Client in PipelineStore gets provided via a guice provider
Processor and Factory throw Exception instead of IOException
Removed PipelineExecutionService.Listener with ActionListener
2015-11-26 18:12:15 +01:00
Martijn van Groningen
9aff8c6352
fix compile errors
2015-11-26 17:00:16 +01:00
Martijn van Groningen
9d1fa0d6da
ingest: Add meta processor that allows to modify the metadata attributes of document being processed
2015-11-26 15:46:32 +01:00
Martijn van Groningen
afc9069c99
* Inlined PipelineStoreClient class into the PipelineStore class
...
* Moved PipelineReference to a top level class and named it PipelineDefinition
* Pulled some logic from the crud transport classes to the PipelineStore
* Use IOUtils#close(...) where appropriate
2015-11-26 14:42:39 +01:00
javanna
1a7391070f
Simulate api improvements
...
Move ParsedSimulateRequest to SimulatePipelineRequest and remove Parser class in favor of static parse methods.
Simplified execute methods in SimulateExecutionService.
2015-11-26 13:24:45 +01:00
Martijn van Groningen
2890432421
made updatePipelines() to not make it prone to race conditions
2015-11-25 18:32:56 +01:00
javanna
5d510b59c8
use MetaData enum for metadata field names
...
Also rename getName to getFieldName in MetaData to prevent confusion with name() enum method.
2015-11-25 18:08:53 +01:00
javanna
ec162c458e
Replace property with field in IngestDocument
...
getPropertyValue => getFieldValue
hasPropertyValue => hasFieldValue
setPropertyValue => setFieldValue
removeProperty => removeField
2015-11-25 18:08:53 +01:00
Luca Cavanna
e15fa99ee3
Merge pull request #15019 from javanna/enhancement/java8_date_parser
...
date formats: use a function instead of our own interface
2015-11-25 16:17:39 +01:00
javanna
c4cf55c196
[TEST] generate random timezone out of the available ones in joda
2015-11-25 15:58:01 +01:00
javanna
5daa73b350
date formats: use a function instead of our own interface
...
Also turn the different date formats into an enum.
2015-11-25 15:37:35 +01:00
javanna
e0fcee642e
[TEST] fix locale comparison
2015-11-25 10:26:46 +01:00
javanna
388e637fa9
add a few more asserts to IngestActionFilterTests
2015-11-24 19:43:54 +01:00
javanna
49bfe6410e
Rename Data leftovers
2015-11-24 15:49:25 +01:00
javanna
8f1f5d4da0
Split mutate processor into one processor per function
2015-11-24 14:31:53 +01:00
Martijn van Groningen
1e9d5c7b22
test: also test what happens if all index requests fail to be processed by the pipeline
2015-11-24 13:41:40 +01:00
Martijn van Groningen
8b1f117e51
Instead of failing the entire bulk request if the pipeline fails, only fail a bulk item.
2015-11-24 12:40:30 +01:00
Martijn van Groningen
ecc8158b89
renamed Data to IngestDocument
...
moved all metadata related fields to a single metadata map
removed specific metadata getters with a generic getMetadata()
2015-11-23 18:24:42 +01:00
javanna
bb298ed27a
add some javadocs to Data#getDocument
2015-11-20 18:01:27 +01:00
javanna
be3e913349
set Data modified flag when a property is removed and improve behaviour when adding a field
2015-11-20 17:59:45 +01:00
javanna
404ae395ca
add javadocs for Data#removeProperty
2015-11-18 18:00:25 +01:00
javanna
59868cd02e
add support for removeProperty
2015-11-18 17:28:37 +01:00
javanna
ab5b649184
accept null values and adapt hasPropertyValue return value
2015-11-18 10:46:03 +01:00
javanna
ba8f8810ea
rename getProperty, containsProperty and addField methods
...
For more consistency we now have getPropertyValue, hasPropertyValue, and setPropertyValue
2015-11-18 10:46:03 +01:00
javanna
126df4ca9a
type safety in Data#getProperty and proper error when type found is not the expected one
2015-11-18 10:46:03 +01:00
javanna
e616e8398a
add missing javadocs
2015-11-18 10:46:03 +01:00
javanna
044a86d6c6
remove dependency on core and resolve generics compiler warning
2015-11-18 10:46:03 +01:00
javanna
ba7e536e1d
better error and tests for empty and null values in Data containsProperty addField and getProperty
2015-11-18 10:46:03 +01:00
javanna
9d7d5bd9bc
Throw a proper error when add field fails due to existing field type mismatch
...
Instead of throwing ClassCastException whenever we try and add a field to a parent that is not a Map, we now throw a clearer error (IAE).
2015-11-18 10:46:02 +01:00
Luca Cavanna
68cefe1d81
Merge pull request #14759 from talevy/fix_null_mutate_report
...
adds tests and guards against null values in some mutate methods
2015-11-16 10:32:32 +01:00
javanna
5169d9d80f
minor formatting changes
2015-11-16 10:28:29 +01:00
Tal Levy
de33f5a911
adds tests and guards against null values in some mutate methods
2015-11-15 21:57:27 -08:00
javanna
446fa0c10b
remove unnecessary line breaks
2015-11-13 19:37:06 +01:00
javanna
26569045ef
remove leftover equals/hashcode
2015-11-13 19:36:18 +01:00
javanna
97f4f27b14
remove equals/hashcode as part of Pipeline and adapt tests
...
Only MutateProcessor implemented equals / hashcode hence we would only use that one in our tests, since they relied on them. Better to not rely on equals/hashcode, drop them and mock processor/pipeline in our tests that need them. That also allow to make MutateProcessor constructor package private as the other processors.
2015-11-13 19:35:09 +01:00
javanna
d093600729
simplify serialization for simulate response depending on verbose flag
...
Removed equals and hashcode whenever they wouldn't be reliable because of exception comparison. at the end of the day we use them for testing and we can simplify our tests without requiring equals and hashcode in prod code, which also would require more tests if maintained.
Add equals/hashcode test for Data/TransportData and randomize existing serialization tests
2015-11-13 16:22:24 +01:00
Tal Levy
20384aedf0
split out SimulateDocumentResult into subclasses, add tests for equalTo and streamable
2015-11-12 18:27:49 -08:00
Tal Levy
af1de8e1cc
updated with cosmetic changes
2015-11-12 13:43:05 -08:00
javanna
c4951ef74f
update get pipeline param names to id for consistency
2015-11-12 17:40:14 +01:00
javanna
979fa81618
make description optional as part of a Pipeline
2015-11-12 15:45:35 +01:00
javanna
75371b2381
restore initial simulate endpoint url, adapt get pipeline param name
2015-11-12 15:45:17 +01:00
javanna
66330539a3
Merge branch 'feature/ingest' into pr/14572
2015-11-12 15:39:22 +01:00
javanna
5bd4493ea2
use ConfigurationUtils to read string value from config
2015-11-12 15:38:36 +01:00
Tal Levy
674084973d
moar updates
2015-11-11 21:51:45 -08:00
Tal Levy
b40af1bcfd
updates, moar verbose
2015-11-11 19:39:23 -08:00
Tal Levy
c22c1e0f54
remove simulate executor service call and move to simple execution
2015-11-11 10:35:09 -08:00
Tal Levy
1f29fa4fe9
update rest status
2015-11-11 10:35:09 -08:00
Tal Levy
bce7f6c7ad
Add simulate endpoint
2015-11-11 10:35:09 -08:00
javanna
c12c9e6e29
add equals and hashcode to GsubExpression
2015-11-11 11:10:26 +01:00
javanna
2b31f4fff7
Mutate processor improvements
...
Remove code duplications from ConfigurationUtils
Make sure that the mutate processor doesn't use Tuple as that would require to depend on core.
Also make sure that the MutateProcessor tests don't end up testing the factory as well.
Make processor getters package private as they are only needed in tests.
Add new tests to MutateProcessorFactoryTests
2015-11-11 11:10:26 +01:00
