1f700bb880
Suppress JsonPath exceptions in AvroFlattener ( #5793 )
...
Re: #5791
- Make the AvroFlattenerMake consistent with the JSONFlattenerMaker
2018-06-14 17:38:15 -07:00
693e3575f9
Remove unused code and exception declarations ( #5461 )
...
* Remove unused code and exception declarations
* Address comments
* Remove redundant Exception declarations
* Make FirehoseFactoryV2.connect() to throw IOException again
2018-03-16 22:11:12 +01:00
7e02408510
Update versions to 0.13.0-SNAPSHOT. ( #5323 )
2018-02-02 12:06:38 -06:00
5787d04fad
Bump Druid version to 0.12.0 ( #5138 )
2017-12-15 07:37:01 -08:00
7c01f77b04
Parse Batch support ( #5081 )
...
* add parseBatch and deprecate parse method in InputRowParser
add addAll method, skip max rows in memory check for it
remove parse method from implemetations
transform transformers
add string multiplier input row parser
fix withParseSpec
fix kafka batch indexing
fix isPersistRequired
comments
* add unit test
* make persist async
* review comments
2017-12-04 16:06:16 -06:00
2487152b59
Update Avro to 1.8.2 ( #5075 )
...
And add exclusions that are required to have a single version of
Apache Avro on the classpath.
2017-11-20 20:29:17 -08:00
3541b7544b
Prohibit and remove unused declarations in the processing module ( #4930 )
...
* Prohibit and remove unused declarations in the processing module
* Fix tests
* Fix integration tests
* Suppress unused
* Try to remove SuppressWarnings unused in VirtualColumn
* Remove reset 'false positives'
* Annotate CliCommandCreator as ExtensionPoint
* Unused import warning instead of error in IntelliJ
* Fixes
* Add comment
* Fix AzureBlob
* Fix CloudFilesBlob
* Address comments
* Add Project SDK section to INTELLIJ_SETUP.md
* Fix image
2017-11-09 09:27:27 -08:00
21e1bf68f6
Update Avro to 1.8.0 ( #5015 )
...
The druid parquet extensions uses Avro 1.8 and therefore it is
required to update the Avro version itself also to 1.8 to avoid
classpath conflicts
2017-11-02 09:08:41 -06:00
dc7cb117a1
Refactor ColumnSelectorFactory; Rely on ColumnValueSelector's polymorphism ( #4886 )
...
* Refactor ColumnSelectorFactory; Rely on ColumnValueSelector's polymorphism
* Fix MapVirtualColumn.makeColumnValueSelector()
* Minor fixes
* Fix IndexGeneratorCombinerTest
* DimensionSelector to return zeros when treated as numeric ColumnValueSelector
* Fix IncrementalIndexTest
* Fix IncrementalIndex.makeColumnSelectorFactory()
* Optimize MapBasedRow.getMetric()
* Fix VarianceAggregatorTest
* Simplify IncrementalIndex.makeColumnSelectorFactory()
* Address comments
* More comments
* Test
2017-10-13 21:44:17 -05:00
1f2074c247
Bump versions in master to 0.11.1-SNAPSHOT. ( #4878 )
...
* Bump versions in master to 0.11.1-SNAPSHOT.
* Missed a few.
2017-09-28 17:09:51 -05:00
bf8fd4c203
Add flattenSpec support to the Avro parser. ( #4832 )
...
* Add flattenSpec support to the Avro parser.
Also:
- Refactor the JSONPathParser a bit so it can share flattening code
with Avro (see ObjectFlatteners).
- Remove the JSONParser. It was only used in two places: by
UriNamespaceExtractor, and as a base for JSONToLowerParser. Migrated
the former to JSONPathParser and made the latter a standalone.
- Move GenericRecordAsMap to the Parquet extension, since the Avro
extension no longer uses it.
* Fix indentation.
* Fix equals/hashCode.
2017-09-26 09:26:06 -07:00
cbd1902db8
Add forbidden-apis plugin; prohibit using system time zone ( #4611 )
...
* Forbidden APIs WIP
* Remove some tests
* Restore io.druid.math.expr.Function
* Integration tests fix
* Add comments
* Fix in SimpleWorkerProvisioningStrategy
* Formatting
* Replace String.format() with StringUtils.format() in RemoteTaskRunnerTest
* Address comments
* Fix GroupByMultiSegmentTest
2017-08-21 13:02:42 -07:00
c0beb78ffd
Enforce brace formatting with Checkstyle ( #4564 )
2017-07-21 10:26:59 -05:00
60cdf94677
Add PMD and prohibit unnecessary fully qualified class names in code ( #4350 )
...
* Add PMD and prohibit unnecessary fully qualified class names in code
* Extra fixes
* Remove extra unnecessary fully-qualified names
* Remove qualifiers
* Remove qualifier
2017-07-17 22:22:29 +09:00
9ae457f7ad
Avoid using the default system Locale and printing to System.out in production code ( #4409 )
...
* Avoid usages of Default system Locale and printing to System.out or System.err in production code
* Fix Charset in DruidKerberosUtil
* Remove redundant string format in GenericIndexed
* Rename StringUtils.safeFormat() to unimportantSafeFormat(); add StringUtils.format() which fails as well as String.format()
* Fix testSafeFormat()
* More fixes of redundant StringUtils.format() inside ISE
* Rename unimportantSafeFormat() to nonStrictFormat()
2017-06-29 14:06:19 -07:00
ae900a4934
Update versions to 0.11.0-SNAPSHOT ( #4483 )
2017-06-28 17:05:58 -07:00
5285eb961b
Update dependencies ( #4313 )
...
* Update dependencies
* Downgrade curator
* Rollback aws-java-sdk dependency to 1.10.77
* Revert exclusions in integration-tests
* Depend only on aws-java-sdk-ec2 instead of umbrella aws-java-sdk (fixes #4382 )
2017-06-09 14:32:07 -07:00
31d33b333e
Make using implicit system Charset an error ( #4326 )
...
* Make using implicit system charset an error
* Use StringUtils.toUtf8() and fromUtf8() instead of String.getBytes() and new String()
* Use English locale in StringUtils.safeFormat()
* Restore comment
2017-06-05 23:57:25 -07:00
b7a52286e8
Make @Override annotation obligatory ( #4274 )
...
* Make MissingOverride an error
* Make travis stript to fail fast
* Add missing Override annotations
* Comment
2017-05-16 13:30:30 -05:00
e823085866
Improve `collection` related things that reusing a immutable object instead of creating a new object ( #4135 )
2017-05-17 01:38:51 +09:00
5ca67644e7
Remove slf4j as dependencies ( #4233 )
...
From the kafka-schema-registry-client in the avro extension slf4j
will be packaged into the distribution. We don't want this as it
will conflict and throw a slf4j multiple bindings warning.
This will cause slf4j to fall back to no-operation (NOP) binding.
2017-05-12 15:59:14 +09:00
3b92220015
Reduce log spam from Avro decoders. ( #4205 )
...
These objects get constructed semi-frequently (any time a parser is
deserialized) and so info logs are spammy. They'll still appear in
task logs at least once, since they're part of the task definition and
will get logged due to that.
2017-04-25 23:59:59 +05:30
de815da942
Some code refactor for better performance of `Avro-Extension` ( #4092 )
...
* 1. Collections.singletonList instand of Arrays.asList; 2. close FSDataInputStream/ByteBufferInputStream for releasing resource; 3. convert com.google.common.base.Function into java.util.function.Function; 4. others code refactor
* Put each param on its own line for code style
* Revert GenericRecordAsMap back about `Function`
2017-04-25 12:46:32 +09:00
2ca7b00346
Update versions to 0.10.1-SNAPSHOT. ( #4191 )
2017-04-20 18:12:28 -07:00
23f77ebd20
Explain Avro's unnecessary EOFException ( #4098 ) ( #4100 )
...
* Explain Avro's unnecessary EOFException (#4098 )
* add jira link into log message
2017-03-24 10:45:45 -05:00
12317fd001
Bump version to 0.10.0-SNAPSHOT. ( #3913 )
2017-02-06 17:54:35 -08:00
d80bec83cc
Enable auto license checking ( #3836 )
...
* Enable license checking
* Clean duplicated license headers
2017-01-10 18:13:47 -08:00
37ecffb648
Add support for Confluent Schema Registry in the avro extension ( #3529 )
2016-11-08 16:10:45 -06:00
4b3bd8bd63
Migrating java-util from Metamarkets. ( #3585 )
...
* Migrating java-util from Metamarkets.
* checkstyle and updated license on java-util files.
* Removed unused imports from whole project.
* cherry pick metamx/java-util@826021f .
* Copyright changes on java-util pom, address review comments.
2016-10-21 14:57:07 -07:00
40f2fe7893
Bump versions to 0.9.3-SNAPSHOT ( #3524 )
2016-09-29 13:53:32 -07:00
a069257d37
avro-extension -- feature to specify multiple avro reader schemas inline ( #3368 )
...
* rename SimpleAvroBytesDecoder to InlineSchemaAvroBytesDecoder
* feature to specify multiple schemas inline in avro module
2016-09-13 14:54:31 -07:00
48dce88aab
Add flag binaryAsString for parquet ingestion ( #3381 )
2016-08-30 17:30:50 -07:00
46da682231
avro-extensions -- feature to specify avro reader schema inline in the task json for all events ( #3249 )
2016-08-10 10:49:26 -07:00
ebd654228b
fix: avro types exception in sketch ( #3167 )
2016-06-22 15:54:20 -05:00
ebf890fe79
Update master version to 0.9.2-SNAPSHOT. ( #3133 )
2016-06-13 13:10:38 -07:00
7e7a886f65
Move druid-api into the druid repo.
...
This is from druid-api-0.3.17, as of commit 51884f1d05d5512cacaf62cedfbb28c6ab2535cf
in the druid-api repo.
2016-03-24 11:04:34 -07:00
a59c9ee1b1
Support use of DimensionSchema class in DimensionsSpec
2016-03-21 13:12:04 -07:00
738dcd8cd9
Update version to 0.9.1-SNAPSHOT.
...
Fixes #2462
2016-03-17 10:34:20 -07:00
e3e932a4d4
refactor extensions into core and contrib
2016-03-08 17:12:09 -08:00
Dylan Wylie
Roman Leventov
Gian Merlino
Parag Jain
Fokko Driesprong
Benedict Jin
Jihoon Son
Nicolas Colomer
Akash Dwivedi
Himanshu
Stéphane Derosiaux
du00cs
jon-wei
fjy