Apache/druid
1
0
Fork 0
mirror of https://github.com/apache/druid.git synced 2025-02-07 02:28:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
8,502 Commits 115 Branches 507 Tags
Commit Graph

38 Commits

Author SHA1 Message Date
Roman Leventov
693e3575f9
Remove unused code and exception declarations (#5461) 
* Remove unused code and exception declarations

* Address comments

* Remove redundant Exception declarations

* Make FirehoseFactoryV2.connect() to throw IOException again
 2018-03-16 22:11:12 +01:00
Gian Merlino
7e02408510 Update versions to 0.13.0-SNAPSHOT. (#5323) 2018-02-02 12:06:38 -06:00
Roman Leventov
5787d04fad Bump Druid version to 0.12.0 (#5138) 2017-12-15 07:37:01 -08:00
Parag Jain
7c01f77b04 Parse Batch support (#5081) 
* add parseBatch and deprecate parse method in InputRowParser

add addAll method, skip max rows in memory check for it

remove parse method from implemetations

transform transformers

add string multiplier input row parser

fix withParseSpec

fix kafka batch indexing

fix isPersistRequired

comments

* add unit test

* make persist async

* review comments
 2017-12-04 16:06:16 -06:00
Fokko Driesprong
2487152b59 Update Avro to 1.8.2 (#5075) 
And add exclusions that are required to have a single version of
Apache Avro on the classpath.
 2017-11-20 20:29:17 -08:00
Roman Leventov
3541b7544b Prohibit and remove unused declarations in the processing module (#4930) 
* Prohibit and remove unused declarations in the processing module

* Fix tests

* Fix integration tests

* Suppress unused

* Try to remove SuppressWarnings unused in VirtualColumn

* Remove reset 'false positives'

* Annotate CliCommandCreator as ExtensionPoint

* Unused import warning instead of error in IntelliJ

* Fixes

* Add comment

* Fix AzureBlob

* Fix CloudFilesBlob

* Address comments

* Add Project SDK section to INTELLIJ_SETUP.md

* Fix image
 2017-11-09 09:27:27 -08:00
Fokko Driesprong
21e1bf68f6 Update Avro to 1.8.0 (#5015) 
The druid parquet extensions uses Avro 1.8 and therefore it is
required to update the Avro version itself also to 1.8 to avoid
classpath conflicts
 2017-11-02 09:08:41 -06:00
Roman Leventov
dc7cb117a1 Refactor ColumnSelectorFactory; Rely on ColumnValueSelector's polymorphism (#4886) 
* Refactor ColumnSelectorFactory; Rely on ColumnValueSelector's polymorphism

* Fix MapVirtualColumn.makeColumnValueSelector()

* Minor fixes

* Fix IndexGeneratorCombinerTest

* DimensionSelector to return zeros when treated as numeric ColumnValueSelector

* Fix IncrementalIndexTest

* Fix IncrementalIndex.makeColumnSelectorFactory()

* Optimize MapBasedRow.getMetric()

* Fix VarianceAggregatorTest

* Simplify IncrementalIndex.makeColumnSelectorFactory()

* Address comments

* More comments

* Test
 2017-10-13 21:44:17 -05:00
Gian Merlino
1f2074c247 Bump versions in master to 0.11.1-SNAPSHOT. (#4878) 
* Bump versions in master to 0.11.1-SNAPSHOT.

* Missed a few.
 2017-09-28 17:09:51 -05:00
Gian Merlino
bf8fd4c203 Add flattenSpec support to the Avro parser. (#4832) 
* Add flattenSpec support to the Avro parser.

Also:

- Refactor the JSONPathParser a bit so it can share flattening code
  with Avro (see ObjectFlatteners).
- Remove the JSONParser. It was only used in two places: by
  UriNamespaceExtractor, and as a base for JSONToLowerParser. Migrated
  the former to JSONPathParser and made the latter a standalone.
- Move GenericRecordAsMap to the Parquet extension, since the Avro
  extension no longer uses it.

* Fix indentation.

* Fix equals/hashCode.
 2017-09-26 09:26:06 -07:00
Roman Leventov
cbd1902db8 Add forbidden-apis plugin; prohibit using system time zone (#4611) 
* Forbidden APIs WIP

* Remove some tests

* Restore io.druid.math.expr.Function

* Integration tests fix

* Add comments

* Fix in SimpleWorkerProvisioningStrategy

* Formatting

* Replace String.format() with StringUtils.format() in RemoteTaskRunnerTest

* Address comments

* Fix GroupByMultiSegmentTest
 2017-08-21 13:02:42 -07:00
Roman Leventov
c0beb78ffd Enforce brace formatting with Checkstyle (#4564) 2017-07-21 10:26:59 -05:00
Roman Leventov
60cdf94677 Add PMD and prohibit unnecessary fully qualified class names in code (#4350) 
* Add PMD and prohibit unnecessary fully qualified class names in code

* Extra fixes

* Remove extra unnecessary fully-qualified names

* Remove qualifiers

* Remove qualifier
 2017-07-17 22:22:29 +09:00
Roman Leventov
9ae457f7ad Avoid using the default system Locale and printing to System.out in production code (#4409) 
* Avoid usages of Default system Locale and printing to System.out or System.err in production code

* Fix Charset in DruidKerberosUtil

* Remove redundant string format in GenericIndexed

* Rename StringUtils.safeFormat() to unimportantSafeFormat(); add StringUtils.format() which fails as well as String.format()

* Fix testSafeFormat()

* More fixes of redundant StringUtils.format() inside ISE

* Rename unimportantSafeFormat() to nonStrictFormat()
 2017-06-29 14:06:19 -07:00
Roman Leventov
ae900a4934 Update versions to 0.11.0-SNAPSHOT (#4483) 2017-06-28 17:05:58 -07:00
Roman Leventov
5285eb961b Update dependencies (#4313) 
* Update dependencies

* Downgrade curator

* Rollback aws-java-sdk dependency to 1.10.77

* Revert exclusions in integration-tests

* Depend only on aws-java-sdk-ec2 instead of umbrella aws-java-sdk (fixes #4382)
 2017-06-09 14:32:07 -07:00
Roman Leventov
31d33b333e Make using implicit system Charset an error (#4326) 
* Make using implicit system charset an error

* Use StringUtils.toUtf8() and fromUtf8() instead of String.getBytes() and new String()

* Use English locale in StringUtils.safeFormat()

* Restore comment
 2017-06-05 23:57:25 -07:00
Roman Leventov
b7a52286e8 Make @Override annotation obligatory (#4274) 
* Make MissingOverride an error

* Make travis stript to fail fast

* Add missing Override annotations

* Comment
 2017-05-16 13:30:30 -05:00
Benedict Jin
e823085866 Improve collection related things that reusing a immutable object instead of creating a new object (#4135) 2017-05-17 01:38:51 +09:00
Fokko Driesprong
5ca67644e7 Remove slf4j as dependencies (#4233) 
From the kafka-schema-registry-client in the avro extension slf4j
will be packaged into the distribution. We don't want this as it
will conflict and throw a slf4j multiple bindings warning.
This will cause slf4j to fall back to no-operation (NOP) binding.
 2017-05-12 15:59:14 +09:00
Gian Merlino
3b92220015 Reduce log spam from Avro decoders. (#4205) 
These objects get constructed semi-frequently (any time a parser is
deserialized) and so info logs are spammy. They'll still appear in
task logs at least once, since they're part of the task definition and
will get logged due to that.
 2017-04-25 23:59:59 +05:30
Benedict Jin
de815da942 Some code refactor for better performance of Avro-Extension (#4092) 
* 1. Collections.singletonList instand of Arrays.asList; 2. close FSDataInputStream/ByteBufferInputStream for releasing resource; 3. convert com.google.common.base.Function into java.util.function.Function; 4. others code refactor

* Put each param on its own line for  code style

* Revert GenericRecordAsMap back about `Function`
 2017-04-25 12:46:32 +09:00
Gian Merlino
2ca7b00346 Update versions to 0.10.1-SNAPSHOT. (#4191) 2017-04-20 18:12:28 -07:00
Benedict Jin
23f77ebd20 Explain Avro's unnecessary EOFException (#4098) (#4100) 
* Explain Avro's unnecessary EOFException (#4098)

* add jira link into log message
 2017-03-24 10:45:45 -05:00
Gian Merlino
12317fd001 Bump version to 0.10.0-SNAPSHOT. (#3913) 2017-02-06 17:54:35 -08:00
Jihoon Son
d80bec83cc Enable auto license checking (#3836) 
* Enable license checking

* Clean duplicated license headers
 2017-01-10 18:13:47 -08:00
Nicolas Colomer
37ecffb648 Add support for Confluent Schema Registry in the avro extension (#3529) 2016-11-08 16:10:45 -06:00
Akash Dwivedi
4b3bd8bd63 Migrating java-util from Metamarkets. (#3585) 
* Migrating java-util from Metamarkets.

* checkstyle and updated license on java-util files.

* Removed unused imports from whole project.

* cherry pick metamx/java-util@826021f.

* Copyright changes on java-util pom, address review comments.
 2016-10-21 14:57:07 -07:00
Gian Merlino
40f2fe7893 Bump versions to 0.9.3-SNAPSHOT (#3524) 2016-09-29 13:53:32 -07:00
Himanshu
a069257d37 avro-extension -- feature to specify multiple avro reader schemas inline (#3368) 
* rename SimpleAvroBytesDecoder to InlineSchemaAvroBytesDecoder

* feature to specify multiple schemas inline in avro module
 2016-09-13 14:54:31 -07:00
Stéphane Derosiaux
48dce88aab Add flag binaryAsString for parquet ingestion (#3381) 2016-08-30 17:30:50 -07:00
Himanshu
46da682231 avro-extensions -- feature to specify avro reader schema inline in the task json for all events (#3249) 2016-08-10 10:49:26 -07:00
du00cs
ebd654228b fix: avro types exception in sketch (#3167) 2016-06-22 15:54:20 -05:00
Gian Merlino
ebf890fe79 Update master version to 0.9.2-SNAPSHOT. (#3133) 2016-06-13 13:10:38 -07:00
Gian Merlino
7e7a886f65 Move druid-api into the druid repo. 
This is from druid-api-0.3.17, as of commit 51884f1d05d5512cacaf62cedfbb28c6ab2535cf
in the druid-api repo.
 2016-03-24 11:04:34 -07:00
jon-wei
a59c9ee1b1 Support use of DimensionSchema class in DimensionsSpec 2016-03-21 13:12:04 -07:00
Gian Merlino
738dcd8cd9 Update version to 0.9.1-SNAPSHOT. 
Fixes #2462
 2016-03-17 10:34:20 -07:00
fjy
e3e932a4d4 refactor extensions into core and contrib 2016-03-08 17:12:09 -08:00