* Add lookup optimization for InDimFilter
* tests for in filter with lookup extraction fn
* refactor
* refactor2 and modified filter test
* make optimizeLookup private
* Datasource as lookup tier
* Adds an option to let indexing service tasks pull their lookup tier from the datasource they are working for.
* Fix bad docs for lookups lookupTier
* Add Datasource name holder
* Move task and datasource to be pulled from Task file
* Make LookupModule pull from bound dataSource
* Fix test
* Fix code style on imports
* Fix formatting
* Make naming better
* Address code comments about naming
* Allows RegisteredLookupExtractionFn to find its lookups lazily
* Use raw variables instead of AtomicReference
* Make sure to use volatile
* Remove extra local variable.
* Move from BAOS to ByteBuffer
* new interval based cost function
Addresses issues with balancing of segments in the existing cost function
- `gapPenalty` led to clusters of segments ~30 days apart
- `recencyPenalty` caused imbalance among recent segments
- size-based cost could be skewed by compression
New cost function is purely based on segment intervals:
- assumes each time-slice of a partition is a constant cost
- cost is additive, i.e. cost(A, B union C) = cost(A, B) + cost(A, C)
- cost decays exponentially based on distance between time-slices
* comments and formatting
* add more comments to explain the calculation
* Allow user to set cost balancer threads more than the number of cores.
Allow user to set cost balancer threads more than the number of cores.
* modify test
* Optimize filter for timeseries, search, and select queries
* exception at failed toolchest type check
* took out query type check
* java7 error fix and test improvement
* Make URIExtraction not require FileSystem impls for URIs it understands
* Fixes#2928
* Preserve URI information
* Simply case for exact matching
* Move unused variable
* Optimize CostBalancerStrategy
Ignore benchmark test in normal run
fix test
review comments
fix compilation
fix test
* review comments
* review comment
* Make S3DataSegmentPuller do GET requests less often
* Fixes#2894
* Run intellij formatting on S3Utils
* Remove forced stream fetching on getVersion
* Remove unneeded finalize
* Allow initial object fetching to fail and be retried
* Fix parsing fail of segment id with underscored datasource (Fix for #2786)
* addressed comment
* renamed and moved code into api. added log4 dependency for tests
* addressed comments
* fixed test fails
Dave Li
Fangjin Yang
Charles Allen
Nishant
Gian Merlino
Xavier Léauté
Shekhar Gulati
Parag Jain
Himanshu
Slim
Jonathan Wei
Joe Pettersson
Walton Seymour
David Lim
Navis Ryu
binlijin