Commit Graph

6127 Commits

Author SHA1 Message Date
Xavier Léauté af86c0e6ea update druid-api + java-util for timstamp parsing speedup 2015-09-21 09:57:29 -07:00
Himanshu Gupta 117a6754b5 improving msging when indexing service is not found 2015-09-18 23:07:44 -05:00
Fangjin Yang 6605a60ef0 Merge pull request #1747 from jon-wei/realtime_corrupt_sink_fix
Account for potential gaps in hydrants in sink initialization, hydrant swapping (e.g. h0, h1, h4)
2015-09-18 15:22:50 -07:00
jon-wei 9f6bb03ef4 Account for potential gaps in hydrants in sink initialization, hydrant swapping (e.g. h0, h1, h4) 2015-09-18 14:53:37 -07:00
Charles Allen d325bb55ae Merge pull request #1744 from metamx/memcached-connection-pooling
Memcached connection pooling
2015-09-18 10:24:56 -07:00
Xavier Léauté df4c8a3aa5 balance memcached cache across multiple connections 2015-09-18 09:52:16 -07:00
Xavier Léauté 1f897257b5 add simple load balancing pool 2015-09-18 09:43:14 -07:00
Gian Merlino aaa8a88464 Merge pull request #1739 from jon-wei/segment_realtime
Allow SegmentAnalyzer to read columns from StorageAdapter, allow SegmentMetadataQuery to query IncrementalIndexSegments on realtime node
2015-09-17 18:36:53 -07:00
Fangjin Yang ea2a7733f5 Merge pull request #1751 from metamx/softerIndexTestConcurrency
Soften concurrency requirements on IncrementalIndexTest
2015-09-17 17:21:01 -07:00
Charles Allen df4c2bab10 Soften concurrency requirements on IncrementalIndexTest 2015-09-17 15:51:07 -07:00
Fangjin Yang e48f6dd660 Merge pull request #1736 from gianm/additional-ingest-segment-timeline-test
IngestSegmentFirehostFactoryTimelineTest for overshadowing of the middle of a segment.
2015-09-17 14:42:29 -07:00
Fangjin Yang aab8c627c6 Merge pull request #1741 from metamx/nicerConcurrencyTesting
Add better concurrency testing to IncrementalIndexTest
2015-09-17 08:35:40 -07:00
Himanshu Gupta a7b1cacbbf make NumberedShardSpec "elastic" and UTs 2015-09-17 08:38:48 -05:00
Himanshu Gupta 22dfa8ecf7 change ParitionHolder.isComplete() to accept chunks beyond the "end" 2015-09-16 21:33:11 -05:00
jon-wei 367c50d4ba Allow SegmentAnalyzer to read columns from StorageAdapter, allow SegmentMetadataQuery to query IncrementalIndexSegments on realtime node 2015-09-16 18:39:31 -07:00
Fangjin Yang 46a520c07c Merge pull request #1743 from dclim/example-segment-publisher-fix
Disable metadata publishing attempt in example script
2015-09-16 16:25:35 -07:00
Gian Merlino aed7cab11e Merge pull request #1742 from druid-io/add-search-filter
add docs for search filter
2015-09-16 16:09:35 -07:00
fjy c9ccf9a565 add docs for search filter 2015-09-16 16:08:59 -07:00
David Lim 13e808368f Disable metadata publishing attempt in example script
Example config no-ops DataSegmentPusher and DataSegmentAnnouncer but
still binds SegmentPublisher to MetadataSegmentPublisher which was
throwing exceptions when it tried to publish and no DB was configured.
Setting this binds NoopSegmentPublisher instead.
2015-09-16 15:57:04 -07:00
Charles Allen 6e1eb3b7fe Add better concurrency testing to IncrementalIndexTest 2015-09-16 14:04:20 -07:00
Gian Merlino 9705c5139b Merge pull request #1732 from jon-wei/segmentmeta
Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups
2015-09-16 12:36:25 -07:00
Himanshu 2ccd0a11c7 Merge pull request #1737 from gianm/hadoopyString-docs
Docs: Suggest hadoopyString parser for Hadoop.
2015-09-16 13:30:15 -05:00
Gian Merlino b29cbf97a6 Docs: Suggest hadoopyString parser for Hadoop. 2015-09-16 10:19:42 -07:00
Gian Merlino 64e33b2bcb IngestSegmentFirehostFactoryTimelineTest for overshadowing of the middle of a segment. 2015-09-16 10:17:43 -07:00
Charles Allen 42bd4f6049 Merge pull request #1695 from himanshug/allow_hadoop_based_input_row_parser
Allow writing InputRowParser extensions that use hadoop/any libraries
2015-09-16 10:16:18 -07:00
Himanshu Gupta e8b9ee85a7 HadoopyStringInputRowParser to convert stringy Text, BytesWritable etc into InputRow 2015-09-16 10:58:13 -05:00
Himanshu Gupta 74f4572bd4 Lazily deserialize "parser" to InputRowParser in DataSchema
so that user hadoop related InputRowParsers are created only when needed
this allows overlord to accept a HadoopIndexTask with a hadoopy InputRowParser
and not fail because hadoopy InputRowParser might need hadoop libraries
2015-09-16 10:58:13 -05:00
Xavier Léauté b464da438c Merge pull request #1688 from metamx/moreMemcachedMetrics
More memcached metrics
2015-09-15 15:33:51 -07:00
Himanshu 52f02a43c4 Merge pull request #1735 from druid-io/add-pivor
add pivot as a UI
2015-09-15 17:30:36 -05:00
Xavier Léauté 8a21b4cae3 Merge pull request #1697 from metamx/betterMissingQTLLogging
Better logging of URIExtractionNamespace failures due to missing files
2015-09-15 15:29:27 -07:00
fjy beab6fd487 add pivot as a UI 2015-09-15 14:58:32 -07:00
Charles Allen f5ed6e885c Merge pull request #1702 from himanshug/double_datasource_in_storage_dir
do not have dataSource twice in path to segment storage on hdfs
2015-09-15 14:00:35 -07:00
Fangjin Yang 8b071a7230 Merge pull request #1710 from metamx/incrementalIndexConcurrentTestLatching
Add some basic latching to concurrency testing in IncrementalIndexTest
2015-09-15 13:55:52 -07:00
Fangjin Yang 518a1a3f9e Merge pull request #1734 from metamx/test
fix broken integration-test
2015-09-15 11:35:18 -07:00
Nishant 9c5166c89c fix broken integration-test 2015-09-15 23:20:22 +05:30
jon-wei 193fb4fdfc Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups 2015-09-14 19:41:42 -07:00
Charles Allen bd605a097e Merge pull request #1731 from metamx/regex-extraction-npe
fix NPE with regex extraction function
2015-09-14 15:55:05 -07:00
Xavier Léauté 5f36e7a992 Merge pull request #1712 from metamx/searchQueryDimExtraction
Add dimension extraction functionality to SearchQuery
2015-09-14 15:26:13 -07:00
Xavier Léauté 08a527d01a fix NPE with regex extraction function 2015-09-14 14:45:30 -07:00
Charles Allen e569f4b6a7 Add dimension extraction functionality to SearchQuery
* Add IdentityExtractionFn
2015-09-14 11:36:15 -07:00
Himanshu 5ff92664f8 Merge pull request #1696 from metamx/cpuTimeReporting
Add CPU time to metrics for segment scanning.
2015-09-14 10:53:55 -05:00
Fangjin Yang 34ef81572d Merge pull request #1700 from himanshug/update_agg_test_helper
update indexing in the helper to use multiple persists and merge
2015-09-14 06:56:29 -07:00
Fangjin Yang 1548fa8afc Merge pull request #1721 from himanshug/hdfs_fs_init_fix
fix for "java.io.IOException: No FileSystem for scheme: hdfs" error
2015-09-13 08:26:01 +08:00
Xavier Léauté 1715849642 Merge pull request #1718 from metamx/task-duration
Adds task duration to indexer console for completed tasks.
2015-09-11 22:35:08 -07:00
Xavier Léauté 43012d3c48 Merge pull request #1725 from metamx/genericIndexedWriteOptimizationObjectsSorted
Don't check for sortedness if we already know GenericIndexedWriter isn't sorted
2015-09-11 22:33:53 -07:00
Charles Allen 8d3cdd8572 Don't check for sortedness if we already know GenericIndexedWriter isn't sorted 2015-09-11 16:32:09 -07:00
Fangjin Yang 5f9be8dd0a Merge pull request #1723 from guobingkun/fix_typo
fix typo in segments.md
2015-09-12 05:51:55 +08:00
Bingkun Guo e222e6b86b fix typo in segments.md 2015-09-11 16:32:50 -05:00
Himanshu Gupta b989a7054c fix for "java.io.IOException: No FileSystem for scheme: hdfs" error
aka workaround for https://issues.apache.org/jira/browse/HDFS-8750
2015-09-11 15:35:59 -05:00
Gian Merlino 252bb5a6bc Merge pull request #1720 from himanshug/faq_update
update ingestion faq to mention dataSource inputSpec
2015-09-11 12:40:34 -07:00