Commit Graph

5857 Commits

Author SHA1 Message Date
jon-wei 9f6bb03ef4 Account for potential gaps in hydrants in sink initialization, hydrant swapping (e.g. h0, h1, h4) 2015-09-18 14:53:37 -07:00
Gian Merlino 9705c5139b Merge pull request #1732 from jon-wei/segmentmeta
Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups
2015-09-16 12:36:25 -07:00
Himanshu 2ccd0a11c7 Merge pull request #1737 from gianm/hadoopyString-docs
Docs: Suggest hadoopyString parser for Hadoop.
2015-09-16 13:30:15 -05:00
Gian Merlino b29cbf97a6 Docs: Suggest hadoopyString parser for Hadoop. 2015-09-16 10:19:42 -07:00
Charles Allen 42bd4f6049 Merge pull request #1695 from himanshug/allow_hadoop_based_input_row_parser
Allow writing InputRowParser extensions that use hadoop/any libraries
2015-09-16 10:16:18 -07:00
Himanshu Gupta e8b9ee85a7 HadoopyStringInputRowParser to convert stringy Text, BytesWritable etc into InputRow 2015-09-16 10:58:13 -05:00
Himanshu Gupta 74f4572bd4 Lazily deserialize "parser" to InputRowParser in DataSchema
so that user hadoop related InputRowParsers are created only when needed
this allows overlord to accept a HadoopIndexTask with a hadoopy InputRowParser
and not fail because hadoopy InputRowParser might need hadoop libraries
2015-09-16 10:58:13 -05:00
Xavier Léauté b464da438c Merge pull request #1688 from metamx/moreMemcachedMetrics
More memcached metrics
2015-09-15 15:33:51 -07:00
Himanshu 52f02a43c4 Merge pull request #1735 from druid-io/add-pivor
add pivot as a UI
2015-09-15 17:30:36 -05:00
Xavier Léauté 8a21b4cae3 Merge pull request #1697 from metamx/betterMissingQTLLogging
Better logging of URIExtractionNamespace failures due to missing files
2015-09-15 15:29:27 -07:00
fjy beab6fd487 add pivot as a UI 2015-09-15 14:58:32 -07:00
Charles Allen f5ed6e885c Merge pull request #1702 from himanshug/double_datasource_in_storage_dir
do not have dataSource twice in path to segment storage on hdfs
2015-09-15 14:00:35 -07:00
Fangjin Yang 8b071a7230 Merge pull request #1710 from metamx/incrementalIndexConcurrentTestLatching
Add some basic latching to concurrency testing in IncrementalIndexTest
2015-09-15 13:55:52 -07:00
Fangjin Yang 518a1a3f9e Merge pull request #1734 from metamx/test
fix broken integration-test
2015-09-15 11:35:18 -07:00
Nishant 9c5166c89c fix broken integration-test 2015-09-15 23:20:22 +05:30
jon-wei 193fb4fdfc Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups 2015-09-14 19:41:42 -07:00
Charles Allen bd605a097e Merge pull request #1731 from metamx/regex-extraction-npe
fix NPE with regex extraction function
2015-09-14 15:55:05 -07:00
Xavier Léauté 5f36e7a992 Merge pull request #1712 from metamx/searchQueryDimExtraction
Add dimension extraction functionality to SearchQuery
2015-09-14 15:26:13 -07:00
Xavier Léauté 08a527d01a fix NPE with regex extraction function 2015-09-14 14:45:30 -07:00
Charles Allen e569f4b6a7 Add dimension extraction functionality to SearchQuery
* Add IdentityExtractionFn
2015-09-14 11:36:15 -07:00
Himanshu 5ff92664f8 Merge pull request #1696 from metamx/cpuTimeReporting
Add CPU time to metrics for segment scanning.
2015-09-14 10:53:55 -05:00
Fangjin Yang 34ef81572d Merge pull request #1700 from himanshug/update_agg_test_helper
update indexing in the helper to use multiple persists and merge
2015-09-14 06:56:29 -07:00
Fangjin Yang 1548fa8afc Merge pull request #1721 from himanshug/hdfs_fs_init_fix
fix for "java.io.IOException: No FileSystem for scheme: hdfs" error
2015-09-13 08:26:01 +08:00
Xavier Léauté 1715849642 Merge pull request #1718 from metamx/task-duration
Adds task duration to indexer console for completed tasks.
2015-09-11 22:35:08 -07:00
Xavier Léauté 43012d3c48 Merge pull request #1725 from metamx/genericIndexedWriteOptimizationObjectsSorted
Don't check for sortedness if we already know GenericIndexedWriter isn't sorted
2015-09-11 22:33:53 -07:00
Charles Allen 8d3cdd8572 Don't check for sortedness if we already know GenericIndexedWriter isn't sorted 2015-09-11 16:32:09 -07:00
Fangjin Yang 5f9be8dd0a Merge pull request #1723 from guobingkun/fix_typo
fix typo in segments.md
2015-09-12 05:51:55 +08:00
Bingkun Guo e222e6b86b fix typo in segments.md 2015-09-11 16:32:50 -05:00
Himanshu Gupta b989a7054c fix for "java.io.IOException: No FileSystem for scheme: hdfs" error
aka workaround for https://issues.apache.org/jira/browse/HDFS-8750
2015-09-11 15:35:59 -05:00
Gian Merlino 252bb5a6bc Merge pull request #1720 from himanshug/faq_update
update ingestion faq to mention dataSource inputSpec
2015-09-11 12:40:34 -07:00
Himanshu Gupta 075b6d4385 update ingestion faq to mention dataSource inputSpec as an option of reindexing via hadoop 2015-09-10 14:41:13 -05:00
Charles Allen d6849805ea Add some basic latching to concurrency testing in IncrementalIndexTest 2015-09-10 10:06:51 -07:00
Nishant 4681ff22ed add task duration in response for completed tasks 2015-09-10 13:51:50 +05:30
Gian Merlino e5532367e1 Merge pull request #1717 from himanshug/doc_fix
in configuration/index.md s/instantialize/initialize
2015-09-09 21:35:21 -07:00
Himanshu Gupta 39d23ffc45 in configuration/index.md s/instantialize/initialize 2015-09-09 21:18:26 -05:00
Charles Allen 7c5d2fa686 Merge pull request #1699 from himanshug/composing_emitter
composing emitter module to use multiple emitters together
2015-09-09 19:05:39 -07:00
Himanshu Gupta ebdb612933 composing emitter module to use multiple emitters together 2015-09-09 16:45:50 -05:00
Himanshu Gupta 67aa3dc153 on HDFS store segments in "dataSource/interval/.." and not in "dataSource/dataSource/interval.." 2015-09-09 11:12:01 -05:00
Himanshu Gupta fe0233adf2 removing unused imports from HadoopIndexTask 2015-09-09 11:12:01 -05:00
Himanshu Gupta 5da58e48e0 use Rule based TemporaryFolder for cleanup of temp directory/files 2015-09-09 11:10:33 -05:00
Himanshu Gupta 44911039c5 update indexing in the helper to use multiple persists and final merge to
catch further issues in aggregator implementations
2015-09-09 11:10:33 -05:00
Charles Allen 5813856819 More memcached metrics 2015-09-08 13:34:58 -07:00
Charles Allen fcf5cae81d Add CPU time to metrics for segment scanning. 2015-09-08 13:34:19 -07:00
Charles Allen 1977ac9c5d Better logging of URIExtractionNamespace failures due to missing files 2015-09-08 13:33:32 -07:00
Fangjin Yang 07266d682a Merge pull request #1694 from metamx/namespaceExtractionCacheManagerTestImprovements
Better timing and locking in NamespaceExtractionCacheManagerExecutorsTest
2015-09-04 15:06:22 -07:00
Charles Allen 0b8a3035c6 Better timing and locking in NamespaceExtractionCacheManagerExecutorsTest 2015-09-04 13:02:14 -07:00
Charles Allen 9b5889b393 Merge pull request #1703 from metamx/fix-npe
add null check for task context.
2015-09-04 11:04:12 -07:00
Nishant 47aac991ec add null check for task context.
make variable final
2015-09-04 22:19:01 +05:30
Fangjin Yang 75a582974b Merge pull request #1639 from gianm/new-plumber
New plumber
2015-09-03 18:52:57 -07:00
Himanshu 66a9a332d0 Merge pull request #1604 from metamx/task-jvm-config
Allow task to override ForkingTaskRunner tunings and jvm settings
2015-09-03 16:43:49 -05:00