Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,844 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3355 Commits

Author SHA1 Message Date
Arpit Agarwal 365a27099c HADOOP-13467. Shell#getSignalKillCommand should use the bash builtin on Linux. (Arpit Agarwal) 2016-08-04 10:08:23 -07:00
Kihwal Lee c7fe1f5ec5 HADOOP-13442. Optimize UGI group lookups. Contributed by Daryn Sharp. 
(cherry picked from commit 9422515239)
 2016-08-04 10:48:17 -05:00
Xiao Chen bb10750067 HADOOP-13443. KMS should check the type of underlying keyprovider of KeyProviderExtension before falling back to default. Contributed by Anthony Young-Garner. 
(cherry picked from commit 05db64913d)
 2016-08-04 16:08:25 +08:00
Wei-Chiu Chuang 5964d65a4d HADOOP-13458. LoadBalancingKMSClientProvider#doOp should log IOException stacktrace. Contributed by Chen Liang. 
(cherry picked from commit a1f6564d31)
 2016-08-03 14:27:33 -07:00
Jason Lowe 5a34fa670e HADOOP-10980. TestActiveStandbyElector fails occasionally in trunk. Contributed by Eric Badger 
(cherry picked from commit c82745432a)
 2016-08-03 20:19:52 +00:00
Kihwal Lee 9d62caa440 HADOOP-13483. Optimize IPC server protobuf decoding. Contributed by Daryn Sharp. 
(cherry picked from commit 580a833496)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java
 2016-08-03 14:12:13 -05:00
Kihwal Lee dd21d242eb HADOOP-13426. More efficiently build IPC responses. Contributed by Daryn Sharp. 
(cherry picked from commit 2d8227605f)
 2016-08-03 09:33:04 -05:00
Arpit Agarwal 9850cf07fa HDFS-9353. Code and comment mismatch in JavaKeyStoreProvider. (Andras Bokor) 2016-08-02 15:18:01 -07:00
Arpit Agarwal 158dd2d5a2 HADOOP-13434. Add bash quoting to Shell class. (Owen O'Malley) 2016-08-02 13:40:56 -07:00
Chris Nauroth ec522a1907 HADOOP-13081. add the ability to create multiple UGIs/subjects from one kerberos login. Contributed by Sergey Shelukhin. 
(cherry picked from commit 0458a2af6e)
 2016-08-02 12:43:51 -07:00
Kihwal Lee 41882bd5ff HADOOP-13429. Dispose of unnecessary SASL servers. Contributed by Daryn Sharp. 
(cherry picked from commit b3018e73cc)
 2016-08-02 10:52:10 -05:00
Akira Ajisaka 5469e128d3 HADOOP-13444. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 2016-08-02 10:17:33 +09:00
Xiao Chen 8fe4b2429a HADOOP-13381. KMS clients should use KMS Delegation Tokens from current UGI. Contributed by Xiao Chen. 
(cherry picked from commit 8ebf2e95d2)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2016-07-28 18:39:59 -07:00
Xiao Chen 24d464a150 HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge 
(cherry picked from commit d9aae22fdf)
 2016-07-28 16:41:31 -07:00
Chris Nauroth df2ed6b2c4 HADOOP-13422. ZKDelegationTokenSecretManager JaasConfig does not work well with other ZK users in process. Contributed by Sergey Shelukhin. 
(cherry picked from commit 255ea45e50)
 2016-07-26 15:33:30 -07:00
Chris Nauroth e6ba69b4db HADOOP-13207. Specify FileSystem listStatus, listFiles and RemoteIterator. Contributed by Steve Loughran. 
(cherry picked from commit 5fda66e4a3)
 2016-07-22 15:38:34 -07:00
Chris Nauroth 382bcf21e7 HADOOP-13240. TestAclCommands.testSetfaclValidations fail. Contributed by John Zhuge. 
(cherry picked from commit 43cf6b101d)
 2016-07-21 14:12:39 -07:00
Jakob Homan 43a40fa009 HADOOP-12009. Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus. backport to 2.8 2016-07-21 21:50:38 +01:00
Steve Loughran 5490ed5f87 HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem (Rajesh Balamohan) 2016-07-20 13:42:33 +01:00
Akira Ajisaka 8fe2c450b3 HADOOP-12991. Conflicting default ports in DelegateToFileSystem. Contributed by Kai Sasaki. 
(cherry picked from commit dc065dd64c)
 2016-07-19 16:06:16 -07:00
Allen Wittenauer 84c785036f HDFS-9852. hdfs dfs -setfacl error message is misleading (Wei-Chiu Chuang via aw) 
(cherry picked from commit b3649adf6a)
 2016-07-19 14:40:51 -07:00
Akira Ajisaka 09375baad1 HADOOP-13202. Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes. Contributed by Kai Sasaki. 
(cherry picked from commit c2bcffb34e)
 2016-07-18 11:50:27 -07:00
Robert Kanter 04c8294b6f HADOOP-13254. Create framework for configurable disk checkers (yufeigu via rkanter) 
(cherry picked from commit 442162048a)
 2016-07-15 13:41:41 -07:00
Akira Ajisaka 2231ef22f5 HADOOP-13289. Remove unused variables in TestFairCallQueue. Contributed by Ye Zhou. 
(cherry picked from commit 1381483233)
 2016-07-14 23:15:34 -07:00
Zhe Zhang b95f1af8a9 HADOOP-13290. Appropriate use of generics in FairCallQueue. Contributed by Jonathan Hung. 
(cherry picked from commit 728bf7f698)
 2016-07-13 16:38:09 -07:00
Akira Ajisaka ee1e45e4b2 HADOOP-13366. Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc. Contributed by Rakesh R. 
(cherry picked from commit 505ce88437)
 2016-07-13 13:08:06 -07:00
Tsuyoshi Ozawa 22e719b16e HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Vinayakumar B, Yongjun Zhang, and Brahma Reddy Battula. (ozawa) 
(cherry picked from commit 77ffe76212)
 2016-07-13 21:28:20 +09:00
Lei Xu 82c3c66dcb HADOOP-13315. FileContext#umask is not initialized properly. (John Zhuge via lei) 
(cherry picked from commit a290a98b6a)
 2016-07-12 16:50:12 -07:00
Steve Loughran 89121745c0 HADOOP-13139 Branch-2: S3a to use thread pool that blocks clients. Contributed by Pieter Reuse. 2016-07-11 17:21:00 +01:00
Akira Ajisaka f4b63a391b Revert "HADOOP-10465. Fix use of generics within SortedMapWritable. Contributed by Bertrand Dechoux." 
This reverts commit 1d175f0f3f.
 2016-07-09 21:00:13 -07:00
Jitendra Pandey e4023f8b13 HADOOP-13352. Make X-FRAME-OPTIONS configurable in HttpServer2. Contributed by Anu Engineer. 2016-07-08 14:20:00 -07:00
Jitendra Pandey 63b64a20c2 HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu. 2016-07-07 15:00:34 -07:00
Akira Ajisaka f78be97469 MAPREDUCE-6717. Remove deprecated StringUtils.getFormattedTimeWithDiff. Contributed by Shen Yinjie. 
(cherry picked from commit d1626a7b37)
 2016-07-07 11:19:33 -07:00
Jian He 46f8204992 YARN-5296. NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit d792a90206)
 2016-07-05 18:06:03 -07:00
Jitendra Pandey 06b89f0d14 HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu. 2016-07-01 15:35:50 -07:00
Vinayakumar B 1458e66593 HADOOP-13314. Remove 'package-info.java' from 'test\java\org\apache\hadoop\fs\shell\' to remove eclipse compile error (Contributed by Vinayakumar B) 
(cherry picked from commit c25021fb71)
 2016-07-01 12:21:47 +05:30
Andrew Wang af581c0e23 HADOOP-12345. Pad hostname correctly in CredentialsSys.java. Contributed by Pradeep Nayak Udupi Kadbet. 
(cherry picked from commit 846ada2de3)
 2016-06-29 17:32:49 -07:00
Masatake Iwasaki b7b5807d2c HADOOP-12242. Add in-page TOC to filesystem specification pages. (iwasakims) 
(cherry picked from commit 5c07c573db)
 2016-06-30 07:23:30 +09:00
Steve Loughran a82ea9275e HADOOP-12229 Fix inconsistent subsection titles in filesystem.md. Contributed by Masatake Iwasaki 2016-06-29 14:31:27 +01:00
Andrew Wang 22eb53eb71 HADOOP-13316. Enforce Kerberos authentication for required ops in DelegationTokenAuthenticator. Contributed by Xiao Chen. 
(cherry picked from commit 7d2070493e)
 2016-06-27 18:46:50 -07:00
Andrew Wang d8a69c8737 HADOOP-13251. Authenticate with Kerberos credentials when renewing KMS delegation token. Contributed by Xiao Chen. 
(cherry picked from commit 771f798edf)

 Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2016-06-27 18:29:39 -07:00
Arpit Agarwal f73e748754 HADOOP-13263. Reload cached groups in background after expiry. (Contributed bt Stephen O'Donnell) 2016-06-27 09:49:04 -07:00
Tsz-Wo Nicholas Sze 20ee619e32 HADOOP-13227. AsyncCallHandler should use an event driven architecture to handle async calls. 2016-06-25 15:46:33 +08:00
Sangjin Lee ae90d4dd90 HDFS-7959. WebHdfs logging is missing on Datanode (Kihwal Lee via sjlee) 2016-06-24 14:48:36 -07:00
Colin Patrick Mccabe e751bb4497 Fix accidental revert of HADOOP-13072: WindowsGetSpaceUsed constructor should be public (Vinayakumar B via cmccabe) 
(cherry picked from commit 975786492c)
 2016-06-24 09:47:25 -07:00
Andrew Wang 8a1ee4d2b5 Revert "HADOOP-13228. Add delegation token to the connection in DelegationTokenAuthenticator. Contributed by Xiao Chen." 
This reverts commit 55eda958cc.
 2016-06-23 11:08:52 -07:00
Allen Wittenauer 2a956baaab HADOOP-13245. Fix up some misc create-release issues (aw) 
(cherry picked from commit e2f640942b)
 2016-06-23 15:15:23 +09:00
Masatake Iwasaki 81497a4451 HADOOP-12588. addendum patch to fix intermittent failure of TestGangliaMetrics. (iwasakims) 
(cherry picked from commit 67089875f0)
 2016-06-22 23:48:06 +09:00
Steve Loughran fb9167fdef HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads. Contributed by Rajesh Balamohan and stevel. 2016-06-22 15:42:30 +01:00
Akira Ajisaka 65b4f2682e HADOOP-13296. Cleanup javadoc for Path. Contributed by Daniel Templeton. 
(cherry picked from commit e15cd43369)
 2016-06-21 20:17:00 +09:00
Colin Patrick Mccabe 14df17383c HADOOP-13288. Guard null stats key in FileSystemStorageStatistics (Mingliang Liu via Colin P. McCabe) 
(cherry picked from commit 8c1f81d4bf)
 2016-06-20 14:30:21 -07:00
Colin Patrick Mccabe 7db1e57cc1 HADOOP-13280. FileSystemStorageStatistics#getLong(“readOps“) should return readOps + largeReadOps (Mingliang Liu via cmccabe) 
(cherry picked from commit 5370a6ffae)
 2016-06-20 10:50:30 -07:00
Akira Ajisaka 39ea0891d2 HADOOP-13192. org.apache.hadoop.util.LineReader cannot handle multibyte delimiters correctly. Contributed by binde. 
(cherry picked from commit fc6b50cc57)
 2016-06-20 17:08:02 +09:00
Brahma Reddy Battula 3f27f40503 HDFS-10474. hftp copy fails when file name with Chinese+special char in branch-2 (Contributed by Brahma Reddy Battula) 2016-06-20 12:38:18 +05:30
Arpit Agarwal bd33cc0179 HDFS-10437. ReconfigurationProtocol not covered by HDFSPolicyProvider. (Arpit Agarwal) 2016-06-18 16:17:10 -07:00
Chris Nauroth b726b5c52c Revert "HADOOP-12718. Incorrect error message by fs -put local dir without permission. (John Zhuge via Yongjun Zhang)" 
This reverts commit a1f913fb6e.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-06-17 23:50:55 -07:00
Xiaoyu Yao ed5d3a62ce HADOOP-13285. DecayRpcScheduler MXBean should only report decayed CallVolumeSummary. Contributed by Xiaoyu Yao. 
(cherry picked from commit 0761379fe4)
 2016-06-17 15:30:51 -07:00
Akira Ajisaka a36aa920f4 HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang. 
(cherry picked from commit 09e82acaf9)
 2016-06-17 16:21:05 +09:00
Xiaoyu Yao ddf66427ff HADOOP-13255. KMSClientProvider should check and renew tgt when doing delegation token operations. Contributed by Xiao Chen. 2016-06-16 20:12:17 -07:00
Akira Ajisaka 1ef89a767d MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 
(cherry picked from commit 51d497fa93)
 2016-06-17 11:05:13 +09:00
Konstantin V Shvachko 77ed9bafdd HADOOP-13189. FairCallQueue makes callQueue larger than the configured capacity. Contributed by Vinitha Gankidi. 2016-06-16 18:26:46 -07:00
Colin Patrick Mccabe 56c997a1a6 HADOOP-12975. Add jitter to CachingGetSpaceUsed's thread (Elliott Clark via Colin P. McCabe) 
(cherry picked from commit bf780406f2)
 2016-06-16 16:59:46 -07:00
Chris Nauroth e4b63ed7b4 HADOOP-13241. document s3a better. Contributed by Steve Loughran. 
(cherry picked from commit 127d2c7281)
 2016-06-16 11:18:40 -07:00
Vinayakumar B 6ab1f545fb HDFS-10256. Use GenericTestUtils.getTestDir method in tests for temporary directories (Contributed by Vinayakumar B) 
(cherry picked from commit cc6c265171)
 2016-06-16 16:56:27 +05:30
Jitendra Pandey 14b849489a HADOOP-12291. Add support for nested groups in LdapGroupsMapping. Contributed by Esther Kundin. 2016-06-15 13:29:29 -07:00
Masatake Iwasaki 9c66fffe4a HADOOP-13244. o.a.h.ipc.Server#Server should honor handlerCount when queueSizePerHandler is specified in consturctor. Contributed by Kai Sasaki. 
(cherry picked from commit 20b13d109a)
 2016-06-14 19:46:33 +09:00
Vinayakumar B 06b76f1586 HADOOP-13179. GenericOptionsParser is not thread-safe because commons-cli OptionBuilder is not thread-safe. (Contributed by hongbin ma) 
(cherry picked from commit 5cd19e429e7aef8751599dc5f54ffe29a9dc3ea9)
 2016-06-14 15:59:07 +05:30
Akira Ajisaka d219550e8b HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki. 
(cherry picked from commit e3ba9ad3f1)
 2016-06-14 10:20:43 +09:00
Andrew Wang 6a8d3233fc HADOOP-13079. Add -q option to Ls to print ? instead of non-printable characters. Contributed by John Zhuge. 
(cherry picked from commit 0accc3306d)
 2016-06-13 11:43:56 -07:00
Karthik Kambatla 63da641910 HADOOP-13243. TestRollingFileSystemSink.testSetInitialFlushTime() fails intermittently. (Daniel Templeton via kasha) 
(cherry picked from commit 49b4064644)
 2016-06-12 14:28:19 -07:00
Akira Ajisaka 035eea4d12 HADOOP-13213. Small Documentation bug with AuthenticatedURL in hadoop-auth. Contributed by Tom Ellis. 
This closes #97.

(cherry picked from commit 8a1dccecce)
 2016-06-11 03:33:19 +09:00
Jing Zhao 9c7002b122 HADOOP-13249. RetryInvocationHandler need wrap InterruptedException in IOException when call Thread.sleep. Contributed by Zhihai Xu. 
(cherry picked from commit 0bbb4ddd79)
 2016-06-10 10:40:53 -07:00
Wangda Tan e91e9e596d YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 
(cherry picked from commit 03fc6b1bb0)
 2016-06-10 09:53:40 -07:00
Chris Nauroth 1925942294 Revert "HADOOP-12666. Support Microsoft Azure Data Lake - as a file system in Hadoop. Contributed by Vishwajeet Dusane." 
This reverts commit a8f03ef7ea.
 2016-06-09 15:17:18 -07:00
Chris Nauroth a8f03ef7ea HADOOP-12666. Support Microsoft Azure Data Lake - as a file system in Hadoop. Contributed by Vishwajeet Dusane. 
(cherry picked from commit 9581fb715c)

Conflicts:
	hadoop-tools/hadoop-tools-dist/pom.xml
	hadoop-tools/pom.xml
 2016-06-09 14:49:05 -07:00
Steve Loughran 6350e4b1fb HADOOP-12537 S3A to support Amazon STS temporary credentials. Contributed by Sean Mackrory. 2016-06-09 20:59:54 +01:00
Steve Loughran df29f77846 HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials. Contributed by Chris Nauroth 2016-06-09 16:36:27 +01:00
Junping Du 19eb997f67 HADOOP-10048. LocalDirAllocator should avoid holding locks while accessing the filesystem. Contributed by Jason Lowe. 
(cherry picked from commit c14c1b298e)
 2016-06-07 09:20:51 -07:00
Tsz-Wo Nicholas Sze 074588d78b Revert "Revert "HADOOP-13226 Support async call retry and failover."" 
This reverts commit 5bfadb8e6b.
 2016-06-06 16:52:24 +08:00
Tsz-Wo Nicholas Sze 09a543fc86 Revert "Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls."" 
This reverts commit 071aeab585.
 2016-06-06 16:52:02 +08:00
Tsz-Wo Nicholas Sze 7b9062f405 Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"" 
This reverts commit 7d4c16bae9.
 2016-06-06 16:51:48 +08:00
Tsz-Wo Nicholas Sze c1947a71ff Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"" 
This reverts commit c11c2ee649.
 2016-06-06 16:51:41 +08:00
Andrew Wang c11c2ee649 Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou" 
This reverts commit d316f44059.
 2016-06-03 18:11:12 -07:00
Andrew Wang 7d4c16bae9 Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou" 
This reverts commit 7e0824faa9.
 2016-06-03 18:10:49 -07:00
Andrew Wang 071aeab585 Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls." 
This reverts commit 658072d62e.
 2016-06-03 18:10:49 -07:00
Andrew Wang 5bfadb8e6b Revert "HADOOP-13226 Support async call retry and failover." 
This reverts commit f7189d6a1c.
 2016-06-03 18:10:47 -07:00
Andrew Wang 03c4724c88 HADOOP-13155. Implement TokenRenewer to renew and cancel delegation tokens in KMS. Contributed by Xiao Chen. 
(cherry picked from commit 713cb71820)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyProviderDelegationTokenExtension.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java
 2016-06-03 17:01:11 -07:00
Chris Nauroth 5e6ee5aafb HADOOP-13105. Support timeouts in LDAP queries in LdapGroupsMapping. Contributed by Mingliang Liu. 
(cherry picked from commit d82bc85018)
 2016-06-03 16:38:38 -07:00
Jian He 7f865bf438 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29)
 2016-06-03 11:11:20 -07:00
Chris Nauroth 043a0c2e6c HADOOP-13171. Add StorageStatistics to S3A; instrument some more operations. Contributed by Steve Loughran. 2016-06-03 08:56:07 -07:00
Akira Ajisaka b6d5546e24 HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (aajisaka) 2016-06-03 16:04:11 +09:00
Masatake Iwasaki 3740974413 HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit aadb77e412)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
 2016-06-02 20:59:40 +09:00
Akira Ajisaka 6843e55208 HADOOP-13232. Typo in exception in ValueQueue.java. Contributed by Jiayi Zhou. 
(cherry picked from commit 99675e00df)
 2016-06-02 19:15:48 +09:00
Andrew Wang 55eda958cc HADOOP-13228. Add delegation token to the connection in DelegationTokenAuthenticator. Contributed by Xiao Chen. 
(cherry picked from commit 35356de1ba)
 2016-06-01 13:13:28 -07:00
Steve Loughran b29bf5d335 HADOOP-13162. Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs. (Rajesh Balamohan via stevel) 2016-06-01 14:17:50 +01:00
Colin Patrick Mccabe a64e1425ed HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe) 
(cherry picked from commit 8ceb06e239)
 2016-05-31 18:05:28 -07:00
Tsz-Wo Nicholas Sze f7189d6a1c HADOOP-13226 Support async call retry and failover. 2016-05-31 17:01:43 -07:00
Andrew Wang cbf87869b8 HADOOP-13132. Handle ClassCastException on AuthenticationException in LoadBalancingKMSClientProvider. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit bca31fe276)
 2016-05-31 09:35:28 -07:00
Xiaoyu Yao 001c8f5d7b HADOOP-13197. Add non-decayed call metrics for DecayRpcScheduler. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4ca8859583)
 2016-05-27 18:48:55 -07:00
Yongjun Zhang 41c02757a5 HADOOP-12847. hadoop daemonlog should support https and SPNEGO for Kerberized cluster. (Wei-Chiu Chuang via Yongjun Zhang) 2016-05-26 22:54:07 -07:00
Karthik Kambatla b4c8729cf9 HDFS-10463. TestRollingFileSystemSinkWithHdfs needs some cleanup. (Daniel Templeton via kasha) 2016-05-26 14:39:07 -07:00
Kai Zheng 4c8ab5e162 HADOOP-12782. Faster LDAP group name resolution with ActiveDirectory. Contributed by Wei-Chiu Chuang 2016-05-27 17:16:32 +08:00
Karthik Kambatla afe603d973 HDFS-9782. RollingFileSystemSink should have configurable roll interval. (Daniel Templeton via kasha) 
(cherry picked from commit 57c31a3fef)
 2016-05-24 16:34:45 -07:00
Steve Loughran 15f997cca6 HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran) 2016-05-21 14:19:56 +01:00
Chris Nauroth d38f2090fa HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth. 
(cherry picked from commit bcde1562d2)
 2016-05-20 12:47:52 -07:00
Steve Loughran eb160561e0 HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong. 2016-05-20 13:52:30 +01:00
Tsz-Wo Nicholas Sze 658072d62e HADOOP-13168. Support Future.get with timeout in ipc async calls. 2016-05-19 15:58:15 -07:00
Jian He 2d846fdcbd YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca)
 2016-05-19 13:02:05 -07:00
Steve Loughran 151ecdf26d HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:44:44 +01:00
Andrew Wang 686691df60 HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder. 2016-05-18 14:55:31 -07:00
Chris Nauroth 42f64b3bcc HADOOP-13138. Unable to append to a SequenceFile with Compression.NONE. Contributed by Vinayakumar B. 
(cherry picked from commit 0a527196dcb35a699e12121165bc3738766b569e)
 2016-05-18 09:32:15 -07:00
Colin Patrick Mccabe feabfe9cda HADOOP-13140. FileSystem#initialize must not attempt to create StorageStatistics objects with null or empty schemes (Mingliang Liu via cmccabe) 
(cherry picked from commit f2c1d9181f)
 2016-05-18 09:19:45 -07:00
Akira Ajisaka e9dd0d84e5 Revert "HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder." 
This reverts commit e205421555.
 2016-05-18 17:57:46 +09:00
Andrew Wang e205421555 HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder. 
(cherry picked from commit 7154ace712)
 2016-05-17 17:44:34 -07:00
Xiaoyu Yao 937bc924cb HADOOP-13159. Fix potential NPE in Metrics2 source for DecayRpcScheduler. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9478484845)
(cherry picked from commit 744d5edc87)
 2016-05-17 15:51:47 -07:00
Akira Ajisaka 7f03c1c90e HDFS-10242. Cannot create space quota of zero. Contributed by Takashi Ohnishi. 
(cherry picked from commit 9fe5828f05)
 2016-05-17 15:56:21 +09:00
Sangjin Lee 9330a7b4de HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee) 
(cherry picked from commit 61f46be071)
 2016-05-16 18:59:19 -07:00
Jing Zhao af70fc42e7 HADOOP-13146. Refactor RetryInvocationHandler. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit a9a8297cad)
 2016-05-16 15:26:48 -07:00
Sangjin Lee 8a316676b7 HADOOP-12971. FileSystemShell doc should explain relative path (John Zhuge via sale) 
(cherry picked from commit 1f2794b4fa)
 2016-05-13 10:02:47 -07:00
Andrew Wang d6f6b99c83 HADOOP-11180. Change log message "token.Token: Cannot find class for token kind kms-dt" to debug. Contributed by Yi Liu. 
(cherry picked from commit 3bfd29949f)
 2016-05-12 23:48:58 -07:00
Steve Loughran 1370dfc775 HADOOP-13028 add low level counter metrics for S3A; use in read performance tests. contributed by: stevel 
patch includes
HADOOP-12844 Recover when S3A fails on IOException in read()
HADOOP-13058 S3A FS fails during init against a read-only FS if multipart purge
HADOOP-13047 S3a Forward seek in stream length to be configurable
 2016-05-12 19:23:31 +01:00
Sangjin Lee d6d13ec670 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 
(cherry picked from commit 0bbe01f8d5)
 2016-05-12 10:14:24 -07:00
Steve Loughran 257b2f5beb HADOOP-13122 Customize User-Agent header sent in HTTP requests by S3A. Chris Nauroth via stevel. 2016-05-12 13:57:05 +01:00
Colin Patrick Mccabe 15bfcde238 HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe) 
(cherry picked from commit 687233f20d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
 2016-05-11 14:02:03 -07:00
Larry McCay 788f80b617 HADOOP-12942. hadoop credential commands non-obviously use password of "none" (Mike Yoder via lmccay) 2016-05-11 14:38:21 -04:00
Chris Nauroth 31279ae45e HADOOP-13008. Add XFS Filter for UIs to Hadoop Common. Contributed by Larry McCay. 
(cherry picked from commit dee279b532)
 2016-05-11 10:58:42 -07:00
Steve Loughran 0ff3ec1f67 HADOOP-13125 FS Contract tests don't report FS initialization errors well. (stevel) 2016-05-11 14:19:41 +01:00
Tsuyoshi Ozawa eb4182f8f5 HADOOP-10694. Remove synchronized input streams from Writable deserialization. Contributed by Gopal V and Rajesh Balamohan. 
(cherry picked from commit 6e56578031)
 2016-05-10 17:27:43 -07:00
Eric Payne de4daf9d22 YARN-4768. getAvailablePhysicalMemorySize can be inaccurate on linux. Contributed by Nathan Roberts. 
(cherry picked from commit 6b1c1cb01c)
 2016-05-10 21:37:24 +00:00
Steve Loughran f47ea0cc3f HADOOP-12982 Document missing S3A and S3 properties. (Wei-Chiu Chuang via stevel) 2016-05-10 21:40:13 +01:00
Steve Loughran 092b199741 HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple. (Bolke de Bruin via stevel). 2016-05-10 21:33:11 +01:00
Akira Ajisaka 7cbf306e4c HADOOP-13118. Fix IOUtils#cleanup and IOUtils#closeStream javadoc. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 0f0c6415af)
 2016-05-10 10:03:41 -07:00
Harsh J e2216a96d7 HADOOP-7256. Resource leak during failure scenario of closing of resources. Contributed by Ramkrishna S. Vasudevan. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1388893 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 28023b7759)
 2016-05-10 09:49:00 -07:00
Andrew Wang 1d563f4d7e HDFS-9939. Increase DecompressorStream skip buffer size. Contributed by John Zhuge. 
(cherry picked from commit 8ef75a702e)
 2016-05-06 17:23:31 -07:00
Xiaoyu Yao 35859872b7 HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit dacd1f50fe)
 2016-05-05 18:15:32 -07:00
Xiaoyu Yao d18a991817 HADOOP-13098. Dynamic LogLevel setting page should accept case-insensitive log level string. (Junping Du via xyao) 
(cherry picked from commit 4e5e87ddd4)
 2016-05-05 17:58:41 -07:00
Tsz-Wo Nicholas Sze b66b85094a HADOOP-13103 Group resolution from LDAP may fail on javax.naming.ServiceUnavailableException 2016-05-05 15:50:50 -07:00
Ming Ma 5b53833171 HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma) 
(cherry picked from commit 1268cf5fbe)
 2016-05-04 17:04:44 -07:00
Arpit Agarwal cb0a035bd1 HADOOP-13068. Clean up RunJar and related test class. (Contributed by Andras Bokor) 2016-05-04 09:49:51 -07:00
Masatake Iwasaki 2d01ffaf24 HADOOP-12101. Add automatic search of default Configuration variables to TestConfigurationFieldsBase. Contributed by Ray Chiang. 
(cherry picked from commit 355325bcc7)
 2016-05-04 15:18:41 +09:00
Chris Douglas a50dc05a60 HADOOP-13080. Refresh time in SysInfoWindows is in nanoseconds. Contributed by Inigo Goiri 
(cherry picked from commit c1cc6ac667)
 2016-05-02 22:37:36 -07:00
Tsz-Wo Nicholas Sze 7e0824faa9 HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou 2016-05-02 11:20:10 -07:00
Colin Patrick Mccabe b16c376c3b HADOOP-13072. WindowsGetSpaceUsed constructor should be public (Vinayakumar B via cmccabe) 
(cherry picked from commit 2beedead72)
 2016-05-02 09:43:42 -07:00
Robert Kanter ac8fb579c6 Remove parent's env vars from child processes 2016-04-29 09:26:09 -07:00
Andrew Wang 7271e91b79 HADOOP-13030. Handle special characters in passwords in KMS startup script. Contributed by Xiao Chen. 2016-04-27 17:13:50 -07:00
Lei Xu 4289cb8b36 HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 0a152103f1)
 2016-04-27 15:06:35 -07:00
Arpit Agarwal 14b8701a63 HADOOP-5470. RunJar.unJar() should write the last modified time found in the jar entry to the uncompressed file. (Contributed by Andras Bakor) 2016-04-27 10:07:08 -07:00
Jian He 9d3ddb0b4d YARN-4983. JVM and UGI metrics disappear after RM transitioned to standby mode 
(cherry picked from commit 4beff01354)
 2016-04-26 21:02:04 -07:00
Arpit Agarwal dba737f1e5 HADOOP-13039. Add documentation for configuration property ipc.maximum.data.length (Contributed by Mingliang Liu) 2016-04-26 20:46:47 -07:00
Masatake Iwasaki c12fcbde44 HADOOP-12738. Create unit test to automatically compare Common related classes and core-default.xml. Contributed by Ray Chiang. 
(cherry picked from commit 68b4564e78)
 2016-04-27 10:48:26 +09:00
Tsz-Wo Nicholas Sze d316f44059 HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-04-26 17:21:20 -07:00
Masatake Iwasaki 5bceaa9683 HADOOP-11418. Property "io.compression.codec.lzo.class" does not work with other value besides default. (Yuanbo Liu via iwasakims) 
(cherry picked from commit bec5b4cd8c)
 2016-04-25 22:46:50 +09:00
Ravi Prakash 88e1427a03 HDFS-9525. hadoop utilities need to support provided delegation tokens. Contributed by HeeSoo Kim 2016-04-23 20:50:56 -07:00
Akira Ajisaka 5949436010 HADOOP-13033. Add missing Javadoc entries to Interns.java. Contributed by Andras Bokor. 
(cherry picked from commit c610031cab)
 2016-04-23 06:20:44 +09:00
Kihwal Lee d0718ed466 HADOOP-13052. ChecksumFileSystem mishandles crc file permissions. Contributed by Daryn Sharp. 
(cherry picked from commit 9dbdc8e12d)
 2016-04-22 15:03:56 -05:00
Steve Loughran edcb2a8410 HADOOP-12891. S3AFileSystem should configure Multipart Copy threshold and chunk size. (Andrew Olson via stevel) 2016-04-22 11:24:50 +01:00
Larry McCay a0605ff002 HADOOP-13011 - Clearly Document the Password Details for Keystore-based Credential Providers 2016-04-21 20:16:08 -07:00
Jing Zhao 5353289c8c HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou. 
(cherry picked from commit 7149cdb3c2d9dd390cd8668883cbe5db94090e0a)
 2016-04-21 20:16:02 -07:00
Xiaoyu Yao 432531dcf7 HDFS-10207. Support enable Hadoop IPC backoff without namenode restart. Contributed by Xiaobing Zhou. 
(cherry picked from commit b4be288c5d)
 2016-04-21 10:19:35 -07:00
Xiaoyu Yao a2a65922dd HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao. 
(cherry picked from commit 5bd7b592e5)
 2016-04-20 23:11:42 -07:00
Steve Loughran 8d11a771b3 HADOOP-13026 Should not wrap IOExceptions into a AuthenticationException in KerberosAuthenticator. Xuan Gong via stevel 2016-04-15 17:43:52 +01:00
Akira Ajisaka 5ae74507c0 HADOOP-12989. Some tests in org.apache.hadoop.fs.shell.find occasionally time out. Contributed by Takashi Ohnishi. 
(cherry picked from commit 6e6b6dd5aa)
 2016-04-15 14:15:33 +09:00
Steve Loughran e124c3a2ae HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel) 2016-04-14 12:45:10 +01:00
Steve Loughran a59e05ad0a HADOOP-12969 Mark IPC.Client and IPC.Server as @Public, @Evolving (Xiaobing Zhou via stevel) 2016-04-14 10:35:20 +01:00
Akira Ajisaka 03bf09865f HADOOP-12822. Change "Metrics intern cache overflow" log level from WARN to INFO. Contributed by Andras Bokor. 
(cherry picked from commit 27b131e79c)
 2016-04-14 10:57:50 +09:00
Robert Kanter d9d12b9e72 HADOOP-12964. Http server vulnerable to clickjacking (haibochen via rkanter) 
(cherry picked from commit 042a3ae960)
 2016-04-13 17:42:28 -07:00
Xiaoyu Yao 42040f706a HADOOP-12993. Change ShutdownHookManger complete shutdown log from INFO to DEBUG. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8ced42daff)
 2016-04-13 16:42:55 -07:00
Colin Patrick Mccabe 2b0b332e2f HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe) 
(cherry picked from commit 35f0770555)
 2016-04-12 16:29:15 -07:00
Akira Ajisaka bb1ed426d1 HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang. 2016-04-12 15:28:05 +09:00
Akira Ajisaka 8f968171bd HDFS-10277. PositionedReadable test testReadFullyZeroByteFile failing in HDFS. Contributed by Steve Loughran. 
(cherry picked from commit a409508b3f)
 2016-04-12 14:55:25 +09:00
Vinod Kumar Vavilapalli 4041d2b49e HADOOP-12406. Fixed AbstractMapWritable.readFields to use the thread's ClassLoader to load class instead of System ClassLoader. Contributed by Nadeem Douba. 
(cherry picked from commit 069c6c62de)
 2016-04-11 12:03:39 -07:00
Chris Nauroth 26a23eff82 HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran. 
(cherry picked from commit 843ee8d59d)
 2016-04-08 13:37:46 -07:00
Steve Loughran c8797e0e39 HADOOP-13006 FileContextMainOperationsBaseTest.testListStatusThrowsExceptionForNonExistentFile() doesnt run. (Kai Sasaki via stevel) 2016-04-08 12:59:01 +01:00
Andrew Wang 2f983b161f HADOOP-12951. Improve documentation on KMS ACLs and delegation tokens. Contributed by Xiao Chen. 
(cherry picked from commit 594c70f779)
 2016-04-07 23:50:36 -07:00
Tsz-Wo Nicholas Sze 46352de1a2 HADOOP-12909. Change ipc.Client to support asynchronous calls. Contributed by Xiaobing Zhou 2016-04-07 14:49:48 +08:00
Suresh Srinivas d72db7d3aa HADOOP-8813. Add InterfaceAudience and InterfaceStability annotations to RPC Server and Client classes. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1389649 13f79535-47bb-0310-9956-ffa450edef68
 2016-04-07 14:39:22 +08:00
Vinayakumar B aea518ef30 Addendum : HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for emporary directory in tests (Contributed by Steve Loughran and Vinayakumar B) 2016-04-07 14:33:48 +08:00
Vinayakumar B c2b271fe05 HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for emporary directory in tests (Contributed by Steve Loughran and Vinayakumar B 
This closes #91

(cherry picked from commit 8d29e2451f)
 2016-04-07 10:17:43 +05:30
Jian He 42bc565630 YARN-4769. Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI. Contributed by Varun Vasudev 2016-04-06 16:14:13 -07:00
Akira Ajisaka 1371f993a9 HADOOP-12022. fix site -Pdocs -Pdist in hadoop-project-dist; cleanout remaining forrest bits. Contributed by Allen Wittenauer. 2016-04-06 17:43:31 +09:00
Masatake Iwasaki 2542e9bccf HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims) 
(cherry picked from commit 85ec5573eb)
 2016-04-06 03:26:33 +09:00
Kihwal Lee ef3da82357 HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
 2016-04-05 09:16:05 -05:00
Arpit Agarwal 30e12edb67 HADOOP-11212. NetUtils.wrapException to handle SocketException explicitly. (Contributed by Steve Loughran) 2016-04-04 10:50:28 -07:00
Steve Loughran 20e808d83e HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse. 2016-04-03 16:39:51 +01:00
Akira Ajisaka 80d122d7fe HADOOP-11661. Deprecate FileUtil#copyMerge. Contributed by Brahma Reddy Battula. 
(cherry picked from commit a8d8b80a20)
 2016-04-01 13:59:45 +09:00
Xiaoyu Yao ab9dc84d4a HADOOP-12950. ShutdownHookManager should have a timeout for each of the Registered shutdown hook. Contributed by Xiaoyu Yao. 
(cherry picked from commit aac4d65bf9)
(cherry picked from commit dc78250aa4)
 2016-03-31 15:50:17 -07:00
Jian He 3afc2caec8 YARN-4811. Generate histograms in ContainerMetrics for actual container resource usage 2016-03-31 14:31:38 -07:00
Xiaoyu Yao 6db3c9d394 HADOOP-12916. Allow RPC scheduler/callqueue backoff using response times. Contributed by Xiaoyu Yao. 
(cherry picked from commit d95c6eb32c)
(cherry picked from commit a0c001c0b9)
 2016-03-31 09:05:37 -07:00
Akira Ajisaka b4fbc22688 HADOOP-12902. JavaDocs for SignerSecretProvider are out-of-date in AuthenticationFilter. Contributed by Gabor Liptak. 
(cherry picked from commit acca149ec9)
 2016-03-31 16:05:51 +09:00
Zhe ZHang 9216993b02 HADOOP-12886. Exclude weak ciphers in SSLFactory through ssl-server.xml. Contributed by Wei-Chiu Chuang. 2016-03-30 14:14:05 -07:00
Colin Patrick Mccabe 9df2e54160 HADOOP-12972. Lz4Compressor#getLibraryName returns the wrong version number (cmccabe) 
(cherry picked from commit e3d15a2e23)
 2016-03-29 17:14:36 -07:00
Arpit Agarwal 759d0acdd4 HDFS-9478. Reason for failing ipc.FairCallQueue contruction should be thrown. (Contributed by Ajith S) 2016-03-29 12:09:10 -07:00
Andrew Wang a61a90c562 HADOOP-10965. Print fully qualified path in CommandWithDestination error messages. Contributed by John Zhuge. 
(cherry picked from commit 8bfaa80037)
 2016-03-28 14:14:05 -07:00
Robert Kanter 5a552973f4 HADOOP-12954. Add a way to change hadoop.security.token.service.use_ip (rkanter) 
(cherry picked from commit 8cac1bb09f)
 2016-03-28 10:37:58 -07:00
Junping Du ae29b1a17b MAPREDUCE-6655. Fix a typo (STRICT_IE6) in Encrypted Shuffle. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 90fcb16c0f)
 2016-03-28 05:31:46 -07:00
Akira Ajisaka 42d312663f HADOOP-12873. Remove MRv1 terms from HttpAuthentication.md. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 01cfee6381)
 2016-03-28 15:58:28 +09:00
Jason Lowe 5e823d839d HADOOP-12958. PhantomReference for filesystem statistics can trigger OOM. Contributed by Sangjin Lee 
(cherry picked from commit 9a09200a1f)
 2016-03-25 22:59:14 +00:00
Andrew Wang c58599acba HADOOP-12962. KMS key names are incorrectly encoded when creating key. Contributed by Xiao Chen. 
(cherry picked from commit d4df7849a5)
 2016-03-25 15:29:06 -07:00
Andrew Wang daf24eeb6b HADOOP-12947. Update documentation Hadoop Groups Mapping to add static group mapping, negative cache. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 938222b2e2)
 2016-03-23 14:57:18 -07:00
Benoy Antony 4dd02f9484 HADOOP-12929. JWTRedirectAuthenticationHandler must accommodate null expiration time. Contributed by Larry McCay. 
(cherry picked from commit e7ed05e4f5)
 2016-03-21 13:21:54 -07:00
Andrew Wang e826d860a7 HDFS-9405. Warmup NameNode EDEK caches in background thread. Contributed by Xiao Chen. 
(cherry picked from commit e3bb38d625)
 2016-03-21 11:39:19 -07:00
Sangjin Lee d956e0a0bb HDFS-9579. Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level (Ming Ma via sjlee) 2016-03-21 10:56:32 -07:00
Haohui Mai 478a25b929 HADOOP-12819. Migrate TestSaslRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-03-20 17:41:20 -07:00
Chris Nauroth 283e44f978 HADOOP-12926. lz4.c does not detect 64-bit mode properly. Contributed by Alan Burlison. 
(cherry picked from commit 33239c9925)
 2016-03-18 16:33:28 -07:00
Steve Loughran 186a7c6bb3 YARN-4593 Deadlock in AbstractService.getConfig() (stevel) 2016-03-16 14:35:13 +00:00
Steve Loughran 1463400aad HADOOP-12888 Shell to disable bash and setsid support when running under JVM security manager (Costin Leau via stevel) 2016-03-16 14:31:50 +00:00
Tsz-Wo Nicholas Sze f85d979a3f HADOOP-12923. Move the test code in ipc.Client to test. 2016-03-14 15:49:34 -07:00
Colin Patrick Mccabe b67b5b09a7 HDFS-9942. Add an HTrace span when refreshing the groups for a username (cmccabe) 
(cherry picked from commit 6e9a582eb1)
 2016-03-11 12:27:12 -08:00
Steve Loughran 1c96fd0a23 Revert "HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims)" 
This reverts commit 7994ab3fa0.
 2016-03-11 16:59:42 +00:00
Masatake Iwasaki 7994ab3fa0 HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims) 
(cherry picked from commit 682adc6ba9)
 2016-03-11 15:05:34 +09:00
Li Lu 92125e6057 HADOOP-12906. AuthenticatedURL should convert a 404/Not Found into an FileNotFoundException. (Steve Loughran via gtcarrera9) 
(cherry-picked from commit 9a79b738c5)
 2016-03-10 11:39:44 -08:00
Harsh J 2e32aa5475 HADOOP-11404. Clarify the "expected client Kerberos principal is null" authorization message. Contributed by Stephen Chu 
(cherry picked from commit 318c9b68b0)
 2016-03-10 17:09:56 +05:30
Allen Wittenauer 95d30a6ec6 HADOOP-12798. Update changelog and release notes (2016-03-04) (aw) 
(cherry picked from commit 55f73a1cbd)
 2016-03-09 19:39:13 -08:00
Allen Wittenauer 140a33df9b HADOOP-11791. Update src/site/markdown/releases to include old versions of Hadoop (aw) 
(cherry picked from commit d759b4bd74)
 2016-03-09 19:39:05 -08:00
Allen Wittenauer 9f4b3eaac0 Revert "HADOOP-12366. expose calculated paths (aw)" 
This reverts commit 45bb467918.
 2016-03-09 09:59:19 -08:00
Arpit Agarwal a06ecb085e HADOOP-12903. IPC Server should allow suppressing exception logging by type, not log 'server too busy' messages. (Arpit Agarwal) 2016-03-08 23:30:36 -08:00
Akira Ajisaka 0d8ad99e0d HADOOP-12688. Fix deadlinks in Compatibility.md. Contributed by Gabor Liptak. 
(cherry picked from commit 4cbefafcac4e7b20a23b7b416af1a2efe71e1080)
 2016-03-09 15:33:14 +09:00
Chris Nauroth 38a031ce68 HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth. 
(cherry picked from commit 89b16d27e2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/pom.xml
	hadoop-mapreduce-project/pom.xml
 2016-03-08 21:22:57 -08:00
Andrew Wang 7e29d732ee HADOOP-12895. SSLFactory#createSSLSocketFactory exception message is wrong. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit a3cc6e2511)
 2016-03-08 13:52:01 -08:00
Akira Ajisaka 0586386472 HADOOP-12860. Expand section "Data Encryption on HTTP" in SecureMode documentation. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit f86850b544)
 2016-03-08 14:29:46 +09:00
Arpit Agarwal 9d67f98e05 HDFS-9882. Add heartbeatsTotal in Datanode metrics. (Contributed by Hua Liu) 2016-03-07 21:10:34 -08:00
Ming Ma aae39ff973 HADOOP-12789. log classpath of ApplicationClassLoader at INFO level. (Sangjin Lee via mingma) 
(cherry picked from commit 49eedc7ff0)
 2016-03-07 20:27:10 -08:00
Andrew Wang fed2339e28 HADOOP-12901. Add warning log when KMSClientProvider cannot create a connection to the KMS server. (Xiao Chen via wang) 
(cherry picked from commit 391da36d93)
 2016-03-07 14:00:08 -08:00
Chris Nauroth 592f161e13 HDFS-9239. DataNode Lifeline Protocol: an alternative protocol for reporting DataNode liveness. Contributed by Chris Nauroth. 
(cherry picked from commit 2759689d7d)
 2016-03-04 15:30:28 -08:00
Masatake Iwasaki 722182fdfb HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 
(cherry picked from commit cbd31328a6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
	hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md
 2016-03-04 14:12:44 +09:00
Andrew Wang 670944810c HADOOP-11792. Remove all of the CHANGES.txt files. 2016-03-03 21:02:58 -08:00
Allen Wittenauer b0d3b0c1f8 HADOOP-12651. Replace dev-support with wrappers to Yetus (aw) 
(cherry picked from commit 8cecad2d56)

 Conflicts:
	dev-support/test-patch.sh
	hadoop-common-project/hadoop-common/pom.xml
 2016-03-03 20:54:08 -08:00
Akira Ajisaka 899715a15e HADOOP-12872. Fix formatting in ServiceLevelAuth.md. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0a9f00af5e)
 2016-03-03 23:32:51 +09:00
Akira Ajisaka 96a0939d5b HADOOP-12871. Fix dead link to NativeLibraries.html in CommandsManual.md. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1c3136cbd8)
 2016-03-03 23:23:12 +09:00
Akira Ajisaka 9e87f59dea HADOOP-12870. Fix typo admininistration in CommandsManual.md. Contributed by John Zhuge. 
(cherry picked from commit eb864d35d6)
 2016-03-03 18:51:50 +09:00
Masatake Iwasaki 69e024f8fa HADOOP-12793. Write a new group mapping service guide (Wei-Chiu Chuang via iwasakims) 
(cherry picked from commit f70e317ccc)
 2016-03-03 17:14:53 +09:00
Arpit Agarwal 7b4a1001ea HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou) 2016-03-02 18:35:38 -08:00
Colin Patrick Mccabe 5998b20d63 HDFS-9835. OIV: add ReverseXML processor which reconstructs an fsimage from an XML file (cmccabe) 
(cherry picked from commit 700b0e4019)
 2016-03-02 18:18:45 -08:00
Andrew Wang 3fecfe1352 HADOOP-12859. Disable hiding field style checks in class setters. Contributed by Kai Zheng. 
(cherry picked from commit 480302b4ba)
 2016-03-02 09:40:33 -08:00
Chris Nauroth bce33b5179 HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty. 
(cherry picked from commit f98dff329b)
 2016-03-01 13:18:57 -08:00
Akira Ajisaka 67e8489ac6 HADOOP-12843. Fix findbugs warnings in hadoop-common (branch-2). (aajisaka) 2016-03-02 00:20:57 +09:00
Chris Nauroth e402371b6a HADOOP-12851. S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders. Contributed by Larry McCay. 
(cherry picked from commit d251e55415)
 2016-02-29 22:00:27 -08:00
Karthik Kambatla c1855a33eb HADOOP-10321. TestCompositeService should cover all enumerations of adding a service to a parent service. (Ray Chiang via kasha) 
(cherry picked from commit 680f3fc02d)
 2016-02-29 19:00:22 -08:00
Jian He 3e842de1d5 HADOOP-12622. Improve the loggings in RetryPolicies and RetryInvocationHandler. Contributed by Junping Du 2016-02-29 16:24:30 -08:00
Haohui Mai c5db4ab0b4 HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-02-29 14:10:18 -08:00
Masatake Iwasaki 32c7791a6c HADOOP-12552. Fix undeclared/unused dependency to httpclient (iwasakims) 
(cherry picked from commit 809ebc0b14)
 2016-02-29 18:23:33 +09:00
Suresh Srinivas fca24fbbaa HADOOP-9121. InodeTree.java has redundant check for vName while throwing exception. Contributed by Arup Malakar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1418005 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit a5331eeead)
 2016-02-29 11:35:19 +09:00
Chris Nauroth 2e761dabd8 HADOOP-12846. Credential Provider Recursive Dependencies. Contributed by Larry McCay. 
(cherry picked from commit 7634d404b7)
 2016-02-28 11:23:02 -08:00
Li Lu 090f74c741 HADOOP-12831. LocalFS/FSOutputSummer NPEs in constructor if bytes per checksum set to 0 (Mingliang Liu via gtcarrera9) 
(cherry-picked from commit 7545ce6636)
 2016-02-27 22:03:49 -08:00
Steve Loughran 4d870b1faa HADOOP-12825. Log slow name resolutions. (Sidharta Seethana via stevel) 2016-02-27 20:05:53 +00:00
Chris Nauroth c692d7c52f HADOOP-12849. TestSymlinkLocalFSFileSystem fails intermittently. Contributed by Mingliang Liu. 
(cherry picked from commit 798babf661)
 2016-02-26 21:59:14 -08:00
Lei Xu 49a10a3cec HADOOP-12841. Update s3-related properties in core-default.xml. (Wei-Chiu Chuang via lei) 
(cherry picked from commit 2093acf6b6)
 2016-02-26 10:48:35 -08:00
Suresh Srinivas b1d497a7de HADOOP-8818. Use equals instead == in MD5MD5CRC32FileChecksum and TFileDumper. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1385374 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 0c53ed4cd1)
 2016-02-27 01:27:00 +09:00
Masatake Iwasaki b0091338f2 HADOOP-12711. Remove dependency on commons-httpclient for ServletUtil (Wei-Chiu Chuang via iwasakims) 2016-02-26 15:32:07 +09:00
Karthik Kambatla 6722e17f6d HDFS-9858. RollingFileSystemSink can throw an NPE on non-secure clusters. (Daniel Templeton via kasha) 
(cherry picked from commit c2460dad64)
 2016-02-25 16:36:07 -08:00
Xiaoyu Yao 7ad6ca1eb2 HADOOP-12824. Collect network and disk usage on the node running Windows. Contributed by Inigo Goiri. 2016-02-25 15:50:36 -08:00
cnauroth 92e49cdd04 HADOOP-12716. KerberosAuthenticator#doSpnegoSequence use incorrect class to determine isKeyTab in JDK8. Contributed by Xiaoyu Yao. 
(cherry picked from commit d6b181c6fa)
 2016-02-24 14:05:55 -08:00
cnauroth 9ea19a861f HADOOP-12535. Run FileSystem contract tests with hadoop-azure. Contributed by Madhumita Chakraborty. 
(cherry picked from commit 3369a4f691)
 2016-02-23 21:37:56 -08:00
Colin Patrick Mccabe 7dfcf40352 HADOOP-12829. StatisticsDataReferenceCleaner swallows interrupt exceptions (Gregory Chanan via cmccabe) 
(cherry picked from commit d9c409a428)
 2016-02-23 11:27:55 -08:00
cnauroth a5511debdb HADOOP-12555. WASB to read credentials from a credential provider. Contributed by Larry McCay. 
(cherry picked from commit 27b77751c1)
 2016-02-22 17:30:32 -08:00
Zhe Zhang 56b7f2c2c1 HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh. 
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
 2016-02-22 14:14:39 -08:00
cnauroth 3f14eed580 HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. Contributed by Masatake Iwasaki. 
(cherry picked from commit d4f5fc23b2)
 2016-02-22 13:39:45 -08:00
Junping Du e38c2ef6c5 Support additional compression levels for GzipCodec. Contributed by Ravi Mutyala. 
(cherry picked from commit 18f9b77a32)
 2016-02-19 14:22:39 -08:00
Yongjun Zhang 36c08f793a HADOOP-12828. Print user when services are started. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit a963baba10)
 2016-02-19 09:57:45 -08:00
cnauroth b4c4f30fa4 HADOOP-11031. Design Document for Credential Provider API. Contributed by Larry McCay. 
(cherry picked from commit 8ab7658025)
 2016-02-18 14:07:09 -08:00
Tsz-Wo Nicholas Sze 38fc086940 HADOOP-10865. Add a Crc32 chunked verification benchmark for both directly and non-directly buffer cases. 2016-02-18 12:02:00 -08:00
Robert Kanter f864faaafa HADOOP-12817. Enable TLS v1.1 and 1.2 (rkanter) 
(cherry picked from commit a365a3941c)
 2016-02-18 11:11:32 -08:00
cnauroth 779a69e28d HDFS-9711. Integrate CSRF prevention filter in WebHDFS. Contributed by Chris Nauroth. 
(cherry picked from commit 5d1889a66d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
 2016-02-18 10:09:33 -08:00
Xiaoyu Yao ad4fcd1f9f HDFS-8855. Webhdfs client leaks active NameNode connections. Contributed by Xiaobing Zhou. 
(cherry picked from commit fe5624b85d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
 2016-02-17 16:05:07 -08:00
Zhe Zhang e226dbbc8a HDFS-9799. Reimplement getCurrentTrashDir to remove incompatibility. (zhz) 
Change-Id: I7834bcebffed38cb384db5395ddb8b6dd9e79a0b
 2016-02-17 13:31:48 -08:00
cnauroth 6731bb468d HADOOP-12548. Read s3a creds from a Credential Provider. Contributed by Larry McCay. 
(cherry picked from commit 76fab26c5c)

Conflicts:
	hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
 2016-02-17 12:20:41 -08:00
Xiaoyu Yao 91c4a3a816 HADOOP-12878. KMS SPNEGO sequence does not work with WEBHDFS. Contributed by Xiaoyu Yao. 
(cherry picked from commit fd1befb6ba)
 2016-02-17 08:44:37 -08:00
Masatake Iwasaki 3e80aece2e HADOOP-12710. Remove dependency on commons-httpclient for TestHttpServerLogs (Wei-Chiu Chuang via iwasakims) 
(cherry picked from commit 1a1c2f5bab)
 2016-02-18 00:25:20 +09:00
Vinayakumar B 2bf5a005fa HADOOP-12810. FileSystem#listLocatedStatus causes unnecessary RPC calls (Contributed by Ryan Blue) 
(cherry picked from commit 96ea309431)
 2016-02-17 10:15:08 +05:30
Vinayakumar B e79a5e4460 HADOOP-12805. Annotate CanUnbuffer with @InterfaceAudience.Public (Contributed by Ted Yu) 
(cherry picked from commit 4b0e59fdc0)
 2016-02-16 17:32:35 +05:30
Akira Ajisaka a670165bbd HADOOP-12786. "hadoop key" command usage is not documented. Contributed by Xiao Chen. 
(cherry picked from commit 1c48e50ce7)
 2016-02-15 11:38:54 +09:00
Junping Du a28be9edc7 Set the release date for 2.6.4. 
(cherry picked from commit ec12ce8f48)
 2016-02-13 01:44:12 -08:00
Masatake Iwasaki 7a67f8f709 HADOOP-12589. Fix intermittent test failure of TestCopyPreserveFlag (iwasakims) 
(cherry picked from commit 36ba5cac37)
 2016-02-13 15:01:08 +09:00
cnauroth ac5a793590 HADOOP-12780. During WASB atomic rename handle crash when one directory has been renamed but not file under it. Contributed by Madhumita Chakraborty. 
(cherry picked from commit 91a96eaa53)
 2016-02-12 15:50:19 -08:00
Arpit Agarwal 63f51208a8 HDFS-9801. ReconfigurableBase should update the cached configuration. (Arpit Agarwal) 2016-02-12 12:41:12 -08:00
Vinayakumar B 488e6cbe16 HADOOP-12776. Remove getaclstatus call for non-acl commands in getfacl. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit c78740a979)
 2016-02-12 11:55:16 +05:30
Karthik Kambatla 383f3ff28c HDFS-9780. RollingFileSystemSink doesn't work on secure clusters. (Daniel Templeton via kasha) 
(cherry picked from commit 33ce7f6c07)
 2016-02-11 20:45:23 -08:00
Andrew Wang fb57c01eaa HADOOP-12699. TestKMS#testKMSProvider intermittently fails during 'test rollover draining'. Contributed by Xiao Chen. 
(cherry picked from commit 8fdef0bd9d)
 2016-02-11 17:21:50 -08:00
cnauroth 25dcb88630 HADOOP-12795. KMS does not log detailed stack trace for unexpected errors. Contributed by Chris Nauroth. 
(cherry picked from commit 70c756d35e)
 2016-02-11 16:53:12 -08:00
Zhe Zhang dfd49a027e Add HADOOP-12764 to CHANGES.txt. 
Change-Id: Ie32b71b7bd8892a3bb434aec12cdf2c43c61b897
 2016-02-11 09:34:32 -08:00
Zhe Zhang 84dcd1a802 HADOOP-12764. Increase default value of KMS maxHttpHeaderSize and make it configurable. (zhz) 
Change-Id: Iabb766f6311b83a1ac03093b0ba26a0284b74d70
 2016-02-11 09:29:14 -08:00
Uma Maheswara Rao G e90f7ef988 HADOOP-12788. OpensslAesCtrCryptoCodec should log which random number generator is used. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit df4dcd3e75)
 2016-02-10 15:04:42 -08:00
Jason Lowe fa9b3e6302 HADOOP-12792. TestUserGroupInformation#testGetServerSideGroups fails in chroot. Contributed by Eric Badger 
(cherry picked from commit 6f03959e4d)
 2016-02-10 21:54:50 +00:00
Sangjin Lee 463195dbe6 HADOOP-12773. HBase classes fail to load with client/job classloader enabled (sjlee) 
(cherry picked from commit 58acbf940a)
 2016-02-08 13:57:44 -08:00
cnauroth a76611d4ec HADOOP-12752. Improve diagnostics/use of envvar/sysprop credential propagation. Contributed by Steve Loughran. 
(cherry picked from commit cf3261570a)
 2016-02-08 09:36:18 -08:00
Varun Vasudev 2451a0a8e0 HADOOP-12749. Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors. Contributed by Sidharta Seethana. 
(cherry picked from commit d37eb828ff)
 2016-02-08 20:19:51 +05:30
Varun Vasudev 368a11f3dd Revert "HADOOP-12749. Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors. Contributed by Sidharta Seethana." 
This reverts commit f3bbe0bd02.

(cherry picked from commit af218101e5)
 2016-02-08 20:17:10 +05:30
Varun Vasudev fd971debee HADOOP-12749. Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors. Contributed by Sidharta Seethana. 
(cherry picked from commit f3bbe0bd02)
 2016-02-08 14:23:19 +05:30
Andrew Wang 915ce5d960 HADOOP-12759. RollingFileSystemSink should eagerly rotate directories. Contributed by Daniel Templeton. 
(cherry picked from commit 5b59a0ea85)
 2016-02-06 20:52:49 -08:00
Steve Loughran 60474a769e HADOOP-12292. Make use of DeleteObjects optional. (Thomas Demoor via stevel) 2016-02-06 18:20:24 +00:00
cnauroth 4db6cc44cd HADOOP-12426. Add Entry point for Kerberos health check. Contributed by Steve Loughran. 
(cherry picked from commit cfa8513890)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
 2016-02-05 15:52:14 -08:00
cnauroth 5752df2362 HADOOP-12758. Extend CSRF Filter with UserAgent Checks. Contributed by Larry McCay. 
(cherry picked from commit a37e423e84)
 2016-02-05 14:38:31 -08:00
Kihwal Lee 51a705432c HADOOP-12772. NetworkTopologyWithNodeGroup.getNodeGroup() can loop infinitely for invalid 'loc' values. Contributed by Kuhu Shukla. 
(cherry picked from commit 49e176c29f)
 2016-02-05 15:47:56 -06:00
Colin Patrick Mccabe 86760c0f52 HADOOP-12766. The default value of "hadoop.workaround.non.threadsafe.getpwuid" is different between core-default.xml and NativeIO.java (Akira AJISAKA via cmccabe) 
(cherry picked from commit 9086dd58c3)
 2016-02-05 10:57:47 -08:00
Xiaoyu Yao e0a356fee3 HADOOP-12771. Fix typo in JvmPauseMonitor#getNumGcWarnThreadholdExceeded. Contributed by Xiaobing Zhou. 
(cherry picked from commit 5d00067ca7)
 2016-02-05 10:38:02 -08:00
Xuan dfad608b22 YARN-4662. Document some newly added metrics. Contributed by Jian He 
(cherry picked from commit 63c63e298c)
 2016-02-03 20:06:25 -08:00
Sangjin Lee b0ab6df079 HADOOP-12761. incremental maven build is not really incremental (sjlee) 
(cherry picked from commit 4dc0a39493)
 2016-02-03 09:00:41 -08:00
Junping Du 3583c77ee8 Adding release 2.6.5 to CHANGES.txt 2016-02-02 18:09:17 -08:00
Andrew Wang b4ad58796b HADOOP-12755. Fix typo in defaultFS warning message. 
(cherry picked from commit 8171874dd1)
 2016-02-02 11:56:47 -08:00
Karthik Kambatla 363a8212ec HADOOP-12702. Add an HDFS metrics sink. (Daniel Templeton via kasha) 
(cherry picked from commit ee005e010c)
 2016-01-28 17:45:50 -08:00