Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,053 Commits 326 Branches 371 Tags
Commit Graph

13053 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mingliang Liu
18a5925256 HADOOP-14363. Inconsistent default block location in FileSystem javadoc. Contributed by Chen Liang 
(cherry picked from commit c735362df719227adf9a648b2d425288cd4e4c2e)
 2017-04-29 23:50:04 -07:00
Daniel Templeton
a8512d5aeb YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton) 
(cherry picked from commit 2e52789edf68016e7a3f450164f8bd3d8e6cb210)
 2017-04-28 13:39:11 -07:00
Eric Payne
92e5650f98 YARN-6536. TestAMRMClient.testAMRMClientWithSaslEncryption fails intermittently. Contributed by Jason Lowe. 
(cherry picked from commit fdf5192bbbb3c81e5fb221758297605271139dc9)
 2017-04-28 15:21:58 -05:00
Eric Payne
2bf29ca2f9 HADOOP-14320. TestIPC.testIpcWithReaderQueuing fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit 6fe11f61556407f8efb9b376623cd61609a40d5f)
 2017-04-28 14:49:02 -05:00
Junping Du
5d81b6b29a YARN-679. Add an entry point that can start any Yarn service. Contributed by Steve Loughran. 
(cherry picked from commit 373bb4931fb392e3ca6bfd78992887e5a405e186)
 2017-04-28 11:10:00 -07:00
Robert Kanter
816d98e7cd YARN-5894. fixed license warning caused by de.ruedigermoeller:fst:jar:2.24 (haibochen via rkanter) 
(cherry picked from commit 371b6467dcee8517ef0d49b9f391302a6d9ed648)
 2017-04-28 09:15:40 -07:00
Zhe Zhang
fc46f398d9 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a33098309f17dfb0e3a000934f68394de44bf7)
 2017-04-27 09:51:04 -07:00
Steve Loughran
b500714e98
HADOOP-14346. CryptoOutputStream throws IOException if stream is closed. 
Contributed by Pierre Lacave

(cherry picked from commit 40b9c30d64ee1b07ade423533e627a7d4b099261)
 2017-04-27 12:18:30 +01:00
Konstantin V Shvachko
4cbf5c5c41 HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 28eb2aabebd15c15a357d86e23ca407d3c85211c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
 2017-04-26 19:02:48 -07:00
Mingliang Liu
e84588eb03 HADOOP-14351. Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache. Contributed by Santhosh G Nayak 
(cherry picked from commit 8b5f2c372e70999f3ee0a0bd685a494e06bc3652)
 2017-04-26 13:51:58 -07:00
Haibo Chen
894521673b YARN-6510. Fix profs stat file warning caused by process names that includes parenthesis. (Wilfred Spiegelenburg via Haibo Chen) 
(cherry picked from commit 4f3ca0396a810f54f7fd0489a224c1bb13143aa4)
 2017-04-26 11:47:23 -07:00
Akira Ajisaka
cc66153e12
HDFS-11417. Add datanode admin command to get the storage info. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 93fa48fcf243dc759db1736af145633da760f937)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
 2017-04-26 18:08:12 +09:00
John Zhuge
1e60cfed7e HADOOP-14341. Support multi-line value for ssl.server.exclude.cipher.list. Contributed by John Zhuge. 
(cherry picked from commit 9ccb849eb69f05bccc3435306ec4bde104e411bf)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/SSLFactory.java
 2017-04-25 22:36:21 -07:00
Mingliang Liu
5ceb61cb43 HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu 
(cherry picked from commit e4c32bccde3147f51263cb58cdb480453b1cb3e9)
 2017-04-25 20:55:56 -07:00
Xiaoyu Yao
fc0e6de929 HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore. 
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
 2017-04-25 18:49:44 -07:00
Mingliang Liu
6fd072f924 HADOOP-12856. FileUtil.checkDest() and RawLocalFileSystem.mkdirs() to throw stricter IOEs; RawLocalFS contract tests to verify. Contributed by Steve Loughran 
(cherry picked from commit d11d280b8b527d572ca0b29ef57434e9ff533ffe)
 2017-04-25 18:34:26 -07:00
Jason Lowe
31af85c18e HDFS-11691. Add a proper scheme to the datanode links in NN web UI. Contributed by Kihwal Lee 
(cherry picked from commit e4321ec84321672a714419278946fe1012daac71)
 2017-04-25 17:31:31 -05:00
Steve Loughran
d1da5ba3af
HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus. 
Contributed by Steve Moist.
 2017-04-25 17:42:30 +01:00
Wangda Tan
503dd650bf YARN-6392. Add submit time to Application Summary log. (Zhihai Xu via wangda) 
(cherry picked from commit 2ba21d63767e11535d3210dc58a03b41e83df949)
 2017-04-24 13:18:47 -07:00
Haibo Chen
49a2f286f7 YARN-6500. Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler. (Miklos Szegedi via Haibo Chen) 
(cherry picked from commit 8ac50e1322cb3f84bd998635924d85846aa47c94)
 2017-04-24 11:39:55 -07:00
Eric Payne
421e51cb9c YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe. 
(cherry picked from commit 52adf719143c20f4f2af369c6c40dd98677e7410)
 2017-04-24 12:08:35 -05:00
Sunil G
a9495b206b YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu. 2017-04-24 12:16:18 +05:30
Robert Kanter
d686560b35 MAPREDUCE-6871. Allow users to specify racks and nodes for strict locality for AMs (rkanter) 
(cherry picked from commit 3721cfe1fbd98c5b6aa46aefdfcf62276c28c4a4)
 2017-04-21 16:12:45 -07:00
Yongjun Zhang
31d7d47070 HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang. 
(cherry picked from commit 5078df7be317e635615c05c5da3285798993ff1f)
 2017-04-21 14:01:47 -07:00
Mingliang Liu
3b7bb7b94b HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran 2017-04-20 17:07:57 -07:00
Daniel Templeton
d77137a03f YARN-6453. fairscheduler-statedump.log gets generated regardless of service 
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit e74ca7216aecc998d277e5ca1f4d0abae43c7e31)
 2017-04-20 14:14:38 -07:00
John Zhuge
1bbba00a20 HADOOP-14141. Store KMS SSL keystore password in catalina.properties. Contributed by John Zhuge. 
Change-Id: I8326b16aab5c3908529e1a0cc7bfe121d18f42f3
 2017-04-20 12:36:40 -07:00
Jason Lowe
f5a9287bb1 YARN-6501. FSSchedulerNode.java fails to compile with JDK7. Contributed by John Zhuge 2017-04-20 09:52:44 -05:00
John Zhuge
6865746ea4 HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 2017-04-19 22:22:36 -07:00
John Zhuge
5046a45507 HDFS-11685. TestDistributedFileSystem.java fails to compile. Contributed by John Zhuge. 2017-04-19 22:10:18 -07:00
Andrew Wang
efb790b0d6 HDFS-11660. TestFsDatasetCache#testPageRounder fails intermittently with AssertionError. 
(cherry picked from commit 74a723852d9ef265d0fd46334f2f885bfb247a3a)
 2017-04-19 18:10:16 -07:00
John Zhuge
ef74a5661b Revert "HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge." 
This reverts commit 3a39dd5391527623a70296410f4754d8d235310a.
 2017-04-19 13:33:38 -07:00
John Zhuge
3a39dd5391 HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 
(cherry picked from commit 0344bea3fd4031622edd828a610c9fdc23c53d26)
 2017-04-19 12:04:03 -07:00
Yiqun Lin
e278eb3ab5 HDFS-11316. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 724bb8635144dd16fc18cf95a78db6c3d5d1c468)
 2017-04-19 12:15:52 -05:00
Daniel Templeton
5225ae50b7 HADOOP-13997. Typo in metrics docs (Contributed by Ana Krasteva via Daniel Templeton) 
(cherry picked from commit d4483a2fe47ccc589e71c6fe5501878e6142c26f)
 2017-04-19 09:42:30 -07:00
Steve Loughran
1ae6be46b4
HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs. 
Contributed by Steve Loughran
 2017-04-19 10:19:43 +01:00
Arpit Agarwal
5c092097b6 HDFS-11603. Improve slow mirror/disk warnings in BlockReceiver. 2017-04-18 16:20:43 -07:00
Arpit Agarwal
6534bc0a01 HDFS-11551. Handle SlowDiskReport from DataNode at the NameNode. Contributed by Hanisha Koneru. 
This is an empty commit with the correct commit message for HDFS-11511.
The actual fix was made via commit ID f16bb799573bea6da2535ee863ed15ca271ef504.
 2017-04-18 15:35:17 -07:00
Arpit Agarwal
fc358b89b8 HDFS-11511. Support Timeout when checking single disk. Contributed by Hanisha Koneru. 
This is an empty commit with the correct commit message for HDFS-11511.
The fix was previously made via commit ID fbe86471cf6d6397a89e473eb328d86a66c513e2.
 2017-04-18 15:34:46 -07:00
Kengo Seki
6cfceee865
HADOOP-14315. Python example in the rack awareness document doesn't work due to bad indentation 
This closes #214

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 654372db859656a2201ae9f9f7c374c6564ea34d)
 2017-04-18 17:15:35 +09:00
Akira Ajisaka
a17bd70892
HADOOP-14318. Remove non-existent setfattr command option from FileSystemShell.md. Contributed by Doris Gu. 
(cherry picked from commit 84a8848aaecd56b4cc85185924c2e33674165e01)
 2017-04-18 16:45:14 +09:00
Konstantin V Shvachko
c3840bde54 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 8dfcd95d580bb090af7f40af0a57061518c18c8c)
 2017-04-17 17:06:18 -07:00
Zhe Zhang
051ab77806 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 
(cherry picked from commit ad49098eb324e238d97db68d7239ed2c4d84afa0)
 2017-04-17 16:28:58 -07:00
Junping Du
9289f4527d YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6ed9d362242d36bfae4083bf3b1c2eb32fbb2f72)
 2017-04-17 15:52:02 -07:00
Karthik Kambatla
6f5912c106 YARN-6432. FairScheduler: Reserve preempted resources for corresponding applications. (Miklos Szegedi via kasha) 2017-04-17 14:34:28 -07:00
Wangda Tan
6ade9e6ce6 YARN-6406. Remove SchedulerRequestKeys when no more pending ResourceRequest. (Arun Suresh via wangda) 
(cherry picked from commit 87e2ef8c985bb72a916477e8783359f2859f7890)
 2017-04-15 21:34:29 -07:00
Haibo Chen
d4f553d42f MAPREDUCE-6673. Add a test example job that grows in memory usage over time (Karthik Kambatla via Haibo Chen) 
Change-Id: Iccfc8c67c38c526cc61726d87bfcbcf69ac36fea
(cherry picked from commit 25ac44709b4bbed78b607ea48021237b64e01b9f)
 2017-04-14 17:37:37 -07:00
Arun Suresh
8515d35bd5 HADOOP-14311. Add python2.7-dev to Dockerfile (Allen Wittenauer via asuresh) 
(cherry picked from commit 0ac17dc644c0429ff8a6f743bf9d3ecdd7458e58)
 2017-04-14 16:58:00 -07:00
Jason Lowe
1699b20526 YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger 
(cherry picked from commit 416880550214e58f2284a045ad0c96ba4aa78ea8)
 2017-04-14 17:17:46 -05:00
Karthik Kambatla
53da65ef8b YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha) 
(cherry picked from commit 8a1d7480f73906d8e0342690ec6c6b008d6de21b)
 2017-04-14 15:07:54 -07:00