Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,347 Commits 326 Branches 371 Tags
Commit Graph

12347 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kihwal Lee
c9bf21b0f3 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 2017-05-01 17:34:19 -05:00
Kihwal Lee
d1e6b6db7c HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:28:27 -05:00
Steve Loughran
993ab15b39 YARN-524 TestYarnVersionInfo failing if generated properties doesn't include an SVN URL 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463300 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 7c65dbb9385567fe1d8e280b81ecb97b109d18a8)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 5ca4f0f82e41c2833fd8520bc1f764a3f958bdf6)
 2017-05-01 08:58:53 -05:00
Eric Payne
5b9460301a YARN-6536. TestAMRMClient.testAMRMClientWithSaslEncryption fails intermittently. Contributed by Jason Lowe. 
(cherry picked from commit fdf5192bbbb3c81e5fb221758297605271139dc9)
 2017-04-28 15:23:05 -05:00
Eric Payne
84106e80a8 HADOOP-14320. TestIPC.testIpcWithReaderQueuing fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit 6fe11f61556407f8efb9b376623cd61609a40d5f)
 2017-04-28 14:55:14 -05:00
Robert Kanter
26492de185 YARN-5894. fixed license warning caused by de.ruedigermoeller:fst:jar:2.24 (haibochen via rkanter) 
(cherry picked from commit 371b6467dcee8517ef0d49b9f391302a6d9ed648)
 2017-04-28 09:21:41 -07:00
Konstantin V Shvachko
cb12ada03f HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 28eb2aabebd15c15a357d86e23ca407d3c85211c)
 2017-04-27 11:42:48 -07:00
Zhe Zhang
70f18911c7 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a33098309f17dfb0e3a000934f68394de44bf7)
(cherry picked from commit fc46f398d9192e83c7197115589eca7805c49943)
 2017-04-27 09:54:10 -07:00
Steve Loughran
40b9c30d64
HADOOP-14346. CryptoOutputStream throws IOException if stream is closed. 
Contributed by Pierre Lacave
 2017-04-27 12:17:20 +01:00
Eric Payne
73563bc9b6 YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe. 2017-04-26 14:33:27 -05:00
John Zhuge
19e94bfd1b HADOOP-14341. Support multi-line value for ssl.server.exclude.cipher.list. Contributed by John Zhuge. 
(cherry picked from commit 9ccb849eb69f05bccc3435306ec4bde104e411bf)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/SSLFactory.java
 2017-04-25 22:41:27 -07:00
Mingliang Liu
c3c9a91900 HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu 
(cherry picked from commit e4c32bccde3147f51263cb58cdb480453b1cb3e9)
(cherry picked from commit e7a08cc6dc1f729d04dea9239f0a8c90c8be5598)
 2017-04-25 20:55:42 -07:00
Jason Lowe
d85dbd11ae HDFS-11691. Add a proper scheme to the datanode links in NN web UI. Contributed by Kihwal Lee 
(cherry picked from commit e4321ec84321672a714419278946fe1012daac71)
 2017-04-25 17:32:07 -05:00
Yongjun Zhang
c88e5e1836 HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang. 
(cherry picked from commit 5078df7be317e635615c05c5da3285798993ff1f)
 2017-04-21 14:12:04 -07:00
John Zhuge
28f0de875d HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 
(cherry picked from commit 6865746ea444c57569e9b2464d8240dafcdbaaa5)
 2017-04-19 22:24:33 -07:00
John Zhuge
e813110734 Revert "HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge." 
This reverts commit 935ea448c1364befb58dd1ccedd4d7f736c8002a.
 2017-04-19 13:34:43 -07:00
John Zhuge
935ea448c1 HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 
(cherry picked from commit 0344bea3fd4031622edd828a610c9fdc23c53d26)
 2017-04-19 12:09:09 -07:00
Yiqun Lin
daa532b902 HDFS-11316. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 724bb8635144dd16fc18cf95a78db6c3d5d1c468)
 2017-04-19 12:31:14 -05:00
Steve Loughran
952113ebfd
HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs. 
Contributed by Steve Loughran

(cherry picked from commit 1ae6be46b4a02870f9471b05f8d39101b8885830)
 2017-04-19 15:37:27 +01:00
Kengo Seki
525b258944
HADOOP-14315. Python example in the rack awareness document doesn't work due to bad indentation 
This closes #214

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 654372db859656a2201ae9f9f7c374c6564ea34d)
(cherry picked from commit 6cfceee86547211c71f8cb6f9c63798a229de03e)
 2017-04-18 17:16:03 +09:00
Akira Ajisaka
c8bcc96500
HADOOP-14318. Remove non-existent setfattr command option from FileSystemShell.md. Contributed by Doris Gu. 
(cherry picked from commit 84a8848aaecd56b4cc85185924c2e33674165e01)
(cherry picked from commit a17bd7089263511b46918948843d05bb18750b1f)
 2017-04-18 16:46:01 +09:00
Konstantin V Shvachko
8b3b3aac10 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 8dfcd95d580bb090af7f40af0a57061518c18c8c)
 2017-04-17 17:09:38 -07:00
Zhe Zhang
07047335a3 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 
(cherry picked from commit ad49098eb324e238d97db68d7239ed2c4d84afa0)
(cherry picked from commit 051ab778065d3659c5284b5d95d04156c4ee7ffa)
 2017-04-17 16:36:21 -07:00
Junping Du
0de6f28020 YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6ed9d362242d36bfae4083bf3b1c2eb32fbb2f72)
(cherry picked from commit 9289f4527d8fc3418fdeb8bf213b0f62525092d5)
 2017-04-17 15:59:00 -07:00
Arun Suresh
5970e8297d HADOOP-14311. Add python2.7-dev to Dockerfile (Allen Wittenauer via asuresh) 
(cherry picked from commit 0ac17dc644c0429ff8a6f743bf9d3ecdd7458e58)
(cherry picked from commit 8515d35bd5424d0cf27e44d3618c3ea5ae7ea969)
 2017-04-14 16:59:25 -07:00
Jason Lowe
a61ec7c374 YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger 
(cherry picked from commit 416880550214e58f2284a045ad0c96ba4aa78ea8)
 2017-04-14 17:18:08 -05:00
Eric Payne
d27f74d915 YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger 
(cherry picked from commit 157a9f310429f6be71d2d3e2bce42d0b36e9a1be)
 2017-04-14 11:54:30 -05:00
Wei-Chiu Chuang
ecaedca5c4
HDFS-10506. OIV's ReverseXML processor cannot reconstruct some snapshot details. Contributed by Akira Ajisaka. 
(cherry picked from commit 14414705f79495eda11e302f38c792128fe0182b)
 2017-04-14 21:11:31 +09:00
Junping Du
c43ee00587 YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen. 
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
(cherry picked from commit 9d9087a67ad599e678948d5c1036e51cb8cf5e3a)
(cherry picked from commit d77cada182ef805679dc79498d6b6cc6e7f6b6d4)
 2017-04-12 14:17:04 -07:00
Mingliang Liu
9eebbcf459 HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou 2017-04-12 13:39:20 -07:00
Kihwal Lee
9dfe0b3515 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 
(cherry picked from commit 8ed230c805625549b1cecc830e909a7027bb4961)
 2017-04-12 13:30:41 -05:00
Chris Nauroth
c4bf504395 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
(cherry picked from commit d5e2bd4096bf2b4d8a5a22042145a08905f93cd4)
 2017-04-11 21:55:28 -07:00
Steve Loughran
32475a7b6c HADOOP-14066 VersionInfo should be marked as public API 
(cherry picked from commit 2ed65540a3fe73db5d2dd0d25e658ccf13c9188a)
 2017-04-11 15:06:45 -07:00
Junping Du
dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Jason Lowe
c5b1a8910d YARN-6461. TestRMAdminCLI has very low test timeouts. Contributed by Eric Badger 
(cherry picked from commit 90d97372ed11aaa95d338db8583f578da159834a)
 2017-04-11 09:18:56 -05:00
Andrew Wang
c8d494197f HADOOP-14293. Initialize FakeTimer with a less trivial value. 
(cherry picked from commit be144117a885cb39bc192279c96cbe3790dc77b1)
(cherry picked from commit dab1deb9a0c96471b6da4ef95be86cbc81d55305)
 2017-04-10 11:37:35 -07:00
John Zhuge
fa8e411d47 HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge. 
Signed-off-by: John Zhuge <jzhuge@apache.org>
(cherry picked from commit 56e81f2a204d5e65d29222df1b2ba4d892f9c0d5)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2017-04-07 15:05:09 -07:00
Zhe Zhang
ef8b30dd4b HADOOP-14276. Add a nanosecond API to Time/Timer/FakeTimer. Contributed by Erik Krogen. 
(cherry picked from commit 95b7f1d29a5e2dadd70a56fca5faa006c5bd74fc)
(cherry picked from commit c85026038a6ef8d38b7a8563fbb64ac6edd1d4ce)
 2017-04-07 10:52:02 -07:00
Xiaoyu Yao
6367bc1f09 HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 
(cherry picked from commit 0eacd4c13be9bad0fbed9421a6539c64bbda4df1)
(cherry picked from commit 0391c920221f4fbfede44fdfd06e678cd1230f1c)
 2017-04-06 16:26:57 -07:00
Andrew Wang
b1dfdea5b4 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:39:56 -07:00
Jason Lowe
6e41aec109 YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 2017-04-06 16:26:24 -05:00
Andrew Wang
ab91ca7338 HADOOP-13688. Stop bundling HTML source code in javadoc JARs. 
(cherry picked from commit 72a2ae6452e615c66d10829da38737896814e02b)
(cherry picked from commit 832c2f52f8543bb966596d0626af99fe995b6aaf)
 2017-04-06 13:16:15 -07:00
Junping Du
f7a35c6489 HADOOP-13996. Fix some release build issues, include gpg timeout and disable sign for hadoop-dist. Contributed by Andrew Wang. 
(cherry picked from commit 98759b426c9aef2304c937fddba02c4d21177dbb)
 2017-04-06 13:06:50 -07:00
Andrew Wang
d12a0a25b4 HDFS-11538. Move ClientProtocol HA proxies into hadoop-hdfs-client. Contributed by Huafeng Wang. 
(cherry picked from commit e154893d3b04d84579a45f0906fa674bda8b4952)
 2017-04-05 16:36:07 -07:00
Wei-Chiu Chuang
eff4b2fa07 HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge. 
(cherry picked from commit c5ca2169151a5eec57152775789b6f53664e102c)
(cherry picked from commit 434403a2a0863e4d572e04e05e85d4f333bd6339)
 2017-04-05 16:35:49 -07:00
Xiao Chen
7ad5b278a7 HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge 
(cherry picked from commit d9aae22fdf2ab22ae8ce4a9d32ac71b3dde084d3)
(cherry picked from commit 24d464a15015a7bf6d02d259568fedab1cb97b84)
 2017-04-05 16:35:44 -07:00
Jing Zhao
e216c15625 HDFS-11395. RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode. Contributed by Nandakumar. 
(cherry picked from commit 55796a0946f80a35055701a34379e374399009c5)
(cherry picked from commit e03d8ff488c5818fbe167fa0143c252d7b6287f0)
 2017-04-05 16:29:04 -07:00
Andrew Wang
ab673aa6cc HDFS-11629. Revert "HDFS-11431. hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider." 
This reverts commit 72fbdf708ee5f6b1004a5d45dd1cf2cb50fedb24.
 2017-04-05 16:26:57 -07:00
Xiaobing Zhou
b9b6bd2932 HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou. 2017-04-05 15:34:27 -07:00
Varun Saxena
bf2c63ce46 YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena) 
(cherry picked from commit 7d963c477ab997e0daad760dd3c568d7010fe511)
 2017-04-06 02:09:32 +05:30