Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,873 Commits 326 Branches 371 Tags
Commit Graph

8873 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Colin Patrick Mccabe
b26ba22a90 HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit ef591b1d6a08f08358b19763a874de6010227307)
 2015-04-03 16:56:25 -07:00
Colin Patrick Mccabe
0c5069c432 HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 023133cef9a7ca05364cefbcead57c921589eda7)
 2015-04-03 14:26:45 -07:00
Colin Patrick Mccabe
386b90a700 HADOOP-11785. Reduce the number of listStatus operation in distcp buildListing (Zoran Dimitrijevic via Colin P. McCabe) 
(cherry picked from commit 932730df7d62077f7356464ad27f69469965d77a)
 2015-04-03 14:08:53 -07:00
cnauroth
28e0602e7b HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit db80e428919e4e763f56b9344c748e7f8fb46491)
 2015-04-03 10:44:32 -07:00
Gera Shegalov
3757e731e6 MAPREDUCE-4844. Counters / AbstractCounters have constant references not declared final. (Brahma Reddy Battula via gera) 
(cherry picked from commit a0795c1e81685b812288cf2f4df6f8d23da30ab8)
 2015-04-03 10:35:18 -07:00
cnauroth
0dde8c079d HDFS-8034. Fix TestDFSClientRetries#testDFSClientConfigurationLocateFollowingBlockInitialDelay for Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit dc5596c36a261626fafda7f30638a000839d8392)
 2015-04-03 10:10:29 -07:00
Akira Ajisaka
5ea64e6722 HADOOP-11800. Clean up some test methods in TestCodec.java. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 228ae9aaa40750cb796bbdfd69ba5646c28cd4e7)
 2015-04-04 00:33:10 +09:00
Jing Zhao
2c1f33d178 HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao. 
(cherry picked from commit 72f6bd4893dcf10d6dad24753f9be99505a87a1f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-02 21:22:43 -07:00
Wangda Tan
405aead879 YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda) 
(cherry picked from commit bad070fe15a642cc6f3a165612fbd272187e03cb)
 2015-04-02 17:23:44 -07:00
Vinod Kumar Vavilapalli
d8e17c58bc YARN-3365. Enhanced NodeManager to support using the 'tc' tool via container-executor for outbound network traffic control. Contributed by Sidharta Seethana. 
(cherry picked from commit b21c72777ae664b08fd1a93b4f88fa43f2478d94)
 2015-04-02 16:55:00 -07:00
Haohui Mai
ddb5bb8fcf Addendum for HDFS-7893 in branch-2. 2015-04-02 16:17:14 -07:00
cnauroth
aabd8c8e04 HADOOP-9805. Refactor RawLocalFileSystem#rename for improved testability. Contributed by Jean-Pierre Matsumoto. 
(cherry picked from commit 5763b173d34dcf7372520076f00b576f493662cd)
 2015-04-02 16:13:12 -07:00
Haohui Mai
1278310c21 HDFS-7893. Update the POM to create a separate hdfs-client jar. Contributed by Haohui Mai. 2015-04-02 15:39:40 -07:00
Sandy Ryza
6286cfd711 YARN-3415. Non-AM containers can be counted towards amResourceUsage of a fairscheduler queue (Zhihai Xu via Sandy Ryza) 2015-04-02 13:57:53 -07:00
Junping Du
cacadea632 MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer. Contributed by Jason Lowe. 
(cherry picked from commit eccb7d46efbf07abcc6a01bd5e7d682f6815b824)
 2015-04-02 12:15:12 -07:00
Haohui Mai
6e36dbf03a HDFS-8035. Move checkBlocksProperlyReplicated() in FSNamesystem to BlockManager. Contributed by Haohui Mai. 2015-04-02 11:38:02 -07:00
Tsz-Wo Nicholas Sze
81ed3a2bb8 HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao 2015-04-02 11:12:27 -07:00
Tsz-Wo Nicholas Sze
ce3f32590d HDFS-7888. Change DFSOutputStream and DataStreamer for convenience of subclassing. Contributed by Li Bo 2015-04-02 11:00:47 -07:00
Vinayakumar B
beb0fd0d60 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
(cherry picked from commit 867d5d2675b8fb73c40fac1e581b02b005459d95)
 2015-04-02 08:12:51 +05:30
Brandon Li
27970b3f46 HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot 
(cherry picked from commit 4d14816c269f110445e1ad3e03ac53b0c1cdb58b)
 2015-04-01 17:25:28 -07:00
Brandon Li
8e2f1a93e4 HADOOP-11757. NFS gateway should shutdown when it can't start UDP or TCP server. Contributed by Brandon Li 
(cherry picked from commit 60ce825a71850fe0622d551159e8d66f32448bb5)
 2015-04-01 17:05:35 -07:00
Haohui Mai
cfcf795492 HDFS-8008. Support client-side back off when the datanodes are congested. Contributed by Haohui Mai. 2015-04-01 16:54:53 -07:00
Haohui Mai
9e114ee607 HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:51:09 -07:00
Colin Patrick Mccabe
6f753da4a9 HDFS-7922. ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors (Rakesh R via Colin P. McCabe) 
(cherry picked from commit 3c7adaaf3571c91fee80585472d2a81402a53e2b)
 2015-04-01 16:11:44 -07:00
Colin Patrick Mccabe
4b74aa7182 HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk (cmccabe) 
(cherry picked from commit c94d594a57806dec515e2a2053a1221f8ce48cc4)
 2015-04-01 14:10:25 -07:00
Zhijie Shen
6d0bebc912 YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong. 
(cherry picked from commit 8366a36ad356e6318b8ce6c5c96e201149f811bd)
 2015-04-01 13:49:00 -07:00
Andrew Wang
b17d8a53fa HDFS-7978. Add LOG.isDebugEnabled() guard for some LOG.debug(..). Contributed by Walter Su. 
(cherry picked from commit ed72daa5df97669906234e8ac9a406d78136b206)
 2015-04-01 12:54:46 -07:00
cnauroth
726024faa5 HADOOP-11787. OpensslSecureRandom.c pthread_threadid_np usage signature is wrong on 32-bit Mac. Contributed by Kiran Kumar M R. 
(cherry picked from commit a3a96a07faf0c6f6aa3ed33608271c2b1657e437)
 2015-04-01 11:40:16 -07:00
Haohui Mai
2ca2515b33 Add the missing files for HDFS-8009. 2015-04-01 10:58:41 -07:00
Haohui Mai
a5bcfe0d33 HDFS-8009. Signal congestion on the DataNode. Contributed by Haohui Mai. 2015-04-01 10:57:07 -07:00
Wangda Tan
865be70b02 YARN-3425. NPE from RMNodeLabelsManager.serviceStop when NodeLabelsManager.serviceInit failed. (Bibin A Chundatt via wangda) 
(cherry picked from commit 492239424a3ace9868b6154f44a0f18fa5318235)
 2015-04-01 10:15:47 -07:00
Xuan
5e4d457674 YARN-3248. Correct fix version from branch-2.7 to branch-2.8 in the change log. 
(cherry picked from commit 2e79f1c2125517586c165a84e99d3c4d38ca0938)
 2015-04-01 04:33:32 -07:00
Xuan
e26b6e55e9 YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev

(cherry picked from commit 4728bdfa15809db4b8b235faa286c65de4a48cf6)
 2015-04-01 04:20:37 -07:00
Tsuyoshi Ozawa
80755edb7a YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. 
(cherry picked from commit c69ba81497ae4da329ddb34ba712a64a7eec479f)
 2015-04-01 17:44:40 +09:00
Karthik Kambatla
93b29d962d YARN-3428. Debug log resources to be localized for a container. (kasha) 
(cherry picked from commit 2daa478a6420585dc13cea2111580ed5fe347bc1)
 2015-03-31 17:35:11 -07:00
Vinod Kumar Vavilapalli
c980e34bc5 YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) 
(cherry picked from commit 7610925e90155dfe5edce05da31574e4fb81b948)
 2015-03-31 17:28:48 -07:00
Akira Ajisaka
b85bbca745 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 
(cherry picked from commit 18a91fe4df0448d9f7de91602646ecf5a51c52e4)
 2015-04-01 09:08:22 +09:00
Vinod Kumar Vavilapalli
4cf44bef5c Reverted MAPREDUCE-6286, MAPREDUCE-6199, and MAPREDUCE-5875 from branch-2.7. Editing CHANGES.txt to reflect this. 
(cherry picked from commit e428fea73029ea0c3494c71a50c5f6c994888fd2)
 2015-03-31 13:30:22 -07:00
Karthik Kambatla
85c420a4d4 YARN-3412. RM tests should use MockRM where possible. (kasha) 
(cherry picked from commit 79f7f2aabfd7a69722748850f4d3b1ff54af7556)
 2015-03-31 09:22:16 -07:00
Akira Ajisaka
7f6811ad73 HDFS-7671. hdfs user guide should point to the common rack awareness doc. Contributed by Kai Sasaki. 
(cherry picked from commit 859cab2f2273f563fd70e3e616758edef91ccf41)
 2015-04-01 00:26:54 +09:00
yliu
e4cd67e9b0 HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) 2015-03-31 21:16:13 +08:00
Tsuyoshi Ozawa
99b8255693 YARN-3258. FairScheduler: Need to add more logging to investigate allocations. Contributed by Anubhav Dhoot. 
(cherry picked from commit b5a22e983832d4843b5df1d07858988e8bbf37e3)
 2015-03-31 17:43:07 +09:00
Arpit Agarwal
312e50f9c4 HDFS-7944. Minor cleanup of BlockPoolManager#getAllNamenodeThreads. (Arpit Agarwal) 2015-03-30 21:24:59 -07:00
Harsh J
32766b6563 HDFS-3918. EditLogTailer shouldn't log WARN when other node is in standby mode. Contributed by Todd Lipcon. 
(cherry picked from commit cce66ba3c9ec293e8ba1afd0eb518c7ca0bbc7c9)
 2015-03-31 08:17:22 +05:30
Arpit Agarwal
abf3ad988d HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 2015-03-30 15:25:26 -07:00
Vinod Kumar Vavilapalli
defae1d399 Revert "MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter." 
This reverts commit 5358b83167777a7108b32c9900fb0d01ca0fe961.

(cherry picked from commit cc0a01c50308fe599b4bdd5a80068d6244062e4f)
 2015-03-30 14:52:36 -07:00
Colin Patrick Mccabe
0048ec452b HADOOP-11660. Add support for hardware crc of HDFS checksums on ARM aarch64 architecture (Edward Nevill via Colin P. McCabe) 
(cherry picked from commit d9ac5ee2c4dcd4a108ca892af501618caaea450c)
 2015-03-30 14:03:11 -07:00
Haohui Mai
02fcf622fb Addendum for HDFS-7748. 2015-03-30 13:11:24 -07:00
Wangda Tan
cba4ed1678 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 
(cherry picked from commit 2a945d24f7de1a7ae6e7bd6636188ce3b55c7f52)
 2015-03-30 12:05:54 -07:00
Haohui Mai
dd5b2dac5a HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. 2015-03-30 11:59:32 -07:00