Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,542 Commits 326 Branches 371 Tags
Commit Graph

10542 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Chris Douglas
4150d08a4b HADOOP-12636. Prevent ServiceLoader failure init for unused FileSystems. Contributed by Inigo Goiri 
(cherry picked from commit 8652cce5b21825f6f835e4ea31de82eb59fb06c1)
(cherry picked from commit d984ba4fd73df64b7dca1ec10b35956e4ae9bd57)
 2015-12-18 18:31:36 -08:00
cnauroth
d6f1d3b6ca HADOOP-12570. HDFS Secure Mode Documentation updates. Contributed by Arpit Agarwal. 
(cherry picked from commit 932956458dddabd9583b911c693386a56d723c0d)
(cherry picked from commit dd91cf9fa51df11c80f52e6845555bb9f2b8acef)
 2015-12-18 16:54:36 -08:00
Wangda Tan
b74391ce14 YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue. (Naganarasimha G R via wangda) 
(cherry picked from commit 9b856d9787be5ec88ef34574b9b98755d7b669ea)
(cherry picked from commit 5060fd1840ba2962cdfd7d91e2f405d7bdcfd169)
 2015-12-18 16:34:45 -08:00
Wangda Tan
c4b1386b4e YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda) 
(cherry picked from commit 7faa406f27f687844c941346f59a27a375af3233)
(cherry picked from commit 2f5435acbda53470724d833cfa1c58dc3c3c9bb1)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:45 -08:00
Wangda Tan
9ed9eb265f YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83e981ae74cb8b58ffcf7907b7612ad4)
(cherry picked from commit 9f4a6ec6630f3776859f8d677532370095a58d0c)
 2015-12-18 16:34:45 -08:00
Zhihai Xu
e4f638d815 YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time. Contributed by Lin Yiqun 
(cherry picked from commit 2aaed1032719e978660d1f14a8ed299e83b5f65e)
(cherry picked from commit be50df687bf800d35f418128d0c3f05ab6c429c8)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:45 -08:00
Wangda Tan
1d1c6c1f8d YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda) 
(cherry picked from commit 07b0fb996a32020678bd2ce482b672f0434651f0)
(cherry picked from commit c915a658c8a5481ca88d8b2f6ce9a72f2e1f5b6a)
 2015-12-18 16:34:44 -08:00
Wangda Tan
c7e10d70de YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 
(cherry picked from commit dfcbbddb0963c89c0455d41223427165b9f9e537)
(cherry picked from commit 9f979107c22f365633811a38363d0dbd34573374)
 2015-12-18 16:34:44 -08:00
Wangda Tan
8ee83765f4 YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda) 
(cherry picked from commit 6cb0af3c39a5d49cb2f7911ee21363a9542ca2d7)
(cherry picked from commit a1a723fdffee44be9a5d2a1ced896442a5318375)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:44 -08:00
Xiaoyu Yao
4f21f9408e HDFS-8326. Documentation about when checkpoints are run is out of date. (Misty Stanley-Jones via xyao) 
(cherry picked from commit d0e75e60fb16ffd6c95648a06ff3958722f71e4d)
(cherry picked from commit 5595b912e417286594ec0703d3e6062b00985fcc)
 2015-12-18 16:34:44 -08:00
Xuan
41986d5373 YARN-3623-Addendum: Improve the description for Timeline Service Version configuration in yarn-default.xml 
(cherry picked from commit 21daa6c68a0bff51a14e748bf14d56b2f5a5580f)
(cherry picked from commit 455c6f39c582f5f482f0dc255d111e6fe149e66a)
 2015-12-18 16:34:43 -08:00
Wangda Tan
a4ba6c8525 YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 
(cherry picked from commit 7e4715186d31ac889fba26d453feedcebb11fc70)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 7013f9d6cda88e72a839b1c55757615b55101beb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:43 -08:00
Jonathan Eagles
c4ae007fd4 YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles) 
(cherry picked from commit 4ff973f96ae7f77cda3b52b38427e2991819ad31)
(cherry picked from commit 7c604c3d065ab15927ef7e14be6acf7772f6077f)
 2015-12-18 16:34:43 -08:00
Xuan
46cb47c03e YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b6762c18ba150d80a8976eb51a8290c)
(cherry picked from commit 7decba83ef541b1f89745ad040c6137d24b187b0)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:43 -08:00
Xiaoyu Yao
f3ee0e7857 HDFS-8831. Trash Support for deletion in HDFS encryption zone. Contributed by Xiaoyu Yao. 
(cherry picked from commit cbc7b6bf97a80c39d4bbb3005e42dacae6726baf)
(cherry picked from commit eb7f9901b4355c5488172dc93f734ee496b3e4dc)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-12-18 16:34:43 -08:00
Lei Xu
134195df55 HDFS-9490. MiniDFSCluster should change block generation stamp via FsDatasetTestUtils. (Tony Wu via lei) 
(cherry picked from commit 0ac8fb4b336dd03a037092c9712b962e7ed8f852)
(cherry picked from commit 58714f3b7357d05604987e1db8f6734a7d2827d9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-12-18 16:34:42 -08:00
Jian He
12772f5a45 Add missing file for YARN-4419 
(cherry picked from commit 8a97ca4b00c4eea044d90e8ed0dc75038bd67bc6)
 2015-12-18 16:34:42 -08:00
Jian He
f355645ca4 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 
(cherry picked from commit 755dda8dd8bb23864abc752bad506f223fcac010)

(cherry picked from commit be404e168060c03a33a35c8d30925e317a64cad7)
 2015-12-18 16:34:42 -08:00
Konstantin V Shvachko
0b88a7f259 HDFS-9484. NNThroughputBenchmark$BlockReportStats should not send empty block reports. Contributed by Mingliang Liu. 
(cherry picked from commit 33d4588a7c57a0945f82fb14d8127d49c9ec68d2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-12-18 16:34:42 -08:00
Konstantin V Shvachko
69d3a71f14 HDFS-9436. Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default. Contributed by Mingliang. 
(cherry picked from commit 3aa73a867a3a5048de72e08d73ffaabe84f8ddee)
 2015-12-18 16:33:53 -08:00
Wangda Tan
15b3ff1971 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda) 
(cherry picked from commit a2c3bfc8c1349102a7f2bc4ea96b80b429ac227b)
(cherry picked from commit 51e448d295f6d2214c1d6455446a8e05e8a967b1)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:33:53 -08:00
Jian He
9c22910570 YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 
(cherry picked from commit 48cbde908319775b82ae30f262cca0503866e88e)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:33:53 -08:00
Arpit Agarwal
cef21b7aaa HADOOP-12656. MiniKdc throws "address in use" BindException. (Contributed by Wei-Chiu Chuang) 2015-12-18 16:08:22 -08:00
Zhe Zhang
65effb76d9 HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang. 
Change-Id: Idde8522e11c77f82b376e0c90a9bf33d163d5cb9
 2015-12-18 16:06:42 -08:00
Vinayakumar B
384628a2d6 HDFS-9393. After choosing favored nodes, choosing nodes for remaining replicas should go through BlockPlacementPolicy (Contributed by J.Andreina) 
(cherry picked from commit bfadf11b36e9d97e03d6ed1e71829907c2301412)
(cherry picked from commit c887bcd1f03244300a45ff70df9a9c9f2ad5a431)
 2015-12-18 11:41:27 +05:30
Akira Ajisaka
e09d026105 HADOOP-12657. Add a option to skip newline on empty files with getMerge -nl. Contributed by Kanaka Kumar Avvaru. 
(cherry picked from commit 061c05cc05ff6257b14c5c4f25cbcec2d184cda7)
(cherry picked from commit af498234997d68b718abed2be7638819e93f2266)
 2015-12-18 14:00:42 +09:00
Akira Ajisaka
c18bf20154 HADOOP-12588. Fix intermittent test failure of TestGangliaMetrics. Contributed by Masatake Iwasaki. 
(cherry picked from commit bd5e207432c54cc75fd8c040e232241678987e53)
(cherry picked from commit e06c291245902a6014770134443140eb78048240)
 2015-12-18 13:13:12 +09:00
cnauroth
9a6814f37c HDFS-9571. Fix ASF Licence warnings in Jenkins reports. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6da9deec204a4fa69dfc2d891ce934979d59b0ab)
(cherry picked from commit 7e5306e7952c1020b4cf9b4f71057ebb40db4ad7)
 2015-12-17 16:26:16 -08:00
Kihwal Lee
e5e5ba495f HDFS-9533. seen_txid in the shared edits directory is modified during bootstrapping. Contributed by Kihwal Lee. 
(cherry picked from commit 5cb1e0118b173a95c1f7bdfae1e58d7833d61c26)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestBootstrapStandby.java
(cherry picked from commit 3c0ae5914ca673b055a9673b4c17c8aaa570c258)
 2015-12-17 17:23:55 -06:00
Junping Du
8812be3dc1 YARN-3226. UI changes for decommissioning node. Contributed by Sunil G. 
(cherry picked from commit 1de56b0448d332717c8316c621b4f6af542a85cc)
(cherry picked from commit 6d2914a69791d66862630516142573e02b9d55d7)
 2015-12-17 15:22:37 -08:00
cnauroth
12e6f38f52 HDFS-9572. Prevent DataNode log spam if a client connects on the data transfer port but sends no data. Contributed by Chris Nauroth 
(cherry picked from commit 03bab8dea163a9ee45d09d2a0483d45cf6fe57c9)
(cherry picked from commit 294d0f6f6a90926f23cbcf8006cbc893bdb488e6)
 2015-12-17 14:05:19 -08:00
Arpit Agarwal
c8e5d2eba4 HDFS-9515. NPE when MiniDFSCluster#shutdown is invoked on uninitialized reference. (Contributed by Wei-Chiu Chuang) 2015-12-17 13:57:03 -08:00
cnauroth
1e81e1f891 Revert "HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang." 
This reverts commit a8dc35dfdf9f3ab82d88a7ef2d03e172d594499b.
 2015-12-17 13:50:02 -08:00
Allen Wittenauer
b11362fd2a HDFS-9570. Minor typos, grammar, and case sensitivity cleanup in HdfsPermissionsGuide.md's (Travis Campbell via aw) 
(cherry picked from commit 8d278d8d2e5a8a53308608a8fbc6ba96228e53ef)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit a5c74cd921d0305f3baa23d7dc53064cb3d09968)
 2015-12-17 11:41:11 -08:00
cnauroth
7cc11b22dd HDFS-9552. Document types of permission checks performed for HDFS operations. Contributed by Chris Nauroth. 
(cherry picked from commit 724e71f63b35695482642ed7aedacd70d08e6cb1)
(cherry picked from commit 940026fc6b2682a3b62d3e33ba977f5ae6375fbf)
 2015-12-17 10:44:00 -08:00
Arpit Agarwal
12df736673 HDFS-9565. TestDistributedFileSystem.testLocatedFileStatusStorageIdsTypes is flaky due to race condition. (Contributed by Wei-Chiu Chuang) 2015-12-17 10:38:58 -08:00
Kihwal Lee
a8dc35dfdf HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang. 
(cherry picked from commit eb6939cea0343840c62b930d4adb377f5eaf879f)
 2015-12-17 10:31:37 -06:00
Junping Du
92de57bdca YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 
(cherry picked from commit 50bd067e1d63d4c80dc1e7bf4024bfaf42cf4416)
(cherry picked from commit 1a2ef845b54166194b133e524ad9533cc259aed2)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java
(cherry picked from commit 0f9dd488428ae65a17043f3a976c6ef778806328)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-12-17 06:16:08 -08:00
Plamen Jeliazkov
cffa262f11 HDFS-9516. Truncate file fails with data dirs on multiple disks. Contributed by Plamen Jeliazkov. 2015-12-17 00:03:51 -08:00
Uma Mahesh
e6dc6f24a2 HDFS-9198. Coalesce IBR processing in the NN. (Daryn Sharp via umamahesh) 
(cherry picked from commit f741476146574550a1a208d58ef8be76639e5ddc)
 2015-12-16 22:50:12 -08:00
Xuan
93c483889c MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 
(cherry picked from commit d4e766de931c1cc478561a211215c517ba1f341c)
(cherry picked from commit 5891d6ed55fdc73da6afa8517ebe9f4d2b666473)
 2015-12-16 15:31:09 -08:00
Jason Lowe
d939c23cc1 YARN-4461. Redundant nodeLocalityDelay log in LeafQueue. Contributed by Eric Payne 
(cherry picked from commit 91828fef6b9314f72d1f973f00e81404dc6bba91)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-12-16 23:24:51 +00:00
Vinod Kumar Vavilapalli
753c9292c2 HADOOP-12415. Fixed pom files to correctly include compile-time dependency on netty. (Tom Zeng via cos) 
Addendum commit to include CHANGES.txt entry.

(cherry picked from commit ce16541c62728f69634111dc2ddb321690d3d29b)
 2015-12-16 14:42:09 -08:00
Siddharth Seth
7415870d13 YARN-4207. Add a non-judgemental YARN app completion status. Contributed by Rich Haase. 
(cherry picked from commit 0f708d465fbc4260f2c36e8067e27cd8b285fde7)
(cherry picked from commit 433212ef878205b0c14e940dd660d1a6e8dc0238)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-16 11:39:37 -08:00
cnauroth
d3e1161b8d HDFS-9557. Reduce object allocation in PB conversion. Contributed by Daryn Sharp. 
(cherry picked from commit c470c8953d4927043b6383fad8e792289c634c09)
(cherry picked from commit e8e3737ce8224df41f57b5d03ed097e8ab413bda)
 2015-12-16 11:11:34 -08:00
Zhihai Xu
c4febc7d02 MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d06921893200163a3d29c8901c3c0107cb)
 2015-12-15 22:42:19 -08:00
Uma Mahesh
58cc106378 HADOOP-12639. Improve JavaDoc for getTrimmedStrings. (BELUGA BEHR via umamahesh) 
(cherry picked from commit ae67c8424c188a2b00920108e6104690137c61b7)
 2015-12-15 15:21:58 -08:00
Andrew Wang
8e5de6cc42 HDFS-8894. Set SO_KEEPALIVE on DN server sockets. Contributed by Kanaka Kumar Avvaru. 
(cherry picked from commit 49949a4bb03aa81cbb9115e91ab1c61cc6dc8a62)
(cherry picked from commit 2796c1099f5c770d534f4bf10c1d80886d0bc194)
 2015-12-15 14:40:23 -08:00
Xuan
f22d0046de YARN-4439. Clarify NMContainerStatus#toString method. Contributed by 
Jian He

(cherry picked from commit d8a45425eba372cdebef3be50436b6ddf1c4e192)
(cherry picked from commit 992f65dd8b53ee31b6b7347e2cfe59a78d031268)
 2015-12-14 17:16:07 -08:00
Yongjun Zhang
4bc1aab49d HDFS-9514. TestDistributedFileSystem.testDFSClientPeerWriteTimeout failing; exception being swallowed. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit bf5295b118a24e762f21bb2b9054e5e0c1d1d14a)
 2015-12-14 15:40:55 -08:00