Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,064 Commits 326 Branches 371 Tags
Commit Graph

9064 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Lowe
8ab642989c YARN-4439. Clarify NMContainerStatus#toString method. Contributed by Jian He 2015-12-15 18:06:13 +00:00
Jason Lowe
97a20ff2c6 Revert "YARN-4439. Clarify NMContainerStatus#toString method. Contributed by" 
This reverts commit 1507d30bc126cac3f18a87de0936c2d0d0d1ccff.

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-12-15 17:59:45 +00:00
Zhihai Xu
35a5b8929e MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d06921893200163a3d29c8901c3c0107cb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-15 01:24:03 -08:00
Xuan
1507d30bc1 YARN-4439. Clarify NMContainerStatus#toString method. Contributed by 
Jian He

(cherry picked from commit d8a45425eba372cdebef3be50436b6ddf1c4e192)
(cherry picked from commit 992f65dd8b53ee31b6b7347e2cfe59a78d031268)
 2015-12-14 15:42:38 -08:00
Kihwal Lee
a02312546a HDFS-9445. Datanode may deadlock while handling a bad volume. Contributed by Walter Su. 
(cherry picked from commit a48301791e9564363bc2abad4e89e344b0d7a5ff)
 2015-12-11 08:47:22 -06:00
Akira Ajisaka
02f666b324 HADOOP-12602. TestMetricsSystemImpl#testQSize occasionally fails. Contributed by Masatake Iwasaki. 
(cherry picked from commit eee0cf4611b02171e8a043f1cc5f7dbad47fc3b4)
(cherry picked from commit 5047ee56af9edd2e6240286e2238b638a738f570)
 2015-12-11 16:19:28 +09:00
Akira Ajisaka
c9be259f26 YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang. 
(cherry picked from commit 16dc65811d4e0e7ab61c61295149848fe53ddd89)
(cherry picked from commit 925bb94d87db7e916ed380d4512225e796f9e03d)
 2015-12-09 16:26:58 +09:00
Wangda Tan
fd18377255 YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 
(cherry picked from commit 7e4715186d31ac889fba26d453feedcebb11fc70)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 7013f9d6cda88e72a839b1c55757615b55101beb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-08 14:31:31 -08:00
Vinod Kumar Vavilapalli
067d14f503 HDFS-9273. Moving to 2.6.3 CHANGES section to reflect the backport. 2015-12-08 11:36:15 -08:00
cnauroth
e23ab9d02b HDFS-9273. ACLs on root directory may be lost after NN restart. Contributed by Xiao Chen. 
(cherry picked from commit 1b525a9c32fabd8919c80717a58afbfa7fdce27e)
(cherry picked from commit faf1c608990408e2aa59a5e676f86d0c82dbd7f3)
 2015-12-08 11:22:19 -08:00
Tsuyoshi Ozawa
0460b8a8a3 YARN-4348. ZKRMStateStore.syncInternal shouldn't wait for sync completion for avoiding blocking ZK's event thread. (ozawa) 2015-12-08 13:33:10 +09:00
Xuan
ba02e8f6d1 MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 
(cherry picked from commit fc470840a0b1f5dc8b3b13f7ed99fd68ba728216)
(cherry picked from commit 3b10ff6a6f7becea181506c306a52973602b0481)
 2015-12-07 14:18:26 -08:00
Jonathan Eagles
22be47d747 YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles) 
(cherry picked from commit 4ff973f96ae7f77cda3b52b38427e2991819ad31)
 2015-12-07 15:09:54 -06:00
Zhe Zhang
670617f325 HDFS-9314. Improve BlockPlacementPolicyDefault's picking of excess replicas. Contributed by Xiao Chen. 
Change-Id: Idf99293085531165239369155c039b55db0eed83
 2015-12-04 09:49:43 -08:00
Zhe Zhang
b39de3b0bd HDFS-9313. Possible NullPointerException in BlockManager if no excess replica can be chosen. Contributed by Ming Ma. Branch-2.7 patch contributed by Xiao Chen. 
Change-Id: Iadaee6be2b44f7b0322d7b0e58f8fc983ef463f6
 2015-12-03 23:19:17 -08:00
Jian He
6a30d93aa4 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad735f4843ce2c38355de9f434838d4507)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebApp.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
 2015-12-03 14:11:48 -08:00
Zhe Zhang
1ae8a8342e HDFS-8647. Abstract BlockManager's rack policy into BlockPlacementPolicy. Contributed by Brahma Reddy Battula. Branch-2.7 patch contributed by Xiao Chen. 
Change-Id: I1b32627bc2a2f30f5debeaba7663fb2777958079
 2015-12-03 10:30:48 -08:00
Akira Ajisaka
7404506111 HADOOP-12565. Replace DSA with RSA for SSH key type in SingleCluster.md. Contributed by Mingliang Liu. 
(cherry picked from commit 3857fed2c8fc601b46e8331f73a3104f4f33e498)
(cherry picked from commit 15c2f709c685986518feba60fc35222574ce3f0f)
 2015-12-03 11:52:28 +08:00
Tsz-Wo Nicholas Sze
4123650e92 HDFS-9294. DFSClient deadlock when close file and failed to renew lease. Contributed by Brahma Reddy Battula 2015-12-02 17:55:10 -08:00
Jian He
42a2bddcb7 YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 
(cherry picked from commit 6b9a5beb2b2f9589ef86670f2d763e8488ee5e90)
 2015-12-02 11:09:08 -08:00
Andrew Wang
10e8a67d23 HDFS-9470. Encryption zone on root not loaded from fsimage after NN restart. Xiao Chen via wang. 
(cherry picked from commit 9b8e50b424d060e16c1175b1811e7abc476e2468)
(cherry picked from commit ce1111ceea830cce5f0833db55201e0e88c3b199)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
 2015-11-30 14:53:04 -08:00
Jason Lowe
e1b6a5413f MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) 
(cherry picked from commit 7fd00b3db4b7d73afd41276ba9a06ec06a0e1762)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-30 17:45:27 +00:00
Vinayakumar B
0c48c17ae9 HDFS-9426. Rollingupgrade finalization is not backward compatible (Contributed by Kihwal Lee) 
(cherry picked from commit c62d42cd8bb09a5ffc0c5eefa2d87913e71b9e7e)
 2015-11-26 09:42:26 +05:30
Tsuyoshi Ozawa
f50f889c17 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83af6a48a8d8d0e37cdf1f813124f366)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
 2015-11-26 01:22:11 +09:00
Jason Lowe
b68f527b9f YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla 
(cherry picked from commit f5acf94ecafb301a0cc8e8f91f19c8bcbc8da701)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestFileSystemNodeLabelsStore.java
 2015-11-24 23:52:46 +00:00
Jason Lowe
4ba5667aed MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts 
(cherry picked from commit cab3c7c8892ad33a7eb0955b01e99872ab95e192)
 2015-11-24 22:02:21 +00:00
Tsz-Wo Nicholas Sze
af811030e5 Move HDFS-9434 to 2.6.3 in CHANGES.txt. 2015-11-24 12:49:34 -08:00
Tsz-Wo Nicholas Sze
09f4a4eefd HDFS-9434. Recommission a datanode with 500k blocks may pause NN for 30 seconds for printing info log messags. 2015-11-24 12:47:50 -08:00
Jason Lowe
2946621a53 Revert "HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula." 
This reverts commit 4356e8a5ef0ac6d11a34704b80ef360a710e623a.

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
(cherry picked from commit 17b1a5482b32dab82225e8233648990bc77674ba)
 2015-11-24 19:18:37 +00:00
Jason Lowe
9bd9544bda Update CHANGES.txt for recent commits to branch-2.6. 
(cherry picked from commit aa2009c37bc3c47035a5aea4a7449d3fe48569d4)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-23 22:00:27 +00:00
Jason Lowe
5bd40d7099 YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 
(cherry picked from commit d36b6e045f317c94e97cb41a163aa974d161a404)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java
 2015-11-23 20:47:12 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
f62cdc38f3 HADOOP-12577. Bumped up commons-collections version to 3.2.2 to address a security flaw. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 70c26703f462e97361924eaf6cbf80be1fce309f)
 2015-11-23 10:01:38 -08:00
Wangda Tan
c7e23255ae YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda) 2015-11-20 16:36:11 -08:00
Wangda Tan
2ad1386b2b YARN-3849. Too much of preemption activity causing continuos killing of containers across queues. (Sunil G via wangda) 2015-11-20 16:17:23 -08:00
Jason Lowe
9074714a18 YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 
(cherry picked from commit 060cdcbe5db56383e74e282ac61eedff8248c11b)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java
 2015-11-20 23:25:27 +00:00
Xuan
eb9296d316 YARN-2859 addendum: move the entry from 2.6.2 to 2.6.3 in hadoop-yarn/CHANGES.txt. 
(cherry picked from commit 866dce4e8e105d0e2c7e041b5d30213536a61702)
(cherry picked from commit 8347b1048d063adaf40fc4f537aaa9bfd1157c37)
 2015-11-19 10:18:47 -08:00
Xuan
fe67c6f43f YARN-2859.addendum: fix the remaining issue from the previous patch 
(cherry picked from commit f114e728da6e19f3d35ff0cfef9fceea26aa5d46)
(cherry picked from commit be65d319fae8f8644c46ee7c44108ce72dafc9c7)
 2015-11-19 10:10:22 -08:00
Akira Ajisaka
bb6983e153 HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed by Kazuho Fujii. 
(cherry picked from commit ac1aa6c8197268ea83f44747199d560be9e299ae)
(cherry picked from commit c74e42b4a2293c8adb99b22929047a8d07242d06)
 2015-11-19 20:55:52 +09:00
Wangda Tan
82de3e1c11 YARN-4287. Capacity Scheduler: Rack Locality improvement. (Nathan Roberts via wangda) 2015-11-18 13:58:47 -08:00
Jason Lowe
b96377334c MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.
(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-18 17:12:50 +00:00
Ming Ma
1b0f277eae HDFS-9413. getContentSummary() on standby should throw StandbyException. (Brahma Reddy Battula via mingma) 
(cherry picked from commit 02653add98f34deedc27f4da2254d25e83e55b58)
(cherry picked from commit 42b55ff23e5fa30e00a6b56df92777a1899a1952)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 83200450f67c1fe2b17385ade23051d83c7354bf)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-11-17 16:12:58 +05:30
Akira Ajisaka
ab04c9ab5f HDFS-8615. Correct HTTP method in WebHDFS document. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1a169a26bcc4e4bab7697965906cb9ca7ef8329e)
(cherry picked from commit 9f9315008e75765de4b905f7d57de9505fb5ca10)
 2015-11-17 16:32:07 +09:00
Jian He
27b786660e Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit cb4d4d5918c4d488056e5521e343aaa9ffbd24d1.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:20:46 -08:00
Jian He
f95f416fad MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du 2015-11-16 17:13:49 -08:00
Jason Lowe
e1716c7cf9 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b6115e2cfbd97a94d6c1a3ddf0e94bb)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
 2015-11-16 15:02:56 +00:00
Jason Lowe
38f37ca2eb Revert "YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe." 
This reverts commit 8ea88ab0b87482b779ec9f629b46e9ac7ad53cb5.
 2015-11-16 14:30:22 +00:00
Junping Du
8ea88ab0b8 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b6115e2cfbd97a94d6c1a3ddf0e94bb)
(cherry picked from commit 1d107d805841e9b39db01aa236068a0ff1ca46d1)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
 2015-11-15 04:50:28 -08:00
Akira Ajisaka
854a0fa6c9 Fix conflict in CHANGES.txt when backporting HADOOP-12482 to branch-2.7. 2015-11-15 19:04:30 +09:00
Lei Xu
929699f7eb HADOOP-12482. Race condition in JMX cache update. (Tony Wu via lei) 
(cherry picked from commit 0eb9c60c5bec79f531da8cb3226d7e8b1d7e6639)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-11-14 14:58:50 +09:00
Robert Kanter
d833950317 HADOOP-12348. MetricsSystemImpl creates MetricsSourceAdapter with wrong time unit parameter. (zxu via rkanter) 
(cherry picked from commit 764f8baa99de35a0ced297b19b0cbe090c615a3b)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-11-14 14:50:01 +09:00