Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,422 Commits 326 Branches 371 Tags
Commit Graph

13422 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arun Suresh
2d62af6545 YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh) 
(cherry picked from commit aaf106fde35ec97e2e2ea4d7a67434038c4273ac)
 2017-08-04 16:19:44 -07:00
Lei Xu
d87a92bf55 HDFS-12251. Add document for StreamCapabilities. (Lei (Eddy) Xu) 
(cherry picked from commit fe3341786a0d61f404127bf21d1afc85b2f21d38)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
 2017-08-04 11:25:21 -07:00
Steve Loughran
ced547d5f0
HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws. 
Contributed by Steve Loughran
 2017-08-04 11:07:34 +01:00
wenxinhe
11e1e0c275
HADOOP-14706. Adding a helper method to determine whether a log is Log4j implement. 
This closes #258

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-08-04 14:42:35 +09:00
Arun Suresh
88d6adf419 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 35dc7829236f92054d5ce6ea78d3a44ca6c8f3d3)
 2017-08-03 21:18:43 -07:00
Yufei Gu
b6729a7be5 YARN-6895. [FairScheduler] Preemption reservation may cause regular reservation leaks. (Miklos Szegedi via Yufei Gu) 2017-08-03 16:56:15 -07:00
Andrew Wang
2da20df1dd HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:45:21 -07:00
Yufei Gu
ce609d06c0 YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 13:52:58 -07:00
Yufei Gu
3ef7adc748 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 
(cherry picked from commit b8e8241854904e3a92b636ffaa6f4c651e47cea2)
 2017-08-03 11:47:09 -07:00
Sunil G
82ec700241 YARN-6678. Handle IllegalStateException in Async Scheduling mode of CapacityScheduler. Contributed by Tao Yang. 
(cherry picked from commit f64cfeaf61ec65a465decdd8215f567d4e6677a9)
 2017-08-03 19:28:16 +05:30
Manoj Govindassamy
065a906be7 HDFS-9388. Decommission related code to support Maintenance State for datanodes. 2017-08-02 16:58:54 -07:00
Arpit Agarwal
45cde760b0 HDFS-12043. Add counters for block re-replication. Contributed by Chen Liang. 2017-08-02 16:45:08 -07:00
Eric Payne
223229b99c YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 8ce8672b6b551dacb9467924fc70f88790f5891f)
 2017-08-02 12:58:07 -05:00
Manoj Govindassamy
a77ae7abdb HDFS-12217. HDFS snapshots doesn't capture all open files when one of the open files is deleted. 
(cherry picked from commit 52d7bafcf49916887197436ddb0f08f021d248d9)
 2017-08-02 10:16:30 -07:00
Eric Payne
686bdc48b8 YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe. 
(cherry picked from commit 48899134d2a77935a821072b5388ab1b1b7b399c)
 2017-08-02 11:29:56 -05:00
Jason Lowe
f34c01b92a Revert "HDFS-12217. HDFS snapshots doesn't capture all open files when one of the open files is deleted." 
This reverts commit b0330c15da8483302007d1df797f4398b7129cdb.
 2017-08-02 11:15:58 -05:00
Steve Loughran
cb46792610
HADOOP-14709. Fix checkstyle warnings in ContractTestUtils. 
Contributed by Thomas Marquardt.
 2017-08-02 12:46:30 +01:00
Jian He
6ee0fe70c7 YARN-6872. [Addendum patch] Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 2017-08-02 00:03:59 -07:00
Masatake Iwasaki
d9a07b3904 HADOOP-14702. Fix formatting issue and regression caused by conversion from APT to Markdown. Contributed by Doris Gu. 2017-08-02 06:21:37 +00:00
Manoj Govindassamy
b0330c15da HDFS-12217. HDFS snapshots doesn't capture all open files when one of the open files is deleted. 
(cherry picked from commit 52d7bafcf49916887197436ddb0f08f021d248d9)
 2017-08-01 17:15:29 -07:00
Manoj Govindassamy
70c511598f HDFS-11402. HDFS Snapshots should capture point-in-time copies of OPEN files. 2017-08-01 17:11:27 -07:00
Andrew Wang
5467d1a4d1 HADOOP-14701. Configuration can log misleading warnings about an attempt to override final parameter. Contributed by Andrew Sherman. 
(cherry picked from commit a11c230236036317a6c12deeca401e3ae8dce698)

 Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
 2017-08-01 15:18:18 -07:00
Arpit Agarwal
d19504866c HDFS-14676. Wrong default value for fs.df.interval. Contributed by xiangguang zheng. 2017-08-01 13:31:55 -07:00
Jian He
4006cc266b YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 2017-08-01 10:00:04 -07:00
Akira Ajisaka
72bde1dad5
HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor. 
(cherry picked from commit b38a1eea8e2917989d83d169a7b5773163e6832e)
 2017-08-01 15:16:21 +09:00
Akira Ajisaka
338d84ab71
MAPREDUCE-6921. TestUmbilicalProtocolWithJobToken#testJobTokenRpc fails. Contributed by Sonia Garudi. 
(cherry picked from commit ceacadc51e58bb94ad3f3669488515a61e886d88)
 2017-08-01 15:02:20 +09:00
Lei Xu
f09d20cffb HADOOP-14397. Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it. (Lei (Eddy) Xu) 
(cherry picked from commit 667ee003bf47e44beb3fdff8d06a7264a13dd22c)
 2017-07-31 20:07:13 -07:00
Weiwei Yang
60ae10b14f HDFS-12154. Incorrect javadoc description in StorageLocationChecker#check. Contributed by Nandakumar. 2017-07-31 17:02:54 -07:00
Andrew Wang
b161516705 HADOOP-14644. Increase max heap size of Maven javadoc plugin. Contributed by Andras Bokor. 
(cherry picked from commit 2be9412b73ae4308c5cee0186520fc2ad6d54e43)
 2017-07-31 15:09:43 -07:00
Jason Lowe
67d31a8e7e Revert "MAPREDUCE-6829. Add peak memory usage counter for each task. (Miklos Szegedi via kasha)" 
This reverts commit a91cc63d374a5770b17cd8465080caf7f91da2a8.
 2017-07-31 16:44:13 -05:00
Jason Lowe
b9640c02bf YARN-6628. Unexpected jackson-core-2.2.3 dependency introduced. Contributed by Jonathan Eagles 2017-07-31 16:20:54 -05:00
Junping Du
cae33098e6 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit e8a31f2e1c34514fba2f480e8db652f6e2ed65d8.

(cherry picked from commit fbb7d6bcbb887ce52ab1e9d5a1fed67a7f8a4be8)
 2017-07-31 14:15:06 -07:00
Junping Du
bf360da63f Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2cffc4e3ac37cb7da4e535ca5c6d3913.
 2017-07-31 14:13:26 -07:00
Junping Du
c0f8fd6432 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 7c72c7f5290f0991f65102c2bd413f965a8ca409.
 2017-07-31 14:11:37 -07:00
Arpit Agarwal
8c63580981 HDFS-12082. BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured. Contributed by Weiwei Yang. 2017-07-31 11:34:00 -07:00
Akira Ajisaka
b19415f1a1
HADOOP-14677. mvn clean compile fails. Contributed by Andras Bokor. 
(cherry picked from commit 0fd6d0f2d361536b2baf859ddbb082eb2eadcfcf)
 2017-07-31 17:01:46 +09:00
Akira Ajisaka
9c8a296169
HADOOP-14690. RetryInvocationHandler should override toString(). Contributed by Yeliang Cang. 
(cherry picked from commit f14be0d24126747887ddc7580f4a9a70768de23d)
 2017-07-31 14:10:18 +09:00
Akira Ajisaka
98c3544e94
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout. 
(cherry picked from commit f8bed5e9a7d1ece127fb3da123bbfc26ada0016f)
 2017-07-31 11:10:10 +09:00
Brahma Reddy Battula
8bfb9971ca HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 2017-07-30 14:01:41 +08:00
Arun Suresh
7e643130f2 YARN-6870. Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker. (Brook Zhou via asuresh) 
(cherry picked from commit 890e14c02a612c772cecd5dff2411060efd418a3)
 2017-07-28 16:33:42 -07:00
John Zhuge
ba94c68e0b HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge. 
(cherry picked from commit f735ad1b67ed82d9b11b1afd7ae39035a6aed18b)
 2017-07-28 09:51:32 -07:00
Yufei Gu
e29d1c75ea YARN-6864. FSPreemptionThread cleanup for readability. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit 9902be72cbf7a170caa5cb1f13c227d881a39064)
 2017-07-27 23:21:30 -07:00
Yongjun Zhang
c3e8c7d56e HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang. 
(cherry picked from commit c6330f22a5e5c2370bab885f9bea4bf8f5e9cf44)
 2017-07-27 17:05:30 -07:00
Brahma Reddy Battula
1c5095b3cb HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694fae3f814ab4e7f3c172da1df0e0e353)
 2017-07-27 12:30:57 -07:00
Varun Saxena
05bbbcbdbe Addendum for YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit 11ece0bda1f6e5dd9d0f828b7c29acacf6087baa)
 2017-07-28 00:33:41 +05:30
Varun Saxena
0b342a644a YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit f66fd11e514fb326fd1f37a88b444a5276f0947b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
 2017-07-28 00:33:37 +05:30
Kihwal Lee
4edb849611 HADOOP-14578. Bind IPC connections to kerberos UPN host for proxy users. Contributed by Daryn Sharp. 
(cherry picked from commit 27a1a5fde94d4d7ea0ed172635c146d594413781)
 2017-07-26 13:16:34 -05:00
Sunil G
08d5f3acf6 Addendum patch for YARN-6102. 2017-07-26 18:40:52 +05:30
Jitendra Pandey
554c3cd21d HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt. 2017-07-25 16:35:40 -07:00
Daniel Templeton
00f4a80e8f YARN-6307. Refactor FairShareComparator#compare (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit f81a4efb8c40f99a9a6b7b42d3b6eeedf43eb27a)
 2017-07-25 13:02:00 -07:00