Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,224 Commits 326 Branches 371 Tags
Commit Graph

14224 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Lowe
fd09f72151 Revert "HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)" 
This reverts commit 896dc7c7801adaa4460fa6c19a4f452a6a6112d8.
 2018-01-17 09:47:43 -06:00
Vinayakumar B
b3eeea0b6e HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 
(cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)
 2018-01-17 14:22:29 +05:30
Sammi Chen
896dc7c780 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302028ed3921d1016ac2fa5754f06ebf0)
(cherry picked from commit 55142849db02a9191db0dd6f4e1401ff19ec242a)
(cherry picked from commit 082a707bae4bb97444a34c00eecd62975807388d)
 2018-01-17 16:36:03 +08:00
Brahma Reddy Battula
8e7ce0eb4c HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 
(cherry picked from commit 880b9d24ff7b5f350ec99bac9b0862009460b486)
 2018-01-16 16:29:13 +05:30
Rohith Sharma K S
8985fccbbc YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch. 
(cherry picked from commit d09058b2fd18803d12f0835fdf78aef5e0b99c90)
 2018-01-16 09:01:02 +05:30
Yiqun Lin
e5612f0389 HDFS-12972. RBF: Display mount table quota info in Web UI and admin command. Contributed by Yiqun Lin. 
(cherry picked from commit 9afb8025d6549f0ade0ae7d36f5e67cd20c500f4)
 2018-01-13 13:20:07 +08:00
Miklos Szegedi
09428b137f YARN-7705. Create the container log directory with correct sticky bit in C code. Contributed by Yufei Gu. 
(cherry picked from commit 2dcfc18)
 2018-01-12 17:44:31 -08:00
Wei-Chiu Chuang
2273499aef HDFS-11915. Sync rbw dir on the first hsync() to avoid file lost on power failure. Contributed by Vinayakumar B. 2018-01-12 10:01:57 -08:00
Sunil G
e7a8d6bd1e YARN-7727. Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector. Contributed by Prabhu Joseph. 
(cherry picked from commit 128d773a2315fa6baaa3a52b13c53c77e741b69c)
 2018-01-12 15:56:08 +05:30
Miklos Szegedi
35527db157 YARN-7590. Improve container-executor validation check. Contributed by Eric Yang. 
(cherry picked from commit bc285da)
 2018-01-11 10:30:15 -08:00
Akira Ajisaka
68b7908aa8
YARN-7735. Fix typo in YARN documentation. Contributed by Takanobu Asanuma. 
(cherry picked from commit fbbbf59c82658e18dad7e0e256613187b5b75d0f)
 2018-01-11 16:00:23 +09:00
Robert Kanter
ff67c68d3c YARN-7622. Allow fair-scheduler configuration on HDFS (gphillips via rkanter) 2018-01-10 15:03:33 -08:00
Miklos Szegedi
679ad2ced0 HADOOP-15060. TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky. Contributed by Miklos Szegedi. 
(cherry picked from commit 12d0645)
 2018-01-10 11:15:58 -08:00
Yiqun Lin
66d58d2b9a HDFS-12934. RBF: Federation supports global quota. Contributed by Yiqun Lin. 2018-01-10 19:48:47 +08:00
bibinchundatt
73478e3bbe YARN-7508. NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode. Contributed by Tao Yang. 
(cherry picked from commit b26e30ab1655a8cdf369862a5512db5c2a88ec46)
 2018-01-10 10:31:46 +05:30
Inigo Goiri
215a3d800a HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 
(cherry picked from commit d9006d8a4e34eae78dfa1cf3be50eeb81c2fc11f)
 2018-01-09 19:01:10 -08:00
Haibo Chen
a58034cdfd YARN-7716. metricsTimeStart and metricsTimeEnd should be all lower case in the doc. 
(cherry picked from commit f725b9e267c604fbcec09956e3a39caf04798809)
(cherry picked from commit cf816bc751e3b82be1f1931d6d0d5cbd77cb3e17)
 2018-01-09 09:00:23 -08:00
Jason Lowe
c892789896 YARN-6948. Invalid event: ATTEMPT_ADDED at FINAL_SAVING. Contributed by lujie 
(cherry picked from commit 90c86d6e090d668414681fcb5eb02add5d419b33)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2018-01-09 10:59:33 -06:00
Jason Lowe
f15c7e86e8 YARN-7663. RMAppImpl:Invalid event: START at KILLED. Contributed by lujie 
(cherry picked from commit d2c2d8c913dfc12585394a1168d4a92934a4cfb4)
 2018-01-09 10:01:06 -06:00
Ray Chiang
18c3982851 YARN-4227. Ignore expired containers from removed nodes in FairScheduler. (Wilfred Spiegelenburg via rchiang) 
(cherry picked from commit bc2d67d6c10619716ef7acce263f3269a86c3150)
 2018-01-08 16:23:29 -08:00
Manoj Govindassamy
eadd3cecf9 HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 2018-01-08 16:21:21 -08:00
Wei Yan
223bb935e2 HDFS-12945. Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods. Contributed by Wei Yan. 
(cherry picked from commit 2ee0d64aceed876f57f09eb9efe1872b6de98d2e)
 2018-01-08 15:23:57 -08:00
Steve Loughran
d2ceef0f20 HADOOP-15086. NativeAzureFileSystem file rename is not atomic. 
Contributed by Thomas Marquardt
(Backported to branch-2 via /HADOOP-15156)

(cherry picked from commit a35267b47abf62d2ab32bc6860d2080284323042)
 2018-01-08 15:13:50 +00:00
Rohith Sharma K S
37a8224101 YARN-7699. queueUsagePercentage is coming as INF for getApp REST api call. Contributed by Sunil G. 
(cherry picked from commit 281350674c70d65233363849a8b60e9b7dc965cf)
 2018-01-08 15:52:41 +05:30
Inigo Goiri
f7fedac6cd HDFS-12988. RBF: Mount table entries not properly updated in the local cache. Contributed by Inigo Goiri. 
(cherry picked from commit 83b513ac6d5448f3771b0b95f91e7aa7961ae2cc)
 2018-01-05 09:13:52 -08:00
Sunil G
cac8a8616e YARN-7619. Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user. Contributed by Eric Payne. 
(cherry picked from commit a79abbc03e75a5d2982de12587bd3b45169cc884)
 2018-01-05 14:51:29 +05:30
Jason Lowe
42964c2a66 YARN-7678. Ability to enable logging of container memory stats. Contributed by Jim Brennan 2018-01-04 10:21:40 -06:00
Jason Lowe
f2a1d5f3a1 MAPREDUCE-7028. Concurrent task progress updates causing NPE in Application Master. Contributed by Gergo Repas 
(cherry picked from commit fe35103591ece0209f8345aba5544313e45a073c)
 2018-01-03 11:08:51 -06:00
Rohith Sharma K S
e46142a8c2 YARN-7692. Skip validating priority acls while recovering applications. Contributed by Sunil G. 
(cherry picked from commit c9bf813c9a6c018d14f2bef49ba086ec0e60c761)
 2018-01-03 18:29:38 +05:30
Jitendra Pandey
cb60c05c3a HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 2018-01-01 21:51:48 -08:00
Xiao Chen
e20173d557 HADOOP-15149. CryptoOutputStream should implement StreamCapabilities. 
(cherry picked from commit 81127616c571b7cd25686e60a1105f96ca0626b7)
(cherry picked from commit 8c275c63ffd06db2cde26cbadc6a2ce4195c7b43)
 2017-12-29 13:57:39 -08:00
Xiao Chen
50b233d356 HDFS-9023. When NN is not able to identify DN for replication, reason behind it can be logged. 2017-12-28 11:55:24 -08:00
Rohith Sharma K S
0052353ecd YARN-7674. Update Timeline Reader web app address in UI2. Contributed by Sunil G. 
(cherry picked from commit 13ad7479b0e35a2c2d398e28c676871d9e672dc3)
 2017-12-20 22:30:29 +05:30
Sunil G
320f564bae YARN-7032. [ATSv2] NPE while starting hbase co-processor when HBase authorization is enabled. Contributed by Rohith Sharma K S. 
(cherry picked from commit d62932c3b2fcacc81dc1f5048cdeb60fb0d38504)
 2017-12-20 11:39:38 +05:30
Varun Saxena
d72d2440f4 YARN-7662. [ATSv2] Define new set of configurations for reader and collectors to bind (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit c0aeb666a4d43aac196569d9ec6768d62139d2b9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2017-12-19 22:51:56 +05:30
Yiqun Lin
df3212aad5 HDFS-12937. RBF: Add more unit tests for router admin commands. Contributed by Yiqun Lin. 
(cherry picked from commit e040c97b7743469f363eeae52c8abcf4fe7c65d5)
 2017-12-19 15:35:40 +08:00
Tsz-Wo Nicholas Sze
c4e58ea823 HDFS-12347. TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently. Contributed by Bharat Viswanadham 2017-12-19 10:04:00 +08:00
Jason Lowe
31a06baca0 YARN-7661. NodeManager metrics return wrong value after update node resource. Contributed by Yang Wang 
(cherry picked from commit 811fabdebe881248756c0165bf7667bfc22be9bb)
 2017-12-18 15:27:27 -06:00
Erik Krogen
da4e2f38e1 HDFS-12818. Support multiple storages in DataNodeCluster / SimulatedFSDataset. Contributed by Erik Krogen. 
(cherry picked from commit 94576b17fbc19c440efafb6c3322f53ec78a5b55)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
 2017-12-18 12:23:20 -08:00
Steve Loughran
1ef906e29e HADOOP-15123. KDiag tries to load krb5.conf from KRB5CCNAME instead of KRB5_CONFIG. 
Contributed by Vipin Rathor.
 2017-12-18 18:48:32 +00:00
Jason Lowe
93fdc3d935 HADOOP-15085. Output streams closed with IOUtils suppressing write errors. Contributed by Jim Brennan 2017-12-15 17:17:25 -06:00
Jason Lowe
0f8de8cd91 YARN-7595. Container launching code suppresses close exceptions after writes. Contributed by Jim Brennan 2017-12-15 15:54:56 -06:00
Jason Lowe
9ae8d1a8de HDFS-12881. Output streams closed with IOUtils suppressing write errors. Contributed by Ajay Kumar 2017-12-15 14:47:54 -06:00
Sammi Chen
1fee447bf2 HADOOP-15111. Aliyun OSS: backport HADOOP-14993 to branch-2. Contributed by Genmao Yu. 2017-12-15 14:28:39 +08:00
Yiqun Lin
24c4d2a7a2 HDFS-12895. RBF: Add ACL support for mount table. Contributed by Yiqun Lin. 
(cherry picked from commit ee028bfdf1c88a27cd925bed93ebb599a164dd2e)
 2017-12-15 14:13:54 +08:00
Sammi Chen
27807e4ccb HADOOP-15104. Aliyun OSS: change the default value of max error retry. Contributed by Jinhu Wu. 
(cherry picked from commit ce04340ec73617daff74378056a95c5d0cc0a790)
 2017-12-15 13:53:48 +08:00
Weiwei Yang
a85036aed1 YARN-7642. Add test case to verify context update after container promotion or demotion with or without auto update. Contributed by Weiwei Yang. 
(cherry picked from commit 89b6c482c1720a7f7ac86ce022c403825a086fa0)
(cherry picked from commit 108f8b8fa1dd556eb27f3fd9c2a700947cfe4aa0)
 2017-12-15 12:44:14 +08:00
Subru Krishnan
8d3d7fa1c5 YARN-7630. Fix AMRMToken rollover handling in AMRMProxy. Contributed by Botong Huang. 
(cherry picked from commit 17ba74be29193ac15474f73baaaf4e647a95078b)
 2017-12-14 14:07:44 -08:00
Kihwal Lee
08a50da95f HDFS-12907. Allow read-only access to reserved raw for non-superusers. Contributed by Rushabh S Shah. 2017-12-14 15:03:47 -06:00
Yiqun Lin
22bc62058b HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 
(cherry picked from commit 91c96bdf8eb9a06193b719186b527563091d7666)
 2017-12-14 16:03:34 +08:00