Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,347 Commits 326 Branches 371 Tags
Commit Graph

12347 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kihwal Lee
e229ffee64 HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 
(cherry picked from commit 8e3a992eccff26a7344c3f0e719898fa97706b8c)
 2017-07-21 09:22:28 -05:00
Brahma Reddy Battula
f65dc6ee95 HDFS-12177. NameNode exits due to setting BlockPlacementPolicy loglevel to Debug. Contributed by Jiandan Yang. 2017-07-21 21:13:20 +08:00
Konstantin V Shvachko
f3bf1a6eb9 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 
(cherry picked from commit 2a5a313539e211736fef12010918a60f9edad030)
 2017-07-20 16:09:56 -07:00
Kihwal Lee
5d58ba0e3d HADOOP-14659. UGI getShortUserName does not need to search the Subject. Contributed by Daryn Sharp. 
(cherry picked from commit b3269f7cc119cbc51b16118b918f345f970804bc)
 2017-07-20 16:22:11 -05:00
Jason Lowe
6e45c543cb YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/impl/pb/TestApplicationClientProtocolRecords.java
 2017-07-20 11:11:41 -05:00
Wangda Tan
a39617df63 YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda) 
Change-Id: I5951f0997547de7d2e4a30b4ad87ab0a59b3066a
 2017-07-19 15:29:45 -07:00
Anu Engineer
6ed569df21 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 413b23eb04eee24275257ab462133e0818f87449)
(cherry picked from commit e0297ffbc89e9f037d5f6a8c5874ce8794656e0c)
 2017-07-19 10:46:21 -07:00
Rohith Sharma K S
4daf5741e0 YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena. 
(cherry picked from commit e15c20edba1e9a23475ee6a4dfbadbdb8c1f668a)
 2017-07-17 14:52:06 -05:00
Brahma Reddy Battula
6039a2a55f HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b778887af59d96f1fac30cae14be1cabbdb74c8b)
 2017-07-15 11:12:56 +08:00
Xiao Chen
8b7d1df766 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-07-14 22:23:13 -07:00
Kihwal Lee
e28c74102d HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp. 
(cherry picked from commit e7d187a1b6a826edd5bd0f708184d48f3674d489)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
 2017-07-14 16:17:44 -05:00
Kihwal Lee
561b72970f HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 
(cherry picked from commit a47820d72a4c4aba857fd7325fa9ae9a95fe8faa)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2017-07-14 15:50:18 -05:00
Jason Lowe
1edc16ee54 YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt 
(cherry picked from commit a5ae5ac50e97cf829c41dcf01655cd9bd4d36a00)
 2017-07-14 15:06:31 -05:00
Ravi Prakash
d6228fbebd MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin 
(cherry picked from commit 43f0503286eccbc6bb8ae77584b635bfd0c48e50)
(cherry picked from commit 756a06814355465c85b9d66f262ee875dd86dbb7)
 2017-07-13 22:57:44 -07:00
Jason Lowe
8c69b040ac YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe 
(cherry picked from commit a77fb561efcea03060674d37ebadc15069766b6f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2017-07-13 18:00:05 -05:00
Jason Lowe
9cff53fed9 YARN-6654. RollingLevelDBTimelineStore backwards incompatible after fst upgrade. Contributed by Jonathan Eagles 
(cherry picked from commit 5f1ee72b0ebf0330417b7c0115083bc851923be4)
 2017-07-13 17:30:57 -05:00
Akira Ajisaka
c180005fcf MAPREDUCE-6697. Concurrent task limits should only be applied when necessary. Contributed by Nathan Roberts. 
(cherry picked from commit a5c0476a990ec1e7eb34ce2462a45aa52cc1350d)
 2017-07-13 13:30:33 -07:00
Masatake Iwasaki
7c8d4a1b3f HADOOP-14646. FileContextMainOperationsBaseTest#testListStatusFilterWithSomeMatches never runs. Contributed by Andras Bokor. 
(cherry picked from commit b61ab8573eb2f224481118004f620fe9f18db74b)
 2017-07-13 21:52:34 +09:00
Xiaoyu Yao
c9adca70e6 HDFS-11502. Datanode UI should display hostname based on JMX bean instead of window.location.hostname. Contributed by Jeffrey E Rodriguez and Yuanbo Liu. 
(cherry picked from commit e15e2713e1e344b14d63726639d1c83451921515)
 2017-07-12 16:39:22 -07:00
Jason Lowe
4f1dfcc118 YARN-6797. TimelineWriter does not fully consume the POST response. Contributed by Jason Lowe 
(cherry picked from commit 655110393bf90a7dac5d85f831ea225071f5f3c2)
 2017-07-12 15:54:49 -05:00
Junping Du
28be69ae6d MAPREDUCE-5621. mr-jobhistory-daemon.sh doesn't have to execute mkdir and chown all the time. Contributed by Shinichi Yamashita. 
(cherry picked from commit ea3dba37cd81a238ffcca4bb41dd69089187bc11)
 2017-07-12 12:54:35 -07:00
Akira Ajisaka
ebb66d2038
YARN-6809. Fix typo in ResourceManagerHA.md. Contributed by Yeliang Cang. 
(cherry picked from commit b628d0da5196faa5abf4c95dc7e1bb4eb67f6707)
(cherry picked from commit 3419381e988b489d4fde677f2ab191a955724371)
 2017-07-12 19:39:29 +09:00
Akira Ajisaka
d1fb4948d2
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 
(cherry picked from commit 9144fd9e9b5d84d71158451428341746a6567152)
(cherry picked from commit 2653cf4466ab99960ea65a937ea01fc4d4ca2f00)
 2017-07-12 11:44:15 +09:00
Xiao Chen
83eacccc17 HADOOP-14044. Synchronization issue in delegation token cancel functionality. Contributed by Hrishikesh Gadre. 
(cherry picked from commit ba75bc759334c8987e5f7dd4b21d025f0cccbde7)
(cherry picked from commit 05ed48b75a53df9ad4456ecddc981250006540ae)
 2017-07-10 13:48:12 -07:00
Naganarasimha
0f9f7bb0c6 YARN-6770. A small mistake in the example of TimelineClient. Contributed by Jinjiang Ling. 
(cherry picked from commit ba5b056e8a151fad9d7573c6e62cffc40cde7da3)
 2017-07-09 21:40:16 +05:30
Naganarasimha
ad70e929b2 YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundutt. 2017-07-09 17:34:47 +05:30
Junping Du
5ad710bc7a MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff602d48413556a1d046670e2003c71c2e)
(cherry picked from commit f823f9fd784fa4944178247a82df24fba03c2051)
 2017-07-07 13:32:21 -07:00
Wei-Chiu Chuang
d657c05177 HADOOP-14563. LoadBalancingKMSClientProvider#warmUpEncryptedKeys swallows IOException. Contributed by Rushabh S Shah. 
(cherry picked from commit 8153fe2bd35fb4df0b64f93ac0046e34d4807ac3)
(cherry picked from commit 730b21e9f90524673932411b48e1cb04dd3633d0)
 2017-07-07 06:14:32 -07:00
Eric Payne
8949a9d541 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G) 2017-07-06 10:53:47 -05:00
Jason Lowe
cd895d0586 YARN-6708. Nodemanager container crash after ext3 folder limit. Contributed by Bibin A Chundatt 
(cherry picked from commit 7576a688ea84aed7206321b1f03594e43a5f216e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestContainerLocalizer.java
 2017-07-06 09:53:13 -05:00
Jason Lowe
c8e6012f62 MAPREDUCE-6911. TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently. Contributed by Eric Badger 
(cherry picked from commit f0435db0891fd51b54241edf5bda42125490a06c)
 2017-07-06 09:24:01 -05:00
Mingliang Liu
db6e8ff747 HDFS-12089. Fix ambiguous NN retry log message in WebHDFS. Contributed by Eric Badger 
(cherry picked from commit 6436768baf1b2ac05f6786edcd76fd3a66c03eaa)
 2017-07-05 11:12:41 -07:00
Vinayakumar B
cd6ee04274 HADOOP-13414. Hide Jetty Server version header in HTTP responses. Contributed by Surendra Singth Lilhore. 
(cherry picked from commit 1e8301b215f79d48bbb58e2348945144d4c37997)
 2017-07-05 16:07:18 +05:30
Akira Ajisaka
71f49406f2
MAPREDUCE-6909. LocalJobRunner fails when run on a node from multiple users. Contributed by Jason Lowe. 
(cherry picked from commit ac333155fbcbbc50477869fb29c8da124e5968c0)
 2017-07-04 15:26:46 +09:00
Akira Ajisaka
9f9d554edf
HDFS-12078. Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml. Contributed by Weiwei Yang. 
(cherry picked from commit 186650d21d482e2f3bc4523ae989ebe76081b0e3)
(cherry picked from commit 063f2132ef54436e8a537d437e7a39256a74214f)
 2017-07-04 14:53:02 +09:00
Naganarasimha
acc504dc58 MAPREDUCE-6905. HADOOP_JOB_HISTORY_OPTS should be HADOOP_JOB_HISTORYSERVER_OPTS in mapred-config.sh. Contributed by LiXin Ge. 
(cherry picked from commit bf1f59937dd5d860c9ed7fefce203d6a9f645182)
 2017-07-02 16:18:30 +05:30
Naganarasimha
69480ba686 YARN-6681. Eliminate double-copy of child queues in canAssignToThisQueue. Contributed by Daryn Sharp. 2017-07-01 12:58:36 +05:30
Wei-Chiu Chuang
8027246306 HDFS-12042. Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption. Contributed by Misha Dmitriev. 
(cherry picked from commit bcba844d1144cc334e2babbc34c9d42eac1c203a)
(cherry picked from commit 94bc5cdbb3d0fd9133a8cab9d3daac6798e8d8dd)
 2017-06-30 11:43:41 -07:00
bibinchundatt
f6fcbf426f YARN-6749. TestAppSchedulingInfo.testPriorityAccounting fails consistently. Contributed by Naganarasimha G R 2017-06-30 21:52:16 +05:30
John Zhuge
2956c584c3 HADOOP-14024. KMS JMX endpoint throws ClassNotFoundException. Contributed by John Zhuge. 
(cherry picked from commit 1272ee153b1a1578abf822b2141d8336d9c5652f)
 2017-06-29 16:27:50 -07:00
Wei-Chiu Chuang
7c076f917f HDFS-11881. NameNode consumes a lot of memory for snapshot diff report generation. Contributed by Manoj Govindassamy. 
(cherry picked from commit 16c8dbde574f49827fde5ee9add1861ee65d4645)
(cherry picked from commit 6917c19167060df7879b0e0ead4db1ded62714b3)
 2017-06-29 06:44:37 -07:00
Naganarasimha
29c68ccc6f YARN-6467. CSQueueMetrics needs to update the current metrics for default partition only. Contributed by Manikandan R. 2017-06-29 00:25:32 +05:30
Xiao Chen
ddd3e7a4db HADOOP-14515. Addendum. Specifically configure zookeeper-related log levels in KMS log4j. 
(cherry picked from commit aff7a3f8eddd55501366a43d2f44f531c6e8ef48)
 2017-06-27 23:49:23 -07:00
Jason Lowe
486dc3f9eb YARN-6738. LevelDBCacheTimelineStore should reuse ObjectMapper instances. Contributed by Zoltan Haindrich 
(cherry picked from commit 63ce1593c5b78eb172773e7498d9c321debe81e8)
 2017-06-27 17:14:30 -05:00
Kihwal Lee
bdbf737ef3 Addendum for HADOOP-14146. Use commons Base64 instead of jdk8 Base64. 
(cherry picked from commit 07193459091cfaf372634637096485154d39a176)
 2017-06-23 13:28:16 -05:00
Kihwal Lee
fa695817e5 Addendum for HADOOP-14146 jdk7 support. 
(cherry picked from commit 4c6184b41cfdab589a2fea4fb7665e28d044c681)
 2017-06-23 10:00:26 -05:00
Daryn Sharp
81eb06b553 HADOOP-14146. KerberosAuthenticationHandler should authenticate with SPN in AP-REQ. Contributed by Daryn Sharp 2017-06-21 11:48:05 -05:00
Kihwal Lee
a560104fab HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee. 
(cherry picked from commit 8f533ca76b3f3592dc8ec6e318382a54becf10f5)
 2017-06-20 12:12:33 -05:00
Kihwal Lee
e87bf17129 HDFS-9754. Avoid unnecessary getBlockCollection calls in BlockManager. Contributed by Jing Zhao. 
(cherry picked from commit 972782d9568e0849484c027f27c1638ba50ec56e)
(cherry picked from commit f8c9c0ff0e2d977fc0f69bde4cdbb03371c0bac4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetadataConsistency.java
 2017-06-20 12:10:43 -05:00
Kihwal Lee
1f9255672d Revert "HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee." 
This reverts commit 40e3cd24b068d08be8cf05668290b908f6b43c11.
 2017-06-20 09:22:25 -05:00