Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,472 Commits 326 Branches 371 Tags
Commit Graph

13472 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Subru Krishnan
ad46a90872 HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru). 
(cherry picked from commit 75dd866bfb8b63cb9f13179d4365b05c48e0907d)
 2017-08-15 16:54:48 -07:00
Ravi Prakash
895a35eac0 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652310442cb5339aabbbb269f10dfa838)
 2017-08-15 15:45:37 -07:00
Steve Loughran
b30522c59e
HADOOP-14662. Update azure-storage sdk to version 5.4.0. 
Contributed by Thomas Marquard
 2017-08-15 16:27:31 +01:00
Arun Suresh
7b22df3da6 YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 4d7be1d8575e9254c59d41460960708e3718503a)
 2017-08-14 19:57:34 -07:00
Arpit Agarwal
bdc425057e HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru. 2017-08-14 15:53:43 -07:00
Jason Lowe
56d0e757c7 YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894169425bff054364a1ab4c48b347fb9)
 2017-08-14 15:34:50 -05:00
Jason Lowe
e5ba40cc98 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d6f364a82dd32485d9ef6d987380ce3)
 2017-08-14 14:41:38 -05:00
Subru Krishnan
a70efb6138 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-14 11:03:50 -07:00
Carlo Curino
394573780b YARN-6687. Validate that the duration of the periodic reservation is less than the periodicity. (subru via curino) 
(cherry picked from commit 28d97b79b69bb2be02d9320105e155eeed6f9e78)
 2017-08-14 11:01:56 -07:00
Jian He
630ada1b4e YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang 2017-08-14 10:56:39 -07:00
John Zhuge
0efc590e6d HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 
(cherry picked from commit 7769e9614956283a86eda9e4e69aaa592c0ca960)
 2017-08-13 00:27:33 -07:00
John Zhuge
ed4d6aa2c1 HDFS-11303. Hedged read might hang infinitely if read data from all DN failed . Contributed by Chen Zhang, Wei-chiu Chuang, and John Zhuge. 
(cherry picked from commit 8b242f09a61a7536d2422546bfa6c2aaf1d57ed6)
 2017-08-11 21:55:23 -07:00
Daniel Templeton
7d3cea232d Revert "YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator" 
This reverts commit ce0cdc50d9700118139e7907dede274d2db93fda.
 2017-08-11 20:52:07 -07:00
Daniel Templeton
ce0cdc50d9 YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator 
(Contributed by Larry Lo via Daniel Templeton)

(cherry picked from commit 0996acde6c325667aa19ae0740eb6b40bf4a682a)
 2017-08-11 14:51:42 -07:00
Daniel Templeton
11e3673748 YARN-6967. Limit application attempt's diagnostic message size thoroughly 
(Contributed by Chengbing Liu via Daniel Templeton)

(cherry picked from commit 65364defb4a633ca20b39ebc38cd9c0db63a5835)
 2017-08-11 14:31:11 -07:00
Daniel Templeton
2566e5fcf7 YARN-6884. AllocationFileLoaderService.loadQueue() has an if without braces 
(Contributed by weiyuan via Daniel Templeton)

(cherry picked from commit c7680d4cc4d9302a5b5efcf2467bd32ecea99585)
 2017-08-11 14:23:17 -07:00
Karthik Kambatla
f6c74ac5ea YARN-5927. BaseContainerManagerTest::waitForNMContainerState timeout accounting is not accurate. (Kai Sasaki via kasha) 
(cherry picked from commit 8c4b6d16a526610a03ccc85665744ad071e37400)
 2017-08-11 12:18:29 -07:00
John Zhuge
e208a4e7ee HADOOP-14260. Configuration.dumpConfiguration should redact sensitive information. Contributed by John Zhuge. 
(cherry picked from commit 582648befaf9908159f937d2cc8f549583a3483e)
 2017-08-11 10:54:41 -07:00
John Zhuge
76c4aee2e3 HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang 
(cherry picked from commit 00160f71b6d98244fcb1cb58b2db9fc24f1cd672)
 2017-08-11 10:54:30 -07:00
John Zhuge
9ba3268fa3 HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang 
(cherry picked from commit 59d59667a8b1d3fb4a744a41774b2397fd91cbb3)
 2017-08-11 10:54:25 -07:00
Steve Loughran
9c6e8ece98
HADOOP-14754. TestCommonConfigurationFields failed: core-default.xml has 2 wasb properties missing in classes. 
Contributed by John Zhuge.

(cherry picked from commit d964062f66c0772f4b1a029bfcdff921fbaaf91c)
 2017-08-11 10:58:26 +01:00
Wei-Chiu Chuang
304cc1a856 HADOOP-14743. CompositeGroupsMapping should not swallow exceptions. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit a8b75466b21edfe8b12beb4420492817f0e03147)
 2017-08-10 09:36:21 -07:00
Steve Loughran
96befde480
HADOOP-14183. Remove service loader config file for wasb fs. 
Contributed by Esfandiar Manii.

(cherry picked from commit 54356b1e8366a23fff1bb45601efffc743306efc)
 2017-08-10 16:49:51 +01:00
Steve Loughran
bbf8c4bacb
HADOOP-14715. TestWasbRemoteCallHelper failing. 
Contributed by Esfandiar Manii.
 2017-08-10 16:49:19 +01:00
Akira Ajisaka
1f80e05eb4
YARN-6958. Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice. Contributed by Yeliang Cang. 2017-08-10 22:14:44 +09:00
Ravi Prakash
c556cdaaf4 MAPREDUCE-6923. Optimize MapReduce Shuffle I/O for small partitions. Contributed by Robert Schmidtke. 
(cherry picked from commit ac7d0604bc73c0925eff240ad9837e14719d57b7)
 2017-08-09 15:40:57 -07:00
Kihwal Lee
306abf4980 HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 
(cherry picked from commit b5c02f95b5a2fcb8931d4a86f8192caa18009ea9)
 2017-08-09 16:48:55 -05:00
Jian He
8b7590057e YARN-6804. Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi 2017-08-09 11:12:38 -07:00
Kihwal Lee
74bcc8d1f2 HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by Vinayakumar B. 2017-08-09 09:31:53 -05:00
Haohui Mai
3e2753daa1 HADOOP-14598. Blacklist Http/HttpsFileSystem in FsUrlStreamHandlerFactory. Contributed by Steve Loughran. 2017-08-08 16:33:54 -07:00
Haohui Mai
21d76f710e HDFS-10326. Disable setting tcp socket send/receive buffers for write pipelines. Contributed by Daryn Sharp. 2017-08-08 14:58:44 -07:00
Subru Krishnan
e9ec1e8f87 YARN-6897. Refactoring RMWebServices by moving some util methods to RMWebAppUtil. (Giovanni Matteo Fumarola via Subru). 2017-08-08 14:01:22 -07:00
Daniel Templeton
c5b3d80a4b YARN-6879. TestLeafQueue.testDRFUserLimits() has commented out code 
(Contributed by Angela Wang via Daniel Templeton)

(cherry picked from commit e0c24145d2c2a7d2cf10864fb4800cb1dcbc2977)
 2017-08-08 13:36:49 -07:00
Jason Lowe
50dfef7275 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit 735fce5bec17f4e1799daf922625c475cf588114)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
 2017-08-08 14:55:55 -05:00
Junping Du
e9c21d14b4 YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S. 2017-08-08 12:45:55 -07:00
Daniel Templeton
675d368950 YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path 
(Contributed by Miklos Szegedi via Daniel Templeton)

(cherry picked from commit 47b145b9b4e81d781891abce8a6638f0b436acc4)
 2017-08-08 11:12:34 -07:00
Jian He
ad1567c487 YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du 2017-08-08 11:10:29 -07:00
Yiqun Lin
817c0f4564 HDFS-12198. Document missing namenode metrics that were added recently. Contributed by Yiqun Lin. 2017-08-08 19:30:03 +08:00
Akira Ajisaka
9fd5a03611
YARN-6873. Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice. Contributed by Yeliang Cang. 2017-08-08 19:28:33 +09:00
Arun Suresh
3f50800d41 YARN-6920. Fix resource leak that happens during container re-initialization. (asuresh) 
(cherry picked from commit 8d3fd81980275fa81e7a5539b1751f38a63b6911)
 2017-08-07 19:01:13 -07:00
Arpit Agarwal
2906c494c5 HDFS-12264. DataNode uses a deprecated method IoUtils#cleanup. Contributed by Ajay Yadav. 2017-08-07 15:33:36 -07:00
Wangda Tan
e18415a350 YARN-4161. Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration. (Wei Yan via wangda) 
Change-Id: Ic441ae4e0bf72e7232411eb54243ec143d5fd0d3
(cherry picked from commit adb84f34db7e1cdcd72aa8e3deb464c48da9e353)
 2017-08-07 11:34:45 -07:00
Xiao Chen
050c50e3c5 HADOOP-14727. Socket not closed properly when reading Configurations with BlockReaderRemote. Contributed by Jonathan Eagles. 
(cherry picked from commit 9a755e6da7158783789f55796644578330653efb)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
 2017-08-07 10:26:10 -07:00
Akira Ajisaka
cfdf29785b
YARN-6957. Moving logging APIs over to slf4j in hadoop-yarn-server-sharedcachemanager. Contributed by Yeliang Cang. 
(cherry picked from commit b0fbf1796585900a37dc4d6a271c5b5b32e9a9da)
 2017-08-07 19:26:18 +09:00
Sunil G
d15bac141e YARN-6951. Fix debug log when Resource Handler chain is enabled. Contributed by Yang Wang. 
(cherry picked from commit 46b7054fa7eae9129c21c9f3dc70acff46bfdc41)
 2017-08-07 13:17:04 +05:30
Steve Loughran
7f7ab0302f
HADOOP-14722. Azure: BlockBlobInputStream position incorrect after seek. 
Contributed by Thomas Marquardt

(cherry picked from commit d91b7a8451489f97bdde928cea774764155cfe03)
 2017-08-06 20:30:40 +01:00
Yufei Gu
b9549e108e YARN-6802. Add max AM resource and AM resource usage to leaf queue view in FairScheduler WebUI. (YunFan Zhou via Yufei Gu) 2017-08-04 17:23:44 -07:00
Carlo Curino
7654b52167 YARN-6634. [API] Refactor ResourceManager WebServices to make API explicit (Contributed by Giovanni Matteo Fumarola via curino). 
(Edited cherry-pick of 547cb53e8ae2f19b55f9de7670b91b009d8f3bdd).
 2017-08-04 17:04:20 -07:00
Arun Suresh
4cfd248a72 YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh) 
(cherry picked from commit 077fcf6a96e420e7f36350931722b8603d010cf1)
 2017-08-04 16:50:21 -07:00
Arun Suresh
043b7d133e YARN-6776. Refactor ApplicaitonMasterService to move actual processing logic to a separate class. (asuresh) 
(cherry picked from commit 5496a34c0cb2b1a83cfa6b0aba5a77b05ff2d8f0)
 2017-08-04 16:43:35 -07:00