Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,338 Commits 326 Branches 371 Tags
Commit Graph

12338 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Akira Ajisaka
2b92c1be04
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka. 
(cherry picked from commit 588c190afd49bdbd5708f7805bf6c68f09fee142)
(cherry picked from commit a28a3dc8f2709738c28f70fc394f51d3b1266296)
 2017-08-16 14:07:23 +09:00
Ravi Prakash
a611922de4 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652310442cb5339aabbbb269f10dfa838)
(cherry picked from commit 895a35eac0422c1845ebe3c5bed1ac2bda1ac807)
 2017-08-15 15:46:22 -07:00
Jason Lowe
87c36d8b8b YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894169425bff054364a1ab4c48b347fb9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
 2017-08-14 15:41:54 -05:00
Jason Lowe
00e51a9cd3 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d6f364a82dd32485d9ef6d987380ce3)
 2017-08-14 14:48:48 -05:00
John Zhuge
de50bb6365 HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 
(cherry picked from commit 7769e9614956283a86eda9e4e69aaa592c0ca960)
 2017-08-13 00:32:25 -07:00
John Zhuge
e967744558 HADOOP-14260. Configuration.dumpConfiguration should redact sensitive information. Contributed by John Zhuge. 
(cherry picked from commit 582648befaf9908159f937d2cc8f549583a3483e)
 2017-08-11 11:02:18 -07:00
John Zhuge
407bbe39fa HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang 
(cherry picked from commit 00160f71b6d98244fcb1cb58b2db9fc24f1cd672)
 2017-08-11 11:02:14 -07:00
John Zhuge
8fdcf86e64 HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang 
(cherry picked from commit 59d59667a8b1d3fb4a744a41774b2397fd91cbb3)
 2017-08-11 11:02:09 -07:00
Kihwal Lee
a54c3437af HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 2017-08-09 16:51:19 -05:00
Kihwal Lee
639380efff HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by Vinayakumar B. 
(cherry picked from commit 74bcc8d1f213ffd18056b7d0590d0cd346abaff9)
 2017-08-09 09:33:07 -05:00
Naganarasimha
49d459a3ed YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash. 
(cherry picked from commit 4c431a694059e40e78365b02a1497a6c7e479a70)
(cherry picked from commit a7cdd39234c3a236cd9b85853636d42dc8c850a4)
 2017-08-08 15:51:31 -07:00
Haohui Mai
6801b898d5 HDFS-10326. Disable setting tcp socket send/receive buffers for write pipelines. Contributed by Daryn Sharp. 2017-08-08 15:28:39 -07:00
Jason Lowe
ea67e1f266 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit 735fce5bec17f4e1799daf922625c475cf588114)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
 2017-08-08 15:03:34 -05:00
Jian He
bc87a293d9 YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du 2017-08-08 11:11:46 -07:00
Andrew Wang
22ef7c6bf2 HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:45:47 -07:00
Eric Payne
8943cf9d30 YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 8ce8672b6b551dacb9467924fc70f88790f5891f)
 2017-08-02 13:07:39 -05:00
Eric Payne
d5150e3148 YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe. 
(cherry picked from commit 48899134d2a77935a821072b5388ab1b1b7b399c)
 2017-08-02 12:03:53 -05:00
Jian He
e84a3f43a1 YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 2017-08-02 00:09:25 -07:00
Masatake Iwasaki
a01d92156d HADOOP-14702. Fix formatting issue and regression caused by conversion from APT to Markdown. Contributed by Doris Gu. 
(cherry picked from commit d9a07b39046c323d430b98eff6603eaf58e18372)

Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
 2017-08-02 06:33:35 +00:00
Manoj Govindassamy
fc66e76a4a HDFS-10480. Add an admin command to list currently open files. 2017-08-01 17:37:45 -07:00
Junping Du
168d8e0c04 HADOOP-14683. FileStatus.compareTo binary compatible issue. Contributed by Akira Ajisaka. 
(cherry picked from commit cf6794ebe93906c406a54acc5c8854dc600a9a7f)
(cherry picked from commit 60c10ea7b47f185997b215aea7a3f375ed713ac5)
 2017-08-01 17:19:18 -07:00
Wangda Tan
f2f09e2bb0 YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. (wangda) 
Change-Id: Ie8c5145f449582253dcf8c0f6f388e3660ab1c6b
 2017-08-01 15:24:19 -07:00
Jason Lowe
0020a8e7d9 YARN-6628. Unexpected jackson-core-2.2.3 dependency introduced. Contributed by Jonathan Eagles 
(cherry picked from commit b9640c02bf5072014299124cfd44880d7361a5a7)
 2017-07-31 16:39:05 -05:00
Akira Ajisaka
55c4eb3cf4
HADOOP-14677. mvn clean compile fails. Contributed by Andras Bokor. 
(cherry picked from commit 0fd6d0f2d361536b2baf859ddbb082eb2eadcfcf)
(cherry picked from commit b19415f1a18f4a2a121408f777fd9eaaeb52d074)
 2017-07-31 17:29:31 +09:00
Akira Ajisaka
d57784cbda
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout. 
(cherry picked from commit f8bed5e9a7d1ece127fb3da123bbfc26ada0016f)
(cherry picked from commit 98c3544e9407e3e1dfae9806d4a8181508952bed)
 2017-07-31 12:34:25 +09:00
Brahma Reddy Battula
90f3108b8e HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 8bfb9971cad1a83da852de4d12f1de4197d25d21)
 2017-07-30 14:02:25 +08:00
John Zhuge
a1682876c9 HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge. 
(cherry picked from commit f735ad1b67ed82d9b11b1afd7ae39035a6aed18b)
 2017-07-28 09:55:14 -07:00
Sunil G
e5ada740ff YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne. 2017-07-28 20:08:20 +05:30
Brahma Reddy Battula
9a168ae884 HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694fae3f814ab4e7f3c172da1df0e0e353)
 2017-07-27 12:37:36 -07:00
Kihwal Lee
c586b7ab7d HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit e407449ddb52c36b74358f22db498424e82ec4a7)
(cherry picked from commit 2d6be7ea2311deb13cbad738b4c8fc68ad25ab2a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestRMAdminCLI.java
 2017-07-26 14:15:39 -05:00
Kihwal Lee
e15a08fc9a HADOOP-14578. Bind IPC connections to kerberos UPN host for proxy users. 
Contributed by Daryn Sharp.
 2017-07-26 13:29:07 -05:00
Kihwal Lee
38c56bbb46 HDFS-12171. Reduce IIP object allocations for inode lookup. Contributed by Daryn Sharp. 
(cherry picked from commit 3d8c7d701b8f22b3b4d67f587f7ff2318c60c541)
 2017-07-25 11:08:57 -05:00
Jitendra Pandey
c0e294ffc8 HADOOP-10829. Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe. Contributed by Benoy Antony and Rakesh R. 
(cherry picked from commit b82485d6fed2194bf0dc2eedfab7e226e30a7cf0)
 2017-07-25 09:04:49 -07:00
Kihwal Lee
7be52c9c79 HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp. 
(cherry picked from commit 1a79dcfc457969d6a6c08ffffe4152fd7638e48a)
 2017-07-25 10:33:09 -05:00
Jason Lowe
db0bc8fafa YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 
(cherry picked from commit 24853bf32a045b8f029fb136edca2af03836c8d5)
 2017-07-24 10:25:04 -05:00
Allen Wittenauer
02ef086824
HADOOP-13178. TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X (Kai Sasaki via aw) 
(cherry picked from commit d8c1fd1944160ad5b5d093731ae987d701802321)
(cherry picked from commit daa768534409f08a83759634b5c3966e32e6f170)
 2017-07-24 12:03:40 +09:00
Brahma Reddy Battula
a1421de70e HADOOP-13867. FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementations. Contributed By Vinayakumar B. 
(cherry picked from commit 0ef796174ecb5383f79cfecfcbfc4f309d093cd7)
(cherry picked from commit 6f445408022f210e01f826499d447d4e7792b429)
 2017-07-23 16:09:29 +08:00
Brahma Reddy Battula
587d47cfec HDFS-8312. Trash does not descent into child directories to check for permissions. Contributed By Weiwei Yang via Eric Yang. 
(cherry picked from commit 820496fbbc00ede0484bec5511c6b12913e97356)
 2017-07-23 16:09:10 +08:00
Jason Lowe
4fc070ac59 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 
(cherry picked from commit c58a6d53c58209a8f78ff64e04e9112933489fb5)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-07-21 14:46:18 -05:00
Jason Lowe
293ac4c18b YARN-4594. container-executor fails to remove directory tree when chmod required. Contributed by Colin Patrick McCabe 
(cherry picked from commit fa328e2d39eda1c479389b99a5c121e640a1e0ad)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-07-21 14:33:34 -05:00
Eric Payne
95be843851 YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 
(cherry picked from commit a16ab2be91e29f0aa4ad3488d35977cb5d1e1f5c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java
 2017-07-21 10:32:35 -05:00
Kihwal Lee
e229ffee64 HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 
(cherry picked from commit 8e3a992eccff26a7344c3f0e719898fa97706b8c)
 2017-07-21 09:22:28 -05:00
Brahma Reddy Battula
f65dc6ee95 HDFS-12177. NameNode exits due to setting BlockPlacementPolicy loglevel to Debug. Contributed by Jiandan Yang. 2017-07-21 21:13:20 +08:00
Konstantin V Shvachko
f3bf1a6eb9 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 
(cherry picked from commit 2a5a313539e211736fef12010918a60f9edad030)
 2017-07-20 16:09:56 -07:00
Kihwal Lee
5d58ba0e3d HADOOP-14659. UGI getShortUserName does not need to search the Subject. Contributed by Daryn Sharp. 
(cherry picked from commit b3269f7cc119cbc51b16118b918f345f970804bc)
 2017-07-20 16:22:11 -05:00
Jason Lowe
6e45c543cb YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/impl/pb/TestApplicationClientProtocolRecords.java
 2017-07-20 11:11:41 -05:00
Wangda Tan
a39617df63 YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda) 
Change-Id: I5951f0997547de7d2e4a30b4ad87ab0a59b3066a
 2017-07-19 15:29:45 -07:00
Anu Engineer
6ed569df21 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 413b23eb04eee24275257ab462133e0818f87449)
(cherry picked from commit e0297ffbc89e9f037d5f6a8c5874ce8794656e0c)
 2017-07-19 10:46:21 -07:00
Rohith Sharma K S
4daf5741e0 YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena. 
(cherry picked from commit e15c20edba1e9a23475ee6a4dfbadbdb8c1f668a)
 2017-07-17 14:52:06 -05:00
Brahma Reddy Battula
6039a2a55f HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b778887af59d96f1fac30cae14be1cabbdb74c8b)
 2017-07-15 11:12:56 +08:00