Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,352 Commits 326 Branches 371 Tags
Commit Graph

12352 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Lowe
dfdda969ad YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:33:00 -05:00
Kihwal Lee
b0e951725f HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5a83ffa396089972e23c533eca33c9cba231c45a)
 2017-08-25 15:04:37 -05:00
Arpit Agarwal
b07eef43ae HDFS-12319. Addendum commit for missed changes. 2017-08-25 12:02:59 -07:00
Arpit Agarwal
22fb9bb843 HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 11:10:25 -07:00
Colin Patrick Mccabe
945c916eec HDFS-7764. DirectoryScanner shouldn't abort the scan if one directory had an error (Rakesh R via cmccabe) 
(cherry picked from commit f67149ab08bb49381def6c535ab4c4610e0a4221)
 2017-08-25 11:02:05 -07:00
Jason Lowe
2d548759ad YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169dc3da9df0158ba6f44a9bc8576e217)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java
 2017-08-25 10:16:58 -05:00
John Zhuge
832b0b82e2 HADOOP-14251. Credential provider should handle property key deprecation. Contributed by John Zhuge. 
(cherry picked from commit 7e6463d2fb5f9383d88baec290461868cf476e4c)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
 2017-08-23 12:18:36 -07:00
John Zhuge
bc1c8f3e5a HDFS-11738. Hedged pread takes more time when block moved from initial locations. Contributed by Vinayakumar B. 
(cherry picked from commit b6bfb2fcb2391d51b8de97c01c1290880779132e)
(cherry picked from commit c54c3500ed720e39926f4cc7ca3db0cb4854ff55)
 2017-08-23 20:05:54 +05:30
Jason Lowe
f6892f5821 HADOOP-14687. AuthenticatedURL will reuse bad/expired session cookies. Contributed by Daryn Sharp 2017-08-22 16:55:14 -05:00
Jason Lowe
ff9cbcbbf4 YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger 2017-08-22 16:32:45 -05:00
Jason Lowe
5cb8a92bdb YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 
(cherry picked from commit 657dd59cc840bce3426170baf68f9d46842f4f53)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestRMDelegationTokens.java
 2017-08-22 13:28:44 -05:00
Jason Lowe
0b64773678 YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 
(cherry picked from commit 3efcd51c3b3eb667d83e08b500bb7a7ea559fabe)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
 2017-08-22 13:02:52 -05:00
Jian He
3f735ad640 YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang 2017-08-18 13:08:54 -07:00
Jason Lowe
2810e6ab9c YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 
(cherry picked from commit 14553061be0a341df3e628dcaf06717b4630b05e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMProxy.java
 2017-08-16 13:14:25 -05:00
Akira Ajisaka
2b92c1be04
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka. 
(cherry picked from commit 588c190afd49bdbd5708f7805bf6c68f09fee142)
(cherry picked from commit a28a3dc8f2709738c28f70fc394f51d3b1266296)
 2017-08-16 14:07:23 +09:00
Ravi Prakash
a611922de4 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652310442cb5339aabbbb269f10dfa838)
(cherry picked from commit 895a35eac0422c1845ebe3c5bed1ac2bda1ac807)
 2017-08-15 15:46:22 -07:00
Jason Lowe
87c36d8b8b YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894169425bff054364a1ab4c48b347fb9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
 2017-08-14 15:41:54 -05:00
Jason Lowe
00e51a9cd3 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d6f364a82dd32485d9ef6d987380ce3)
 2017-08-14 14:48:48 -05:00
John Zhuge
de50bb6365 HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 
(cherry picked from commit 7769e9614956283a86eda9e4e69aaa592c0ca960)
 2017-08-13 00:32:25 -07:00
John Zhuge
e967744558 HADOOP-14260. Configuration.dumpConfiguration should redact sensitive information. Contributed by John Zhuge. 
(cherry picked from commit 582648befaf9908159f937d2cc8f549583a3483e)
 2017-08-11 11:02:18 -07:00
John Zhuge
407bbe39fa HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang 
(cherry picked from commit 00160f71b6d98244fcb1cb58b2db9fc24f1cd672)
 2017-08-11 11:02:14 -07:00
John Zhuge
8fdcf86e64 HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang 
(cherry picked from commit 59d59667a8b1d3fb4a744a41774b2397fd91cbb3)
 2017-08-11 11:02:09 -07:00
Kihwal Lee
a54c3437af HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 2017-08-09 16:51:19 -05:00
Kihwal Lee
639380efff HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by Vinayakumar B. 
(cherry picked from commit 74bcc8d1f213ffd18056b7d0590d0cd346abaff9)
 2017-08-09 09:33:07 -05:00
Naganarasimha
49d459a3ed YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash. 
(cherry picked from commit 4c431a694059e40e78365b02a1497a6c7e479a70)
(cherry picked from commit a7cdd39234c3a236cd9b85853636d42dc8c850a4)
 2017-08-08 15:51:31 -07:00
Haohui Mai
6801b898d5 HDFS-10326. Disable setting tcp socket send/receive buffers for write pipelines. Contributed by Daryn Sharp. 2017-08-08 15:28:39 -07:00
Jason Lowe
ea67e1f266 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit 735fce5bec17f4e1799daf922625c475cf588114)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
 2017-08-08 15:03:34 -05:00
Jian He
bc87a293d9 YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du 2017-08-08 11:11:46 -07:00
Andrew Wang
22ef7c6bf2 HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:45:47 -07:00
Eric Payne
8943cf9d30 YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 8ce8672b6b551dacb9467924fc70f88790f5891f)
 2017-08-02 13:07:39 -05:00
Eric Payne
d5150e3148 YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe. 
(cherry picked from commit 48899134d2a77935a821072b5388ab1b1b7b399c)
 2017-08-02 12:03:53 -05:00
Jian He
e84a3f43a1 YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 2017-08-02 00:09:25 -07:00
Masatake Iwasaki
a01d92156d HADOOP-14702. Fix formatting issue and regression caused by conversion from APT to Markdown. Contributed by Doris Gu. 
(cherry picked from commit d9a07b39046c323d430b98eff6603eaf58e18372)

Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
 2017-08-02 06:33:35 +00:00
Manoj Govindassamy
fc66e76a4a HDFS-10480. Add an admin command to list currently open files. 2017-08-01 17:37:45 -07:00
Junping Du
168d8e0c04 HADOOP-14683. FileStatus.compareTo binary compatible issue. Contributed by Akira Ajisaka. 
(cherry picked from commit cf6794ebe93906c406a54acc5c8854dc600a9a7f)
(cherry picked from commit 60c10ea7b47f185997b215aea7a3f375ed713ac5)
 2017-08-01 17:19:18 -07:00
Wangda Tan
f2f09e2bb0 YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. (wangda) 
Change-Id: Ie8c5145f449582253dcf8c0f6f388e3660ab1c6b
 2017-08-01 15:24:19 -07:00
Jason Lowe
0020a8e7d9 YARN-6628. Unexpected jackson-core-2.2.3 dependency introduced. Contributed by Jonathan Eagles 
(cherry picked from commit b9640c02bf5072014299124cfd44880d7361a5a7)
 2017-07-31 16:39:05 -05:00
Akira Ajisaka
55c4eb3cf4
HADOOP-14677. mvn clean compile fails. Contributed by Andras Bokor. 
(cherry picked from commit 0fd6d0f2d361536b2baf859ddbb082eb2eadcfcf)
(cherry picked from commit b19415f1a18f4a2a121408f777fd9eaaeb52d074)
 2017-07-31 17:29:31 +09:00
Akira Ajisaka
d57784cbda
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout. 
(cherry picked from commit f8bed5e9a7d1ece127fb3da123bbfc26ada0016f)
(cherry picked from commit 98c3544e9407e3e1dfae9806d4a8181508952bed)
 2017-07-31 12:34:25 +09:00
Brahma Reddy Battula
90f3108b8e HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 8bfb9971cad1a83da852de4d12f1de4197d25d21)
 2017-07-30 14:02:25 +08:00
John Zhuge
a1682876c9 HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge. 
(cherry picked from commit f735ad1b67ed82d9b11b1afd7ae39035a6aed18b)
 2017-07-28 09:55:14 -07:00
Sunil G
e5ada740ff YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne. 2017-07-28 20:08:20 +05:30
Brahma Reddy Battula
9a168ae884 HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694fae3f814ab4e7f3c172da1df0e0e353)
 2017-07-27 12:37:36 -07:00
Kihwal Lee
c586b7ab7d HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit e407449ddb52c36b74358f22db498424e82ec4a7)
(cherry picked from commit 2d6be7ea2311deb13cbad738b4c8fc68ad25ab2a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestRMAdminCLI.java
 2017-07-26 14:15:39 -05:00
Kihwal Lee
e15a08fc9a HADOOP-14578. Bind IPC connections to kerberos UPN host for proxy users. 
Contributed by Daryn Sharp.
 2017-07-26 13:29:07 -05:00
Kihwal Lee
38c56bbb46 HDFS-12171. Reduce IIP object allocations for inode lookup. Contributed by Daryn Sharp. 
(cherry picked from commit 3d8c7d701b8f22b3b4d67f587f7ff2318c60c541)
 2017-07-25 11:08:57 -05:00
Jitendra Pandey
c0e294ffc8 HADOOP-10829. Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe. Contributed by Benoy Antony and Rakesh R. 
(cherry picked from commit b82485d6fed2194bf0dc2eedfab7e226e30a7cf0)
 2017-07-25 09:04:49 -07:00
Kihwal Lee
7be52c9c79 HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp. 
(cherry picked from commit 1a79dcfc457969d6a6c08ffffe4152fd7638e48a)
 2017-07-25 10:33:09 -05:00
Jason Lowe
db0bc8fafa YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 
(cherry picked from commit 24853bf32a045b8f029fb136edca2af03836c8d5)
 2017-07-24 10:25:04 -05:00
Allen Wittenauer
02ef086824
HADOOP-13178. TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X (Kai Sasaki via aw) 
(cherry picked from commit d8c1fd1944160ad5b5d093731ae987d701802321)
(cherry picked from commit daa768534409f08a83759634b5c3966e32e6f170)
 2017-07-24 12:03:40 +09:00