Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,183 Commits 326 Branches 371 Tags
Commit Graph

12183 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Chris Douglas
9791ecc9d8 YARN-6577. Remove unused ContainerLocalization classes. Contributed by ZhangBing Lin 
(cherry picked from commit b23fcc86c670aa896151a2bd8878154d7bb45d13)
(cherry picked from commit 3d2afb209c9a296c05da4ddf62bfa69cd13b700d)
 2017-05-17 20:42:27 -07:00
Chris Douglas
475b50b44c HADOOP-14434. Use MoveFileEx to allow renaming a file when the destination exists. Contributed by Lukas Majercak 
(cherry picked from commit ef9e536a7137d209985d25a4769712c2db2c52bf)
(cherry picked from commit 4bb056544341726dd051353f1cc8430b801bf391)
 2017-05-17 17:25:08 -07:00
Rohith Sharma K S
c61956dd1e HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe. 2017-05-17 08:26:17 +05:30
Kihwal Lee
95e8f4c80d HDFS-11641. Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus. Contributed by Daryn Sharp. 
(cherry picked from commit 82ea3f4545c88f2dc106e63afd6fcd616bb120be)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-05-16 15:55:53 -05:00
Akira Ajisaka
9b595ba40b
HDFS-11833. HDFS architecture documentation descibes outdated placement policy. Contributed by Chen Liang. 
(cherry picked from commit 1d1c52b42feae5a4271ef4b771d0d8de43e83c15)
(cherry picked from commit c17cb03a23f2ea9f1af8f9c147ac68d8441be935)
 2017-05-16 11:54:27 -04:00
Jason Lowe
c2d7c12477 YARN-6603. NPE in RMAppsBlock. Contributed by Jason Lowe 
(cherry picked from commit 489f85933c508bc26de607b921e56e23b979fce8)
 2017-05-16 09:29:18 -05:00
Jonathan Eagles
c992cf638e YARN-6598. History server getApplicationReport NPE when fetching report for pre-2.8 job (Jason Lowe via jeagles) 2017-05-15 12:10:12 -05:00
Jason Lowe
f42222a75b HDFS-11818. TestBlockManager.testSufficientlyReplBlocksUsesNewRack fails intermittently. Contributed by Nathan Roberts 
(cherry picked from commit aeca471ec13135064f9605174cd6daa120007b18)
 2017-05-12 17:48:00 -05:00
Jason Lowe
dfa2657abf HADOOP-14376. Memory leak when reading a compressed file using the native library. Contributed by Eli Acherkan 
(cherry picked from commit 7bc217224891b7f7f0a2e35e37e46b36d8c5309d)
 2017-05-12 16:59:21 -05:00
Vinayakumar B
b121e047b4 HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B) 
(cherry picked from commit 1411612aa4e70c704b941723217ed4efd8a0125b)
(cherry picked from commit 53d9f56a18214ff351384589ebfadd6afae2c62f)
 2017-05-12 07:47:57 +05:30
Min Shen
aae0600a67 YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen. 
(cherry picked from commit 2ada100da7cfe12946e43da2929bd80c2a8bd833)
 2017-05-11 13:31:02 -07:00
Allen Wittenauer
d4ab50f9b0 HADOOP-11703. git should ignore .DS_Store files on Mac OS X (Abin Shahab via aw) 
(cherry picked from commit 344d7cb6dcd3b1752c82b62271799be9e4e10416)
 2017-05-10 19:55:37 -07:00
Kihwal Lee
a8f209dee7 HDFS-11755. Underconstruction blocks can be considered missing. Contributed by Nathan Roberts. 2017-05-10 14:22:17 -05:00
Jason Lowe
ef120bed7b HDFS-11745. Increase HDFS test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit fab73748a1ce982904d332e6ddd46169f186ad54)
 2017-05-10 13:03:48 -05:00
Jason Lowe
030d521a69 MAPREDUCE-6882. Increase MapReduce test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit 1e71fe8c42b7945170c47023e82a0412397ff57e)
 2017-05-10 09:59:10 -05:00
Jason Lowe
f32db1be1e YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit 6099deebcb0ea23adc0f0c62db749c8f528869e9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java
 2017-05-10 08:53:33 -05:00
Akira Ajisaka
4f8dd60e49
HADOOP-14373. License error in org.apache.hadoop.metrics2.util.Servers. Contributed by hu xiaodong. 
(cherry picked from commit a5fa954684f517bdf0a50f3fa7a0314f24cc7049)
(cherry picked from commit d9c3c78b7809d48e965046b2f76ac01e70942937)
 2017-05-10 05:30:55 -05:00
Masatake Iwasaki
59702efb78 HADOOP-14405. Fix performance regression due to incorrect use of DataChecksum. Contributed by LiXin Ge. 
(cherry picked from commit 06ffeb8d4d705d14e8755364099286ac4ed3bb83)
 2017-05-10 14:45:23 +09:00
Jason Lowe
7489f0fd1a HADOOP-14377. Increase Common test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit f9ba4ef7d5e50be86ffca71b6c4bbc5575f64326)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/service/TestCompositeService.java
 2017-05-09 16:23:31 -05:00
Lei Xu
5b8564f986 HDFS-11687. Add new public encryption APIs required by Hive. (lei) 
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
(cherry picked from commit 25f5d9ad5ee5ead349d259a99b49541a70b1604d)
(cherry picked from commit 1f30a8a105d178e0e554e85d713320bf5356e035)
 2017-05-09 12:11:48 -07:00
Haibo Chen
3799625e90 YARN-6457. Allow custom SSL configuration to be supplied in WebApps. (Sanjay M Pujare via Haibo Chen) 
(cherry picked from commit 1769b12a773dc6c83f13663a77da33fa78878730)
(cherry picked from commit 86fcb8a66ccc0a0cd0c41073bb3a49315033aed5)
 2017-05-09 08:52:37 -07:00
Akira Ajisaka
8007d024b8
HADOOP-14374. License error in GridmixTestUtils.java. Contributed by lixinglong. 
(cherry picked from commit c60164fb60d3d8afb0fb8410936ccc231c4593f9)
(cherry picked from commit 753954f7beb3a1ca4a60cc3f59c809aed3839dd2)
 2017-05-09 05:39:26 -05:00
Varun Vasudev
341457a0f2 YARN-4562. YARN WebApp ignores the configuration passed to it for keystore settings. Contributed by Sergey Shelukhin. 
(cherry picked from commit 07920acc165f17e8fe3ca9d21283a2123223f438)
(cherry picked from commit 4baee3eb674cfdab6e1bfa678211354656829ae3)
 2017-05-09 10:59:15 +05:30
Kihwal Lee
f8b6aa1ed1 HADOOP-13437. KMS should reload whitelist and default key ACLs when hot-reloading. Contributed by Xiao Chen. 
(cherry picked from commit 9daa9979a1f92fb3230361c10ddfcc1633795c0e)
 2017-05-08 08:36:53 -05:00
Kihwal Lee
9d8d3a84bd HDFS-11702. Remove indefinite caching of key provider uri in DFSClient. Contributed by Rushabh S Shah. 
(cherry picked from commit cef2815cf48154fe82f44082dcbdce6373c81284)
 2017-05-08 08:31:18 -05:00
Xiaoyu Yao
a80ac0822b HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore. 
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
(cherry picked from commit 81883e3150c42e866ba9e68d68b734f2772344b2)
 2017-05-04 16:03:39 -07:00
Kihwal Lee
103dfae7b9 HADOOP-14372. TestSymlinkLocalFS timeouts are too low. Contributed by Eric Badger. 
(cherry picked from commit d4631e466bc85ce605061673a31f451353da9713)
 2017-05-03 09:49:50 -05:00
Erik Krogen
bcafaf2c61 HDFS-11717. Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d9014bda93760f223789d2ec9f5e35f40de157d4)
(cherry picked from commit 9d6b873d3c249aa76420ab03ceefc6180295915f)
 2017-05-02 18:44:26 -07:00
Jason Lowe
e4af087d92 HADOOP-14306. TestLocalFileSystem tests have very low timeouts. Contributed by Eric Badger 
(cherry picked from commit cedaf4cab9d5013acd559dcb92d4976823ccdf85)
 2017-05-02 16:54:00 -05:00
Karthik Kambatla
dd46bc0f54 YARN-5343. TestContinuousScheduling#testSortedNodes fails intermittently. (Yufei Gu via kasha) 
(cherry picked from commit 7992c0b42ceb10fd3ca6c4ced4f59b8e8998e046)
 2017-05-02 16:46:10 -05:00
Karthik Kambatla
d382a21423 YARN-4812. TestFairScheduler#testContinuousScheduling fails intermittently. (kasha) 
(cherry picked from commit f84af8bd588763c4e99305742d8c86ed596e8359)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestContinuousScheduling.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
 2017-05-02 16:40:46 -05:00
Ming Ma
d1ae1fb449 HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma) 
(cherry picked from commit b48f27e794e42ba90836314834e872616437d7c9)
(cherry picked from commit 32b115da1d75b9b5b6a770371fbecfda43928bb9)
 2017-05-02 12:48:51 -07:00
Andrew Wang
ab511286fd HADOOP-14369. NetworkTopology calls expensive toString() when logging. Contributed by Inigo Goiri. 
(cherry picked from commit dcc292d7ded200a4976f6d348952ecba10f01db2)
(cherry picked from commit 484bab39442939e1fe8424def25d1aa4ca99a2e5)
 2017-05-02 10:53:59 -07:00
Ming Ma
305a9d886a HDFS-9016. Display upgrade domain information in fsck. 2017-05-02 09:52:16 -07:00
Akira Ajisaka
42fa35d531
HADOOP-14371. License error in TestLoadBalancingKMSClientProvider.java. Contributed by hu xiaodong. 
(cherry picked from commit 20cde552748cc6be1c5ca590755d4e5a67baf187)
(cherry picked from commit 036624900be0bcc2af7f60a28a72cea0295f7300)
 2017-05-02 23:39:25 +09:00
Ming Ma
c4c5533216 HDFS-9005. Provide configuration support for upgrade domain. 2017-05-02 06:53:32 -07:00
Kihwal Lee
c9bf21b0f3 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 2017-05-01 17:34:19 -05:00
Kihwal Lee
d1e6b6db7c HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:28:27 -05:00
Steve Loughran
993ab15b39 YARN-524 TestYarnVersionInfo failing if generated properties doesn't include an SVN URL 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463300 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 7c65dbb9385567fe1d8e280b81ecb97b109d18a8)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 5ca4f0f82e41c2833fd8520bc1f764a3f958bdf6)
 2017-05-01 08:58:53 -05:00
Eric Payne
5b9460301a YARN-6536. TestAMRMClient.testAMRMClientWithSaslEncryption fails intermittently. Contributed by Jason Lowe. 
(cherry picked from commit fdf5192bbbb3c81e5fb221758297605271139dc9)
 2017-04-28 15:23:05 -05:00
Eric Payne
84106e80a8 HADOOP-14320. TestIPC.testIpcWithReaderQueuing fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit 6fe11f61556407f8efb9b376623cd61609a40d5f)
 2017-04-28 14:55:14 -05:00
Robert Kanter
26492de185 YARN-5894. fixed license warning caused by de.ruedigermoeller:fst:jar:2.24 (haibochen via rkanter) 
(cherry picked from commit 371b6467dcee8517ef0d49b9f391302a6d9ed648)
 2017-04-28 09:21:41 -07:00
Konstantin V Shvachko
cb12ada03f HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 28eb2aabebd15c15a357d86e23ca407d3c85211c)
 2017-04-27 11:42:48 -07:00
Zhe Zhang
70f18911c7 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a33098309f17dfb0e3a000934f68394de44bf7)
(cherry picked from commit fc46f398d9192e83c7197115589eca7805c49943)
 2017-04-27 09:54:10 -07:00
Steve Loughran
40b9c30d64
HADOOP-14346. CryptoOutputStream throws IOException if stream is closed. 
Contributed by Pierre Lacave
 2017-04-27 12:17:20 +01:00
Eric Payne
73563bc9b6 YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe. 2017-04-26 14:33:27 -05:00
John Zhuge
19e94bfd1b HADOOP-14341. Support multi-line value for ssl.server.exclude.cipher.list. Contributed by John Zhuge. 
(cherry picked from commit 9ccb849eb69f05bccc3435306ec4bde104e411bf)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/SSLFactory.java
 2017-04-25 22:41:27 -07:00
Mingliang Liu
c3c9a91900 HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu 
(cherry picked from commit e4c32bccde3147f51263cb58cdb480453b1cb3e9)
(cherry picked from commit e7a08cc6dc1f729d04dea9239f0a8c90c8be5598)
 2017-04-25 20:55:42 -07:00
Jason Lowe
d85dbd11ae HDFS-11691. Add a proper scheme to the datanode links in NN web UI. Contributed by Kihwal Lee 
(cherry picked from commit e4321ec84321672a714419278946fe1012daac71)
 2017-04-25 17:32:07 -05:00
Yongjun Zhang
c88e5e1836 HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang. 
(cherry picked from commit 5078df7be317e635615c05c5da3285798993ff1f)
 2017-04-21 14:12:04 -07:00