Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,507 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

9507 Commits

Author SHA1 Message Date
Konstantin V Shvachko cf5abf8a7c HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 28eb2aabeb)
# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
 2017-04-27 11:57:36 -07:00
Zhe Zhang bec23593b5 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a3309830)
(cherry picked from commit fc46f398d9)
(cherry picked from commit 70f18911c7)
 2017-04-27 09:56:14 -07:00
Brahma Reddy Battula ac12063eeb HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 
(cherry picked from commit 1c5708f6de)
(cherry picked from commit c895f08936)
 2017-04-26 15:59:21 -07:00
Konstantin V Shvachko 54bbdb4c02 Update CHANGES.txt for HDFS-11634. 2017-04-18 13:20:21 -07:00
Konstantin V Shvachko 5a714fee43 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 8dfcd95d58)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
 2017-04-17 17:39:29 -07:00
Zhe Zhang ccd24f0917 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 
(cherry picked from commit ad49098eb3)
(cherry picked from commit 051ab77806)
(cherry picked from commit 07047335a3)
 2017-04-17 16:55:24 -07:00
John Zhuge fac5bdc023 HADOOP-14195. CredentialProviderFactory$getProviders is not thread-safe. Contributed by Vihang Karajgaonkar. 
(cherry picked from commit 128015584d)
(cherry picked from commit a92f654aa2)
 2017-04-12 20:05:15 -07:00
Steve Loughran 7a3f9e8fac HADOOP-14066 VersionInfo should be marked as public API 
(cherry picked from commit 2ed65540a3)
 2017-04-11 15:06:48 -07:00
Arpit Agarwal 9f4585c95f HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 2017-04-11 11:43:25 -07:00
Arpit Agarwal f2d1d3d159 Revert "HDFS-11551. Handle SlowDiskReport from DataNode at the NameNode. Contributed by Xiaobing Zhou." 
This reverts commit ac76dc10dd.
 2017-04-11 11:42:57 -07:00
Xiaobing Zhou ac76dc10dd HDFS-11551. Handle SlowDiskReport from DataNode at the NameNode. Contributed by Xiaobing Zhou. 2017-04-11 11:16:36 -07:00
Andrew Wang 60a3a63990 HADOOP-14293. Initialize FakeTimer with a less trivial value. 
(cherry picked from commit be144117a8)
(cherry picked from commit dab1deb9a0)
 2017-04-10 11:37:38 -07:00
Zhe Zhang e5d5d0ddb5 HADOOP-14276. Add a nanosecond API to Time/Timer/FakeTimer. Contributed by Erik Krogen. 
(cherry picked from commit 95b7f1d29a)
(cherry picked from commit c85026038a)
(cherry picked from commit ef8b30dd4b)
 2017-04-07 10:59:44 -07:00
Xiaobing Zhou ea38aa727a HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou. 2017-04-05 15:34:40 -07:00
Zhe Zhang 79f7bfbf22 MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen. 
(cherry picked from commit de7efd2687)
(cherry picked from commit 3f5dc51ec9)
(cherry picked from commit 0acf2bca9c)
 2017-03-29 22:35:12 -07:00
Masatake Iwasaki 0ece3c8dd9 HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin. 
(cherry picked from commit ca3ede5fda)
 2017-03-28 18:17:45 +09:00
Andrew Wang 295aab8e3a HADOOP-14211. FilterFs and ChRootedFs are too aggressive about enforcing 'authorityNeeded'. Contributed by Erik Krogen. 
(cherry picked from commit 0e556a5ba6)
(cherry picked from commit 96fe940e59)
(cherry picked from commit 5130128a31)
 2017-03-24 11:38:39 -07:00
Zhe Zhang 4cc53b51f6 HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen. 
(cherry picked from commit 59d69257a8)
(cherry picked from commit f98f4bb6ca)
(cherry picked from commit 8ca9915d0f)
 2017-03-22 22:48:04 -07:00
Erik Krogen 1d035c8149 HADOOP-14169. Implement listStatusIterator, listLocatedStatus for ViewFs. Contributed by Erik Krogen. 
(cherry picked from commit e1a99802fc)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFsBaseTest.java
 2017-03-17 17:38:03 -07:00
Wei-Chiu Chuang 830a602375 HDFS-11499. Decommissioning stuck because of failing recovery. Contributed by Lukas Majercak and Manoj Govindassamy. 
(cherry picked from commit 385d2cb777)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 60be2e5d8a)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2017-03-13 13:45:12 -07:00
Jason Lowe ef99e5ed89 YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen 
(cherry picked from commit deb9f56946)
 2017-03-10 11:11:25 -06:00
Akira Ajisaka a47d8283b1
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f)
(cherry picked from commit 120f680318)
 2017-03-07 13:40:16 +09:00
Jason Lowe 39b483e065 YARN-6274. Documentation refers to incorrect nodemanager health checker interval property. Contributed by WeiWei Yang 
(cherry picked from commit 05237636d3)
 2017-03-03 08:03:05 -06:00
Steve Loughran c67d2f96da HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. 
Contributed by Steve Loughran

(cherry picked from commit 51fefbe38b)
 2017-03-02 16:01:29 -06:00
Gera Shegalov c8b1112ed9 YARN-1728. Workaround guice3x-undecoded pathInfo in YARN WebApp. (Yuanbo Liu via gera) 
(cherry picked from commit df35ba81fe)
 2017-02-28 10:07:18 -08:00
Zhe Zhang 33f4263345 HDFS-11466. Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms. Contributed by Andrew Wang. 
(cherry picked from commit d269b488a7)
(cherry picked from commit 5fb785e154)
(cherry picked from commit 265ddb20c3)
 2017-02-28 09:07:23 -08:00
Xiao Chen c2a4ce7717 HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp. 2017-02-17 15:38:55 -08:00
Wei-Chiu Chuang f133e0161c HDFS-11333. Print a user friendly error message when plugins are not found. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 859bd159ae)
(cherry picked from commit a8531d5d52)
(cherry picked from commit 4c47cb68e8)
 2017-02-15 03:05:01 -08:00
Xiao Chen 18dfff8a3c HADOOP-13433 Race in UGI.reloginFromKeytab. Contributed by Duo Zhang. 2017-02-08 23:48:24 -08:00
Eric Payne bc3e188e11 YARN-6152. Used queue percentage not accurate in UI for 2.7 and below when using DominantResourceCalculator. Contributed by Jonathan Hung 2017-02-07 11:33:28 -06:00
Junping Du 086ee0adf8 YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong 
(cherry picked from commit d81109e588)

(cherry picked from commit f40f17489c)
 2017-02-02 09:57:14 -08:00
Zhe Zhang f5a4f69e9d Addendum for HDFS-10534: revert accidental change to MiniHadoopClusterManager.java. 2017-01-26 10:50:48 -08:00
Zhe Zhang 03a67e1e62 HDFS-10534. NameNode WebUI should display DataNode usage histogram. Contributed by Kai Sasaki. 2017-01-26 10:34:00 -08:00
Akira Ajisaka ab62484240 HDFS-11352. Add missing entry in CHANGES.txt. 2017-01-24 14:57:21 +09:00
Akira Ajisaka 3acc96edbf HDFS-11352. Potential deadlock in NN when failing over. Contributed by Erik Krogen. 2017-01-24 14:55:27 +09:00
Akira Ajisaka b0b3966851 HDFS-11290. TestFSNameSystemMBean should wait until JMX cache is cleared. Contributed by Erik Krogen. 
(cherry picked from commit b1a9ec856b)
(cherry picked from commit db47bd15dd)
 2017-01-24 14:19:21 +09:00
Eric Yang b13492a0ab HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu. 
(cherry picked from commit a847903b6e)
 2017-01-21 15:50:42 -08:00
Akira Ajisaka 1cf20b37ed HADOOP-14001. Improve delegation token validity checking. 
(cherry picked from commit 1763467210)
(cherry picked from commit c6c29d0080)
 2017-01-19 18:07:54 +09:00
Vinitha Reddy Gankidi a6625e69a2 HDFS-10733. NameNode terminated after full GC thinking QJM is unresponsive. Contributed by Vinitha Gankidi. 2017-01-18 12:59:37 -08:00
Junping Du 8f2fa65a8e HADOOP-13958. Bump up release year to 2017. Contributed by Junping Du. 
(cherry picked from commit 94f1d06f20)
(cherry picked from commit 7b5b57e5b2)
 2017-01-08 15:49:42 -08:00
Junping Du 01dff41699 MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 
(cherry picked from commit 679478d0c6)
(cherry picked from commit bf87c523ab)
(cherry picked from commit 031ad740b9)
 2017-01-07 22:14:44 -08:00
Haohui Mai 977fb0c524 HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2017-01-04 21:16:20 -08:00
Masatake Iwasaki 04ccb1bec5 HADOOP-13839. Fix outdated tracing documentation. Contributed by Elek, Marton. 2017-01-05 09:24:15 +09:00
Brahma Reddy Battula a3f4ee53c8 Revert "HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao." 
This reverts commit d90f7412f2.
 2017-01-03 19:32:39 +05:30
Haohui Mai d90f7412f2 HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2016-12-31 01:12:28 -08:00
Sunil 7820eeb267 YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 2016-12-28 16:02:39 +05:30
Brahma Reddy Battula 3e0bd9951f HDFS-11263. ClassCastException when we use Bzipcodec for Fsimage compression. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1b401f6a73)
 2016-12-20 21:30:18 +05:30
Wei-Chiu Chuang bbf380a493 HDFS-11160. VolumeScanner reports write-in-progress replicas as corrupt incorrectly. Contributed by Wei-Chiu Chuang and Yongjun Zhang. 
(cherry picked from commit 0cb99db9d9)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockScanner.java

(cherry picked from commit 18be0447cde622dfaaad27f7c2b9cccb30469fef)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
 2016-12-16 13:40:20 -08:00
Wei-Chiu Chuang 2123b664ca HDFS-11229. HDFS-11056 failed to close meta file. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 292bd78b44)
(cherry picked from commit fa3b2bee61)
 2016-12-10 09:34:05 -08:00
Kihwal Lee deae1f8323 HADOOP-11859. PseudoAuthenticationHandler fails with httpcomponents v4.4. Contributed by Eugene Koifman. 
(cherry picked from commit db4419df6b)
 2016-12-09 11:05:04 -06:00