Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,532 Commits 326 Branches 371 Tags
Commit Graph

9532 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Brahma Reddy Battula
3004358639 HDFS-8312. Trash does not descent into child directories to check for permissions. Contributed By Weiwei Yang via Eric Yang. 2017-05-16 23:27:57 +05:30
Brahma Reddy Battula
b95d63cdac HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed By Brahma Reddy Battula. 2017-05-16 10:50:51 +05:30
Jason Lowe
3252064eac HADOOP-14376. Memory leak when reading a compressed file using the native library. Contributed by Eli Acherkan 
(cherry picked from commit 7bc217224891b7f7f0a2e35e37e46b36d8c5309d)
 2017-05-12 17:04:31 -05:00
Vinayakumar B
b0b6de38c9 HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B) 2017-05-12 14:26:28 +05:30
Min Shen
28b947603b YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen and Jonathan Hung. 
(cherry picked from commit 2ada100da7cfe12946e43da2929bd80c2a8bd833)
Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingMonitor.java
 2017-05-11 13:47:12 -07:00
Brahma Reddy Battula
764fd3ed49 HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. Contributed By Brahma Reddy Battula. 2017-05-11 10:22:59 +05:30
Chris Douglas
db72660f30 HADOOP-12173. NetworkTopology::add calls toString always. Contributed by Inigo Goiri 
(cherry picked from commit e59f6fad6a8849cfab6acbf012f338d9cc7dd63c)

# Conflicts:
#	hadoop-common-project/hadoop-common/CHANGES.txt
 2017-05-10 20:48:38 -07:00
Konstantin V Shvachko
f30484e042 Update CHANGES.txt for HADOOP-11703. 2017-05-10 20:01:43 -07:00
Allen Wittenauer
879dc57145 HADOOP-11703. git should ignore .DS_Store files on Mac OS X (Abin Shahab via aw) 
(cherry picked from commit 344d7cb6dcd3b1752c82b62271799be9e4e10416)
 2017-05-10 19:57:11 -07:00
Konstantin V Shvachko
834d4217b7 HADOOP-12975. Add jitter to CachingGetSpaceUsed's thread (Elliott Clark via Colin P. McCabe). 
HADOOP-14408. Backport contributed by Erik Krogen.

(cherry picked from commit bf780406f2b30e627bdf36ac07973f6931f81106)
(cherry picked from commit 56c997a1a61188693ed306b3ceddb13a2f2e6456)
(cherry picked from commit 04682cc6b00e2ba9c0a0a3d1f71a06f36cd4c1a9)
 2017-05-10 19:16:01 -07:00
Akira Ajisaka
f052976ff1
HDFS-11795. Fix ASF License warnings in branch-2.7. Contributed by Yiqun Lin. 2017-05-10 14:17:00 -05:00
Brahma Reddy Battula
48c5e37abf HDFS-11373. Backport HDFS-11258 and HDFS-11272(File mtime change could not save to editlog). (Contributed by Jimmy Xiang, backported by Akira Ajisaka) 2017-05-10 11:06:57 +05:30
Akira Ajisaka
2875462c2b
HADOOP-14100. Upgrade Jsch jar to latest version to fix vulnerability in old versions. Contributed by Vinayakumar B and Brahma Reddy Battula. 2017-05-09 11:47:28 -05:00
Akira Ajisaka
9088f8358a
Update CHANGES.txt for YARN-6457 branch-2.7. 2017-05-09 11:41:08 -05:00
Akira Ajisaka
6599e57a79
HDFS-11766. Fix findbugs warning in branch-2.7. Contributed by Chen Liang. 2017-05-09 11:22:57 -05:00
Haibo Chen
f8e893f7ea YARN-6457. Allow custom SSL configuration to be supplied in WebApps. (Sanjay M Pujare via Haibo Chen) 
(cherry picked from commit 1769b12a773dc6c83f13663a77da33fa78878730)
(cherry picked from commit 86fcb8a66ccc0a0cd0c41073bb3a49315033aed5)
 2017-05-09 08:56:56 -07:00
Akira Ajisaka
0187892151
HADOOP-14374. License error in GridmixTestUtils.java. Contributed by lixinglong. 
(cherry picked from commit c60164fb60d3d8afb0fb8410936ccc231c4593f9)
(cherry picked from commit 753954f7beb3a1ca4a60cc3f59c809aed3839dd2)
 2017-05-09 05:43:12 -05:00
Varun Vasudev
0a149f3413 YARN-4562. YARN WebApp ignores the configuration passed to it for keystore settings. Contributed by Sergey Shelukhin. 
Added CHANGES.txt entry

(cherry picked from commit 07920acc165f17e8fe3ca9d21283a2123223f438)
(cherry picked from commit 4baee3eb674cfdab6e1bfa678211354656829ae3)
 2017-05-09 10:51:31 +05:30
Konstantin V Shvachko
c7c410898e Update CHANGES.txt for HDFS-11717 branch-2.7. 2017-05-02 22:30:18 -07:00
Erik Krogen
a00cef6079 HDFS-11717. Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d9014bda93760f223789d2ec9f5e35f40de157d4)
(cherry picked from commit 9d6b873d3c249aa76420ab03ceefc6180295915f)
 2017-05-02 18:46:58 -07:00
Akira Ajisaka
60e74edd05
HADOOP-14371. License error in TestLoadBalancingKMSClientProvider.java. Contributed by hu xiaodong. 
(cherry picked from commit 20cde552748cc6be1c5ca590755d4e5a67baf187)
(cherry picked from commit 036624900be0bcc2af7f60a28a72cea0295f7300)
 2017-05-02 23:41:17 +09:00
Kihwal Lee
130a0e2156 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 
Added a CHANGES.txt entry
(cherry picked from commit f77d138725662197ca0be874e9ec4dc430db2a4f)
 2017-05-01 17:36:58 -05:00
Kihwal Lee
139327d34b HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:31:59 -05:00
Zhe Zhang
8ad9efbe14 HDFS-8873. Allow the directoryScanner to be rate-limited. Contributed by Daniel Templeton. 2017-04-27 15:04:10 -07:00
Konstantin V Shvachko
5e67eb7560 Update CHANGES.txt for HDFS-11384 branch-2.7. 2017-04-27 12:23:41 -07:00
Konstantin V Shvachko
cf5abf8a7c HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 28eb2aabebd15c15a357d86e23ca407d3c85211c)
# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
 2017-04-27 11:57:36 -07:00
Zhe Zhang
bec23593b5 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a33098309f17dfb0e3a000934f68394de44bf7)
(cherry picked from commit fc46f398d9192e83c7197115589eca7805c49943)
(cherry picked from commit 70f18911c71c61f9b3918e195bfb9a99d22d230c)
 2017-04-27 09:56:14 -07:00
Brahma Reddy Battula
ac12063eeb HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 
(cherry picked from commit 1c5708f6ded6ba9fc007eed93c1f65ba44451241)
(cherry picked from commit c895f0893621fd6196c42ed46b97cb99cc2e9bee)
 2017-04-26 15:59:21 -07:00
Konstantin V Shvachko
54bbdb4c02 Update CHANGES.txt for HDFS-11634. 2017-04-18 13:20:21 -07:00
Konstantin V Shvachko
5a714fee43 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 8dfcd95d580bb090af7f40af0a57061518c18c8c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
 2017-04-17 17:39:29 -07:00
Zhe Zhang
ccd24f0917 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 
(cherry picked from commit ad49098eb324e238d97db68d7239ed2c4d84afa0)
(cherry picked from commit 051ab778065d3659c5284b5d95d04156c4ee7ffa)
(cherry picked from commit 07047335a3bc12b719ca608742bef1e565151035)
 2017-04-17 16:55:24 -07:00
John Zhuge
fac5bdc023 HADOOP-14195. CredentialProviderFactory$getProviders is not thread-safe. Contributed by Vihang Karajgaonkar. 
(cherry picked from commit 128015584d69492806fd1700c8f840d78aa9c729)
(cherry picked from commit a92f654aa2859dd962b6d75f4b25a7f8344cc7aa)
 2017-04-12 20:05:15 -07:00
Steve Loughran
7a3f9e8fac HADOOP-14066 VersionInfo should be marked as public API 
(cherry picked from commit 2ed65540a3fe73db5d2dd0d25e658ccf13c9188a)
 2017-04-11 15:06:48 -07:00
Arpit Agarwal
9f4585c95f HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 2017-04-11 11:43:25 -07:00
Arpit Agarwal
f2d1d3d159 Revert "HDFS-11551. Handle SlowDiskReport from DataNode at the NameNode. Contributed by Xiaobing Zhou." 
This reverts commit ac76dc10dde45ed00885bb96e89ed78015435790.
 2017-04-11 11:42:57 -07:00
Xiaobing Zhou
ac76dc10dd HDFS-11551. Handle SlowDiskReport from DataNode at the NameNode. Contributed by Xiaobing Zhou. 2017-04-11 11:16:36 -07:00
Andrew Wang
60a3a63990 HADOOP-14293. Initialize FakeTimer with a less trivial value. 
(cherry picked from commit be144117a885cb39bc192279c96cbe3790dc77b1)
(cherry picked from commit dab1deb9a0c96471b6da4ef95be86cbc81d55305)
 2017-04-10 11:37:38 -07:00
Zhe Zhang
e5d5d0ddb5 HADOOP-14276. Add a nanosecond API to Time/Timer/FakeTimer. Contributed by Erik Krogen. 
(cherry picked from commit 95b7f1d29a5e2dadd70a56fca5faa006c5bd74fc)
(cherry picked from commit c85026038a6ef8d38b7a8563fbb64ac6edd1d4ce)
(cherry picked from commit ef8b30dd4beb8baf59f70e81ee938d7852922a42)
 2017-04-07 10:59:44 -07:00
Xiaobing Zhou
ea38aa727a HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou. 2017-04-05 15:34:40 -07:00
Zhe Zhang
79f7bfbf22 MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen. 
(cherry picked from commit de7efd2687ccb74608249abf4c83c84a9345ec0a)
(cherry picked from commit 3f5dc51ec94af5a20a319dd9bc807fc9e4db4f6a)
(cherry picked from commit 0acf2bca9c19bcf76526d2ff59886931cd4041dd)
 2017-03-29 22:35:12 -07:00
Masatake Iwasaki
0ece3c8dd9 HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin. 
(cherry picked from commit ca3ede5fda60f317c24be8a378e7450ae581f0c5)
 2017-03-28 18:17:45 +09:00
Andrew Wang
295aab8e3a HADOOP-14211. FilterFs and ChRootedFs are too aggressive about enforcing 'authorityNeeded'. Contributed by Erik Krogen. 
(cherry picked from commit 0e556a5ba645570d381beca60114a1239b27d49f)
(cherry picked from commit 96fe940e59127dc7c3e4182c3ed450c3cd8d858e)
(cherry picked from commit 5130128a31a75587f6ccca08c487e44f73685227)
 2017-03-24 11:38:39 -07:00
Zhe Zhang
4cc53b51f6 HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen. 
(cherry picked from commit 59d69257a888347f0fb9c51bb000afc986b64f98)
(cherry picked from commit f98f4bb6ca45e6f807ca747d673690232e39da52)
(cherry picked from commit 8ca9915d0f833bff6dae75fdcd071b6ae79b6320)
 2017-03-22 22:48:04 -07:00
Erik Krogen
1d035c8149 HADOOP-14169. Implement listStatusIterator, listLocatedStatus for ViewFs. Contributed by Erik Krogen. 
(cherry picked from commit e1a99802fc5c2f2ad3ceb000378db1c0133069b3)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFsBaseTest.java
 2017-03-17 17:38:03 -07:00
Wei-Chiu Chuang
830a602375 HDFS-11499. Decommissioning stuck because of failing recovery. Contributed by Lukas Majercak and Manoj Govindassamy. 
(cherry picked from commit 385d2cb777a0272ac20c62336c944fad295d5d12)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 60be2e5d8a1a6a8921c68f8b0f428b55152d05db)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2017-03-13 13:45:12 -07:00
Jason Lowe
ef99e5ed89 YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen 
(cherry picked from commit deb9f569465bb760e661e60a313dad1605635236)
 2017-03-10 11:11:25 -06:00
Akira Ajisaka
a47d8283b1
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f4cc7d925d078a7ce32e5800da4e429)
(cherry picked from commit 120f680318d766f44787fd0eec88270d61172523)
 2017-03-07 13:40:16 +09:00
Jason Lowe
39b483e065 YARN-6274. Documentation refers to incorrect nodemanager health checker interval property. Contributed by WeiWei Yang 
(cherry picked from commit 05237636d3a359dc502e8fa2d66bfdcea274f86c)
 2017-03-03 08:03:05 -06:00
Steve Loughran
c67d2f96da HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. 
Contributed by Steve Loughran

(cherry picked from commit 51fefbe38bf40373d2855163fa6444821b0f994c)
 2017-03-02 16:01:29 -06:00
Gera Shegalov
c8b1112ed9 YARN-1728. Workaround guice3x-undecoded pathInfo in YARN WebApp. (Yuanbo Liu via gera) 
(cherry picked from commit df35ba81fe26f526a1534b72089fbb310efccdd9)
 2017-02-28 10:07:18 -08:00