Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,572 Commits 326 Branches 371 Tags
Commit Graph

9572 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wangda Tan
12f819a6b7 YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda) Backport by Jonathan Hung. 
(cherry picked from commit def374e666ed0c1d665aeb1b7307e09769448138)
 2017-06-03 13:59:36 -07:00
Naganarasimha
4f2ff8540b YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt 
(cherry picked from commit f04c81c9ce93512bc714531a7731debbe6b794ce)
 2017-06-02 16:53:38 -07:00
Xiao Chen
67e87c9aff HADOOP-14474. Use OpenJDK 7 instead of Oracle JDK 7 to avoid oracle-java7-installer failures. Contributed by Akira Ajisaka. 
(cherry picked from commit 5e40efe0ec38f1e1228846a5448d93bf040868ff)
(cherry picked from commit d946387850ae2fe383df90dd2722560986236e31)

Conflicts:
	dev-support/docker/Dockerfile
 2017-06-02 09:36:08 -07:00
Kihwal Lee
d3b86234b2 HDFS-11856. Ability to re-add upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-06-01 16:45:55 -05:00
Kihwal Lee
01cdea732f HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-06-01 12:41:00 -05:00
Brahma Reddy Battula
38bc12c683 Updating CHANGES.txt for HDFS-11078 2017-06-02 00:21:12 +08:00
Brahma Reddy Battula
2b19c7b3ff HDFS-11893. Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0dcf843c008f2b9cece8c0a0cef78140398ac464)
 2017-06-01 22:53:06 +08:00
Sunil G
fd112535c8 YARN-5333. Some recovered apps are put into default queue when RM HA. Contributed by Jun Gong. 2017-06-01 11:19:37 +05:30
Kihwal Lee
1903665b23 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 
(cherry picked from commit 8ed230c805625549b1cecc830e909a7027bb4961)
(cherry picked from commit 9dfe0b35158cc0046a82cbfea2218a4ad61f329e)
 2017-05-31 08:50:26 -07:00
Kihwal Lee
81712e84fb HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp. 
(cherry picked from commit 90020624b05230ad4a7fbd666d0177ecb107a4d6)
(cherry picked from commit 3a89a8814f8e443a5f7ceddcecd6d953cfc2c6f3)
 2017-05-31 08:50:26 -07:00
Akira Ajisaka
fd834915f2
Update CHANGES.txt for YARN-6304. 2017-05-30 18:05:05 +09:00
Junping Du
b60e11e6bf YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6ed9d362242d36bfae4083bf3b1c2eb32fbb2f72)
 2017-05-30 14:19:27 +05:30
Kihwal Lee
cafee1109c HDFS-8674. Improve performance of postponed block scans. Contributed by Daryn Sharp. 
(cherry picked from commit 0d8a35bd6de5d2a5a9b816ca98f31975e94bd7c6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2017-05-26 19:00:03 -07:00
Inigo Goiri
948d0ac72d Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri. 
(cherry picked from commit 303c8dc9b6c853c0939ea9ba14388897cc258071)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-05-26 13:36:34 -07:00
Brahma Reddy Battula
724a5f3dbc HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula. 2017-05-25 23:00:35 +08:00
Steve Loughran
f225b5514f HADOOP-13026 Should not wrap IOExceptions into a AuthenticationException in KerberosAuthenticator. Xuan Gong via stevel 2017-05-24 19:00:18 -07:00
Vinayakumar B
b6afb4a951 HDFS-6291. FSImage may be left unclosed in BootstrapStandby#doRun() ( Contributed by Sanghyun Yun) 
(cherry picked from commit e5e492a9631ff78302fccedcb64d7b64b9407991)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2017-05-24 18:47:31 -07:00
Amar Kamat
61ba70c978 HADOOP-7851. Configuration.getClasses() never returns the default value. (amarrk) 
(cherry picked from commit 82d57ee7fee82919b42066af38063a27d0998806)
 2017-05-24 17:19:54 -07:00
Konstantin V Shvachko
2b067895a1 Commit missing file TestBatchIbr for HDFS-9710 backport. 2017-05-24 12:21:48 -07:00
Konstantin V Shvachko
ac1b8ff78f HDFS-9710. DN can be configured to send block receipt IBRs in batches. Contributed by Tsz-Wo Nicholas Sze. 
Backport HDFS-11837 by Vinitha Reddy Gankidi

(cherry picked from commit 5b95971f8a6dee09d1143c6cf121afa22fa6c16e)
 2017-05-24 11:42:52 -07:00
Jason Lowe
f99a48aa63 YARN-6615. AmIpFilter drops query parameters on redirect. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 7ef01c7c49e38529557322dbb6fce3e310cb7826)
 2017-05-24 11:31:39 -05:00
Konstantin V Shvachko
6d657054eb Update CHANGES.txt for HDFS-11377. 2017-05-23 15:02:30 -07:00
Brahma Reddy Battula
7ae1bc3c20 HDFS-11864. Document Metrics to track usage of memory for writes. Contributed by Yiqun Lin. 
(cherry picked from commit 52661e0912a79d1e851afc2b46c941ce952ca63f)
 2017-05-24 00:04:25 +08:00
Brahma Reddy Battula
0b53e9f9af Update CHANGES.txt for HDFS-11849 2017-05-23 23:31:03 +08:00
Wangda Tan
cf3bf3c17b Update CHANGES.txt for YARN-6493 
Change-Id: I61fbb26c4e7abb5ee4817856eb7c65f072e9d764
 2017-05-22 22:22:42 -07:00
Yiqun Lin
f46781261f HDFS-11377. Balancer hung due to no available mover threads. Contributed by yunjiong zhao. 
(cherry picked from commit 9cbbd1eae893b21212c9bc9e6745c6859317a667)
 2017-05-22 17:44:12 -07:00
Wangda Tan
e9575d7dae YARN-6493. Print requested node partition in assignContainer logs. (Jonathan Hung via wangda) 
Change-Id: I6b045ebf0be09bcc73d4bf009ca60a59f9626916
 2017-05-22 15:04:09 -07:00
Brahma Reddy Battula
cecedcf558 HDFS-11849. JournalNode startup failure exception should be logged in log file. Contributed By Surendra Singh Lilhore. 
(cherry picked from commit 9cab42cc797986081fef184748044f1790a4f039)
 2017-05-23 02:52:41 +08:00
Akira Ajisaka
8a0e5e3b70 HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu. 
(cherry picked from commit ef432579a7763cc0e482fe049027c6e5325eb034)
(cherry picked from commit 68638661271e6e69c220b19fec7cbc7ae5e6208e)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/AvailableSpaceBlockPlacementPolicy.java
 2017-05-21 17:47:51 -07:00
Kihwal Lee
9440ec2771 HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui. 
(cherry picked from commit de30d66b2673d0344346fb985e786247ca682317)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
 2017-05-21 17:19:54 -07:00
Konstantin V Shvachko
321f730ca8 HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi. Backport HDFS-11855 by Vinitha Reddy Gankidi. 
(cherry picked from commit 67523ffcf491f4f2db5335899c00a174d0caaa9b)
(cherry picked from commit a6d0e9e56a7eef226c8a891cf741ab22d8205e1c)
 2017-05-21 13:56:44 -07:00
Xuan
43ea501e33 YARN-4612. Fix rumen and scheduler load simulator handle killed tasks 
properly. Contributed by Ming Ma.

(cherry picked from commit 4efdf3a979c361348612f817a3253be6d0de58f7)
 2017-05-21 13:00:32 -07:00
Karthik Kambatla
94031b1978 YARN-4367. SLS webapp doesn't load. (kasha) 2017-05-21 11:42:38 -07:00
Varun Vasudev
6feef1b039 YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt. 
(cherry picked from commit c28e16b40caf1e22f72cf2214ebc2fe2eaca4d03)
 2017-05-21 11:31:23 -07:00
Konstantin V Shvachko
eecc68d4d8 HDFS-9726. Refactor IBR code to a new class. Contributed by Tsz-Wo Nicholas Sze. Backport HDFS-11839 by Vinitha Reddy Gankidi. 
(cherry picked from commit 494b6c7c4b5e0a861fcbdcfb89cfc4869247e710)
 2017-05-18 18:09:36 -07:00
rohithsharmaks
0a4d8f6584 YARN-4109. Exception on RM scheduler page loading with labels. (Mohammad Shahid Khan via rohithsharmaks) 
(cherry picked from commit 8c180a13c82ab9d60f595e6942e35d51024dab53)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2017-05-18 17:26:30 -07:00
Konstantin V Shvachko
42c2d36ba6 MAPREDUCE-6304. Specifying node labels when submitting MR jobs. Contributed by Naganarasimha G R. Backport by Inigo Goiri. 
(cherry picked from commit 3164e7d83875aa6b7435d1dfe61ac280aa277f1c)
 2017-05-18 00:20:15 -07:00
Konstantin V Shvachko
2c3f6aedf1 HDFS-7990. IBR delete ack should not be delayed. Contributed by Daryn Sharp. Backport HDFS-11838 by Vinitha Gankidi. 
(cherry picked from commit 60882ab26d49f05cbf0686944af6559f86b3417d)
 2017-05-17 23:58:27 -07:00
Chris Douglas
2b58fb8b03 HADOOP-14434. Use MoveFileEx to allow renaming a file when the destination exists. Contributed by Lukas Majercak 
(cherry picked from commit ef9e536a7137d209985d25a4769712c2db2c52bf)
(cherry picked from commit 4bb056544341726dd051353f1cc8430b801bf391)
(cherry picked from commit 475b50b44cebf5e5b4622e4092f4a447569b1b2b)
 2017-05-17 17:30:20 -07:00
Konstantin V Shvachko
0823fb72a9 HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao. Backport HDFS-11732 by Zhe Zhang. 2017-05-16 11:03:12 -07:00
Brahma Reddy Battula
3004358639 HDFS-8312. Trash does not descent into child directories to check for permissions. Contributed By Weiwei Yang via Eric Yang. 2017-05-16 23:27:57 +05:30
Brahma Reddy Battula
b95d63cdac HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed By Brahma Reddy Battula. 2017-05-16 10:50:51 +05:30
Jason Lowe
3252064eac HADOOP-14376. Memory leak when reading a compressed file using the native library. Contributed by Eli Acherkan 
(cherry picked from commit 7bc217224891b7f7f0a2e35e37e46b36d8c5309d)
 2017-05-12 17:04:31 -05:00
Vinayakumar B
b0b6de38c9 HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B) 2017-05-12 14:26:28 +05:30
Min Shen
28b947603b YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen and Jonathan Hung. 
(cherry picked from commit 2ada100da7cfe12946e43da2929bd80c2a8bd833)
Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingMonitor.java
 2017-05-11 13:47:12 -07:00
Brahma Reddy Battula
764fd3ed49 HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. Contributed By Brahma Reddy Battula. 2017-05-11 10:22:59 +05:30
Chris Douglas
db72660f30 HADOOP-12173. NetworkTopology::add calls toString always. Contributed by Inigo Goiri 
(cherry picked from commit e59f6fad6a8849cfab6acbf012f338d9cc7dd63c)

# Conflicts:
#	hadoop-common-project/hadoop-common/CHANGES.txt
 2017-05-10 20:48:38 -07:00
Konstantin V Shvachko
f30484e042 Update CHANGES.txt for HADOOP-11703. 2017-05-10 20:01:43 -07:00
Allen Wittenauer
879dc57145 HADOOP-11703. git should ignore .DS_Store files on Mac OS X (Abin Shahab via aw) 
(cherry picked from commit 344d7cb6dcd3b1752c82b62271799be9e4e10416)
 2017-05-10 19:57:11 -07:00
Konstantin V Shvachko
834d4217b7 HADOOP-12975. Add jitter to CachingGetSpaceUsed's thread (Elliott Clark via Colin P. McCabe). 
HADOOP-14408. Backport contributed by Erik Krogen.

(cherry picked from commit bf780406f2b30e627bdf36ac07973f6931f81106)
(cherry picked from commit 56c997a1a61188693ed306b3ceddb13a2f2e6456)
(cherry picked from commit 04682cc6b00e2ba9c0a0a3d1f71a06f36cd4c1a9)
 2017-05-10 19:16:01 -07:00