Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,262 Commits 326 Branches 371 Tags
Commit Graph

12262 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiao Chen
d946387850 HADOOP-14474. Use OpenJDK 7 instead of Oracle JDK 7 to avoid oracle-java7-installer failures. Contributed by Akira Ajisaka. 
(cherry picked from commit 5e40efe0ec38f1e1228846a5448d93bf040868ff)
 2017-06-02 09:30:08 -07:00
Xiao Chen
ec10a81bdb HDFS-11741. Long running balancer may fail due to expired DataEncryptionKey. Contributed by Wei-Chiu Chuang and Xiao Chen. 
(cherry picked from commit 068e23b896c63b0f817e6b91d73c994be1551eb2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
(cherry picked from commit 6f2391e0ad6fbaa6786cc9f0e3a09b955e0cb21c)
 2017-06-01 14:05:31 -07:00
Kihwal Lee
89d59c292f HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-06-01 12:25:27 -05:00
Brahma Reddy Battula
0cba282261 HDFS-11893. Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0dcf843c008f2b9cece8c0a0cef78140398ac464)
 2017-06-01 22:40:43 +08:00
Kihwal Lee
fe6de5e594 HDFS-11856. Ability to re-add upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-05-31 13:08:51 -05:00
Nathan Roberts
1f077f45f9 YARN-6649. RollingLevelDBTimelineServer throws RuntimeException if object decoding ever fails runtime exception. Contributed by Jon Eagles. 
(cherry picked from commit 4369690ce63566131aee28696bf2683a3cb20205)
 2017-05-31 12:47:38 -05:00
John Zhuge
f8a3de64f8 HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge. 
(cherry picked from commit 6c6a7a59622ba7c1e4faa5534f4479de0cd84993)
 2017-05-28 22:29:49 -07:00
Arpit Agarwal
76c15121d8 HDFS-11891. DU#refresh should print the path of the directory when an exception is caught. Contributed by Chen Liang. 2017-05-26 16:03:04 -07:00
Inigo Goiri
a20789f299 Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri. 
(cherry picked from commit 303c8dc9b6c853c0939ea9ba14388897cc258071)
 2017-05-26 13:42:55 -07:00
Jason Lowe
b89d59e21e YARN-6641. Non-public resource localization on a bad disk causes subsequent containers failure. Contributed by Kuhu Shukla 2017-05-26 09:45:49 -05:00
Kihwal Lee
bbef16b84e HDFS-11817. A faulty node can cause a lease leak and NPE on accessing data. Contributed by Kihwal Lee. 
(Updated TestBlockUnderConstruction for nextGenerationStamp method)
(cherry picked from commit 2cb63433abb21cd2b74bd266b70a682caf9e2d98)
 2017-05-25 17:34:56 -05:00
Jason Lowe
8739c4d2f8 YARN-6643. TestRMFailover fails rarely due to port conflict. Contributed by Robert Kanter 
(cherry picked from commit 3fd6a2da4e537423d1462238e10cc9e1f698d1c2)
 2017-05-25 16:09:29 -05:00
Brahma Reddy Battula
4f5846f1e3 HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 7bad748091b6a5f90e919c781a915e13000e1be6)
 2017-05-25 22:40:11 +08:00
Varun Vasudev
c1f3cd765f Addendum patch to fix Docker sanitization. 
(cherry picked from commit 2ff2a1f50e8c7c0f33676b010b256d6c8daf912d)
(cherry picked from commit 983c4437c2b1aaafddfc97b19371cdb992389a61)
 2017-05-25 14:54:17 +05:30
Akira Ajisaka
d50f240784
YARN-6141. ppc64le on Linux doesn't trigger __linux get_executable codepath. Contributed by Sonia Garudi and Ayappan. 
(cherry picked from commit bc28da65fb1c67904aa3cefd7273cb7423521014)
(cherry picked from commit ca1c0cbc62b577e03ed59efb3f9050cba59be8a0)
 2017-05-25 17:08:18 +09:00
Wei-Chiu Chuang
caad191c0e Revert "HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory." 
This reverts commit 998e44a972ad06b48675afac89656ffba8c4d595.
 2017-05-24 18:12:03 -07:00
Wei-Chiu Chuang
6c50341ea8 Revert "HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang." 
This reverts commit f9d3bb3b8335889b30691baca4331f6f9ed28f69.
 2017-05-24 18:11:42 -07:00
Amar Kamat
425d597d48 HADOOP-7851. Configuration.getClasses() never returns the default value. (amarrk) 
(cherry picked from commit 82d57ee7fee82919b42066af38063a27d0998806)
 2017-05-24 17:06:26 -07:00
Brahma Reddy Battula
d9492839d0 HADOOP-14166. Reset the DecayRpcScheduler AvgResponseTime metric to zero when queue is not used. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit dcf4559ebdb5bb11d03b264a9875bff316b89eef)
 2017-05-25 01:28:36 +08:00
Brahma Reddy Battula
cf4e6d6b1d HADOOP-14449. The ASF Header in ComparableVersion.java and SSLHostnameVerifier.java is not correct. Contributed by ZhangBing Lin. 
(cherry picked from commit b7a0c0e7019528fac492e1cb32d374ed014f8673)
 2017-05-25 01:09:20 +08:00
Jason Lowe
7ef01c7c49 YARN-6615. AmIpFilter drops query parameters on redirect. Contributed by Wilfred Spiegelenburg 2017-05-24 11:27:21 -05:00
Brahma Reddy Battula
462c6fb5b1 HDFS-11864. Document Metrics to track usage of memory for writes. Contributed by Yiqun Lin. 
(cherry picked from commit 52661e0912a79d1e851afc2b46c941ce952ca63f)
 2017-05-24 23:47:20 +08:00
Brahma Reddy Battula
a7f063576c HDFS-11849. JournalNode startup failure exception should be logged in log file. Contributed By Surendra Singh Lilhore. 
(cherry picked from commit 9cab42cc797986081fef184748044f1790a4f039)
 2017-05-24 23:45:57 +08:00
Yiqun Lin
94a80974d3 HDFS-11377. Balancer hung due to no available mover threads. Contributed by yunjiong zhao. 
(cherry picked from commit 9cbbd1eae893b21212c9bc9e6745c6859317a667)
 2017-05-22 17:11:52 -07:00
Wangda Tan
836e8f3e69 YARN-6493. Print requested node partition in assignContainer logs. (Jonathan Hung via wangda) 
Change-Id: I3c5c012eb8d0887a4a37d58662bc37aac07ea86e
 2017-05-22 14:57:54 -07:00
Xuan
cef5ef2377 YARN-4612. Fix rumen and scheduler load simulator handle killed tasks 
properly. Contributed by Ming Ma.

(cherry picked from commit 4efdf3a979c361348612f817a3253be6d0de58f7)
 2017-05-21 12:51:16 -07:00
Eric Payne
71d91d53bf YARN-6618. TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once. Contributed by Jason Lowe. 
(cherry picked from commit 19482e0d0990feb719678d71971aa849095982fd)
 2017-05-19 11:36:18 -05:00
Varun Vasudev
23bbbce63e Sanitize arguments before launching Docker containers. 2017-05-18 12:03:52 +05:30
Varun Vasudev
51e65cc710 Validate docker image name before launching container. 2017-05-18 11:53:16 +05:30
Chris Douglas
9791ecc9d8 YARN-6577. Remove unused ContainerLocalization classes. Contributed by ZhangBing Lin 
(cherry picked from commit b23fcc86c670aa896151a2bd8878154d7bb45d13)
(cherry picked from commit 3d2afb209c9a296c05da4ddf62bfa69cd13b700d)
 2017-05-17 20:42:27 -07:00
Chris Douglas
475b50b44c HADOOP-14434. Use MoveFileEx to allow renaming a file when the destination exists. Contributed by Lukas Majercak 
(cherry picked from commit ef9e536a7137d209985d25a4769712c2db2c52bf)
(cherry picked from commit 4bb056544341726dd051353f1cc8430b801bf391)
 2017-05-17 17:25:08 -07:00
Rohith Sharma K S
c61956dd1e HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe. 2017-05-17 08:26:17 +05:30
Kihwal Lee
95e8f4c80d HDFS-11641. Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus. Contributed by Daryn Sharp. 
(cherry picked from commit 82ea3f4545c88f2dc106e63afd6fcd616bb120be)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-05-16 15:55:53 -05:00
Akira Ajisaka
9b595ba40b
HDFS-11833. HDFS architecture documentation descibes outdated placement policy. Contributed by Chen Liang. 
(cherry picked from commit 1d1c52b42feae5a4271ef4b771d0d8de43e83c15)
(cherry picked from commit c17cb03a23f2ea9f1af8f9c147ac68d8441be935)
 2017-05-16 11:54:27 -04:00
Jason Lowe
c2d7c12477 YARN-6603. NPE in RMAppsBlock. Contributed by Jason Lowe 
(cherry picked from commit 489f85933c508bc26de607b921e56e23b979fce8)
 2017-05-16 09:29:18 -05:00
Jonathan Eagles
c992cf638e YARN-6598. History server getApplicationReport NPE when fetching report for pre-2.8 job (Jason Lowe via jeagles) 2017-05-15 12:10:12 -05:00
Jason Lowe
f42222a75b HDFS-11818. TestBlockManager.testSufficientlyReplBlocksUsesNewRack fails intermittently. Contributed by Nathan Roberts 
(cherry picked from commit aeca471ec13135064f9605174cd6daa120007b18)
 2017-05-12 17:48:00 -05:00
Jason Lowe
dfa2657abf HADOOP-14376. Memory leak when reading a compressed file using the native library. Contributed by Eli Acherkan 
(cherry picked from commit 7bc217224891b7f7f0a2e35e37e46b36d8c5309d)
 2017-05-12 16:59:21 -05:00
Vinayakumar B
b121e047b4 HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B) 
(cherry picked from commit 1411612aa4e70c704b941723217ed4efd8a0125b)
(cherry picked from commit 53d9f56a18214ff351384589ebfadd6afae2c62f)
 2017-05-12 07:47:57 +05:30
Min Shen
aae0600a67 YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen. 
(cherry picked from commit 2ada100da7cfe12946e43da2929bd80c2a8bd833)
 2017-05-11 13:31:02 -07:00
Allen Wittenauer
d4ab50f9b0 HADOOP-11703. git should ignore .DS_Store files on Mac OS X (Abin Shahab via aw) 
(cherry picked from commit 344d7cb6dcd3b1752c82b62271799be9e4e10416)
 2017-05-10 19:55:37 -07:00
Kihwal Lee
a8f209dee7 HDFS-11755. Underconstruction blocks can be considered missing. Contributed by Nathan Roberts. 2017-05-10 14:22:17 -05:00
Jason Lowe
ef120bed7b HDFS-11745. Increase HDFS test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit fab73748a1ce982904d332e6ddd46169f186ad54)
 2017-05-10 13:03:48 -05:00
Jason Lowe
030d521a69 MAPREDUCE-6882. Increase MapReduce test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit 1e71fe8c42b7945170c47023e82a0412397ff57e)
 2017-05-10 09:59:10 -05:00
Jason Lowe
f32db1be1e YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit 6099deebcb0ea23adc0f0c62db749c8f528869e9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java
 2017-05-10 08:53:33 -05:00
Akira Ajisaka
4f8dd60e49
HADOOP-14373. License error in org.apache.hadoop.metrics2.util.Servers. Contributed by hu xiaodong. 
(cherry picked from commit a5fa954684f517bdf0a50f3fa7a0314f24cc7049)
(cherry picked from commit d9c3c78b7809d48e965046b2f76ac01e70942937)
 2017-05-10 05:30:55 -05:00
Masatake Iwasaki
59702efb78 HADOOP-14405. Fix performance regression due to incorrect use of DataChecksum. Contributed by LiXin Ge. 
(cherry picked from commit 06ffeb8d4d705d14e8755364099286ac4ed3bb83)
 2017-05-10 14:45:23 +09:00
Jason Lowe
7489f0fd1a HADOOP-14377. Increase Common test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit f9ba4ef7d5e50be86ffca71b6c4bbc5575f64326)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/service/TestCompositeService.java
 2017-05-09 16:23:31 -05:00
Lei Xu
5b8564f986 HDFS-11687. Add new public encryption APIs required by Hive. (lei) 
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
(cherry picked from commit 25f5d9ad5ee5ead349d259a99b49541a70b1604d)
(cherry picked from commit 1f30a8a105d178e0e554e85d713320bf5356e035)
 2017-05-09 12:11:48 -07:00
Haibo Chen
3799625e90 YARN-6457. Allow custom SSL configuration to be supplied in WebApps. (Sanjay M Pujare via Haibo Chen) 
(cherry picked from commit 1769b12a773dc6c83f13663a77da33fa78878730)
(cherry picked from commit 86fcb8a66ccc0a0cd0c41073bb3a49315033aed5)
 2017-05-09 08:52:37 -07:00