Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,751 Commits 326 Branches 371 Tags
Commit Graph

18751 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Weiwei Yang
ce4a0898df YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach. 2018-10-09 22:30:42 +08:00
Yiqun Lin
665036c5f7 HDFS-13962. Add null check for add-replica pool to avoid lock acquiring. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 1043795f7fe44c98a34f8ea3cea708c801c3043b)
 2018-10-09 10:36:45 +08:00
Sunil G
dd445e0368 YARN-7825. [UI2] Maintain constant horizontal application info bar for Application Attempt page. Contributed by Akhil PB. 
(cherry picked from commit 228dc192f1108438e2a221ec2ed7b6fd44080295)
 2018-10-08 11:56:28 +05:30
Inigo Goiri
9bbf494c7f HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice. Contributed by Ayush Saxena. 
(cherry picked from commit 3bfff0f79e7246cbb2989c720c14cfa2566a9d1a)
 2018-10-06 10:41:49 -07:00
Jason Lowe
df7ec58ced HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe 
(cherry picked from commit f13e231025333ebf80b30bbdce1296cef554943b)
 2018-10-05 09:27:23 -05:00
Sunil G
f3f0a9a2ca YARN-8797. [UI2] Improve error pages in new YARN UI. Contributed by Akhil PB. 
(cherry picked from commit 94d82f416258b65c8ab7849a459af78abebdf7d0)
 2018-10-05 14:34:55 +05:30
Sunil G
6407f2a0ed YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB. 
(cherry picked from commit 751f626e506b6cf8cc87ca65b65550871dca74fb)
 2018-10-05 14:10:57 +05:30
Sunil G
fae24b4747 Revert "YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB." 
This reverts commit dd70029255e9621675f2bcf1083d787d24896733.
 2018-10-05 13:38:21 +05:30
Sunil G
dd70029255 YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB. 
(cherry picked from commit 751f626e506b6cf8cc87ca65b65550871dca74fb)
 2018-10-05 12:29:33 +05:30
Takanobu Asanuma
af19ad5bfe HADOOP-15816. Upgrade Apache Zookeeper version due to security concerns. Contributed by Akira Ajisaka. 
(cherry picked from commit 241cbec2dab3c3d49b48f42b86e8bd85cd1f08f3)
 2018-10-05 15:31:04 +09:00
Yiqun Lin
dd70adf318 HDFS-13957. Fix incorrect option used in description of InMemoryAliasMap. 
(cherry picked from commit 619e490333fa89601fd476dedac6d16610e9a52a)
 2018-10-05 10:08:28 +08:00
Wangda Tan
86a1ad4428 YARN-8844. TestNMProxy unit test is failing. (Eric Yang via wangda) 
Change-Id: I241fa8701b6f1dbcad87fd2e9a429e32e7aa40f5
(cherry picked from commit b3ac8869338faeabedbc27e95e3166c830a9a761)
 2018-10-04 10:49:29 -07:00
Wangda Tan
30f50a0682 YARN-8758. Support getting PreemptionMessage when using AMRMClientAsyn. (Zian Chen via wangda) 
Change-Id: Ibf5d165f49957b582eeadeb41dc285c84d2f05e7
(cherry picked from commit 6926fd0ec634df2576bbc9f45e9636b99260db72)
(cherry picked from commit abe4a8e5d82e70ce991e2830f20eba9f25a2491a)
 2018-10-04 10:49:23 -07:00
Yiqun Lin
1a890b17b9 HADOOP-15817. Reuse Object Mapper in KMSJSONReader. Contributed by Jonathan Eagles. 
(cherry picked from commit 81f635f47f0737eb551bef1aa55afdf7b268253d)
 2018-10-04 10:35:16 +08:00
Akira Ajisaka
58693c63db
YARN-8840. Add missing cleanupSSLConfig() call for TestTimelineClient test. Contributed by Aki Tanaka. 
(cherry picked from commit 39b35036ba47064149003046a7b59feb01575d1e)
 2018-10-04 09:53:22 +09:00
Shane Kumpf
adbc010d0f YARN-8785. Improve the error message when a bind mount is not whitelisted. Contributed by Simon Prewo 
(cherry picked from commit 5edb9d3b972ad51e23b9cac7928b38876068a463)
 2018-10-02 07:26:45 -06:00
Yiqun Lin
65af98b58a HDFS-13768. Adding replicas to volume map makes DataNode start slowly. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 5689355783de005ebc604f4403dc5129a286bfca)
 2018-10-02 09:46:23 +08:00
Eric E Payne
c306da08ec YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang. 
(cherry picked from commit 8598b498bcaf4deffa822f871a26635bdf3d9d5c)
 2018-09-28 15:34:23 +00:00
Vrushali C
5b72aa04e1 YARN-8270 Adding JMX Metrics for Timeline Collector and Reader. Contributed by Sushil Ks. 
(cherry picked from commit 90e2e493b3dc8be54f655b957b98a4bc0e003684)
 2018-09-28 10:31:38 +05:30
Jason Lowe
a56a345e07 YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang 
(cherry picked from commit 6b988d821e62d29c118e10a7213583b92c302baf)
 2018-09-26 16:15:48 -07:00
Brahma Reddy Battula
02f8b5da47 HDFS-13840. RBW Blocks which are having less GS should be added to Corrupt. Contributed by Brahma Reddy Battula 
(cherry picked from commit e0ff8e2c10f38298b81624df1e62743ac171f18d)
 2018-09-26 23:51:59 +05:30
Brahma Reddy Battula
97b75f47fd HDFS-13790. RBF: Move ClientProtocol APIs to its own module Contributed by Chao Sun. 2018-09-26 22:40:48 +05:30
Rohith Sharma K S
d473152e6a YARN-8824. App Nodelabel missed after RM restart for finished apps. Contributed by Bibin A Chundatt. 2018-09-26 20:36:34 +05:30
Akira Ajisaka
7dbdb7560d
YARN-8752. yarn-registry.md has wrong word ong-lived, it should be long-lived. Contributed by leiqiang. 
(cherry picked from commit 94b5c54e8af51d178474d2f0eb5505366750ecf4)
 2018-09-26 09:43:33 +09:00
Shane Kumpf
eea10c7b20 YARN-8623. Update Docker examples to use image which exists. Contributed by Craig Condit 
(cherry picked from commit 26c94a0fd07dab40fb5e62614ae71d98110a7467)
 2018-09-25 06:42:20 -06:00
Sammi Chen
5da3e83597 HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu. 
(cherry picked from commit 2b635125fb059fc204ed35bc0e264c42dd3a9fe9)
 2018-09-25 19:50:39 +08:00
Rohith Sharma K S
47306cc2db YARN-8815. RM fails to recover finished unmanaged AM. Contributed by Bibin A Chundatt. 
(cherry picked from commit 50bc7746d7593bf6adc92fc9807a99cf53b4aef3)
 2018-09-25 11:40:09 +05:30
Wangda Tan
daa14bf06a MAPREDUCE-7125. JobResourceUploader creates LocalFileSystem when it's not necessary. (Peter Cseh via wangda) 
Change-Id: I1aa720ed03739f6f4abeec46f6068e2ab332987a
(cherry picked from commit 230d90c5b9cc0d6e74bf16b1cf6685a163a24d48)
 2018-09-24 13:53:51 -07:00
Sunil G
0196ffaa22 YARN-8742. [UI2] Container logs on Application / Service pages on UI2 are not available many case, improve error messages in such cases. Contributed by Akhil PB. 
(cherry picked from commit 32a35dc94b5914a782e8f85e8bef0466b57c5534)
 2018-09-24 12:21:15 +05:30
Sunil G
6699b9658e YARN-8628. [UI2] Few duplicated or inconsistent information displayed in UI2. Contributed by Akhil PB. 
(cherry picked from commit a2752779ac1545f5e0a52fce3cff02a7007e95fb)
 2018-09-21 16:06:22 +05:30
Eric Yang
e9315f6688 YARN-8801. Fixed header comments for docker utility functions. 
Contributed by Zian Chen
 2018-09-20 13:12:29 -04:00
Steve Loughran
9dc180d14a
HADOOP-15748. S3 listing inconsistency can raise NPE in globber. 
Contributed by Steve Loughran.

(cherry picked from commit 646874c326139457b79cf8cfa547b3c91a78c7b4)
 2018-09-20 13:05:46 +01:00
Vinayakumar B
2aa3854633 HADOOP-15736. Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. Contributed by Ayush Saxena. 
(cherry picked from commit 7ad27e97f05b13b33fdcef9cb63ace9c1728bfb5)
 2018-09-20 09:33:22 +05:30
Jason Lowe
3fb6787295 YARN-8784. DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk. Contributed by Eric Badger 
(cherry picked from commit 6b5838ed3220f992092c7348f92f1d9d0d4a3061)
 2018-09-19 16:49:21 -05:00
Chen Liang
b4069343bb HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-19 14:28:35 -07:00
Arpit Agarwal
58f339e491 HADOOP-15772. Remove the 'Path ... should be specified as a URI' warnings on startup. Contributed by Ayush Saxena. 
(cherry picked from commit 2eb597b1511f8f46866abe4eeec820f4191cc295)
 2018-09-19 13:49:06 -07:00
Inigo Goiri
2f7222a27b HADOOP-15684. triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens. Contributed by Rong Tang. 
(cherry picked from commit 7f9a89e1b54a9712af50ffef70bed7cfb91ed34e)
 2018-09-19 12:59:16 -07:00
Inigo Goiri
e1af3c9bb7 HDFS-13908. TestDataNodeMultipleRegistrations is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 56e0d635e0f49772d001c1fdb385110c705a622a)
 2018-09-19 09:31:18 -07:00
Weiwei Yang
aaf0b119e5 YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang. 
(cherry picked from commit 0712537e799bc03855d548d1f4bd690dd478b871)
 2018-09-19 19:38:09 +08:00
Wei-Chiu Chuang
7442b7941d HDFS-13868. WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is not. Contributed by Pranay Singh. 
(cherry picked from commit 28ceb34a725cd06d28fb51361c49bb45464f5368)
 2018-09-19 03:12:55 -07:00
Wei-Chiu Chuang
595ce94570 HDFS-13886. HttpFSFileSystem.getFileStatus() doesn't return "snapshot enabled" bit. Contributed by Siyao Meng. 
(cherry picked from commit 44857476fa993fbf9c97f979b91e19d27632c10a)
 2018-09-18 15:40:10 -07:00
Jason Lowe
3a13fa1b7b MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 8382b860d4ef4f20d000537ded42a88e98bd2190)
 2018-09-18 17:12:11 -05:00
Jason Lowe
85ae097f3c MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 34b2237e420cfbe3a97ddd44968de8bbe1ed30ab)
 2018-09-18 17:08:35 -05:00
Jason Lowe
610b00bdf4 HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia 
(cherry picked from commit e71f61ecb87e04727a5a76e578a75714c9db6706)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestStringUtils.java
 2018-09-18 16:00:48 -05:00
Jason Lowe
3d77094cf2 YARN-8648. Container cgroups are leaked when using docker. Contributed by Jim Brennan 
(cherry picked from commit 2df0a8dcb3dfde15d216481cc1296d97d2cb5d43)
 2018-09-18 15:43:10 -05:00
Eric E Payne
af2390cf49 MAPREDUCE-3801: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently. Contributed by Jason Lowe 
(cherry picked from commit 2cf8927683d49f4cdc3cba038480128c473c5461)
 2018-09-18 20:31:33 +00:00
Mingliang Liu
bff2230591 HDFS-11719. Arrays.fill() wrong index in BlockSender.readChecksum() exception handling. Contributed by Tao Zhang 2018-09-18 10:40:16 -07:00
Sammi Chen
53528d5ec5 HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu. 
(cherry picked from commit e4fca6aae46a3c04fc56897986a4ab4e5aa98503)
 2018-09-18 18:44:01 +08:00
Weiwei Yang
46eeba6234 YARN-8787. Fix broken list items in PlacementConstraints documentation. Contributed by Masahiro Tanaka. 
(cherry picked from commit 78a0d173e4f0c2f2679a04edd62a60fb76dde4f0)
 2018-09-18 17:21:21 +08:00
Sunil G
75d2d0fdb1 YARN-8652. [UI2] YARN UI2 breaks if getUserInfo REST API is not available in older versions. Contributed by Akhil PB. 
(cherry picked from commit bbeca0107e247ae14cfe96761f9e5fbb1f02e53d)
 2018-09-18 12:45:08 +05:30