Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,110 Commits 326 Branches 371 Tags
Commit Graph

13110 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Junping Du
9289f4527d YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6ed9d362242d36bfae4083bf3b1c2eb32fbb2f72)
 2017-04-17 15:52:02 -07:00
Karthik Kambatla
6f5912c106 YARN-6432. FairScheduler: Reserve preempted resources for corresponding applications. (Miklos Szegedi via kasha) 2017-04-17 14:34:28 -07:00
Wangda Tan
6ade9e6ce6 YARN-6406. Remove SchedulerRequestKeys when no more pending ResourceRequest. (Arun Suresh via wangda) 
(cherry picked from commit 87e2ef8c985bb72a916477e8783359f2859f7890)
 2017-04-15 21:34:29 -07:00
Haibo Chen
d4f553d42f MAPREDUCE-6673. Add a test example job that grows in memory usage over time (Karthik Kambatla via Haibo Chen) 
Change-Id: Iccfc8c67c38c526cc61726d87bfcbcf69ac36fea
(cherry picked from commit 25ac44709b4bbed78b607ea48021237b64e01b9f)
 2017-04-14 17:37:37 -07:00
Arun Suresh
8515d35bd5 HADOOP-14311. Add python2.7-dev to Dockerfile (Allen Wittenauer via asuresh) 
(cherry picked from commit 0ac17dc644c0429ff8a6f743bf9d3ecdd7458e58)
 2017-04-14 16:58:00 -07:00
Jason Lowe
1699b20526 YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger 
(cherry picked from commit 416880550214e58f2284a045ad0c96ba4aa78ea8)
 2017-04-14 17:17:46 -05:00
Karthik Kambatla
53da65ef8b YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha) 
(cherry picked from commit 8a1d7480f73906d8e0342690ec6c6b008d6de21b)
 2017-04-14 15:07:54 -07:00
Yongjun Zhang
c411adac32 HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li. 2017-04-14 10:14:02 -07:00
Eric Payne
157a9f3104 YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger 
(cherry-picked from commit a41f8dd58e27d8835fbb64eeaba5d7416df0499d)
 2017-04-14 11:04:18 -05:00
Wangda Tan
5756256280 YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda) 
(cherry picked from commit eac6b4c35c50e555c2f1b5f913bb2c4d839f1ff4)
 2017-04-13 13:16:31 -07:00
Wangda Tan
408d23477f YARN-6040. Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes. (wangda) 2017-04-13 11:11:57 -07:00
Hanisha Koneru
107f685f64 HDFS-11630. TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds. Contributed by Hanisha Koneru. 2017-04-12 17:11:36 -07:00
Hanisha Koneru
ed8bb51f59 HDFS-11560. Expose slow disks via NameNode JMX. Contributed by Hanisha Koneru. 2017-04-12 16:59:37 -07:00
Mingliang Liu
03aa142e35 HADOOP-14274. Azure: Simplify Ranger-WASB policy model. Contributed by Sivaguru Sankaridurg 
(cherry picked from commit 0cab57223e3f54be17a5f27cefdb6d1da1b073e5)
 2017-04-12 16:09:20 -07:00
Mingliang Liu
508389ee54 HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu 
(cherry picked from commit 507845074970b90330db86bc1ff0124def550bb5)
 2017-04-12 14:30:54 -07:00
Junping Du
d77cada182 YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen. 
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
(cherry picked from commit 9d9087a67ad599e678948d5c1036e51cb8cf5e3a)
 2017-04-12 13:54:58 -07:00
Kihwal Lee
8ed230c805 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 2017-04-12 13:28:26 -05:00
Daniel Templeton
f20aa38a1d HADOOP-14246. Authentication Tokens should use SecureRandom instead of Random and 256 bit secrets 
(Conttributed by Robert Konter via Daniel Templeton)

(cherry picked from commit 4dd6206547de8f694532579e37ba8103bafaeb12)
 2017-04-12 11:19:53 -07:00
Chris Nauroth
1bfb382296 HADOOP-14301. Deprecate SharedInstanceProfileCredentialsProvider in branch-2. Contributed by Mingliang Liu. 2017-04-12 10:07:00 -07:00
Eric Payne
607295d21f YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 
(cherry picked from commit a16ab2be91e29f0aa4ad3488d35977cb5d1e1f5c)
 2017-04-12 10:26:31 -05:00
Chris Nauroth
d5e2bd4096 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
 2017-04-11 21:18:47 -07:00
Mingliang Liu
f1a85b2be3 HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou 2017-04-11 18:09:43 -07:00
Hanisha Koneru
f16bb79957 HDFS-11551. Support Timeout when checking single disk. Contributed by Hanisha Koneru. 2017-04-11 16:54:28 -07:00
Steve Loughran
5efb3ade1f HADOOP-14066 VersionInfo should be marked as public API 
(cherry picked from commit 2ed65540a3fe73db5d2dd0d25e658ccf13c9188a)
 2017-04-11 15:06:40 -07:00
Wangda Tan
2f08c86b7a YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (Carlo Curino via wangda) 
(cherry picked from commit 4d4ad0ebb76198c90c5317f19ce85a6696132c8f)
 2017-04-11 14:59:34 -07:00
Naganarasimha
b04c09163b YARN-6463. Correct spelling mistake in FileSystemRMStateStore. Contributed by Yeliang Cang. 
(cherry picked from commit 7d873c465b815816e9955cf40f4e141dd2e7d82b)
 2017-04-11 22:46:09 +05:30
Jason Lowe
b29fb55798 YARN-6461. TestRMAdminCLI has very low test timeouts. Contributed by Eric Badger 
(cherry picked from commit 90d97372ed11aaa95d338db8583f578da159834a)
 2017-04-11 09:17:42 -05:00
Jason Lowe
0cb22a0f15 YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu 
(cherry picked from commit 0e065f2edead41f6f894c9d70def389af1c0f239)
 2017-04-11 08:45:16 -05:00
Akira Ajisaka
121fcb74f3
HADOOP-13545. Update HSQLDB to 2.3.4. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit aabf08dd0707995e471d41ec4158c6af597c55dd)
 2017-04-11 14:39:31 +09:00
Konstantinos Karanasos
9b3ba25980 YARN-6344. Add parameter for rack locality delay in CapacityScheduler. (kkaranasos) 
(cherry picked from commit 3587b46164514e6751d7cebb09c8ab5e988853b4)
 2017-04-10 17:22:29 -07:00
Daniel Templeton
c1ae753de1 YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit e486569197e10bf48731d9e7ef5c031987c26acc)
 2017-04-10 14:57:19 -07:00
Andrew Wang
dab1deb9a0 HADOOP-14293. Initialize FakeTimer with a less trivial value. 
(cherry picked from commit be144117a885cb39bc192279c96cbe3790dc77b1)
 2017-04-10 11:37:31 -07:00
Daniel Templeton
2dae9299ef YARN-6343. Docker docs MR example is broken (Contributed by Prashant Jha via Daniel Templeton) 
(cherry picked from commit 14a39907c636420800210b984f2f78b639c8382c)
 2017-04-10 10:03:06 -07:00
Akira Ajisaka
21af5be51a
HADOOP-14290. Update SLF4J from 1.7.10 to 1.7.25. 
(cherry picked from commit 5d38504071d72745c42588c7a6e76bcffb13832c)
 2017-04-09 01:48:03 +09:00
Karthik Kambatla
e0b143e51a Revert "YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha)" 
This reverts commit dcfce5fcddcfdd23392586c87adf7c14e8ef265e.
 2017-04-07 17:37:31 -07:00
Karthik Kambatla
dcfce5fcdd YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha) 
(cherry picked from commit 2aa8967809b05e558b36ab6fc8c5110ddb723b45)
 2017-04-07 17:36:39 -07:00
John Zhuge
ff2870310e HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge. 
Signed-off-by: John Zhuge <jzhuge@apache.org>
(cherry picked from commit 56e81f2a204d5e65d29222df1b2ba4d892f9c0d5)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java
 2017-04-07 15:01:29 -07:00
Robert Kanter
cbb1c76195 YARN-6368. Decommissioning an NM results in a -1 exit code (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 63f7322522e0ab223ceb91440636eb62ca0a3e41)
 2017-04-07 14:28:39 -07:00
Zhe Zhang
c85026038a HADOOP-14276. Add a nanosecond API to Time/Timer/FakeTimer. Contributed by Erik Krogen. 
(cherry picked from commit 95b7f1d29a5e2dadd70a56fca5faa006c5bd74fc)
 2017-04-07 10:39:40 -07:00
Akira Ajisaka
a467e5412e
YARN-6288. Addendum patch to fix compile error. 2017-04-07 12:19:46 +09:00
Ming Ma
db5b4c292b YARN-5797. Add metrics to the node manager for cleaning the PUBLIC and PRIVATE caches. (Chris Trezzo via mingma) 2017-04-06 17:08:59 -07:00
Xiaoyu Yao
0391c92022 HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 
(cherry picked from commit 0eacd4c13be9bad0fbed9421a6539c64bbda4df1)
 2017-04-06 16:15:56 -07:00
Andrew Wang
ed8f738be5 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:38:22 -07:00
Xiaoyu Yao
325be6e720 HDFS-11362. StorageDirectory should initialize a non-null default StorageDirType. Contribute by Hanisha Koneru. 
(cherry picked from commit dde3bbf351ef3121ae80a96d06668d8fe33c788f)
 2017-04-06 14:33:30 -07:00
Jason Lowe
84b7f2e956 YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 
(cherry picked from commit 1b081ca27e05e97d8b7d284ca24200d43763e481)
 2017-04-06 16:25:41 -05:00
Andrew Wang
832c2f52f8 HADOOP-13688. Stop bundling HTML source code in javadoc JARs. 
(cherry picked from commit 72a2ae6452e615c66d10829da38737896814e02b)
 2017-04-06 13:14:59 -07:00
Junping Du
98759b426c HADOOP-13996. Fix some release build issues, include gpg timeout and disable sign for hadoop-dist. Contributed by Andrew Wang. 2017-04-06 13:06:10 -07:00
Xiaoyu Yao
9016614eb1 HDFS-11302. Improve Logging for SSLHostnameVerifier. Contributed by Chen Liang. 
(cherry picked from commit 32bb36b750ab656f2f32f6c74eaa1a3e68ae956e)
 2017-04-05 18:30:33 -07:00
Arpit Agarwal
75e60b6bb7 HDFS-11131. TestThrottledAsyncChecker#testCancellation is flaky. 2017-04-05 17:54:27 -07:00
Daniel Templeton
2f32accaaf MAPREDUCE-6846. Fragments specified for libjar paths are not handled correctly 
(Contributed by Chris Trezzo via Daniel Templeton)

(cherry picked from commit a92bc8a6c1c50d34d8229a67dc9d197823001551)
 2017-04-05 17:26:06 -07:00