12787 Commits

Author SHA1 Message Date
Arpit Agarwal
ff08f54f41 HDFS-11479. Socket re-use address option should be used in SimpleUdpServer. Contributed by Mukul Kumar Singh. 2017-03-01 12:59:09 -08:00
Brahma Reddy Battula
a545ee4d7f HDFS-11432. Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands. Contributed by Brahma Reddy Battula.
(cherry picked from commit dcd03df9f9e0080d7e179060ffc8148336c31b3e)
2017-03-01 10:47:47 +05:30
Jian He
c7ddf95d4a YARN-6153. KeepContainer does not work when AM retry window is set. Contributed by kyungwan nam
(cherry picked from commit 235203dffda1482fb38762fde544c4dd9c3e1fa8)
2017-02-28 13:26:55 -08:00
Xuan
95bd3c3d55 YARN-6189: Improve application status log message when RM restarted when
app is in NEW state. Contributed by Junping Du

(cherry picked from commit e0bb867c3fa638c9f689ee0b044b400481cf02b5)
2017-02-28 11:06:07 -08:00
Gera Shegalov
df35ba81fe YARN-1728. Workaround guice3x-undecoded pathInfo in YARN WebApp. (Yuanbo Liu via gera) 2017-02-28 09:46:57 -08:00
Zhe Zhang
5fb785e154 HDFS-11466. Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms. Contributed by Andrew Wang.
(cherry picked from commit d269b488a71a158d3ddcbdea96992abe29725c69)
2017-02-28 09:02:59 -08:00
Karthik Kambatla
58eefc71c7 YARN-6231. FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness. (kasha)
(cherry picked from commit f187d63816584b783fbfe238475c8f37decdb6dc)
2017-02-27 22:16:32 -08:00
Lei Xu
76bf47cb91 Add missing file hadoop-common-project/hadoop-kms/src/main/tomcat/catalina-default.properties for HADOOP-14083. (lei) 2017-02-27 20:49:35 -08:00
Yongjun Zhang
16ca70553c HADOOP-14127. Add log4j configuration to enable logging in hadoop-distcp's tests. (Xiao Chen via Yongjun Zhang)
(cherry picked from commit d235dcdf0b6f36162280e32c7c0342b7970c9b7b)
2017-02-27 20:46:17 -08:00
Lei Xu
d440ad46b5 HADOOP-14083. KMS should support old SSL clients. (John Zhuge via Lei Xu) 2017-02-27 18:12:00 -08:00
Naganarasimha
1eec911cd9 YARN-5703. ReservationAgents are not correctly configured. Contributed by Manikandan R.
(cherry picked from commit 5f5b031d1f20cb7f621db41979e963eaa42cf52f)
2017-02-27 20:42:10 +05:30
Sunil G
68b08e96a0 YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan. 2017-02-27 17:54:47 +05:30
Yiqun Lin
6cd39803ba HADOOP-14119. Remove unused imports from GzipCodec.java. Contributed by Yiqun Lin.
(cherry picked from commit 9db2e0c8d4b29971da60e2628ff55b5dabed2f7b)
2017-02-27 18:41:47 +08:00
Karthik Kambatla
c24325bbcb YARN-6172. FSLeafQueue demand update needs to be atomic. (Miklos Szegedi via kasha)
(cherry picked from commit fbfe86deea5f2aa857cd13fee913b7becee57f93)
2017-02-26 20:37:18 -08:00
Karthik Kambatla
a95d3e1fe7 YARN-6215. FairScheduler preemption and update should not run concurrently. (Tao Jie via kasha)
(cherry picked from commit 815d53506fb0c5ca029c993d6b094db2ac0ca6eb)
2017-02-26 20:31:03 -08:00
Wei-Chiu Chuang
f3cdf29af4 HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao. 2017-02-25 21:13:51 -08:00
Steve Loughran
29fe5af017
HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures.
Contributed by Steve Loughran.
2017-02-25 15:31:26 +00:00
Arpit Agarwal
98af813098 HDFS-11462. Fix occasional BindException in TestNameNodeMetricsLogger. 2017-02-24 19:53:45 -08:00
Yiqun Lin
31f9ef2203 HDFS-11433. Document missing usages of OfflineEditsViewer processors. Contributed by Yiqun Lin.
(cherry picked from commit 4a58870a0427ecdf4c5c895088b96a0b4d1c0c19)
2017-02-25 10:03:02 +08:00
Arpit Agarwal
fa86ec99a1 HDFS-11295. Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete(). Contributed by Marton Elek. 2017-02-24 15:44:19 -08:00
Xuan
5c509f5f0c HADOOP-14116:FailoverOnNetworkExceptionRetry does not wait when failover
on certain exception. Contributed by Jian He

(cherry picked from commit 289bc50e663b882956878eeaefe0eaa1ef4ed39e)
2017-02-24 11:43:22 -08:00
Akira Ajisaka
bd3c2a2ee9 HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg. 2017-02-25 04:17:17 +09:00
Xuan
19c5b3d622 YARN-6228: EntityGroupFSTimelineStore should allow configurable cache
stores. Contributed by Li Lu

(cherry picked from commit 53d372a2550c970f3dd3c49738af3c1789ae589b)
2017-02-24 11:00:16 -08:00
Harsh J
c097d0343c HADOOP-13817. Add a finite shell command timeout to ShellBasedUnixGroupsMapping. (harsh)
(cherry picked from commit e8694deb6ad180449f8ce6c1c8b4f84873c0587a)
2017-02-24 21:34:31 +05:30
Akira Ajisaka
b5a8c49cf3 HADOOP-14097. Remove Java6 specific code from GzipCodec.java. Contributed by Elek, Marton.
(cherry picked from commit 50decd36130945e184734dcd55b8912be6f4550a)
2017-02-25 00:29:40 +09:00
Steve Loughran
c6d7da2112
HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory.
(cherry picked from commit ff87ca84418a710c6dc884fe8c70947fcc6489d5)
2017-02-24 10:40:58 +00:00
Wangda Tan
650ff95e00 YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)
(cherry picked from commit 0a55bd841ec0f2eb89a0383f4c589526e8b138d4)
2017-02-23 17:22:25 -08:00
Karthik Kambatla
40bc9e7ddb YARN-6222. TestFairScheduler.testReservationMetrics is flaky. (Yufei Gu via kasha)
(cherry picked from commit 694e680d20dc07f634b539537021b09d9316601c)
2017-02-23 15:30:29 -08:00
Arpit Agarwal
9b947611a7 HADOOP-14100. Upgrade Jsch jar to latest version to fix vulnerability in old versions. Contributed by Vinayakumar B 2017-02-23 14:25:17 -08:00
Mingliang Liu
b0f8de184a HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari.
(cherry picked from commit 82607fce39151fc6ba5bced738088e2bc176dc77)
2017-02-23 14:11:08 -08:00
Wangda Tan
a82480c7d4 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-23 13:59:20 -08:00
Sunil G
801357726d YARN-6211. Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority. Contributed by Bibin A Chundatt.
(cherry picked from commit a207aa9930e7ee4f10228e2db4b4e733794eb8ea)
2017-02-23 14:25:14 +05:30
Karthik Kambatla
37edbd35f1 YARN-6194. Cluster capacity in SchedulingPolicy is updated only on allocation file reload. (Yufei Gu via kasha)
(cherry picked from commit b10e962224a8ae1c6031a05322b0cc5e564bd078)
2017-02-22 15:59:11 -08:00
Karthik Kambatla
ca7a6a7365 YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha)
(cherry picked from commit 718ad9f6ee93d4145f2bb19b7582ce4e1174feaf)
2017-02-22 15:52:44 -08:00
Daniel Templeton
4c883f331c MAPREDUCE-6825. YARNRunner#createApplicationSubmissionContext method is longer than 150 lines (Contributed by Gergely Novák via Daniel Templeton)
(cherry picked from commit 732ee6f0b58a12500198c0d934cc570c7490b520)
2017-02-22 15:40:12 -08:00
Andrew Wang
6c5fd82773 HDFS-11438. Fix typo in error message of StoragePolicyAdmin tool. Contributed by Alison Yu.
(cherry picked from commit d150f061f4ebde923fda28ea898a9606b8789758)
2017-02-22 15:16:12 -08:00
Mingliang Liu
db955f0844 HADOOP-14102. Relax error message assertion in S3A test ITestS3AEncryptionSSEC. Contributed by Mingliang Liu
(cherry picked from commit 0013090fb4340eadf147054e65a73de20a62c1c1)
2017-02-22 14:38:57 -08:00
Ming Ma
0add328c8c HDFS-11411. Avoid OutOfMemoryError in TestMaintenanceState test runs. (Manoj Govindassamy via mingma)
(cherry picked from commit cfcd527323352cf2a851c5c41f5d243d375d88d0)
2017-02-22 09:42:47 -08:00
Steve Loughran
171b18693a HADOOP-14099 Split S3 testing documentation out into its own file. Contributed by Steve Loughran. 2017-02-22 11:41:24 +00:00
Andrew Wang
4428c3da69 HDFS-11406. Remove unused getStartInstance and getFinalizeInstance in FSEditLogOp. Contributed by Alison Yu.
(cherry picked from commit 8ef7ebbc7112e1868c9b12ff1df4a40fe7afa8af)
2017-02-21 14:54:32 -08:00
Steve Loughran
6843659689 HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory. 2017-02-21 18:29:30 +00:00
Eric Payne
41d31588d1 HDFS-11404. Increase timeout on TestShortCircuitLocalRead.testDeprecatedGetBlockLocalPathInfoRpc. Contributed by Eric Badger
(cherry picked from commit a07ddef10115fd0082832f1c338b2484507a8f49)
2017-02-21 12:09:29 -06:00
Vinayakumar B
00ca9f1792 HDFS-11177. 'storagepolicies -getStoragePolicy' command should accept URI based path. (Contributed by Surendra Singh Lilhore)
(cherry picked from commit 480405063063f564ae0cdb34e0757ac3990569aa)
2017-02-21 18:15:03 +05:30
Steve Loughran
274c02d2fa HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan 2017-02-20 16:21:00 +00:00
Steve Loughran
8a88e8ea57 HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge
(cherry picked from commit b1c1f05b1dc997906390d653dfafb4f0d7e193c4)
2017-02-18 18:17:54 +00:00
Daniel Templeton
93219b6454 YARN-6125. The application attempt's diagnostic message should have a maximum size
(Contributed by Andras Piros via Daniel Templeton)

(cherry picked from commit c7a36e613053ec8b111146004b887c2f13535469)
2017-02-17 14:38:06 -08:00
Karthik Kambatla
5575328bb6 YARN-6193. FairScheduler might not trigger preemption when using DRF. (kasha)
(cherry picked from commit dbbfcf74ab44d7bfdc805b63affd0defc57182b8)
2017-02-17 14:07:47 -08:00
Daniel Templeton
10fd5627bb YARN-6188. Fix OOM issue with decommissioningNodesWatcher in the case of clusters with
large number of nodes (Contributed by Ajay Jadhav via Daniel Templeton)

(cherry picked from commit 585168cc1bdb87398d4f388f3eeccd1c71114982)
2017-02-17 13:14:37 -08:00
Xiao Chen
06386b7e78 HDFS-11410. Use the cached instance when edit logging SetAclOp, SetXAttrOp and RemoveXAttrOp.
(cherry picked from commit 02c549484a4fe6215c7f1a18d89389dbba6ea723)
2017-02-16 18:08:16 -08:00
Mingliang Liu
b23e93d465 HADOOP-14040. Use shaded aws-sdk uber-JAR 1.11.86. Contributed by Steve Loughran and Sean Mackrory
(cherry picked from commit 658702efffdf52cf5ddf8e92f959f1157c95a348)
2017-02-16 16:59:31 -08:00