Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,987 Commits 326 Branches 371 Tags
Commit Graph

13987 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sidharta S
3ecbfff937 YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe 2017-10-04 14:05:01 -07:00
Wangda Tan
175ff0585d YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (wangda via Jian He) 
Change-Id: I582566e09d5ef67e9cc9e9805231ccbabe57c3d0
 2017-10-03 12:19:11 -07:00
Steve Loughran
28e05dc1da
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. 
Contributed by Steve Loughran
 2017-10-03 20:06:45 +01:00
Wangda Tan
42e873d4ea YARN-6625. yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment. (Yufei Gu) 2017-10-03 12:04:56 -07:00
Wangda Tan
598329e6f8 YARN-6550. Capture launch_container.sh logs to a separate log file. (Suma Shivaprasad via wangda) 
Change-Id: I66c9b50ae3955dda0b5bf0674a9bfb728db27acb
 2017-10-03 09:46:35 -07:00
Wangda Tan
8beae14a07 YARN-7259. Add size-based rolling policy to LogAggregationIndexedFileController. (xgong via wangda) 
Change-Id: Ifaf82c0aee6b73b9b6ebf103aa72e131e3942f31
(cherry picked from commit 280080fad01304c85a9ede4d4f7b707eb36c0155)
 2017-10-02 15:36:11 -07:00
Steve Loughran
1eecf8a97c
HADOOP-14768. Honoring sticky bit during Deletion when authorization is enabled in WASB 
Contributed by Varada Hemeswari
 2017-10-02 14:28:59 +01:00
Xiao Chen
78d6dd414a HADOOP-14095. Document caveats about the default JavaKeyStoreProvider in KMS. 
(cherry picked from commit d7ecac379a02876919d3e6081d42f0937f54e664)

Conflicts:
	hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
 2017-09-29 19:34:50 -07:00
Yufei Gu
e84d508c1b YARN-6333. Improve doc for minSharePreemptionTimeout, fairSharePreemptionTimeout and fairSharePreemptionThreshold. (Contributed by Chetna Chaudhari via Yufei Gu) 
(cherry picked from commit 66c417167ae0d24467360ae3b222a10f00fa2700)
 2017-09-29 18:08:55 -07:00
Wangda Tan
5fa5ba7f4d YARN-6509. Add a size threshold beyond which yarn logs will require a force option. (Xuan Gong via wangda) 
Change-Id: I755fe903337d4ff9ec35dae5b9cce638794e1d0f
(cherry picked from commit ec2ae3060a807c8754826af2135a68c08b2e4f13)
(cherry picked from commit d58d75ca501c6d1bb20605c1c964c0d351012763)
 2017-09-29 16:53:16 -07:00
Eric Payne
b0ba31c518 YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe. 
(cherry picked from commit 7f6118f918e0fbe81cb64e8b9f6959b8d66679f6)
 2017-09-29 10:50:21 -05:00
Jason Lowe
c0bfb385a8 YARN-7260. yarn.router.pipeline.cache-max-size is missing in yarn-default.xml. Contributed by Jason Lowe 2017-09-29 09:53:05 -05:00
Steve Loughran
a1afc6aa59
HADOOP-13345 S3Guard: Improved Consistency for S3A. 
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.

(cherry-picked from/based on commit 621b43e254afaff708cd6fc4698b29628f6abc33)
 2017-09-29 11:29:22 +01:00
Steve Loughran
a641bcec01
HADOOP-14845. Azure wasb: getFileStatus not making any auth check. 
Contributed by Sivaguru Sankaridurg
 2017-09-29 11:16:31 +01:00
Chris Trezzo
cba1891b61 YARN-7250. Update Shared cache client api to use URLs. 
(cherry picked from commit c114da5e64d14b1d9e614081c4171ea0391cb1aa)
 2017-09-28 15:32:18 -07:00
Jason Lowe
11ac10fe1d HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly calculated. Contributed by Hanisha Koneru 
(cherry picked from commit 6f789fe05766a61b12ca10df3f26ee354eac84aa)
 2017-09-28 16:40:18 -05:00
Subru Krishnan
ffcf5ba1ce YARN-6962. Add support for updateContainers when allocating using FederationInterceptor. (Botong Huang via Subru). 
(cherry picked from commit ca669f9f8bc7abe5b7d4648c589aa1756bd336d1)
 2017-09-28 13:11:32 -07:00
Subru Krishnan
d6da014f67 YARN-6955. Handle concurrent register AM requests in FederationInterceptor. (Botong Huang via Subru). 
(cherry picked from commit c61f2c419830e40bb47fb2b1fe1f7d6109ed29a9)
 2017-09-28 13:11:19 -07:00
Jason Lowe
913a64e4c9 YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh 
(cherry picked from commit 85d81ae58ec4361a944c84753a900460a0888b9b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
 2017-09-28 14:25:01 -05:00
Arun Suresh
53c1115908 YARN-6691. Update YARN daemon startup/shutdown scripts to include Router service. (Giovanni Matteo Fumarola via asuresh) 2017-09-28 11:41:43 -07:00
Xuan
c143708acf YARN-7257. AggregatedLogsBlock reports a bad 'end' value as a bad 
'start' value. Contributed by Jason Lowe

(cherry picked from commit 28c4957fccebe2d7e63ec9fe9af58313b4f21d4f)
(cherry picked from commit 81e78727938a1c0369f164849cc3a08add31be46)
 2017-09-27 16:04:06 -07:00
Rohith Sharma K S
c570ddaa66 YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G. 2017-09-27 11:49:59 +05:30
Akira Ajisaka
61f41b071d
HADOOP-14822. hadoop-project/pom.xml is executable. Contributed by Ajay Kumar. 
(cherry picked from commit dd378775df5bd79e23c12120535404c4ca0fb386)
 2017-09-27 14:27:13 +09:00
Yiqun Lin
2e2f8e42cf HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit e9b790db9de7c3fbe451bcdc3fd928a27a0d8b4e)
 2017-09-27 10:45:44 +08:00
Daryn Sharp
7f3ed5f61e [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:21:37 -05:00
Steve Loughran
27a41848ac
HADOOP-14890 Move up to AWS SDK 1.11.199. 
Contributed by Steve Loughran.
 2017-09-26 20:11:10 +01:00
Jason Lowe
c4de765fcb Revert "YARN-6570. No logs were found for running application, running" 
This reverts commit 6aaca3d08edf02925bcc0b4521b21b6bf9aacb99.
 2017-09-26 10:44:16 -05:00
Jason Lowe
0aac5af6e1 HADOOP-14891. Remove references to Guava Objects.toStringHelper. Contributed by Jonathan Eagles 2017-09-26 10:08:06 -05:00
Rohith Sharma K S
c27f682032 YARN-65. Reduce RM app memory footprint once app has completed. Contributed by Manikandan R. 
(cherry picked from commit 06e5a7b5cf141420d3a411088b87acba72e68cad)
 2017-09-26 14:50:34 +05:30
Akira Ajisaka
b017af2391
HADOOP-14893. WritableRpcEngine should use Time.monotonicNow. Contributed by Chetna Chaudhari. 
(cherry picked from commit d08b8c801a908b4242e7b21a54f3b1e4072f1eae)
 2017-09-26 16:47:50 +09:00
Wangda Tan
179d8dc83c YARN-7118. AHS REST API can return NullPointerException. (Billie Rinaldi via wangda) 
Change-Id: I16ca93385ab80e3680d4d173b8d1b79929ea7174
 2017-09-25 21:41:00 -07:00
Arun Suresh
e7199cb494 YARN-7240. Add more states and transitions to stabilize the NM Container state machine. (Kartheek Muthyala via asuresh) 
(cherry picked from commit df800f6cf3ea663daf4081ebe784808b08d9366d)
 2017-09-25 16:36:58 -07:00
Anu Engineer
17b17aed0f HADOOP-14901. ReuseObjectMapper in Hadoop Common. Contributed by Hanisha Koneru. 2017-09-25 13:44:51 -07:00
Jason Lowe
09be0acae0 HADOOP-14881. LoadGenerator should use Time.monotonicNow() to measure durations. Contributed by Bharat Viswanadham 
(cherry picked from commit ac05a51bbb2a3fad4e85f9334a3408571967900a)
 2017-09-25 15:38:52 -05:00
Akira Ajisaka
c0dacda546
YARN-7153. Remove duplicated code in AMRMClientAsyncImpl.java. Contributed by Sen Zhao. 
(cherry picked from commit e928ee583c5a1367e24eab34057f8d8496891452)
 2017-09-26 02:05:35 +09:00
Akira Ajisaka
6004fb4676
HADOOP-14892. MetricsSystemImpl should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit 3a10367a17b99e2c3035ec54048f94f334d93227)
 2017-09-26 01:48:24 +09:00
Anu Engineer
de5744c903 HDFS-12531. Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2. Contributed by Bharat Viswanadham. 2017-09-23 11:16:50 -07:00
Anu Engineer
d0c0bd573b Revert "YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi." 
This reverts commit de1d747644727ed7c469570bad56400741fb034e.
 2017-09-23 11:12:16 -07:00
Junping Du
de1d747644 YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi. 
(cherry picked from commit 4002bf0a9e7e3619a3bdcff071f0369cbf2873ad)
 2017-09-22 13:59:03 -07:00
Jason Lowe
2c10abaddd MAPREDUCE-6967. gridmix/SleepReducer should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari 
(cherry picked from commit b133dc5700b66318ac6d80e463b049723c511f37)
 2017-09-22 15:30:53 -05:00
Arpit Agarwal
09290659e7 HDFS-12530. Processor argument in Offline Image Viewer should be case insensitive. Contributed by Hanisha Koneru. 2017-09-22 09:48:10 -07:00
Jason Lowe
b94acb6efb MAPREDUCE-6965. QuasiMonteCarlo should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari 
(cherry picked from commit 908d8e914ef55aca69cb8e725e62bda29d1073cc)
 2017-09-22 09:37:09 -05:00
Akira Ajisaka
d80e2f2eae
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit c71d13796039815350d3c73c51f9fe48531af0fa)
 2017-09-22 15:10:19 +09:00
Akira Ajisaka
9d4de9c556
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergely Novák. 
(cherry picked from commit 53be075241f1ba92bfe47e89c2dfc3f0664e2578)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
 2017-09-22 13:29:17 +09:00
Carlo Curino
7cd9018b13 [YARN FEDERATION BACKPORT] Fixing more Java 1.7 compilation issues 2017-09-21 18:19:36 -07:00
Carlo Curino
88b32edb8f YARN-7010. Federation: routing REST invocations transparently to multiple RMs (part 2 - getApps). (Contributed by Giovanni Matteo Fumarola via curino) 
(cherry picked from commit cc8893edc0b7960e958723c81062986c12f06ade)
 2017-09-21 18:09:44 -07:00
Arun Suresh
2aacb9d3fb YARN-6923. Metrics for Federation Router. (Giovanni Matteo Fumarola via asuresh) 
(cherry picked from commit ae8fb13b312b30de50d65b5450b565d50d690e9e)
 2017-09-21 18:09:37 -07:00
Arun Suresh
ac090b38ad YARN-5603. Metrics for Federation StateStore. (Ellen Hui via asuresh) 
(cherry picked from commit 75abc9a8e2cf1c7d2c574ede720df59421512be3)
 2017-09-21 18:09:30 -07:00
Subru Krishnan
261f769d79 YARN-6900. ZooKeeper based implementation of the FederationStateStore. (Íñigo Goiri via Subru). 
(cherry picked from commit de462da04e167a04b89ecf0f40d464cf39dc6549)
 2017-09-21 18:09:23 -07:00
Subru Krishnan
9ad067efe9 YARN-6996. Change javax.cache library implementation from JSR107 to Apache Geronimo. (Ray Chiang via Subru). 
(cherry picked from commit 18f3603bce37e0e07c9075811b1179afc2c227eb)
 2017-09-21 18:09:16 -07:00