17115 Commits

Author SHA1 Message Date
Kai Zheng
c27951e98d HADOOP-15104. AliyunOSS: change the default value of max error retry. Contributed by Jinhu Wu 2017-12-08 23:02:17 +08:00
Sammi Chen
6e24fe2998 HADOOP-15024. AliyunOSS: Support user agent configuration and include that & Hadoop version information to oss server.
(cherry picked from commit abaabb5deccc770bb38c933174a4e91c081a45c7)
2017-12-08 21:43:05 +08:00
Sammi Chen
1e0f7a1631 HADOOP-14993. AliyunOSS: Override listFiles and listLocatedStatus. Contributed Genmao Yu.
(cherry picked from commit fb809e05dca29b87d730ccd2123220d5a3b7c479)
2017-12-08 20:44:40 +08:00
Lei Xu
e309d25d2b HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu.
(cherry picked from commit 67662e2ac9e68f32b725c8118cf2be79a662fca5)
(cherry picked from commit f3143d225afec0ad95d9e4b81b91d760b5b77c52)
2017-12-07 11:24:24 -08:00
Sammi Chen
cb307d5b8e HADOOP-15080. Aliyun OSS: update oss sdk from 2.8.1 to 2.8.3 to remove its dependency on Cat-x json-lib 2017-12-07 23:11:40 +08:00
Weiwei Yang
7f354097aa YARN-7607. Remove the trailing duplicated timestamp in container diagnostics message. Contributed by Weiwei Yang. 2017-12-07 17:40:16 +08:00
Akira Ajisaka
99bc03f56b
HDFS-12889. Addendum patch to add missing file.
(cherry picked from commit 6cca5b3bcb440095f12d3eda88101fa250ac000a)
(cherry picked from commit a8be4b559ae0c09930effc136c782e33f6081b8e)
2017-12-07 13:24:10 +09:00
Xiao Chen
859f57e32c HDFS-12872. EC Checksum broken when BlockAccessToken is enabled.
(cherry picked from commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f)
2017-12-05 20:48:53 -08:00
Akira Ajisaka
c49b4f7977
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham.
(cherry picked from commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b)
(cherry picked from commit c81057d936353e385339ff0578aadaceeb465cb9)
2017-12-06 11:42:12 +09:00
Wangda Tan
243a0812b7 YARN-7381. Enable the configuration: yarn.nodemanager.log-container-debug-info.enabled by default in yarn-default.xml. (Xuan Gong via wangda)
Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87
(cherry picked from commit 0311cf05358cd75388f48f048c44fba52ec90f00)
2017-12-05 13:10:36 -08:00
Sean Mackrory
c938a54e87 HADOOP-15058. create-release site build outputs dummy shaded jars due to skipShade. Contributed by Andrew Wang.
(cherry picked from commit 5533648241b11f547ac10b64a126f3ae7a2f54c4)
(cherry picked from commit 486311cdc487966e9f0f8fff9dd58b7ce6f9c93e)
2017-12-04 15:45:37 -07:00
Chris Douglas
0d41249bec HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak
(cherry picked from commit 42307e3c3abbfe0b83d9a2581deba327435b910f)
(cherry picked from commit 2399c96ee7395bed8318820018d5285d460ac9bc)
2017-12-02 23:32:59 -08:00
Jason Lowe
419a83e133 YARN-7589. TestPBImplRecords fails with NullPointerException. Contributed by Daniel Templeton
(cherry picked from commit 25df5054216a6a76d09d9c49984f8075ebc6a197)
(cherry picked from commit f222b9d362936621e72c1460d3a9e05e2de4c78b)
2017-12-01 14:27:54 -08:00
Chris Douglas
60adf0b8bd Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak"
This reverts commit be664bd64d0dba3356f44605f870e988abfec932.
2017-12-01 11:19:25 -08:00
Chris Douglas
be664bd64d HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak
(cherry picked from commit 5304698dc8c5667c33e6ed9c4a827ef57172a723)
(cherry picked from commit 482fd5a880994f37fc3ad9e0cc2d127737b70aef)
2017-12-01 10:35:31 -08:00
Robert Kanter
7dd435cd81 HADOOP-13493. Compatibility Docs should clarify the policy for what takes precedence when a conflict is found (templedf via rkanter)
(cherry picked from commit 75a3ab88f5f4ea6abf0a56cb8058e17b5a5fe403)
(cherry picked from commit 2c2ff7da052c35f62dff3ede22bf6c1ddec6c5bb)
2017-11-30 09:27:12 -08:00
Daniel Templeton
9de795f12a YARN-7541. Node updates don't update the maximum cluster capability for resources other than CPU and memory
(cherry picked from commit 8498d287cd3beddcf8fe19625227e09982ec4be2)
(cherry picked from commit 0e73efb1ad68398ce5f7cde51466cea8c2153659)
2017-11-30 09:26:46 -08:00
Wangda Tan
60b35eccfd YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda)
Change-Id: I8a6d821abb2fe32a93cc88658ed95094e79e3e8b
2017-11-29 16:08:37 -08:00
Kihwal Lee
2e1532a712 HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev.
(cherry picked from commit 53509f295b5274059541565d7216bf98aa35347d)
2017-11-29 15:13:49 -06:00
Jason Lowe
e03725c262 YARN-6647. RM can crash during transitionToStandby due to InterruptedException. Contributed by Bibin A Chundatt
(cherry picked from commit a2c7a73e33045ce42cce19aacbe45c0421a61994)
2017-11-28 11:26:47 -06:00
Xiao Chen
b9239e0b79 HADOOP-15067. GC time percentage reported in JvmMetrics should be a gauge, not counter. Contributed by Misha Dmitriev.
(cherry picked from commit d162252d7a7223631ff66ba0210953296407e55f)
2017-11-23 09:02:15 -08:00
Daniel Templeton
d7c9b58ec7 HADOOP-14876. Create downstream developer docs from the compatibility guidelines
(cherry picked from commit bfd588789a879b8583ea4abd59f4f5843c5ac285)
(cherry picked from commit 4a70e7538678d104b4b11b8889e14f908af10c00)
2017-11-22 12:27:37 -08:00
Kihwal Lee
6c633e64a0 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla.
(cherry picked from commit 738d1a206aba05f0b4be7d633b17db7fcd1c74bc)
2017-11-22 14:20:44 -06:00
Lei Xu
781c01aa2c HDFS-12847. Regenerate editsStored and editsStored.xml in HDFS tests. Contributed by Lei (Eddy) Xu. 2017-11-22 10:23:25 -08:00
Varun Vasudev
098eb777cf YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang.
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
(cherry picked from commit 6681e7238b15835317365fab10fc50505a7f04ed)
2017-11-22 20:50:35 +05:30
Andrew Wang
3853c33466 Update maven versions in preparation for 3.0.0 release. release-3.0.0-RC0 2017-11-14 11:12:22 -08:00
Andrew Wang
3f96ecf5c3 HADOOP-15037. Add site release notes for OrgQueue and resource types.
(cherry picked from commit 8b125741659a825c71877bd1b1cb8f7e3ef26436)
2017-11-13 18:49:50 -08:00
Junping Du
ef212b8550 YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi.
(cherry picked from commit e14f03dfbf078de63126a1e882261081b9ec6778)
2017-11-13 15:28:21 -08:00
Wangda Tan
7b3cd10131 YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda)
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
2017-11-13 14:45:40 -08:00
Arun Suresh
cb48eea922 HADOOP-15036. Update LICENSE.txt for HADOOP-14840. (asuresh)
(cherry picked from commit f871b7541a5375eb117eafb9a091e4f59401231f)
2017-11-13 14:38:05 -08:00
Arun Suresh
f6de2b741e Addendum patch for Configuration fix. (Jason Lowe via asuresh)
(cherry picked from commit b07e68b02a34d272114dda4194992a847928aef8)
2017-11-13 14:05:41 -08:00
Daniel Templeton
c2f479c248 YARN-7369. Improve the resource types docs 2017-11-13 11:08:58 -08:00
Daniel Templeton
5b55a74bad YARN-7442. [YARN-7069] Limit format of resource type name (Contributed by Wangda Tan via Daniel Templeton)
(cherry picked from commit 2c6213a44280f5b3950167131293ff83f48ff56f)
2017-11-13 11:03:56 -08:00
Xiaoyu Yao
f90f772384 HADOOP-14987. Improve KMSClientProvider log around delegation token checking. Contributed by Xiaoyu Yao and Xiao Chen.
(cherry picked from commit 59d78a5088700350a5122c3a3ba5e76cd26d6a80)
2017-11-12 19:44:18 -08:00
Anu Engineer
ba4355a62f HADOOP-14901. ReuseObjectMapper in Hadoop Common. Contributed by Hanisha Koneru.
(cherry picked from commit e1b32e0959dea5f5a40055157476f9320519a618)
2017-11-12 19:44:18 -08:00
Subru Krishnan
37ae818660 YARN-7475. Fix Container log link in new YARN UI. (Sunil G via Subru).
(cherry picked from commit 3c5b46c2edd69bb238d635ae61ff91656dec23df)
(cherry picked from commit 3e26077848ed1d7461576116a9ae841d38aa3ef1)
2017-11-12 09:54:55 -08:00
bibinchundatt
bfbf94197f YARN-7406. Moving logging APIs over to slf4j in hadoop-yarn-api. Contributed by Yeliang Cang.
(cherry picked from commit 2c2b7a3672e0744ce6a77a117cedefba04fed603)
2017-11-11 10:38:47 +05:30
Chris Douglas
7f80c2f293 HADOOP-8522. ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used. Contributed by Mike Percy
(cherry picked from commit 796a0d3a5c661f0c3b23af9c0db2d8f3db83c322)
2017-11-10 17:45:15 -08:00
Xiao Chen
65d7968afd HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev.
(cherry picked from commit 3c6adda291745c592741b87cd613214ae11887e4)
2017-11-09 21:06:43 -08:00
Arun Suresh
72d7bedac9 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh)
(cherry picked from commit 5eb7dbe9b31a45f57f2e1623aa1c9ce84a56c4d1)
2017-11-09 15:16:55 -08:00
Sean Mackrory
abaf90792e YARN-7465. start-yarn.sh fails to start ResourceManagers unless running as root.
(cherry picked from commit 1883a0024949f7946264a4d7649b03fd1881567a)
2017-11-09 14:49:54 -07:00
Daniel Templeton
38f5d4cbba YARN-7413. Support resource type in SLS (Contributed by Yufei Gu via Daniel Templeton)
Change-Id: Ic0a897c123c5d2f57aae757ca6bcf1dad7b90d2b
(cherry picked from commit ba8136615ab66c450884614557eddc6509d63b7c)
2017-11-09 13:39:08 -08:00
Yufei Gu
ddef7d05a4 YARN-6505. Define the strings used in SLS JSON input file format. (Contributed by Gergely Novak)
(cherry picked from commit 99880d0a16727c770da053464da87960c5b02065)
2017-11-09 13:38:57 -08:00
Konstantinos Karanasos
e55dc1700d YARN-7437. Rename PlacementSet and SchedulingPlacementSet. (Wangda Tan via kkaranasos)
(cherry picked from commit ac4d2b1081d8836a21bc70e77f4e6cd2071a9949)
2017-11-09 13:07:10 -08:00
Robert Kanter
82abc7224f YARN-7386. Duplicate Strings in various places in Yarn memory (misha@cloudera.com via rkanter)
(cherry picked from commit a2c150a7369cc629bbfaa2dfa3a8495b6f9c42e2)
2017-11-09 12:13:09 -08:00
Daniel Templeton
933a09e886 YARN-7143. FileNotFound handling in ResourceUtils is inconsistent
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
(cherry picked from commit db82a41d94872cea4d0c1bb1336916cebc2faeec)
2017-11-09 11:59:16 -08:00
Wangda Tan
51d3693a9c YARN-7205. Log improvements for the ResourceUtils. (Sunil G via wangda)
Change-Id: I0f5b7a7f68ec5d3e1d52211f83fdd089bc0bfd37
(cherry picked from commit 8bcc49e6771ca75f012211e27870a421b19233e7)
2017-11-09 11:59:16 -08:00
Haibo Chen
91a7f8d246 YARN-7388. TestAMRestart should be scheduler agnostic.
(cherry picked from commit a1382a18dff8a70aa25240d6fbba6e22832a7679)
2017-11-09 10:50:49 -08:00
bibinchundatt
4370b70e63 YARN-7454. RMAppAttemptMetrics#getAggregateResourceUsage can NPE due to double lookup. Contributed by Jason Lowe.
(cherry picked from commit 0a72c2f56c37063609de72eef1f74632890c048a)
2017-11-09 21:08:47 +05:30
Akira Ajisaka
9051d1b851
HDFS-12732. Correct spellings of ramdomly to randomly in log. Contributed by hu xiaodong.
(cherry picked from commit 3a3566e1d1ab5f78cfb734796b41802fe039196d)
2017-11-09 15:18:32 +09:00