Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,583 Commits 326 Branches 371 Tags
Commit Graph

406 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
c205841d49 YARN-2788. Fixed backwards compatiblity issues with log-aggregation feature that were caused when adding log-upload-time via YARN-2703. Contributed by Xuan Gong. 
(cherry picked from commit 58e9f24e0f06efede21085b7ffe36af042fa7b38)
 2014-11-03 13:20:01 -08:00
Vinod Kumar Vavilapalli
dd82a1eb75 YARN-2798. Fixed YarnClient to populate the renewer correctly for Timeline delegation tokens. Contributed by Zhijie Shen. 
(cherry picked from commit 71fbb474f531f60c5d908cf724f18f90dfd5fa9f)
 2014-11-03 12:51:30 -08:00
Jason Lowe
239c4cc572 YARN-2707. Potential null dereference in FSDownload. Contributed by Gera Shegalov 
(cherry picked from commit 7bc179f9f5e1e270fd5e6f1ad29b0f32fe371d15)
 2014-10-31 22:55:06 +00:00
Jian He
a859adcc23 YARN-2770. Added functionality to renew/cancel TimeLineDelegationToken. Contributed by Zhijie Shen 
(cherry picked from commit 1b4be918664b09272b120bc42de3e5fc02d79047)
 2014-10-31 13:17:45 -07:00
Vinod Kumar Vavilapalli
85fcdaf8d8 YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan. 
(cherry picked from commit e0233c16ebd06bb3aabeb523cd17259008e243ef)
 2014-10-30 23:03:02 -07:00
Zhijie Shen
dcb484fac0 YARN-2747. Fixed the test failure of TestAggregatedLogFormat when native I/O is enabled. Contributed by Xuan Gong. 
(cherry picked from commit ec63a3ffbd9413e7434594682fdbbd36eef7413c)
 2014-10-28 22:47:05 -07:00
Vinod Kumar Vavilapalli
0af8cc9ca2 YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He. 
(cherry picked from commit a16d022ca4313a41425c8e97841c841a2d6f2f54)
 2014-10-27 15:51:20 -07:00
Zhijie Shen
ab7a21569c YARN-2734. Skipped sub-folders in the local log dir when aggregating logs. Contributed by Xuan Gong. 
(cherry picked from commit caecd9fffe7c6216be31f3ab65349182045451fa)
 2014-10-26 13:00:26 -07:00
Vinod Kumar Vavilapalli
259db7a726 YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He. 
(cherry picked from commit 018664550507981297fd9f91e29408e6b7801ea9)
 2014-10-26 11:29:06 -07:00
Jian He
c8d48d2981 YARN-2314. Disable ContainerManagementProtocolProxy cache by default to prevent creating thousands of threads in a large cluster. Contributed by Jason Lowe 
(cherry picked from commit f44cf99599119b5e989be724eeab447b2dc4fe53)
 2014-10-24 23:06:38 -07:00
Jian He
728cc2394c YARN-1915. Fixed a race condition that client could use the ClientToAMToken to contact with AM before AM actually receives the ClientToAMTokenMasterKey. Contributed by Jason Lowe 
(cherry picked from commit 5864dd99a419b8a0fb973a4984c0f1d1e02ccf16)
 2014-10-24 22:49:19 -07:00
Zhijie Shen
a41371e173 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong. 
(cherry picked from commit f81dc3f995579c1b94b11d60e9fc6da56c8a9496)
 2014-10-24 14:13:28 -07:00
Zhijie Shen
38fa39aa17 YARN-2724. Skipped uploading a local log file to HDFS if exception is raised when opening it. Contributed by Xuan Gong. 
(cherry picked from commit e31f0a6558b106662c83e1f797216e412b6689a9)
 2014-10-24 11:16:44 -07:00
Zhijie Shen
09e422302f YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be1a3b2e4624f31f84656f4a32dadce9)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-23 22:18:27 -07:00
Zhijie Shen
7df8cbdd0f YARN-2709. Made timeline client getDelegationToken API retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit b2942762d7f76d510ece5621c71116346a6b12f6)
 2014-10-21 16:09:23 -07:00
Zhijie Shen
fcdeba204c YARN-2717. Avoided duplicate logging when container logs are not found. Contributed by Xuan Gong. 
(cherry picked from commit 171f2376d23d51b61b9c9b3804ee86dbd4de033a)
 2014-10-20 22:24:19 -07:00
Zhijie Shen
2af7f48625 YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. 
(cherry picked from commit e90718fa5a0e7c18592af61534668acebb9db51b)
 2014-10-20 13:43:03 -07:00
Zhijie Shen
6d78dc3ee4 YARN-2673. Made timeline client put APIs retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit 89427419a3c5eaab0f73bae98d675979b9efab5f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2014-10-20 12:31:33 -07:00
Vinod Kumar Vavilapalli
88b7e5ca65 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan. 
(cherry picked from commit 82567664988b673f1b819a42a4baf31cb0dcb331)
 2014-10-18 12:09:02 -07:00
Vinod Kumar Vavilapalli
24b0529a9f YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 
(cherry picked from commit 39063cd36f96e351e4a6bf0bc2b6185711d4b059)
 2014-10-17 22:04:14 -07:00
Vinod Kumar Vavilapalli
ce42210651 YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 
(cherry picked from commit e9c66e8fd2ccb658db2848e1ab911f1502de4de5)
 2014-10-17 18:27:34 -07:00
Vinod Kumar Vavilapalli
778fbcd5d3 YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan. 
(cherry picked from commit abae63caf9c53b404f2f2db7d482555484eaeaf8)
 2014-10-17 08:59:49 -07:00
Vinod Kumar Vavilapalli
8a09fdccb3 YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan. 
(cherry picked from commit b3056c266a628a65cf7ceb61b55ab3bd0a09baf2)
 2014-10-15 18:49:06 -07:00
Jian He
bd45dba807 YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit f93d2ea27e7554272845fb11d97b3564704834b3)
 2014-10-15 15:31:57 -07:00
Karthik Kambatla
e6102b1828 YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha) 
(cherry picked from commit cc93e7e683fa74eb1a7aa2b357a36667bd21086a)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-13 16:42:47 -07:00
Zhijie Shen
6da637ec12 YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong. 
(cherry picked from commit 4aed2d8e91c7dccc78fbaffc409d3076c3316289)
 2014-10-13 10:56:22 -07:00
Vinod Kumar Vavilapalli
bb6c79f76c YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 
(cherry picked from commit db7f1653198b950e89567c06898d64f6b930a0ee)
 2014-10-10 11:46:10 -07:00
Zhijie Shen
16d7144337 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong. 
(cherry picked from commit cb81bac0029fce3a9726df3523f0b692cd3375b8)
 2014-10-10 00:18:24 -07:00
Zhijie Shen
647874206c YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan. 
(cherry picked from commit e532ed8faa8db4b008a5b8d3f82b48a1b314fa6c)
 2014-10-09 18:01:13 -07:00
Vinod Kumar Vavilapalli
967c758373 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 596702a02501e9cb09aabced168027189eaf02ba)
 2014-10-09 15:47:51 -07:00
Zhijie Shen
a4522e90f6 YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen. 
(cherry picked from commit 1d4612f5ad9678c952b416e798dccd20c88f96ef)
 2014-10-09 13:03:03 -07:00
Vinod Kumar Vavilapalli
ad7baad21e YARN-2493. Added user-APIs for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 180afa2f86f9854c536c3d4ff7476880e41ac48d)
 2014-10-09 11:27:49 -07:00
Steve Loughran
088ae9c5bf YARN-913 service registry: YARN-2652 add hadoop-yarn-registry package under hadoop-yarn 2014-10-08 12:54:37 -07:00
Jian He
73a1b77223 YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du 
(cherry picked from commit ea26cc0b4ac02b8af686dfda80f540e5aa70c358)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
 2014-10-06 10:52:20 -07:00
Chris Douglas
0b2dedc42d YARN-1051. Add a system for creating reservations of cluster capacity. 
Contributed by Subru Krishnan and Carlo Curino.
(cherry picked from commit c8212bacb1b2a7e6ee83cc56f72297465ce99390)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit ebf4aae60f7a641ac7406e27d43528cace20d4ff)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-06 10:30:04 -07:00
subru
bf4394e160 YARN-2576. Making test patch pass in branch. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 90ac0be86b898aefec5471db4027554c8e1b310c)
(cherry picked from commit 5e10a13bb4759984494c6a870c7f08fb6693c9c0)
(cherry picked from commit ae0f16ccc837de2e32d7137cf48d5b17cb7c18d4)
 2014-10-06 10:29:14 -07:00
subru
f4522fd987 YARN-1708. Public YARN APIs for creating/updating/deleting reservations. 
(cherry picked from commit 3f2e3b275bcf29264a112c5d2f4eae289fe7f52c)
(cherry picked from commit c9266df40434fdde05bd3a4c75f7a6bc29db760f)
(cherry picked from commit 3a167056d10707bb3144e0d0a2a2affc169e8461)
 2014-10-06 10:29:13 -07:00
Vinod Kumar Vavilapalli
3542eef75b YARN-2468. Enhanced NodeManager to support log handling APIs (YARN-2569) for use by long running services. Contributed by Xuan Gong. 
(cherry picked from commit 34cdcaad71cad76c0874a4e5266b4074009d2ffc)
 2014-10-03 12:17:38 -07:00
Jian He
f97252e7e5 YARN-2562. Changed ContainerId#toString() to be more readable. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit eb6ce5e97c6a379caf9de0b34cc7770c4447bb1a)
 2014-10-03 11:05:17 -07:00
Zhijie Shen
20fb25cde2 YARN-2527. Fixed the potential NPE in ApplicationACLsManager and added test cases for it. Contributed by Benoy Antony. 
(cherry picked from commit 1c93025a1b370db46e345161dbc15e03f829823f)
 2014-10-02 14:58:32 -07:00
Vinod Kumar Vavilapalli
ae7a3235be YARN-2446. Augmented Timeline service APIs to start taking in domains as a parameter while posting entities and events. Contributed by Zhijie Shen. 
(cherry picked from commit 9e40de6af7959ac7bb5f4e4d2833ca14ea457614)
 2014-10-01 18:02:03 -07:00
junping_du
8871bf37fe YARN-2613. Support retry in NMClient for rolling-upgrades. (Contributed by Jian He) 2014-10-01 17:11:59 -07:00
Karthik Kambatla
c206265b7e YARN-2610. Hamlet should close table tags. (Ray Chiang via kasha) 
(cherry picked from commit f7743dd07dfbe0dde9be71acfaba16ded52adba7)
 2014-09-30 15:50:13 -07:00
Jason Lowe
ab3cab0b53 YARN-2387. Resource Manager crashes with NPE due to lack of synchronization. Contributed by Mit Desai 
(cherry picked from commit feaf139b4f327d33011e5a4424c06fb44c630955)
 2014-09-30 22:42:53 +00:00
Jian He
cb08ed1484 YARN-668. Changed NMTokenIdentifier/AMRMTokenIdentifier/ContainerTokenIdentifier to use protobuf object as the payload. Contributed by Junping Du. 
(cherry picked from commit 5391919b09ce9549d13c897aa89bb0a0536760fe)
 2014-09-26 17:53:35 -07:00
Zhijie Shen
3a2e400377 YARN-2581. Passed LogAggregationContext to NM via ContainerTokenIdentifier. Contributed by Xuan Gong. 
(cherry picked from commit c86674a3a4d99aa56bb8ed3f6df51e3fef215eba)
 2014-09-24 17:51:54 -07:00
Vinod Kumar Vavilapalli
5d251d99d6 YARN-2102. Added the concept of a Timeline Domain to handle read/write ACLs on Timeline service event data. Contributed by Zhijie Shen. 
(cherry picked from commit d78b452a4f413c6931a494c33df0666ce9b44973)
 2014-09-24 11:29:01 -07:00
Zhijie Shen
19cfe2f800 YARN-2569. Added the log handling APIs for the long running services. Contributed by Xuan Gong. 
(cherry picked from commit 5338ac416ab8ab3e7e0a7bfb4a53151fc457f673)
 2014-09-23 10:38:27 -07:00
Zhijie Shen
b8d645539d MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 10:41:55 -07:00
Allen Wittenauer
0037b8f67b YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw) 2014-09-19 14:08:04 -07:00