Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,703 Commits 326 Branches 371 Tags
Commit Graph

7703 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Uma Maheswara Rao G
157f86fca5 HADOOP-11133. Should trim the content of keystore password file for JavaKeyStoreProvider (Yi Liu via umamahesh) 
(cherry picked from commit 8d7c54967d5a08cdff75a782998ee89e7537b5c4)
 2014-10-10 23:04:12 -07:00
Vinod Kumar Vavilapalli
8dacf89b3e YARN-2501. Enhanced AMRMClient library to support requests against node labels. Contributed by Wangda Tan. 
(cherry picked from commit a5ec3d080978a67837946a991843a081ea712539)
 2014-10-10 19:59:30 -07:00
Vinod Kumar Vavilapalli
bb6c79f76c YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 
(cherry picked from commit db7f1653198b950e89567c06898d64f6b930a0ee)
 2014-10-10 11:46:10 -07:00
Zhijie Shen
16d7144337 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong. 
(cherry picked from commit cb81bac0029fce3a9726df3523f0b692cd3375b8)
 2014-10-10 00:18:24 -07:00
cnauroth
281041a57e YARN-2662. TestCgroupsLCEResourcesHandler leaks file descriptors. Contributed by Chris Nauroth. 
(cherry picked from commit d3afd730acfa380ab5032be5ee296c5d73744518)
(cherry picked from commit b81641a31050b6a5cf5db3af636125e0617ffd56)
 2014-10-09 22:49:00 -07:00
cnauroth
308e336853 MAPREDUCE-6123. TestCombineFileInputFormat incorrectly starts 2 MiniDFSCluster instances. Contributed by Chris Nauroth. 
(cherry picked from commit 777877d079ecb025c293deac741800800e413179)
(cherry picked from commit f6278cbac50522302ee7ad3cbbe21d0aeefa76d8)
 2014-10-09 22:40:34 -07:00
cnauroth
4573bbe33d MAPREDUCE-6122. Fix CHANGES.txt. 
(cherry picked from commit 2f0166db560b180e8d07c10ba9ad4bfaf2aa7b36)
(cherry picked from commit 61be28f501ba3de0915c165c9d43fa38463722c0)
 2014-10-09 22:33:54 -07:00
cnauroth
3fe4e0d3df MAPREDUCE-6122. TestLineRecordReader may fail due to test data files checked out of git with incorrect line endings. Contributed by Chris Nauroth. 
(cherry picked from commit 684170d5efdc11bcd9967168f44a428d1a72af3c)
(cherry picked from commit 85a2b907407c7719a31c64d3747810263b3ec57d)
 2014-10-09 22:30:25 -07:00
Zhijie Shen
ce704583a7 YARN-2671. Fix the Jira number in the change log. 
(cherry picked from commit 5b12df6587eb4f37d09c9ffc35a0ea59694df831)
 2014-10-09 21:00:38 -07:00
Zhijie Shen
647874206c YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan. 
(cherry picked from commit e532ed8faa8db4b008a5b8d3f82b48a1b314fa6c)
 2014-10-09 18:01:13 -07:00
Colin Patrick Mccabe
14080ea3d2 HADOOP-11184. Update Hadoop's lz4 to r123 (cmccabe) 
(cherry picked from commit d8d628d1e0c42a32e2079ed430e8817996bbe4c4)
(cherry picked from commit 8addbe2b8eaf9fabbb0b6dc0dd3351a0145e0cae)
 2014-10-09 17:12:48 -07:00
Vinod Kumar Vavilapalli
967c758373 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 596702a02501e9cb09aabced168027189eaf02ba)
 2014-10-09 15:47:51 -07:00
Andrew Wang
2ff182c890 HADOOP-11174. Delegation token for KMS should only be got once if it already exists. (Yi Liu via wang) 
(cherry picked from commit 8d941144952a77a714cd4368cba21b01904926e9)
 2014-10-09 13:21:34 -07:00
Zhijie Shen
a4522e90f6 YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen. 
(cherry picked from commit 1d4612f5ad9678c952b416e798dccd20c88f96ef)
 2014-10-09 13:03:03 -07:00
Andrew Wang
e7b215dfa7 HADOOP-11178. Fix findbugs exclude file. (Arun Suresh via wang) 
(cherry picked from commit d7b647f0ee2c918143ff71abd9ce8b89717b2fc5)
 2014-10-09 11:59:15 -07:00
cnauroth
754718aed5 HDFS-7195. Update user doc of secure mode about Datanodes don't require root or jsvc. Contributed by Chris Nauroth. 
(cherry picked from commit 4e544c84e9e2cf3fb031cd8797bac4e8bcc7eca9)
 2014-10-09 11:57:00 -07:00
Mayank Bansal
cf0ad82c5a YARN-2598 GHS should show N/A instead of null for the inaccessible information (Zhijie Shen via mayank) 2014-10-09 11:39:25 -07:00
Vinod Kumar Vavilapalli
ad7baad21e YARN-2493. Added user-APIs for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 180afa2f86f9854c536c3d4ff7476880e41ac48d)
 2014-10-09 11:27:49 -07:00
cnauroth
cf7eaf28e9 HADOOP-11175. Fix several issues of hadoop security configuration in user doc. Contributed by Yi Liu. 
(cherry picked from commit 1123a06e3658f26228f1eedaed796aca8efc7bad)
(cherry picked from commit 5af4c021cc3558de0ac8610170a1eb749e0171da)
 2014-10-09 11:12:58 -07:00
Kihwal Lee
0280e9aee0 HDFS-7217. Better batching of IBRs. Contributed by Kihwal Lee. 
(cherry picked from commit db71bb54bcc75b71c5841b25ceb03fb0218c6d4f)
 2014-10-09 12:11:42 -05:00
Aaron T. Myers
f86c9c6c71 HADOOP-11161. Expose close method in KeyProvider to give clients of Provider implementations a hook to release resources. Contribued by Arun Suresh. 
(cherry picked from commit d9556e873ef4d3e68c4f0c991f856d1faa747f07)
(cherry picked from commit 3a2565c7be80cf6e9cdfec0f5460ed8ed2252768)
 2014-10-08 18:00:37 -07:00
Zhijie Shen
ab448565c9 HADOOP-11179. Java untar should handle the case that the file entry comes without its parent directory entry. Contributed by Craig Welch. 
(cherry picked from commit a1690519317068d9855174752d22ff45f0e4c962)
 2014-10-08 15:39:30 -07:00
Colin Patrick Mccabe
204148f0d4 HADOOP-10404. Some accesses to DomainSocketWatcher#closed are not protected by the lock (cmccabe) 2014-10-08 13:58:28 -07:00
Kihwal Lee
522b6505f4 HDFS-7203. Concurrent appending to the same file can cause data 
corruption. Contributed by Kihwal Lee.
(cherry picked from commit 853cb704edf54207313c0e70c9c375212d288b60)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-08 15:10:03 -05:00
Steve Loughran
088ae9c5bf YARN-913 service registry: YARN-2652 add hadoop-yarn-registry package under hadoop-yarn 2014-10-08 12:54:37 -07:00
Jian He
7ec65c8617 YARN-2649. Fixed TestAMRMRPCNodeUpdates test failure. Contributed by Ming Ma 
(cherry picked from commit e16e25ab1beac89c8d8be4e9f2a7fbefe81d35f3)
 2014-10-08 11:03:28 -07:00
Andrew Wang
2c27b7b26f HADOOP-11169. Fix DelegationTokenAuthenticatedURL to pass the connection Configurator to the authenticator. (Arun Suresh via wang) 
(cherry picked from commit b2f619752355d4ef6733935c020f57c8a26d82e1)
 2014-10-07 14:51:52 -07:00
Kihwal Lee
06587e4876 Fix version in pom.xml from HADOOP-11007 in branch-2.6 2014-10-07 16:40:01 -05:00
Kihwal Lee
0a36b6808f Missing files from the previous commit of HADOOP-11007 2014-10-07 16:32:55 -05:00
Kihwal Lee
3f200fddfd HADOOP-11007. Reinstate building of ant tasks support. Contributed by 
Jason Lowe.
(cherry picked from commit 99d7a452be91b3aefe9bcde9b51893ff6719742e)
 2014-10-07 16:24:23 -05:00
Jian He
5ff984f33f YARN-1857. CapacityScheduler headroom doesn't account for other AM's running. Contributed by Chen He and Craig Welch 
(cherry picked from commit 30d56fdbb40d06c4e267d6c314c8c767a7adc6a3)
 2014-10-07 13:48:46 -07:00
Jason Lowe
e835cb5c86 MAPREDUCE-6029. TestCommitterEventHandler fails in trunk. Contributed by Mit Desai 
(cherry picked from commit 2e789eb2261fa607489d8ae630fb77e8147d0266)
 2014-10-07 14:13:12 +00:00
Haohui Mai
1bd45fa165 HADOOP-11168. Remove duplicated entry dfs.webhdfs.enabled in the user doc. Contributed by Yi Liu. 2014-10-06 22:08:01 -06:00
Jian He
efc482d1f7 YARN-2644. Fixed CapacityScheduler to return up-to-date headroom when AM allocates. Contributed by Craig Welch 
(cherry picked from commit 519e5a7dd2bd540105434ec3c8939b68f6c024f8)
 2014-10-06 15:52:05 -07:00
Jian He
73a1b77223 YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du 
(cherry picked from commit ea26cc0b4ac02b8af686dfda80f540e5aa70c358)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
 2014-10-06 10:52:20 -07:00
Chris Douglas
0b2dedc42d YARN-1051. Add a system for creating reservations of cluster capacity. 
Contributed by Subru Krishnan and Carlo Curino.
(cherry picked from commit c8212bacb1b2a7e6ee83cc56f72297465ce99390)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit ebf4aae60f7a641ac7406e27d43528cace20d4ff)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-06 10:30:04 -07:00
subru
114f09226e YARN-2611. Fixing jenkins findbugs warning and TestRMWebServicesCapacitySched for branch YARN-1051. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit c47464aba407d1dafe10be23fe454f0489cc4367)
(cherry picked from commit a2986234be4e02f9ccb589f9ff5f7ffb28bc6400)
(cherry picked from commit fb5e9df7fd09a5cbc2c98bc9c875faf6a3b8f7c0)
 2014-10-06 10:29:14 -07:00
subru
bf4394e160 YARN-2576. Making test patch pass in branch. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 90ac0be86b898aefec5471db4027554c8e1b310c)
(cherry picked from commit 5e10a13bb4759984494c6a870c7f08fb6693c9c0)
(cherry picked from commit ae0f16ccc837de2e32d7137cf48d5b17cb7c18d4)
 2014-10-06 10:29:14 -07:00
subru
256a951049 MAPREDUCE-6103.Adding reservation APIs to MR resource manager delegate. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit aa92dd45f2d8c89a8a17ad2e4449aa3ff08bc53a)
(cherry picked from commit 3f282762d1afc916de9207d3adeda852ca344853)
(cherry picked from commit 30a370e70504a4cc3222da0dc706c871fcebfa78)
 2014-10-06 10:29:13 -07:00
subru
d244b2ae3a YARN-2080. Integrating reservation system with ResourceManager and client-RM protocol. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 8baeaead8532898163f1006276b731a237b1a559)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
(cherry picked from commit 6261f7cc69a0eb3eebc9898c7599c7c20f432b4e)
(cherry picked from commit cbfbdf60d69e70c1821d5b3d343a4f5c0c2a410f)
 2014-10-06 10:29:13 -07:00
subru
8e92b3ec9d YARN-1712. Plan follower that synchronizes the current state of reservation subsystem with the scheduler. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 169085319b8b76641f8b9f6840a3fef06d221e2b)
(cherry picked from commit 3418c56bcf4bbddaf483bdaa1a15a8bbc4039bfe)
(cherry picked from commit fec639cda14f56b87052512c166c8e42a0ba6cf4)
 2014-10-06 10:29:13 -07:00
carlo curino
56972123ff YARN-1711. Policy to enforce instantaneous and over-time quotas on user reservation. Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit c4918cb4cb5a267a8cfd6eace28fcfe7ad6174e8)
(cherry picked from commit b6df0dddcdafd7ec67c76ea92aea3ff3e94db247)
(cherry picked from commit 6bfdaf06c496c9db87c054977617c16e372325ac)
 2014-10-06 10:29:13 -07:00
carlo curino
4ee027b9d6 YARN-1710. Logic to find allocations within a Plan that satisfy user ReservationRequest(s). Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit aef7928899b37262773f3dc117157bb746bf8918)
(cherry picked from commit f66ffcf832235e0da0bb050fff08e248b547c360)
(cherry picked from commit 6a3c167175ba98e002b6a67ce73355974c1263da)
 2014-10-06 10:29:13 -07:00
subru
056b7f5799 YARN-1709. In-memory data structures used to track resources over time to enable reservations. 
(cherry picked from commit 0d8b2cd88b958b1e602fd4ea4078ef8d4742a7c3)
(cherry picked from commit cf4b34282aafee9f6b09d3433c4de1ae4b359168)
(cherry picked from commit 63250ef9d6410f5db20dfe421870c655e2c0fd65)
 2014-10-06 10:29:13 -07:00
subru
f4522fd987 YARN-1708. Public YARN APIs for creating/updating/deleting reservations. 
(cherry picked from commit 3f2e3b275bcf29264a112c5d2f4eae289fe7f52c)
(cherry picked from commit c9266df40434fdde05bd3a4c75f7a6bc29db760f)
(cherry picked from commit 3a167056d10707bb3144e0d0a2a2affc169e8461)
 2014-10-06 10:29:13 -07:00
carlo curino
950da32b4c YARN-2475. Logic for responding to capacity drops for the ReservationSystem. Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit f83a07f266f2c5e6eead554d8a331ed7e75e10d5)
(cherry picked from commit 1c6950354f3c35a7824770dc251d5aec3be4876a)
(cherry picked from commit b81f571e6064cee119bcbfed090ab646cdd927a7)
 2014-10-06 10:29:12 -07:00
carlo curino
5ac08c5398 YARN-1707. Introduce APIs to add/remove/resize queues in the CapacityScheduler. Contributed by Carlo Curino and Subru Krishnan 
(cherry picked from commit aac47fda7fecda9fc18ade34d633eca895865a70)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
(cherry picked from commit eb3e40b833b9d82c1556843f960194dc42e482f3)
(cherry picked from commit cf5ef00b96cba3ee1e971b7280f987fa65044ef5)
 2014-10-06 10:29:12 -07:00
cnauroth
08ccd30063 HADOOP-11163. Correct CHANGES.txt. 
(cherry picked from commit bbb3b1acc0e8ce10b8689ded2be73b0965f09e52)
(cherry picked from commit 1329d1e701a52bb99759a9f0176865674887a90f)
 2014-10-06 09:16:31 -07:00
cnauroth
29c203585e HADOOP-11163. MetricsSystemImpl may miss a registered source. Contributed by Chuan Liu. 
(cherry picked from commit 69c9af91031f345d9511618877646b8756b17863)
(cherry picked from commit a828ef15b0e2c63dac6854910bc9ae3b5f5ef755)
 2014-10-06 09:16:22 -07:00
Haohui Mai
f567f3c833 HDFS-7172. Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. 
(cherry picked from commit 5db04a5a1a27d4199bfcc1c31a001ff4db32810b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-06 09:15:41 -07:00