Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,712 Commits 326 Branches 371 Tags
Commit Graph

7712 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jian He
e7fc071906 YARN-2992. ZKRMStateStore crashes due to session expiry. Contributed by Karthik Kambatla 
(cherry picked from commit 1454efe5d4fe4214ec5ef9142d55dbeca7dab953)

(cherry picked from commit ca0349b87ab1b2d0d2b9dc93de7806d26713165c)
(cherry picked from commit 2f6be218fa41fd0f39633ec5ed0df6e0fa0f54b6)
 2015-08-30 20:42:06 -07:00
Jian He
8cdcb792a1 YARN-2340. Fixed NPE when queue is stopped during RM restart. Contributed by Rohith Sharmaks 
(cherry picked from commit 0d89859b51157078cc504ac81dc8aa75ce6b1782)

(cherry picked from commit 8b398a66ca3728f47363fc8b2fcf7e556e6bbf5a)
(cherry picked from commit 37f1789f419fa741c2fbae9855549d405e856bff)
 2015-08-30 20:40:09 -07:00
Jian He
6dfdcf094d YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 808cba3821d5bc4267f69d14220757f01cd55715)

(cherry picked from commit 9180d11b3bbb2a49127d5d25f53b38c5113bf7ea)
 2015-08-30 20:34:14 -07:00
Jason Lowe
17a23b8191 YARN-1984. LeveldbTimelineStore does not handle db exceptions properly. Contributed by Varun Saxena 
(cherry picked from commit 1ce4d33c2dc86d711b227a04d2f9a2ab696a24a1)

(cherry picked from commit 89ef49fb0814baea3640798fdf66d2ae3a550896)
 2015-08-30 20:25:39 -07:00
Colin Patrick Mccabe
c2e00e0012 HDFS-7531. Improve the concurrent access on FsVolumeList (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 3b173d95171d01ab55042b1162569d1cf14a8d43)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java

(cherry picked from commit dda1fc169db2e69964cca746be4ff8965eb8b56f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeList.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java

(cherry picked from commit ba28192f9d5a8385283bd717bca494e6981d378f)
 2015-08-30 20:11:41 -07:00
Jason Lowe
570d52e53c YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie). Contributed by Jian He 
(cherry picked from commit 0402bada1989258ecbfdc437cb339322a1f55a97)

(cherry picked from commit 173664d70f0ed3b1852b6703d32e796778fb1c78)
(cherry picked from commit 04e71db1ce9572ae0641234a02b7db5d174668fd)
 2015-08-30 20:06:07 -07:00
Gera Shegalov
1931fa5f4b MAPREDUCE-6166. Reducers do not validate checksum of map outputs when fetching directly to disk. (Eric Payne via gera) 
(cherry picked from commit af006937e8ba82f98f468dc7375fe89c2e0a7912)
(cherry picked from commit 7c7bccfc31ede6f4afc043d81c8204046148c02e)
(cherry picked from commit 2e9dca584ba25809d9c6269e5f9326e09f55ed99)
 2015-08-30 19:50:43 -07:00
Colin Patrick Mccabe
1698110acc HADOOP-11238. Update the NameNode's Group Cache in the background when possible (Chris Li via Colin P. McCabe) 
(cherry picked from commit e5a692519956aefb3a540ed0137b63cf598ac10d)

(cherry picked from commit b521d91c0f5b6d114630b6727f6a01db56dba4f1)
 2015-08-30 19:47:15 -07:00
Jian He
3d5627b563 YARN-2917. Fixed potential deadlock when system.exit is called in AsyncDispatcher. Contributed by Rohith Sharmaks 
(cherry picked from commit 614b6afea450ebb897fbb2519c6f02e13b9bd12d)

(cherry picked from commit 41f0d20fcb4fec5b932b8947a44f93345205222c)
(cherry picked from commit aa4f8184ee7aa1817a9b23b3b4e7e78da95dc031)
 2015-08-30 13:49:25 -07:00
Karthik Kambatla
6e954bc25c YARN-2910. FSLeafQueue can throw ConcurrentModificationException. (Wilfred Spiegelenburg via kasha) 
(cherry picked from commit a2e07a54561a57a83b943628ebbc53ed5ba52718)
(cherry picked from commit 1986ea8dd223267ced3e3aef69980b46e2fef740)
(cherry picked from commit 2b827a18d7b4eb41dc0095ea7277239273e7e396)
 2015-08-30 13:44:52 -07:00
Andrew Wang
2a2888f2f3 HADOOP-11368. Fix SSLFactory truststore reloader thread leak in KMSClientProvider. Contributed by Arun Suresh. 
(cherry picked from commit 74d4bfded98239507511dedb515bc6a54958d5a8)
(cherry picked from commit deaa172e7a2ab09656cc9eb431a3e68a73e0bd96)
 2015-08-30 13:41:40 -07:00
Andrew Wang
65180617fb HADOOP-11343. Overflow is not properly handled in caclulating final iv for AES CTR. Contributed by Jerry Chen. 
(cherry picked from commit 0707e4eca906552c960e3b8c4e20d9913145eca6)
(cherry picked from commit dabdd2d746d1e1194c124c5c7fe73fcc025e78d2)
 2015-08-30 13:31:43 -07:00
Karthik Kambatla
7d686eccc3 YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps. (Naganarasimha G R via kasha) 
(cherry picked from commit 799353e2c7db5af6e40e3521439b5c8a3c5c6a51)

(cherry picked from commit 25be97808b99148412c0efd4d87fc750db4d6607)
(cherry picked from commit d82bf536d44c6e7ba06a01105545b3979b731d80)
 2015-08-30 13:25:37 -07:00
Jian He
3600f30c35 YARN-2894. Fixed a bug regarding application view acl when RM fails over. Contributed by Rohith Sharmaks 
(cherry picked from commit 392c3aaea8e8f156b76e418157fa347256283c56)

(cherry picked from commit d6f3d4893d750f19dd8c539fe28eecfab2a54576)
(cherry picked from commit 61efbc1cba0c4a81b8aafb1d45c2f7b3cf7857d8)
 2015-08-30 13:08:42 -07:00
Colin Patrick Mccabe
70f33f5eff HDFS-7035. Make adding a new data directory to the DataNode an atomic operation and improve error handling (Lei Xu via Colin P. McCabe) 
(cherry picked from commit a9331fe9b071fdcdae0c6c747d7b6b306142e671)

(cherry picked from commit ec2621e907742aad0264c5f533783f0f18565880)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java

(cherry picked from commit d79a5849999cdb0bc315938b80ed71b4f2dcb720)
 2015-08-30 12:13:33 -07:00
Arpit Agarwal
46e9926ce3 HDFS-8486. Addendum patch to fix backporting bug. 
(cherry picked from commit 6c9e26609ceac5d16434cc7f4a1013e6faacd92f)
 2015-08-30 12:02:23 -07:00
Kihwal Lee
8549891cb3 HDFS-7213. processIncrementalBlockReport performance degradation. 
Contributed by Eric Payne.
(cherry picked from commit e226b5b40d716b6d363c43a8783766b72734e347)

(cherry picked from commit 946463efefec9031cacb21d5a5367acd150ef904)
(cherry picked from commit f94aa4d25c2f96faf5164e807c2c3eb031e9a1fe)
 2015-08-30 11:46:16 -07:00
Zhijie Shen
8a47d1aa55 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367de94689770f57c64bea3b244d7755f6)
(cherry picked from commit d21ef79707a0f32939d9a5af4fed2d9f5fe6f2ec)
(cherry picked from commit db325c053c6ef3ee8731d7273d1d92da7e5deee7)
 2015-08-27 19:47:43 -07:00
Jason Lowe
2f0a34b5ba YARN-2905. AggregatedLogsBlock page can infinitely loop if the aggregated log file is corrupted. Contributed by Varun Saxena 
(cherry picked from commit 0f9528b99addbb0fd9a19d84db22a8c8e934b05f)

(cherry picked from commit 38ea1419f60d2b8176dba4931748f1f0e52ca84e)
(cherry picked from commit 3877166754956bf66a8b1c81440dba2d279a1e03)
 2015-08-27 19:27:24 -07:00
Colin Patrick Mccabe
bfc0a19d6d HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full (zhaoyunjiong via cmccabe) 
(cherry picked from commit 86e3993def01223f92b8d1dd35f6c1f8ab6033f5)

(cherry picked from commit f6d1bf5ed1cf647d82e676df15587de42b1faa42)
 2015-08-27 19:18:45 -07:00
Jian He
888ab4a6e7 YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users. Contributed by Jason Lowe 
(cherry picked from commit 8a7ca13b13c0c3f008a6490cc96d4d48a051d1f7)

(cherry picked from commit ae35b0e14d3438237f4b5d3b5d5268d45e549846)
(cherry picked from commit 65acee3e19a147e5c5a8688319ab75357bdf51b5)
 2015-08-27 19:10:43 -07:00
Jian He
7f97189bcf YARN-2865. Fixed RM to always create a new RMContext when transtions from StandBy to Active. Contributed by Rohith Sharmaks 
(cherry picked from commit 9cb8b75ba57f18639492bfa3b7e7c11c00bb3d3b)

(cherry picked from commit db31ef7e7f55436bbf88c6d93e2273c4463ca9f0)
(cherry picked from commit e669974ae94c03914c9181a4481b4879fd4acc0d)
 2015-08-27 18:56:22 -07:00
Andrew Wang
0898c21e24 HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang) 
(cherry picked from commit 406c09ad1150c4971c2b7675fcb0263d40517fbf)
(cherry picked from commit 2e15754a92c6589308ccbbb646166353cc2f2456)
(cherry picked from commit 014d07de2e9b39be4b6793f0e09fcf8548570ad5)
 2015-08-27 18:48:17 -07:00
Jason Lowe
f307b426f3 YARN-2414. RM web UI: app page will crash if app is failed before any attempt has been created. Contributed by Wangda Tan 
(cherry picked from commit 81c9d17af84ed87b9ded7057cb726a3855ddd32d)

(cherry picked from commit 242fd0e39ad1c5d51719cd0f6c197166066e3288)
(cherry picked from commit a9d5acd898b34e1050a78f2d70ed62fdb82948a6)
 2015-08-27 18:36:50 -07:00
Jason Lowe
f83d898944 YARN-2816. NM fail to start with NPE during container recovery. Contributed by Zhihai Xu 
(cherry picked from commit 49c38898b0be64fc686d039ed2fb2dea1378df02)

(cherry picked from commit ad140d1fc831735fb9335e27b38d2fc040847af1)
(cherry picked from commit 85b23c323c80c5303bd0b7bdb066258792ca67d8)
 2015-08-27 18:32:59 -07:00
Jian He
81ba30211e YARN-2856. Fixed RMAppImpl to handle ATTEMPT_KILLED event at ACCEPTED state on app recovery. Contributed by Rohith Sharmaks 
(cherry picked from commit d005404ef7211fe96ce1801ed267a249568540fd)

(cherry picked from commit beb184ac580b0d89351a3f3a7201da34a26db1c1)
(cherry picked from commit 325bb33988743d60cb333002f9da60314241632e)
 2015-08-27 18:29:39 -07:00
Vinayakumar B
d6666e8d26 HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit d6050f06a3b7e049541b1cb4597c388abf00a5be)
 2015-08-27 18:18:32 -07:00
Haohui Mai
7405de4970 HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu. 
(cherry picked from commit 9e63cb4492896ffb78c84e27f263a61ca12148c8)
(cherry picked from commit 7adb6f9501e8efaa712fb70fe5a97e233622e3e1)
 2015-08-27 18:18:25 -07:00
Tao Luo
6d2f80107f HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 
(cherry picked from commit 8bfef590295372a48bd447b1462048008810ee17)

(cherry picked from commit 3827a1acdbc4f9fec3179dcafa614734b5fa31bc)
 2015-08-27 17:53:06 -07:00
Colin Patrick Mccabe
d3a7769bff HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
(cherry picked from commit ac9ab037e9a9b03e4fa9bd471d3ab9940beb53fb)

(cherry picked from commit 842a54a5f66e76eb79321b66cc3b8820fe66c5cd)
(cherry picked from commit 1aa9e34c5106c496ffd390f6b2c822d387fb1908)
 2015-08-27 17:47:39 -07:00
Ming Ma
dda8e0e328 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 
(cherry picked from commit fbd88f1062f3c4b208724d208e3f501eb196dfab)
(cherry picked from commit 516bbf1c20547dc513126df0d9f0934bb65c10c7)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java

(cherry picked from commit fb1bf424bdad20fff7ab390ce75c4bec558e7e6d)
 2015-08-27 17:38:06 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
4239513ec0 Adding release 2.6.1 to CHANGES.txt 
(cherry picked from commit 7c796fd12ca5092d74e6cc7044f28496f9884983)
 2015-08-11 13:23:27 -07:00
Arpit Agarwal
aa2303088f HDFS-8486. DN startup may cause severe data loss. (Contributed by Daryn Sharp) 2015-08-06 15:17:45 -07:00
Akira Ajisaka
0daf9dd552 Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0febe546e350a5fd4f37e717f2b8e52d6f.
 2015-04-16 16:40:31 -05:00
Wangda Tan
881084fe5c YARN-3251. Fixed a deadlock in CapacityScheduler when computing absoluteMaxAvailableCapacity in LeafQueue (Craig Welch via wangda) 2015-02-26 17:05:25 -08:00
Tsuyoshi Ozawa
5b3d9bf636 MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah. 
(cherry picked from commit 241336ca2b7cf97d7e0bd84dbe0542b72f304dc9)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-02-10 03:54:54 +09:00
Arpit Agarwal
ef212fbe7e HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. (Contributed by Arpit Agarwal) 2015-02-04 16:26:00 -08:00
Colin Patrick Mccabe
ebb515f5e7 HADOOP-11466. FastByteComparisons: do not use UNSAFE_COMPARER on the SPARC architecture because it is slower there (Suman Somasundar via Colin P. McCabe) 
(cherry picked from commit ee7d22e90ce67de3e7ee92f309c048a1d4be0bbe)
(cherry picked from commit 2c69f8cf81aed731762524e821088fd779210601)
 2015-01-26 11:19:24 -08:00
Tsz-Wo Nicholas Sze
4f5d330623 HDFS-3443. Fix NPE when namenode transition to active during startup by adding checkNNStartup() in NameNodeRpcServer. Contributed by Vinayakumar B 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
 2015-01-21 11:54:26 -08:00
Tsuyoshi Ozawa
07b3329e3f Revert "CHANGES.txt: add YARN-2977" 
This reverts commit 3419091631777ec04dd1a9e2bf24adab2496c5ce.
 2014-12-21 12:47:30 +09:00
Tsuyoshi Ozawa
ca121a78fd Revert "YARN-2977. Fixed intermittent TestNMClient failure. (Contributed by Junping Du)" 
This reverts commit d79f63120e16cb03f3f85b5d79a84202881a95a5.
 2014-12-21 12:47:28 +09:00
Tsuyoshi Ozawa
3419091631 CHANGES.txt: add YARN-2977 
(cherry picked from commit 76b0370a27c482caff9498e15ef889d37f413ce7)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-12-20 23:11:04 +09:00
Tsuyoshi Ozawa
d79f63120e YARN-2977. Fixed intermittent TestNMClient failure. (Contributed by Junping Du) 
(cherry picked from commit cf7fe583d14ebb16fc1b6e29dc2afbf67d24b9d1)
 2014-12-20 23:09:58 +09:00
Colin Patrick Mccabe
51d49ffffd HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume (cmccabe) 
(cherry picked from commit 8fa265a290792ff42635ff9b42416c634f88bdf3)
(cherry picked from commit bcaf15e2fa94db929b8cd11ed7c07085161bf950)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
 2014-12-19 13:25:33 -08:00
arp
91042c950f HDFS-7503. Namenode restart after large deletions can cause slow processReport (Arpit Agarwal) 2014-12-10 23:38:55 -08:00
Colin Patrick Mccabe
d93bfbace4 HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes (Noah Lorang via Colin P. McCabe) 
(cherry picked from commit d8352b9b2b99aa46679c5880a724ba3f0ceb41ff)
(cherry picked from commit a037d6030b5ae9422fdb265f5e4880d515be9e37)
 2014-12-09 10:59:49 -08:00
Colin Patrick Mccabe
575d7bd5b0 HDFS-4882. Prevent the Namenode's LeaseManager from looping forever in checkLeases (Ravi Prakash via Colin P. McCabe) 
(cherry picked from commit daacbc18d739d030822df0b75205eeb067f89850)
 2014-11-24 11:42:46 -08:00
cnauroth
2379932968 HDFS-7425. NameNode block deletion logging uses incorrect appender. Contributed by Chris Nauroth. 
(cherry picked from commit b8c094b0754099f13d668081b4afcf0644e325c3)
(cherry picked from commit 8d8eb8dcec94e92d94eedef883cdece8ba333087)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-21 16:58:06 -08:00
Karthik Kambatla
5913c5143b Update release date for 2.5.2 
(cherry picked from commit 1e9a3f42ff49be788ac99e5f06a7984d1d0638cc)
 2014-11-21 14:35:06 -08:00
Jonathan Eagles
a9b76813a2 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 
(cherry picked from commit c298a9a845f89317eb9efad332e6657c56736a4d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-20 23:54:59 -06:00