Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,774 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

7774 Commits

Author SHA1 Message Date
Jian He e917ef3cfe MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name properly. Contributed by Jason Lowe 
(cherry picked from commit cff05bff1f)

(cherry picked from commit 43b3b43cea)
(cherry picked from commit 013e271036554020e3d26de4fcbb5b6970f5a9c0)
 2015-08-31 15:49:24 -07:00
Jian He 8658945b3a YARN-3103. AMRMClientImpl does not update AMRM token properly. Contributed by Jason Lowe 
(cherry picked from commit 6d2bdbd7da)

(cherry picked from commit 12522fd9cb)
(cherry picked from commit f50f5ad49d3b70448647384fc5f020214cb58f10)
 2015-08-31 15:42:03 -07:00
Jian He 994c3d049a YARN-3011. Possible IllegalArgumentException in ResourceLocalizationService might lead NM to crash. Contributed by Varun Saxena 
(cherry picked from commit 4e15fc0841)

(cherry picked from commit 8100c8a68c)
(cherry picked from commit 10a6c4f349e6f32ed2a520bf669a0cbfff31c824)
 2015-08-31 15:38:45 -07:00
Andrew Wang 24da944241 HADOOP-11482. Use correct UGI when KMSClientProvider is called by a proxy user. Contributed by Arun Suresh. 
(cherry picked from commit 4b00935643)
(cherry picked from commit 7b69719455)
 2015-08-31 15:14:41 -07:00
Arpit Agarwal f81f97eb7c HDFS-7575. Upgrade should generate a unique storage ID for each volume. (Contributed by Arpit Agarwal) 
(cherry picked from commit 1d9d166c0b)
(cherry picked from commit ca8e1b0739b6653833f9bc8990ab126420703f66)
 2015-08-31 15:08:42 -07:00
Haohui Mai 31c99a3dbf HADOOP-11350. The size of header buffer of HttpServer is too small when HTTPS is enabled. Contributed by Benoy Antony. 
(cherry picked from commit 2e4df87104)
 2015-08-31 12:59:50 -07:00
Kihwal Lee ae288f7291 HDFS-7533. Datanode sometimes does not shutdown on receiving upgrade shutdown command. Contributed by Eric Payne. 
(cherry picked from commit 6bbf9fdd04)

(cherry picked from commit 33534a0c9a)
(cherry picked from commit e9a28251ee46e64e1b99b2dd54b0432bdc0b9578)
 2015-08-31 11:53:24 -07:00
cnauroth 8382e52c94 HDFS-7596. NameNode should prune dead storages from storageMap. Contributed by Arpit Agarwal. 
(cherry picked from commit ef3c3a832c)
(cherry picked from commit 75e4e55e12)
(cherry picked from commit cc637d6ece64dfeb89e78c7e9766836149e098be)
 2015-08-31 11:26:48 -07:00
Jing Zhao c8ce11d067 HDFS-7182. JMX metrics aren't accessible when NN is busy. Contributed by Ming Ma. 
(cherry picked from commit 4b589e7cfa)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSNamesystemMBean.java

(cherry picked from commit 96f0813c5d6140aabe7b2837f30971936276e689)
 2015-08-30 21:21:05 -07:00
cnauroth e6b52fc3e9 HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb. 
(cherry picked from commit 7e2d9a3242)
(cherry picked from commit f0acb7c2a2)
(cherry picked from commit 33fb7b45195a89b6464e4a1cb3fbf6bbad2bcecb)
 2015-08-30 21:17:34 -07:00
Jian He 3f8da2a9eb YARN-2997. Fixed NodeStatusUpdater to not send alreay-sent completed container statuses on heartbeat. Contributed by Chengbing Liu 
(cherry picked from commit cc2a745f7e)

(cherry picked from commit e7e6173049)
(cherry picked from commit 3c4ed2497b14140f09b3cae4959be6474c4cdc99)
 2015-08-30 20:45:45 -07:00
Tsuyoshi Ozawa 03f9ac2de7 YARN-2922. ConcurrentModificationException in CapacityScheduler's LeafQueue. Contributed by Rohith Sharmaks. 
(cherry picked from commit ddc5be48fc)
(cherry picked from commit c116743bdd)
(cherry picked from commit 3c9d26ae14625de3e9437c07eceda0d05f1985b2)
 2015-08-30 20:43:14 -07:00
Jian He e7fc071906 YARN-2992. ZKRMStateStore crashes due to session expiry. Contributed by Karthik Kambatla 
(cherry picked from commit 1454efe5d4)

(cherry picked from commit ca0349b87a)
(cherry picked from commit 2f6be218fa41fd0f39633ec5ed0df6e0fa0f54b6)
 2015-08-30 20:42:06 -07:00
Jian He 8cdcb792a1 YARN-2340. Fixed NPE when queue is stopped during RM restart. Contributed by Rohith Sharmaks 
(cherry picked from commit 0d89859b51)

(cherry picked from commit 8b398a66ca)
(cherry picked from commit 37f1789f419fa741c2fbae9855549d405e856bff)
 2015-08-30 20:40:09 -07:00
Jian He 6dfdcf094d YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 808cba3821)

(cherry picked from commit 9180d11b3b)
 2015-08-30 20:34:14 -07:00
Jason Lowe 17a23b8191 YARN-1984. LeveldbTimelineStore does not handle db exceptions properly. Contributed by Varun Saxena 
(cherry picked from commit 1ce4d33c2d)

(cherry picked from commit 89ef49fb08)
 2015-08-30 20:25:39 -07:00
Colin Patrick Mccabe c2e00e0012 HDFS-7531. Improve the concurrent access on FsVolumeList (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 3b173d9517)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java

(cherry picked from commit dda1fc169d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeList.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java

(cherry picked from commit ba28192f9d5a8385283bd717bca494e6981d378f)
 2015-08-30 20:11:41 -07:00
Jason Lowe 570d52e53c YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie). Contributed by Jian He 
(cherry picked from commit 0402bada19)

(cherry picked from commit 173664d70f)
(cherry picked from commit 04e71db1ce9572ae0641234a02b7db5d174668fd)
 2015-08-30 20:06:07 -07:00
Gera Shegalov 1931fa5f4b MAPREDUCE-6166. Reducers do not validate checksum of map outputs when fetching directly to disk. (Eric Payne via gera) 
(cherry picked from commit af006937e8)
(cherry picked from commit 7c7bccfc31)
(cherry picked from commit 2e9dca584ba25809d9c6269e5f9326e09f55ed99)
 2015-08-30 19:50:43 -07:00
Colin Patrick Mccabe 1698110acc HADOOP-11238. Update the NameNode's Group Cache in the background when possible (Chris Li via Colin P. McCabe) 
(cherry picked from commit e5a6925199)

(cherry picked from commit b521d91c0f)
 2015-08-30 19:47:15 -07:00
Jian He 3d5627b563 YARN-2917. Fixed potential deadlock when system.exit is called in AsyncDispatcher. Contributed by Rohith Sharmaks 
(cherry picked from commit 614b6afea4)

(cherry picked from commit 41f0d20fcb)
(cherry picked from commit aa4f8184ee7aa1817a9b23b3b4e7e78da95dc031)
 2015-08-30 13:49:25 -07:00
Karthik Kambatla 6e954bc25c YARN-2910. FSLeafQueue can throw ConcurrentModificationException. (Wilfred Spiegelenburg via kasha) 
(cherry picked from commit a2e07a5456)
(cherry picked from commit 1986ea8dd2)
(cherry picked from commit 2b827a18d7b4eb41dc0095ea7277239273e7e396)
 2015-08-30 13:44:52 -07:00
Andrew Wang 2a2888f2f3 HADOOP-11368. Fix SSLFactory truststore reloader thread leak in KMSClientProvider. Contributed by Arun Suresh. 
(cherry picked from commit 74d4bfded9)
(cherry picked from commit deaa172e7a)
 2015-08-30 13:41:40 -07:00
Andrew Wang 65180617fb HADOOP-11343. Overflow is not properly handled in caclulating final iv for AES CTR. Contributed by Jerry Chen. 
(cherry picked from commit 0707e4eca9)
(cherry picked from commit dabdd2d746)
 2015-08-30 13:31:43 -07:00
Karthik Kambatla 7d686eccc3 YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps. (Naganarasimha G R via kasha) 
(cherry picked from commit 799353e2c7)

(cherry picked from commit 25be97808b)
(cherry picked from commit d82bf536d44c6e7ba06a01105545b3979b731d80)
 2015-08-30 13:25:37 -07:00
Jian He 3600f30c35 YARN-2894. Fixed a bug regarding application view acl when RM fails over. Contributed by Rohith Sharmaks 
(cherry picked from commit 392c3aaea8)

(cherry picked from commit d6f3d4893d)
(cherry picked from commit 61efbc1cba0c4a81b8aafb1d45c2f7b3cf7857d8)
 2015-08-30 13:08:42 -07:00
Colin Patrick Mccabe 70f33f5eff HDFS-7035. Make adding a new data directory to the DataNode an atomic operation and improve error handling (Lei Xu via Colin P. McCabe) 
(cherry picked from commit a9331fe9b0)

(cherry picked from commit ec2621e907)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java

(cherry picked from commit d79a5849999cdb0bc315938b80ed71b4f2dcb720)
 2015-08-30 12:13:33 -07:00
Arpit Agarwal 46e9926ce3 HDFS-8486. Addendum patch to fix backporting bug. 
(cherry picked from commit 6c9e26609ceac5d16434cc7f4a1013e6faacd92f)
 2015-08-30 12:02:23 -07:00
Kihwal Lee 8549891cb3 HDFS-7213. processIncrementalBlockReport performance degradation. 
Contributed by Eric Payne.
(cherry picked from commit e226b5b40d)

(cherry picked from commit 946463efef)
(cherry picked from commit f94aa4d25c2f96faf5164e807c2c3eb031e9a1fe)
 2015-08-30 11:46:16 -07:00
Zhijie Shen 8a47d1aa55 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367d)
(cherry picked from commit d21ef79707)
(cherry picked from commit db325c053c6ef3ee8731d7273d1d92da7e5deee7)
 2015-08-27 19:47:43 -07:00
Jason Lowe 2f0a34b5ba YARN-2905. AggregatedLogsBlock page can infinitely loop if the aggregated log file is corrupted. Contributed by Varun Saxena 
(cherry picked from commit 0f9528b99a)

(cherry picked from commit 38ea1419f6)
(cherry picked from commit 3877166754956bf66a8b1c81440dba2d279a1e03)
 2015-08-27 19:27:24 -07:00
Colin Patrick Mccabe bfc0a19d6d HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full (zhaoyunjiong via cmccabe) 
(cherry picked from commit 86e3993def)

(cherry picked from commit f6d1bf5ed1)
 2015-08-27 19:18:45 -07:00
Jian He 888ab4a6e7 YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users. Contributed by Jason Lowe 
(cherry picked from commit 8a7ca13b13)

(cherry picked from commit ae35b0e14d)
(cherry picked from commit 65acee3e19a147e5c5a8688319ab75357bdf51b5)
 2015-08-27 19:10:43 -07:00
Jian He 7f97189bcf YARN-2865. Fixed RM to always create a new RMContext when transtions from StandBy to Active. Contributed by Rohith Sharmaks 
(cherry picked from commit 9cb8b75ba5)

(cherry picked from commit db31ef7e7f)
(cherry picked from commit e669974ae94c03914c9181a4481b4879fd4acc0d)
 2015-08-27 18:56:22 -07:00
Andrew Wang 0898c21e24 HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang) 
(cherry picked from commit 406c09ad11)
(cherry picked from commit 2e15754a92)
(cherry picked from commit 014d07de2e9b39be4b6793f0e09fcf8548570ad5)
 2015-08-27 18:48:17 -07:00
Jason Lowe f307b426f3 YARN-2414. RM web UI: app page will crash if app is failed before any attempt has been created. Contributed by Wangda Tan 
(cherry picked from commit 81c9d17af8)

(cherry picked from commit 242fd0e39a)
(cherry picked from commit a9d5acd898b34e1050a78f2d70ed62fdb82948a6)
 2015-08-27 18:36:50 -07:00
Jason Lowe f83d898944 YARN-2816. NM fail to start with NPE during container recovery. Contributed by Zhihai Xu 
(cherry picked from commit 49c38898b0)

(cherry picked from commit ad140d1fc8)
(cherry picked from commit 85b23c323c80c5303bd0b7bdb066258792ca67d8)
 2015-08-27 18:32:59 -07:00
Jian He 81ba30211e YARN-2856. Fixed RMAppImpl to handle ATTEMPT_KILLED event at ACCEPTED state on app recovery. Contributed by Rohith Sharmaks 
(cherry picked from commit d005404ef7)

(cherry picked from commit beb184ac58)
(cherry picked from commit 325bb33988743d60cb333002f9da60314241632e)
 2015-08-27 18:29:39 -07:00
Vinayakumar B d6666e8d26 HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit d6050f06a3b7e049541b1cb4597c388abf00a5be)
 2015-08-27 18:18:32 -07:00
Haohui Mai 7405de4970 HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu. 
(cherry picked from commit 9e63cb4492)
(cherry picked from commit 7adb6f9501e8efaa712fb70fe5a97e233622e3e1)
 2015-08-27 18:18:25 -07:00
Tao Luo 6d2f80107f HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 
(cherry picked from commit 8bfef59029)

(cherry picked from commit 3827a1acdbc4f9fec3179dcafa614734b5fa31bc)
 2015-08-27 17:53:06 -07:00
Colin Patrick Mccabe d3a7769bff HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
(cherry picked from commit ac9ab037e9)

(cherry picked from commit 842a54a5f6)
(cherry picked from commit 1aa9e34c5106c496ffd390f6b2c822d387fb1908)
 2015-08-27 17:47:39 -07:00
Ming Ma dda8e0e328 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 
(cherry picked from commit fbd88f1062)
(cherry picked from commit 516bbf1c20)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java

(cherry picked from commit fb1bf424bdad20fff7ab390ce75c4bec558e7e6d)
 2015-08-27 17:38:06 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 4239513ec0 Adding release 2.6.1 to CHANGES.txt 
(cherry picked from commit 7c796fd12c)
 2015-08-11 13:23:27 -07:00
Arpit Agarwal aa2303088f HDFS-8486. DN startup may cause severe data loss. (Contributed by Daryn Sharp) 2015-08-06 15:17:45 -07:00
Akira Ajisaka 0daf9dd552 Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0feb.
 2015-04-16 16:40:31 -05:00
Wangda Tan 881084fe5c YARN-3251. Fixed a deadlock in CapacityScheduler when computing absoluteMaxAvailableCapacity in LeafQueue (Craig Welch via wangda) 2015-02-26 17:05:25 -08:00
Tsuyoshi Ozawa 5b3d9bf636 MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah. 
(cherry picked from commit 241336ca2b)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-02-10 03:54:54 +09:00
Arpit Agarwal ef212fbe7e HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. (Contributed by Arpit Agarwal) 2015-02-04 16:26:00 -08:00
Colin Patrick Mccabe ebb515f5e7 HADOOP-11466. FastByteComparisons: do not use UNSAFE_COMPARER on the SPARC architecture because it is slower there (Suman Somasundar via Colin P. McCabe) 
(cherry picked from commit ee7d22e90c)
(cherry picked from commit 2c69f8cf81)
 2015-01-26 11:19:24 -08:00