Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,177 Commits 326 Branches 371 Tags
Commit Graph

9177 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan
ceefabc44a YARN-4313. Race condition in MiniMRYarnCluster when getting history 
server address. Contributed by Jian He

(cherry picked from commit 7412ff48eeb967c972c19c1370c77a41c5b3b81f)
(cherry picked from commit b505bf9e582f5bca59be28296a4b4a640b3389eb)
 2015-10-29 17:38:09 -07:00
Jonathan Eagles
3287a50cfe YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles) 
(cherry picked from commit c293c58954cdab25c8c69418b0e839883b563fa4)
 2015-10-29 16:45:31 -05:00
Jason Lowe
7499981d3c MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d58443a7431ff33ba49ee19278ebe42b)
(cherry picked from commit b9865cbf3d435903835afa23c74c5e2ecfbba1bc)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-10-29 18:29:45 +00:00
Jason Lowe
c43cc8fd67 YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena 
(cherry picked from commit e2267de2076245bd9857f6a30e3c731df017fef8)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DockerContainerExecutor.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
 2015-10-29 16:48:57 +00:00
Kihwal Lee
13b256ed22 HDFS-9083. Replication violates block placement policy. Contributed by Rushabh Shah. 2015-10-28 14:49:41 -05:00
Xuan
336be63dad YARN-2859. ApplicationHistoryServer binds to default port 8188 in 
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli

(cherry picked from commit 27414dac66f278b61fc23762204b01a1c508178a)
(cherry picked from commit 9ce5069d16edefb0b2a24d6b7988347039624942)
 2015-10-28 10:34:28 -07:00
Akira Ajisaka
ce01a5951b HDFS-9317. Document fsck -blockId and -storagepolicy options in branch-2.7. (aajisaka) 2015-10-28 13:14:58 +09:00
Kihwal Lee
59a2072135 HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list. Contributed by Daniel Templeton. 2015-10-27 16:42:33 -05:00
Kihwal Lee
1d23e1ec07 HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma. 
(cherry picked from commit 8104d522690fe9556177893770a388291cea0749)
 2015-10-27 11:09:05 -05:00
Arpit Agarwal
653ef52ef2 HDFS-9305. Delayed heartbeat processing causes storm of subsequent heartbeats. (Contributed by Arpit Agarwal) 2015-10-26 15:47:37 -07:00
Kihwal Lee
f2fd011cc1 HDFS-9290. DFSClient#callAppend() is not backward compatible for slightly older NameNodes. Contributed by Tony Wu. 
(cherry picked from commit b9e0417bdf2b9655dc4256bdb43683eca1ab46be)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
 2015-10-23 16:46:14 -05:00
Jason Lowe
2b6ce20e1c YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G 
(cherry picked from commit d3a34a4f388155f6a7ef040e244ce7be788cd28b)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-10-23 20:59:28 +00:00
Jonathan Eagles
49a7d70f53 YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 
(cherry picked from commit f8adeb712dc834c27cec15c04a986f2f635aba83)
 2015-10-23 11:18:32 -05:00
Jian He
7021e015d7 YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena 2015-10-21 16:37:21 -07:00
Jason Lowe
0dc69afdc7 MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li 
(cherry picked from commit ee9b80acf84718b952cbb82380cd0a4dbe22dc79)
 2015-10-21 21:46:59 +00:00
Jonathan Eagles
7d3d0d6052 YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) 
(cherry picked from commit 25f8f801d15e3d9f27f4a2a198262407203e14a5)
 2015-10-21 15:40:24 -05:00
Tsuyoshi Ozawa
c74a32ea42 Moved CHANGES.txt entries to 2.6.2 
YARN-2019, YARN-4092, YARN-4101, YARN-4087, YARN-3727, YARN-4005,
YARN-3780, YARN-3802, YARN-3896, YARN-3554, YARN-3194, YARN-3798

(cherry picked from commit 14f0e2787c7c590d8ec1fc52f437866f34b098bd)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-10-21 23:28:25 +09:00
Jason Lowe
29838ff5b1 YARN-4281. 2.7 RM app page is broken. Contributed by Chang Li 2015-10-21 13:43:25 +00:00
Kihwal Lee
3f3829e3a8 HADOOP-12464. Interrupted client may try to fail-over and retry. Contributed by Kihwal Lee. 
(cherry picked from commit 6144e0137bb51bd04b46ea5ce42c59c2d4f7657d)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryInvocationHandler.java
 2015-10-19 11:00:48 -05:00
Tsuyoshi Ozawa
08c02199ff YARN-3798. ZKRMStateStore shouldn't create new session without occurrance of SESSIONEXPIED. (ozawa and Varun Saxena) 2015-10-19 10:30:28 +09:00
Kihwal Lee
4cf7f8441a HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao. 
(cherry picked from commit c7c36cbd6218f46c33d7fb2f60cd52cb29e6d720)
 2015-10-15 11:26:14 -05:00
Kihwal Lee
efba749760 HDFS-8676. Delayed rolling upgrade finalization can cause heartbeat expiration. Contributed by Walter Su. 
(cherry picked from commit 5b43db47a313decccdcca8f45c5708aab46396df)
 2015-10-13 13:25:10 -05:00
Kihwal Lee
61388b41e7 HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization. 
(cherry picked from commit 60b858bfa65e0feb665e1a84784a3d45e9091c66)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java
 2015-10-13 10:46:20 -05:00
Kihwal Lee
3a5b9f49df HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 
(cherry picked from commit abf3ad988ddf07450fccb7ca8f4bb4dd688c118a)
 2015-10-13 10:29:42 -05:00
Jason Lowe
fe874e9f69 YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks) 
(cherry picked from commit feaf0349949e831ce3f25814c1bbff52f17bfe8f)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-10-08 16:08:55 +00:00
Jason Lowe
784d00e0e2 YARN-3802. Two RMNodes for the same NodeId are used in RM sometimes 
after NM is reconnected. Contributed by zhihai xu
(cherry picked from commit 5b5bb8dcdc888ba1ebc7e4eba0fa0e7e79edda9a)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-10-08 15:56:44 +00:00
Jason Lowe
aaff9dc39b YARN-3780. Should use equals when compare Resource in 
RMNodeImpl#ReconnectNodeTransition. Contributed by zhihai xu.
(cherry picked from commit c7ee6c151c5771043a6de3b8a951cea13f59dd7b)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-10-08 15:30:25 +00:00
Jason Lowe
84f1d99d02 YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong 
(cherry picked from commit 38aed1a94ed7b6da62e2445b5610bc02b1cddeeb)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-10-08 15:06:58 +00:00
Akira Ajisaka
aa0231f921 HADOOP-12465. Incorrect javadoc in WritableUtils.java. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 7fbf69bf47bb733f95b4afd733f9b65e1e7f2b46)
(cherry picked from commit 613c7088fd1accc59fdb7c700ef89409b0fb0d43)
 2015-10-08 06:09:32 +09:00
Kihwal Lee
5cb80992fb HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee. 
(cherry picked from commit 99e5204ff5326430558b6f6fd9da7c44654c15d7)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java
 2015-10-07 10:24:55 -05:00
Rohith Sharma K S
a231075964 YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks) 2015-10-07 09:35:10 +05:30
Jason Lowe
9f6ed41b95 YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 2015-10-02 20:19:14 +00:00
Kihwal Lee
b2bc72a051 HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) 
(cherry picked from commit d2b941f94a835f7bdde7714d21a470b505aa582b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-10-02 14:59:29 -05:00
Akira Ajisaka
3249c3ce56 HADOOP-12230. hadoop-project declares duplicate, conflicting curator dependencies. Contributed by Rakesh R. 
(cherry picked from commit 966f9508c3ce9dcc63dd2b169150f573699f94a7)
 2015-10-02 12:40:35 +02:00
Karthik Kambatla
49ff1e5276 HADOOP-12451. Setting HADOOP_HOME explicitly should be allowed. (kasha) 
(cherry picked from commit db145e0c9add81c74d853baa8187005f97ea1b53)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
 2015-10-01 11:36:12 -07:00
Akira Ajisaka
3096d3e1a1 MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita. 
(cherry picked from commit 195793c6f3e53a5c0527020477fe9c9158576f77)
(cherry picked from commit e42f57623251f0a5d5607e6a8b3d732de8c8a5a8)
 2015-10-01 10:26:24 +02:00
Jason Lowe
493f072008 YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu 
(cherry picked from commit 854d25b0c30fd40f640c052e79a8747741492042)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
 2015-09-30 15:08:10 +00:00
Anubhav Dhoot
7c9a368b45 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 
(cherry picked from commit 9735afe967a660f356e953348cb6c34417f41055)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java

(cherry picked from commit 22f2501476d987afb7bc19080a7a0db94ea72be6)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
 2015-09-28 20:57:08 -07:00
Jason Lowe
fc10c99839 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81353e1617875deb10f3c8e2a8b91a1e)
 2015-09-28 22:13:40 +00:00
Kihwal Lee
6e7c76a5bd HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. 2015-09-28 15:19:57 -05:00
Xuan
4c6e24528d YARN-3624. ApplicationHistoryServer should not reverse the order of the 
filters it gets. Contributed by Mit Desai

(cherry picked from commit d1b9b852442cb93b413eff8a137db6213e730d9c)
(cherry picked from commit bf3f08cc634c97fd31cc481b9fef3ccb2a53e5e9)
 2015-09-24 15:04:37 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
2b4c7fcc67 Release process for 2.6.1: Set the release date for 2.6.1 
(cherry picked from commit 1f707ecffd80142c25ef0dee8708da49920646bc)
(cherry picked from commit f48af332a76caa89c9d8fa1d2654d290bffdc183)
 2015-09-23 15:13:09 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
63213d70a9 YARN-3248. Moving it to improvements section in CHANGES.txt. 
(cherry picked from commit df52ff92ae42977bc743630191d1cbe1e17d00cf)
 2015-09-23 15:03:35 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
3de54ad4e1 CHANGES.txt: Moving YARN-1884, YARN-3171, YARN-3740, YARN-3248, YARN-3544 to 2.6.1 given the backport. 
(cherry picked from commit f3e5bc67661ebc7e14509f2a267131ae314699dc)
(cherry picked from commit 23f4cd1c5a1ccff86a0f2579cc6486d27e33fa59)
 2015-09-23 14:59:00 -07:00
Xuan
dce9223622 YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev

(cherry picked from commit 4728bdfa15809db4b8b235faa286c65de4a48cf6)
 2015-09-23 14:44:42 -07:00
Zhijie Shen
6540d5aed7 YARN-3740. Fixed the typo in the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS. Contributed by Xuan Gong. 
(cherry picked from commit eb6bf91eeacf97afb4cefe590f75ba94f3187d2b)
(cherry picked from commit 68cddb894a95a1ee63291af162849a23cb4eeb5f)
 2015-09-23 13:54:30 -07:00
Jason Lowe
3fac56e824 YARN-3975. WebAppProxyServlet should not redirect to RM page if AHS is enabled. Contributed by Mit Desai 2015-09-23 16:41:49 +00:00
Vinayakumar B
d800e2e9ed HDFS-9043. Doc updation for commands in HDFS Federation (Contributed by J.Andreina) 
(cherry picked from commit 57003fa971658c8482240f70445a6822c7692844)
(cherry picked from commit 96e3fbf401763c10e040b9d7dac2123f97895090)
 2015-09-22 16:40:19 +05:30
Jason Lowe
7429438c85 MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 2015-09-18 19:21:35 +00:00
Jason Lowe
521196874c MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 
(cherry picked from commit 58d1a02b8d66b1d2a6ac2158be32bd35ad2e69bd)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:10:38 +00:00