Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,009 Commits 326 Branches 371 Tags
Commit Graph

8009 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Lowe
7fe08d762f YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong 
(cherry picked from commit f0520a2c94a9052e9907847cd8381844285a7f5f)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-05-25 21:56:50 +00:00
Jason Lowe
07b702d9d5 MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 9227dfc25f373a99cb66ad7d6bacef8dcf336f77)
 2016-05-13 14:38:57 +00:00
Colin Patrick Mccabe
167e54ed34 HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe) 
(cherry picked from commit f0252ad2525c90b2b89ae2ff1de12482da2c8ddd)
(cherry picked from commit 43f32d3f4a86d8f2a302530831003496c78fd561)
(cherry picked from commit 154332bb5463998d5e8f2251c8010d624f5e85a9)
(cherry picked from commit a61e04171f36ce75a8b5b501bc883e46fa67ef08)
 2016-05-10 11:22:42 -07:00
Jason Lowe
5c457e37e9 Addendum to CHANGES.txt for MR-6689 2016-05-06 22:56:35 +00:00
Jason Lowe
0bd88aa78b MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 
(cherry picked from commit c9bb96fa81fc925e33ccc0b02c98cc2d929df120)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
 2016-05-06 22:34:12 +00:00
Haohui Mai
a9beec0221 HDFS-7258. CacheReplicationMonitor rescan schedule log should use DEBUG level instead of INFO level. Contributed by Xiaoyu Yao. 
(cherry picked from commit 0942c99eba12f6baf5609c9621cd07b09618a97e)
 2016-05-06 14:05:00 -07:00
Vinod Kumar Vavilapalli
ab4fad0d82 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
Updated the 2.7 branch patch to fix a minor issue in TestRMContainerAllocator.java and a minor fix in the 2.6 patch.

(cherry picked from commit 8d48266720dcf0e71cfd87fef18b60a53aa1bef9)
(cherry picked from commit 9f51a3f430e4a8efc54193af4fe9fc1509f9980f)
 2016-05-05 20:43:07 -07:00
Harsh J
916f949131 HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. Contributed by harsh. (harsh) 
(cherry picked from commit 54e612bfb9f877e58f7f153c43cb4147876826d3)
 2016-05-02 17:50:30 -07:00
Arpit Agarwal
fffa13b2d6 HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-21 14:55:54 -07:00
Andrew Wang
36a9150b2b HADOOP-13043. Add LICENSE.txt entries for bundled javascript dependencies. 
(cherry picked from commit 08b7efa95202b6d6ada143cab9369fac4ebb4c49)
(cherry picked from commit e16040b2af5046e3919a1f20cb801ba2ebc823c9)
(cherry picked from commit 22403be0c3b311c54a66d9d8a5d606bf5045fc1d)
(cherry picked from commit 7cf625188cbc2b369986bf5c98c3f1bbfb8cd198)
 2016-04-21 14:12:09 -07:00
Andrew Wang
ccd4b83972 HADOOP-13042. Restore lost leveldbjni LICENSE and NOTICE changes. 
(cherry picked from commit fea50c5440d83225958c5e346299334559fc37a4)
(cherry picked from commit c28e4b0651cfd5ed142980dc10f4c2a7388a39a0)
(cherry picked from commit bae229ae27605d039d692fdd1edc5b4d946ddbe8)

 Conflicts:
	LICENSE.txt

(cherry picked from commit b8febb774a195f185a41824de673fa17aad8e5f0)
 2016-04-21 13:52:03 -07:00
Vinayakumar B
5eb7b68efe Changed version to 2.6.5-SNAPSHOT 2016-04-12 12:06:27 +05:30
Vinayakumar B
a8b1a63b8f Addendum: HDFS-10271. 
Added CHANGES.txt entry

(cherry picked from commit b52ac3cb4d49a9ec9f3e5f1ed7a5f40521ef3cf1)
 2016-04-12 11:57:28 +05:30
Vinayakumar B
713ba2a732 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 37e4e45390007536cb53f375d529910d65cd5d19)
 2016-04-12 11:33:13 +05:30
Wangda Tan
74aca34882 MAPREDUCE-6302. Preempt reducers after a configurable timeout irrespective of headroom. (kasha via wangda) 2016-04-08 14:34:15 -07:00
Vinayakumar B
783c99d292 HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos() (Contributed by kanaka kumar avvaru) 
(cherry picked from commit 48ca23def1d1c28448a65238814070e79c8f4c4e)
(cherry picked from commit d21bc811d82f685b0a1338bc513d9a925d305a17)

(cherry picked from commit 62da8f6fa688b503ed124313bc2f6f664175d225)

 Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
        hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
 2016-04-06 10:52:28 +08:00
Jason Lowe
95b8700969 YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 2016-04-05 14:05:11 +00:00
Walter Su
173681a477 HDFS-10182. Hedged read might overwrite user's buf. Contributed by zhouyingchao. 2016-03-29 16:10:50 +08:00
Jason Lowe
c092a8c1bb HADOOP-12958. PhantomReference for filesystem statistics can trigger OOM. Contributed by Sangjin Lee 
(cherry picked from commit 9a09200a1f5f752e266d4fb8e0c808073080bde8)
 2016-03-25 23:05:29 +00:00
Akira Ajisaka
160a8c74df MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina. 
(cherry picked from commit 368c77376a3704e3fa395d72b1816d388b5cd782)
(cherry picked from commit 7a3fd1bc1a2ce93100b6887a97733e262b5e1d56)
 2016-03-23 23:28:12 +09:00
Xiaoyu Yao
77deb15c17 HADOOP-12682. Fix TestKMS#testKMSRestart* failure. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit ab725cff66e8a047e9437e42ac49ac8685ee7a94)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-03-22 14:03:59 -07:00
Xiaoyu Yao
c3f24c6353 HADOOP-12559. KMS connection failures should trigger TGT renewal. Contributed by Zhe Zhang. 
Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-03-18 17:50:05 -07:00
Junping Du
c84b32bdda YARN-4785. inconsistent value type of the type field for LeafQueueInfo in response of RM REST API. (Varun Vasudev via junping-du) 2016-03-17 09:49:09 -07:00
Steve Loughran
e17fe26c85 Revert "HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims)" 
This reverts commit f311023f1205b1ea4d51a5b9da5df89f4040af32.
 2016-03-11 16:58:50 +00:00
Masatake Iwasaki
f311023f12 HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims) 
(cherry picked from commit acafc950d9347769c3729d571121b3525c6d5eb2)
 2016-03-11 15:39:57 +09:00
Ming Ma
d61a2fd1d0 Update CHANGES.txt for HADOOP-12789 
(cherry picked from commit d6e40fd270c7b552b02c407e2f2dba7deb97177d)
 2016-03-07 20:52:25 -08:00
Ming Ma
5579ee37e3 HADOOP-12789. log classpath of ApplicationClassLoader at INFO level. (Sangjin Lee via mingma) 
(cherry picked from commit 49eedc7ff02ea61764f416f0e2ddf81370aec5fb)
(cherry picked from commit f01f1940c4006eeffae679f822463d156c790c04)
 2016-03-07 20:52:12 -08:00
Zhihai Xu
80034621ea YARN-4761. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations on fair scheduler. Contributed by Sangjin Lee 
(cherry picked from commit e1ccc9622b2f1fbefea1862fa74d1fb56d8eb264)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java

(cherry picked from commit 274e18286054ee99a4de557d8c429a3f3b9ccf6b)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java
 2016-03-06 21:43:40 -08:00
Robert Kanter
909c96f054 MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 
(cherry picked from commit 0f72da7e281376f4fcbfbf3fb33f5d7fedcdb1aa)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-03-06 18:25:32 -08:00
Sangjin Lee
84233f013d YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee) 
(cherry picked from commit 553b591ba06bbf0b18dca674d25a48218fed0a26)
 2016-02-24 09:37:28 -08:00
Jason Lowe
9e10dd6077 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 2016-02-23 20:54:42 +00:00
Varun Vasudev
d32100d7fc MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du. 
(cherry picked from commit c6f2d761d5430eac6b9f07f137a7028de4e0660c)
(cherry picked from commit f1999fe2754cbf11b138fb048c7486cab9b02c97)

 Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-02-23 14:08:14 +05:30
Zhe Zhang
3fea7f0a38 HDFS-9688. Test the effect of nested encryption zones in HDFS downgrade. 
Change-Id: I2a802325a2bc82c97012d2c9fb388b292fe3af8f
 2016-02-22 15:11:00 -08:00
Zhe Zhang
44eef0ed7c HADOOP-12800. Copy docker directory from 2.8 to 2.7/2.6 repos to enable pre-commit Jenkins runs. (zhz) 
Change-Id: I893319c53e497420e2b20511f0d5c0803d8515ba
 2016-02-19 14:34:39 -08:00
Sangjin Lee
319790432d YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when possible (Ming Ma via sjlee) 
(cherry picked from commit 7de70680fe44967e2afc92ba4c92f8e7afa7b151)
 2016-02-17 20:57:09 -08:00
Vinayakumar B
89e90becfc HADOOP-12805. Annotate CanUnbuffer with @InterfaceAudience.Public (Contributed by Ted Yu) 
(cherry picked from commit 4b0e59fdc010be517dd67ddf35c962da204df547)
(cherry picked from commit e79a5e4460b5199eebb0d788fdc1eaa36d6b0ac4)
 2016-02-16 17:34:19 +05:30
Junping Du
43ac0b09e3 Set the release date for 2.6.4. 
(cherry picked from commit ec12ce8f48944c38271399a6e452aa2c2a9f5b80)
(cherry picked from commit a28be9edc72cc4fe42fab107c17bcac6789af4e5)
 2016-02-13 01:51:27 -08:00
Masatake Iwasaki
39970794d7 HADOOP-12589. Fix intermittent test failure of TestCopyPreserveFlag (iwasakims) 
(cherry picked from commit 36ba5cac37dee6b60e607401ac06d263e4ac5d8f)
 2016-02-13 15:03:54 +09:00
Jason Lowe
1b013a4e83 MAPREDUCE-6191. Improve clearing stale state of Java serialization 
testcase.  (Sam Liu via Eric Yang)
(cherry picked from commit c379e102ddab1b83fd69e4492a40c9901fb50675)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
(cherry picked from commit 303c650d0dfcb10760cea589b8cef6ace8e36417)
 2016-02-10 16:49:25 +00:00
Arpit Agarwal
d833823daf HDFS-9752. Permanent write failures may happen to slow writers during datanode rolling upgrades. (Contributed by Walter Su) 2016-02-09 09:41:35 -08:00
Jason Lowe
f2e3935bd6 MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host. Contributed by zhihai xu 
(cherry picked from commit aa5b15b03be61ebb76a226e0de485d5228c8e3d0)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-09 16:38:59 +00:00
Zhe Zhang
1b25855b9b HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang. 
Change-Id: Idde8522e11c77f82b376e0c90a9bf33d163d5cb9
(cherry picked from commit ef3f3f6bb14cf44bef1778f1091d8ed8a4b764a3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-09 10:20:40 +08:00
Sangjin Lee
7f8c89c33a HADOOP-12773. HBase classes fail to load with client/job classloader enabled (sjlee) 
(cherry picked from commit 58acbf940a92ef8a761208a7a743175ee7b3377d)
 2016-02-08 14:00:55 -08:00
Vinayakumar B
ea517acfb6 HDFS-9629. Update the footer of Web UI to show year 2016 (Contributed by Xiao Chen and Vinayakumar B) 
(cherry picked from commit ddcd6954d03d3caadc8b69f7b886a3a17c3c9d0c)
(cherry picked from commit 8cdfdbf815bf89eee3184ec7cd00e1924c1ea316)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/index.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/explorer.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/index.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/status.html
 2016-02-04 13:34:54 +05:30
Junping Du
89bfc884a5 Adding release 2.6.5 to CHANGES.txt. 2016-02-02 20:28:20 -08:00
Kihwal Lee
d8416aec3c HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao. 
(cherry picked from commit c7c36cbd6218f46c33d7fb2f60cd52cb29e6d720)
(cherry picked from commit 4cf7f8441a288cefd44f126d60dae1998239892a)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend2.java
 2016-02-02 17:10:32 -08:00
Jason Lowe
fc01ec866a MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong 
(cherry picked from commit 43e669b22d9ac0b81ab8cc397c5d6d92bac986c6)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-02 19:33:32 +00:00
Junping Du
cd93382a59 HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee. 2016-02-02 10:37:29 -08:00
Junping Du
30626ad35f HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee. 2016-02-02 10:29:31 -08:00
Junping Du
572b5872fe HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 2016-02-02 10:16:59 -08:00