Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,672 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4668 Commits

Author SHA1 Message Date
Arpit Agarwal 492a597942 HDFS-11968. ViewFS: StoragePolicies commands fail with HDFS federation. Contributed by Mukul Kumar Singh. 
(cherry picked from commit b91305119b)
 2018-04-06 15:19:01 -07:00
Wei Yan 9638c3fe67 HDFS-13353. RBF: TestRouterWebHDFSContractCreate failed. Contributed by Takanobu Asanuma. 
(cherry picked from commit 3121e8c293)
 2018-04-05 13:19:05 -07:00
Xiao Chen 973a7b16b9 HADOOP-15355. TestCommonConfigurationFields is broken by HADOOP-15312. Contributed by LiXin Ge. 
(cherry picked from commit f614b44fe583033947cc78d8f04cfe6fb64fc892)
 2018-04-02 23:10:26 -07:00
Xiao Chen ce37f586f5 HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 
(cherry picked from commit 57374c4737)
 2018-04-02 22:49:03 -07:00
Konstantin V Shvachko 4ba5efa96f HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 
(cherry picked from commit acfd764fcc)
 2018-03-30 19:05:01 -07:00
Konstantin V Shvachko 3912bdb2b0 HADOOP-12862. LDAP Group Mapping over SSL can not specify trust store. Contributed by Wei-Chiu Chuang and Konstantin Shvachko. 
(cherry picked from commit 2216bde322)
 2018-03-29 17:22:53 -07:00
Wei-Chiu Chuang c6939e9330 HADOOP-15312. Undocumented KeyProvider configuration keys. Contributed by LiXin Ge. 
(cherry picked from commit 3fe41c65d8)
(cherry picked from commit 1176a128d6)
 2018-03-27 13:37:14 -07:00
Lei Xu 7b47400ce0 Update releasenotes and changelogs for 3.0.1 release 2018-03-23 11:51:32 -07:00
Inigo Goiri 981fda128f HDFS-13268. TestWebHdfsFileContextMainOperations fails on Windows. Contributed by Xiao Liang. 
(cherry picked from commit fe224ff972)
 2018-03-20 11:21:09 -07:00
Inigo Goiri a97fe585b3 HDFS-13296. GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases. Contributed by Xiao Liang. 
(cherry picked from commit e65ff1c8be)
 2018-03-19 17:21:20 -07:00
Wangda Tan 78f103d6e6 Revert "HADOOP-13707. If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed. Contributed by Yuanbo Liu" 
Change-Id: I946a466a43d56c73bb0135384e73cb8513595461
(cherry picked from commit 80ee5248b2dda1cb8d122d4f362f2f8cf02b9467)
(cherry picked from commit 252c2b4d52)
 2018-03-14 13:40:46 -07:00
Inigo Goiri 95a4665ad5 HADOOP-15308. TestConfiguration fails on Windows because of paths. Contributed by Xiao Liang. 
(cherry picked from commit 427fd027a3)
 2018-03-13 20:02:06 -07:00
Chris Douglas 9889e55e5a HADOOP-15311. HttpServer2 needs a way to configure the acceptor/selector count. Contributed by Erik Krogen 
(cherry picked from commit 9d6994da19)
(cherry picked from commit 80641508c7)
 2018-03-13 14:20:44 -07:00
Xiaoyu Yao 2a93ba690c HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 
(cherry picked from commit 107c177782)
 2018-03-12 20:23:52 -07:00
Chris Douglas a03c8ea61f HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 
(cherry picked from commit 45d1b0fdcc)
(cherry picked from commit 09940b1eb3)
 2018-03-12 20:06:22 -07:00
Owen O'Malley 0f3b6102ef Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpengo.java
 2018-03-10 13:35:47 -08:00
Owen O'Malley 0f106df4d6 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-10 13:33:52 -08:00
Steve Loughran ca5c4d454e HADOOP-15277. Remove .FluentPropertyBeanIntrospector from CLI operation log output. 
Contributed by Steve Loughran.
 2018-03-09 10:50:04 +00:00
Steve Loughran a5cb7f3616 HADOOP-15289. FileStatus.readFields() assertion incorrect. 
Contributed by Steve Loughran.

(cherry picked from commit 12ecb55ffe)
 2018-03-06 10:08:28 +00:00
Nanda kumar 7193c95584
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 55d04a6db1)
 2018-03-01 19:04:41 +09:00
Xiaoyu Yao 908ca63c03 HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 
(cherry picked from commit f20e10b2dd)
 2018-02-28 09:48:12 -08:00
Arpit Agarwal e22a731070 MAPREDUCE-7061. SingleCluster setup document needs to be updated. Contributed by Bharat Viswanadham. 2018-02-27 10:28:03 -08:00
Kihwal Lee 107efcdc5d HADOOP-9747. Reduce unnecessary UGI synchronization. Contributed by Daryn Sharp. 
(cherry picked from commit 1f0952bfc7)
 2018-02-23 13:14:44 -06:00
Steve Loughran c2bbe22c5a HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-21 11:11:28 +00:00
Jason Lowe f173e9cc04 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9)
 2018-02-16 14:54:55 -06:00
Anu Engineer 4b4157441f HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 
(cherry picked from commit 8f66affd62)
 2018-02-14 13:30:33 -08:00
Xiao Chen 2f3415a4d6 Revert "HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar." 
This reverts commit 16b2cad8e8.
 2018-02-14 10:25:15 -08:00
Arpit Agarwal 16b2cad8e8 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-13 10:14:29 -08:00
Anu Engineer 2d6ab20007 HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 
(cherry picked from commit 4304fcd5bd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java

(cherry picked from commit 9264f10bb3)
 2018-02-07 11:16:53 -08:00
Kihwal Lee d436c403b2 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de98)
 2018-02-06 15:34:52 -06:00
Lei Xu 7b377f43d0 Preparing for 3.0.2 development 2018-02-06 10:50:37 -08:00
Xiao Chen abdf8abf1a HADOOP-15197. Remove tomcat from the Hadoop-auth test bundle. 
(cherry picked from commit 09dd709d6e)
 2018-02-01 15:35:27 -08:00
Xiaoyu Yao e260a0af24 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 
(cherry picked from commit b3ac6f7ce3f5e014af7ad6bf882431a97716e020)
 2018-01-31 22:40:13 -08:00
Xiao Chen 75be15cf02 HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 
(cherry picked from commit b63dcd583f)
 2018-01-28 22:24:00 -08:00
Steve Loughran ec4f5f0f0d HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu
 2018-01-26 15:23:08 -08:00
Chris Douglas 7f35487788 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 
(cherry picked from commit 91db424c4360d7556660e8c57ac9a266e6688e01)
 2018-01-26 09:15:03 -08:00
Wei Yan c5c042135d HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 
(cherry picked from commit 0c139d5bcf)
 2018-01-25 12:52:45 -08:00
Hanisha Koneru 304642322f HADOOP-15121. Encounter NullPointerException when using DecayRpcScheduler. Contributed by Tao Jie. 
(cherry picked from commit 3fde0f1db5)
 2018-01-22 16:26:23 -08:00
Masahiro Tanaka dcb09c30b9
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e27b20b4d)
 2018-01-22 17:15:14 +09:00
Jitendra Pandey 169dacf1ee HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 
(cherry picked from commit d31c9d8c49)
 2018-01-18 20:02:48 -08:00
Brahma Reddy Battula e575406715 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 
(cherry picked from commit 08332e12d0)
 2018-01-18 10:56:12 +05:30
Miklos Szegedi 542cb8480c HADOOP-15060. TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky. Contributed by Miklos Szegedi. 
(cherry picked from commit 12d0645)
 2018-01-10 11:04:40 -08:00
Haibo Chen 889bebf8bb YARN-7602. NM should reference the singleton JvmMetrics instance. 
(cherry picked from commit 2f6c038be6)
 2018-01-03 09:44:33 -08:00
Xiao Chen 8c275c63ff HADOOP-15149. CryptoOutputStream should implement StreamCapabilities. 
(cherry picked from commit 81127616c5)
 2017-12-29 13:41:43 -08:00
Junping Du 53b9d81f46 Add 2.8.3 release jdiff files. 
(cherry picked from commit c89f99aade)
 2017-12-18 22:40:04 -08:00
Steve Loughran f61edab1d0 HADOOP-15123. KDiag tries to load krb5.conf from KRB5CCNAME instead of KRB5_CONFIG. 
Contributed by Vipin Rathor.

(cherry picked from commit 1ef906e29e)
 2017-12-18 18:49:41 +00:00
Jason Lowe 6e253c0391 HADOOP-15085. Output streams closed with IOUtils suppressing write errors. Contributed by Jim Brennan 
(cherry picked from commit f8af0e2feb)
 2017-12-14 10:19:07 -06:00
Yiqun Lin 925f3e62db HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 
(cherry picked from commit 91c96bdf8e)
 2017-12-14 16:02:37 +08:00
Andrew Wang 58225e4030 Update CHANGES, RELEASENOTES, jdiff for 3.0.0 release. 
(cherry picked from commit d447152d49)
 2017-12-13 13:05:45 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) ba874d6787 HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 
(cherry picked from commit f19638333b)
 2017-12-08 08:03:23 -08:00
John Zhuge fc9e156484 HADOOP-14872. CryptoInputStream should implement unbuffer. Contributed by John Zhuge. 
(cherry picked from commit 6c32ddad30)
 2017-12-07 21:08:16 -08:00
Xiao Chen 1d1401ad38 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 
(cherry picked from commit 19e0894209)
 2017-12-07 21:06:25 -08:00
John Zhuge 9942952dd8 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 
(cherry picked from commit bf6a660232)
 2017-12-07 21:06:25 -08:00
Sean Mackrory 63d81cbbe7 HADOOP-15098. TestClusterTopology#testChooseRandom fails intermittently. Contributed by Zsolt Venczel. 
(cherry picked from commit acb92904d0)
 2017-12-07 13:24:12 -07:00
Lei Xu f3143d225a HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 67662e2ac9)
 2017-12-07 11:20:19 -08:00
Xiao Chen 55fa509eb6 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 
(cherry picked from commit 404eab4dc0)
 2017-12-04 10:49:58 -08:00
Chris Douglas 2399c96ee7 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 
(cherry picked from commit 42307e3c3a)
 2017-12-02 23:32:44 -08:00
Chris Douglas 6266c52680 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 482fd5a880.
 2017-12-01 11:19:17 -08:00
Chris Douglas 482fd5a880 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 
(cherry picked from commit 5304698dc8)
 2017-12-01 10:30:08 -08:00
Robert Kanter 2c2ff7da05 HADOOP-13493. Compatibility Docs should clarify the policy for what takes precedence when a conflict is found (templedf via rkanter) 
(cherry picked from commit 75a3ab88f5)
 2017-11-30 07:39:51 -08:00
Xiao Chen 4f71c27f2e HADOOP-15067. GC time percentage reported in JvmMetrics should be a gauge, not counter. Contributed by Misha Dmitriev. 
(cherry picked from commit d162252d7a)
 2017-11-23 09:02:07 -08:00
Xiao Chen a78dbd1571 HADOOP-15023. ValueQueue should also validate (int) (lowWatermark * numValues) > 0 on construction. 
(cherry picked from commit b1941b200d)
 2017-11-15 16:44:48 -08:00
Daniel Templeton 4a70e75386 HADOOP-14876. Create downstream developer docs from the compatibility guidelines 
(cherry picked from commit bfd588789a879b8583ea4abd59f4f5843c5ac285)
 2017-11-14 13:20:27 -08:00
Andrew Wang 1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Arun Suresh f6de2b741e Addendum patch for Configuration fix. (Jason Lowe via asuresh) 
(cherry picked from commit b07e68b02a)
 2017-11-13 14:05:41 -08:00
Xiaoyu Yao f90f772384 HADOOP-14987. Improve KMSClientProvider log around delegation token checking. Contributed by Xiaoyu Yao and Xiao Chen. 
(cherry picked from commit 59d78a5088)
 2017-11-12 19:44:18 -08:00
Anu Engineer ba4355a62f HADOOP-14901. ReuseObjectMapper in Hadoop Common. Contributed by Hanisha Koneru. 
(cherry picked from commit e1b32e0959)
 2017-11-12 19:44:18 -08:00
Chris Douglas 7f80c2f293 HADOOP-8522. ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used. Contributed by Mike Percy 
(cherry picked from commit 796a0d3a5c)
 2017-11-10 17:45:15 -08:00
Xiao Chen 65d7968afd HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev. 
(cherry picked from commit 3c6adda291)
 2017-11-09 21:06:43 -08:00
Arun Suresh 72d7bedac9 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 
(cherry picked from commit 5eb7dbe9b3)
 2017-11-09 15:16:55 -08:00
Chris Douglas b5f66b005c Revert "HDFS-7878. API - expose a unique file identifier." 
This reverts commit 1ce0eb7d23.
 2017-10-31 13:56:41 -07:00
Wei-Chiu Chuang cba1fa496b HADOOP-14880. [KMS] Document&test missing KMS client side configs. Contributed by Gabor Bota. 
(cherry picked from commit 97c70c7ac6)
 2017-10-31 12:54:53 -07:00
Chen Liang af52a585f0 HADOOP-14915. method name is incorrect in ConfServlet. Contributed by Bharat Viswanadham. 
(cherry picked from commit 563dcdfc1d)
 2017-10-31 12:54:28 -07:00
Wei-Chiu Chuang 0b3e3e4895 HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar. 
(cherry picked from commit e906108fc9)
 2017-10-31 12:54:03 -07:00
Chris Douglas 1ce0eb7d23 HDFS-7878. API - expose a unique file identifier. 
(cherry picked from commit d015e0bbd5)
 2017-10-31 09:46:39 -07:00
Jason Lowe 75605766fd HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 
(cherry picked from commit 2fae63aa60)
 2017-10-31 09:32:02 -05:00
Junping Du ce10c1ef05 Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf)
 2017-10-26 23:18:50 -07:00
Junping Du f36b2c6990 Update CHANGES.md and RELEASENOTES for 2.8.2 release. 
(cherry picked from commit 1ad863e220)
 2017-10-26 23:18:23 -07:00
Xiao Chen e18d2c548b HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 
(cherry picked from commit e3f25f361720b544b464c5e4fbeb4f31e6c53c67)
 2017-10-25 14:11:38 -07:00
Akira Ajisaka 79ea3e8ea7
HADOOP-14030. PreCommit TestKDiag failure. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4c970088af)
 2017-10-25 17:18:49 +09:00
Varun Saxena 5be228e43b HADOOP-9657. NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports. Contributed by Varun Saxena. 
(cherry picked from commit 67e7673750)
 2017-10-25 03:22:18 +05:30
Naganarasimha faf23e6298 HADOOP-14966. Handle JDK-8071638 for hadoop-common. Contributed by Bibin A Chundatt. 
(cherry picked from commit 6b8122458e)
 2017-10-23 23:43:57 +05:30
Kai Zheng b7cb58a21d HDFS-12448. Make sure user defined erasure coding policy ID will not overflow. Contributed by Huafeng Wang 2017-10-20 09:34:58 +08:00
Junping Du 9433f9eb09 HADOOP-14958. Fix source-level compatibility after HADOOP-11252. Contributed by Junping Du. 
(cherry picked from commit b016f08f67)
 2017-10-18 15:07:20 -07:00
Anu Engineer 0160a708ae HADOOP-14752. TestCopyFromLocal#testCopyFromLocalWithThreads is fleaky. Contributed by Andras Bokor. 
(cherry picked from commit 442ea85545)
 2017-10-18 11:26:23 -07:00
Xiao Chen 12d19d3978 HADOOP-14944. Add JvmMetrics to KMS. 
(cherry picked from commit 86ee0c5e4e)
 2017-10-17 19:07:19 -07:00
Lei Xu cdfcd384b2 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 
(cherry picked from commit 31ebccc962)
 2017-10-16 19:46:26 -07:00
Robert Kanter a9cbfb0398 HADOOP-14938. Configuration.updatingResource map should be initialized lazily (misha@cloudera.com via rkanter) 2017-10-16 16:39:25 -07:00
Xiao Chen 23ec4a788a HADOOP-14949. TestKMS#testACLs fails intermittently. 
(cherry picked from commit 152317d40ece8abfa76fb5bd29f04878c1b0d3d8)
 2017-10-16 14:19:44 -07:00
Anu Engineer beb54e5279 HADOOP-13102. Update GroupsMapping documentation to reflect the new changes. Contributed by Esther Kundin. 
(cherry picked from commit 075358eb6f)
 2017-10-11 16:24:08 -07:00
Arun Suresh 52844ec0a1 HADOOP-13556. Change Configuration.getPropsWithPrefix to use getProps instead of iterator. (Larry McCay via asuresh) 
(cherry picked from commit b6c2c9058e)
 2017-10-11 15:22:18 -07:00
Jason Lowe c0d56cad5c HADOOP-14912. FairCallQueue may defer servicing calls. Contributed by Daryn Sharp 
(cherry picked from commit 1123f8f0b6)
 2017-10-10 14:49:15 -05:00
Xiao Chen 3106552b75 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit 212342d6db14673f8f5652a5209e1c4fcc0f0ede)
 2017-10-05 19:47:13 -07:00
Akira Ajisaka 5c2f07eed7
HADOOP-14616. Client fails to read a block with erasure code (XOR, native) when one of the data block is lost. Contributed by Huafeng Wang. 
(cherry picked from commit fe84cdc9da)
 2017-10-05 11:07:33 +09:00
Daniel Templeton 280b1fe1b4 HADOOP-14459. SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined 
(Contributed by Nandor Kollar via Daniel Templeton)

(cherry picked from commit 20e9ce3ab3)
 2017-10-04 15:11:03 +02:00
Akira Ajisaka b658ae002f
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari. 
(cherry picked from commit c9b525bd94)
 2017-10-04 07:02:31 +09:00
Xiao Chen 3dd3f924fc HADOOP-14095. Document caveats about the default JavaKeyStoreProvider in KMS. 
(cherry picked from commit d7ecac379a02876919d3e6081d42f0937f54e664)
 2017-09-29 19:34:42 -07:00
Junping Du c3eade44d0 YARN-7072. Add a new log aggregation file format controller. Contributed by Xuan Gong. 
(cherry picked from commit 3fddabc2fe)
 2017-09-29 16:49:44 -07:00
Jason Lowe a6630f703c HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly calculated. Contributed by Hanisha Koneru 
(cherry picked from commit 6f789fe057)
 2017-09-28 16:39:47 -05:00
Andrew Wang 5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00