Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,129 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

22129 Commits

Author SHA1 Message Date
avijayanhwx 4b099b8b89 HDDS-1501 : Create a Recon task interface to update internal DB on updates from OM. (#819) 2019-05-23 15:34:55 -07:00
Giovanni Matteo Fumarola 6a0e7dd454 YARN-9482. DistributedShell job with localization fails in unsecure cluster. Contributed by Prabhu Joseph. 2019-05-23 13:27:52 -07:00
Arpit Agarwal b4b9120e28
HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka (#832) 
* HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka

* Fixed checkstyle issues.

* verifyContainerData also does fixup, renamed. Added a Javadoc comment, both as per review discussion

* fixed merge error. adapted to new signature of BlockUtils getDB

* fixed checkstyle issue post merge
 2019-05-23 10:42:08 -07:00
Xiaoyu Yao 20a4ec351c HDDS-700. Support rack awared node placement policy based on network topology. Contributed by Sammi Chen. 2019-05-23 10:40:15 -07:00
Christopher Gregorian f96a2df38d HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian. 2019-05-23 10:28:37 -07:00
Igor Rudenko e2b79912c3 YARN-9469. Fix typo in YarnConfiguration. 2019-05-23 19:00:38 +02:00
Eric Yang 7b03072fd4 YARN-9080. Added clean up of bucket directories. 
Contributed by Prabhu Joseph, Peter Bacsko, Szilard Nemeth
 2019-05-23 12:08:44 -04:00
Eric Yang ea0b1d8fba HADOOP-16287. Implement ProxyUserAuthenticationFilter for web protocol impersonation. 
Contributed by Prabhu Joseph
 2019-05-23 11:36:32 -04:00
sdeka ca93760504 fixed checkstyle issue post merge 2019-05-23 20:49:52 +05:30
sdeka 72bef0f6cb fixed merge error. adapted to new signature of BlockUtils getDB 2019-05-23 11:36:40 +05:30
Akira Ajisaka a771e2a638
HADOOP-12948. Remove the defunct startKdc profile from hadoop-common. Contributed by Wei-Chiu Chuang. 2019-05-23 13:59:42 +09:00
supratimdeka 9da62f33be
Merge branch 'trunk' into HDDS-1535 2019-05-23 09:51:33 +05:30
Zhankun Tang 03aa70fe19 SUBMARINE-83. Refine the documents of submarine targeting 0.2.0 release. Contributed by Zhankun Tang. 2019-05-23 10:02:00 +08:00
Akira Ajisaka 5565f2c532
MAPREDUCE-7198. mapreduce.task.timeout=0 configuration used to disable timeout doesn't work. 2019-05-23 10:21:11 +09:00
Ajay Yadav 9c61494c02 HDDS-1065. OM and DN should persist SCM certificate as the trust root. Contributed by Ajay Kumar. (#834) 2019-05-22 11:47:32 -07:00
Shashikant Banerjee a315913c48 HDDS-1517. AllocateBlock call fails with ContainerNotFoundException (#826). Contributed by Shashikant Banerjee. 2019-05-22 17:24:27 +05:30
Mukul Kumar Singh 2fc6f8599a HDDS-1449. JVM Exit in datanode while committing a key. Contributed by Mukul Kumar Singh. (#825) 2019-05-22 17:18:40 +05:30
Wanqiang Ji 67f9a7b165
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event. 
This closes #821

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-05-22 14:23:13 +09:00
Siyao Meng 9dff6eff81 HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-22 13:22:26 +09:00
sdeka 456bb8a3e0 verifyContainerData also does fixup, renamed. Added a Javadoc comment, both as per review discussion 2019-05-22 09:34:21 +05:30
Bharat Viswanadham 77c49f2963
HDDS-1406. Avoid usage of commonPool in RatisPipelineUtils. (#714) 2019-05-21 18:41:30 -07:00
Ajay Yadav ef1cc725b8
HDDS-1538. Update ozone protobuf message for ACLs. Contributed by Ajay Kumar. (#828) 2019-05-21 15:53:40 -07:00
Giovanni Matteo Fumarola a85451c8d2 YARN-9575. Fix TestYarnConfigurationFields testcase failing. Contributed by Prabhu Joseph. 2019-05-21 11:25:30 -07:00
Sahil Takiar 5906268f0d HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors 2019-05-21 11:30:45 -06:00
Ajay Kumar ff5691ca06
HDDS-1442. add spark container to ozonesecure-mr compose files 
Closes #746
 2019-05-21 17:10:46 +02:00
Lokesh Jain c1d7d68c78
HDDS-1461. Optimize listStatus api in OzoneFileSystem (#782) 2019-05-21 14:47:48 +05:30
Inigo Goiri 1cb2eb0df3 HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong. 2019-05-20 17:22:58 -07:00
Vivek Ratnavel Subramanian 05db2a598e HDDS-1487. Bootstrap React framework for Recon UI (#831) 
HDDS-1487. Bootstrap React framework for Recon UI.
Contributed by Vivek Ratnavel Subramanian.
 2019-05-20 22:09:58 +02:00
avijayanhwx f63300228e HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected. (#799) 
* HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected.

* HDDS-1451 : Address review comments.
 2019-05-20 22:02:56 +02:00
Sunil G 24c53e057a YARN-9546. Add configuration option for YARN Native services AM classpath. Contributed by Gergely Pollak. 2019-05-20 10:53:01 -04:00
sdeka 64c39856d2 Fixed checkstyle issues. 2019-05-20 08:13:37 +05:30
Bharat Viswanadham 0d1d7c86ec HDDS-1499. OzoneManager Cache. (#798) 2019-05-19 19:23:02 -07:00
Ben Roling a36274d699
HADOOP-16085. S3Guard: use object version or etags to protect against inconsistent read after replace/overwrite. 
Contributed by Ben Roling.

S3Guard will now track the etag of uploaded files and, if an S3
bucket is versioned, the object version.

You can then control how to react to a mismatch between the data
in the DynamoDB table and that in the store: warn, fail, or, when
using versions, return the original value.

This adds two new columns to the table: etag and version.
This is transparent to older S3A clients -but when such clients
add/update data to the S3Guard table, they will not add these values.
As a result, the etag/version checks will not work with files uploaded by older clients.

For a consistent experience, upgrade all clients to use the latest hadoop version.
 2019-05-19 22:29:54 +01:00
Zhankun Tang 729ccb2cab SUBMARINE-58. Submarine client needs to generate fat jar. Contributed by Zac Zhou. 2019-05-19 21:18:33 +08:00
sdeka 869a1ab41a HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka 2019-05-18 16:16:33 +05:30
Arpit Agarwal 732133cb2a HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit A… (#725) 
* HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit Agarwal.

Change-Id: I6db6bdd19839a45e5341ed7e745cd38f68af8378

* Suppress spurious findbugs warning.

* Remove log file that got committed in error
 2019-05-18 11:47:19 +02:00
Giovanni Matteo Fumarola 12c81610e0 YARN-9505. Add container allocation latency for Opportunistic Scheduler. Contributed by Abhishek Modi. 2019-05-17 12:03:21 -07:00
Srinivasu Majeti 3e5e5b028a HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-05-17 19:20:28 +02:00
Alexis Daboville 4cb3da6ac7
HADOOP-16248. MutableQuantiles leak memory under heavy load. 
Contributed by Alexis Daboville,
 2019-05-17 15:15:22 +01:00
Siddharth c183bd8e20 HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle. 
This closes #822.
 2019-05-16 15:13:10 -07:00
Eric Yang fab5b80a36 YARN-9554. Fixed TimelineEntity DAO serialization handling. 
Contributed by Prabhu Joseph
 2019-05-16 16:39:50 -04:00
Márton Elek 03ea8ea92e
HDDS-1297. Fix IllegalArgumentException thrown with MiniOzoneCluster Initialization. Contributed by Yiqun Lin. 2019-05-16 17:04:46 +02:00
Márton Elek 9248b7d58b
HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. 2019-05-16 17:04:46 +02:00
Sahil Takiar b067f8acaa HADOOP-16050: s3a SSL connections should use OpenSSL 
(cherry picked from commit aebf229c17)
 2019-05-16 08:57:54 -06:00
Elek, Márton b7de735159 HDDS-1522. Provide intellij runConfiguration for Ozone components. Contributed by Elek, Marton. 2019-05-16 16:38:50 +02:00
Andrew Olson c15b3bca86
HADOOP-16294: Enable access to input options by DistCp subclasses. 
Adding a protected-scope getter for the DistCpOptions, so that a subclass does
not need to save its own copy of the inputOptions supplied to its constructor,
if it wishes to override the createInputFileListing method with logic similar
to the original implementation, i.e. calling CopyListing#buildListing with a path and input options.

Author:    Andrew Olson
 2019-05-16 16:11:12 +02:00
David Mollitor 2713dcf6e9
HADOOP-16307. Intern User Name and Group Name in FileStatus. 
Author:    David Mollitor
 2019-05-16 16:02:07 +02:00
Shashikant Banerjee e66ecc564a HDDS-1531. Disable the sync flag by default during chunk writes in Datanode (#820). Contributed by Shashikant Banerjee. 2019-05-16 18:38:52 +05:30
Sunil G de01422c2e SUBMARINE-56. Update documentation to describe single-node PyTorch integration. Contributed by Szilard Nemeth. 2019-05-15 21:26:48 -07:00
Bharat Viswanadham d4c8858586
HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy. 2019-05-15 17:41:36 -07:00