Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,925 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

21925 Commits

Author SHA1 Message Date
Weiwei Yang ec143cbf67 YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang. 2019-04-06 19:59:36 +08:00
Sunil G 1b9ba0ebb2 SUBMARINE-47. Provide an implementation to parse configuration values from a YAML file for submarine run CLI. Contributed by Szilard Nemeth. 2019-04-06 07:39:13 +05:30
Shweta e9b859f749 HDFS-14371. Improve Logging in FSNamesystem by adding parameterized logging. Contributed by Shweta. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-04-05 16:09:00 -07:00
Anu Engineer dc013f7a35 HDDS-1329. Update documentation for Ozone-0.4.0 release. 
Contributed By Ajay Kumar.
 2019-04-05 14:14:42 -07:00
Doroszlai, Attila 274ab4e86c HDDS-1332. Attempt to fix flaky test testStartStopDatanodeStateMachine (#697) 2019-04-05 13:21:58 -07:00
avijayanhwx 26dc940813 HDDS-1390. Fix OzoneS3 Gateway server due to exclusion of hk2-api. (#698) 2019-04-05 12:36:00 -07:00
Vrushali C 22362c876d YARN-9335 [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls. Contributed by Abhishesk Modi. 2019-04-05 12:06:51 -07:00
Vrushali C 27039a29ae YARN-9382 Publish container killed, paused and resumed events to ATSv2. Contributed by Abhishesk Modi. 2019-04-05 12:02:43 -07:00
Nanda kumar 5750bb94ed HDDS-1389. Fix testSCMChillModeRestrictedOp. (#696) 2019-04-05 10:42:00 -07:00
Elek, Márton 89f9c8604b HDDS-1385. Make the ozonesecure-mr environment definition version independent. Contributed by Elek, Marton. (#695) 
(cherry picked from commit d758c97873)
 2019-04-05 09:30:40 -07:00
Ajay Kumar 0356b3699d HDDS-1333. OzoneFileSystem can't work with spark/hadoop2.7 because incompatible security classes. Contributed by Elek, Marton. (#653) 
(cherry picked from commit 7ec6a31eb3)
 2019-04-05 09:29:21 -07:00
Inigo Goiri 260d843b25 HDFS-14416. Fix TestHdfsConfigFields for field dfs.client.failover.resolver.useFQDN. Contributed by Fengnan Li. 2019-04-05 09:26:06 -07:00
Bharat Viswanadham 77fe51e136
HDDS-1379. Convert all OM Volume related operations to HA model. (#689) 2019-04-04 21:09:50 -07:00
Akira Ajisaka 67020f0950
HDFS-14407. Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes. Contributed by Wanqiang Ji. 2019-04-05 12:29:11 +09:00
Bharat Viswanadham d2141ef886
HDDS-1189. Build failing due to rat check failure introduced by HDDS-1189. Contributed by Siddharth Wagle. 2019-04-04 17:38:02 -07:00
Steve Loughran 215ffc792e HADOOP-16197 S3AUtils.translateException to map CredentialInitializationException to AccessDeniedException 
Contributed by Steve Loughran.

Change-Id: Ie98ca5210bf0009f297edbcacf1fc6dfe5ea70cd.
 2019-04-04 21:14:18 +01:00
Eric Yang 2bf82aee82 YARN-9441. Updated YARN app catalog name for consistency. 
Contributed by Weiwei Yang
 2019-04-04 13:21:30 -04:00
Eric Yang 8d150067e2 YARN-9396. Fixed duplicated RM Container created event to ATS. 
Contributed by Prabhu Joseph
 2019-04-04 13:01:56 -04:00
David Mollitor c90736350b
HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.
 2019-04-04 16:15:57 +01:00
avijayanhwx 993f36ee33 HDDS-1353 : Metrics scm_pipeline_metrics_num_pipeline_creation_failed keeps increasing because of BackgroundPipelineCreator. (#681) 2019-04-04 19:18:32 +05:30
Stephen O'Donnell d2637cb176 HDFS-14389. getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-04-04 04:07:18 -07:00
Shashikant Banerjee 33e159bf8d HDDS-1349. Remove watchClient from XceiverClientRatis. Contributed by Shashikant Banerjee. 2019-04-04 16:35:37 +05:30
Nanda kumar 48a58bce37
HDDS-1207. Refactor Container Report Processing logic and plugin new Replication Manager. (#662) 2019-04-04 16:32:59 +05:30
Weiwei Yang 945e8c6064 YARN-9394. Use new API of RackResolver to get better performance. Contributed by Lantao Jin. 2019-04-04 18:05:01 +08:00
Yiqun Lin a92806d05a HDDS-1189. Recon Aggregate DB schema and ORM. Contributed by Siddharth Wagle. 2019-04-04 17:33:37 +08:00
Vrushali C eb03f7c419 YARN-9303 Username splits won't help timelineservice.app_flow table. Contributed by Prabhu Joseph. 2019-04-03 22:53:05 -07:00
Hanisha Koneru f09a78f73f
HDDS-1339. Implement ratis snapshots on OM (#651) 2019-04-03 22:50:28 -07:00
Inigo Goiri 7b5b783f66 HDFS-14327. Using FQDN instead of IP to access servers with DNS resolving. Contributed by Fengnan Li. 2019-04-03 16:11:13 -07:00
Hanisha Koneru 4e64f8d763 HDDS-1324. TestOzoneManagerHA tests are flaky (#676) 2019-04-03 16:02:19 -07:00
Bharat Viswanadham 358f7f9b99
HDDS-1211. Test SCMChillMode failing randomly in Jenkins run (#543) 2019-04-03 15:02:00 -07:00
avijayanhwx 8ff41d6243 HDDS-1358 : Recon Server REST API not working as expected. (#668) 2019-04-03 14:52:06 -07:00
Siyao Meng e62cbcbc83 HADOOP-16011. OsSecureRandom very slow compared to other SecureRandom implementations. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-04-03 14:29:52 -07:00
Steve Loughran 366186d999
HADOOP-16233. S3AFileStatus to declare that isEncrypted() is always true (#685) 
This is needed to fix up some confusion about caching of job.addCache() handling of S3A paths; all parent dirs -the files are downloaded by the NM without  using the DTs of the user submitting the job. This means that when you submit jobs to an EC2 cluster with lower IAM permissions than the user, cached resources don't get downloaded and the job doesn't start.

Production code changes:
* S3AFileStatus Adds "true" to the superclass's encrypted flag during construction.

Tests
* Base AbstractContractOpenTest can control whether zero byte files created in tests are encrypted. Not done via an XML attribute, just a subclass point. Thoughts?
* Verify that the filecache considers paths to not have the permissions which trigger reduce-privilege downloads
* And extend ITestDelegatedMRJob to test a completely different bucket (open street map), to verify that cached resources do get their tokens picked up

Docs:
* Advise FS developers to say all files are encrypted. It's otherwise harmless and it'll stop other people seeing impossible to debug error messages on app launch.

Contributed by Steve Loughran.

Change-Id: Ifaae4c9d735ccc5eafeebd2584b65daf2d4e5da3
 2019-04-03 21:23:40 +01:00
avijayanhwx 59022b28b7 HDDS-1330 : Add a docker compose for Ozone deployment with Recon. (#669) 2019-04-03 13:20:51 -07:00
Gabor Bota d7979079ea HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota. 2019-04-03 12:59:39 -06:00
Xiaoyu Yao d6c233fce6 HDDS-1377. OM failed to start with incorrect hostname set as ip address in CSR. (#683) 2019-04-03 11:53:51 -07:00
Wei-Chiu Chuang be488b6070 HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang. 2019-04-03 11:00:59 -07:00
Sahil Takiar 3b0c5016b2
HDFS-14394: Add -std=c99 / -std=gnu99 to libhdfs compile flags 
Signed-off-by: Todd Lipcon <todd@apache.org>
 2019-04-03 10:56:33 -07:00
Shashikant Banerjee 7cd7045eea HDDS-1164. Add New blockade Tests to test Replica Manager. Contributed by Nilotpal Nandi. 2019-04-03 22:19:10 +05:30
Sunil G 002dcc4ebf YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko. 2019-04-03 18:57:28 +05:30
Akira Ajisaka 8b6deebb1d
HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji. 2019-04-03 19:35:02 +09:00
Yiqun Lin f96fb05a2b HDDS-1365. Fix error handling in KeyValueContainerCheck. Contributed by Supratim Deka. 2019-04-03 14:01:30 +08:00
Akira Ajisaka aaaf856f4b
HADOOP-16226. new Path(String str) does not remove all the trailing slashes of str 2019-04-03 13:16:59 +09:00
Lokesh Jain cf268114c9 HDFS-13960. hdfs dfs -checksum command should optionally show block size in output. Contributed by Lokesh Jain. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-04-02 12:24:55 -07:00
Ajay Kumar bfc90bdc64
HDDS-1355. Only FQDN is accepted for OM rpc address in secure environment. Contributed by Ajay Kumar. 
Closes #677
 2019-04-02 15:43:01 +02:00
Kitti Nanasi aeb43dfba3
HDDS-1153. Make tracing instrumentation configurable 
Closes #673
 2019-04-02 14:21:47 +02:00
Yiqun Lin d31c86892e HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee. 2019-04-02 16:32:30 +08:00
Steve Loughran 61d19110d4
HADOOP-16218. Findbugs warning of null param to non-nullable method in Configuration with Guava update. (#655) 
Change-Id: I461e518ce9a4730b91a8138ad55b39e9a4b0a4b8
 2019-04-02 09:15:11 +01:00
Yufei Gu 2f752830ba YARN-9214. Add AbstractYarnScheduler#getValidQueues method to remove duplication. Contributed by Wanqiang Ji. 2019-04-01 20:05:15 -07:00
Akira Ajisaka ebd0d21538
HADOOP-16225. Fix links to the developer mailing lists in DownstreamDev.md. Contributed by Wanqiang Ji. 2019-04-02 10:51:47 +09:00