Commit Graph

277 Commits

Author SHA1 Message Date
Gautham B A 5736b34b2a
HDFS-16736. Link to Boost library in libhdfspp (#4782) 2022-08-26 09:11:44 -07:00
Gautham B A 6ba2c53720
HDFS-16681. Do not pass GCC flags for MSVC in libhdfspp (#4615)
* This PR ensures that the GCC flag
  -Wno-missing-field-initializers isn't passed
  for MSVC.
2022-07-25 22:55:38 +05:30
Gautham B A 8f83d9f56d
HDFS-16680. Skip libhdfspp Valgrind tests on Windows (#4611)
* The CMake test libhdfs_mini_stress_valgrind
  requires Valgrind.
* This PR skips this test on Windows since
  Valgrind isn't available.
2022-07-23 23:22:13 +05:30
Gautham B A 7de9b5ee27
HDFS-16467. Ensure Protobuf generated headers are included first (#4601)
* This PR ensures that the Protobuf generated headers
  are always included first, even when these headers
  are included transitively.
* This problem is specific to Windows only.
2022-07-23 23:20:15 +05:30
Gautham B A d07256a96d
HDFS-16667. Use malloc for buffer allocation in uriparser2 (#4576)
* Windows doesn't support variables for specifying
  the array size.
* This PR uses malloc to fix this issue.
2022-07-20 21:57:28 +05:30
Gautham B A 6415eb04e8
HDFS-16665. Fix duplicate sources for HDFS test (#4573)
* The library target hdfspp_test_shim_static is
  built using the following sources, which
  causes duplicate symbols to be defined -
  - hdfs_shim.c
  - ${LIBHDFSPP_BINDING_C}/hdfs.cc
* ${LIBHDFSPP_BINDING_C}/hdfs.cc is redundant
  and removing this fixes the issue.
2022-07-19 21:39:06 +05:30
Gautham B A 4fb799e6c5
HDFS-16464. Create only libhdfspp static libraries for Windows (#4571)
* Added the appropriate CMake flags and
  commands to enable only statically
  linked libraries and executables to
  be built on Windows.
2022-07-19 21:37:22 +05:30
Gautham B A 21b8952125
HDFS-16666. Pass CMake args for Windows in pom.xml (#4574)
* This PR passes the necessary CMake args in the
  pom.xml needed for building HDFS native client
  on Windows.
* These arguments are exposed as maven options
  and can be passed from the command-line.
2022-07-19 10:45:59 +05:30
Gautham B A 440f4c2b28
HDFS-16654. Link OpenSSL lib for CMake deps check (#4538)
* The check_c_source_compiles fails on Windows
  while linking with an "unable to resolve
  external symbol" error.
* This PR links OpenSSL lib for this check to
  fix this issue.
2022-07-17 20:47:30 +05:30
Gautham B A 8e39e35bea
HDFS-16466. Implement Linux permission flags on Windows (#4526)
* HDFS-16466. Implement Linux permission flags on Windows

* statinfo.cc uses POSIX permission flags.
  These flags aren't available for Windows.
* This PR implements the equivalent flags
  on Windows to make this cross platform
  compatible.
2022-07-08 09:29:13 +05:30
Gautham B A 5a40224b53
HDFS-16469. Locate protoc-gen-hrpc across platforms (#4434)
* We use the TARGET_FILE CMake
  generator expression to get
  the location of the
  protoc-gen-hrpc CMake target.
2022-06-15 15:28:10 +05:30
Gautham B A d557c44d94
HDFS-16463. Make dirent cross platform compatible (#4370)
* jnihelper.c in HDFS native client uses
  dirent.h. This header file isn't available
  on Windows.
* This PR provides a cross platform
  compatible implementation for dirent
  under the XPlatform library.
2022-06-10 09:59:07 +05:30
Gautham B A a406f49bc0
HDFS-16602. Use "defined" directive along with #if (#4371)
* The #if directive expects a
  boolean expression. This PR
  uses the more appropriate
  "defined" directive.
2022-06-03 22:05:20 +05:30
Gautham B A 4496e2daf6
HDFS-16604. Install gtest via FetchContent_Declare in CMake (#4374)
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-06-01 13:45:44 +09:00
Rishabh Sharma 0be1fde962
HDFS-16561. Handle error returned by strtol
* strtol is used in hdfs-chmod.cc. The call
  to strtol could error out when an invalid
  input is provided.
* This PR handles the error given out by
   strtol.
2022-05-26 11:09:49 +05:30
Gautham B A 1350539f2d
HDFS-16465. Remove redundant strings.h inclusions (#4279)
* Remove redundant strings.h inclusions

* strings.h was included in a bunch of
  C/C++ files and were redundant.
* Also, strings.h is not available on
  Windows and thus isn't cross-platform
  compatible.

* Build for all platforms in CI

* Revert "Build for all platforms in CI"

This reverts commit 2650f047bd6791a5908cfbe50cc8e70d42c512cb.

* Debug failure on Centos 8

* Skipping pipeline run on
  Centos 7 to debug the
  failure on Centos 8.

* Revert "Debug failure on Centos 8"

This reverts commit e365e34d6f.
2022-05-11 23:04:22 +05:30
Gautham B A d346be9bd2
HDFS-16564. Use uint32_t for hdfs_find (#4245)
* hdfs_find uses u_int32_t type for
  storing the value for the max-depth
  command line argument.
* The type u_int32_t isn't standard,
  isn't available on Windows and thus
  breaks cross-platform compatibility.
  We need to replace this with uint32_t
  which is available on all platforms since
  it's part of the C++ standard.
2022-05-04 20:19:08 +05:30
Gautham B A 88155cebe9
HDFS-16468. Define ssize_t for Windows (#4228)
* Some C/C++ files use ssize_t data type.
  This isn't available for Windows and we
  need to define an alias for this and set it
  to an appropriate type to make it cross
  platform compatible.
2022-04-29 22:25:09 +05:30
Steve Loughran a7b4e8f03e
HDFS-14478. Add libhdfs APIs for openFile (#4166)
Contributed by Sahil Takiar
2022-04-13 14:15:27 +01:00
Gautham B A e89fd9645b
HDFS-16474. Make HDFS tail tool cross platform (#4157)
* The source files for hdfs_tail
  uses getopt for parsing the
  command line arguments.
* getopt is available only on
  Linux and thus, isn't cross
  platform.
* We need to replace getopt
  with boost::program_options
  to make these tools cross
  platform.
2022-04-12 20:31:24 +05:30
Gautham B A d5e97fe4d6
HDFS-16473. Make HDFS stat tool cross platform (#4145)
* The source files for hdfs_stat
  uses getopt for parsing the
  command line arguments.
* getopt is available only on
  Linux and thus, isn't cross platform.
* We need to replace getopt with
  boost::program_options to make
  this tool cross platform.
2022-04-08 23:06:39 +05:30
Gautham B A 4ef1d3eef9
HDFS-16472. Make HDFS setrep tool cross platform (#4130)
* The source files for hdfs_setrep
   uses getopt for parsing the
   command line arguments.
* getopt is available only on Linux
   and thus, isn't cross platform.
* We need to replace getopt
  with boost::program_options
  to make this tool cross platform.
2022-04-05 22:59:11 +05:30
Gautham B A 81879eb9cb
HDFS-16471. Make HDFS ls tool cross platform (#4086)
* The source files for hdfs_ls
   uses getopt for parsing the
   command line arguments.
* getopt is available only on
   Linux and thus, isn't cross
   platform.
* Thus, we need to replace
   getopt with
   boost::program_options to
   make this tool cross platform.
2022-03-22 23:12:00 +05:30
Gautham B A a631f45a99
HDFS-16470. Make HDFS find tool cross platform (#4076)
* The source files for hdfs_find uses
   getopt for parsing the command
   line arguments. getopt is available
   only on Linux and thus, isn't cross
   platform.
* Thus, we need to replace getopt
   with boost::program_options to
   make hdfs_find cross platform.
2022-03-18 21:41:01 +05:30
Gautham B A 8f07081789
HDFS-16462. Make HDFS get tool cross platform (#4003) 2022-03-05 23:05:14 +05:30
Gautham B A 19561846df
HADOOP-18151. Switch the baseurl for Centos 8 (#4047)
* Centos 8 has reached its
  End-of-Life and thus its
  packages are no longer
  accessible from
  mirror.centos.org.
* This PR switches the baseurl
  to vault.centos.org where
  the packages are archived.
2022-03-05 07:11:22 +05:30
Gautham B A ed44662968
HDFS-16445. Make HDFS count, mkdir, rm cross platform (#3945) 2022-02-02 11:12:17 +05:30
daimin d69938994e
HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background (#3842)
Reviewed-by: Istvan Fajth <pifta@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-01-25 13:02:37 +08:00
Gautham B A 271ac0e091
HDFS-16419. Make HDFS data transfer tools cross platform (#3873) 2022-01-12 19:57:27 +05:30
Gautham B A c3006be516
HDFS-16407. Make hdfs_du tool cross platform (#3848) 2022-01-04 22:29:54 +05:30
Gautham B A 6dddbd42ed
HDFS-16285. Make HDFS ownership tools cross platform (#3588) 2021-12-09 10:17:41 +05:30
Gautham B A 0a952e4d1f
HDFS-16304. Locate OpenSSL libs for libhdfspp (#3625) 2021-11-08 08:58:07 -08:00
Gautham B A 1032724aa3
HDFS-16300. Use libcrypto in Windows for libhdfspp (#3617) 2021-11-04 09:19:11 -07:00
Gautham B A a2f089df4d
HDFS-16278. Make HDFS snapshot tools cross platform (#3563) 2021-10-21 10:24:33 -07:00
Viraj Jasani 516f36c6f1
HADOOP-17967. Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom (#3555) 2021-10-21 16:54:25 +09:00
Gautham B A 107fe227eb
HDFS-16267. Make hdfs_df tool cross platform (#3542) 2021-10-13 09:11:21 -07:00
Gautham B A a05eb23a85
HDFS-16260. Make hdfs_deleteSnapshot tool cross platform (#3532) 2021-10-11 18:25:28 -07:00
Viraj Jasani 3ac0cc9e6f
HADOOP-17957. Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-hdfs-project modules (#3530)
Reviewed-by: Ahmed Hussein <ahussein@apache.org>
2021-10-11 15:32:49 +09:00
Gautham B A 4f3dfb7c1c
HDFS-16265. Refactor HDFS tool tests for better reuse (#3536) 2021-10-09 11:00:43 -07:00
Gautham B A cc95fc4cc6
HDFS-16263. Add CMakeLists for hdfs_allowSnapshot (#3531) 2021-10-08 08:49:56 -07:00
Gautham B A 6bad3badf5
HDFS-16251. Make hdfs_cat tool cross platform (#3523) 2021-10-07 10:57:11 -07:00
Gautham B A e12cd0c638
HDFS-16254. Cleanup protobuf on exit of hdfs_allowSnapshot (#3518) 2021-10-06 09:24:27 -07:00
Gautham B A 84f10fd78b
HDFS-16250. Refactor AllowSnapshotMock using GMock (#3513) 2021-10-05 09:16:42 -07:00
Gautham B A 35a8d48872
HDFS-16205. Make hdfs_allowSnapshot tool cross platform (#3388) 2021-09-22 10:29:38 -07:00
Gautham B A b6d1971820
HDFS-16178. Make recursive rmdir in libhdfs++ cross platform (#3311) 2021-08-20 10:04:55 -07:00
Gautham B A 409878ef80
HDFS-16174. Refactor TempFile and TempDir in libhdfs++ (#3303) 2021-08-17 09:49:52 -07:00
Gautham B A 77383a4d55
HDFS-15976. Make mkdtemp cross platform (#2908) 2021-08-10 09:23:28 -07:00
Akira Ajisaka 4972e7a246
Fix potential heap buffer overflow in hdfs.c. Contributed by Igor Chervatyuk. 2021-08-07 11:27:25 +09:00
Gautham B A ef5dbc7283
HADOOP-17766. CI for Debian 10 (#3129) 2021-06-23 10:02:33 -07:00
Gautham B A bcaeb1ac8c
Revert "Revert "HDFS-15971. Make mkstemp cross platform (#2898)"" (#3044) 2021-05-26 14:12:53 -07:00