Commit Graph

276 Commits

Author SHA1 Message Date
Akira Ajisaka f0241ec216
HDFS-15977. Call explicit_bzero only if it is available. (#2914)
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-04-16 13:26:45 +09:00
Masatake Iwasaki 2717203f85
HADOOP-17569. Building native code fails on Fedora 33. (#2886) 2021-04-15 21:33:18 +09:00
Gautham B A b088f46b68
HDFS-15971. Make mkstemp cross platform (#2898) 2021-04-13 08:54:45 -07:00
Gautham B A 3148791da4
HDFS-15962. Make strcasecmp cross platform (#2883) 2021-04-09 09:01:33 -07:00
Gautham B A bf66116407
HDFS-15955. Make explicit_bzero cross platform (#2875) 2021-04-08 09:44:47 -07:00
Gautham B A 42ddb5c6fe
HDFS-15909. Make fnmatch cross platform (#2792) 2021-04-06 09:23:21 -07:00
Gautham B A 36014b8282
HDFS-15949. Fix integer overflow (#2857)
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-04-06 18:22:54 +09:00
Gautham B A 6e1df1c048
HDFS-15947. Replace deprecated protobuf APIs (#2856) 2021-04-05 09:52:48 -07:00
Gautham B A 70536ba1f9
HDFS-15944. Prevent truncation by snprintf (#2853) 2021-04-02 09:37:33 -07:00
Gautham B A 7dfff496fa
HDFS-15929. Replace RAND_pseudo_bytes in util.cc (#2826) 2021-03-30 13:36:02 -07:00
Gautham B A 8668abf87e
HDFS-15927. Catch polymorphic type by reference (#2824) 2021-03-30 13:35:45 -07:00
Gautham B A 50b1e1c0c4
HDFS-15928. Replace RAND_pseudo_bytes in rpc_engine.cc (#2825) 2021-03-30 10:37:10 -07:00
Gautham B A 6a3e135316
HDFS-15922. Use memcpy for copying non-null terminated string. (#2827) 2021-03-29 10:38:59 -07:00
Gautham B A 1fed18bb2d
HDFS-15922. Use memcpy for copying non-null terminated string in jni_helper.c (#2818) 2021-03-26 09:26:35 -07:00
Gautham B A 654555783d
HDFS-15918. Replace deprecated RAND_pseudo_bytes (#2811) 2021-03-24 10:52:33 -07:00
Gautham B A 714427c414
HDFS-15910. Improve security with explicit_bzero (#2793) 2021-03-23 09:58:19 -07:00
Gautham B A 61bfa42389
HDFS-15903. Refactor X-Platform lib (#2783) 2021-03-22 08:40:54 -07:00
Gautham B A 47620f8821
HDFS-15843. Make write cross-platform (#2710) 2021-03-16 09:06:51 -07:00
Akira Ajisaka 0ed9e36282
HDFS-15834. Remove the usage of org.apache.log4j.Level (#2696) 2021-02-17 01:59:24 -05:00
Gautham B A ad483fd66e
HDFS-15740. Add x-platform utilities (#2567) 2021-01-29 10:11:38 -08:00
Gautham B A 4ffec79b7c
HDFS-15712. Upgrade googletest to 1.10.0 (#2523)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-12-08 17:08:14 +09:00
Hui Fei 474fa80bfb
HADOOP-17277. Correct spelling errors for separator (#2322)
Contributed by Hui Fei.
2020-09-22 17:10:33 +01:00
Gautham B A 909f1e82d3
HADOOP-17196. Fix C/C++ standard warnings (#2208)
* Passing C/C++ standard flags -std is
  not cross-compiler friendly as not all
  compilers support all values.
* Thus, we need to make use of the
  appropriate flags provided by CMake in
  order to specify the C/C++ standards.

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-08-11 16:35:08 +09:00
Yuan e756fe3590
HDFS-14950. fix missing libhdfspp lib in dist-package (#1947)
libhdfspp.{a,so} are missed in dist-package.
This patch fixed this by copying these libs to the right directory

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
2020-07-31 16:49:49 +09:00
Suraj Naik 4101b0c0ed
HDFS-15476 Make AsyncStream executor private (#2151) 2020-07-19 23:26:54 +05:30
Gautham B A cce5a6f609
HDFS-15385 Upgrade boost library to 1.72 (#2051)
* Removed the asio-1.10.2 that was
  checked in under libhdfspp/third_party
  directory.
2020-07-15 13:39:48 +09:00
Babneet Singh 1996351b0b
HDFS-15270. Account for *env == NULL in hdfsThreadDestructor (#1951)
OpenJ9 JVM properly terminates the thread before hdfsThreadDestructor is
invoked. JNIEnv is a mirror of J9VMThread in OpenJ9. After proper thread
termination, accessing JNIEnv in hdfsThreadDestructor (*env)->GetJavaVM,
yields a SIGSEGV since *env is NULL after thread cleanup is performed.

The main purpose of hdfsThreadDestructor is to invoke
DetachCurrentThread, which performs thread cleanup in OpenJ9. Since
OpenJ9 performs thread cleanup before hdfsThreadDestructor is invoked,
hdfsThreadDestructor should account for *env == NULL and skip
DetachCurrentThread.

Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>
2020-05-04 13:08:53 -07:00
Aryan Gupta 816042e62b
HDFS-15301. statfs function in hdfs-fuse not working. Contributed by Aryan Gupta.(#1980) 2020-04-29 09:19:59 +05:30
Brahma Reddy Battula 8914cf9167 Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
Akira Ajisaka f59f6891c8
HDFS-15232. Fix libhdfspp test failures with GCC 7. (#1906) 2020-03-24 11:36:14 +09:00
Istvan Fajth 57aa048516
HDFS-14668 Support Fuse with Users from multiple Security Realms (#1739) 2020-02-27 07:48:14 -08:00
Kihwal Lee 10a60fbe20 HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan. 2020-02-04 12:22:35 -06:00
Sahil Takiar 13b427fc05 HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable (#963) Contributed by Sahil Takiar.
Reviewed-by: Siyao Meng <smeng@cloudera.com>
2019-09-27 12:25:03 -07:00
Sahil Takiar 3cf6e4272f HDFS-14846: libhdfs tests are failing on trunk due to jni usage bugs
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-17 15:51:39 -07:00
hunshenshi 2fe450cb5e HDFS-14425. Native build fails on macos due to jlong in hdfs.c (#741) 2019-07-26 15:33:12 -07:00
Akira Ajisaka a95e87d827 HDFS-11950. Disable libhdfs zerocopy test on Mac. Contributed by Akira Ajisaka.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-17 17:03:25 -07:00
Akira Ajisaka afd844059c HADOOP-16331. Fix ASF License check in pom.xml
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
Akira Ajisaka 9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Sahil Takiar e2dfdaee7c HDFS-14482: Crash when using libhdfs with bad classpath 2019-05-14 11:04:06 -07:00
Sahil Takiar 4877f0aa51 HDFS-3246: pRead equivalent for direct read path (#597)
HDFS-3246: pRead equivalent for direct read path

Contributed by Sahil Takiar
2019-04-30 14:52:16 -07:00
Sahil Takiar 18c57cf046 HDFS-14304: High lock contention on hdfsHashMutex in libhdfs
This closes #595

Signed-off-by: Todd Lipcon <todd@apache.org>
2019-03-26 20:32:51 -07:00
Sahil Takiar fe29b3901b
HDFS-14348: Fix JNI exception handling issues in libhdfs
This closes #600

Signed-off-by: Todd Lipcon <todd@apache.org>
2019-03-26 16:33:34 -07:00
Sahil Takiar 618e009ac0 HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 15:04:06 -08:00
Wei-Chiu Chuang 6192c1fe3b Revert "HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar."
This reverts commit f5a4b43a49.
2019-03-06 15:02:18 -08:00
Sahil Takiar f5a4b43a49 HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 14:58:45 -08:00
Sahil Takiar cb0fa0ce3f HDFS-14321. Fix -Xcheck:jni issues in libhdfs, run ctest with -Xcheck:jni enabled. Contributed by Sahil Takiar.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-04 12:37:57 -08:00
Sahil Takiar f19c844e75 HDFS-14285. libhdfs hdfsRead copies entire array even if its only partially filled. Contributed by Sahil Takiar.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-22 17:54:32 -08:00
Sahil Takiar a30059bb61 HDFS-14267. Add test_libhdfs_ops to libhdfs tests, mark libhdfs_read/write.c as examples. Contributed by Sahil Takiar.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-20 11:38:37 -08:00
Takanobu Asanuma ec77e95bbd HDFS-14232. libhdfs is not included in binary tarball. Contributed by Akira Ajisaka. 2019-02-04 10:12:09 +09:00
Akira Ajisaka 1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Chen Liang 652b257478 HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang. 2018-12-24 09:34:01 -08:00
Daniel Templeton e56d9f2618 HDFS-14015. Improve error handling in hdfsThreadDestructor in native thread local storage
Change-Id: Ida1e888c9231b9e46081338e3a206d8f6faabd36
2018-11-16 16:24:40 -08:00
Daniel Templeton 349168c4b3 HDFS-14047. [libhdfs++] Fix hdfsGetLastExceptionRootCause bug in test_libhdfs_threaded.c
(Contributed by Antoli Shein via Daniel Templeton)

Change-Id: Ia5546fd5f8270f285c373b9023a9bd9835059845
2018-11-06 09:22:27 -08:00
Sunil G 9c438abe52 HDFS-14033. [libhdfs++] Disable libhdfs++ build on systems that do not support thread_local. Contributed by Anatoli Shein. 2018-10-31 12:32:49 +05:30
Sunil G 58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Giovanni Matteo Fumarola 7b1fa5693e HDFS-13849. Migrate logging to slf4j in hadoop-hdfs-httpfs, hadoop-hdfs-nfs, hadoop-hdfs-rbf, hadoop-hdfs-native-client. Contributed by Ian Pickering. 2018-08-27 10:18:05 -07:00
Jason Lowe a17eed1b87 HDFS-13822. speedup libhdfs++ build (enable parallel build). Contributed by Allen Wittenauer and Pradeep Ambati 2018-08-17 12:25:36 -05:00
Aaron Fabbri 40f9b0c5c1
HDFS-13322 fuse dfs - uid persists when switching between ticket caches. Contributed by Istvan Fajth. 2018-07-31 18:44:49 -07:00
Allen Wittenauer 5d0f01e1fe
HDFS-13722. HDFS Native Client Fails Compilation on Ubuntu 18.04 (contributed by Jack Bearden) 2018-07-10 12:17:44 -07:00
James Clampffer 7969cc4667 HDFS-13615. libhdfs++ SaslProtocol hanging while accessing invalid lock. Contributed by Mitchell Tracy 2018-06-07 16:02:57 -04:00
James Clampffer b79ae5d93d HDFS-13534. libhdfs++: Fix GCC7 build. Contributed by James Clampffer. 2018-06-07 14:16:17 -04:00
James Clampffer 19ae588fde HDFS-11807. libhdfs++: Get minidfscluster tests running under valgrind. Contributed by Anatoli Shein. 2018-05-02 11:49:12 -04:00
James Clampffer eefe2a147c HDFS-13403: libhdfs++ Use hdfs::IoService object rather than asio::io_service. Contributed by James Clampffer. 2018-04-11 10:27:23 -04:00
James Clampffer 8620d2bdf9 HDFS-12110: libhdfs++: Rebase HDFS-8707 on trunk, minor conflict resolution in this patch. Contributed by Deepak Majeti and Anatoli Shein. 2018-03-22 17:19:47 -04:00
James fc5e44d9ed HDFS-12427: libhdfs++: Prevent requests from holding dangling pointer to RpcEngine. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James 48db24a430 HDFS-11294: libhdfs++: Segfault in HA failover if DNS lookup for both Namenodes fails. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James Clampffer 12942f679a HDFS-10787: libhdfs++: Public API should expose configuration parser. Original patch contributed by Mitchell Tracy, followup work and compile warning fixes contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James Clampffer eeb49d0ca7 HDFS-12237: libhdfs++: PROTOC_IS_COMPATIBLE check fails if protobuf library is built from source. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James Clampffer 22ea06a3dd HDFS-12134: libhdfs++: Add a synchronization interface for the GSSAPI. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James Clampffer 033433bce7 HDFS-12026: libhdfs++: Fix compilation errors and warnings when compiling with Clang. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James Clampffer 53e40bc718 HDFS-10874: libhdfs++: Public API headers should not depend on internal implementation. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James Clampffer 3db261d45b HDFS-12103: libhdfs++: Provide workaround to support cancel on filesystem connect until HDFS-11437 is resolved. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James Clampffer 23afd50865 HDFS-11908: libhdfs++: Authentication failure when first NN of a kerberized HA cluster is standby. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James Clampffer e64ba87486 HDFS-12104: libhdfs++: Make sure all steps in SaslProtocol end up calling AuthComplete. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James Clampffer 3c743b475e HDFS-12013: libhdfs++: read with offset at EOF should return 0 bytes instead of error. Contributed by Xiaowei Zhu 2018-03-22 17:19:47 -04:00
James Clampffer c5e7a69523 HDFS-11971: libhdfs++: [Fix] a few portability issues. Contributed by Anatoli Schein 2018-03-22 17:19:47 -04:00
James Clampffer 1c9b174663 HDFS-11518: libhdfs++: Add a build option to skip building examples, tests, and tools. Contributed by Anatoli Schein. 2018-03-22 17:19:47 -04:00
James Clampffer 869317be0a HDFS-10785: libhdfs++: Implement the rest of the tools. Contributed by Anatoli Schein 2018-03-22 17:19:47 -04:00
James Clampffer 52445fd4ee HDFS-11544: libhdfs++: Revert HDFS-11544.HDFS-8707.001.patch due to invalid memory access regression. Done by James Clampffer 2018-03-22 17:19:47 -04:00
James Clampffer c2386bc2f3 HDFS-11758: libhdfs++: Catch exceptions thrown by runtime hooks. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James 57cdad73de HDFS-11767: libhdfs++: SASL events should be scoped closer to usage. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James b584e34f2f HDFS-11730: libhdfs++: RpcConnection should handle authorization error call id. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James fdb88eb57e HDFS-11544: libhdfs++: Improve C API error reporting. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James 3e53da2d62 HDFS-11106: libhdfs++: Some refactoring to better organize files (part 2). Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James 0f3f8db113 HDFS-11449: libhdfs++: Log Datanode read size when reading from an HDFS block. Contributed by Xiaowei Zhu. 2018-03-22 17:19:47 -04:00
James 606d20c6c0 HDFS-11436: libhdfs++: Fix race condition in ScopedResolver. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James 8783461e2e HDFS-11028: libhdfs++: FileSystem needs to be able to cancel pending connections. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James 58de2df860 HDFS-10685: libhdfs++: return explicit error when non-secured client connects to secured server. Contributed by Kai Jiang. 2018-03-22 17:19:47 -04:00
James 69d55340f8 HDFS-11305: libhdfs++: Log Datanode information when reading an HDFS block. Contributed by Xiaowei Zhu. 2018-03-22 17:19:47 -04:00
James a30cf6a369 HDFS-10796: libhdfs++: Previous commit was missing test/hdfs_ioservice_test.cc, adding it. 2018-03-22 17:19:47 -04:00
James fbff671e0f HDFS-10796: libhdfs++: rationalize ioservice interactions. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James dd7837c429 HDFS-11106: libhdfs++: Some refactoring to better organize files. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James 2524afbc20 HDFS-11099: libhdfs++: Expose rack id in hdfsDNInfo. Contributed by Xiaowei Zhu. 2018-03-22 17:19:47 -04:00
James b7d9a0f827 HDFS-11027: libbhdfs++: Don't retry if there is an authentication failure. Contributed by Xiaowei Zhu. 2018-03-22 17:19:47 -04:00
James 59a3926946 HDFS-11014: libhdfs++: Make connection to HA clusters faster. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James 6dd47cae86 libhdfs++: Get rid of lock in RpcConnectionImpl destructor. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James 7ebecaeede HDFS-10898: libhdfs++: Make log levels consistent. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James 2a42eeb66f HDFS-10931: libhdfs++: Fix object lifecycle issues in the BlockReader (see follow up work in jira). Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
Bob Hansen 69cb05d292 HDFS-10595: libhdfs++: deconflate client name and client id 2018-03-22 17:19:47 -04:00
Bob Hansen d22e4b2eb7 HDFS-10450b: libhdfs++: Add Cyrus SASL support (engines). Contributed by Don Davis. 2018-03-22 17:19:47 -04:00