honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/dev-tools/forbidden/core-signatures.txt

114 lines
7.4 KiB
Plaintext
Raw Normal View History

Integrate forbiddenAPI checks into Maven build. This commit integrates the forbiddenAPI checks that checks Java byte code against a list of "forbidden" API signatures. The commit also contains the fixes of the current source code that didn't pass the default API checks. See https://code.google.com/p/forbidden-apis/ for details. Closes #3059
2013-05-18 17:49:04 -04:00
@defaultMessage spawns threads with vague names; use a custom thread factory and name threads so that you can tell (by its name) which executor it is associated with
java.util.concurrent.Executors#newFixedThreadPool(int)
java.util.concurrent.Executors#newSingleThreadExecutor()
java.util.concurrent.Executors#newCachedThreadPool()
java.util.concurrent.Executors#newSingleThreadScheduledExecutor()
java.util.concurrent.Executors#newScheduledThreadPool(int)
java.util.concurrent.Executors#defaultThreadFactory()
java.util.concurrent.Executors#privilegedThreadFactory()
Forbid usage of Character.codePoint(At|Before) and Collections.sort. Character.codePointAt and codePointBefore have two versions: one which only accepts an offset, and one which accepts an offset and a limit. The former can be dangerous when working with buffers of characters because if the offset is the last char of the buffer, a char outside the buffer might be used to compute the code point, so one should always use the version which accepts a limit. Collections.sort is wasteful on random-access lists: it dumps data into an array, sorts the list and then adds elements back to the list. However, the sorting can easily be performed in-place by using Lucene's CollectionUtil.(merge|quick|tim)Sort.
2013-06-13 03:23:01 -04:00
java.lang.Character#codePointBefore(char[],int) @ Implicit start offset is error-prone when the char[] is a buffer and the first chars are random chars
java.lang.Character#codePointAt(char[],int) @ Implicit end offset is error-prone when the char[] is a buffer and the last chars are random chars
@defaultMessage Collections.sort dumps data into an array, sorts the array and reinserts data into the list, one should rather use Lucene's CollectionUtil sort methods which sort in place
java.util.Collections#sort(java.util.List)
java.util.Collections#sort(java.util.List,java.util.Comparator)
Forbid usage of StringReader in favor of FastStringReader. StringReader is synchronized although input streams should always be consumed by a single thread at a time. FastStringReader on the other hand is completely thread unsafe. Closes #3411
2013-07-30 12:50:58 -04:00
java.io.StringReader#<init>(java.lang.String) @ Use FastStringReader instead
Add RamUsageEstimator#sizeOf(Object) to forbidden APIs This method can be a performance trap since it traverse the entire object tree that is referenced by the provided object. See LUCENE-5373
2014-01-31 15:38:39 -05:00
Add IndexReader reference counting methods to forbidden APIs.
2014-02-06 14:47:31 -05:00
@defaultMessage Reference management is tricky, leave it to SearcherManager
org.apache.lucene.index.IndexReader#decRef()
org.apache.lucene.index.IndexReader#incRef()
org.apache.lucene.index.IndexReader#tryIncRef()
Force merges to not happen when indexing a doc / flush Today, even though our merge policy doesn't return new merge specs on SEGMENT_FLUSH, merge on the scheduler is still called on flush time, and can cause merges to stall indexing during merges. Both for the concurrent merge scheduler (the default) and the serial merge scheduler. This behavior become worse when throttling kicks in (today at 20mb per sec). In order to solve it (outside of Lucene for now), we wrap the merge scheduler with an EnableMergeScheduler, where, on the thread level, using a thread local, the call to merge can be enabled/disabled. A Merges helper class is added where all explicit merges operations should go through. If the scheduler is the enabled one, it will enable merges before calling the relevant explicit method call. In order to make sure Merges is the only class that calls the explicit merge calls, the IW variant of them is added to the forbidden APIs list. closes #5319
2014-03-05 07:26:26 -05:00
typo fixes - https://github.com/vlajos/misspell_fixer Closes #8323
2014-11-02 18:36:06 -05:00
@defaultMessage QueryWrapperFilter is cacheable by default - use Queries#wrap instead
Enforce query instance checking before it wrapper as a filter We have the default QueryWrapperFilter as well as our custom one while our wrapper is explicitly marked as no_cache such that it will never be included in a cache. This was not consistenly used and caused several problems during tests where p/c related queries were used as filters and ended up in the cache. This commit adds the QueryWrapperFilter ctor to the forbidden APIs to enforce the query instance checks.
2014-03-14 09:13:06 -04:00
org.apache.lucene.search.QueryWrapperFilter#<init>(org.apache.lucene.search.Query)
Close Directory / Store once all resources have been released Currently we close the store and therefor the underlying directory when the engine / shard is closed ie. during relocation etc. We also just close it while there are still searches going on and/or we are recovering from it. The recoveries might fail which is ok but searches etc. will be working like pending fetch phases. The contract of the Directory doesn't prevent to read from a stream that was already opened before the Directory was closed but from a system boundary perspective and from lifecycles that we test it seems to be the right thing to do to wait until all resources are released. Additionally it will also help to make sure everything is closed properly before directories are closed itself. Note: this commit adds Object#wait & Object@#notify/All to forbidden APIs Closes #5432
2013-12-14 14:46:16 -05:00
Change default numeric precision_step Change the default numeric precision_step to 16 for 64-bit types, 8 for 32-bit and 16-bit types. Disable precision_step for the 8-bit byte type. Closes #5905
2014-04-23 09:01:25 -04:00
@defaultMessage Pass the precision step from the mappings explicitly instead
org.apache.lucene.search.NumericRangeQuery#newDoubleRange(java.lang.String,java.lang.Double,java.lang.Double,boolean,boolean)
org.apache.lucene.search.NumericRangeQuery#newFloatRange(java.lang.String,java.lang.Float,java.lang.Float,boolean,boolean)
org.apache.lucene.search.NumericRangeQuery#newIntRange(java.lang.String,java.lang.Integer,java.lang.Integer,boolean,boolean)
org.apache.lucene.search.NumericRangeQuery#newLongRange(java.lang.String,java.lang.Long,java.lang.Long,boolean,boolean)
org.apache.lucene.search.NumericRangeFilter#newDoubleRange(java.lang.String,java.lang.Double,java.lang.Double,boolean,boolean)
org.apache.lucene.search.NumericRangeFilter#newFloatRange(java.lang.String,java.lang.Float,java.lang.Float,boolean,boolean)
org.apache.lucene.search.NumericRangeFilter#newIntRange(java.lang.String,java.lang.Integer,java.lang.Integer,boolean,boolean)
org.apache.lucene.search.NumericRangeFilter#newLongRange(java.lang.String,java.lang.Long,java.lang.Long,boolean,boolean)
Close Directory / Store once all resources have been released Currently we close the store and therefor the underlying directory when the engine / shard is closed ie. during relocation etc. We also just close it while there are still searches going on and/or we are recovering from it. The recoveries might fail which is ok but searches etc. will be working like pending fetch phases. The contract of the Directory doesn't prevent to read from a stream that was already opened before the Directory was closed but from a system boundary perspective and from lifecycles that we test it seems to be the right thing to do to wait until all resources are released. Additionally it will also help to make sure everything is closed properly before directories are closed itself. Note: this commit adds Object#wait & Object@#notify/All to forbidden APIs Closes #5432
2013-12-14 14:46:16 -05:00
@defaultMessage Only use wait / notify when really needed try to use concurrency primitives, latches or callbacks instead.
java.lang.Object#wait()
java.lang.Object#wait(long)
java.lang.Object#wait(long,int)
java.lang.Object#notify()
java.lang.Object#notifyAll()
Forbid Math.abs(int/long). We have had a couple of bugs because of the use of these methods without paying attention that it might return a negative value when provided with MIN_VALUE. There is one common and legitimate usage of this method in order to perform a modulo operation which would always return a positive number. This use-case has been extracted to MathUtils.mod. Close #5562
2014-03-26 20:14:26 -04:00
@defaultMessage Beware of the behavior of this method on MIN_VALUE
java.lang.Math#abs(int)
java.lang.Math#abs(long)
[JAVA7 Upgrade] Move to Long.compare
2014-03-27 10:48:12 -04:00
Disable explicit GC by default We don't rely upon GC to cleanup mappedbytebuffers, we unmap them explicitly on close in lucene. But the JDK has crazy loops with explicit GCs in exceptional cases to try to force unmapping. In general we don't want any of our code or library code calling this method: so its banned in forbidden-apis as well.
2014-06-27 07:02:26 -04:00
@defaultMessage Please do not try to stop the world
java.lang.System#gc()
[JAVA7 Upgrade] Move to Long.compare
2014-03-27 10:48:12 -04:00
@defaultMessage Use Long.compare instead we are on Java7
Change default numeric precision_step Change the default numeric precision_step to 16 for 64-bit types, 8 for 32-bit and 16-bit types. Disable precision_step for the 8-bit byte type. Closes #5905
2014-04-23 09:01:25 -04:00
com.google.common.primitives.Longs#compare(long,long)
Better support for partial buffer reads/writes in translog infrastructure Some IO api can return after writing & reading only a part of the requested data. On these rare occasions, we should call the methods again to read/write the rest of the data. This has cause rare translog corruption while writing huge documents on Windows. Noteful parts of the commit: - A new Channels class with utility methods for reading and writing to channels - Writing or reading to channels is added to the forbidden API list - Added locking to SimpleFsTranslogFile - Removed FileChannelInputStream which was not used Closes #6441 , #6576
2014-06-20 09:33:17 -04:00
@defaultMessage Use Channels.* methods to write to channels. Do not write directly.
java.nio.channels.WritableByteChannel#write(java.nio.ByteBuffer)
java.nio.channels.FileChannel#write(java.nio.ByteBuffer, long)
java.nio.channels.GatheringByteChannel#write(java.nio.ByteBuffer[], int, int)
java.nio.channels.GatheringByteChannel#write(java.nio.ByteBuffer[])
java.nio.channels.ReadableByteChannel#read(java.nio.ByteBuffer)
java.nio.channels.ScatteringByteChannel#read(java.nio.ByteBuffer[])
Fix forbidden API syntax error
2014-07-01 13:48:37 -04:00
java.nio.channels.ScatteringByteChannel#read(java.nio.ByteBuffer[], int, int)
Better support for partial buffer reads/writes in translog infrastructure Some IO api can return after writing & reading only a part of the requested data. On these rare occasions, we should call the methods again to read/write the rest of the data. This has cause rare translog corruption while writing huge documents on Windows. Noteful parts of the commit: - A new Channels class with utility methods for reading and writing to channels - Writing or reading to channels is added to the forbidden API list - Added locking to SimpleFsTranslogFile - Removed FileChannelInputStream which was not used Closes #6441 , #6576
2014-06-20 09:33:17 -04:00
java.nio.channels.FileChannel#read(java.nio.ByteBuffer, long)
[CORE] Support parsing lucene minor version strings We parse the version that is shipped with the Lucene segments in order to find the version of lucene that wrote a particular segment. Yet, some lucene version ie: * 4.3.1 (Elasticsearch 0.90.2) * 4.5.1 (Elasticsearch 0.90.7) * 3.6.1 (pre Elasticsearch 0.90.0) wrote illegal strings containing the minor version which causes IAE exceptions being thrown from lucenes parsing method. Closes #7055
2014-07-28 06:43:37 -04:00
@defaultMessage Use Lucene.parseLenient instead it strips off minor version
org.apache.lucene.util.Version#parseLeniently(java.lang.String)
Internal: Add all unsafe variants of LZF compress library functions to forbidden APIs. The "optimized" encoders/decoders have been unreliable and error prone. Also, fix LZFCompressor.compress to use LZFEncoder.safeEncode, which creates a new safe encoder, instead of using a shared encoder (which is not threadsafe). closes #7468
2014-08-26 23:17:07 -04:00
@defaultMessage unsafe encoders/decoders have problems in the lzf compress library. Use variants of encode/decode functions which take Encoder/Decoder.
com.ning.compress.lzf.impl.UnsafeChunkEncoders#createEncoder(int)
com.ning.compress.lzf.impl.UnsafeChunkEncoders#createNonAllocatingEncoder(int)
com.ning.compress.lzf.impl.UnsafeChunkEncoders#createEncoder(int, com.ning.compress.BufferRecycler)
com.ning.compress.lzf.impl.UnsafeChunkEncoders#createNonAllocatingEncoder(int, com.ning.compress.BufferRecycler)
com.ning.compress.lzf.impl.UnsafeChunkDecoder#<init>()
com.ning.compress.lzf.parallel.CompressTask
com.ning.compress.lzf.util.ChunkEncoderFactory#optimalInstance()
com.ning.compress.lzf.util.ChunkEncoderFactory#optimalInstance(int)
com.ning.compress.lzf.util.ChunkEncoderFactory#optimalNonAllocatingInstance(int)
com.ning.compress.lzf.util.ChunkEncoderFactory#optimalInstance(com.ning.compress.BufferRecycler)
com.ning.compress.lzf.util.ChunkEncoderFactory#optimalInstance(int, com.ning.compress.BufferRecycler)
com.ning.compress.lzf.util.ChunkEncoderFactory#optimalNonAllocatingInstance(int, com.ning.compress.BufferRecycler)
com.ning.compress.lzf.util.ChunkDecoderFactory#optimalInstance()
com.ning.compress.lzf.util.LZFFileInputStream#<init>(java.io.File)
com.ning.compress.lzf.util.LZFFileInputStream#<init>(java.io.FileDescriptor)
com.ning.compress.lzf.util.LZFFileInputStream#<init>(java.lang.String)
com.ning.compress.lzf.util.LZFFileOutputStream#<init>(java.io.File)
com.ning.compress.lzf.util.LZFFileOutputStream#<init>(java.io.File, boolean)
com.ning.compress.lzf.util.LZFFileOutputStream#<init>(java.io.FileDescriptor)
com.ning.compress.lzf.util.LZFFileOutputStream#<init>(java.lang.String)
com.ning.compress.lzf.util.LZFFileOutputStream#<init>(java.lang.String, boolean)
com.ning.compress.lzf.LZFEncoder#encode(byte[])
com.ning.compress.lzf.LZFEncoder#encode(byte[], int, int)
com.ning.compress.lzf.LZFEncoder#encode(byte[], int, int, com.ning.compress.BufferRecycler)
com.ning.compress.lzf.LZFEncoder#appendEncoded(byte[], int, int, byte[], int)
com.ning.compress.lzf.LZFEncoder#appendEncoded(byte[], int, int, byte[], int, com.ning.compress.BufferRecycler)
com.ning.compress.lzf.LZFCompressingInputStream#<init>(java.io.InputStream)
com.ning.compress.lzf.LZFDecoder#fastDecoder()
com.ning.compress.lzf.LZFDecoder#decode(byte[])
com.ning.compress.lzf.LZFDecoder#decode(byte[], int, int)
com.ning.compress.lzf.LZFDecoder#decode(byte[], byte[])
com.ning.compress.lzf.LZFDecoder#decode(byte[], int, int, byte[])
com.ning.compress.lzf.LZFInputStream#<init>(java.io.InputStream)
com.ning.compress.lzf.LZFInputStream#<init>(java.io.InputStream, boolean)
com.ning.compress.lzf.LZFInputStream#<init>(java.io.InputStream, com.ning.compress.BufferRecycler)
com.ning.compress.lzf.LZFInputStream#<init>(java.io.InputStream, com.ning.compress.BufferRecycler, boolean)
com.ning.compress.lzf.LZFOutputStream#<init>(java.io.OutputStream)
com.ning.compress.lzf.LZFOutputStream#<init>(java.io.OutputStream, com.ning.compress.BufferRecycler)
com.ning.compress.lzf.LZFUncompressor#<init>(com.ning.compress.DataHandler)
com.ning.compress.lzf.LZFUncompressor#<init>(com.ning.compress.DataHandler, com.ning.compress.BufferRecycler)
[TEST] Add a more restrictive thread leaks filter Today all threads are allowed to leak a suite. This is tricky since it essentially allows resource leaks by default where for instance test private TransportClients will never get closed and consume resources influencing other tests. It also hides threads that are not fully under elasticsearchs control like the Lucene TimeLimitingCollector thread. This commit restricts the threads that can leak a suite to the threads spawned from testclusters and fixes sevearl places that leaked threads. Closes #7833
2014-09-22 16:15:59 -04:00
@defaultMessage Spawns a new thread which is solely under lucenes control use ThreadPool#estimatedTimeInMillisCounter instead
org.apache.lucene.search.TimeLimitingCollector#getGlobalTimerThread()
org.apache.lucene.search.TimeLimitingCollector#getGlobalCounter()