java.lang.Object
org.apache.lucene.store.Directory
org.apache.lucene.store.FilterDirectory
org.apache.lucene.misc.store.DirectIODirectory
- All Implemented Interfaces:
Closeable
,AutoCloseable
A
Directory
implementation for all Unixes and Windows that uses DIRECT I/O to bypass OS
level IO caching during merging. For all other cases (searching, writing) we delegate to the
provided Directory instance.
See Overview for more details.
WARNING: this code is very new and quite easily could contain horrible bugs.
This directory passes Solr and Lucene tests on Linux, OS X, and Windows; other systems should work but have not been tested! Use at your own risk.
@throws UnsupportedOperationException if the operating system, file system or JDK does not support Direct I/O or a sufficient equivalent.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprivate static final class
private static final class
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate final int
static final int
Default buffer size before writing to disk (256 KB); larger means less IO load but more RAM and direct buffer storage space consumed during merging.static final long
Default min expected merge size before direct IO is used (10 MB):(package private) static final OpenOption
Reference tocom.sun.nio.file.ExtendedOpenOption.DIRECT
by reflective class and enum lookup.(package private) boolean
private final int
private final long
Fields inherited from class org.apache.lucene.store.FilterDirectory
in
-
Constructor Summary
ConstructorsConstructorDescriptionDirectIODirectory
(FSDirectory delegate) Create a new DirectIODirectory for the named location.DirectIODirectory
(FSDirectory delegate, int mergeBufferSize, long minBytesDirect) Create a new DirectIODirectory for the named location. -
Method Summary
Modifier and TypeMethodDescriptionvoid
close()
Closes the directory.createOutput
(String name, IOContext context) Creates a new, empty file in the directory and returns anIndexOutput
instance for appending data to this file.protected void
Ensures this directory is still open.private static OpenOption
Opens a stream for reading an existing file.protected boolean
useDirectIO
(String name, IOContext context, OptionalLong fileLength) Determines if direct IO should be used for a file.Methods inherited from class org.apache.lucene.store.FilterDirectory
createTempOutput, deleteFile, fileLength, getDelegate, getPendingDeletions, listAll, obtainLock, rename, sync, syncMetaData, toString, unwrap
Methods inherited from class org.apache.lucene.store.Directory
copyFrom, getTempFileName, openChecksumInput
-
Field Details
-
DEFAULT_MERGE_BUFFER_SIZE
public static final int DEFAULT_MERGE_BUFFER_SIZEDefault buffer size before writing to disk (256 KB); larger means less IO load but more RAM and direct buffer storage space consumed during merging.- See Also:
-
DEFAULT_MIN_BYTES_DIRECT
public static final long DEFAULT_MIN_BYTES_DIRECTDefault min expected merge size before direct IO is used (10 MB):- See Also:
-
blockSize
private final int blockSize -
mergeBufferSize
private final int mergeBufferSize -
minBytesDirect
private final long minBytesDirect -
isOpen
volatile boolean isOpen -
ExtendedOpenOption_DIRECT
Reference tocom.sun.nio.file.ExtendedOpenOption.DIRECT
by reflective class and enum lookup. There are two reasons for using this instead of directly referencing ExtendedOpenOption.DIRECT:- ExtendedOpenOption.DIRECT is OpenJDK's internal proprietary API. This API causes un-suppressible(?) warning to be emitted when compiling with --release flag and value N, where N is smaller than the version of javac used for compilation. For details, please refer to https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8259039.
- It is possible that Lucene is run using JDK that does not support ExtendedOpenOption.DIRECT. In such a case, dynamic lookup allows us to bail out with UnsupportedOperationException with meaningful error message.
This reference is
null
, if the JDK does not support direct I/O.
-
-
Constructor Details
-
DirectIODirectory
public DirectIODirectory(FSDirectory delegate, int mergeBufferSize, long minBytesDirect) throws IOException Create a new DirectIODirectory for the named location.- Parameters:
delegate
- Directory for non-merges, also used as reference to file system path.mergeBufferSize
- Size of buffer to use for merging.minBytesDirect
- Merges, or files to be opened for reading, smaller than this will not use direct IO. SeeDEFAULT_MIN_BYTES_DIRECT
anduseDirectIO(java.lang.String, org.apache.lucene.store.IOContext, java.util.OptionalLong)
.- Throws:
IOException
- If there is a low-level I/O error
-
DirectIODirectory
Create a new DirectIODirectory for the named location.- Parameters:
delegate
- Directory for non-merges, also used as reference to file system path.- Throws:
IOException
- If there is a low-level I/O error
-
-
Method Details
-
getDirectory
- Returns:
- the underlying file system directory
-
ensureOpen
Description copied from class:Directory
Ensures this directory is still open.- Overrides:
ensureOpen
in classFilterDirectory
- Throws:
AlreadyClosedException
- if this directory is closed.
-
useDirectIO
Determines if direct IO should be used for a file. By default this tests if it is a merge context and if the merge or file length extends the minimum size (seeDEFAULT_MIN_BYTES_DIRECT
). Subclasses may override method to enforce direct IO for specific file types.- Parameters:
name
- file name (unused by default implementation)context
- information about merge sizefileLength
- if available, gives the file length. Will be empty when requesting anIndexOutput
.- Returns:
true
if direct IO should be used;false
if input/output should be requested from delegate directory.
-
openInput
Description copied from class:Directory
Opens a stream for reading an existing file.This method must throw either
NoSuchFileException
orFileNotFoundException
ifname
points to a non-existing file.- Overrides:
openInput
in classFilterDirectory
- Parameters:
name
- the name of an existing file.- Throws:
IOException
- in case of I/O error
-
createOutput
Description copied from class:Directory
Creates a new, empty file in the directory and returns anIndexOutput
instance for appending data to this file.This method must throw
FileAlreadyExistsException
if the file already exists.- Overrides:
createOutput
in classFilterDirectory
- Parameters:
name
- the name of the file to create.- Throws:
IOException
- in case of I/O error
-
close
Description copied from class:Directory
Closes the directory.- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Overrides:
close
in classFilterDirectory
- Throws:
IOException
-
getDirectOpenOption
-