public class ScratchFile
extends java.lang.Object
implements java.io.Closeable
RandomAccess
buffers each having its set of pages (implemented by
ScratchFileBuffer
). A buffer is created calling createBuffer()
.
Pages can be stored in main memory or in a temporary file. A mixed mode is supported storing a certain amount of pages in memory and only the additional ones in temporary file (defined by maximum main memory to be used).
Pages can be marked as 'free' in order to re-use them. For in-memory pages this will release the used memory while for pages in temporary file this simply marks the area as free to re-use.
If a temporary file was created (done with the first page to be stored
in temporary file) it is deleted when close()
is called.
Using this class for RandomAccess
buffers allows for a direct control
on the maximum memory usage and allows processing large files for which we
otherwise would get an OutOfMemoryError
in case of using RandomAccessBuffer
.
This base class for providing pages is thread safe (the buffer implementations are not).
Modifier and Type | Field and Description |
---|---|
private static int |
ENLARGE_PAGE_COUNT
number of pages by which we enlarge the scratch file (reduce I/O-operations)
|
private java.io.File |
file
scratch file; only to be accessed under synchronization of
ioLock |
private java.util.BitSet |
freePages |
private static int |
INIT_UNRESTRICTED_MAINMEM_PAGECOUNT
in case of unrestricted main memory usage this is the initial number of pages
inMemoryPages is setup for |
private int |
inMemoryMaxPageCount |
private byte[][] |
inMemoryPages
holds pointers to in-memory page content; will be initialized once in case of restricted
main memory, otherwise it is enlarged as needed and first initialized to a size of
INIT_UNRESTRICTED_MAINMEM_PAGECOUNT |
private java.lang.Object |
ioLock |
private boolean |
isClosed |
private static org.apache.commons.logging.Log |
LOG |
private boolean |
maxMainMemoryIsRestricted |
private int |
maxPageCount |
private static int |
PAGE_SIZE |
private int |
pageCount |
private java.io.RandomAccessFile |
raf
random access to scratch file; only to be accessed under synchronization of
ioLock |
private java.io.File |
scratchFileDirectory |
private boolean |
useScratchFile |
Constructor and Description |
---|
ScratchFile(java.io.File scratchFileDirectory)
Initializes page handler.
|
ScratchFile(MemoryUsageSetting memUsageSetting)
Initializes page handler.
|
Modifier and Type | Method and Description |
---|---|
(package private) void |
checkClosed()
Checks if this page handler has already been closed.
|
void |
close()
Closes and deletes the temporary file.
|
RandomAccess |
createBuffer()
Creates a new buffer using this page handler.
|
RandomAccess |
createBuffer(java.io.InputStream input)
Creates a new buffer using this page handler and initializes it with the
data read from provided input stream (input stream is copied to buffer).
|
private void |
enlarge()
This will provide new free pages by either enlarging the scratch file
by a number of pages defined by
ENLARGE_PAGE_COUNT - in case
scratch file usage is allowed - or increase the inMemoryPages
array in case main memory was not restricted. |
static ScratchFile |
getMainMemoryOnlyInstance()
Getter for an instance using only unrestricted main memory for buffering
(same as
new ScratchFile(MemoryUsageSetting.setupMainMemoryOnly()) ). |
(package private) int |
getNewPage()
Returns a new free page, either from free page pool
or by enlarging scratch file (may be created).
|
(package private) int |
getPageSize()
Returns byte size of a page.
|
(package private) void |
markPagesAsFree(int[] pageIndexes,
int off,
int count)
Allows a buffer which is cleared/closed to release its pages to be re-used.
|
(package private) byte[] |
readPage(int pageIdx)
Reads the page with specified index.
|
(package private) void |
writePage(int pageIdx,
byte[] page)
Writes updated page.
|
private static final org.apache.commons.logging.Log LOG
private static final int ENLARGE_PAGE_COUNT
private static final int INIT_UNRESTRICTED_MAINMEM_PAGECOUNT
inMemoryPages
is setup forprivate static final int PAGE_SIZE
private final java.lang.Object ioLock
private final java.io.File scratchFileDirectory
private java.io.File file
ioLock
private java.io.RandomAccessFile raf
ioLock
private volatile int pageCount
private final java.util.BitSet freePages
private volatile byte[][] inMemoryPages
INIT_UNRESTRICTED_MAINMEM_PAGECOUNT
private final int inMemoryMaxPageCount
private final int maxPageCount
private final boolean useScratchFile
private final boolean maxMainMemoryIsRestricted
private volatile boolean isClosed
public ScratchFile(java.io.File scratchFileDirectory) throws java.io.IOException
scratchFileDirectory
is supplied,
then the scratch file will be created in that directory.
All pages will be stored in the scratch file.
scratchFileDirectory
- The directory in which to create the scratch file
or null
to created it in the default temporary directory.java.io.IOException
- If scratch file directory was given but don't exist.public ScratchFile(MemoryUsageSetting memUsageSetting) throws java.io.IOException
scratchFileDirectory
is supplied,
then the scratch file will be created in that directory.
Depending on the size of allowed memory usage a number of pages (memorySize/PAGE_SIZE
)
will be stored in-memory and only additional pages will be written to/read from scratch file.
memUsageSetting
- set how memory/temporary files are used for buffering streams etc.java.io.IOException
- If scratch file directory was given but don't exist.public static ScratchFile getMainMemoryOnlyInstance()
new ScratchFile(MemoryUsageSetting.setupMainMemoryOnly())
).int getNewPage() throws java.io.IOException
java.io.IOException
private void enlarge() throws java.io.IOException
ENLARGE_PAGE_COUNT
- in case
scratch file usage is allowed - or increase the inMemoryPages
array in case main memory was not restricted. If neither of both is
allowed/the case than free pages count won't be changed. The same is true
if no new pages could be added because we reached the maximum of
Integer.MAX_VALUE
pages.
If scratch file usage is allowed and scratch file does not exist already it will be created.
Only to be called under synchronization on freePages
.
java.io.IOException
int getPageSize()
byte[] readPage(int pageIdx) throws java.io.IOException
pageIdx
- index of page to readPAGE_SIZE
filled with page data read from filejava.io.IOException
void writePage(int pageIdx, byte[] page) throws java.io.IOException
inMemoryMaxPageCount
or is written to scratch file.
Provided page byte array must not be re-used for other pages since we store it as is in case of in-memory handling.
pageIdx
- index of page to writepage
- page to write (length has to be 4096)java.io.IOException
- in case page index is out of range or page has wrong length
or writing to file failedvoid checkClosed() throws java.io.IOException
IOException
is thrown.java.io.IOException
- If close()
has already been called.public RandomAccess createBuffer() throws java.io.IOException
java.io.IOException
- If an error occurred.public RandomAccess createBuffer(java.io.InputStream input) throws java.io.IOException
input
- The input stream that is to be copied into the buffer.java.io.IOException
- If an error occurred.void markPagesAsFree(int[] pageIndexes, int off, int count)
pageIndexes
- pages indexes of pages to releasecount
- number of page indexes contained in provided arraypublic void close() throws java.io.IOException
close
in interface java.io.Closeable
close
in interface java.lang.AutoCloseable
java.io.IOException
- If there was a problem closing or deleting the temporary file.