Package org.apache.pdfbox.pdfparser
Class COSParser
java.lang.Object
org.apache.pdfbox.pdfparser.BaseParser
org.apache.pdfbox.pdfparser.COSParser
PDF-Parser which first reads startxref and xref tables in order to know valid objects and parse only these objects.
First
PDFParser.parse()
or FDFParser.parse()
must be called before page objects
can be retrieved, e.g. PDFParser.getPDDocument()
.
This class is a much enhanced version of QuickParser
presented in PDFBOX-1104 by Jeremy Villalobos.-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate AccessPermission
private Map<COSObjectKey,
Long> Contains all found objects of a brute force search.private static final int
How many trailing bytes to read for EOF marker.private PDEncryption
private static final byte[]
private static final byte[]
protected static final char[]
EOF-marker.private static final String
private static final String
protected long
file length.protected boolean
private boolean
is parser using auto healing capacity ?private String
private InputStream
private Long
private static final org.apache.commons.logging.Log
private static final long
protected static final char[]
obj-marker.private static final char[]
ObjStream-marker.private String
private static final String
private static final String
private int
how many trailing bytes to read for EOF marker.protected SecurityHandler
The security handler.protected final RandomAccessRead
private static final char[]
private final byte[]
private static final int
private final byte[]
private static final int
static final String
The range within the %%EOF marker will be searched.static final String
Only parse the PDF file minimally allowing access to basic information.static final String
The prefix for the temp file being used.private static final char[]
trailer-marker.private long
private boolean
private static final int
private static final char[]
private static final char[]
protected XrefTrailerResolver
Collects all Xref/trailer objects and resolves them into single object using startxref reference.Fields inherited from class org.apache.pdfbox.pdfparser.BaseParser
A, ASCII_CR, ASCII_LF, B, D, DEF, document, E, ENDOBJ_STRING, ENDSTREAM_STRING, J, M, MAX_LENGTH_LONG, N, O, R, S, seqSource, STREAM_STRING, T
-
Constructor Summary
ConstructorsConstructorDescriptionCOSParser
(RandomAccessRead source) Default constructor.COSParser
(RandomAccessRead source, String password, InputStream keyStore, String keyAlias) Constructor for encrypted pdfs. -
Method Summary
Modifier and TypeMethodDescriptionprivate void
addExcludedToList
(COSName[] excludeObjects, COSDictionary dict, Set<Long> parsedObjects) private void
addNewToList
(Queue<COSBase> toBeParsedList, Collection<COSBase> newObjects, Set<Long> addedObjects) Adds all from newObjects to toBeParsedList if it is not an COSObject or we didn't add this COSObject already (checked via addedObjects).private void
Adds newObject to toBeParsedList if it is not an COSObject or we didn't add this COSObject already (checked via addedObjects).private void
Brute force search for the last EOF marker.private void
Brute force search for every object in the pdf.private void
Brute force search for all object streams.private boolean
bfSearchForTrailer
(COSDictionary trailer) Brute force search for all trailer marker.private long
bfSearchForXRef
(long xrefOffset, boolean streamsOnly) Search for the offset of the given xref table/stream among those found by a brute force search.private void
Brute force search for all /XRef entries (streams).private void
Brute force search for all xref entries (tables).private long
calculateXRefFixedOffset
(long objectOffset, boolean streamsOnly) Try to find a fixed offset for the given xref table/stream.protected void
checkPages
(COSDictionary root) Check if all entries of the pages dictionary are present.private int
checkPagesDictionary
(COSDictionary pagesDict, Set<COSObject> set) private long
checkXRefOffset
(long startXRefOffset) Check if the cross reference table/stream can be found at the current offset.private void
Check the XRef table by dereferencing all objects and fixing the offset if necessary.private boolean
checkXRefStreamOffset
(long startXRefOffset) Check if the cross reference stream can be found at the current offset.private COSObject
compareCOSObjects
(COSObject newObject, Long newOffset, COSObject currentObject, Long currentOffset) private COSObjectKey
findObjectKey
(COSObjectKey objectKey, long offset) Check if the given object can be found at the given offset.This will get the AccessPermission.This will get the document that was parsed.This will get the encryption dictionary.private COSNumber
Returns length value referred to or defined in given object.private long
getObjectId
(COSObject obj) Creates a unique object id using object number and object generation number.protected final long
Looks for and parses startxref.protected boolean
isCatalog
(COSDictionary dictionary) Tell if the dictionary is a PDF catalog.private boolean
isInfo
(COSDictionary dictionary) Tell if the dictionary is an info dictionary.boolean
Return true if parser is lenient.private boolean
isString
(byte[] string) Checks if the given string can be found at the current offset.private boolean
isString
(char[] string) Checks if the given string can be found at the current offset.protected int
lastIndexOf
(char[] pattern, byte[] buf, int endOff) Searches last appearance of pattern within buffer.protected COSStream
This will read a COSStream from the input stream using length attribute within dictionary.private void
parseDictionaryRecursive
(COSObject dictionaryObject) Resolves all not already parsed objects of a dictionary recursively.protected void
parseDictObjects
(COSDictionary dict, COSName... excludeObjects) Will parse every object necessary to load a single page from the pdf document.protected boolean
Parse the header of a fdf.private void
parseFileObject
(Long offsetOrObjstmObNr, COSObjectKey objKey, COSObject pdfObject) private boolean
parseHeader
(String headerMarker, String defaultVersion) protected COSBase
parseObjectDynamically
(long objNr, int objGenNr, boolean requireExistingNotCompressedObj) This will parse the next object from the stream and add it to the local state.protected final COSBase
parseObjectDynamically
(COSObject obj, boolean requireExistingNotCompressedObj) This will parse the next object from the stream and add it to the local state.private void
parseObjectStream
(int objstmObjNr) protected boolean
Parse the header of a pdf.private long
This will parse the startxref section from the stream.private boolean
This will parse the trailer from the stream and add it to the state.protected COSBase
Parse the values of the trailer dictionary and return the root object.protected COSDictionary
parseXref
(long startXRefOffset) Parses cross reference tables.private long
parseXrefObjStream
(long objByteOffset, boolean isStandalone) Parses an xref object stream starting with indirect object id.private void
parseXrefStream
(COSStream stream, long objByteOffset, boolean isStandalone) Fills XRefTrailerResolver with data of given stream.protected boolean
parseXrefTable
(long startByteOffset) This will parse the xref table from the stream and add it to the state The XrefTable contents are ignored.private void
Prepare for decryption.private void
This method will read through the current stream object until we find the keyword "endstream" meaning we're at the end of this object.private void
readValidStream
(OutputStream out, COSNumber streamLengthObj) protected final COSDictionary
Rebuild the trailer dictionary if startxref can't be found.private COSDictionary
retrieveCOSDictionary
(COSObject object) private COSDictionary
retrieveCOSDictionary
(COSObjectKey key, long offset) protected COSDictionary
Read the trailer information and provide a COSDictionary containing the trailer information.private boolean
searchForTrailerItems
(COSDictionary trailer) Search for the different parts of the trailer dictionary.private long
searchNearestValue
(List<Long> values, long offset) void
setEOFLookupRange
(int byteCount) Sets how many trailing bytes of PDF file are searched for EOF marker and 'startxref' marker.void
setLenient
(boolean lenient) Change the parser leniency flag.private boolean
validateStreamLength
(long streamLength) private boolean
validateXrefOffsets
(Map<COSObjectKey, Long> xrefOffset) Methods inherited from class org.apache.pdfbox.pdfparser.BaseParser
isClosing, isClosing, isDigit, isDigit, isEndOfName, isEOL, isEOL, isSpace, isSpace, isWhitespace, isWhitespace, parseBoolean, parseCOSArray, parseCOSDictionary, parseCOSName, parseCOSString, parseDirObject, readExpectedChar, readExpectedString, readExpectedString, readGenerationNumber, readInt, readLine, readLong, readObjectNumber, readString, readString, readStringNumber, skipSpaces, skipWhiteSpaces
-
Field Details
-
PDF_HEADER
- See Also:
-
FDF_HEADER
- See Also:
-
PDF_DEFAULT_VERSION
- See Also:
-
FDF_DEFAULT_VERSION
- See Also:
-
XREF_TABLE
private static final char[] XREF_TABLE -
XREF_STREAM
private static final char[] XREF_STREAM -
STARTXREF
private static final char[] STARTXREF -
ENDSTREAM
private static final byte[] ENDSTREAM -
ENDOBJ
private static final byte[] ENDOBJ -
MINIMUM_SEARCH_OFFSET
private static final long MINIMUM_SEARCH_OFFSET- See Also:
-
X
private static final int X- See Also:
-
STRMBUFLEN
private static final int STRMBUFLEN- See Also:
-
strmBuf
private final byte[] strmBuf -
source
-
accessPermission
-
keyStoreInputStream
-
password
-
keyAlias
-
SYSPROP_PARSEMINIMAL
Only parse the PDF file minimally allowing access to basic information.- See Also:
-
SYSPROP_EOFLOOKUPRANGE
The range within the %%EOF marker will be searched. Useful if there are additional characters after %%EOF within the PDF.- See Also:
-
DEFAULT_TRAIL_BYTECOUNT
private static final int DEFAULT_TRAIL_BYTECOUNTHow many trailing bytes to read for EOF marker.- See Also:
-
EOF_MARKER
protected static final char[] EOF_MARKEREOF-marker. -
OBJ_MARKER
protected static final char[] OBJ_MARKERobj-marker. -
TRAILER_MARKER
private static final char[] TRAILER_MARKERtrailer-marker. -
OBJ_STREAM
private static final char[] OBJ_STREAMObjStream-marker. -
trailerOffset
private long trailerOffset -
fileLen
protected long fileLenfile length. -
isLenient
private boolean isLenientis parser using auto healing capacity ? -
initialParseDone
protected boolean initialParseDone -
trailerWasRebuild
private boolean trailerWasRebuild -
bfSearchCOSObjectKeyOffsets
Contains all found objects of a brute force search. -
lastEOFMarker
-
bfSearchXRefTablesOffsets
-
bfSearchXRefStreamsOffsets
-
encryption
-
securityHandler
The security handler. -
readTrailBytes
private int readTrailByteshow many trailing bytes to read for EOF marker. -
LOG
private static final org.apache.commons.logging.Log LOG -
xrefTrailerResolver
Collects all Xref/trailer objects and resolves them into single object using startxref reference. -
TMP_FILE_PREFIX
The prefix for the temp file being used.- See Also:
-
STREAMCOPYBUFLEN
private static final int STREAMCOPYBUFLEN- See Also:
-
streamCopyBuf
private final byte[] streamCopyBuf
-
-
Constructor Details
-
COSParser
Default constructor.- Parameters:
source
- input representing the pdf.
-
COSParser
Constructor for encrypted pdfs.- Parameters:
source
- input representing the pdf.password
- password to be used for decryption.keyStore
- key store to be used for decryption when using public key securitykeyAlias
- alias to be used for decryption when using public key security
-
-
Method Details
-
setEOFLookupRange
public void setEOFLookupRange(int byteCount) Sets how many trailing bytes of PDF file are searched for EOF marker and 'startxref' marker. If not set we use default valueDEFAULT_TRAIL_BYTECOUNT
.We check that new value is at least 16. However for practical use cases this value should not be lower than 1000; even 2000 was found to not be enough in some cases where some trailing garbage like HTML snippets followed the EOF marker.
In case system property
SYSPROP_EOFLOOKUPRANGE
is defined this value will be set on initialization but can be overwritten later.- Parameters:
byteCount
- number of trailing bytes
-
retrieveTrailer
Read the trailer information and provide a COSDictionary containing the trailer information.- Returns:
- a COSDictionary containing the trailer information
- Throws:
IOException
- if something went wrong
-
parseXref
Parses cross reference tables.- Parameters:
startXRefOffset
- start offset of the first table- Returns:
- the trailer dictionary
- Throws:
IOException
- if something went wrong
-
parseXrefObjStream
Parses an xref object stream starting with indirect object id.- Returns:
- value of PREV item in dictionary or
-1
if no such item exists - Throws:
IOException
-
getStartxrefOffset
Looks for and parses startxref. We first look for last '%%EOF' marker (within lastDEFAULT_TRAIL_BYTECOUNT
bytes (or range set viasetEOFLookupRange(int)
) and go back to findstartxref
.- Returns:
- the offset of StartXref
- Throws:
IOException
- If something went wrong.
-
lastIndexOf
protected int lastIndexOf(char[] pattern, byte[] buf, int endOff) Searches last appearance of pattern within buffer. Lookup before _lastOff and goes back until 0.- Parameters:
pattern
- pattern to search forbuf
- buffer to search pattern inendOff
- offset (exclusive) where lookup starts at- Returns:
- start offset of pattern within buffer or
-1
if pattern could not be found
-
isLenient
public boolean isLenient()Return true if parser is lenient. Meaning auto healing capacity of the parser are used.- Returns:
- true if parser is lenient
-
setLenient
public void setLenient(boolean lenient) Change the parser leniency flag. This method can only be called before the parsing of the file.- Parameters:
lenient
- try to handle malformed PDFs.
-
getObjectId
Creates a unique object id using object number and object generation number. (requires object number < 2^31)) -
addNewToList
private void addNewToList(Queue<COSBase> toBeParsedList, Collection<COSBase> newObjects, Set<Long> addedObjects) Adds all from newObjects to toBeParsedList if it is not an COSObject or we didn't add this COSObject already (checked via addedObjects). -
addNewToList
Adds newObject to toBeParsedList if it is not an COSObject or we didn't add this COSObject already (checked via addedObjects). Simple objects are not added because nothing is done with them when toBeParsedList is processed. -
parseDictObjects
Will parse every object necessary to load a single page from the pdf document. We try our best to order objects according to offset in file before reading to minimize seek operations.- Parameters:
dict
- the COSObject from the parent pages.excludeObjects
- dictionary object reference entries with these names will not be parsed- Throws:
IOException
- if something went wrong
-
addExcludedToList
private void addExcludedToList(COSName[] excludeObjects, COSDictionary dict, Set<Long> parsedObjects) -
parseObjectDynamically
protected final COSBase parseObjectDynamically(COSObject obj, boolean requireExistingNotCompressedObj) throws IOException This will parse the next object from the stream and add it to the local state.- Parameters:
obj
- object to be parsed (we only take object number and generation number for lookup start offset)requireExistingNotCompressedObj
- iftrue
object to be parsed must not be contained within compressed stream- Returns:
- the parsed object (which is also added to document object)
- Throws:
IOException
- If an IO error occurs.
-
parseObjectDynamically
protected COSBase parseObjectDynamically(long objNr, int objGenNr, boolean requireExistingNotCompressedObj) throws IOException This will parse the next object from the stream and add it to the local state. It's reduced to parsing an indirect object.- Parameters:
objNr
- object number of object to be parsedobjGenNr
- object generation number of object to be parsedrequireExistingNotCompressedObj
- iftrue
the object to be parsed must be defined in xref (comment: null objects may be missing from xref) and it must not be a compressed object within object stream (this is used to circumvent being stuck in a loop in a malicious PDF)- Returns:
- the parsed object (which is also added to document object)
- Throws:
IOException
- If an IO error occurs.
-
parseFileObject
private void parseFileObject(Long offsetOrObjstmObNr, COSObjectKey objKey, COSObject pdfObject) throws IOException - Throws:
IOException
-
parseObjectStream
- Throws:
IOException
-
getLength
Returns length value referred to or defined in given object.- Throws:
IOException
-
parseCOSStream
This will read a COSStream from the input stream using length attribute within dictionary. If length attribute is a indirect reference it is first resolved to get the stream length. This means we copy stream data without testing for 'endstream' or 'endobj' and thus it is no problem if these keywords occur within stream. We require 'endstream' to be found after stream data is read.- Parameters:
dic
- dictionary that goes with this stream.- Returns:
- parsed pdf stream.
- Throws:
IOException
- if an error occurred reading the stream, like problems with reading length attribute, stream does not end with 'endstream' after data read, stream too short etc.
-
readUntilEndStream
This method will read through the current stream object until we find the keyword "endstream" meaning we're at the end of this object. Some pdf files, however, forget to write some endstream tags and just close off objects with an "endobj" tag so we have to handle this case as well. This method is optimized using buffered IO and reduced number of byte compare operations.- Parameters:
out
- stream we write out to.- Throws:
IOException
- if something went wrong
-
readValidStream
- Throws:
IOException
-
validateStreamLength
- Throws:
IOException
-
checkXRefOffset
Check if the cross reference table/stream can be found at the current offset.- Parameters:
startXRefOffset
-- Returns:
- the revised offset
- Throws:
IOException
-
checkXRefStreamOffset
Check if the cross reference stream can be found at the current offset.- Parameters:
startXRefOffset
- the expected start offset of the XRef stream- Returns:
- the revised offset
- Throws:
IOException
- if something went wrong
-
calculateXRefFixedOffset
Try to find a fixed offset for the given xref table/stream.- Parameters:
objectOffset
- the given offset where to look atstreamsOnly
- search for xref streams only- Returns:
- the fixed offset
- Throws:
IOException
- if something went wrong
-
validateXrefOffsets
- Throws:
IOException
-
checkXrefOffsets
Check the XRef table by dereferencing all objects and fixing the offset if necessary.- Throws:
IOException
- if something went wrong.
-
findObjectKey
Check if the given object can be found at the given offset. Returns the provided object key if everything is ok. If the generation number differs it will be fixed and a new object key is returned.- Parameters:
objectKey
- the key of object we are looking foroffset
- the offset where to look- Returns:
- returns the found/fixed object key
- Throws:
IOException
- if something went wrong
-
bfSearchForObjects
Brute force search for every object in the pdf.- Throws:
IOException
- if something went wrong
-
bfSearchForXRef
Search for the offset of the given xref table/stream among those found by a brute force search.- Parameters:
streamsOnly
- search for xref streams only- Returns:
- the offset of the xref entry
- Throws:
IOException
- if something went wrong
-
searchNearestValue
-
bfSearchForTrailer
Brute force search for all trailer marker.- Throws:
IOException
- if something went wrong
-
bfSearchForLastEOFMarker
Brute force search for the last EOF marker.- Throws:
IOException
- if something went wrong
-
bfSearchForObjStreams
Brute force search for all object streams.- Throws:
IOException
- if something went wrong
-
bfSearchForXRefTables
Brute force search for all xref entries (tables).- Throws:
IOException
- if something went wrong
-
bfSearchForXRefStreams
Brute force search for all /XRef entries (streams).- Throws:
IOException
- if something went wrong
-
rebuildTrailer
Rebuild the trailer dictionary if startxref can't be found.- Returns:
- the rebuild trailer dictionary
- Throws:
IOException
- if something went wrong
-
searchForTrailerItems
Search for the different parts of the trailer dictionary.- Parameters:
trailer
-- Returns:
- true if the root was found, false if not.
- Throws:
IOException
-
compareCOSObjects
-
retrieveCOSDictionary
- Throws:
IOException
-
retrieveCOSDictionary
- Throws:
IOException
-
checkPages
Check if all entries of the pages dictionary are present. Those which can't be dereferenced are removed.- Parameters:
root
- the root dictionary of the pdf
-
checkPagesDictionary
-
isCatalog
Tell if the dictionary is a PDF catalog. Override this for an FDF catalog.- Parameters:
dictionary
-- Returns:
- true if the given dictionary is a root dictionary
-
isInfo
Tell if the dictionary is an info dictionary.- Parameters:
dictionary
-- Returns:
- true if the given dictionary is an info dictionary
-
parseStartXref
This will parse the startxref section from the stream. The startxref value is ignored.- Returns:
- the startxref value or -1 on parsing error
- Throws:
IOException
- If an IO error occurs.
-
isString
Checks if the given string can be found at the current offset.- Parameters:
string
- the bytes of the string to look for- Returns:
- true if the bytes are in place, false if not
- Throws:
IOException
- if something went wrong
-
isString
Checks if the given string can be found at the current offset.- Parameters:
string
- the bytes of the string to look for- Returns:
- true if the bytes are in place, false if not
- Throws:
IOException
- if something went wrong
-
parseTrailer
This will parse the trailer from the stream and add it to the state.- Returns:
- false on parsing error
- Throws:
IOException
- If an IO error occurs.
-
parsePDFHeader
Parse the header of a pdf.- Returns:
- true if a PDF header was found
- Throws:
IOException
- if something went wrong
-
parseFDFHeader
Parse the header of a fdf.- Returns:
- true if a FDF header was found
- Throws:
IOException
- if something went wrong
-
parseHeader
- Throws:
IOException
-
parseXrefTable
This will parse the xref table from the stream and add it to the state The XrefTable contents are ignored.- Parameters:
startByteOffset
- the offset to start at- Returns:
- false on parsing error
- Throws:
IOException
- If an IO error occurs.
-
parseXrefStream
private void parseXrefStream(COSStream stream, long objByteOffset, boolean isStandalone) throws IOException Fills XRefTrailerResolver with data of given stream. Stream must be of type XRef.- Parameters:
stream
- the stream to be readobjByteOffset
- the offset to start atisStandalone
- should be set to true if the stream is not part of a hybrid xref table- Throws:
IOException
- if there is an error parsing the stream
-
getDocument
This will get the document that was parsed. The document must be parsed before this is called. When you are done with this document you must call close() on it to release resources.- Returns:
- The document that was parsed.
- Throws:
IOException
- If there is an error getting the document.
-
getEncryption
This will get the encryption dictionary. The document must be parsed before this is called.- Returns:
- The encryption dictionary of the document that was parsed.
- Throws:
IOException
- If there is an error getting the document.
-
getAccessPermission
This will get the AccessPermission. The document must be parsed before this is called.- Returns:
- The access permission of document that was parsed.
- Throws:
IOException
- If there is an error getting the document.
-
parseTrailerValuesDynamically
Parse the values of the trailer dictionary and return the root object.- Parameters:
trailer
- The trailer dictionary.- Returns:
- The parsed root object.
- Throws:
IOException
- If an IO error occurs or if the root object is missing in the trailer dictionary.
-
prepareDecryption
Prepare for decryption.- Throws:
InvalidPasswordException
- If the password is incorrect.IOException
- if something went wrong
-
parseDictionaryRecursive
Resolves all not already parsed objects of a dictionary recursively.- Parameters:
dictionaryObject
- dictionary to be parsed- Throws:
IOException
- if something went wrong
-