Class BufferedUpdatesStream

java.lang.Object
org.apache.lucene.index.BufferedUpdatesStream
All Implemented Interfaces:
Accountable

final class BufferedUpdatesStream extends Object implements Accountable
Tracks the stream of FrozenBufferedUpdates. When DocumentsWriterPerThread flushes, its buffered deletes and updates are appended to this stream and immediately resolved (to actual docIDs, per segment) using the indexing thread that triggered the flush for concurrency. When a merge kicks off, we sync to ensure all resolving packets complete. We also apply to all segments when NRT reader is pulled, commit/close is called, or when too many deletes or updates are buffered and must be flushed (by RAM usage or by count).

Each packet is assigned a generation, and each flushed or merged segment is also assigned a generation, so we can track which BufferedDeletes packets to apply to any given segment.

  • Field Details

  • Constructor Details

    • BufferedUpdatesStream

      BufferedUpdatesStream(InfoStream infoStream)
  • Method Details

    • push

      long push(FrozenBufferedUpdates packet)
    • getPendingUpdatesCount

      int getPendingUpdatesCount()
    • clear

      void clear()
      Only used by IW.rollback
    • any

      boolean any()
    • numTerms

      int numTerms()
    • ramBytesUsed

      public long ramBytesUsed()
      Description copied from interface: Accountable
      Return the memory usage of this object in bytes. Negative values are illegal.
      Specified by:
      ramBytesUsed in interface Accountable
    • waitApplyAll

      void waitApplyAll(IndexWriter writer) throws IOException
      Waits for all in-flight packets, which are already being resolved concurrently by indexing threads, to finish. Returns true if there were any new deletes or updates. This is called for refresh, commit.
      Throws:
      IOException
    • stillRunning

      boolean stillRunning(long delGen)
      Returns true if this delGen is still running.
    • finishedSegment

      void finishedSegment(long delGen)
    • finished

      void finished(FrozenBufferedUpdates packet)
      Called by indexing threads once they are fully done resolving all deletes for the provided delGen. We track the completed delGens and record the maximum delGen for which all prior delGens, inclusive, are completed, so that it's safe for doc values updates to apply and write.
    • getCompletedDelGen

      long getCompletedDelGen()
      All frozen packets up to and including this del gen are guaranteed to be finished.
    • waitApplyForMerge

      void waitApplyForMerge(List<SegmentCommitInfo> mergeInfos, IndexWriter writer) throws IOException
      Waits only for those in-flight packets that apply to these merge segments. This is called when a merge needs to finish and must ensure all deletes to the merging segments are resolved.
      Throws:
      IOException
    • waitApply

      private void waitApply(Set<FrozenBufferedUpdates> waitFor, IndexWriter writer) throws IOException
      Throws:
      IOException
    • getNextGen

      long getNextGen()
    • checkDeleteStats

      private boolean checkDeleteStats()