XRootD
Loading...
Searching...
No Matches
XrdPfc::Info Class Reference

Status of cached file. Can be read from and written into a binary file. More...

#include <XrdPfcInfo.hh>

+ Collaboration diagram for XrdPfc::Info:

Classes

struct  AStat
 Access statistics. More...
 
struct  Status
 
union  Status.__unnamed126__
 
struct  Status.__unnamed126__.__unnamed128__
 
struct  Store
 

Public Member Functions

 Info (XrdSysTrace *trace, bool prefetchBuffer=false)
 Constructor.
 
 ~Info ()
 Destructor.
 
void CalcCksumMd5 (unsigned char *buff, char *digest)
 
uint32_t CalcCksumStore ()
 Get cksum, MD5 is for backward compatibility with V2 and V3.
 
uint32_t CalcCksumSyncedAndAStats ()
 
void CompactifyAccessRecords ()
 Compactify access records to the configured maximum.
 
int CountBlocksNotWrittenInRng (int firstIdx, int lastIdx) const
 Check download status in given block range.
 
void DowngradeCkSumState (CkSumCheck_e css_ref)
 
size_t GetAccessCnt () const
 Get number of accesses.
 
int GetBitvecSizeInBytes () const
 Get size of download-state bit-vector in bytes.
 
long long GetBufferSize () const
 Get prefetch buffer size.
 
CkSumCheck_e GetCkSumState () const
 
const char * GetCkSumStateAsText () const
 
time_t GetCreationTime () const
 Get file size.
 
long long GetExpectedDataFileSize () const
 Get expected data file size.
 
long long GetFileSize () const
 Get file size.
 
const AStatGetLastAccessStats () const
 Get latest access stats.
 
int GetLastDownloadedBlock () const
 Get number of the last downloaded block.
 
bool GetLatestDetachTime (time_t &t) const
 Get latest detach time.
 
int GetNBlocks () const
 Get number of blocks represented in download-state bit-vector.
 
int GetNDownloadedBlocks () const
 Get number of downloaded blocks.
 
long long GetNDownloadedBytes () const
 Get number of downloaded bytes.
 
time_t GetNoCkSumTime () const
 
time_t GetNoCkSumTimeForUVKeep () const
 
XrdSysTraceGetTrace () const
 
int GetVersion ()
 Get version.
 
bool HasNoCkSumTime () const
 
bool IsCkSumAny () const
 
bool IsCkSumBoth () const
 
bool IsCkSumCache () const
 
bool IsCkSumNet () const
 
bool IsComplete () const
 Get complete status.
 
bool Read (XrdOssDF *fp, const char *dname, const char *fname=0)
 Read content of cinfo file into this object.
 
const std::vector< AStat > & RefAStats () const
 
const StoreRefStoredData () const
 Get stored data.
 
void ResetAllAccessStats ()
 Reset IO Stats.
 
void ResetCkSumCache ()
 
void ResetCkSumNet ()
 
void ResetNoCkSumTime ()
 
void ResizeBits ()
 Reserve bit vectors for file_size / buffer_size bytes.
 
void SetAllBitsSynced ()
 Mark all blocks as synced to disk.
 
void SetBitPrefetch (int i)
 Mark block as obtained through prefetch.
 
void SetBitSynced (int i)
 Mark block as synced to disk.
 
void SetBitWritten (int i)
 Mark block as written to disk.
 
void SetBufferSizeFileSizeAndCreationTime (long long bs, long long fs)
 
void SetCkSumState (CkSumCheck_e css)
 
bool TestBitPrefetch (int i) const
 Test if block at the given index has been prefetched.
 
bool TestBitWritten (int i) const
 Test if block at the given index is written to disk.
 
void UpdateDownloadCompleteStatus ()
 Update complete status.
 
bool Write (XrdOssDF *fp, const char *dname, const char *fname=0)
 
void WriteIOStat (Stats &s)
 Write bytes missed, hits, and disk.
 
void WriteIOStatAttach ()
 Write open time in the last entry of access statistics.
 
void WriteIOStatDetach (Stats &s)
 Write close time together with bytes missed, hits, and disk.
 
void WriteIOStatSingle (long long bytes_disk)
 Write single open/close time for given bytes read from disk.
 
void WriteIOStatSingle (long long bytes_disk, time_t att, time_t dtc)
 Write open/close with given time and bytes read from disk.
 

Static Public Attributes

static const char * m_traceID = "CInfo"
 
static const int s_defaultVersion = 4
 
static const char * s_infoExtension = ".cinfo"
 
static const size_t s_infoExtensionLen = strlen(Info::s_infoExtension)
 
static size_t s_maxNumAccess = 20
 

Protected Attributes

std::vector< AStatm_astats
 access records
 
int m_bitvecSizeInBits
 cached
 
unsigned char * m_buff_prefetch
 prefetch statistics
 
unsigned char * m_buff_synced
 disk written state vector
 
unsigned char * m_buff_written
 download state vector
 
bool m_complete
 cached; if false, set to true when missingBlocks hit zero
 
bool m_hasPrefetchBuffer
 constains current prefetch score
 
int m_missingBlocks
 cached, updated in SetBitWritten()
 
Store m_store
 
XrdSysTracem_trace
 
int m_version
 

Detailed Description

Status of cached file. Can be read from and written into a binary file.

Definition at line 44 of file XrdPfcInfo.hh.


Class Documentation

◆ XrdPfc::Info::Status.__unnamed126__

union XrdPfc::Info::Status.__unnamed126__

Definition at line 48 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed126__:
Class Members
struct Status.__unnamed126__.__unnamed128__ __unnamed__
unsigned int _raw_

◆ XrdPfc::Info::Status.__unnamed126__.__unnamed128__

struct XrdPfc::Info::Status.__unnamed126__.__unnamed128__

Definition at line 49 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed126__.__unnamed128__:
Class Members
int _free_bits_: 29
int f_cksum_check: 3 as in enum CkSumCheck_e

Constructor & Destructor Documentation

◆ Info()

Info::Info ( XrdSysTrace * trace,
bool prefetchBuffer = false )

Constructor.

Definition at line 125 of file XrdPfcInfo.cc.

125 :
126 m_trace(trace),
128 m_version(0),
131 m_complete(false),
132 m_hasPrefetchBuffer(prefetchBuffer),
133 m_cksCalcMd5(0)
134{}
unsigned char * m_buff_prefetch
prefetch statistics
unsigned char * m_buff_written
download state vector
unsigned char * m_buff_synced
disk written state vector
bool m_complete
cached; if false, set to true when missingBlocks hit zero
int m_missingBlocks
cached, updated in SetBitWritten()
int m_bitvecSizeInBits
cached
XrdSysTrace * m_trace
bool m_hasPrefetchBuffer
constains current prefetch score

◆ ~Info()

Info::~Info ( )

Destructor.

Definition at line 136 of file XrdPfcInfo.cc.

137{
138 if (m_buff_synced) free(m_buff_synced);
141 delete m_cksCalcMd5;
142}

References m_buff_prefetch, m_buff_synced, and m_buff_written.

Member Function Documentation

◆ CalcCksumMd5()

void Info::CalcCksumMd5 ( unsigned char * buff,
char * digest )

Definition at line 240 of file XrdPfcInfo.cc.

241{
242 if (m_cksCalcMd5)
243 m_cksCalcMd5->Init();
244 else
245 m_cksCalcMd5 = new XrdCksCalcmd5();
246
247 m_cksCalcMd5->Update((const char*)buff, GetBitvecSizeInBytes());
248 memcpy(digest, m_cksCalcMd5->Final(), 16);
249}
virtual void Update(const char *Buff, int BLen)=0
virtual void Init()=0
virtual char * Final()=0
int GetBitvecSizeInBytes() const
Get size of download-state bit-vector in bytes.

References XrdCksCalc::Final(), GetBitvecSizeInBytes(), XrdCksCalc::Init(), and XrdCksCalc::Update().

+ Here is the call graph for this function:

◆ CalcCksumStore()

uint32_t Info::CalcCksumStore ( )

Get cksum, MD5 is for backward compatibility with V2 and V3.

Definition at line 229 of file XrdPfcInfo.cc.

230{
231 return crc32c(0, &m_store, sizeof(Store));
232}
uint32_t crc32c(uint32_t crc, void const *buf, size_t len)

References crc32c(), and m_store.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CalcCksumSyncedAndAStats()

uint32_t Info::CalcCksumSyncedAndAStats ( )

Definition at line 234 of file XrdPfcInfo.cc.

235{
236 uint32_t cks = crc32c(0, m_buff_synced, GetBitvecSizeInBytes());
237 return crc32c(cks, m_astats.data(), m_astats.size() * sizeof(AStat));
238}
std::vector< AStat > m_astats
access records

References crc32c(), GetBitvecSizeInBytes(), m_astats, and m_buff_synced.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CompactifyAccessRecords()

void Info::CompactifyAccessRecords ( )

Compactify access records to the configured maximum.

Definition at line 381 of file XrdPfcInfo.cc.

382{
383 time_t now = time(0);
384
385 std::vector<AStat> &v = m_astats;
386
387 for (int i = 0; i < (int) v.size() - 1; ++i)
388 {
389 if (v[i].DetachTime == 0)
390 v[i].DetachTime = std::min(v[i].AttachTime + v[i].Duration / v[i].NumIos, v[i+1].AttachTime);
391 }
392
393 while (v.size() > s_maxNumAccess)
394 {
395 double min_s = 1e10;
396 int min_i = -1;
397
398 int M = (int) v.size() - 2;
399 for (int i = 0; i < M; ++i)
400 {
401 AStat &a = v[i], &b = v[i + 1];
402
403 time_t t = std::max((time_t) 1, (now - b.AttachTime) / 2 + (now - a.DetachTime) / 2);
404 double s = (double) (b.AttachTime - a.DetachTime) / t;
405
406 if (s < min_s)
407 {
408 min_s = s;
409 min_i = i;
410 }
411 }
412 assert(min_i != -1);
413
414 v[min_i].MergeWith(v[min_i + 1]);
415
416 v.erase(v.begin() + (min_i + 1));
417 }
418}
static size_t s_maxNumAccess

References XrdPfc::Info::AStat::DetachTime, m_astats, and s_maxNumAccess.

Referenced by Write().

+ Here is the caller graph for this function:

◆ CountBlocksNotWrittenInRng()

int XrdPfc::Info::CountBlocksNotWrittenInRng ( int firstIdx,
int lastIdx ) const
inline

Check download status in given block range.

Definition at line 456 of file XrdPfcInfo.hh.

457{
458 // TODO rewrite to use full byte comparisons outside of edges ?
459 // Also, it seems to be always called with firstIdx = 0, lastIdx = m_bitvecSizeInBits.
460 int cnt = 0;
461 for (int i = firstIdx; i < lastIdx; ++i)
462 if (! TestBitWritten(i)) ++cnt;
463
464 return cnt;
465}
bool TestBitWritten(int i) const
Test if block at the given index is written to disk.

References TestBitWritten().

Referenced by UpdateDownloadCompleteStatus().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ DowngradeCkSumState()

void XrdPfc::Info::DowngradeCkSumState ( CkSumCheck_e css_ref)
inline

Definition at line 299 of file XrdPfcInfo.hh.

299{ m_store.m_status.f_cksum_check &= css_ref; }
Status m_status
status information
Definition XrdPfcInfo.hh:87

References XrdPfc::Info::Store::m_status, and m_store.

◆ GetAccessCnt()

size_t XrdPfc::Info::GetAccessCnt ( ) const
inline

Get number of accesses.

Definition at line 265 of file XrdPfcInfo.hh.

265{ return m_store.m_accessCnt; }
size_t m_accessCnt
total access count for the file
Definition XrdPfcInfo.hh:86

References XrdPfc::Info::Store::m_accessCnt, and m_store.

Referenced by XrdPfc::File::GetAccessCnt().

+ Here is the caller graph for this function:

◆ GetBitvecSizeInBytes()

int XrdPfc::Info::GetBitvecSizeInBytes ( ) const
inline

Get size of download-state bit-vector in bytes.

Definition at line 433 of file XrdPfcInfo.hh.

434{
436 return ((m_bitvecSizeInBits - 1)/8 + 1);
437 else
438 return 0;
439}

References m_bitvecSizeInBits.

Referenced by CalcCksumMd5(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitPrefetch(), SetBitSynced(), SetBitWritten(), TestBitPrefetch(), TestBitWritten(), and Write().

+ Here is the caller graph for this function:

◆ GetBufferSize()

long long XrdPfc::Info::GetBufferSize ( ) const
inline

Get prefetch buffer size.

Definition at line 473 of file XrdPfcInfo.hh.

474{
475 return m_store.m_buffer_size;
476}
long long m_buffer_size
buffer / block size
Definition XrdPfcInfo.hh:82

References XrdPfc::Info::Store::m_buffer_size, and m_store.

Referenced by XrdPfc::File::GetBlockSize().

+ Here is the caller graph for this function:

◆ GetCkSumState()

CkSumCheck_e XrdPfc::Info::GetCkSumState ( ) const
inline

Definition at line 290 of file XrdPfcInfo.hh.

290{ return (CkSumCheck_e) m_store.m_status.f_cksum_check; }

References XrdPfc::Info::Store::m_status, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile().

+ Here is the caller graph for this function:

◆ GetCkSumStateAsText()

const char * Info::GetCkSumStateAsText ( ) const

Definition at line 251 of file XrdPfcInfo.cc.

252{
253 switch (m_store.m_status.f_cksum_check) {
254 case CSChk_None : return "none";
255 case CSChk_Cache : return "cache";
256 case CSChk_Net : return "net";
257 case CSChk_Both : return "both";
258 default : return "unknown";
259 }
260}
@ CSChk_Cache

References XrdPfc::CSChk_Both, XrdPfc::CSChk_Cache, XrdPfc::CSChk_Net, XrdPfc::CSChk_None, XrdPfc::Info::Store::m_status, and m_store.

◆ GetCreationTime()

time_t XrdPfc::Info::GetCreationTime ( ) const
inline

Get file size.

Definition at line 281 of file XrdPfcInfo.hh.

281{ return m_store.m_creationTime; }
time_t m_creationTime
time the info file was created
Definition XrdPfcInfo.hh:84

References XrdPfc::Info::Store::m_creationTime, and m_store.

◆ GetExpectedDataFileSize()

long long XrdPfc::Info::GetExpectedDataFileSize ( ) const
inline

Get expected data file size.

Definition at line 424 of file XrdPfcInfo.hh.

425{
426 int last_block = GetLastDownloadedBlock();
427 if (last_block == m_bitvecSizeInBits - 1)
428 return m_store.m_file_size;
429 else
430 return (last_block + 1) * m_store.m_buffer_size;
431}
int GetLastDownloadedBlock() const
Get number of the last downloaded block.
long long m_file_size
size of file in bytes
Definition XrdPfcInfo.hh:83

References GetLastDownloadedBlock(), m_bitvecSizeInBits, XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_file_size, and m_store.

+ Here is the call graph for this function:

◆ GetFileSize()

long long XrdPfc::Info::GetFileSize ( ) const
inline

Get file size.

Definition at line 446 of file XrdPfcInfo.hh.

447{
448 return m_store.m_file_size;
449}

References XrdPfc::Info::Store::m_file_size, and m_store.

Referenced by XrdPfc::Cache::ConsiderCached(), XrdPfc::Cache::LocalFilePath(), and XrdPfc::Cache::Stat().

+ Here is the caller graph for this function:

◆ GetLastAccessStats()

const Info::AStat * Info::GetLastAccessStats ( ) const

Get latest access stats.

Definition at line 491 of file XrdPfcInfo.cc.

492{
493 return m_astats.empty() ? 0 : & m_astats.back();
494}

References m_astats.

Referenced by XrdPfc::File::GetLastAccessStats().

+ Here is the caller graph for this function:

◆ GetLastDownloadedBlock()

int XrdPfc::Info::GetLastDownloadedBlock ( ) const
inline

Get number of the last downloaded block.

Definition at line 416 of file XrdPfcInfo.hh.

417{
418 for (int i = m_bitvecSizeInBits - 1; i >= 0; --i)
419 if (TestBitWritten(i)) return i;
420
421 return -1;
422}

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by GetExpectedDataFileSize().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetLatestDetachTime()

bool Info::GetLatestDetachTime ( time_t & t) const

Get latest detach time.

Definition at line 472 of file XrdPfcInfo.cc.

473{
474 if (m_astats.empty())
475 {
477 }
478 else
479 {
480 const AStat& ls = m_astats.back();
481
482 if (ls.DetachTime == 0)
483 t = ls.AttachTime + ls.Duration;
484 else
485 t = ls.DetachTime;
486 }
487
488 return t != 0;
489}

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, m_astats, XrdPfc::Info::Store::m_creationTime, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile().

+ Here is the caller graph for this function:

◆ GetNBlocks()

int XrdPfc::Info::GetNBlocks ( ) const
inline

Get number of blocks represented in download-state bit-vector.

Definition at line 441 of file XrdPfcInfo.hh.

442{
443 return m_bitvecSizeInBits;
444}

References m_bitvecSizeInBits.

Referenced by XrdPfc::File::GetNBlocks().

+ Here is the caller graph for this function:

◆ GetNDownloadedBlocks()

int XrdPfc::Info::GetNDownloadedBlocks ( ) const
inline

Get number of downloaded blocks.

Definition at line 402 of file XrdPfcInfo.hh.

403{
404 int cntd = 0;
405 for (int i = 0; i < m_bitvecSizeInBits; ++i)
406 if (TestBitWritten(i)) cntd++;
407
408 return cntd;
409}

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by XrdPfc::File::GetNDownloadedBlocks(), and GetNDownloadedBytes().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetNDownloadedBytes()

long long XrdPfc::Info::GetNDownloadedBytes ( ) const
inline

Get number of downloaded bytes.

Definition at line 411 of file XrdPfcInfo.hh.

412{
414}
int GetNDownloadedBlocks() const
Get number of downloaded blocks.

References GetNDownloadedBlocks(), XrdPfc::Info::Store::m_buffer_size, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile(), and XrdPfc::Cache::ConsiderCached().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetNoCkSumTime()

time_t XrdPfc::Info::GetNoCkSumTime ( ) const
inline

Definition at line 304 of file XrdPfcInfo.hh.

304{ return m_store.m_noCkSumTime; }
time_t m_noCkSumTime
time when first non-cksummed block was detected
Definition XrdPfcInfo.hh:85

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ GetNoCkSumTimeForUVKeep()

time_t XrdPfc::Info::GetNoCkSumTimeForUVKeep ( ) const
inline

Definition at line 305 of file XrdPfcInfo.hh.

References XrdPfc::Info::Store::m_creationTime, XrdPfc::Info::Store::m_noCkSumTime, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile().

+ Here is the caller graph for this function:

◆ GetTrace()

XrdSysTrace * XrdPfc::Info::GetTrace ( ) const
inline

Definition at line 318 of file XrdPfcInfo.hh.

318{return m_trace; }

References m_trace.

◆ GetVersion()

int XrdPfc::Info::GetVersion ( )
inline

Get version.

Definition at line 270 of file XrdPfcInfo.hh.

270{ return m_version; }

References m_version.

◆ HasNoCkSumTime()

bool XrdPfc::Info::HasNoCkSumTime ( ) const
inline

Definition at line 303 of file XrdPfcInfo.hh.

303{ return m_store.m_noCkSumTime != 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

Referenced by ResetCkSumCache(), and ResetCkSumNet().

+ Here is the caller graph for this function:

◆ IsCkSumAny()

bool XrdPfc::Info::IsCkSumAny ( ) const
inline

Definition at line 295 of file XrdPfcInfo.hh.

295{ return m_store.m_status.f_cksum_check & CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumBoth()

bool XrdPfc::Info::IsCkSumBoth ( ) const
inline

Definition at line 296 of file XrdPfcInfo.hh.

296{ return (m_store.m_status.f_cksum_check & CSChk_Both) == CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumCache()

bool XrdPfc::Info::IsCkSumCache ( ) const
inline

Definition at line 293 of file XrdPfcInfo.hh.

293{ return m_store.m_status.f_cksum_check & CSChk_Cache; }

References XrdPfc::CSChk_Cache, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumCache(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsCkSumNet()

bool XrdPfc::Info::IsCkSumNet ( ) const
inline

Definition at line 294 of file XrdPfcInfo.hh.

294{ return m_store.m_status.f_cksum_check & CSChk_Net; }

References XrdPfc::CSChk_Net, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumNet(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsComplete()

bool XrdPfc::Info::IsComplete ( ) const
inline

Get complete status.

Definition at line 451 of file XrdPfcInfo.hh.

452{
453 return m_complete;
454}

References m_complete.

Referenced by XrdPfc::Cache::ConsiderCached(), XrdPfc::Cache::LocalFilePath(), XrdPfc::File::Read(), XrdPfc::File::ReadV(), XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ Read()

bool Info::Read ( XrdOssDF * fp,
const char * dname,
const char * fname = 0 )

Read content of cinfo file into this object.

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 296 of file XrdPfcInfo.cc.

297{
298 // Does not need lock, called only in File::Open before File::Run() starts.
299 // XXXX Wait, how about Purge, and LocalFilePath, Stat?
300
301 TraceHeader trace_pfx("Read()", dname, fname);
302
303 FpHelper r(fp, 0, m_trace, m_traceID, trace_pfx);
304
305 if (r.Read(m_version)) return false;
306
308 {
309 if (m_version == 2)
310 {
311 return ReadV2(fp, r.f_off, dname, fname);
312 }
313 else if (m_version == 3)
314 {
315 return ReadV3(fp, r.f_off, dname, fname);
316 }
317 else
318 {
319 TRACE(Warning, trace_pfx << "File version " << m_version << " not supported.");
320 return false;
321 }
322 }
323
324 uint32_t cksum;
325
326 if (r.Read(m_store) || r.Read(cksum)) return false;
327
328 if (cksum != CalcCksumStore())
329 {
330 TRACE(Error, trace_pfx << "Checksum Store mismatch.");
331 return false;
332 }
333
334 ResizeBits();
336
337 if (r.ReadRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
338 r.ReadRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
339 r.Read(cksum))
340 {
341 return false;
342 }
343
344 if (cksum != CalcCksumSyncedAndAStats())
345 {
346 TRACE(Error, trace_pfx << "Checksum Synced or AStats mismatch.");
347 return false;
348 }
349
351
353
354 return true;
355}
#define TRACE(act, x)
Definition XrdTrace.hh:63
void UpdateDownloadCompleteStatus()
Update complete status.
uint32_t CalcCksumStore()
Get cksum, MD5 is for backward compatibility with V2 and V3.
void ResizeBits()
Reserve bit vectors for file_size / buffer_size bytes.
static const int s_defaultVersion
uint32_t CalcCksumSyncedAndAStats()
static const char * m_traceID
int m_astatSize
size of AStat vector
Definition XrdPfcInfo.hh:88

References CalcCksumStore(), CalcCksumSyncedAndAStats(), Error, GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_buff_written, m_store, m_trace, m_traceID, m_version, ResizeBits(), s_defaultVersion, TRACE, and UpdateDownloadCompleteStatus().

Referenced by XrdPfc::Cache::ConsiderCached(), XrdPfc::Cache::LocalFilePath(), XrdPfc::Cache::Stat(), and XrdPfc::FPurgeState::TraverseNamespace().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ RefAStats()

const std::vector< AStat > & XrdPfc::Info::RefAStats ( ) const
inline

Definition at line 276 of file XrdPfcInfo.hh.

276{ return m_astats; }

References m_astats.

◆ RefStoredData()

const Store & XrdPfc::Info::RefStoredData ( ) const
inline

Get stored data.

Definition at line 275 of file XrdPfcInfo.hh.

275{ return m_store; }

References m_store.

◆ ResetAllAccessStats()

void Info::ResetAllAccessStats ( )

Reset IO Stats.

Definition at line 361 of file XrdPfcInfo.cc.

362{
365 m_astats.clear();
366}

References XrdPfc::Info::Store::m_accessCnt, m_astats, XrdPfc::Info::Store::m_astatSize, and m_store.

◆ ResetCkSumCache()

void Info::ResetCkSumCache ( )

Definition at line 205 of file XrdPfcInfo.cc.

206{
207 if (IsCkSumCache())
208 {
209 m_store.m_status.f_cksum_check &= ~CSChk_Cache;
210 if ( ! HasNoCkSumTime())
211 m_store.m_noCkSumTime = time(0);
212 }
213}
bool IsCkSumCache() const
bool HasNoCkSumTime() const

References HasNoCkSumTime(), IsCkSumCache(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

+ Here is the call graph for this function:

◆ ResetCkSumNet()

void Info::ResetCkSumNet ( )

Definition at line 215 of file XrdPfcInfo.cc.

216{
217 if (IsCkSumNet())
218 {
219 m_store.m_status.f_cksum_check &= ~CSChk_Net;
220 if ( ! HasNoCkSumTime())
221 m_store.m_noCkSumTime = time(0);
222 }
223}
bool IsCkSumNet() const

References HasNoCkSumTime(), IsCkSumNet(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ ResetNoCkSumTime()

void XrdPfc::Info::ResetNoCkSumTime ( )
inline

Definition at line 306 of file XrdPfcInfo.hh.

306{ m_store.m_noCkSumTime = 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ ResizeBits()

void Info::ResizeBits ( )

Reserve bit vectors for file_size / buffer_size bytes.

Definition at line 174 of file XrdPfcInfo.cc.

175{
176 // drop buffer in case of failed/partial reads
177
178 if (m_buff_synced) free(m_buff_synced);
181
183
184 m_buff_written = (unsigned char*) malloc(GetBitvecSizeInBytes());
185 m_buff_synced = (unsigned char*) malloc(GetBitvecSizeInBytes());
188
190 m_complete = false;
191
193 {
194 m_buff_prefetch = (unsigned char*) malloc(GetBitvecSizeInBytes());
196 }
197 else
198 {
199 m_buff_prefetch = 0;
200 }
201}

References GetBitvecSizeInBytes(), m_bitvecSizeInBits, m_buff_prefetch, m_buff_synced, m_buff_written, XrdPfc::Info::Store::m_buffer_size, m_complete, XrdPfc::Info::Store::m_file_size, m_hasPrefetchBuffer, m_missingBlocks, and m_store.

Referenced by Read(), and SetBufferSizeFileSizeAndCreationTime().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetAllBitsSynced()

void Info::SetAllBitsSynced ( )

Mark all blocks as synced to disk.

Definition at line 146 of file XrdPfcInfo.cc.

147{
148 // The following should be:
149 // memset(m_buff_synced, 255, GetBitvecSizeInBytes());
150 // but GCC produces an overzealous 'possible argument transpose warning' and
151 // xrootd build uses warnings->errors escalation.
152 // This workaround can be removed for gcc >= 5.
153 // See also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61294
154 const int nb = GetBitvecSizeInBytes();
155 for (int i = 0; i < nb; ++i)
156 m_buff_synced[i] = 255;
157
158 m_complete = true;
159}

References GetBitvecSizeInBytes(), m_buff_synced, and m_complete.

Referenced by XrdPfc::Cache::ExecuteCommandUrl().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitPrefetch()

void XrdPfc::Info::SetBitPrefetch ( int i)
inline

Mark block as obtained through prefetch.

Definition at line 369 of file XrdPfcInfo.hh.

370{
371 if (!m_buff_prefetch) return;
372
373 const int cn = i/8;
374 assert(cn < GetBitvecSizeInBytes());
375
376 const int off = i - cn*8;
377 m_buff_prefetch[cn] |= cfiBIT(off);
378}

References GetBitvecSizeInBytes(), and m_buff_prefetch.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitSynced()

void XrdPfc::Info::SetBitSynced ( int i)
inline

Mark block as synced to disk.

Definition at line 391 of file XrdPfcInfo.hh.

392{
393 const int cn = i/8;
394 assert(cn < GetBitvecSizeInBytes());
395
396 const int off = i - cn*8;
397 m_buff_synced[cn] |= cfiBIT(off);
398}

References GetBitvecSizeInBytes(), and m_buff_synced.

Referenced by XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitWritten()

void XrdPfc::Info::SetBitWritten ( int i)
inline

Mark block as written to disk.

Definition at line 356 of file XrdPfcInfo.hh.

357{
358 const int cn = i/8;
359 assert(cn < GetBitvecSizeInBytes());
360
361 const int off = i - cn*8;
362
363 m_buff_written[cn] |= cfiBIT(off);
364
365 if (--m_missingBlocks == 0)
366 m_complete = true;
367}

References GetBitvecSizeInBytes(), m_buff_written, m_complete, and m_missingBlocks.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBufferSizeFileSizeAndCreationTime()

void Info::SetBufferSizeFileSizeAndCreationTime ( long long bs,
long long fs )

Definition at line 163 of file XrdPfcInfo.cc.

164{
165 // Needed only when Info object is created for the first time in File::Open()
167 m_store.m_file_size = fs;
168 ResizeBits();
169 m_store.m_creationTime = time(0);
170}

References XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_creationTime, XrdPfc::Info::Store::m_file_size, m_store, and ResizeBits().

Referenced by XrdPfc::Cache::ExecuteCommandUrl().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetCkSumState()

void XrdPfc::Info::SetCkSumState ( CkSumCheck_e css)
inline

Definition at line 298 of file XrdPfcInfo.hh.

298{ m_store.m_status.f_cksum_check = css; }

References XrdPfc::Info::Store::m_status, and m_store.

◆ TestBitPrefetch()

bool XrdPfc::Info::TestBitPrefetch ( int i) const
inline

Test if block at the given index has been prefetched.

Definition at line 380 of file XrdPfcInfo.hh.

381{
382 if (!m_buff_prefetch) return false;
383
384 const int cn = i/8;
385 assert(cn < GetBitvecSizeInBytes());
386
387 const int off = i - cn*8;
388 return (m_buff_prefetch[cn] & cfiBIT(off)) != 0;
389}

References GetBitvecSizeInBytes(), and m_buff_prefetch.

+ Here is the call graph for this function:

◆ TestBitWritten()

bool XrdPfc::Info::TestBitWritten ( int i) const
inline

Test if block at the given index is written to disk.

Definition at line 347 of file XrdPfcInfo.hh.

348{
349 const int cn = i/8;
350 assert(cn < GetBitvecSizeInBytes());
351
352 const int off = i - cn*8;
353 return (m_buff_written[cn] & cfiBIT(off)) != 0;
354}

References GetBitvecSizeInBytes(), and m_buff_written.

Referenced by CountBlocksNotWrittenInRng(), GetLastDownloadedBlock(), GetNDownloadedBlocks(), and XrdPfc::File::Prefetch().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ UpdateDownloadCompleteStatus()

void XrdPfc::Info::UpdateDownloadCompleteStatus ( )
inline

Update complete status.

Definition at line 467 of file XrdPfcInfo.hh.

468{
471}
int CountBlocksNotWrittenInRng(int firstIdx, int lastIdx) const
Check download status in given block range.

References CountBlocksNotWrittenInRng(), m_bitvecSizeInBits, m_complete, and m_missingBlocks.

Referenced by Read().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ Write()

bool Info::Write ( XrdOssDF * fp,
const char * dname,
const char * fname = 0 )

Write number of blocks and read buffer size

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 268 of file XrdPfcInfo.cc.

269{
270 TraceHeader trace_pfx("Write()", dname, fname);
271
273 m_store.m_astatSize = (int32_t) m_astats.size();
274
275 FpHelper w(fp, 0, m_trace, m_traceID, trace_pfx);
276
277 if (w.Write(s_defaultVersion) ||
278 w.Write(m_store) ||
279 w.Write(CalcCksumStore()) ||
280 w.WriteRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
281 w.WriteRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
282 w.Write(CalcCksumSyncedAndAStats()))
283 {
284 return false;
285 }
286
287 return true;
288}
void CompactifyAccessRecords()
Compactify access records to the configured maximum.

References CalcCksumStore(), CalcCksumSyncedAndAStats(), CompactifyAccessRecords(), GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_store, m_trace, m_traceID, s_defaultVersion, and s_maxNumAccess.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), XrdPfc::Cache::LocalFilePath(), and XrdPfc::File::Sync().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStat()

void Info::WriteIOStat ( Stats & s)

Write bytes missed, hits, and disk.

Definition at line 431 of file XrdPfcInfo.cc.

432{
433 m_astats.back().NumIos = s.m_NumIos;
434 m_astats.back().Duration = s.m_Duration;
435 m_astats.back().BytesHit = s.m_BytesHit;
436 m_astats.back().BytesMissed = s.m_BytesMissed;
437 m_astats.back().BytesBypassed = s.m_BytesBypassed;
438}
long long m_BytesMissed
number of bytes served from remote and cached
long long m_BytesBypassed
number of bytes served directly through XrdCl
int m_Duration
total duration of all IOs attached
int m_NumIos
number of IO objects attached during this access
long long m_BytesHit
number of bytes served from disk

References m_astats, XrdPfc::Stats::m_BytesBypassed, XrdPfc::Stats::m_BytesHit, XrdPfc::Stats::m_BytesMissed, XrdPfc::Stats::m_Duration, and XrdPfc::Stats::m_NumIos.

Referenced by XrdPfc::File::Sync(), and WriteIOStatDetach().

+ Here is the caller graph for this function:

◆ WriteIOStatAttach()

void Info::WriteIOStatAttach ( )

Write open time in the last entry of access statistics.

Definition at line 422 of file XrdPfcInfo.cc.

423{
425
426 AStat as;
427 as.AttachTime = time(0);
428 m_astats.push_back(as);
429}

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, and m_store.

◆ WriteIOStatDetach()

void Info::WriteIOStatDetach ( Stats & s)

Write close time together with bytes missed, hits, and disk.

Definition at line 440 of file XrdPfcInfo.cc.

441{
442 m_astats.back().DetachTime = time(0);
443 WriteIOStat(s);
444}
void WriteIOStat(Stats &s)
Write bytes missed, hits, and disk.

References m_astats, and WriteIOStat().

Referenced by XrdPfc::File::FinalizeSyncBeforeExit().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [1/2]

void Info::WriteIOStatSingle ( long long bytes_disk)

Write single open/close time for given bytes read from disk.

Definition at line 446 of file XrdPfcInfo.cc.

447{
449
450 AStat as;
451 as.AttachTime = as.DetachTime = time(0);
452 as.NumIos = 1;
453 as.BytesHit = bytes_disk;
454 m_astats.push_back(as);
455}

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), and XrdPfc::Cache::LocalFilePath().

+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [2/2]

void Info::WriteIOStatSingle ( long long bytes_disk,
time_t att,
time_t dtc )

Write open/close with given time and bytes read from disk.

Definition at line 457 of file XrdPfcInfo.cc.

458{
460
461 AStat as;
462 as.AttachTime = att;
463 as.DetachTime = dtc;
464 as.NumIos = 1;
465 as.Duration = dtc - att;
466 as.BytesHit = bytes_disk;
467 m_astats.push_back(as);
468}

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Member Data Documentation

◆ m_astats

◆ m_bitvecSizeInBits

int XrdPfc::Info::m_bitvecSizeInBits
protected

◆ m_buff_prefetch

unsigned char* XrdPfc::Info::m_buff_prefetch
protected

prefetch statistics

Definition at line 326 of file XrdPfcInfo.hh.

Referenced by ~Info(), ResizeBits(), SetBitPrefetch(), and TestBitPrefetch().

◆ m_buff_synced

unsigned char* XrdPfc::Info::m_buff_synced
protected

disk written state vector

Definition at line 324 of file XrdPfcInfo.hh.

Referenced by ~Info(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitSynced(), and Write().

◆ m_buff_written

unsigned char* XrdPfc::Info::m_buff_written
protected

download state vector

Definition at line 325 of file XrdPfcInfo.hh.

Referenced by ~Info(), Read(), ResizeBits(), SetBitWritten(), and TestBitWritten().

◆ m_complete

bool XrdPfc::Info::m_complete
protected

cached; if false, set to true when missingBlocks hit zero

Definition at line 332 of file XrdPfcInfo.hh.

Referenced by IsComplete(), ResizeBits(), SetAllBitsSynced(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_hasPrefetchBuffer

bool XrdPfc::Info::m_hasPrefetchBuffer
protected

constains current prefetch score

Definition at line 333 of file XrdPfcInfo.hh.

Referenced by ResizeBits().

◆ m_missingBlocks

int XrdPfc::Info::m_missingBlocks
protected

cached, updated in SetBitWritten()

Definition at line 331 of file XrdPfcInfo.hh.

Referenced by ResizeBits(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_store

◆ m_trace

XrdSysTrace* XrdPfc::Info::m_trace
protected

Definition at line 321 of file XrdPfcInfo.hh.

Referenced by GetTrace(), Read(), and Write().

◆ m_traceID

const char * Info::m_traceID = "CInfo"
static

Definition at line 312 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ m_version

int XrdPfc::Info::m_version
protected

Definition at line 329 of file XrdPfcInfo.hh.

Referenced by GetVersion(), and Read().

◆ s_defaultVersion

const int Info::s_defaultVersion = 4
static

Definition at line 316 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ s_infoExtension

◆ s_infoExtensionLen

const size_t Info::s_infoExtensionLen = strlen(Info::s_infoExtension)
static

Definition at line 314 of file XrdPfcInfo.hh.

◆ s_maxNumAccess

size_t Info::s_maxNumAccess = 20
static

Definition at line 315 of file XrdPfcInfo.hh.

Referenced by CompactifyAccessRecords(), XrdPfc::Cache::Config(), and Write().


The documentation for this class was generated from the following files: