Roc Toolkit internal modules
Roc Toolkit: real-time audio streaming
latency_monitor.h
Go to the documentation of this file.
1/*
2 * Copyright (c) 2015 Roc Streaming authors
3 *
4 * This Source Code Form is subject to the terms of the Mozilla Public
5 * License, v. 2.0. If a copy of the MPL was not distributed with this
6 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
7 */
8
9//! @file roc_audio/latency_monitor.h
10//! @brief Latency monitor.
11
12#ifndef ROC_AUDIO_LATENCY_MONITOR_H_
13#define ROC_AUDIO_LATENCY_MONITOR_H_
14
20#include "roc_core/attributes.h"
22#include "roc_core/optional.h"
23#include "roc_core/time.h"
25#include "roc_packet/units.h"
26
27namespace roc {
28namespace audio {
29
30//! Parameters for latency monitor.
32 //! Enable FreqEstimator.
34
35 //! FreqEstimator profile.
37
38 //! FreqEstimator update interval, nanoseconds.
39 //! How often to run FreqEstimator and update Resampler scaling.
41
42 //! Maximum allowed deviation from target latency, nanoseconds.
43 //! If the latency goes out of bounds, the session is terminated.
45
46 //! Maximum allowed deviation of freq_coeff from 1.0.
47 //! If the scaling goes out of bounds, it is trimmed.
48 //! For example, 0.01 allows freq_coeff values in range [0.99; 1.01].
50
52 : fe_enable(true)
56 , scaling_tolerance(0.005f) {
57 }
58
59 //! Automatically deduce FreqEstimator profile from target latency.
60 void deduce_fe_profile(const core::nanoseconds_t target_latency) {
61 fe_profile = target_latency < 30 * core::Millisecond
62 // prefer responsive profile on low latencies, because gradual profile
63 // won't do it at all
65 // prefer gradual profile for higher latencies, because it can handle
66 // higher network jitter
68 }
69
70 //! Automatically deduce latency_tolerance from target_latency.
72 // this formula returns target_latency * N, where N starts with larger
73 // number and approaches 0.5 as target_latency grows
74 // examples:
75 // target=1ms -> tolerance=8ms (x8)
76 // target=10ms -> tolerance=20ms (x2)
77 // target=200ms -> tolerance=200ms (x1)
78 // target=2000ms -> tolerance=1444ms (x0.722)
79 if (target_latency < core::Millisecond) {
80 target_latency = core::Millisecond;
81 }
83 target_latency
84 * (std::log((200 * core::Millisecond) * 2) / std::log(target_latency * 2)));
85 }
86};
87
88//! Metrics of latency monitor.
90 //! Estimated NIQ latency.
91 //! NIQ = network incoming queue.
92 //! Defines how many samples are buffered in receiver packet queue and
93 //! receiver pipeline before depacketizer (packet part of pipeline).
95
96 //! Estimated E2E latency.
97 //! E2E = end-to-end.
98 //! Defines how much time passed between frame entered sender pipeline
99 //! (when it is captured) and leaved received pipeline (when it is played).
101
103 : niq_latency(0)
104 , e2e_latency(0) {
105 }
106};
107
108//! Latency monitor.
109//!
110//! @b Features
111//!
112//! - calculates NIQ latency (network incoming queue) - how many samples are
113//! buffered in receiver packet queue and receiver pipeline before depacketizer
114//! - calculates E2E latency (end-to-end) - how much time passed between frame
115//! was captured on sender and played on receiver (this is based on capture
116//! timestamps, which are populated with the help of RTCP)
117//! - monitors how close actual latency and target latency are (which one to
118//! check, NIQ or E2E, depends on config)
119//! - assuming that the difference between actual latency and target latency is
120//! caused by the clock drift between sender and receiver, calculates scaling
121//! factor for resampler to compensate that clock drift
122//! - passes calculated scaling factor to resampler
123//! - shuts down session if the actual latency goes out of bounds
124//!
125//! @b Flow
126//!
127//! - pipeline periodically calls read() method; it uses references to incoming
128//! packet queue (start of the pipeline) and depacketizer (last pipeline element
129//! that works with packets), asks them about current packet / frame, and calculates
130//! distance between them, which is NIQ latency
131//! - after adding frame to playback buffer, pipeline invokes reclock() method;
132//! it calculates difference between capture and playback time of the frame,
133//! which is E2E latency
134//! - latency monitor has an instance of FreqEstimator (FE); it continously passes
135//! calculated latency to FE, and FE calculates scaling factor for resampler
136//! - latency monitor has a reference to resampler, and periodically passes
137//! updated scaling factor to it;
138//! - pipeline also can query latency monitor for latency statistics on behalf of
139//! request from user
141public:
142 //! Constructor.
143 //!
144 //! @b Parameters
145 //! - @p frame_reader is inner frame reader for E2E latency calculation
146 //! - @p incoming_queue and @p depacketizer are used to NIQ latency calculation
147 //! - @p resampler is used to set the scaling factor to compensate clock
148 //! drift according to calculated latency
149 //! - @p config defines calculation parameters
150 //! - @p target_latency defines target latency that we should maintain
151 //! - @p input_sample_spec is the sample spec of the input packets
152 //! - @p output_sample_spec is the sample spec of the output frames (after
153 //! resampling)
155 const packet::SortedQueue& incoming_queue,
156 const Depacketizer& depacketizer,
157 ResamplerReader* resampler,
158 const LatencyMonitorConfig& config,
159 core::nanoseconds_t target_latency,
160 const SampleSpec& input_sample_spec,
161 const SampleSpec& output_sample_spec);
162
163 //! Check if the object was initialized successfully.
164 bool is_valid() const;
165
166 //! Check if the stream is still alive.
167 bool is_alive() const;
168
169 //! Get metrics.
171
172 //! Read audio frame from a pipeline.
173 //! @remarks
174 //! Forwards frame from underlying reader as-is.
175 virtual bool read(Frame& frame);
176
177 //! Report playback timestamp of last frame returned by read.
178 //! @remarks
179 //! Pipeline invokes this method after adding last frame to
180 //! playback buffer and knowing its playback time.
181 //! @returns
182 //! false if the session is ended
183 bool reclock(core::nanoseconds_t playback_timestamp);
184
185private:
186 void compute_niq_latency_();
187 void compute_e2e_latency_(core::nanoseconds_t playback_timestamp);
188
189 bool update_();
190
191 bool check_bounds_(packet::stream_timestamp_diff_t latency) const;
192
193 bool init_scaling_(size_t input_sample_rate, size_t output_sample_rate);
194 bool update_scaling_(packet::stream_timestamp_diff_t latency);
195
196 void report_();
197
198 IFrameReader& frame_reader_;
199
200 const packet::SortedQueue& incoming_queue_;
201 const Depacketizer& depacketizer_;
202
203 ResamplerReader* resampler_;
205
206 packet::stream_timestamp_t stream_pos_;
207 core::nanoseconds_t stream_cts_;
208
209 const packet::stream_timestamp_t update_interval_;
210 packet::stream_timestamp_t update_pos_;
211
212 const packet::stream_timestamp_t report_interval_;
213 packet::stream_timestamp_t report_pos_;
214
215 float freq_coeff_;
216
219 bool has_niq_latency_;
220 bool has_e2e_latency_;
221
222 const packet::stream_timestamp_diff_t target_latency_;
223 const packet::stream_timestamp_diff_t min_latency_;
224 const packet::stream_timestamp_diff_t max_latency_;
225
226 const float max_scaling_delta_;
227
228 const SampleSpec input_sample_spec_;
229 const SampleSpec output_sample_spec_;
230
231 bool alive_;
232 bool valid_;
233};
234
235} // namespace audio
236} // namespace roc
237
238#endif // ROC_AUDIO_LATENCY_MONITOR_H_
Compiler attributes.
Audio frame.
Definition: frame.h:25
Frame reader interface.
Definition: iframe_reader.h:22
bool reclock(core::nanoseconds_t playback_timestamp)
Report playback timestamp of last frame returned by read.
LatencyMonitorMetrics metrics() const
Get metrics.
virtual bool read(Frame &frame)
Read audio frame from a pipeline.
bool is_valid() const
Check if the object was initialized successfully.
LatencyMonitor(IFrameReader &frame_reader, const packet::SortedQueue &incoming_queue, const Depacketizer &depacketizer, ResamplerReader *resampler, const LatencyMonitorConfig &config, core::nanoseconds_t target_latency, const SampleSpec &input_sample_spec, const SampleSpec &output_sample_spec)
Constructor.
bool is_alive() const
Check if the stream is still alive.
Resampler element for reading pipeline.
Sample specification. Describes sample rate and channels.
Definition: sample_spec.h:26
Base class for non-copyable objects.
Definition: noncopyable.h:23
Optionally constructed object.
Definition: optional.h:25
Sorted packet queue.
Definition: sorted_queue.h:27
Depacketizer.
Frequency estimator.
Frame reader interface.
FreqEstimatorProfile
FreqEstimator paremeter preset.
@ FreqEstimatorProfile_Gradual
Slow and smooth tuning. Good for higher network latency and jitter.
@ FreqEstimatorProfile_Responsive
Fast and responsive tuning. Good for lower network latency and jitter.
const nanoseconds_t Millisecond
One millisecond represented in nanoseconds.
Definition: time.h:67
int64_t nanoseconds_t
Nanoseconds.
Definition: time.h:58
uint32_t stream_timestamp_t
Packet stream timestamp.
Definition: units.h:36
int32_t stream_timestamp_diff_t
Packet stream timestamp delta.
Definition: units.h:41
Root namespace.
Non-copyable object.
Optionally constructed object.
Sample specifications.
Sorted packet queue.
Parameters for latency monitor.
void deduce_latency_tolerance(core::nanoseconds_t target_latency)
Automatically deduce latency_tolerance from target_latency.
FreqEstimatorProfile fe_profile
FreqEstimator profile.
bool fe_enable
Enable FreqEstimator.
void deduce_fe_profile(const core::nanoseconds_t target_latency)
Automatically deduce FreqEstimator profile from target latency.
core::nanoseconds_t fe_update_interval
FreqEstimator update interval, nanoseconds. How often to run FreqEstimator and update Resampler scali...
core::nanoseconds_t latency_tolerance
Maximum allowed deviation from target latency, nanoseconds. If the latency goes out of bounds,...
float scaling_tolerance
Maximum allowed deviation of freq_coeff from 1.0. If the scaling goes out of bounds,...
Metrics of latency monitor.
core::nanoseconds_t e2e_latency
Estimated E2E latency. E2E = end-to-end. Defines how much time passed between frame entered sender pi...
core::nanoseconds_t niq_latency
Estimated NIQ latency. NIQ = network incoming queue. Defines how many samples are buffered in receive...
Time definitions.
Various units used in packets.