Intel(R) Threading Building Blocks Doxygen Documentation  version 4.2.3
tbb::internal::governor Class Reference

The class handles access to the single instance of market, and to TLS to keep scheduler instances. More...

#include <governor.h>

Collaboration diagram for tbb::internal::governor:

Static Public Member Functions

static unsigned default_num_threads ()
 
static size_t default_page_size ()
 
static void one_time_init ()
 
static generic_schedulerinit_scheduler (int num_threads, stack_size_type stack_size, bool auto_init)
 Processes scheduler initialization request (possibly nested) in a master thread. More...
 
static generic_schedulerinit_scheduler_weak ()
 Automatic initialization of scheduler in a master thread with default settings without arena. More...
 
static bool terminate_scheduler (generic_scheduler *s, bool blocking)
 Processes scheduler termination request (possibly nested) in a master thread. More...
 
static void sign_on (generic_scheduler *s)
 Register TBB scheduler instance in thread-local storage. More...
 
static void sign_off (generic_scheduler *s)
 Unregister TBB scheduler instance from thread-local storage. More...
 
static bool is_set (generic_scheduler *s)
 Used to check validity of the local scheduler TLS contents. More...
 
static void assume_scheduler (generic_scheduler *s)
 Temporarily set TLS slot to the given scheduler. More...
 
static uintptr_t tls_value_of (generic_scheduler *s)
 Computes the value of the TLS. More...
 
static generic_schedulertls_scheduler_of (uintptr_t v)
 Converts TLS value to the scheduler pointer. More...
 
static generic_schedulerlocal_scheduler ()
 Obtain the thread-local instance of the TBB scheduler. More...
 
static generic_schedulerlocal_scheduler_weak ()
 
static generic_schedulerlocal_scheduler_if_initialized ()
 
static void terminate_auto_initialized_scheduler ()
 Undo automatic initialization if necessary; call when a thread exits. More...
 
static void print_version_info ()
 
static void initialize_rml_factory ()
 
static bool does_client_join_workers (const tbb::internal::rml::tbb_client &client)
 
static bool speculation_enabled ()
 
static bool rethrow_exception_broken ()
 

Static Private Member Functions

static void acquire_resources ()
 Create key for thread-local storage and initialize RML. More...
 
static void release_resources ()
 Destroy the thread-local storage key and deinitialize RML. More...
 
static rml::tbb_server * create_rml_server (rml::tbb_client &)
 
static void auto_terminate (void *scheduler)
 The internal routine to undo automatic initialization. More...
 

Static Private Attributes

static basic_tls< uintptr_t > theTLS
 TLS for scheduler instances associated with individual threads. More...
 
static unsigned DefaultNumberOfThreads
 Caches the maximal level of parallelism supported by the hardware. More...
 
static size_t DefaultPageSize
 Caches the size of OS regular memory page. More...
 
static rml::tbb_factory theRMLServerFactory
 
static bool UsePrivateRML
 
static bool is_speculation_enabled
 
static bool is_rethrow_broken
 

Friends

class __TBB_InitOnce
 
class market
 

Detailed Description

The class handles access to the single instance of market, and to TLS to keep scheduler instances.

It also supports automatic on-demand initialization of the TBB scheduler. The class contains only static data members and methods.

Definition at line 48 of file governor.h.

Member Function Documentation

◆ acquire_resources()

void tbb::internal::governor::acquire_resources ( )
staticprivate

Create key for thread-local storage and initialize RML.

Definition at line 67 of file governor.cpp.

References auto_terminate(), tbb::internal::cpu_has_speculation(), tbb::internal::basic_tls< T >::create(), tbb::internal::gcc_rethrow_exception_broken(), tbb::internal::handle_perror(), is_rethrow_broken, is_speculation_enabled, and theTLS.

Referenced by tbb::internal::__TBB_InitOnce::add_ref().

67  {
68 #if USE_PTHREAD
69  int status = theTLS.create(auto_terminate);
70 #else
71  int status = theTLS.create();
72 #endif
73  if( status )
74  handle_perror(status, "TBB failed to initialize task scheduler TLS\n");
77 }
bool gcc_rethrow_exception_broken()
Definition: tbb_misc.cpp:198
bool cpu_has_speculation()
check for transaction support.
Definition: tbb_misc.cpp:230
void __TBB_EXPORTED_FUNC handle_perror(int error_code, const char *aux_info)
Throws std::runtime_error with what() returning error_code description prefixed with aux_info...
Definition: tbb_misc.cpp:87
static bool is_rethrow_broken
Definition: governor.h:68
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:54
static void auto_terminate(void *scheduler)
The internal routine to undo automatic initialization.
Definition: governor.cpp:215
static bool is_speculation_enabled
Definition: governor.h:67
Here is the call graph for this function:
Here is the caller graph for this function:

◆ assume_scheduler()

void tbb::internal::governor::assume_scheduler ( generic_scheduler s)
static

Temporarily set TLS slot to the given scheduler.

Definition at line 116 of file governor.cpp.

References s, tbb::internal::basic_tls< T >::set(), theTLS, and tls_value_of().

Referenced by auto_terminate(), tbb::internal::market::cleanup(), init_scheduler(), tbb::internal::generic_scheduler::nested_arena_entry(), sign_off(), sign_on(), and tbb::internal::nested_arena_context::~nested_arena_context().

116  {
117  theTLS.set( tls_value_of(s) );
118 }
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:54
static uintptr_t tls_value_of(generic_scheduler *s)
Computes the value of the TLS.
Definition: governor.cpp:110
void set(T value)
Definition: tls.h:56
void const char const char int ITT_FORMAT __itt_group_sync s
Here is the call graph for this function:
Here is the caller graph for this function:

◆ auto_terminate()

void tbb::internal::governor::auto_terminate ( void scheduler)
staticprivate

The internal routine to undo automatic initialization.

The signature is written with void* so that the routine can be the destructor argument to pthread_key_create.

Definition at line 215 of file governor.cpp.

References __TBB_ASSERT, assume_scheduler(), is_set(), s, and tls_scheduler_of().

Referenced by acquire_resources(), and terminate_auto_initialized_scheduler().

215  {
216  generic_scheduler* s = tls_scheduler_of( uintptr_t(arg) ); // arg is equivalent to theTLS.get()
217  if( s && s->my_auto_initialized ) {
218  if( !--(s->my_ref_count) ) {
219  // If the TLS slot is already cleared by OS or underlying concurrency
220  // runtime, restore its value.
221  if( !is_set(s) )
223  s->cleanup_master( /*blocking_terminate=*/false );
224  __TBB_ASSERT( is_set(NULL), "cleanup_master has not cleared its TLS slot" );
225  }
226  }
227 }
static generic_scheduler * tls_scheduler_of(uintptr_t v)
Converts TLS value to the scheduler pointer.
Definition: governor.h:122
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:120
void const char const char int ITT_FORMAT __itt_group_sync s
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:165
static void assume_scheduler(generic_scheduler *s)
Temporarily set TLS slot to the given scheduler.
Definition: governor.cpp:116
Here is the call graph for this function:
Here is the caller graph for this function:

◆ create_rml_server()

rml::tbb_server * tbb::internal::governor::create_rml_server ( rml::tbb_client &  client)
staticprivate

Definition at line 92 of file governor.cpp.

References __TBB_ASSERT, tbb::internal::rml::make_private_server(), tbb::internal::runtime_warning(), theRMLServerFactory, and UsePrivateRML.

Referenced by tbb::internal::market::market().

92  {
93  rml::tbb_server* server = NULL;
94  if( !UsePrivateRML ) {
95  ::rml::factory::status_type status = theRMLServerFactory.make_server( server, client );
96  if( status != ::rml::factory::st_success ) {
97  UsePrivateRML = true;
98  runtime_warning( "rml::tbb_factory::make_server failed with status %x, falling back on private rml", status );
99  }
100  }
101  if ( !server ) {
102  __TBB_ASSERT( UsePrivateRML, NULL );
103  server = rml::make_private_server( client );
104  }
105  __TBB_ASSERT( server, "Failed to create RML server" );
106  return server;
107 }
tbb_server * make_private_server(tbb_client &client)
Factory method called from task.cpp to create a private_server.
static rml::tbb_factory theRMLServerFactory
Definition: governor.h:62
static bool UsePrivateRML
Definition: governor.h:64
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:165
void __TBB_EXPORTED_FUNC runtime_warning(const char *format,...)
Report a runtime warning.
Here is the call graph for this function:
Here is the caller graph for this function:

◆ default_num_threads()

static unsigned tbb::internal::governor::default_num_threads ( )
inlinestatic

Definition at line 84 of file governor.h.

References tbb::internal::AvailableHwConcurrency(), and DefaultNumberOfThreads.

Referenced by tbb::internal::calc_workers_soft_limit(), tbb::internal::rml::private_server::default_concurrency(), tbb::task_scheduler_init::default_num_threads(), tbb::internal::allowed_parallelism_control::default_value(), tbb::internal::DoOneTimeInitializations(), tbb::internal::market::global_market(), tbb::internal::tbb_thread_v3::hardware_concurrency(), and init_scheduler().

84  {
85  // No memory fence required, because at worst each invoking thread calls AvailableHwConcurrency once.
88  }
int AvailableHwConcurrency()
Returns maximal parallelism level supported by the current OS configuration.
static unsigned DefaultNumberOfThreads
Caches the maximal level of parallelism supported by the hardware.
Definition: governor.h:57
Here is the call graph for this function:
Here is the caller graph for this function:

◆ default_page_size()

static size_t tbb::internal::governor::default_page_size ( )
inlinestatic

Definition at line 89 of file governor.h.

References DefaultPageSize, and tbb::internal::DefaultSystemPageSize().

Referenced by tbb::internal::create_coroutine(), tbb::internal::destroy_coroutine(), and tbb::internal::DoOneTimeInitializations().

89  {
92  }
static size_t DefaultPageSize
Caches the size of OS regular memory page.
Definition: governor.h:60
size_t DefaultSystemPageSize()
Returns OS regular memory page size.
Definition: tbb_misc.cpp:70
Here is the call graph for this function:
Here is the caller graph for this function:

◆ does_client_join_workers()

bool tbb::internal::governor::does_client_join_workers ( const tbb::internal::rml::tbb_client &  client)
static

Definition at line 304 of file market.cpp.

Referenced by tbb::internal::rml::private_worker::start_shutdown(), and tbb::internal::rml::private_worker::wake_or_launch().

304  {
305  return ((const market&)client).must_join_workers();
306 }
friend class market
Definition: governor.h:51
Here is the caller graph for this function:

◆ init_scheduler()

generic_scheduler * tbb::internal::governor::init_scheduler ( int  num_threads,
stack_size_type  stack_size,
bool  auto_init 
)
static

Processes scheduler initialization request (possibly nested) in a master thread.

If necessary creates new instance of arena and/or local scheduler. The auto_init argument specifies if the call is due to automatic initialization.

Definition at line 172 of file governor.cpp.

References __TBB_ASSERT, assume_scheduler(), tbb::task_scheduler_init::automatic, tbb::internal::market::create_arena(), tbb::internal::generic_scheduler::create_master(), default_num_threads(), tbb::internal::basic_tls< T >::get(), is_set(), one_time_init(), s, theTLS, and tls_scheduler_of().

Referenced by tbb::task_scheduler_init::initialize(), and local_scheduler().

172  {
173  one_time_init();
174  if ( uintptr_t v = theTLS.get() ) {
175  generic_scheduler* s = tls_scheduler_of( v );
176  if ( (v&1) == 0 ) { // TLS holds scheduler instance without arena
177  __TBB_ASSERT( s->my_ref_count == 1, "weakly initialized scheduler must have refcount equal to 1" );
178  __TBB_ASSERT( !s->my_arena, "weakly initialized scheduler must have no arena" );
179  __TBB_ASSERT( s->my_auto_initialized, "weakly initialized scheduler is supposed to be auto-initialized" );
180  s->attach_arena( market::create_arena( default_num_threads(), 1, 0 ), 0, /*is_master*/true );
181  __TBB_ASSERT( s->my_arena_index == 0, "Master thread must occupy the first slot in its arena" );
182  s->my_arena_slot->my_scheduler = s;
183 #if __TBB_TASK_GROUP_CONTEXT
184  s->my_arena->my_default_ctx = s->default_context(); // it also transfers implied ownership
185 #endif
186  // Mark the scheduler as fully initialized
187  assume_scheduler( s );
188  }
189  // Increment refcount only for explicit instances of task_scheduler_init.
190  if ( !auto_init ) s->my_ref_count += 1;
191  __TBB_ASSERT( s->my_arena, "scheduler is not initialized fully" );
192  return s;
193  }
194  // Create new scheduler instance with arena
195  if( num_threads == task_scheduler_init::automatic )
196  num_threads = default_num_threads();
197  arena *a = market::create_arena( num_threads, 1, stack_size );
198  generic_scheduler* s = generic_scheduler::create_master( a );
199  __TBB_ASSERT(s, "Somehow a local scheduler creation for a master thread failed");
200  __TBB_ASSERT( is_set(s), NULL );
201  s->my_auto_initialized = auto_init;
202  return s;
203 }
static generic_scheduler * tls_scheduler_of(uintptr_t v)
Converts TLS value to the scheduler pointer.
Definition: governor.h:122
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:120
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:54
static unsigned default_num_threads()
Definition: governor.h:84
void const char const char int ITT_FORMAT __itt_group_sync s
static arena * create_arena(int num_slots, int num_reserved_slots, size_t stack_size)
Creates an arena object.
Definition: market.cpp:308
static const int automatic
Typedef for number of threads that is automatic.
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:165
static void assume_scheduler(generic_scheduler *s)
Temporarily set TLS slot to the given scheduler.
Definition: governor.cpp:116
static void one_time_init()
Definition: governor.cpp:156
static generic_scheduler * create_master(arena *a)
Initialize a scheduler for a master thread.
Definition: scheduler.cpp:1287
Here is the call graph for this function:
Here is the caller graph for this function:

◆ init_scheduler_weak()

generic_scheduler * tbb::internal::governor::init_scheduler_weak ( )
static

Automatic initialization of scheduler in a master thread with default settings without arena.

Definition at line 164 of file governor.cpp.

References __TBB_ASSERT, tbb::internal::generic_scheduler::create_master(), is_set(), one_time_init(), and s.

Referenced by local_scheduler_weak().

164  {
165  one_time_init();
166  __TBB_ASSERT( is_set(NULL), "TLS contains a scheduler?" );
167  generic_scheduler* s = generic_scheduler::create_master( NULL ); // without arena
168  s->my_auto_initialized = true;
169  return s;
170 }
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:120
void const char const char int ITT_FORMAT __itt_group_sync s
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:165
static void one_time_init()
Definition: governor.cpp:156
static generic_scheduler * create_master(arena *a)
Initialize a scheduler for a master thread.
Definition: scheduler.cpp:1287
Here is the call graph for this function:
Here is the caller graph for this function:

◆ initialize_rml_factory()

void tbb::internal::governor::initialize_rml_factory ( )
static

Definition at line 242 of file governor.cpp.

References theRMLServerFactory, and UsePrivateRML.

Referenced by tbb::internal::DoOneTimeInitializations().

242  {
243  ::rml::factory::status_type res = theRMLServerFactory.open();
244  UsePrivateRML = res != ::rml::factory::st_success;
245 }
static rml::tbb_factory theRMLServerFactory
Definition: governor.h:62
static bool UsePrivateRML
Definition: governor.h:64
Here is the caller graph for this function:

◆ is_set()

bool tbb::internal::governor::is_set ( generic_scheduler s)
static

Used to check validity of the local scheduler TLS contents.

Definition at line 120 of file governor.cpp.

References tbb::internal::basic_tls< T >::get(), s, theTLS, and tls_value_of().

Referenced by auto_terminate(), tbb::internal::generic_scheduler::cleanup_master(), tbb::internal::generic_scheduler::generic_scheduler(), init_scheduler(), init_scheduler_weak(), tbb::internal::generic_scheduler::local_spawn(), tbb::internal::generic_scheduler::local_spawn_root_and_wait(), tbb::internal::custom_scheduler< SchedulerTraits >::local_wait_for_all(), tbb::internal::market::process(), tbb::internal::arena::process(), sign_off(), sign_on(), and terminate_scheduler().

120  {
121  return theTLS.get() == tls_value_of(s);
122 }
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:54
static uintptr_t tls_value_of(generic_scheduler *s)
Computes the value of the TLS.
Definition: governor.cpp:110
void const char const char int ITT_FORMAT __itt_group_sync s
Here is the call graph for this function:
Here is the caller graph for this function:

◆ local_scheduler()

static generic_scheduler* tbb::internal::governor::local_scheduler ( )
inlinestatic

Obtain the thread-local instance of the TBB scheduler.

If the scheduler has not been initialized yet, initialization is done automatically. Note that auto-initialized scheduler instance is destroyed only when its thread terminates.

Definition at line 129 of file governor.h.

References tbb::task_scheduler_init::automatic, tbb::internal::basic_tls< T >::get(), init_scheduler(), theTLS, and tls_scheduler_of().

Referenced by tbb::internal::arena::advertise_new_work(), tbb::internal::generic_scheduler::enqueue(), tbb::internal::get_initial_auto_partitioner_divisor(), tbb::internal::affinity_partitioner_base_v3::resize(), tbb::internal::generic_scheduler::spawn(), tbb::task::spawn_and_wait_for_all(), tbb::internal::generic_scheduler::spawn_root_and_wait(), and tbb::internal::custom_scheduler< SchedulerTraits >::wait_for_all().

129  {
130  uintptr_t v = theTLS.get();
131  return (v&1) ? tls_scheduler_of(v) : init_scheduler( task_scheduler_init::automatic, 0, /*auto_init=*/true );
132  }
static generic_scheduler * tls_scheduler_of(uintptr_t v)
Converts TLS value to the scheduler pointer.
Definition: governor.h:122
static generic_scheduler * init_scheduler(int num_threads, stack_size_type stack_size, bool auto_init)
Processes scheduler initialization request (possibly nested) in a master thread.
Definition: governor.cpp:172
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:54
Here is the call graph for this function:
Here is the caller graph for this function:

◆ local_scheduler_if_initialized()

static generic_scheduler* tbb::internal::governor::local_scheduler_if_initialized ( )
inlinestatic

Definition at line 139 of file governor.h.

References tbb::internal::basic_tls< T >::get(), theTLS, and tls_scheduler_of().

Referenced by tbb::internal::market::adjust_demand(), tbb::internal::market::cleanup(), tbb::internal::arena::enqueue_task(), and terminate_auto_initialized_scheduler().

139  {
140  return tls_scheduler_of( theTLS.get() );
141  }
static generic_scheduler * tls_scheduler_of(uintptr_t v)
Converts TLS value to the scheduler pointer.
Definition: governor.h:122
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:54
Here is the call graph for this function:
Here is the caller graph for this function:

◆ local_scheduler_weak()

static generic_scheduler* tbb::internal::governor::local_scheduler_weak ( )
inlinestatic

Definition at line 134 of file governor.h.

References tbb::internal::basic_tls< T >::get(), init_scheduler_weak(), theTLS, and tls_scheduler_of().

Referenced by tbb::internal::allocate_additional_child_of_proxy::allocate(), tbb::internal::allocate_root_proxy::allocate(), tbb::internal::allocate_continuation_proxy::allocate(), tbb::internal::allocate_child_proxy::allocate(), tbb::interface5::internal::task_base::destroy(), tbb::internal::allocate_additional_child_of_proxy::free(), tbb::internal::allocate_root_proxy::free(), tbb::internal::allocate_continuation_proxy::free(), tbb::internal::allocate_child_proxy::free(), and tbb::task::self().

134  {
135  uintptr_t v = theTLS.get();
136  return v ? tls_scheduler_of(v) : init_scheduler_weak();
137  }
static generic_scheduler * tls_scheduler_of(uintptr_t v)
Converts TLS value to the scheduler pointer.
Definition: governor.h:122
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:54
static generic_scheduler * init_scheduler_weak()
Automatic initialization of scheduler in a master thread with default settings without arena...
Definition: governor.cpp:164
Here is the call graph for this function:
Here is the caller graph for this function:

◆ one_time_init()

void tbb::internal::governor::one_time_init ( )
static

Definition at line 156 of file governor.cpp.

References tbb::internal::atomic_do_once(), tbb::internal::DoOneTimeInitializations(), and tbb::internal::__TBB_InitOnce::initialization_done().

Referenced by init_scheduler(), and init_scheduler_weak().

156  {
159 #if __TBB_SURVIVE_THREAD_SWITCH
160  atomic_do_once( &initialize_cilk_interop, cilkrts_load_state );
161 #endif /* __TBB_SURVIVE_THREAD_SWITCH */
162 }
static bool initialization_done()
Definition: tbb_main.h:64
void atomic_do_once(const F &initializer, atomic< do_once_state > &state)
One-time initialization function.
Definition: tbb_misc.h:213
void DoOneTimeInitializations()
Performs thread-safe lazy one-time general TBB initialization.
Definition: tbb_main.cpp:215
Here is the call graph for this function:
Here is the caller graph for this function:

◆ print_version_info()

void tbb::internal::governor::print_version_info ( )
static

Definition at line 229 of file governor.cpp.

References tbb::internal::PrintExtraVersionInfo(), tbb::internal::PrintRMLVersionInfo(), theRMLServerFactory, and UsePrivateRML.

Referenced by tbb::internal::DoOneTimeInitializations().

229  {
230  if ( UsePrivateRML )
231  PrintExtraVersionInfo( "RML", "private" );
232  else {
233  PrintExtraVersionInfo( "RML", "shared" );
234  theRMLServerFactory.call_with_server_info( PrintRMLVersionInfo, (void*)"" );
235  }
236 #if __TBB_SURVIVE_THREAD_SWITCH
237  if( watch_stack_handler )
238  PrintExtraVersionInfo( "CILK", CILKLIB_NAME );
239 #endif /* __TBB_SURVIVE_THREAD_SWITCH */
240 }
void PrintRMLVersionInfo(void *arg, const char *server_info)
A callback routine to print RML version information on stderr.
Definition: tbb_misc.cpp:222
static rml::tbb_factory theRMLServerFactory
Definition: governor.h:62
static bool UsePrivateRML
Definition: governor.h:64
void PrintExtraVersionInfo(const char *category, const char *format,...)
Prints arbitrary extra TBB version information on stderr.
Definition: tbb_misc.cpp:211
Here is the call graph for this function:
Here is the caller graph for this function:

◆ release_resources()

void tbb::internal::governor::release_resources ( )
staticprivate

Destroy the thread-local storage key and deinitialize RML.

Definition at line 79 of file governor.cpp.

References tbb::internal::basic_tls< T >::destroy(), tbb::internal::destroy_process_mask(), dynamic_unlink_all(), tbb::internal::basic_tls< T >::get(), tbb::internal::__TBB_InitOnce::initialization_done(), tbb::internal::runtime_warning(), theRMLServerFactory, and theTLS.

Referenced by tbb::internal::__TBB_InitOnce::remove_ref().

79  {
80  theRMLServerFactory.close();
82 #if TBB_USE_ASSERT
84  runtime_warning( "TBB is unloaded while tbb::task_scheduler_init object is alive?" );
85 #endif
86  int status = theTLS.destroy();
87  if( status )
88  runtime_warning("failed to destroy task scheduler TLS: %s", strerror(status));
90 }
static bool initialization_done()
Definition: tbb_main.h:64
void dynamic_unlink_all()
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:54
static rml::tbb_factory theRMLServerFactory
Definition: governor.h:62
void destroy_process_mask()
Definition: tbb_misc.h:266
void __TBB_EXPORTED_FUNC runtime_warning(const char *format,...)
Report a runtime warning.
Here is the call graph for this function:
Here is the caller graph for this function:

◆ rethrow_exception_broken()

static bool tbb::internal::governor::rethrow_exception_broken ( )
inlinestatic

Definition at line 159 of file governor.h.

References is_rethrow_broken.

159 { return is_rethrow_broken; }
static bool is_rethrow_broken
Definition: governor.h:68

◆ sign_off()

void tbb::internal::governor::sign_off ( generic_scheduler s)
static

Unregister TBB scheduler instance from thread-local storage.

Definition at line 145 of file governor.cpp.

References __TBB_ASSERT, assume_scheduler(), __cilk_tbb_unwatch_thunk::data, is_set(), __cilk_tbb_unwatch_thunk::routine, s, and tbb::internal::suppress_unused_warning().

Referenced by tbb::internal::generic_scheduler::cleanup_scheduler().

145  {
147  __TBB_ASSERT( is_set(s), "attempt to unregister a wrong scheduler instance" );
148  assume_scheduler(NULL);
149 #if __TBB_SURVIVE_THREAD_SWITCH
150  __cilk_tbb_unwatch_thunk &ut = s->my_cilk_unwatch_thunk;
151  if ( ut.routine )
152  (*ut.routine)(ut.data);
153 #endif /* __TBB_SURVIVE_THREAD_SWITCH */
154 }
void suppress_unused_warning(const T1 &)
Utility template function to prevent "unused" warnings by various compilers.
Definition: tbb_stddef.h:398
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:120
__cilk_tbb_pfn_unwatch_stacks routine
void const char const char int ITT_FORMAT __itt_group_sync s
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:165
static void assume_scheduler(generic_scheduler *s)
Temporarily set TLS slot to the given scheduler.
Definition: governor.cpp:116
Here is the call graph for this function:
Here is the caller graph for this function:

◆ sign_on()

void tbb::internal::governor::sign_on ( generic_scheduler s)
static

Register TBB scheduler instance in thread-local storage.

Definition at line 124 of file governor.cpp.

References __TBB_ASSERT, assume_scheduler(), __cilk_tbb_stack_op_thunk::data, is_set(), __cilk_tbb_stack_op_thunk::routine, and s.

Referenced by tbb::internal::generic_scheduler::create_master(), and tbb::internal::generic_scheduler::create_worker().

124  {
125  __TBB_ASSERT( is_set(NULL) && s, NULL );
126  assume_scheduler( s );
127 #if __TBB_SURVIVE_THREAD_SWITCH
128  if( watch_stack_handler ) {
130  o.routine = &stack_op_handler;
131  o.data = s;
132  if( (*watch_stack_handler)(&s->my_cilk_unwatch_thunk, o) ) {
133  // Failed to register with cilkrts, make sure we are clean
134  s->my_cilk_unwatch_thunk.routine = NULL;
135  }
136 #if TBB_USE_ASSERT
137  else
138  s->my_cilk_state = generic_scheduler::cs_running;
139 #endif /* TBB_USE_ASSERT */
140  }
141 #endif /* __TBB_SURVIVE_THREAD_SWITCH */
142  __TBB_ASSERT( is_set(s), NULL );
143 }
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:120
__cilk_tbb_pfn_stack_op routine
void const char const char int ITT_FORMAT __itt_group_sync s
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:165
static void assume_scheduler(generic_scheduler *s)
Temporarily set TLS slot to the given scheduler.
Definition: governor.cpp:116
Here is the call graph for this function:
Here is the caller graph for this function:

◆ speculation_enabled()

static bool tbb::internal::governor::speculation_enabled ( )
inlinestatic

Definition at line 158 of file governor.h.

References is_speculation_enabled.

158 { return is_speculation_enabled; }
static bool is_speculation_enabled
Definition: governor.h:67

◆ terminate_auto_initialized_scheduler()

static void tbb::internal::governor::terminate_auto_initialized_scheduler ( )
inlinestatic

Undo automatic initialization if necessary; call when a thread exits.

Definition at line 144 of file governor.h.

References auto_terminate(), and local_scheduler_if_initialized().

Referenced by tbb::internal::__TBB_InitOnce::~__TBB_InitOnce().

144  {
146  }
static generic_scheduler * local_scheduler_if_initialized()
Definition: governor.h:139
static void auto_terminate(void *scheduler)
The internal routine to undo automatic initialization.
Definition: governor.cpp:215
Here is the call graph for this function:
Here is the caller graph for this function:

◆ terminate_scheduler()

bool tbb::internal::governor::terminate_scheduler ( generic_scheduler s,
bool  blocking 
)
static

Processes scheduler termination request (possibly nested) in a master thread.

Definition at line 205 of file governor.cpp.

References __TBB_ASSERT, is_set(), and s.

Referenced by tbb::task_scheduler_init::internal_terminate().

205  {
206  bool ok = false;
207  __TBB_ASSERT( is_set(s), "Attempt to terminate non-local scheduler instance" );
208  if (0 == --(s->my_ref_count)) {
209  ok = s->cleanup_master( blocking );
210  __TBB_ASSERT( is_set(NULL), "cleanup_master has not cleared its TLS slot" );
211  }
212  return ok;
213 }
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:120
void const char const char int ITT_FORMAT __itt_group_sync s
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:165
Here is the call graph for this function:
Here is the caller graph for this function:

◆ tls_scheduler_of()

static generic_scheduler* tbb::internal::governor::tls_scheduler_of ( uintptr_t  v)
inlinestatic

Converts TLS value to the scheduler pointer.

Definition at line 122 of file governor.h.

Referenced by auto_terminate(), init_scheduler(), local_scheduler(), local_scheduler_if_initialized(), and local_scheduler_weak().

122  {
123  return (generic_scheduler*)(v & ~uintptr_t(1));
124  }
Here is the caller graph for this function:

◆ tls_value_of()

uintptr_t tbb::internal::governor::tls_value_of ( generic_scheduler s)
static

Computes the value of the TLS.

Definition at line 110 of file governor.cpp.

References __TBB_ASSERT, and s.

Referenced by assume_scheduler(), and is_set().

110  {
111  __TBB_ASSERT( (uintptr_t(s)&1) == 0, "Bad pointer to the scheduler" );
112  // LSB marks the scheduler initialized with arena
113  return uintptr_t(s) | uintptr_t((s && (s->my_arena || s->is_worker()))? 1 : 0);
114 }
void const char const char int ITT_FORMAT __itt_group_sync s
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:165
Here is the caller graph for this function:

Friends And Related Function Documentation

◆ __TBB_InitOnce

friend class __TBB_InitOnce
friend

Definition at line 50 of file governor.h.

◆ market

friend class market
friend

Definition at line 51 of file governor.h.

Member Data Documentation

◆ DefaultNumberOfThreads

unsigned tbb::internal::governor::DefaultNumberOfThreads
staticprivate

Caches the maximal level of parallelism supported by the hardware.

Definition at line 57 of file governor.h.

Referenced by default_num_threads().

◆ DefaultPageSize

size_t tbb::internal::governor::DefaultPageSize
staticprivate

Caches the size of OS regular memory page.

Definition at line 60 of file governor.h.

Referenced by default_page_size().

◆ is_rethrow_broken

bool tbb::internal::governor::is_rethrow_broken
staticprivate

Definition at line 68 of file governor.h.

Referenced by acquire_resources(), and rethrow_exception_broken().

◆ is_speculation_enabled

bool tbb::internal::governor::is_speculation_enabled
staticprivate

Definition at line 67 of file governor.h.

Referenced by acquire_resources(), and speculation_enabled().

◆ theRMLServerFactory

rml::tbb_factory tbb::internal::governor::theRMLServerFactory
staticprivate

◆ theTLS

basic_tls< uintptr_t > tbb::internal::governor::theTLS
staticprivate

TLS for scheduler instances associated with individual threads.

Definition at line 54 of file governor.h.

Referenced by acquire_resources(), assume_scheduler(), init_scheduler(), is_set(), local_scheduler(), local_scheduler_if_initialized(), local_scheduler_weak(), and release_resources().

◆ UsePrivateRML

bool tbb::internal::governor::UsePrivateRML
staticprivate

The documentation for this class was generated from the following files:

Copyright © 2005-2020 Intel Corporation. All Rights Reserved.

Intel, Pentium, Intel Xeon, Itanium, Intel XScale and VTune are registered trademarks or trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

* Other names and brands may be claimed as the property of others.