QueueManager

class qcfractal.QueueManager(client: FractalClient, queue_client: BaseAdapter, logger: Optional[logging.Logger] = None, max_tasks: int = 200, queue_tag: Union[List[str], str, None] = None, manager_name: str = 'unlabeled', update_frequency: Union[int, float] = 2, verbose: bool = True, server_error_retries: Optional[int] = 1, stale_update_limit: Optional[int] = 10, cores_per_task: Optional[int] = None, memory_per_task: Optional[float] = None, nodes_per_task: Optional[int] = None, cores_per_rank: Optional[int] = 1, scratch_directory: Optional[str] = None, retries: Optional[int] = 2, configuration: Optional[Dict[str, Any]] = None)[source]

Bases: object

This object maintains a computational queue and watches for finished tasks for different queue backends. Finished tasks are added to the database and removed from the queue.

Variables
  • client (FractalClient) – A FractalClient connected to a server.

  • queue_adapter (QueueAdapter) – The DBAdapter class for queue abstraction

  • errors (dict) – A dictionary of current errors

  • logger (logging.logger. Optional, Default: None) – A logger for the QueueManager

Methods Summary

add_exit_callback(callback, *args, **kwargs)

Adds additional callbacks to perform when closing down the server.

assert_connected()

Raises an error for functions that require a server connection.

await_results()

A synchronous method for testing or small launches that awaits task completion.

close_adapter()

Closes down the underlying adapter.

connected()

Checks the connection to the server.

heartbeat()

Provides a heartbeat to the connected Server.

list_current_tasks()

Provides a list of tasks currently in the queue along with the associated keys.

name()

Returns the Managers full name.

shutdown()

Shutdown the manager and returns tasks to queue.

start()

Starts up all IOLoops and processes.

stop([signame, signum, stack])

Shuts down all IOLoops and periodic updates.

test([n])

Tests all known programs with simple inputs to check if the Adapter is correctly instantiated.

update([new_tasks, allow_shutdown])

Examines the queue for completed tasks and adds successful completions to the database while unsuccessful are logged for future inspection.

Methods Documentation

add_exit_callback(callback: Callable, *args: List[Any], **kwargs: Dict[Any, Any]) → None[source]

Adds additional callbacks to perform when closing down the server.

Parameters
  • callback (callable) – The function to call at exit

  • *args – Arguments to call with the function.

  • **kwargs – Kwargs to call with the function.

assert_connected() → None[source]

Raises an error for functions that require a server connection.

await_results() → bool[source]

A synchronous method for testing or small launches that awaits task completion.

Returns

Return True if the operation completed successfully

Return type

bool

close_adapter() → bool[source]

Closes down the underlying adapter.

connected() → bool[source]

Checks the connection to the server.

heartbeat() → None[source]

Provides a heartbeat to the connected Server.

list_current_tasks() → List[Any][source]

Provides a list of tasks currently in the queue along with the associated keys.

Returns

ret – All tasks currently still in the database

Return type

list of tuples

name() → str[source]

Returns the Managers full name.

shutdown() → Dict[str, Any][source]

Shutdown the manager and returns tasks to queue.

start() → None[source]

Starts up all IOLoops and processes.

stop(signame='Not provided', signum=None, stack=None) → None[source]

Shuts down all IOLoops and periodic updates.

test(n=1) → bool[source]

Tests all known programs with simple inputs to check if the Adapter is correctly instantiated.

update(new_tasks: bool = True, allow_shutdown=True) → bool[source]

Examines the queue for completed tasks and adds successful completions to the database while unsuccessful are logged for future inspection.

Parameters
  • new_tasks (bool, optional, Default: True) – Try to get new tasks from the server

  • allow_shutdown (bool, optional, Default: True) – Allow function to attempt graceful shutdowns in the case of stale job or fatal error limits. Does not prevent errors from being raise, but mostly used to prevent infinite loops when update is called from shutdown itself