marEx.helper.get_cluster_info

marEx.helper.get_cluster_info(client)[source]

Get and print cluster connection information.

Parameters:: client (Client) – Dask client connected to a cluster.
Returns:: Dictionary containing connection information.
Return type:: dict

Examples

Basic cluster info retrieval:

>>> import marEx
>>>
>>> # Start local cluster
>>> client = marEx.start_local_cluster(n_workers=2)
>>>
>>> # Get connection information
>>> info = marEx.get_cluster_info(client)
Hostname: login01
Forward Port: login01:8787
Dashboard Link: localhost:8787/status
>>>
>>> print(f"Connect via: {info['dashboard_link']}")
Connect via: localhost:8787/status
>>> client.close()

SSH tunneling for remote access:

>>> # Start cluster on HPC system
>>> client = marEx.start_distributed_cluster(
...     n_workers=8, workers_per_node=4, dashboard_address=8889
... )
>>>
>>> # Get tunneling information
>>> info = marEx.get_cluster_info(client)
Hostname: levante-login01
Forward Port: levante-login01:8889
Dashboard Link: localhost:8889/status
>>>
>>> # Use this info to set up SSH tunnel:
>>> # ssh -L 8889:localhost:8889 levante-login01.dkrz.de
>>> # Then access dashboard at localhost:8889/status
>>>
>>> client.close()

Monitoring cluster status:

>>> client = marEx.start_local_cluster(n_workers=4)
>>> info = marEx.get_cluster_info(client)
>>>
>>> # Access cluster details
>>> print(f"Dashboard URL: {client.dashboard_link}")
>>> print(f"Cluster address: {client.cluster.scheduler_address}")
>>> print(f"Total threads: {client.nthreads()}")
>>>
>>> # Use port for programmatic access
>>> import requests
>>> try:
...     response = requests.get(f"http://localhost:{info['port']}/info")
...     print(f"Scheduler info: {response.status_code}")
... except:
...     print("Dashboard not accessible")
>>>
>>> client.close()