Setup Overview¶
QCFractal comprises two components:
The Server (
qcfractal-server), which accepts compute and data queries and maintains a database of tasks and results. The Server should be run continuously on a persistent machine.One or more Managers (
qcfractal-manager). The Managers pull work from the Server, use attached compute resources to complete the work, and report results back to the server. Managers may be turned off and on at any time. Managers connect to compute resources through Adapters.
In the Quickstart Tutorial, the above components were combined within a python environment using FractalSnowflake.
In general, the Server and Manager(s) are run separately in different processes, often on different machines.
For detailed information about the relationship between Server and Manager, see Fractal Queue Managers.
Common Use Cases¶
The table below lists some common use cases for QCFractal:
Use case |
|
|
Recommended manager |
|---|---|---|---|
Snowflake |
Snowflake |
Snowflake |
|
Local |
Local |
Pool |
|
Head node |
Head node |
Parsl |
|
Personal server, head node (if permitted) |
Head node |
Parsl |
|
Personal server |
Head node of each cluster |
Parsl |
|
Personal server or cloud instance |
Docker container |
Pool |
QCFractal is highly adaptable and is not limited to the above use cases.
For example, it possible to mix local, cluster, supercomputer, and cloud Managers simultaneously.
In addition, a cloud instance may provide a good option for running qcfractal-server when a persistent web-exposed server is not otherwise available.
Quickstart Setups¶
This section presents quickstart setup guides for the above common use cases. The guides assume that QCFractal has been installed (see Install QCFractal). General guides are also available:
Single Workstation¶
This quickstart guide addresses QCFractal setup on a single computer which will be used for the Server, Manager, user client, and compute. On the workstation, initialize the Server:
qcfractal-server init
Next, start the Server and ProcessPoolExecutor Manager:
nohup qcfractal-server start --local-manager 1 &
The second command starts qcfractal-server in the background.
It also starts one Worker which will pull tasks from the Server and run them.
Test if everything is setup by running a Hartree-Fock calculation on a single hydrogen molecule,
as in the Example (note this requires psi4):
python
>>> import qcfractal.interface as ptl
# Note that server TLS verification is turned off (verify=False) since all components are run locally.
>>> client = ptl.FractalClient(address="localhost:7777", verify=False)
>>> mol = ptl.Molecule(symbols=["H", "H"], geometry=[0, 0, 0, 0, 5, 0])
>>> mol_id = client.add_molecules([mol])[0]
>>> r = client.add_compute("psi4", "HF", "STO-3G", "energy", None, [mol_id])
# Wait a minute for the job to complete
>>> proc = client.query_procedures(id=r.ids)[0]
>>> print(proc)
<ResultRecord(id='0' status='COMPLETE')>
>>> print(proc.properties.scf_total_energy)
-0.6865598095254312
Private Cluster¶
This quickstart guide addresses QCFractal setup on a private cluster comprising a head node and compute nodes, with a Scheduler such as SLURM, PBS, or Torque.
This guide requires Parsl which may be installed with pip or conda.
Begin by initializing the Server on the cluster head node:
qcfractal-server init
Next, start the Server in the background:
nohup qcfractal-server start &
The Manager must be configured before use. Create a configuration file (e.g. in ~/.qca/qcfractal/my_manager.yaml) based on the following template:
common:
adapter: parsl
tasks_per_worker: 1
cores_per_worker: 6
memory_per_worker: 64
max_workers: 5
scratch_directory: "$TMPDIR"
cluster:
node_exclusivity: True
scheduler: slurm
parsl:
provider:
partition: CLUSTER
cmd_timeout: 30
You may need to modify these values to match the particulars of your cluster. In particular:
The
schedulerandpartitionoptions should be set to match the details of your Scheduler (e.g. SLURM, PBS, Torque).Options related to Workers should be set appropriately for the compute node on your cluster. Note that Parsl requires that full nodes be allocated to each Worker (i.e.
node_exclusivity: True).
For more information on Manager configuration, see Fractal Queue Managers and Queue Manager Example YAML Files.
Finally, start the Manager in the background on the cluster head node:
nohup qcfractal-manager --config-file <path to config YAML> --verify=False &
Note that TLS certificate verification is disabled (--verify=False) because the Manager and Server are both run on the head node.
Test if everything is setup by running a Hartree-Fock calculation on a single hydrogen molecule,
as in the Example (note this requires psi4):
python
>>> import qcfractal.interface as ptl
# Note that server TLS verification is turned off (verify=False) since all components are run locally.
>>> client = ptl.FractalClient(address="localhost:7777", verify=False)
>>> mol = ptl.Molecule(symbols=["H", "H"], geometry=[0, 0, 0, 0, 5, 0])
>>> mol_id = client.add_molecules([mol])[0]
>>> r = client.add_compute("psi4", "HF", "STO-3G", "energy", None, [mol_id])
# Wait a minute for the job to complete
>>> proc = client.query_procedures(id=r.ids)[0]
>>> print(proc)
<ResultRecord(id='0' status='COMPLETE')>
>>> print(proc.properties.scf_total_energy)
-0.6865598095254312
Cloud Compute¶
This quickstart guide addresses QCFractal setup using cloud resources for computation. The Server should be set up on a persistent server for which you have permission to expose ports. For example, this may be a dedicated webserver, the head node of a private cluster, or a cloud instance. The Manager will be set up on a Kubernetes cluster as a Deployment.
Begin by initializing the Server on your persistent server:
qcfractal-server init
The QCFractal server receives connections from Managers and clients on TCP port 7777.
You may optionally specify the --port option to choose a custom port.
You may need to configure your firewall to allow access to this port.
Because the Server will be exposed to the internet,
security should be enabled to control access.
Enable security by changing the YAML file (default: ~/.qca/qcfractal/qcfractal_config.yaml)
fractal.security option to local:
- security: null
+ security: local
Start the Server:
nohup qcfractal-server start &
Note
You may optionally provide a TLS certificate to enable host verification for the Server
using the --tls-cert and --tls-key options.
If a TLS certificate is not provided, communications with the server will still be encrypted,
but host verification will be unavailable
(and Managers and clients will need to specify verify=False).
Next, add users for admin, the Manager, and a user (you may choose whatever usernames you like):
qcfractal-server user add admin --permissions admin
qcfractal-server user add manager --permissions queue
qcfractal-server user add user --permissions read write compute
Passwords will be automatically generated and printed. You may instead specify a password with the --password option.
See Fractal Server User for more information.
The Manager will be set up on a Kubernetes cluster as a Deployment, running Docker images which each contain QCEngine, QCFractal, and relevant programs. In this guide, we use the molssi/qcarchive_worker_openff Docker image. For execution, this image includes:
Note
You may wish to set up a custom Docker image for your specific use case. The Dockerfile corresponding to the molssi/qcarchive_worker_openff image is included below as an example.
FROM continuumio/miniconda3
RUN conda install -c psi4/label/dev -c conda-forge psi4 dftd3 mp2d qcengine qcfractal rdkit geometric
RUN groupadd -g 999 qcfractal && \
useradd -m -r -u 999 -g qcfractal qcfractal
USER qcfractal
ENV PATH /opt/local/conda/envs/base/bin/:$PATH
ENTRYPOINT qcfractal-manager --config-file /etc/qcfractal-manager/manager.yaml
Create a manager configuration file (e.g. manager.yaml) following the template below.
common:
adapter: pool
tasks_per_worker: 1
cores_per_worker: 4 # CHANGEME number of cores/worker
memory_per_worker: 16 # CHANGEME memory/worker in Gb
max_workers: 1
scratch_directory: "$TMPDIR"
server:
fractal_uri: api.qcarchive.molssi.org:443 # CHANGEME URI of your server goes here
username: manager
password: foo # CHANGEME manager password goes here
verify: True # False if TLS was skipped earlier
manager:
manager_name: MyManager # CHANGEME name your manager
queue_tag: null
log_file_prefix: null
update_frequency: 30
test: False
Add the manager configuration as a secret in Kubernetes:
kubectl create secret generic manager-config-yaml --from-file=manager.yaml
This allows us to pass the manager configuration into the Docker container securely.
Next, create a Kubernetes deployment configuration file (e.g. deployment.yaml) following the template below.
The cpu and memory fields of the deployment configuration should match the cores_per_worker
and memory_per_worker fields of the manager configuration.
In this setup, replicas determines the number of workers; the max_workers and tasks_per_worker fields
in the manager configuration should be set to 1.
apiVersion: apps/v1
kind: Deployment
metadata:
name: qcfractal-manager
labels:
k8s-app: qcfractal-manager
spec:
replicas: 4 # CHANGEME: number of images here
selector:
matchLabels:
k8s-app: qcfractal-manager
template:
metadata:
labels:
k8s-app: qcfractal-manager
spec:
containers:
- image: molssi/qcarchive_worker_openff # you may wish to specify your own Docker image here
name: qcfractal-manager-pod
resources:
limits:
cpu: 4 # CHANGEME number of cores/worker
memory: 16Gi # CHANGEME memory/worker
volumeMounts:
- name: manager-config-secret
mountPath: "/etc/qcfractal-manager"
readOnly: true
volumes:
- name: manager-config-secret
secret:
secretName: manager-config-yaml
Start the deployment:
kubectl apply -f deployment.yaml
Note
You can view the status of your deployment with:
kubectl get deployments
You can view the status of individual “Pods” (Docker containers) with:
kubectl get pods --show-labels
To get the output of invidual Managers:
kubectl logs <pod name>
To get Kubernetes metadata and status information about a Pod:
kubectl describe pod <pod name>
See the Kubernetes Deployment documentation for more information.
Test if everything is setup by running a Hartree-Fock calculation on a single hydrogen molecule,
as in the Example
(note this requires psi4 to be installed on at least one compute resource).
This test may be run from any machine.
python
>>> import qcfractal.interface as ptl
# Note that server TLS verification may need to be turned off if (verify=False).
# Note that the Server URL and the password for user will need to be filled in.
>>> client = ptl.FractalClient(address="URL:Port", username="user", password="***")
>>> mol = ptl.Molecule(symbols=["H", "H"], geometry=[0, 0, 0, 0, 5, 0])
>>> mol_id = client.add_molecules([mol])[0]
>>> r = client.add_compute("psi4", "HF", "STO-3G", "energy", None, [mol_id])
# Wait a minute for the job to complete
>>> proc = client.query_procedures(id=r.ids)[0]
>>> print(proc)
<ResultRecord(id='0' status='COMPLETE')>
>>> print(proc.properties.scf_total_energy)
-0.6865598095254312