Records in a Project#

A project holds records under project-local names, so that a record can be referred to as "water_scan" rather than by its ID. Every method here that takes a record accepts either the name or the ID.

There are three ways a record ends up in a project:

  • Adding - a new computation is created and submitted to be run

  • Linking - a record that already exists on the server is associated with the project

  • Importing - a computation that was run elsewhere is ingested into the server

Adding records#

add_record() creates a new record in the project and submits it for computation.

This differs from the client’s add_* methods (see Submitting computations) in two ways. It creates exactly one record rather than one per molecule, and it takes the specification and input molecule together as a single record input object rather than as separate arguments. The input types are SinglepointInput, OptimizationInput, and the equivalent for each of the other computation types.

>>> from qcportal.singlepoint import SinglepointInput

>>> inp = SinglepointInput(
...     molecule=Molecule(symbols=['h', 'h'], geometry=[0, 0, 0, 0, 0, 1.5]),
...     specification={
...         "program": "psi4",
...         "driver": "energy",
...         "method": "b3lyp",
...         "basis": "def2-svp",
...     },
... )

>>> r = proj.add_record("hydrogen", inp)
>>> print(r.id, r.status)
118326390 RecordStatusEnum.waiting

Unlike the client’s add_* methods, this returns the record itself rather than metadata and a list of IDs.

The name must be unique within the project - reusing a name raises a PortalRequestError. There is no existing_ok equivalent here, so check record_metadata first if a script may be run more than once.

The optional arguments are keyword-only:

  • description - A longer description of this record within the project

  • tags - A list of strings for categorizing this record within the project

  • compute_tag - The compute tag to run with. Defaults to the project’s default_compute_tag

  • compute_priority - The priority to run at. Defaults to the project’s default_compute_priority

  • find_existing - If False, always create a new record rather than reusing a matching existing one. See Record Deduplication

>>> r = proj.add_record("hydrogen_fresh", inp,
...                     description="Same computation, deliberately not deduplicated",
...                     tags=["scratch"],
...                     compute_tag="small_mem",
...                     compute_priority="high",
...                     find_existing=False)

Note

Deduplication applies as usual. With the default find_existing=True, adding a record whose computation already exists on the server links the project to that existing record instead of running it again - so the returned record may already be complete.

Linking existing records#

link_record() associates a record that already exists on the server with the project. Nothing is copied and the record itself is unchanged; the project gains a reference to it plus a project-local name, description, and tags. All four arguments are required.

>>> meta, ids = client.add_singlepoints(mol, 'psi4', 'energy', 'hf', 'sto-3g')
>>> r = proj.link_record(ids[0], "reference_hf", "HF reference for comparison", ["reference"])
>>> print(r.id)
118326412

The same record may be linked into more than one project. Linking a record that is already in this project is an error.

>>> proj.link_record(ids[0], "reference_hf_again", "", [])
---------------------------------------------------------------------------
PortalRequestError                        Traceback (most recent call last)

...

PortalRequestError: Request failed: Record 118326412 already linked to project 7 (HTTP status 400)

Importing records#

import_record() ingests a computation that was run somewhere else - on another QCArchive server, or by hand - and stores its results as a record on this server. The computation is not run again; the results are stored as they are given.

This is the only way to get externally-computed records onto a server, and it works only into a project.

>>> # A record retrieved from some other server
>>> other_client = PortalClient("https://ml.qcarchive.molssi.org")
>>> external = other_client.get_singlepoints(123, include=['**'])

>>> r = proj.import_record("imported_hf", external,
...                        description="Imported from the ML server",
...                        tags=["imported"])

>>> print(r.status)
RecordStatusEnum.complete

>>> print(r.creator_user)
ben

The imported record is a normal record on this server afterwards, with a new ID. The importing user becomes its creator_user.

Note

Fetch the source record with include=['**'] before importing it. Only the data actually present on the object is transferred, so a partially-fetched record imports partial results.

Records of every computation type can be imported, as can the raw QCSchema results that a compute manager would return (AtomicResult, OptimizationResult, and FailedOperation from qcportal.qcschema_v1).

Important

Importing is new and still limited. Only individual records can be imported - there is no facility for importing a whole dataset - and only into a project. See PR 972.

Getting records#

get_record() fetches a record from the project, by project-local name or by ID.

>>> r = proj.get_record("hydrogen")
>>> print(r.id, r.status)
118326390 RecordStatusEnum.complete

>>> # By ID works too
>>> r = proj.get_record(118326390)

>>> # Fetch all of the record's data up front
>>> r = proj.get_record("hydrogen", include=['**'])

The record that comes back is an ordinary record of the appropriate type - a SinglepointRecord here - and behaves exactly as one retrieved through get_records(). See Record & Computation Types.

Note

Looking a record up by name requires the project’s record metadata to have been fetched, which happens automatically on first use. Records added during this session are also findable by name.

Listing records#

The record_metadata property lists what the project contains without fetching the records themselves. Each entry is a ProjectRecordMetadata object holding the project-local name, description, and tags, plus the record’s ID, type, and status.

>>> for rm in proj.record_metadata:
...     print(rm.record_id, rm.record_type, rm.status, rm.name, rm.tags)
118326390 singlepoint RecordStatusEnum.complete hydrogen []
118326412 singlepoint RecordStatusEnum.complete reference_hf ['reference']

This is fetched from the server the first time it is accessed and then cached. The status values in particular go stale, since they are a snapshot from when the metadata was fetched. Call fetch_record_metadata() to refresh it.

>>> proj.fetch_record_metadata()

For an up-to-date count by status across the whole project, use status() instead - see Project status.

Removing records#

unlink_records() removes records from the project. By default the records stay on the server and only the association with the project is dropped - this is the reverse of linking, and it applies equally to records that were added or imported.

It accepts a single name or ID, or a list, and the two may be mixed.

>>> proj.unlink_records("hydrogen")

>>> proj.unlink_records(["reference_hf", 118326390])

Pass delete_records=True to delete the records themselves as well.

>>> proj.unlink_records("hydrogen_fresh", delete_records=True)

Note

delete_records=True attempts to remove the record from the server, not just from the project, and the deletion is permanent rather than the reversible soft delete.

It is best effort, though. A record that is also in another project or in a dataset is protected by the database and will survive - unlinked from this project, but still on the server - and nothing in the return value tells you so. See Shared records and datasets when deleting.

Records QCPortal API#