diff --git a/.github/workflows/check_links.yml b/.github/workflows/check_links.yml index 526ec827b..f15cac997 100644 --- a/.github/workflows/check_links.yml +++ b/.github/workflows/check_links.yml @@ -13,4 +13,6 @@ jobs: - name: Link Checker uses: lycheeverse/lychee-action@v1.8.0 with: + # materialsproject.org blocks automated requests (Cloudflare 403) + args: --verbose --no-progress --exclude 'materialsproject\.org' './**/*.md' './**/*.html' fail: true diff --git a/README.md b/README.md index 3efca1142..f7bfea407 100644 --- a/README.md +++ b/README.md @@ -160,7 +160,7 @@ In order to run or edit the examples: 3. Optionally, for local Jupyter without OIDC, set legacy API token values in [settings.json](src/py/mat3ra/notebooks_utils/core/api/settings.json). See [Get Authentication Params](examples/system/get_authentication_params.ipynb) for details. API tokens can also be generated in [Account Preferences](https://docs.mat3ra.com/accounts/ui/preferences/api/) on the platform. -NOTE: The Materials Project API key should be set in `settings.json` and obtained from [https://legacy.materialsproject.org/open](https://legacy.materialsproject.org/open). +NOTE: The Materials Project API key should be set in `settings.json` and obtained from [https://next-gen.materialsproject.org/api](https://next-gen.materialsproject.org/api). ## Contribute diff --git a/examples/assets/README.md b/examples/assets/README.md index 69303e2d5..53fba23c8 100644 --- a/examples/assets/README.md +++ b/examples/assets/README.md @@ -2,6 +2,6 @@ This directory contains various files for the example notebooks. -- `mp-978534.poscar` contains a SiGe structure from [https://legacy.materialsproject.org/materials/mp-978534](https://legacy.materialsproject.org/materials/mp-978534). +- `mp-978534.poscar` contains a SiGe structure from [https://next-gen.materialsproject.org/materials/mp-978534](https://next-gen.materialsproject.org/materials/mp-978534). - `bash_workflow_template.json` contains the template JSON schema of Mat3ra workflow. - `Si.pz-vbc.UPF` is the pseudo potential file of silicon used in the Quantum Espresso workflow example. diff --git a/other/materials_designer/workflows/Introduction.ipynb b/other/materials_designer/workflows/Introduction.ipynb index 592d0a641..03d695968 100644 --- a/other/materials_designer/workflows/Introduction.ipynb +++ b/other/materials_designer/workflows/Introduction.ipynb @@ -69,7 +69,7 @@ "#### [6.4.1. Zero-point energy calculation.](zero_point_energy.ipynb)\n", "\n", "### 6.5. Defect Energy\n", - "#### 6.5.1. Defect formation energy. *(to be added)*\n", + "#### [6.5.1. Defect formation energy.](defect_formation_energy.ipynb)\n", "\n", "### 6.6. Formation Energy\n", "#### [6.6.1. Compound formation energy.](formation_energy.ipynb)\n", diff --git a/other/materials_designer/workflows/defect_formation_energy.ipynb b/other/materials_designer/workflows/defect_formation_energy.ipynb new file mode 100644 index 000000000..fa92265c2 --- /dev/null +++ b/other/materials_designer/workflows/defect_formation_energy.ipynb @@ -0,0 +1,623 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "0", + "metadata": {}, + "source": [ + "# Defect Formation Energy\n", + "\n", + "Calculate the neutral defect formation energy (eV) of a defective supercell using a multi-material DFT workflow on the Mat3ra platform.\n", + "\n", + "The workflow takes **two materials, in order**:\n", + "\n", + "- **[0] Defective supercell** — the structure containing the defect(s) (vacancy, substitution, interstitial, or a mix).\n", + "- **[1] Pristine supercell** — the defect-free version of the *same* supercell.\n", + "\n", + "Only the defective cell is computed here (`pw_scf`). The pristine reference energy is fetched from a previously-finished **Total Energy** job, so the pristine must already be converged/relaxed and have such a job on the platform. Elemental chemical potentials are taken from Standata elemental reference materials (each needs a total energy too), as in [Formation Energy](formation_energy.ipynb).\n", + "\n", + "Formula:\n", + "\n", + "$$E_{\\text{defect}} = E_{\\text{defective}}[q] - E_{\\text{pristine}} - \\sum_i \\Delta N_i\\, \\mu_i + q\\,(E_{\\text{VBM}} + E_F) \\quad [\\text{eV}]$$\n", + "\n", + "Set `CHARGE` (cell 1.3) to a non-zero value to charge the defective supercell (`tot_charge` in the QE `&SYSTEM` namelist, compensated by a uniform jellium background). **This notebook does not compute a charged-defect finite-size correction** (e.g. Freysoldt-Neugebauer-Van de Walle) or track the Fermi-level term $q(E_{\\text{VBM}}+E_F)$ -- for `CHARGE = 0` (the default) neither is needed and the value below is directly physical; for `CHARGE != 0` the reported value is the raw, uncorrected total-energy difference only.\n", + "\n", + "where $\\Delta N_i = \\text{count}_i(\\text{defective}) - \\text{count}_i(\\text{pristine})$ is the per-species atom-count change and $\\mu_i = E_{\\text{elemental},i} / n_{\\text{atoms},i}$.\n", + "\n", + "The reported value is the **total** formation energy of the whole defective configuration (per cell), not per defect: for a single isolated defect it equals the point-defect formation energy; for multiple/interacting defects it is the aggregate for that configuration.\n", + "\n", + "

Usage

\n", + "\n", + "1. Build a pristine supercell and a defective version of it (for example via `create_point_defect.ipynb`, which produces both), and export both to `../uploads` — or set the names below to match materials on the platform.\n", + "2. Run [Total Energy](total_energy.ipynb) for the pristine supercell, and for each Standata elemental reference material.\n", + "3. Set parameters in cells 1.2 and 1.3 below.\n", + "4. Click \"Run\" > \"Run All\".\n", + "5. Inspect the resolved references, their total energies, and the final defect formation energy.\n", + "\n", + "## Summary\n", + "\n", + "1. Set up the environment and parameters.\n", + "2. Authenticate and initialize API client.\n", + "3. Load defective and pristine supercells, resolve elemental references, verify the pristine total energy, and assemble the job materials.\n", + "4. Configure the Defect Formation Energy workflow.\n", + "5. Configure compute.\n", + "6. Create, submit, and monitor the multi-material job.\n", + "7. Retrieve results." + ] + }, + { + "cell_type": "markdown", + "id": "1", + "metadata": {}, + "source": [ + "## 1. Set up the environment and parameters\n", + "### 1.1. Install packages (JupyterLite)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2", + "metadata": {}, + "outputs": [], + "source": [ + "from mat3ra.notebooks_utils.packages import install_packages\n", + "\n", + "await install_packages(\"made|api_examples\")" + ] + }, + { + "cell_type": "markdown", + "id": "3", + "metadata": {}, + "source": [ + "### 1.2. Set parameters" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4", + "metadata": {}, + "outputs": [], + "source": [ + "from datetime import datetime\n", + "from mat3ra.ide.compute import QueueName\n", + "\n", + "# 2. Auth and organization parameters\n", + "ORGANIZATION_NAME = None\n", + "\n", + "# 3. Material parameters\n", + "FOLDER = \"../uploads\"\n", + "DEFECTIVE_NAME = \"Si\" # defective supercell: name in uploads or on the platform\n", + "PRISTINE_NAME = \"Si\" # defect-free version of the SAME supercell\n", + "\n", + "# 4. Workflow parameters\n", + "WORKFLOW_SEARCH_TERM = \"defect_formation_energy.json\"\n", + "APPLICATION_NAME = \"espresso\"\n", + "MY_WORKFLOW_NAME = \"Defect Formation Energy\"\n", + "\n", + "# 5. Compute parameters\n", + "CLUSTER_NAME = None # specify full or partial name i.e. \"cluster-001\" to select\n", + "QUEUE_NAME = QueueName.D\n", + "PPN = 1\n", + "\n", + "# 6. Job parameters\n", + "timestamp = datetime.now().strftime(\"%Y-%m-%d %H:%M\")\n", + "POLL_INTERVAL = 30 # seconds" + ] + }, + { + "cell_type": "markdown", + "id": "5", + "metadata": {}, + "source": [ + "### 1.3. Set specific defect formation energy parameters" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6", + "metadata": {}, + "outputs": [], + "source": [ + "# K-grid for the defective-cell SCF (if not set, KPPRA is used by default)\n", + "SCF_KGRID = None # e.g. [4, 4, 4]\n", + "\n", + "# Net charge on the defective supercell (e.g. -3 for a triply negatively charged\n", + "# defect, +1 for a singly positively charged one). 0 = neutral defect (default).\n", + "# NOTE: a non-zero charge requires a charged-defect finite-size correction (e.g.\n", + "# Freysoldt-Neugebauer-Van de Walle) to remove the spurious electrostatic\n", + "# interaction between the charged defect and its periodic images -- this\n", + "# notebook does NOT compute that correction, so CHARGE != 0 results are raw,\n", + "# uncorrected values only.\n", + "CHARGE = 0 # e.g. -3, +1\n", + "\n", + "# Whose total_energy properties to consider for the pristine reference:\n", + "# \"public\" (any owner, highest precision wins), \"curators\" (only curators'),\n", + "# or \"my_account\" (curators' or your own).\n", + "PRISTINE_TOTAL_ENERGY_SOURCE = \"my_account\"\n" + ] + }, + { + "cell_type": "markdown", + "id": "7", + "metadata": {}, + "source": [ + "## 2. Authenticate and initialize API client\n", + "### 2.1. Configure API endpoint\n", + "Local development: point the API client at a local platform instance. Remove or adjust these for the hosted platform." + ] + }, + { + "cell_type": "markdown", + "id": "8", + "metadata": {}, + "source": [ + "### 2.2. Authenticate\n", + "Authenticate in the browser and have credentials stored in environment variable `OIDC_ACCESS_TOKEN`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9", + "metadata": {}, + "outputs": [], + "source": [ + "from mat3ra.notebooks_utils.auth import authenticate\n", + "\n", + "await authenticate()" + ] + }, + { + "cell_type": "markdown", + "id": "10", + "metadata": {}, + "source": [ + "### 2.3. Initialize API client" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "11", + "metadata": {}, + "outputs": [], + "source": [ + "from mat3ra.api_client import APIClient\n", + "\n", + "client = APIClient.authenticate()\n", + "client" + ] + }, + { + "cell_type": "markdown", + "id": "12", + "metadata": {}, + "source": [ + "### 2.4. Select account to work under" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "13", + "metadata": {}, + "outputs": [], + "source": [ + "client.list_accounts()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "14", + "metadata": {}, + "outputs": [], + "source": [ + "selected_account = client.my_account\n", + "\n", + "if ORGANIZATION_NAME:\n", + " selected_account = client.get_account(name=ORGANIZATION_NAME)\n", + "\n", + "ACCOUNT_ID = selected_account.id\n", + "print(f\"✅ Selected account ID: {ACCOUNT_ID}, name: {selected_account.name}\")" + ] + }, + { + "cell_type": "markdown", + "id": "15", + "metadata": {}, + "source": [ + "### 2.5. Select project" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "16", + "metadata": {}, + "outputs": [], + "source": [ + "projects = client.projects.list({\"isDefault\": True, \"owner._id\": ACCOUNT_ID})\n", + "project_id = projects[0][\"_id\"]\n", + "print(f\"✅ Using project: {projects[0]['name']} ({project_id})\")" + ] + }, + { + "cell_type": "markdown", + "id": "17", + "metadata": {}, + "source": [ + "## 3. Load defective and pristine supercells\n", + "### 3.1. Load materials from local files or platform" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "18", + "metadata": {}, + "outputs": [], + "source": [ + "import re\n", + "from mat3ra.made.material import Material\n", + "from mat3ra.notebooks_utils.material import load_material_from_folder\n", + "from mat3ra.notebooks_utils.ipython.entity.material.visualize import visualize_materials as visualize\n", + "\n", + "\n", + "def load_material(name):\n", + " loaded = load_material_from_folder(FOLDER, name)\n", + " if loaded is not None:\n", + " print(f\"✅ Loaded '{name}' from folder: {loaded.name}\")\n", + " return loaded\n", + " matches = client.materials.list({\n", + " \"name\": {\"$regex\": re.escape(name), \"$options\": \"i\"},\n", + " \"owner._id\": ACCOUNT_ID,\n", + " })\n", + " if not matches:\n", + " raise ValueError(f\"No material containing '{name}' was found in '{FOLDER}' or on the platform.\")\n", + " material = Material.create(matches[0])\n", + " print(f\"♻️ Loaded '{name}' from platform: {matches[0]['_id']}\")\n", + " return material\n", + "\n", + "\n", + "defective = load_material(DEFECTIVE_NAME)\n", + "pristine = load_material(PRISTINE_NAME)\n", + "\n", + "visualize([{\"material\": defective, \"title\": \"Defective\"},\n", + " {\"material\": pristine, \"title\": \"Pristine\"}], repetitions=[1, 1, 1], rotation=\"-90x\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "19", + "metadata": {}, + "source": [ + "### 3.2. Prepare materials for Quantum ESPRESSO\n", + "Strip atom labels. Labels are useful for analysis notebooks but are not compatible with QE input generation." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "20", + "metadata": {}, + "outputs": [], + "source": [ + "defective_material = defective.clone()\n", + "defective_material.basis.set_labels_from_list([])\n", + "pristine_material = pristine.clone()\n", + "pristine_material.basis.set_labels_from_list([])\n", + "print(f\"Prepared defective: {defective_material.name}\")\n", + "print(f\"Prepared pristine: {pristine_material.name}\")" + ] + }, + { + "cell_type": "markdown", + "id": "21", + "metadata": {}, + "source": [ + "### 3.3. Resolve unique elements (union of defective and pristine)\n", + "The chemical-potential term covers every species whose count changes, so elements are taken from the union of the two structures (e.g. a species fully removed by a vacancy still appears via the pristine)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "22", + "metadata": {}, + "outputs": [], + "source": [ + "elements = sorted(\n", + " set(defective_material.basis.elements.values) | set(pristine_material.basis.elements.values)\n", + ")\n", + "print(f\"Elements (defective ∪ pristine): {elements}\")" + ] + }, + { + "cell_type": "markdown", + "id": "23", + "metadata": {}, + "source": [ + "### 3.4. Resolve Standata elemental reference materials\n", + "Elemental chemical potentials come from Standata materials tagged `elemental` with `metadata.element`. Each must also have a refined `total_energy` — this is enforced by the workflow at runtime; run [Total Energy](total_energy.ipynb) for any elemental reference that is missing one." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "24", + "metadata": {}, + "outputs": [], + "source": [ + "elemental_materials_data = client.materials.list(\n", + " {\"tags\": \"elemental\", \"metadata.element\": {\"$in\": elements}},\n", + ")\n", + "element_materials_by_symbol = {\n", + " element: [metadata for metadata in elemental_materials_data if\n", + " metadata.get(\"metadata\", {}).get(\"element\") == element]\n", + " for element in elements\n", + "}\n", + "missing = [element for element in elements if not element_materials_by_symbol.get(element)]\n", + "if missing:\n", + " raise RuntimeError(\n", + " f\"Missing elemental reference material(s) for {missing}. \"\n", + " \"Add elemental material(s) from Standata, or add tag 'elemental' with metadata.element = {element}\"\n", + " )\n", + "print(f\"Resolved elemental reference materials for: {', '.join(elements)}\")" + ] + }, + { + "cell_type": "markdown", + "id": "25", + "metadata": {}, + "source": [ + "### 3.5. Save materials and check the pristine total energy\n", + "Save both materials and confirm the pristine already has a finished Total Energy job (its energy is fetched, not recomputed), then assemble the job materials in order: `[0]` defective, `[1]` pristine." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "26", + "metadata": {}, + "outputs": [], + "source": [ + "from mat3ra.notebooks_utils.core.entity.material.api import get_or_create_material\n", + "from mat3ra.notebooks_utils.core.entity.property.api import find_total_energy_for_material\n", + "\n", + "saved_defective = Material.create(get_or_create_material(client, defective_material, ACCOUNT_ID))\n", + "saved_pristine = Material.create(get_or_create_material(client, pristine_material, ACCOUNT_ID))\n", + "\n", + "pristine_te_property = find_total_energy_for_material(\n", + " client, saved_pristine.id, source=PRISTINE_TOTAL_ENERGY_SOURCE\n", + ")\n", + "if pristine_te_property is None:\n", + " raise RuntimeError(\"Run total_energy.ipynb for the pristine material first.\")\n", + "\n", + "# Order matters: [0] defective (computed), [1] pristine (reference).\n", + "materials = [saved_defective, saved_pristine]\n" + ] + }, + { + "cell_type": "markdown", + "id": "27", + "metadata": {}, + "source": [ + "## 4. Configure the Defect Formation Energy workflow\n", + "### 4.1. Select application" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "28", + "metadata": {}, + "outputs": [], + "source": [ + "from mat3ra.ade.application import Application\n", + "from mat3ra.standata.applications import ApplicationStandata\n", + "\n", + "app_config = ApplicationStandata.get_by_name_first_match(APPLICATION_NAME)\n", + "app = Application(**app_config)\n", + "print(f\"Using application: {app.name}\")" + ] + }, + { + "cell_type": "markdown", + "id": "29", + "metadata": {}, + "source": [ + "### 4.2. Load workflow from Standata, apply parameters, and preview" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "30", + "metadata": {}, + "outputs": [], + "source": [ + "from mat3ra.standata.workflows import WorkflowStandata\n", + "from mat3ra.wode.workflows import Workflow\n", + "from mat3ra.wode.context.providers import PointsGridDataProvider\n", + "from mat3ra.notebooks_utils.ipython.entity.workflow.visualize import visualize_workflow\n", + "from mat3ra.notebooks_utils.workflow import patch_workflow_qe_input\n", + "\n", + "defect_workflow_config = WorkflowStandata.filter_by_application(app.name).get_by_name_first_match(\n", + " WORKFLOW_SEARCH_TERM\n", + ")\n", + "defect_workflow = Workflow.create(defect_workflow_config)\n", + "defect_workflow.name = MY_WORKFLOW_NAME\n", + "print(f\"Loaded workflow: {defect_workflow.name}\")\n", + "print(f\"Multi-material: {getattr(defect_workflow, 'isMultiMaterial', False)}\")\n", + "\n", + "# K-grid for the defective-cell SCF.\n", + "if SCF_KGRID is not None:\n", + " new_context = PointsGridDataProvider(dimensions=SCF_KGRID, isEdited=True).get_context_item_data()\n", + " for subworkflow in defect_workflow.subworkflows:\n", + " unit = subworkflow.get_unit_by_name(name=\"pw_scf\")\n", + " if unit:\n", + " unit.add_context(new_context)\n", + " subworkflow.set_unit(unit)\n", + "\n", + "# Net charge on the defective-cell SCF: adds `tot_charge` to the &SYSTEM namelist\n", + "if CHARGE:\n", + " patch_workflow_qe_input(defect_workflow, {\"system\": {\"tot_charge\": CHARGE}}, unit_names=[\"pw_scf\"])\n", + "\n", + "visualize_workflow(defect_workflow)" + ] + }, + { + "cell_type": "markdown", + "id": "31", + "metadata": {}, + "source": [ + "## 5. Create the compute configuration\n", + "### 5.1. Select cluster" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "32", + "metadata": {}, + "outputs": [], + "source": [ + "clusters = client.clusters.list()\n", + "print(f\"Available clusters: {[c['hostname'] for c in clusters]}\")" + ] + }, + { + "cell_type": "markdown", + "id": "33", + "metadata": {}, + "source": [ + "### 5.2. Create compute configuration" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "34", + "metadata": {}, + "outputs": [], + "source": [ + "from mat3ra.ide.compute import Compute\n", + "\n", + "if CLUSTER_NAME:\n", + " cluster = next((c for c in clusters if CLUSTER_NAME in c[\"hostname\"]), None)\n", + "else:\n", + " cluster = clusters[0]\n", + "\n", + "compute = Compute(cluster=cluster, queue=QUEUE_NAME, ppn=PPN)\n", + "print(f\"Using cluster: {compute.cluster.hostname}, queue: {QUEUE_NAME}, ppn: {PPN}\")" + ] + }, + { + "cell_type": "markdown", + "id": "35", + "metadata": {}, + "source": [ + "## 6. Create the Defect Formation Energy job\n", + "### 6.1. Create the multi-material job (defective + pristine)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "36", + "metadata": {}, + "outputs": [], + "source": [ + "from mat3ra.notebooks_utils.api.job import wait_for_jobs_to_finish_async\n", + "from mat3ra.notebooks_utils.job import create_job\n", + "from mat3ra.notebooks_utils.ui import display_JSON\n", + "from mat3ra.utils.namespace import dict_to_namespace_recursive\n", + "\n", + "defect_job_name = f\"{MY_WORKFLOW_NAME} {saved_defective.formula} {timestamp}\"\n", + "defect_job_response = create_job(\n", + " api_client=client,\n", + " materials=materials,\n", + " workflow=defect_workflow,\n", + " project_id=project_id,\n", + " owner_id=ACCOUNT_ID,\n", + " prefix=defect_job_name,\n", + " compute=compute.to_dict(),\n", + ")\n", + "\n", + "defect_job = dict_to_namespace_recursive(defect_job_response)\n", + "defect_job_id = defect_job._id\n", + "print(f\"✅ Defect Formation Energy job created: {defect_job_id}\")\n", + "display_JSON(defect_job_response)" + ] + }, + { + "cell_type": "markdown", + "id": "37", + "metadata": {}, + "source": [ + "### 6.2. Submit the Defect Formation Energy job and monitor the status" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "38", + "metadata": {}, + "outputs": [], + "source": [ + "client.jobs.submit(defect_job_id)\n", + "print(f\"✅ Job {defect_job_id} submitted successfully!\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "39", + "metadata": {}, + "outputs": [], + "source": [ + "await wait_for_jobs_to_finish_async(client.jobs, [defect_job_id], poll_interval=POLL_INTERVAL)" + ] + }, + { + "cell_type": "markdown", + "id": "40", + "metadata": {}, + "source": [ + "## 7. Retrieve results\n", + "### 7.1. Retrieve and visualize defect formation energy" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "41", + "metadata": {}, + "outputs": [], + "source": [ + "from mat3ra.notebooks_utils.ipython.entity.property.visualize import visualize_properties\n", + "\n", + "defect_energy_data = client.properties.get_for_job(defect_job_id)\n", + "visualize_properties(defect_energy_data, title=\"Defect Formation Energy\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "pygments_lexer": "ipython3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/other/materials_designer/workflows/dielectric_tensor.ipynb b/other/materials_designer/workflows/dielectric_tensor.ipynb index 1355b1be1..e5ef233cd 100644 --- a/other/materials_designer/workflows/dielectric_tensor.ipynb +++ b/other/materials_designer/workflows/dielectric_tensor.ipynb @@ -437,7 +437,7 @@ " unit.add_context(cutoffs_context)\n", " swf.set_unit(unit)\n", "\n", - "# epsilon.x energy grid -- this unit has no context provider, so it is patched directly\n", + "# epsilon.x energy grid\n", "patch_workflow_qe_input(\n", " workflow,\n", " {\n", @@ -631,12 +631,7 @@ "dielectric_tensor_data = get_properties_for_job(\n", " client, job_id, property_name=PropertyName.non_scalar.dielectric_tensor.value\n", ")\n", - "visualize_properties(dielectric_tensor_data, title=\"Dielectric Tensor\", extra_config={\"material\": material.to_dict()})\n", - "\n", - "# Sanity check: confirm real, non-trivial values were computed (not an empty/failed result)\n", - "_real_entry = next(v for v in dielectric_tensor_data[0][\"values\"] if v[\"part\"] == \"real\")\n", - "_eps_static = _real_entry[\"components\"][0][0]\n", - "print(f\"Dielectric tensor: eps1(0) = {_eps_static:.2f}\")" + "visualize_properties(dielectric_tensor_data, title=\"Dielectric Tensor\", extra_config={\"material\": material.to_dict()})" ] } ], diff --git a/other/materials_designer/workflows/interfacial_energy.ipynb b/other/materials_designer/workflows/interfacial_energy.ipynb index 18fa5c611..f265de0d7 100644 --- a/other/materials_designer/workflows/interfacial_energy.ipynb +++ b/other/materials_designer/workflows/interfacial_energy.ipynb @@ -120,7 +120,12 @@ "# Number of interfaces in the supercell: 1 if the slab has vacuum (one physical\n", "# interface), 2 for a periodic stack with no vacuum. Substrate and film\n", "# multiplicities are derived automatically from the structures.\n", - "N_INTERFACES = 1" + "N_INTERFACES = 1\n", + "\n", + "# Whose total_energy properties to consider for the substrate/film bulk\n", + "# references: \"public\" (any owner, highest precision wins), \"curators\" (only\n", + "# curators'), or \"my_account\" (curators' or your own).\n", + "BULK_TOTAL_ENERGY_SOURCE = \"my_account\"" ] }, { @@ -245,6 +250,7 @@ "if interface is None:\n", " interface_matches = client.materials.list({\n", " \"name\": {\"$regex\": re.escape(INTERFACE_NAME), \"$options\": \"i\"},\n", + " \"owner._id\": ACCOUNT_ID,\n", " })\n", " if not interface_matches:\n", " raise ValueError(\n", @@ -256,7 +262,7 @@ " print(f\"✅ Loaded interface from folder: {interface.name}\")\n", "\n", "visualize(interface, repetitions=[1, 1, 1])\n", - "visualize(interface, repetitions=[1, 1, 1], rotation=\"-90x\")\n" + "visualize(interface, repetitions=[1, 1, 1], rotation=\"-90x\")" ] }, { @@ -356,14 +362,18 @@ "from mat3ra.notebooks_utils.core.entity.property.api import find_total_energy_for_material\n", "\n", "substrate_bulk = get_bulk_material_by_crystal(client, CRYSTAL_SUBSTRATE, ACCOUNT_ID)\n", - "substrate_te_property = find_total_energy_for_material(client, substrate_bulk.id)\n", + "substrate_te_property = find_total_energy_for_material(\n", + " client, substrate_bulk.id, source=BULK_TOTAL_ENERGY_SOURCE\n", + ")\n", "if substrate_te_property is None:\n", " raise RuntimeError(\"Substrate bulk total energy not found. Run total_energy.ipynb for the substrate bulk first.\")\n", "print(f\"✅ Substrate bulk (material 1): {substrate_bulk.name} ({substrate_bulk.id}), \"\n", " f\"total energy: {substrate_te_property['data']['value']} eV\")\n", "\n", "film_bulk = get_bulk_material_by_crystal(client, CRYSTAL_FILM, ACCOUNT_ID)\n", - "film_te_property = find_total_energy_for_material(client, film_bulk.id)\n", + "film_te_property = find_total_energy_for_material(\n", + " client, film_bulk.id, source=BULK_TOTAL_ENERGY_SOURCE\n", + ")\n", "if film_te_property is None:\n", " raise RuntimeError(\"Film bulk total energy not found. Run total_energy.ipynb for the film bulk first.\")\n", "print(f\"✅ Film bulk (material 2): {film_bulk.name} ({film_bulk.id}), \"\n", @@ -614,14 +624,6 @@ "print(f\"Film bulk (material 2): {film_bulk.name} ({film_bulk.id}), \"\n", " f\"total energy: {film_te_property['data']['value']} eV\")" ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "43", - "metadata": {}, - "outputs": [], - "source": [] } ], "metadata": { diff --git a/other/materials_designer/workflows/phonon_dos_dispersion.ipynb b/other/materials_designer/workflows/phonon_dos_dispersion.ipynb index 47b96ebf7..ac2cc4210 100644 --- a/other/materials_designer/workflows/phonon_dos_dispersion.ipynb +++ b/other/materials_designer/workflows/phonon_dos_dispersion.ipynb @@ -635,11 +635,7 @@ "phonon_dispersions_data = get_properties_for_job(\n", " client, job_id, property_name=PropertyName.non_scalar.phonon_dispersions.value\n", ")\n", - "visualize_properties(phonon_dispersions_data, title=\"Phonon Dispersion\", extra_config={\"material\": material.to_dict()})\n", - "\n", - "# Sanity check: confirm real, non-trivial values were computed (not an empty/failed result)\n", - "_freqs = [f for band in phonon_dispersions_data[0][\"yDataSeries\"] for f in band]\n", - "print(f\"Phonon dispersion: {len(phonon_dispersions_data[0]['yDataSeries'])} bands, frequency range {min(_freqs):.1f} to {max(_freqs):.1f} cm^-1\")" + "visualize_properties(phonon_dispersions_data, title=\"Phonon Dispersion\", extra_config={\"material\": material.to_dict()})" ] }, { @@ -660,12 +656,7 @@ "phonon_dos_data = get_properties_for_job(\n", " client, job_id, property_name=PropertyName.non_scalar.phonon_dos.value\n", ")\n", - "visualize_properties(phonon_dos_data, title=\"Phonon DOS\", extra_config={\"material\": material.to_dict()})\n", - "\n", - "# Sanity check: confirm real, non-trivial values were computed (not an empty/failed result)\n", - "_dos_freqs = phonon_dos_data[0][\"xDataArray\"][0]\n", - "_dos_vals = phonon_dos_data[0][\"yDataSeries\"][0]\n", - "print(f\"Phonon DOS: frequency range {min(_dos_freqs):.1f} to {max(_dos_freqs):.1f} cm^-1, peak DOS {max(_dos_vals):.4f} states/cm^-1\")" + "visualize_properties(phonon_dos_data, title=\"Phonon DOS\", extra_config={\"material\": material.to_dict()})" ] } ], diff --git a/other/materials_designer/workflows/surface_energy.ipynb b/other/materials_designer/workflows/surface_energy.ipynb index cef5240bf..e98e13fc4 100644 --- a/other/materials_designer/workflows/surface_energy.ipynb +++ b/other/materials_designer/workflows/surface_energy.ipynb @@ -109,7 +109,12 @@ "outputs": [], "source": [ "# K-grid for slab SCF (if not set, KPPRA is used by default)\n", - "SCF_KGRID = None # e.g., [4, 4, 1]" + "SCF_KGRID = None # e.g., [4, 4, 1]\n", + "\n", + "# Whose total_energy properties to consider for the bulk reference: \"public\"\n", + "# (any owner, highest precision wins), \"curators\" (only curators'), or\n", + "# \"my_account\" (curators' or your own).\n", + "BULK_TOTAL_ENERGY_SOURCE = \"my_account\"" ] }, { @@ -235,6 +240,7 @@ "if slab is None:\n", " slab_matches = client.materials.list({\n", " \"name\": {\"$regex\": re.escape(SLAB_NAME), \"$options\": \"i\"},\n", + " \"owner._id\": ACCOUNT_ID,\n", " })\n", " if not slab_matches:\n", " raise ValueError(\n", @@ -294,7 +300,9 @@ "\n", "bulk_material = get_bulk_material(client, slab, ACCOUNT_ID)\n", "\n", - "bulk_total_energy_property = find_total_energy_for_material(client, bulk_material.id)\n", + "bulk_total_energy_property = find_total_energy_for_material(\n", + " client, bulk_material.id, source=BULK_TOTAL_ENERGY_SOURCE\n", + ")\n", "if bulk_total_energy_property is None:\n", " raise RuntimeError(\"Bulk total energy not found. Run total_energy.ipynb for the bulk material first.\")\n", "\n", diff --git a/pyproject.toml b/pyproject.toml index 493c5719d..9dd8433f4 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -71,8 +71,7 @@ tests = [ "pytest", "pytest-asyncio", "pytest-cov", - "mat3ra-wode", - "mat3ra-notebooks-utils[materials]", + "mat3ra-notebooks-utils[workflows]", ] docs = [ "mkdocs>=1.4.3", diff --git a/src/py/mat3ra/notebooks_utils/core/entity/property/api.py b/src/py/mat3ra/notebooks_utils/core/entity/property/api.py index e0189e0ac..f57f538ec 100644 --- a/src/py/mat3ra/notebooks_utils/core/entity/property/api.py +++ b/src/py/mat3ra/notebooks_utils/core/entity/property/api.py @@ -71,7 +71,7 @@ def update_property_holder_value(client: APIClient, property_holder_id: str, val return client.properties.update(property_holder_id, {"$set": {"data.value": value}}) -def find_total_energy_for_material(client: APIClient, material_id: str) -> Optional[dict]: +def find_total_energy_for_material(client: APIClient, material_id: str, source: str = "public") -> Optional[dict]: """ Find the best-precision total_energy property for a material. Mirrors the platform's "Resolve Total Energies for Elemental Materials" subworkflow, @@ -84,6 +84,7 @@ def find_total_energy_for_material(client: APIClient, material_id: str) -> Optio Args: client (APIClient): API client instance. material_id (str): Material _id to look up the total_energy property for. + source (str): Source of the total energy property: `public`, `my_account` or `curators`. Returns: The best-precision total_energy property, or None if none exists. @@ -92,11 +93,15 @@ def find_total_energy_for_material(client: APIClient, material_id: str) -> Optio exabyte_id = material.get("exabyteId") if not exabyte_id: return None + query = {"exabyteId": exabyte_id, "slug": "total_energy"} + if source == "curators": + query["owner.slug"] = "curators" + elif source == "my_account": + query["owner._id"] = client.my_account.id + elif source != "public": + raise ValueError(f"Invalid source: {source!r}. Expected 'public', 'curators', or 'my_account'.") properties = client.properties.list( - query={ - "exabyteId": exabyte_id, - "slug": "total_energy", - }, + query=query, projection={"sort": {"precision.value": -1}, "limit": 1}, ) return properties[0] if properties else None diff --git a/tests/py/unit/core/entity/test_property_api_find_total_energy_for_material.py b/tests/py/unit/core/entity/test_property_api_find_total_energy_for_material.py new file mode 100644 index 000000000..3389698db --- /dev/null +++ b/tests/py/unit/core/entity/test_property_api_find_total_energy_for_material.py @@ -0,0 +1,109 @@ +"""Unit tests for find_total_energy_for_material.""" + +from unittest.mock import MagicMock + +import pytest +from mat3ra.notebooks_utils.core.entity.property.api import find_total_energy_for_material + +MATERIAL_ID = "material-a" +EXABYTE_ID = "exabyte-a" +MATERIAL = {"_id": MATERIAL_ID, "exabyteId": EXABYTE_ID} +OWNER_ACCOUNT_ID = "account-a" +TOTAL_ENERGY_PROPERTY = {"data": {"value": -12.34}, "precision": {"value": 0.001}} + + +def _client(): + client = MagicMock() + client.materials.get.return_value = MATERIAL + client.properties.list.return_value = [TOTAL_ENERGY_PROPERTY] + client.my_account.id = OWNER_ACCOUNT_ID + # entity_cache is often None right after auth -- this must not be relied on. + client.my_account.entity_cache = None + return client + + +def test_find_total_energy_for_material_defaults_to_public_scope(): + client = _client() + + result = find_total_energy_for_material(client, MATERIAL_ID) + + client.materials.get.assert_called_once_with(MATERIAL_ID) + client.properties.list.assert_called_once_with( + query={ + "exabyteId": EXABYTE_ID, + "slug": "total_energy", + }, + projection={"sort": {"precision.value": -1}, "limit": 1}, + ) + client.jobs.list.assert_not_called() + assert result == TOTAL_ENERGY_PROPERTY + + +def test_find_total_energy_for_material_my_account_scope(): + client = _client() + + find_total_energy_for_material(client, MATERIAL_ID, source="my_account") + + client.properties.list.assert_called_once_with( + query={ + "exabyteId": EXABYTE_ID, + "slug": "total_energy", + "owner._id": OWNER_ACCOUNT_ID, + }, + projection={"sort": {"precision.value": -1}, "limit": 1}, + ) + + +def test_find_total_energy_for_material_curators_scope(): + client = _client() + + find_total_energy_for_material(client, MATERIAL_ID, source="curators") + + client.properties.list.assert_called_once_with( + query={ + "exabyteId": EXABYTE_ID, + "slug": "total_energy", + "owner.slug": "curators", + }, + projection={"sort": {"precision.value": -1}, "limit": 1}, + ) + + +def test_find_total_energy_for_material_public_scope_has_no_owner_filter(): + client = _client() + + find_total_energy_for_material(client, MATERIAL_ID, source="public") + + client.properties.list.assert_called_once_with( + query={ + "exabyteId": EXABYTE_ID, + "slug": "total_energy", + }, + projection={"sort": {"precision.value": -1}, "limit": 1}, + ) + + +def test_find_total_energy_for_material_rejects_invalid_source(): + client = _client() + + with pytest.raises(ValueError, match="Invalid source"): + find_total_energy_for_material(client, MATERIAL_ID, source="everyone") + + +def test_find_total_energy_for_material_returns_none_when_no_property_found(): + client = _client() + client.properties.list.return_value = [] + + result = find_total_energy_for_material(client, MATERIAL_ID) + + assert result is None + + +def test_find_total_energy_for_material_returns_none_when_material_has_no_exabyte_id(): + client = _client() + client.materials.get.return_value = {"_id": MATERIAL_ID} + + result = find_total_energy_for_material(client, MATERIAL_ID) + + client.properties.list.assert_not_called() + assert result is None