Start by forking this main repository that will house all your plugins.

NOMAD Dev Distribution

Benefits

• One-step installations: Install everything at once with editable mode. Since all packages are installed in editable mode, changes you make to the code are immediately reflected. Edit your code and rerun tests or the application as needed, without needing to reinstall the packages.
• Centralized codebase: Easier navigation and searching across projects.
• Better editor support: Improved autocompletion and refactoring.
• Consistent tooling: Shared linting, testing, and formatting.
• Flexible plugin management: If you're developing plugins for different deployments with varying requirements, you can easily create different branches for each deployment and configure which plugins to use in the specific branch for that deployment.

Below are instructions for how to create a dev environment for developing nomad-lab and its plugins.

Basic infra

1. Ensure you have docker installed. Docker nowadays comes with docker compose built in. Prior, you need to install the stand-alone docker-compose.

2. Install uv (v0.5.14 and above). uv is required to manage your development environment. It's recommended to use the standalone installer or perform a global installation. (brew install uv on macOS or dnf install uv on Fedora).

3. For Windows users, nomad-distro-dev works natively on Windows. However, not all NOMAD plugins are tested on Windows, so there might be some bugs. If you face any issues, please file an issue in the respective plugin repository.

4. Clone the forked repository.

   git clone https://github.com/<your-username>/nomad-distro-dev.git
   cd nomad-distro-dev

5. Run the docker containers with docker compose in detached (--detach or -d) mode.

   docker compose up -d

   To shutdown the containers:

   docker compose down

Developing nomad + plugins locally.

This guide explains how to set up a streamlined development environment for nomad-lab and its plugins using uv workspaces. This approach eliminates the need for multiple pip install commands by leveraging a monorepo and a single installation step.

In this example, we'll set up the development environment for a developer working on the following plugins: nomad-parser-plugins-electronic and nomad-measurements. The first plugin already comes as a dependency in this dev distribution. On the contrary, the second plugin is not listed as a dependency. In the following, we take a look at how to setup the environment in these two situations.

Step-by-Step Setup

1. Update submodules

   This loads the nomad-lab package which is already listed as a submodule.

   git submodule update --init --recursive

   [!TIP]

   To get more information on how git submodules are used to structure bigger software projects, read this GitHub blog entry on this topic.

2. Add local plugins

   Assuming that you already have a git repo for your plugins, add them to the packages/ directory as submodules. In case, you are looking to create a plugin repo from scratch, consider using our plugin template: nomad-plugin-template.

   git submodule add https://github.com/<package_name>.git packages/<package_name>

   Repeat for all the plugin packages you want to add and develop. For our example:

   git submodule add https://github.com/nomad-coe/electronic-parsers.git packages/nomad-parser-plugins-electronic
   
   git submodule add https://github.com/FAIRmat-NFDI/nomad-measurements.git packages/nomad-measurements

3. Modify pyproject.toml

   To ensure uv recognizes the local plugins (a local copy of your plugin repository available in packages/ directory), we need to make some modifications in the pyproject.toml. These include adding the plugin package to [project.dependencies] and [tool.uv.sources] tables. The packages listed under [tool.uv.sources] are loaded by uv using the local code directory made available under packages/ with the previous step. This list will contain all the plugins that we need to actively develop in this environment.

   If a new plugin is not listed under [project.dependencies], we need to first add it as a dependency. After adding the dependencies, update the [tool.uv.sources] section in your pyproject.toml file to reflect the new plugins.

   There are two ways of adding to these two lists:

   a) You can use uv add which adds the dependency and the source in pyproject.toml and sets up the environment. Adding multiple plugins should be done in a single command:

   uv add packages/nomad-measurements packages/PLUGIN_B packages/PLUGIN_C

   In this example, we're just adding one:

   uv add packages/nomad-measurements

   b) You can modify the pyproject.toml file manually:

   [project]
   dependencies = [
   ...
   "nomad-measurements",
   ]
   
   [tool.uv.sources]
   ...
   nomad-measurements = { workspace = true }

   Some of the plugins are already listed under [project.dependencies]. If you want to develop one of them, you have to add them under [tool.uv.sources]. We do this for nomad-parser-plugins-electronics.

   [tool.uv.sources]
   ...
   nomad-parser-plugins-electronic = { workspace = true }

Advanced: Dependency Issues

• Overriding a dependent's version pin. When adding a local project, another package in the tree may pin one of its dependencies to a released version. That pin fights the local workspace source, so uv won't use your local copy and resolution fails. Add an unconstrained override so uv drops every dependent's pin and falls back to the local checkout:

  [tool.uv.sources]
  nomad-lab = { workspace = true }   # already present — points at packages/nomad-FAIR
  
  [tool.uv]
  override-dependencies = ["nomad-lab"]

  Read more about override-dependencies in the uv reference guide. Here are some trade-offs to be aware of when using overrides:

  • It drops the version specifier from every dependent — direct and transitive. A package that pinned nomad-lab>=X for an API added in X silently resolves to whatever the local checkout provides, so functionality can break in subtle ways.
  • uv also stops flagging version conflicts and never reports which pins it overrode, so incompatibilities surface later as runtime errors with no obvious cause.

  Prefer a constrained override (override-dependencies = ["nomad-lab==<local-version>"]) when you can name the local version, so incompatible consumers still fail loudly. Use overrides sparingly.

Note

You can also use uv to install a specific branch of the plugin without adding a submodule locally.

uv add https://github.com/FAIRmat-NFDI/nomad-measurements.git --branch <specific-branch-name>

This command will not include the plugin in the packages/ folder, and hence this plugin will not be editable.

A complete list of plugins maintained by FAIRmat-NFDI can by found in the overview page of the FAIRmat-NFDI organisation.

Day-to-Day Development

After the initial setup, here’s how to manage your daily development tasks.

Tip

You can use uv tool install poethepoet to install it as a CLI tool, allowing you to run commands directly with poe ... (e.g., poe start instead of uv run poe start).

1. Update the environment (this installs the necessary dependencies, including the new GUI):

   uv run poe setup

   As part of the setup command, a nomad.yaml config file will be created. This file is used to configure NOMAD and will be placed in the top-level directory of your repository, where all commands are executed from. The new GUI is still in beta and requires neither Node.js nor a separate setup command.

   For more information on configuration options, refer to the detailed nomad configuration docs.

   [!NOTE]

   uv sync and uv run automatically manages the virtual environment for you. There's no need to manually create or activate a venv. Any uv run commands will automatically use the correct environment by default. Read more about uv commands to manage the dependencies here.

2. Run the NOMAD API app and new GUI (equivalent to running uv run nomad admin run appworker --dev):

   uv run poe start

3. [Optional] Run the legacy GUI

   The legacy GUI requires Node.js v20 and Yarn v1.22. Set it up and start it in a separate terminal:

   uv run poe gui-setup
   uv run poe gui start

   [!TIP]

   uv run poe gui maps to yarn run, so you can replace start with commands such as test or build.

4. [Optional] Run the docs server (only if you wish to run the documentation server):

   Add the nomad-docs repository as a submodule (if you have added it as a submodule already, skip this step):

   git submodule add https://github.com/FAIRmat-NFDI/nomad-docs.git packages/nomad-docs

   Just like adding a new plugin, use uv to add nomad-docs to the environment:

   uv add packages/nomad-docs

   At this moment, you can commit the changes made to your nomad-dev-distro.

   Every time you want to start the docs server, run:

   uv run poe docs

5. [Optional] Run the remote tools hub server (only if you wish to use the remote tools hub):

   uv run poe hub

6. Running tests

   To run tests across the project, use the uv run command to execute pytest in the relevant directory. For instance:

   uv run --directory packages/package_name pytest

   This allows you to run tests for a specific parser or package. For running tests across all packages, simply repeat the command for each directory.

   [!TIP]

   To run tests for a specific package in an isolated venv use: uv run --exact --all-extras --package plugin_a --directory packages/plugin_a pytest

7. Linting & code formatting

   To check for linting issues using ruff, run the following command:

   uv run poe lint

   You can invoke ruff separately using uv run ruff too.

8. Adding new plugins

   To add a new package, follow setup guide and add it into the packages/ directory and ensure it's listed in pyproject.toml under [tool.uv.sources]. Then, install it by running:

   uv sync

9. Removing an existing plugin

   To remove an existing plugin from the workspace in packages/ directory, do the following and commit:

   git rm <path-to-submodule>

   Then you can remove the plugin from [tool.uv.sources] in pyproject.toml to stop uv from using the local plugin repository.

   Additionally, if you want to remove the plugin from being a dependency of your NOMAD installation, you can use uv to entirely remove it:

   uv remove <plugin-name>

10. Modifying dependencies in packages.

    uv add --package <PACKAGE_NAME> <DEPENDENCY_NAME>

    For example:

    uv add --package nomad-measurements "pandas>=2.0"
    uv remove --package nomad-lab numpy

11. Keeping Up-to-Date

    To pull updates from the main repository and submodules, run:

    git pull --recurse-submodules

    Afterward, sync your environment:

    uv sync

Note

The new GUI is served by the NOMAD API at http://localhost:8000/nomad-oasis/gui/v2/. The legacy GUI started with uv run poe gui start is available on http://localhost:3000/nomad-oasis/gui and expects to find the NOMAD API on localhost:8000 and the remote tools hub on localhost:9000. If you are running the instance on a remote server, make sure to forward the relevant ports locally. As an alternative to port forwarding for the legacy GUI, configure its backend URL by changing REACT_APP_BACKEND_URL in packages/nomad-FAIR/gui/.env.development.

Updating the fork

To keep your fork up to date with the latest changes from the original repository (upstream), follow these steps:

1. Add the upstream Remote

   If you haven't already, add the original repository as upstream:

   git remote add upstream https://github.com/FAIRmat-NFDI/nomad-distro-dev.git

2. Fetch the Latest Changes from upstream

   git fetch upstream

3. Merge upstream/main into Your Local Branch

   Switch to the local branch (e.g., main) you want to update, and merge the changes from upstream/main:

   git checkout main
   git merge upstream/main

   Resolve any merge conflicts if necessary, and commit the merge.

4. Push the Updates to Your Fork

   After merging, push the updated branch to your fork on GitHub:

   git push origin main

Common Issues and Solutions

1. Failed to install phonopy.

      uv venv -p 3.12
      uv pip install 'numpy>=1.25'
      uv pip install 'phonopy==2.11.0' --no-build-isolation

2. Failed to install pycifrw.

   The error usually indicates that clang was missing. error: command 'clang'. Installing clang should fix this issue.