Add YAML-LD test suite & decide about YAML-LD implementation in pyld

.github/workflows/main.yaml

@@ -69,6 +69,7 @@ jobs:
         pytest tests/test_manifests.py --tests=./specifications/json-ld-api/tests --loader=${{ matrix.loader }}
         pytest tests/test_manifests.py --tests=./specifications/json-ld-framing/tests --loader=${{ matrix.loader }}
         pytest tests/test_manifests.py --tests=./specifications/normalization/tests --loader=${{ matrix.loader }}
+        pytest tests/test_manifests.py --tests=./specifications/yaml-ld/tests --loader=${{ matrix.loader }}
         pytest --ignore ./tests/test_manifests.py
       env:
         LOADER: ${{ matrix.loader }}

.gitignore

@@ -19,3 +19,6 @@ tests/data/test_caching.json
 
 # Local version file for pyenv
 .python-version
+
+# Local lock file for uv
+uv.lock

.gitmodules

@@ -7,3 +7,6 @@
 [submodule "specifications/normalization"]
 	path = specifications/normalization
 	url = https://github.com/json-ld/normalization.git
+[submodule "specifications/yaml-ld"]
+	path = specifications/yaml-ld
+	url = https://github.com/w3c/yaml-ld.git

README.md

@@ -331,13 +331,14 @@ git submodule update
 #### Cloning manually
 
 You can also avoid using git submodules by manually cloning the `json-ld-api`,
-`json-ld-framing`, and `normalization` repositories hosted on GitHub using the
-following commands:
+`json-ld-framing`, `normalization`, and `yaml-ld` repositories hosted on GitHub
+using the following commands:
 
 ```bash
 git clone https://github.com/w3c/json-ld-api ./specifications/json-ld-api
 git clone https://github.com/w3c/json-ld-framing ./specifications/json-ld-framing
 git clone https://github.com/json-ld/normalization ./specifications/normalization
+git clone https://github.com/w3c/yaml-ld ./specifications/yaml-ld
 ```
 
 Note that you can clone these repositories into any location you wish; however,

docs/project/decisions/choose-where-to-host-yaml-ld-support.md

@@ -0,0 +1,81 @@
+---
+title: Treat YAML-LD support as …
+status: undecided
+date: 2026-07-19
+author: Anatoly Scherbakov
+tags: [decision]
+hide: [toc]
+---
+
+# Treat YAML-LD support as …
+
+{{ adr_metadata(date, status) }}
+
+## :material-text-box-outline: Context
+
+At the moment of writing this document, the [JSON-LD Working Group](https://www.w3.org/groups/wg/json-ld/) is preparing a new specification, [YAML-LD](https://www.w3.org/TR/yaml-ld-10/), for the status of [Recommendation](https://www.w3.org/standards/types/#x2-5-recommendation). It [leverages the readability and conciseness of YAML, but relies on the JSON-LD data model and API](https://www.w3.org/TR/yaml-ld-10/). Here is an example:
+
+{{ source_file('project/decisions/examples/intro.yamlld') }}
+
+The specification is already stable in the normative sense, and the Working Group is inviting implementations for the new standard.
+
+Should we provide such an implementation for Python within [:fontawesome-brands-github: `digitalbazaar/pyld`](https://github.com/digitalbazaar/pyld)?
+
+## :material-arrow-decision-outline: Decision
+
+<table markdown="1">
+  <tr markdown="span">
+    <th></th>
+    <th class="adr-col-undecided">PyLD implements YAML-LD</th>
+    <th class="adr-col-undecided">`python-yaml-ld` implements YAML-LD</th>
+    <th class="adr-col-rejected">PyLD is a façade on top of `python-yaml-ld`</th>
+  </tr>
+  <tr markdown="span">
+    <th>Install</th>
+    <td class="adr-col-undecided">`pip install 'PyLD[yaml-ld]'`</td>
+    <td class="adr-col-undecided">`pip install yaml-ld`</td>
+    <td class="adr-col-rejected">`pip install 'PyLD[yaml-ld]'`</td>
+  </tr>
+  <tr markdown="span">
+    <th>Dependencies</th>
+    <td class="adr-col-undecided">YAML parser, for instance: `ruamel.yaml` or `pyyaml`</td>
+    <td class="adr-col-undecided">[PyLD](https://github.com/iolanta-tech/python-yaml-ld/blob/master/pyproject.toml)</td>
+    <td class="adr-col-rejected">`PyLD` → `yaml-ld` → `PyLD`</td>
+  </tr>
+  <tr markdown="span">
+    <th>Test suite</th>
+    <td class="adr-col-undecided">PyLD (`specifications/yaml-ld`)</td>
+    <td class="adr-col-undecided">`yaml-ld`</td>
+    <td class="adr-col-rejected">`yaml-ld`</td>
+  </tr>
+  <tr markdown="span">
+    <th>YAML parsing</th>
+    <td class="adr-col-undecided">`PyLD`</td>
+    <td class="adr-col-undecided">`yaml-ld`</td>
+    <td class="adr-col-rejected">`yaml-ld`</td>
+  </tr>
+  <tr markdown="span">
+    <th>How to expand()</th>
+    <td class="adr-col-undecided">`pyld.jsonld.expand()`</td>
+    <td class="adr-col-undecided">`yaml_ld.expand()`</td>
+    <td class="adr-col-rejected">`pyld.jsonld.expand()`</td>
+  </tr>
+  <tr markdown="span">
+    <th>Decision</th>
+    <td class="adr-col-undecided">:question:</td>
+    <td class="adr-col-undecided">:question:</td>
+    <td class="adr-col-rejected">:x: Circular dependency (`PyLD` → `yaml-ld` → `PyLD`)</td>
+  </tr>
+</table>
+
+## :material-arrow-right-bold-outline: Consequences
+
+- Docs, optional extras, and ownership of YAML-LD conformance must match the chosen packaging home.
+- Dependency surface (YAML parser and any optional extras) will either grow in PyLD or stay in a separate package.
+- Release cadence for YAML-LD support fixes will either couple to PyLD releases or remain with a separate package.
+
+#### Implementation Steps
+
+- [ ] Record the Decision row outcome in this ADR (flip status to decided)
+- [ ] Align suite location, YAML parsing ownership, `expand()` entry point, and install surface with the chosen column
+- [ ] Update project index / user docs if needed

docs/project/decisions/examples/intro.yamlld

@@ -0,0 +1,19 @@
+"@context":
+  - https://json-ld.org/contexts/dollar-convenience.jsonld
+  - schema: https://schema.org/
+    dbo: http://dbpedia.org/ontology/
+    dbp: http://dbpedia.org/property/
+    dbr: http://dbpedia.org/resource/
+    xsd: http://www.w3.org/2001/XMLSchema#
+    dbp:discovered:
+      "@type": xsd:date
+    dbp:star:
+      "@type": "@id"
+
+$id: dbr:Proxima_Centauri_b
+$type: dbo:Planet
+schema:description: >-
+  The closest known exoplanet to Earth,
+  orbiting in Proxima Centauri's habitable zone.
+dbp:discovered: 2016-08-24
+dbp:star: dbr:Proxima_Centauri

docs/project/index.md

@@ -12,6 +12,12 @@ Architecture Decision Records (ADRs) document the technical choices taken during
 
 <div id="adr-index" markdown>
 
+!!! warning inline "[Treat YAML-LD support as …](decisions/choose-where-to-host-yaml-ld-support/)"
+    :material-calendar-clock: 19 July 2026
+
+    Bundle into PyLD, keep a separate package (with suite under `specifications/`),
+    expose a thin PyLD facade, or treat YAML-LD as out of scope (no suite either).
+
 !!! success inline "[Use `requests-cache` for persistent HTTP caching in synchronous Python code](decisions/use-requests-cache-for-sync-http-caching-in-document-loaders/)"
     :material-calendar-clock: 29 June 2026
 

docs_macros.py

@@ -11,34 +11,53 @@
 from yaml import SafeLoader
 
 ROOT_DIR = Path(__file__).resolve().parent
-EXAMPLES_DIR = ROOT_DIR / 'docs' / 'examples'
+DOCS_DIR = ROOT_DIR / 'docs'
+EXAMPLES_DIR = DOCS_DIR / 'examples'
 sys.path.insert(0, str(ROOT_DIR / 'lib'))
 sys.path.insert(0, str(ROOT_DIR / 'tests'))
 
+_SOURCE_LANG = {
+    '.yamlld': 'yaml',
+    '.yaml': 'yaml',
+    '.yml': 'yaml',
+    '.jsonld': 'json',
+    '.json': 'json',
+    '.py': 'python',
+}
+
 MANIFEST_BASES = {
     'frame-manifest': 'https://w3c.github.io/json-ld-framing/tests',
     'manifest-urgna2012': 'https://w3c.github.io/rdf-canon/tests',
     'manifest-urdna2015': 'https://w3c.github.io/rdf-canon/tests',
+    'manifest': 'https://w3c.github.io/yaml-ld/tests',
 }
 DEFAULT_TEST_BASE = 'https://w3c.github.io/json-ld-api/tests'
 
 _SKIP_ID_PATTERN = re.compile(r'^\.\*(?P<manifest>[^#]+)#(?P<test_id>[^$]+)\$$')
 _MANIFEST_PATHS = (
     ROOT_DIR / 'specifications' / 'json-ld-api' / 'tests',
     ROOT_DIR / 'specifications' / 'json-ld-framing' / 'tests',
+    ROOT_DIR / 'specifications' / 'yaml-ld' / 'tests',
 )
 
 
 def _parse_skip_id_regex(pattern):
     match = _SKIP_ID_PATTERN.fullmatch(pattern)
     if not match:
         return None
-    return match.group('manifest'), match.group('test_id')
+    # Suite-scoped patterns may include a path prefix, e.g.
+    # .*yaml-ld/tests/manifest#two-documents-from-stream$
+    manifest = match.group('manifest')
+    if '/' in manifest:
+        manifest = manifest.rsplit('/', 1)[-1]
+    return manifest, match.group('test_id')
 
 
 def _test_url(manifest, test_id):
-    base = MANIFEST_BASES.get(manifest, DEFAULT_TEST_BASE)
-    return f'{base}/{manifest}.html#{test_id}'
+    # Basename only — suite-scoped regex captures may still contain '/'.
+    manifest_name = manifest.rsplit('/', 1)[-1]
+    base = MANIFEST_BASES.get(manifest_name, DEFAULT_TEST_BASE)
+    return f'{base}/{manifest_name}.html#{test_id}'
 
 
 def _jsonld_values(data, key):
@@ -59,7 +78,11 @@ def _manifest_entries():
     for manifest_dir in _MANIFEST_PATHS:
         if not manifest_dir.exists():
             continue
-        for path in sorted(manifest_dir.glob('*-manifest.jsonld')):
+        paths = {
+            *manifest_dir.glob('*-manifest.jsonld'),
+            *manifest_dir.glob('manifest.jsonld'),
+        }
+        for path in sorted(paths):
             data = json.loads(path.read_text())
             manifest = path.stem
             for entry in _jsonld_values(data, 'sequence'):
@@ -82,6 +105,11 @@ def _skip_reason(test_type, skip, test):
     for pattern in skip.get('idRegex', []):
         if re.match(pattern, test_id):
             return f'Explicit skip (`{test_type}`)'
+        # Suite-scoped patterns (e.g. .*yaml-ld/tests/manifest#id$) still
+        # refer to the same short manifest#id used in local entries.
+        parsed = _parse_skip_id_regex(pattern)
+        if parsed and f'{parsed[0]}#{parsed[1]}' == test_id:
+            return f'Explicit skip (`{test_type}`)'
 
     for pattern in skip.get('descriptionRegex', []):
         if re.match(pattern, entry.get('description', '')):
@@ -114,6 +142,13 @@ def _example_path(name):
     return path
 
 
+def _docs_path(rel):
+    path = (DOCS_DIR / rel).resolve()
+    if not path.is_relative_to(DOCS_DIR.resolve()):
+        raise ValueError(f'Invalid docs path: {rel}')
+    return path
+
+
 def _github_branch():
     branch = os.environ.get('GITHUB_REF_NAME')
     if branch:
@@ -129,10 +164,20 @@ def _github_branch():
     return 'master'
 
 
-def _example_github_url(name, repo_url):
-    rel_path = Path('docs/examples') / name
+def _docs_github_url(rel, repo_url):
     branch = 'master'
-    return f'{repo_url.rstrip("/")}/blob/{branch}/{rel_path.as_posix()}'
+    return f'{repo_url.rstrip("/")}/blob/{branch}/docs/{Path(rel).as_posix()}'
+
+
+def _example_github_url(name, repo_url):
+    return _docs_github_url(Path('examples') / name, repo_url)
+
+
+def _example_admonition(title, body, indent=0):
+    content_indent = indent + 4
+    pad = ' ' * content_indent
+    indented = '\n'.join(f'{pad}{line}' for line in body.splitlines())
+    return f'!!! example "{title}"\n\n{indented}\n'
 
 
 def _human_date(value):
@@ -319,7 +364,19 @@ def example(name, output_syntax=None, indent=0):
             f'```python\n{source}```\n\n'
             f'```{output_lang} title="Output"\n{result.stdout}```'
         )
-        content_indent = indent + 4
-        pad = ' ' * content_indent
-        indented = '\n'.join(f'{pad}{line}' for line in body.splitlines())
-        return f'!!! example "{title}"\n\n{indented}\n'
+        return _example_admonition(title, body, indent=indent)
+
+    @env.macro
+    def source_file(rel, indent=0):
+        """Include a docs file in an example admonition with a GitHub source link."""
+        path = _docs_path(rel)
+        source = path.read_text()
+        github_url = _docs_github_url(rel, env.conf['repo_url'])
+        lang = _SOURCE_LANG.get(path.suffix.lower(), 'text')
+        title = (
+            f'Example<span class="example-source-link" markdown>'
+            f':fontawesome-brands-github: [`{path.name}`]({github_url})'
+            f'</span>'
+        )
+        body = f'```{lang}\n{source}```'
+        return _example_admonition(title, body, indent=indent)

mkdocs.yml

@@ -34,6 +34,10 @@ markdown_extensions:
       emoji_generator: !!python/name:material.extensions.emoji.to_svg
   - pymdownx.highlight:
       anchor_linenums: true
+  - pymdownx.snippets:
+      base_path:
+        - docs
+      check_paths: true
   - pymdownx.superfences:
       custom_fences:
         - name: mermaid