Add a Gaussian Splat and octomap rendering sample by gpx1000 · Pull Request #1247 · KhronosGroup/Vulkan-Samples

gpx1000 · 2024-12-14T05:22:04Z

Description

This is a sample showing a complex interaction with other libraries to demonstrate how to work with libraries and tools int he wild. Here Octomap is used to demonstrate Occupancy grid rendering. We also demonstrate how to use ImGUI in a project with added functionality and positioning of rendered assets.

General Checklist:

Please ensure the following points are checked:

Note: The Samples CI runs a number of checks including:

I have updated the header Copyright to reflect the current year (CI build will fail if Copyright is out of date)
My changes build on Windows, Linux, macOS and Android. Otherwise I have documented any exceptions

If this PR contains framework changes:

I did a full batch run using the batch command line argument to make sure all samples still work properly

Sample Checklist

If your PR contains a new or modified sample, these further checks must be carried out in addition to the General Checklist:

I have tested the sample on at least one compliant Vulkan implementation
If the sample is vendor-specific, I have tagged it appropriately
I have stated on what implementation the sample has been tested so that others can test on different implementations and platforms
Any dependent assets have been merged and published in downstream modules
For new samples, I have added a paragraph with a summary to the appropriate chapter in the readme of the folder that the sample belongs to e.g. api samples readme
For new samples, I have added a tutorial README.md file to guide users through what they need to know to implement code using this feature. For example, see conditional_rendering
For new samples, I have added a link to the Antora navigation so that the sample will be listed at the Vulkan documentation site

… Voxel Point Cloud map using Octomap. Also demonstrates how to use ImGUI in a project with added functionality and positioning of rendered assets.

SaschaWillems · 2024-12-16T19:25:03Z

Wanted to give this a first try, but no matter what I tried, I wasn't able to get the newly added OctoMap submodule to checkout. Any ideas? Tried to clean reset, did a recursive checkout of all submodules, etc.

gary-sweet · 2025-01-13T10:46:25Z

Wanted to give this a first try, but no matter what I tried, I wasn't able to get the newly added OctoMap submodule to checkout. Any ideas? Tried to clean reset, did a recursive checkout of all submodules, etc.

Same problem for me it looks like.

Extends the octomap sample with three view modes: Octomap, GLTF Regular, and GLTF Splats. Implements splat rendering pipeline with per-instance attributes and premultiplied alpha blending. Adds GLTF mesh rendering with vertex color support. Includes new shader files for splat geometry and fragment evaluation. Updates camera to FirstPerson type for WASD movement. Fixes cube winding order and improves command buffer recording. Adds proper license headers to all new and modified files.

Standardizes code formatting with consistent indentation and brace style. Reorders includes alphabetically with core utilities first. Consolidates constructor initializer list to single line. Adjusts spacing and alignment throughout for consistency.

Ensures submodules are fully initialized and updated in each job by adding explicit `git submodule update --init --recursive` command after checkout, complementing the existing `submodules: "recursive"` checkout option.

Replaces spaces with tabs to maintain consistent formatting throughout the .gitmodules file.

Adds update_overlay method override that delegates to custom ImGUI implementation in render() function and executes additional UI callbacks when provided.

Renames header guards in ImGUIUtil.h from ORB_SLAM3_IMGUI_H to IMGUI_UTIL_H to match the actual file name. Excludes samples/complex/render_octomap/octomap directory from CI workflow file checks.

… with static_cast Improves code style consistency by adding braces to all single-statement if blocks. Replaces C-style casts with appropriate static_cast and reinterpret_cast. Filters out third_party/glfw warnings from clang-tidy output in CI workflow.

Replaces leading spaces with tabs to maintain consistent formatting throughout the file.

SaschaWillems · 2026-01-01T21:30:31Z

This does compile for me now (after explicitly updating submodules), but crashes at startup with the following error:

"ERROR: Filestream to assets/scenes/octmap_and_splats/octMap.bin not open, nothing read."

I do see octMap.bin directly below the assets/scene folder.

gpx1000 · 2026-01-02T00:23:24Z

Yeah, I was holding off on uploading the map until I took a good one that includes the splats. Will upload as soon as I can go to the office and capture it

…

On Thu, Jan 1, 2026, 1:30 PM Sascha Willems ***@***.***> wrote: *SaschaWillems* left a comment (KhronosGroup/Vulkan-Samples#1247) <#1247 (comment)> This does compile for me now (after explicitly updating submodules), but crashes at startup with the following error: "ERROR: Filestream to assets/scenes/octmap_and_splats/octMap.bin not open, nothing read." I do see octMap.bin directly below the assets/scene folder. — Reply to this email directly, view it on GitHub <#1247 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA5IAY2XXH22KNPXLRU5TK34EWGY3AVCNFSM6AAAAABTTGIWPOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTOMBUGEZDQMRWGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

gary-sweet · 2026-01-07T11:16:05Z

Not supported on direct-to-display platforms. Doesn't even build the sample.

Replaces GLFW-specific key handling with framework's KeyCode enum for platform-agnostic input. Renames handleKey to handle_key_event to match framework naming conventions. Updates modifier key state tracking to use ImGui's key state. Removes GLFW dependency and platform-specific conditionals from ImGUI implementation. Enables render_octomap sample on all platforms including Android, iOS, and Direct-to-Display.

Updates copyright headers across framework components and shaders from 2025 to 2026.

Updated copyright headers across various project files to reflect the year 2026. This ensures consistency and compliance with licensing requirements.

gpx1000 · 2026-01-08T06:35:29Z

Give it another try Gary, I removed the input handling by GLFW assumption used during development. Although, I still haven't had the time to generate a new map to go along with this; it should build for everyone on all devices and then run until you get to Sascha's problem of not having the map yet.

gary-sweet · 2026-01-08T08:32:58Z

Give it another try Gary, I removed the input handling by GLFW assumption used during development. Although, I still haven't had the time to generate a new map to go along with this; it should build for everyone on all devices and then run until you get to Sascha's problem of not having the map yet.

Confirmed. This now gets to the same point for me.

# Conflicts: # .gitmodules # framework/CMakeLists.txt

…ting spec Moves Gaussian splatting attributes from extension object to primitive attributes using KHR_gaussian_splatting namespace. Updates attribute parsing to check KHR_gaussian_splatting:ROTATION, SCALE, OPACITY, and SH_DEGREE_0_COEF_0 in primitive attributes first, with fallback to old draft extension properties and COLOR_0. Simplifies extension parsing logic and removes nested property structures. Updates code examples in tutorial documentation to reflect new attribute structure.

gary-sweet · 2026-06-01T08:40:18Z

Are these errors expected:

ERROR: Filestream to assets/scenes/octmap_and_splats/octMap.bin not open, nothing read.
[error] Error Message: Failed to open file for reading at path: shaders/render_octomap/glsl/imgui.vert.spv
[error] Failed when running application Octmap rendering

gary-sweet · 2026-06-10T08:48:03Z

Here's what I get when I run this:

Reading binary octree type OcTree
[error] Error Message: Failed to open file for reading at path: shaders/render_octomap/glsl/imgui.vert.spv
[error] Failed when running application Octmap rendering
[error] -1363947098 - VUID-vkDestroyPipeline-pipeline-parameter: vkDestroyPipeline(): pipeline Invalid VkPipeline Object 0x3f401698bf866666.
The Vulkan spec states: If pipeline is not VK_NULL_HANDLE, pipeline must be a valid VkPipeline handle (https://docs.vulkan.org/spec/latest/chapters/pipelines.html#VUID-vkDestroyPipeline-pipeline-parameter)
vulkan_samples: /build/gsweet/vc5_git/thirdparty_vulkan/Vulkan-ValidationLayers/layers/./thread_tracker/thread_safety_validation.h:122: std::shared_ptr<threadsafety::ObjectUseData> threadsafety::Counter<T>::FindObject(T, const Location&) [with T = VkPipeline_T*]: Assertion `object_table.contains(object)' failed.

It looks like there are no .spv files in the shaders/render_octomap/glsl folder at all currently.

gary-sweet · 2026-06-10T09:17:43Z

If I build the spv files manually it does run, but I'm not sure it looks right.

My normal method of driving the UI (using the keyboard) doesn't work either so I can't actually do anything other than show the initial result.

This is what I see:

asuessenbach

I don't get any octomap asset files.
When I checkout this PR, switch to branch 'complex-octomap-rendering', and update the asset submodule, everything seems to run fine, but there is no octmap_and_splats in assets/scenes. Any idea how to get that?

asuessenbach · 2026-06-10T08:11:50Z

+			continue;
+		std::string name = entry.path().filename().string();
+		if (name.find("_cell_") != std::string::npos && name.size() >= 5 &&
+		    name.substr(name.size() - 5) == ".gltf")


Maybe name.ends_with( ".gltf" )?

ends_with is C++20. I don't know if we can use it. I think we have a max upper bound on C++ that we can use and I don't remember off the top of my head if it was above 20 or not.

We upped that to C++20 several months ago for all platforms, so should be good.

I thought we had to downgrade it again as there was a max version of gcc that we were allowed to use as some couldn't upgrade their developer systems.

gpx1000 · 2026-06-10T09:19:34Z

If I build the spv files manually it does run, but I'm not sure it looks right.

My normal method of driving the UI (using the keyboard) doesn't work either so I can't actually do anything other than show the initial result.

This is what I see:

Yep, that worked. WASD and the mouse will let you move around. That's the voxel view of the occupancy grid. Click on the other two buttons to see other views of the same map.

gpx1000 · 2026-06-10T09:21:37Z

I don't get any octomap asset files. When I checkout this PR, switch to branch 'complex-octomap-rendering', and update the asset submodule, everything seems to run fine, but there is no octmap_and_splats in assets/scenes. Any idea how to get that?

It should just be in assets/scenes/octmap_and_splats There should be 21 files in there. Here's the assets commit if you want to grab it directly: KhronosGroup/Vulkan-Samples-Assets@8db8ce9...6900e07

gary-sweet · 2026-06-10T09:22:25Z

Yep, that worked. WASD and the mouse will let you move around. That's the voxel view of the occupancy grid. Click on the other two buttons to see other views of the same map.

But I can't click on the buttons (we don't have a mouse). Normally the arrow keys move around in the UI, but not in this sample it seems.

gpx1000 · 2026-06-10T09:25:47Z

But I can't click on the buttons (we don't have a mouse). Normally the arrow keys move around in the UI, but not in this sample it seems.

Okay, gimme a bit and there'll be a new version that lets you move around arrow keys as normal. The complex sample I original designed to use less of the framework so it doesn't inherit the input. Easy fix working on it and the shader compiler now.

… in built spv files so platforms that oddly don't auto build will work.

gpx1000 · 2026-06-10T10:10:56Z

Yep, that worked. WASD and the mouse will let you move around. That's the voxel view of the occupancy grid. Click on the other two buttons to see other views of the same map.

But I can't click on the buttons (we don't have a mouse). Normally the arrow keys move around in the UI, but not in this sample it seems.

This version should allow you to do keyboard nav in imgui.

gary-sweet · 2026-06-10T10:30:32Z

This version should allow you to do keyboard nav in imgui.

Sadly not:

Reading binary octree type OcTree
    01:46:01.620 nexus_display_hlm: usage 6 enabled
vulkan_samples: /build/gsweet/vc5_git/thirdparty_vulkan/Vulkan-Samples/third_party/imgui/imgui.cpp:1488: void ImGuiIO::AddKeyAnalogEvent(ImGuiKey, bool, float): Assertion `KeyMap[n] == -1 && "Backend needs to either only use io.AddKeyEvent(), either only fill legacy io.KeysDown[] + io.KeyMap[]. Not both!"' failed.
Aborted

SaschaWillems · 2026-06-10T11:04:53Z

Sample does run for me, but I'm seeing a similar issue to the one Gary reported. As soon as I press a key (e.g. "S") I get an assertion:

Also unsure what it's supposed to look like. Can you post/add some reference screenshots?

gpx1000 · 2026-06-10T11:09:53Z

This is a temp map while I make a new one that hopefully looks nicer. The gaussian splats are from a single live run so they don't look that detailed. They are accurate. The octomap renders are as described in the text, no textures, just points in an occupancy grid. So those are by design false colors.

SaschaWillems · 2026-06-10T11:15:41Z

Could documentation be expanded at least a bit? The code has very few comments and the readme is mostly talking about the topic and not the sample itself.

…es from what we do in GLTF_Loader in the framework, but it's a good idea).

gary-sweet · 2026-06-10T11:25:01Z

The UI traversal is now working, thanks.

…g modes, and key implementation details.

gpx1000 · 2026-06-10T11:39:52Z

Could documentation be expanded at least a bit? The code has very few comments and the readme is mostly talking about the topic and not the sample itself.

I updated the README.adoc. If you'd like specific information let me know and I'll add it. I think I'm headed to bed for a nap so might be a few hours before I do another update.

SaschaWillems · 2026-06-10T12:37:02Z

That's perfect. Thank you very much 👍🏻

r-potter · 2026-06-10T12:49:23Z

No reflection on the quality of the sample but we should hold off merging this until internal discussions are fully resolved (but please continue to review).

… the rest of the documentation.

asuessenbach · 2026-06-11T07:18:02Z

-The preceding high-level background describes how we get to a situation where there's plenty of desire to be able to work with and render point clouds that are dynamically generated.  ARCore, and ARKit are both able to create a point cloud map, and everything from Drones to robots use this same basic system to register and deal with the world around them.  To navigate a room, a drone/robot might need to be able to determine if a voxel is occupied or not by using an occupancy grid. This gives rise to solutions which are optimized for storing such maps that can be dynamically updated in real time.
+The preceding high-level background describes how we get to a situation where there's plenty of desire to be able to work with and render point clouds that are dynamically generated. ARCore and ARKit are both able to create a point cloud map, and everything from drones to robots use this same basic system to register and deal with the world around them. To navigate a room, a drone/robot might need to be able to determine if a voxel is occupied or not by using an occupancy grid. This gives rise to solutions which are optimized for storing such maps that can be dynamically updated in real time.

 The library https://octomap.github.io/[octomap] provides just such a library.


"The library ... provides just such a library"?

Maybe better: "... provides such functionality"?

Add a complex sample showing how to do a simplistic tool that shows a…

9afa1e3

… Voxel Point Cloud map using Octomap. Also demonstrates how to use ImGUI in a project with added functionality and positioning of rendered assets.

gpx1000 added 12 commits December 26, 2025 12:54

Merge branch 'main' into complex-octmap-rendering

ca842cb

Add explicit submodule update step to all workflow jobs

6251cdd

Ensures submodules are fully initialized and updated in each job by adding explicit `git submodule update --init --recursive` command after checkout, complementing the existing `submodules: "recursive"` checkout option.

Fix indentation in .gitmodules for octomap submodule

3593a36

Replaces spaces with tabs to maintain consistent formatting throughout the .gitmodules file.

Add octomap file for complex render sample

0d80d0b

Restrict render_octomap sample to platforms with GLFW support

c060bff

Force Windows to build library as static to hopefully fix CI.

c32ff54

Implement update_overlay method for render_octomap sample

97db620

Adds update_overlay method override that delegates to custom ImGUI implementation in render() function and executes additional UI callbacks when provided.

Fix header guard naming and exclude octomap submodule from CI checks

abb9e1f

Renames header guards in ImGUIUtil.h from ORB_SLAM3_IMGUI_H to IMGUI_UTIL_H to match the actual file name. Excludes samples/complex/render_octomap/octomap directory from CI workflow file checks.

Fix indentation in render_octomap sample

cda38bc

Replaces leading spaces with tabs to maintain consistent formatting throughout the file.

gpx1000 added 3 commits January 7, 2026 22:31

Update copyright year to 2026

08b9a90

Updates copyright headers across framework components and shaders from 2025 to 2026.

Update copyright years in project files

d857003

Updated copyright headers across various project files to reflect the year 2026. This ensures consistency and compliance with licensing requirements.

gpx1000 changed the title ~~Add a octomap rendering sample~~ Add a Gaussian Splat and octomap rendering sample Apr 2, 2026

gpx1000 added 4 commits May 31, 2026 19:03

Merge branch 'refs/heads/main' into complex-octmap-rendering

b7a50e5

# Conflicts: # .gitmodules # framework/CMakeLists.txt

clang-format

b49d37e

fix the compile

478d992

janharaldfredriksen-arm reviewed Jun 9, 2026

View reviewed changes

Comment thread samples/complex/render_octomap/README.adoc Outdated

asuessenbach reviewed Jun 10, 2026

View reviewed changes

Allow imgui to update with keyboard nav. fix name.size() >= 5 and add…

ff48c1d

… in built spv files so platforms that oddly don't auto build will work.

Fix for keyboard navigation and numpad look handling

21a11c1

SaschaWillems reviewed Jun 10, 2026

View reviewed changes

Comment thread samples/complex/render_octomap/render_octomap.cpp Outdated

clang-format

00594e4

address Sascha's nit on KHR_gaussian_splatting extension name (Diverg…

3916404

…es from what we do in GLTF_Loader in the framework, but it's a good idea).

Expand README for Octomap sample to include usage, controls, renderin…

a950f84

…g modes, and key implementation details.

r-potter self-requested a review June 10, 2026 12:42

SaschaWillems self-requested a review June 10, 2026 14:58

SaschaWillems previously approved these changes Jun 10, 2026

View reviewed changes

I forgot to add in the complex samples section to nav.adoc along with…

f07190f

… the rest of the documentation.

gpx1000 dismissed SaschaWillems’s stale review via f07190f June 10, 2026 15:43

SaschaWillems self-requested a review June 11, 2026 06:42

SaschaWillems approved these changes Jun 11, 2026

View reviewed changes

asuessenbach reviewed Jun 11, 2026

View reviewed changes

Conversation

gpx1000 commented Dec 14, 2024

Description

General Checklist:

Sample Checklist

Uh oh!

SaschaWillems commented Dec 16, 2024

Uh oh!

gary-sweet commented Jan 13, 2025

Uh oh!

SaschaWillems commented Jan 1, 2026

Uh oh!

gpx1000 commented Jan 2, 2026 via email

Uh oh!

gary-sweet commented Jan 7, 2026

Uh oh!

gpx1000 commented Jan 8, 2026

Uh oh!

gary-sweet commented Jan 8, 2026

Uh oh!

gary-sweet commented Jun 1, 2026

Uh oh!

Uh oh!

gary-sweet commented Jun 10, 2026

Uh oh!

gary-sweet commented Jun 10, 2026

Uh oh!

asuessenbach left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asuessenbach Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

gpx1000 Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

SaschaWillems Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

gpx1000 Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

gpx1000 commented Jun 10, 2026

Uh oh!

gpx1000 commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gary-sweet commented Jun 10, 2026

Uh oh!

gpx1000 commented Jun 10, 2026

Uh oh!

gpx1000 commented Jun 10, 2026

Uh oh!

gary-sweet commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SaschaWillems commented Jun 10, 2026

Uh oh!

gpx1000 commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

SaschaWillems commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gary-sweet commented Jun 10, 2026

Uh oh!

gpx1000 commented Jun 10, 2026

Uh oh!

SaschaWillems commented Jun 10, 2026

Uh oh!

r-potter commented Jun 10, 2026

Uh oh!

asuessenbach Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

asuessenbach left a comment •

edited

Loading

gpx1000 commented Jun 10, 2026 •

edited

Loading

gary-sweet commented Jun 10, 2026 •

edited

Loading

gpx1000 commented Jun 10, 2026 •

edited

Loading

SaschaWillems commented Jun 10, 2026 •

edited

Loading