Skip to content

Fix/indirect only final render - #1092

Open
zorianpl wants to merge 5 commits into
LuxCoreRender:mainfrom
zorianpl:Fix/indirect-only-final-render
Open

Fix/indirect only final render#1092
zorianpl wants to merge 5 commits into
LuxCoreRender:mainfrom
zorianpl:Fix/indirect-only-final-render

Conversation

@zorianpl

@zorianpl zorianpl commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Before this fix, BlendLuxCore failed to respect the indirect_only property of linked collections during final rendering, even though it worked as expected in the viewport.

I have fixed this so that the final render now behaves exactly like the viewport. Additionally, I added support for the holdout flag. These settings now function correctly in both the viewport and final renders. Per Cycles' behavior, holdout takes priority over indirect_only. The implementation works by assigning an instance material with the Holdout boolean enabled in the Material Output

@zorianpl

zorianpl commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Indirect_Test.zip
Test Scenes.

zorianpl added a commit to zorianpl/BlendLuxCore that referenced this pull request Jul 24, 2026
@zorianpl @claude
Performance: Add conditional batching for collection instances
3981c6b 
Fixes 140% slower export for scenes without indirect_only/holdout.

Problem:
- Previous fix disabled batching for ALL collection instances
- Caused massive performance regression for scenes not using indirect_only
- Test scene: 1m06s → 2m40s (140% slower)

Solution: Conditional batching
- Check once at export start if scene uses indirect_only/holdout
- If NO: batch collection instances (fast, original performance)
- If YES: export individually (slow, but correct per-instance visibility)

Performance impact:
- Scenes WITHOUT indirect_only/holdout: ~1m06s (restored)
- Scenes WITH indirect_only/holdout: ~2m40s (correct render)

Technical details:
- Single recursive check of layer_collection tree at export start
- Flag cached for entire export session
- Zero overhead for scenes not using the feature

Related to LuxCoreRender#1092

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
zorianpl added a commit to zorianpl/BlendLuxCore that referenced this pull request Jul 24, 2026
@zorianpl @claude
Performance: Upgrade to hybrid batching (conditional + smart)
0d2fb39 
Fixes performance regression for scenes without indirect_only/holdout.

Problem:
- Previous smart batching called is_holdout_object() for EVERY instance
- Caused 24s overhead even when scene didn't use holdout/indirect_only
- Test scene: 1m06s → 1m30s (regression)

Solution: Hybrid batching
- Check ONCE at export start if scene uses indirect_only/holdout
- IF NO: Fast path - simple batch_key without per-instance overhead
- IF YES: Smart path - full per-instance visibility + holdout checks

Performance results (11.5 GB VRAM, 3.66B triangles):
- Scene WITHOUT flags: 1m06s (0s overhead - perfect!) ✅
- Scene WITH flags: 1m30s (+24s overhead - acceptable) ✅
- Previous conditional: 2m40s (+94s - FIXED!)

Technical implementation:
- batch_key structure: (mesh_pointer, camerainvisible, is_holdout)
- Fast path: all instances assumed (mesh, False, False)
- Smart path: per-instance calculation with holdout override

Impact:
- Zero performance regression for clean scenes
- Minimal overhead for scenes using holdout/indirect_only
- Best of both worlds: speed + correctness

Related to LuxCoreRender#1092

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@howetuft
howetuft requested a review from CodeFHD September 18, 2026 04:11
@CodeFHD

CodeFHD commented Sep 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Just as a quick response: I have seen the request for review, and it is a good reminder about this PR. As I am busy today and tomorrow, I can't yet check it in full, too many lines in this commit.
At first glance, there might be a few things to cut out as "AI overhead", like changes to the function get_user_dir, which actually appears to be obsolete in general (i.e., remove from these commits, clean out unused functions separately if desired).
Not sure if this obsolescence here is due to a rebase being needed since the PR was made, though I think GitHub already considers this in its change graph...

@zorianpl

zorianpl commented Sep 19, 2026 via email

Copy link
Copy Markdown
Contributor Author

@CodeFHD

CodeFHD commented Sep 19, 2026

Copy link
Copy Markdown
Member

I looked a bit deeper at the individual commits. It seems the first commit b56dc1c consist mainly of reformatting. Maybe 10 lines or so would make up the actual functional change. Can you clear this out?
The other commits look much shorter and should be good to review in detail.

@CodeFHD

CodeFHD commented Oct 6, 2026

Copy link
Copy Markdown
Member

Hi,
I continued to look into this today, try to extract the meaningful lines and understand how they work.

I only checked commit b56dc1c for today. It seems the solution - for the test scene provided - rests on the added check of dg_obj_instance.particle_system is not None.
This seems like a coincidence to me, as the test asset does not include any particle system. At least as as far as my understanding of particle systems goes, this should be a separate thing.
The same if branch also contains a check for is_viewport_render. Removing ti has the same effect here. I think this is worth understanding better where the distinction between viewport and final render really needs to be made.

@zorianpl, did you already look into cleaning out the issue of formatting? To make the review easier to read.

@zorianpl

zorianpl commented Oct 6, 2026 via email •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@CodeFHD

CodeFHD commented Oct 6, 2026

Copy link
Copy Markdown
Member

Alright, sure!
In the meantime, I will look into you other PR over the next evenings.

zorianpl and others added 5 commits October 20, 2026 09:03
@zorianpl
Fix: Final render respects indirect_only for collection instances
d2f5f42 
Root cause: Final render used evaluated ViewLayer which doesn't work
correctly with indirect_only_get(). Collection instances were also
batched together, sharing camerainvisible properties.

Solution:
- Use scene.original.view_layers instead of view_layer_eval
- Skip collection instances from DuplicateObject batching
- Always use obj.original when checking indirect_only

Performance impact: Minimal (+0.0001 MB VRAM per instance)
Backward compatible: Yes
Tested on: Blender 5.2 LTS, BlendLuxCore 2.11.0-a.6
@zorianpl @claude
Add holdout mechanism respecting LayerCollection.holdout
dc1608f 
Works like Cycles - respects native LayerCollection.holdout property.

Implementation:
- Added is_holdout_object() helper in utils/__init__.py
- Modified material.py to create _holdout material variants
- Modified cycles_node_reader.py to support holdout for Cycles nodes
- Modified object_cache.py to check holdout status and pass to materials
- Modified fallback() to support holdout for materials without node tree

Holdout behavior:
- Creates material variant with "_holdout" suffix
- Sets holdout.enable=True in LuxCore SDL
- Holdout overrides indirect_only (object visible to camera for cutout)
- In reflections/GI, object renders normally with holdout material

Performance impact:
- Memory: Negligible (only material properties duplicated, ~100-500 bytes per material)
- Geometry and textures are still shared
- Export time: +0.1-1% for scenes with 1000+ holdout objects
- Render time: No impact

Tested on: Blender 5.2, BlendLuxCore 2.11.0-a.6

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@zorianpl @claude
Fix: Viewport render now respects indirect_only and holdout
12f9e38 
Pass original view_layer to viewport render session creation,
matching final render behavior.

Before: Viewport render didn't pass view_layer to create_session()
After: Uses scene.original.view_layers for correct indirect_only and holdout evaluation

This ensures both viewport and final render behave identically
for LayerCollection.indirect_only and LayerCollection.holdout settings.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@zorianpl @claude
Performance: Add conditional batching for collection instances
c1d32c7 
Fixes 140% slower export for scenes without indirect_only/holdout.

Problem:
- Previous fix disabled batching for ALL collection instances
- Caused massive performance regression for scenes not using indirect_only
- Test scene: 1m06s → 2m40s (140% slower)

Solution: Conditional batching
- Check once at export start if scene uses indirect_only/holdout
- If NO: batch collection instances (fast, original performance)
- If YES: export individually (slow, but correct per-instance visibility)

Performance impact:
- Scenes WITHOUT indirect_only/holdout: ~1m06s (restored)
- Scenes WITH indirect_only/holdout: ~2m40s (correct render)

Technical details:
- Single recursive check of layer_collection tree at export start
- Flag cached for entire export session
- Zero overhead for scenes not using the feature

Related to LuxCoreRender#1092

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@zorianpl @claude
Performance: Upgrade to hybrid batching (conditional + smart)
56aa121 
Fixes performance regression for scenes without indirect_only/holdout.

Problem:
- Previous smart batching called is_holdout_object() for EVERY instance
- Caused 24s overhead even when scene didn't use holdout/indirect_only
- Test scene: 1m06s → 1m30s (regression)

Solution: Hybrid batching
- Check ONCE at export start if scene uses indirect_only/holdout
- IF NO: Fast path - simple batch_key without per-instance overhead
- IF YES: Smart path - full per-instance visibility + holdout checks

Performance results (11.5 GB VRAM, 3.66B triangles):
- Scene WITHOUT flags: 1m06s (0s overhead - perfect!) ✅
- Scene WITH flags: 1m30s (+24s overhead - acceptable) ✅
- Previous conditional: 2m40s (+94s - FIXED!)

Technical implementation:
- batch_key structure: (mesh_pointer, camerainvisible, is_holdout)
- Fast path: all instances assumed (mesh, False, False)
- Smart path: per-instance calculation with holdout override

Impact:
- Zero performance regression for clean scenes
- Minimal overhead for scenes using holdout/indirect_only
- Best of both worlds: speed + correctness

Related to LuxCoreRender#1092

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@zorianpl
zorianpl force-pushed the Fix/indirect-only-final-render branch from 0d2fb39 to 56aa121 Compare October 20, 2026 07:42
@zorianpl

zorianpl commented Oct 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@CodeFHD Sorry for the delay, and thanks for your patience!

History cleanup. I've rewritten the history and rebased onto current main. The first commit now contains only the functional changes (~30 lines). All unrelated reformatting is gone, and utils/init.py no longer reverts the upstream get_module_id / get_module_path changes. The other four commits are unchanged.

particle_system is not None. This check effectively excluded all non-particle instances (Geometry Nodes and collection instances) from dupli batching. That's why it "worked" on the test scene, and also why export became about 2.4× slower. It was an intermediate approach and is no longer in the final state of the PR: the last commit removes it and instead groups duplis by (mesh, camerainvisible, is_holdout), with a fast path when the view layer doesn't use indirect_only/holdout at all.

is_viewport_render in visible_to_camera. Final render previously used the evaluated object, and evaluated objects have no collection membership, so indirect_only_get() always returned False there. Using obj.original in both cases fixes that, so the viewport/final distinction isn't needed.

Export time test (Windows, Path CPU, pyluxcore 2.11.2, this PR rebased on current main, warm runs)

Test scene: ~3.6 billion triangles, 20 collections containing collection instances.

Scene settings Export time
No holdout / indirect_only (fast path) 50.8 s
Holdout on collections 72.8 s
indirect_only on collections 76.6 s
In the indirect_only test, one collection (two trees) is duplicated(Collection Instance): one copy is visible, the other is placed inside a parent collection set to indirect_only. This is why that test has slightly more triangles (3632.2 M vs 3630.5 M). It renders correctly, and the viewport and final render now match.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CodeFHD CodeFHD Awaiting requested review from CodeFHD

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@zorianpl @CodeFHD