Skip to content

Implement SVS v3 bootstrap-time state vectors and async pub/sub support#36

Open
matianxing1992 wants to merge 10 commits into
named-data:masterfrom
matianxing1992:master
Open

Implement SVS v3 bootstrap-time state vectors and async pub/sub support#36
matianxing1992 wants to merge 10 commits into
named-data:masterfrom
matianxing1992:master

Conversation

@matianxing1992

Copy link
Copy Markdown
Collaborator

This PR adds bootstrap-time state vectors and several async/pubsub delivery improvements to ndn-svs.

Key changes:

  • encode bootstrap time with state-vector entries
  • add pub/sub selection and name-only publication APIs
  • add async publish and parallel production
  • add parallel sync processing and batching
  • improve piggyback mapping and late piggyback handling
  • ensure monotonic signed Interest timestamps
  • fix fetchAll pending-fetch mutation
  • add regression tests for the new behavior

The final state of this branch has been thoroughly tested and verified through extensive Mini-NDN experiments. However, because Codex was used to help reorganize and structure the commit history, I cannot guarantee that each intermediate commit represents a stable or fully working state. Only the final commit has been comprehensively validated.

Comment thread ndn-svs/mapping-provider.cpp Outdated
Comment thread ndn-svs/mapping-provider.cpp Outdated
Comment thread ndn-svs/mapping-provider.cpp Outdated
Comment thread .gitignore Outdated
Comment thread wscript Outdated
@Pesa Pesa linked an issue Jun 1, 2026 that may be closed by this pull request
Comment thread .waf-tools/default-compiler-flags.py Outdated
Comment thread ndn-svs/core.cpp
" elapsed_ms=" << (result.workerUs / 1000));

m_face.getIoContext().post([this, alive, result = std::move(result)] () mutable {
boost::asio::post(m_face.getIoContext(), [this, alive, result = std::move(result)] () mutable {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this commit should be folded with the commit that first introduced this code, it shouldn't be a separate commit. Same with the other asio::post changes.

Comment thread tests/unit-tests/core.t.cpp Outdated
Comment on lines +50 to +53
// DummyClientFace stores sent Interests in a std::vector. On macOS with
// libc++ container annotations and ASan, parsing the vector-owned Interest
// directly can trip container-overflow checks in ndn-cxx Block internals.
// Re-decode the wire into a local Interest before inspecting parameters.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this and it probably doesn't make sense. Is this AI generated?

Please explain because this may be papering over an actual bug somewhere...

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out. That workaround was added while I was debugging the macOS ASan failure, but I agree that the explanation was not well justified and could mask a real issue.

I removed this workaround in the latest revision. The test now parses the Interest parameters directly again, and the branch has been rebased/cleaned up so this change is no longer part of the PR.

If ASan still reports a problem after the cleanup, I will investigate the actual failing path instead of adding this kind of defensive re-encoding.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this was AI-assisted. I saw a large number of macOS/ASan failures and I did not fully understand the root cause at that moment, so I used AI to help debug it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, when I initially completed this large set of patches, the codebase was in a much worse state than it is now. I used AI to help reorganize and clean up portions of the code; otherwise, it would have been even more difficult to review. I apologize for the additional time and effort this has required from you during the review process.

Comment thread ndn-svs/common.hpp Outdated
Comment on lines +63 to +73
inline Name::Component
makeSeqNoComponent(SeqNo seqNo)
{
return Name::Component::fromSequenceNumber(seqNo);
}

inline SeqNo
readSeqNoComponent(const Name::Component& component)
{
return component.toSequenceNumber();
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why bother with these wrappers at all? just call the function directly

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for your suggestions! I have to admit that I was mostly focused on getting the functionality working, and I did not pay nearly enough attention to code quality and coding standards.

@matianxing1992 matianxing1992 force-pushed the master branch 2 times, most recently from 2d90595 to 0521665 Compare June 1, 2026 22:40
Comment thread ndn-svs/security-options.cpp Outdated
}

time::system_clock::time_point
KeyChainSigner::getFreshInterestTimestamp() const

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can you not use the InterestSigner class from ndn-cxx? If thread safety is a concern, then you have a much bigger problem because KeyChain doesn't guarantee thread safety either.

Add regression coverage for bootstrap-time state vectors, async publish,
piggyback delivery, and related pub/sub behavior in the existing unit test
suites.
Snapshot pending publication keys before issuing fetches so callbacks and cleanup paths can update m_fetchMap without invalidating the active iteration.

This fixes NDNSF dummy-face SVS smoke crashes observed during post-commit validation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SVS v3

2 participants