Skip to content

docs: record the Shodan and capa/knowledge-base research as TODO entries - #2

Merged
AsherDLL merged 1 commit into
mainfrom
public/todo-shodan-capa
Aug 8, 2026
Merged

docs: record the Shodan and capa/knowledge-base research as TODO entries#2
AsherDLL merged 1 commit into
mainfrom
public/todo-shodan-capa

Conversation

@AsherDLL

@AsherDLL AsherDLL commented Aug 8, 2026

Copy link
Copy Markdown
Member

Both features were investigated far enough to know what they cost, then deferred. Writing down what the research found so the next attempt starts from measurements rather than from scratch.

TODO-005, Shodan. The credit model decides the design and is the least obvious part: /shodan/host/{ip}, /dns/resolve and /dns/reverse cost nothing, while /dns/domain/{domain} costs a credit per lookup and a free Developer account has zero credits. So domain -> resolve -> host answers "what is running there" without ever touching a credit-consuming endpoint.

The real blocker is recorded plainly: network IOCs are prose inside EvidenceFinding.claim/detail with no structured value field anywhere, and enforce_network_coverage only counts them. There is nothing machine-readable to feed Shodan, so the entry lists the three ways out with their costs rather than implying the integration is the hard part. Also noted that Shodan does not belong in active_sources, which gates on a file digest and would hand it a SHA-256 it cannot use.

TODO-006, capa and the knowledge base. capa's ATT&CK and MBC output is structured where today's mappings come from a model reading prose, which is the argument for it. Against it, honestly: its .NET extractor is built on dnfile/dncil and is weakest on exactly the protected assemblies that are hardest here, so a SmartAssembly sample will likely yield little. Recorded the six enforced places a new sandbox tool touches, including the evidence-critic allowlist that silently rejects findings from an unlisted tool.

The knowledge base is recorded as the user framed it -- correlation, not skipping. A hash check at intake surfaces the prior report, capa capability sets act as a cross-sample similarity key, and similar capabilities are a lead to follow rather than a reason to skip analysis.

Its blocker is shared and worth knowing before either feature is planned: the memory subsystem has never persisted a single record. The database holds one scope and zero rows, because the scope id is only ever seeded by run_single_query, which the ADK entry points never call. Once that is fixed the rest is unusually cheap -- MemoryQuery.source is an indexed digest key needing no migration, scopes are durable, NoteRecord already exists for annotations, and FINDING_CODEC is a complete tested template whose first writer this would be.

Both features were investigated far enough to know what they cost, then
deferred. Writing down what the research found so the next attempt starts from
measurements rather than from scratch.

TODO-005, Shodan. The credit model decides the design and is the least obvious
part: /shodan/host/{ip}, /dns/resolve and /dns/reverse cost nothing, while
/dns/domain/{domain} costs a credit per lookup and a free Developer account has
zero credits. So domain -> resolve -> host answers "what is running there"
without ever touching a credit-consuming endpoint.

The real blocker is recorded plainly: network IOCs are prose inside
EvidenceFinding.claim/detail with no structured value field anywhere, and
enforce_network_coverage only counts them. There is nothing machine-readable to
feed Shodan, so the entry lists the three ways out with their costs rather than
implying the integration is the hard part. Also noted that Shodan does not
belong in active_sources, which gates on a file digest and would hand it a
SHA-256 it cannot use.

TODO-006, capa and the knowledge base. capa's ATT&CK and MBC output is
structured where today's mappings come from a model reading prose, which is the
argument for it. Against it, honestly: its .NET extractor is built on
dnfile/dncil and is weakest on exactly the protected assemblies that are hardest
here, so a SmartAssembly sample will likely yield little. Recorded the six
enforced places a new sandbox tool touches, including the evidence-critic
allowlist that silently rejects findings from an unlisted tool.

The knowledge base is recorded as the user framed it -- correlation, not
skipping. A hash check at intake surfaces the prior report, capa capability sets
act as a cross-sample similarity key, and similar capabilities are a lead to
follow rather than a reason to skip analysis.

Its blocker is shared and worth knowing before either feature is planned: the
memory subsystem has never persisted a single record. The database holds one
scope and zero rows, because the scope id is only ever seeded by
run_single_query, which the ADK entry points never call. Once that is fixed the
rest is unusually cheap -- MemoryQuery.source is an indexed digest key needing no
migration, scopes are durable, NoteRecord already exists for annotations, and
FINDING_CODEC is a complete tested template whose first writer this would be.
@AsherDLL
AsherDLL merged commit 54de147 into main Aug 8, 2026
2 checks passed
@AsherDLL
AsherDLL deleted the public/todo-shodan-capa branch August 10, 2026 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant