DOC: Add robustness analysis

[Achintyasingh412]

This is a suggested pull request template for NiMARE.
It's designed to capture information we've found to be useful in reviewing pull requests.

If there is other information that would be helpful to include, please don't hesitate to add it!

Please also label your pull request with the relevant tags.
See here for more information and a list of available options:
https://github.com/neurostuff/NiMARE/blob/main/CONTRIBUTING.md#pull-requests
-->

Closes #927

Changes proposed in this pull request:

Summary by Sourcery

Document study-level robustness of subtraction meta-analysis results in the two-sample CBMA example.

Enhancements:

• Add a Jackknife robustness analysis and contribution table to the two-sample subtraction CBMA example.

Documentation:

• Document robustness analysis of subtraction results in the two-sample CBMA example.

[welcome]

Thanks for opening this pull request! We have detected this is the first time you have contributed to NiMARE. Please check out our contributing guidelines.
We invite you to list yourself as a NiMARE contributor, so if your name is not already mentioned, please modify the .zenodo.json file with your data right above Angie's entry. Example:

{
  "name": "Contributor, New",
  "affiliation": "Department of Psychology, Some University",
  "orcid": "<your id>"
},
{
  "name": "Laird, Angela R.",
  "affiliation": "Florida International University",
  "orcid": "0000-0003-3379-8744"
},

Of course, if you want to opt out this time there is no problem at all with adding your name later. You will be always welcome to add it in the future whenever you feel it should be listed.

[sourcery-ai]

Reviewer's Guide

The example now demonstrates how to assess the stability of subtraction clusters under individual study dropouts by running a Jackknife diagnostic and inspecting its contribution table.

Sequence diagram for subtraction Jackknife robustness analysis

sequenceDiagram
    participant Example
    participant Jackknife
    participant MetaResult
    participant DiagnosticResults
    Example->>Jackknife: Jackknife(target_image, voxel_thresh, display_second_group)
    Example->>Jackknife: transform(res_sub)
    Jackknife->>MetaResult: evaluate study dropouts
    MetaResult-->>Jackknife: subtraction cluster results
    Jackknife-->>DiagnosticResults: return sub_diagnostic_results
    Example->>DiagnosticResults: tables[diagnostic_table]
    DiagnosticResults-->>Example: sub_jackknife_table
    Example->>Example: head(10)

Loading

File-Level Changes

Change	Details	Files
Add a Jackknife robustness analysis to the subtraction meta-analysis example.	Import and configure the Jackknife diagnostic for the subtraction cluster-level corrected target image. Run the diagnostic on the subtraction MetaResult, including both groups in the analysis. Extract and display the resulting positive-tail contribution-count table for individual-study influence assessment.	examples/02_meta-analyses/08_plot_cbma_subtraction_conjunction.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[Achintyasingh412]

Hi ,

This PR resolves Issue #927 by integrating the missing Jackknife robustness analysis directly into the two-sample ALE subtraction example script.

To achieve this, we added the Jackknife diagnostic configured with the correct subtraction map name (z_desc-group1MinusGroup2Size_level-cluster_corr-FWE_method-montecarlo) to evaluate study stability across both groups.

We thoroughly verified the workflow by executing the script locally from top to bottom, confirming that all Monte Carlo permutations, diagnostic tables, and conjunction plots complete successfully without errors.

You will notice we did not add a new external citation to the bibliography at the bottom; this is because Jackknife is already a core, built-in diagnostic class in NiMARE rather than introducing a brand-new external paper.

Looking forward to your feedback!

[sourcery-ai]

Hey - I've found 1 issue

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="examples/02_meta-analyses/08_plot_cbma_subtraction_conjunction.py" line_range="209-219" />
<code_context>
+    voxel_thresh=None,
+    display_second_group=True,
+)
+sub_diagnostic_results = jackknife.transform(res_sub)
+
+###############################################################################
+# Subtraction Jackknife Table
+# -----------------------------------------------------------------------------
+# We can inspect the contribution table to see how individual studies from
+# both groups influence the resulting subtraction clusters.
+sub_jackknife_table = sub_diagnostic_results.tables[
+    "z_desc-group1MinusGroup2Size_level-cluster_corr-FWE_method-montecarlo_diag-Jackknife_tab-counts_tail-positive"
+]
+sub_jackknife_table.head(10)
 ###############################################################################
 # Conjunction analysis
</code_context>
<issue_to_address>
**issue:** The robustness diagnostic is run after `run_reports(res_sub, html_dir)`, so the generated HTML report does not contain the new subtraction Jackknife results. The example displays the table in the notebook-style output only after the report has already been written.

**Triggers:** When the example is executed to generate the documented HTML report.

**Suggested fix:** Move `run_reports(res_sub, html_dir)` after the Jackknife analysis, or explicitly regenerate the report after adding the diagnostic results.
</issue_to_address>

Sourcery assessment

Approval pending. 1 finding to address first.

Blocking findings: examples/02_meta-analyses/08_plot_cbma_subtraction_conjunction.py:219

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

issue: The robustness diagnostic is run after run_reports(res_sub, html_dir), so the generated HTML report does not contain the new subtraction Jackknife results. The example displays the table in the notebook-style output only after the report has already been written.

Triggers: When the example is executed to generate the documented HTML report.

Suggested fix: Move run_reports(res_sub, html_dir) after the Jackknife analysis, or explicitly regenerate the report after adding the diagnostic results.

[jdkent]

thanks @Achintyasingh412! please open/reference an existing issue before making a pull request. That allows us get alinged on what the issue is exactly before we try to fix it. That way we don't spend extra effort where it's not necessary. For example, I think this pull request could just be a comment on the relevant line that specifies the diagnostic, not a new section in this notebook.

[jdkent]

this is where the diagnostic is specified, we historically did not not run jackknife in the example because it took so long, I've since made some optimizations, so perhaps jackknife could be added here, but perhaps a note signalling jacnkknife could be used here would suffice.

[Achintyasingh412]

Hi @jdkent, thank you for the guidance! I completely understand about linking the issue first—I'll make sure to do that for all future PRs. Regarding the diagnostic, thanks for pointing out that Pairwise CBMA Workflow handles it via the diagnostics parameter. Should I update the workflow to use Jackknife directly there now that it's optimized, or would you prefer a comment/note indicating where it can be swapped?

[jdkent]

responding to this comment in what I assume to be relevant to this issue: #1127 (comment)

I would keep the diagnostic as focuscounter, there may be some cases that I don't cover with jackknife when the number of studies is low (like in this example), if you don't mind opening an issue, then someone (you if you want) can work on characterizing how that error arises and what a better guard/work around/error message would be. Thanks for looking into this!

[Achintyasingh412]

Done! Opened tracking issue #1130 to document the Jackknife distance matrix error during subtraction report generation. Thanks again for pointing it out