Updated dbtool to support MTA files by jschick04 · Pull Request #485 · microsoft/EventLogExpert

jschick04 · 2026-04-19T04:19:53Z

No description provided.

Copilot

Pull request overview

Updates the EventDbTool CLI to treat provider data as coming from a “provider source” (local machine, .db, exported .evtx + sibling LocaleMetaData/*.MTA, or a folder containing these), enabling workflows that rely on MTA-only metadata rather than local registry/DLL resolution.

Changes:

Replaces showlocal/showdatabase with a unified show command that can read from .db, .evtx+LocaleMetaData, or folders.
Adds ProviderSource and MtaProviderSource helpers to discover/load providers from .db/.evtx/folders.
Extends create, merge, and diff to accept these broader provider sources.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/EventLogExpert.EventDbTool/ShowLocalCommand.cs	Removed legacy local-only “show” command in favor of unified source handling.
src/EventLogExpert.EventDbTool/ShowDatabaseCommand.cs	Removed legacy DB-only “show” command in favor of unified source handling.
src/EventLogExpert.EventDbTool/ShowCommand.cs	New unified `show` command supporting local, `.db`, `.evtx`+MTA, and folders.
src/EventLogExpert.EventDbTool/ProviderSource.cs	New abstraction for enumerating/loading provider details from `.db`/`.evtx`/folders.
src/EventLogExpert.EventDbTool/MtaProviderSource.cs	New logic to discover provider names from `.evtx` and resolve via sibling `LocaleMetaData/*.MTA` only.
src/EventLogExpert.EventDbTool/Program.cs	Switches root command registration to the new unified `show` command.
src/EventLogExpert.EventDbTool/MergeDatabaseCommand.cs	Allows merging from any provider source; adds case-insensitive match/removal behavior.
src/EventLogExpert.EventDbTool/DiffDatabaseCommand.cs	Allows diffing two provider sources and generating a `.db` containing providers only in the second source.
src/EventLogExpert.EventDbTool/DbToolCommand.cs	Adds `LoadLocalProviders` helper and updates output to include `Parameters` count.
src/EventLogExpert.EventDbTool/CreateDatabaseCommand.cs	Adds optional `source` argument and expands skip/filter logic to support non-DB inputs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-20T01:53:17Z

@@ -133,6 +173,12 @@ private void MergeDatabase(string sourceFile, string targetFile, bool overwriteP
            });

            providersCopied.Add(provider);
+            count++;


MergeDatabase streams providers into the target context, but still accumulates the full ProviderDetails objects in providersCopied for end-of-run logging. For large merges (especially from .evtx+MTA sources where each ProviderDetails can be sizable) this negates the intended memory benefits of batching/ChangeTracker.Clear and can lead to high memory usage. Consider logging each provider as it's processed (or storing only ProviderName strings for the final summary) instead of retaining the full details list.

Copilot · 2026-04-20T01:53:17Z

+    [Fact]
+    public void EndOfResults_AfterExhaustion_ShouldKeepBookmarkAndNotSetLastErrorCode()
+    {
+        // Single full-scan that consolidates bookmark stability + LastErrorCode assertions to
+        // avoid scanning the Application log multiple times across the suite.
+
+        // Arrange
+        using var reader = new EventLogReader(Constants.ApplicationLogName, PathType.LogName);
+
+        // Act — exhaust the reader so TryGetEvents returns false with ERROR_NO_MORE_ITEMS
+        while (reader.TryGetEvents(out _)) { }
+
+        string? bookmarkAfterExhaustion = reader.LastBookmark;
+
+        // One additional read past end-of-results
+        reader.TryGetEvents(out _);
+        string? bookmarkAfterExtraRead = reader.LastBookmark;
+
+        // Assert — bookmark is stable past EOF, and normal end-of-results does NOT set LastErrorCode
+        Assert.Equal(bookmarkAfterExhaustion, bookmarkAfterExtraRead);
+        Assert.Null(reader.LastErrorCode);


This test exhausts the entire Application event log (potentially thousands/millions of events) to reach end-of-results. That makes the suite runtime dependent on machine log size and can cause very slow/flaky CI runs. Consider using a small exported .evtx test fixture, constraining the query to a bounded range (e.g., by record ID/time), or otherwise limiting reads so the EOF/LastErrorCode behavior is exercised without a full scan.

Suggested change

[Fact]

public void EndOfResults_AfterExhaustion_ShouldKeepBookmarkAndNotSetLastErrorCode()

{

// Single full-scan that consolidates bookmark stability + LastErrorCode assertions to

// avoid scanning the Application log multiple times across the suite.

// Arrange

using var reader = new EventLogReader(Constants.ApplicationLogName, PathType.LogName);

// Act — exhaust the reader so TryGetEvents returns false with ERROR_NO_MORE_ITEMS

while (reader.TryGetEvents(out _)) { }

string? bookmarkAfterExhaustion = reader.LastBookmark;

// One additional read past end-of-results

reader.TryGetEvents(out _);

string? bookmarkAfterExtraRead = reader.LastBookmark;

// Assert — bookmark is stable past EOF, and normal end-of-results does NOT set LastErrorCode

Assert.Equal(bookmarkAfterExhaustion, bookmarkAfterExtraRead);

Assert.Null(reader.LastErrorCode);

private static string? TryCreateSingleEventFixtureFromApplicationLog()

{

var query = new System.Diagnostics.Eventing.Reader.EventLogQuery(

Constants.ApplicationLogName,

System.Diagnostics.Eventing.Reader.PathType.LogName)

{

ReverseDirection = true,

};

using var systemReader = new System.Diagnostics.Eventing.Reader.EventLogReader(query);

using var latestEvent = systemReader.ReadEvent();

if (latestEvent?.RecordId is not long recordId)

{

return null;

}

string tempFilePath = System.IO.Path.Combine(

System.IO.Path.GetTempPath(),

$"{nameof(EventLogReaderTests)}_{System.Guid.NewGuid():N}.evtx");

using var session = new System.Diagnostics.Eventing.Reader.EventLogSession();

session.ExportLog(

Constants.ApplicationLogName,

System.Diagnostics.Eventing.Reader.PathType.LogName,

$"*[System[EventRecordID={recordId}]]",

tempFilePath);

return tempFilePath;

}

[Fact]

public void EndOfResults_AfterExhaustion_ShouldKeepBookmarkAndNotSetLastErrorCode()

{

string? fixturePath = null;

try

{

fixturePath = TryCreateSingleEventFixtureFromApplicationLog();

using var reader = fixturePath is null

? new EventLogReader(Constants.ApplicationLogName, PathType.LogName)

: new EventLogReader(fixturePath, PathType.FilePath);

while (reader.TryGetEvents(out _)) { }

string? bookmarkAfterExhaustion = reader.LastBookmark;

reader.TryGetEvents(out _);

string? bookmarkAfterExtraRead = reader.LastBookmark;

Assert.Equal(bookmarkAfterExhaustion, bookmarkAfterExtraRead);

Assert.Null(reader.LastErrorCode);

}

finally

{

if (!string.IsNullOrEmpty(fixturePath) && System.IO.File.Exists(fixturePath))

{

System.IO.File.Delete(fixturePath);

}

}

jschick04 requested a review from a team as a code owner April 19, 2026 04:19

Copilot AI review requested due to automatic review settings April 19, 2026 04:19

Copilot started reviewing on behalf of jschick04 April 19, 2026 04:20 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

jschick04 force-pushed the jschick/update_dbtool_mta branch from ceebcf0 to a39b1ce Compare April 19, 2026 04:39

jschick04 requested a review from Copilot April 19, 2026 04:39

Copilot started reviewing on behalf of jschick04 April 19, 2026 04:39 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

jschick04 requested a review from Copilot April 19, 2026 05:05

Copilot started reviewing on behalf of jschick04 April 19, 2026 05:06 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/EventLogExpert.Eventing/EventProviderDatabase/EventProviderDbContext.cs Outdated

jschick04 force-pushed the jschick/update_dbtool_mta branch from 9b2c7dd to 02d6baa Compare April 19, 2026 05:13

jschick04 requested a review from Copilot April 19, 2026 05:13

Copilot started reviewing on behalf of jschick04 April 19, 2026 05:14 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

bill-long previously approved these changes Apr 19, 2026

View reviewed changes

jschick04 dismissed bill-long’s stale review via a42f8c2 April 19, 2026 14:18

jschick04 force-pushed the jschick/update_dbtool_mta branch from 02d6baa to a42f8c2 Compare April 19, 2026 14:18

Updated dbtool to support MTA files

64dc8ce

jschick04 force-pushed the jschick/update_dbtool_mta branch from a42f8c2 to e4c48ff Compare April 19, 2026 14:19

jschick04 requested a review from Copilot April 19, 2026 14:20

Copilot started reviewing on behalf of jschick04 April 19, 2026 14:21 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/EventLogExpert.Eventing/Readers/EventLogReader.cs

Comment thread src/EventLogExpert.EventDbTool/ProviderSource.cs Outdated

jschick04 force-pushed the jschick/update_dbtool_mta branch from e4c48ff to c92ac17 Compare April 19, 2026 14:39

jschick04 requested a review from Copilot April 19, 2026 14:52

Copilot started reviewing on behalf of jschick04 April 19, 2026 14:52 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/EventLogExpert.EventDbTool/MergeDatabaseCommand.cs Outdated

jschick04 force-pushed the jschick/update_dbtool_mta branch 2 times, most recently from d36f50c to 7470e68 Compare April 19, 2026 15:13

jschick04 requested a review from Copilot April 19, 2026 15:13

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/EventLogExpert.EventDbTool/MergeDatabaseCommand.cs

Comment thread src/EventLogExpert.Eventing/EventProviderDatabase/EventProviderDbContext.cs

Comment thread src/EventLogExpert.EventDbTool/ProviderSource.cs

Comment thread src/EventLogExpert.EventDbTool/ShowCommand.cs Outdated

jschick04 force-pushed the jschick/update_dbtool_mta branch from be9681d to 5c0eacc Compare April 19, 2026 16:26

jschick04 requested a review from Copilot April 19, 2026 16:26

Copilot started reviewing on behalf of jschick04 April 19, 2026 16:27 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/EventLogExpert.EventDbTool/CreateDatabaseCommand.cs Outdated

Comment thread src/EventLogExpert.EventDbTool/ShowCommand.cs

Comment thread src/EventLogExpert.Eventing.Tests/Readers/EventLogReaderTests.cs Outdated

jschick04 force-pushed the jschick/update_dbtool_mta branch from 5c0eacc to 3ed3c05 Compare April 19, 2026 16:39

jschick04 requested a review from Copilot April 19, 2026 16:39

Copilot started reviewing on behalf of jschick04 April 19, 2026 16:39 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

jschick04 force-pushed the jschick/update_dbtool_mta branch from 3ed3c05 to a6bf37e Compare April 19, 2026 17:02

jschick04 requested a review from Copilot April 19, 2026 17:11

Copilot started reviewing on behalf of jschick04 April 19, 2026 17:11 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/EventLogExpert.EventDbTool/DiffDatabaseCommand.cs Outdated

Comment thread src/EventLogExpert.EventDbTool/ProviderSource.cs Outdated

Comment thread src/EventLogExpert.Eventing/EventProviderDatabase/EventProviderDbContext.cs

jschick04 force-pushed the jschick/update_dbtool_mta branch from a6bf37e to 0622789 Compare April 19, 2026 17:23

jschick04 requested a review from Copilot April 19, 2026 17:23

Copilot started reviewing on behalf of jschick04 April 19, 2026 17:24 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/EventLogExpert.Eventing/EventProviderDatabase/EventProviderDbContext.cs Outdated

Comment thread src/EventLogExpert.EventDbTool/MergeDatabaseCommand.cs Outdated

Comment thread src/EventLogExpert.Eventing.Tests/EventProviderDatabase/EventProviderDbContextTests.cs Outdated

jschick04 force-pushed the jschick/update_dbtool_mta branch from 0622789 to a80dc86 Compare April 19, 2026 17:56

jschick04 requested a review from Copilot April 19, 2026 17:56

Copilot started reviewing on behalf of jschick04 April 19, 2026 17:57 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/EventLogExpert.EventDbTool/MergeDatabaseCommand.cs Outdated

Comment thread src/EventLogExpert.EventDbTool/Program.cs

jschick04 force-pushed the jschick/update_dbtool_mta branch from a80dc86 to 880b9be Compare April 19, 2026 18:18

jschick04 requested a review from Copilot April 19, 2026 18:19

Copilot started reviewing on behalf of jschick04 April 19, 2026 18:19 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/EventLogExpert.EventDbTool/MergeDatabaseCommand.cs Outdated

Comment thread src/EventLogExpert.EventDbTool/ProviderSource.cs Outdated

Comment thread src/EventLogExpert.EventDbTool/CreateDatabaseCommand.cs Outdated

Fixed CI issue related to needs upgrade checks

361cc0d

jschick04 force-pushed the jschick/update_dbtool_mta branch from 880b9be to 361cc0d Compare April 20, 2026 01:48

jschick04 requested a review from Copilot April 20, 2026 01:48

Copilot started reviewing on behalf of jschick04 April 20, 2026 01:48 View session

Copilot AI reviewed Apr 20, 2026

View reviewed changes

-    [Fact]
-    public void EndOfResults_AfterExhaustion_ShouldKeepBookmarkAndNotSetLastErrorCode()
-    {
-        // Single full-scan that consolidates bookmark stability + LastErrorCode assertions to
-        // avoid scanning the Application log multiple times across the suite.
-        // Arrange
-        using var reader = new EventLogReader(Constants.ApplicationLogName, PathType.LogName);
-        // Act — exhaust the reader so TryGetEvents returns false with ERROR_NO_MORE_ITEMS
-        while (reader.TryGetEvents(out _)) { }
-        string? bookmarkAfterExhaustion = reader.LastBookmark;
-        // One additional read past end-of-results
-        reader.TryGetEvents(out _);
-        string? bookmarkAfterExtraRead = reader.LastBookmark;
-        // Assert — bookmark is stable past EOF, and normal end-of-results does NOT set LastErrorCode
-        Assert.Equal(bookmarkAfterExhaustion, bookmarkAfterExtraRead);
-        Assert.Null(reader.LastErrorCode);
+    private static string? TryCreateSingleEventFixtureFromApplicationLog()
+    {
+        var query = new System.Diagnostics.Eventing.Reader.EventLogQuery(
+            Constants.ApplicationLogName,
+            System.Diagnostics.Eventing.Reader.PathType.LogName)
+        {
+            ReverseDirection = true,
+        };
+        using var systemReader = new System.Diagnostics.Eventing.Reader.EventLogReader(query);
+        using var latestEvent = systemReader.ReadEvent();
+        if (latestEvent?.RecordId is not long recordId)
+        {
+            return null;
+        }
+        string tempFilePath = System.IO.Path.Combine(
+            System.IO.Path.GetTempPath(),
+            $"{nameof(EventLogReaderTests)}_{System.Guid.NewGuid():N}.evtx");
+        using var session = new System.Diagnostics.Eventing.Reader.EventLogSession();
+        session.ExportLog(
+            Constants.ApplicationLogName,
+            System.Diagnostics.Eventing.Reader.PathType.LogName,
+            $"*[System[EventRecordID={recordId}]]",
+            tempFilePath);
+        return tempFilePath;
+    }
+    [Fact]
+    public void EndOfResults_AfterExhaustion_ShouldKeepBookmarkAndNotSetLastErrorCode()
+    {
+        string? fixturePath = null;
+        try
+        {
+            fixturePath = TryCreateSingleEventFixtureFromApplicationLog();
+            using var reader = fixturePath is null
+                ? new EventLogReader(Constants.ApplicationLogName, PathType.LogName)
+                : new EventLogReader(fixturePath, PathType.FilePath);
+            while (reader.TryGetEvents(out _)) { }
+            string? bookmarkAfterExhaustion = reader.LastBookmark;
+            reader.TryGetEvents(out _);
+            string? bookmarkAfterExtraRead = reader.LastBookmark;
+            Assert.Equal(bookmarkAfterExhaustion, bookmarkAfterExtraRead);
+            Assert.Null(reader.LastErrorCode);
+        }
+        finally
+        {
+            if (!string.IsNullOrEmpty(fixturePath) && System.IO.File.Exists(fixturePath))
+            {
+                System.IO.File.Delete(fixturePath);
+            }
+        }

Conversation

jschick04 commented Apr 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment