Skip to content

Speed up test suite via fixture scope optimization and slow test markers#116

Merged
ryan-kipawa merged 3 commits into
mainfrom
copilot/improve-test-suite-speed
Feb 26, 2026
Merged

Speed up test suite via fixture scope optimization and slow test markers#116
ryan-kipawa merged 3 commits into
mainfrom
copilot/improve-test-suite-speed

Conversation

Copilot AI commented Feb 24, 2026

Copy link
Copy Markdown
Contributor

Many tests unnecessarily open a new .NET Database connection per test function despite being read-only or sharing the same underlying database file. Database opens via pythonnet are expensive.

Fixture scope optimizations

  • TestDatabase.db: Function-scoped → class-scoped, eliminating ~10 redundant DB opens. Extracted test_close to its own TestDatabaseClose class to avoid closing the shared connection.
  • TestBaseQuery: Added class-scoped _db fixture so the Database opens once instead of per-test (~11 opens saved, including 8 parametrized test_by_muid cases). base_query remains function-scoped for mutable state isolation.
  • test_scenarios.py: Grouped two read-only tests into TestScenarioReadOnly with a class-scoped db on session_sirius_db. Saves 2 file copies + 2 DB opens + fixes connection leaks.
  • test_create_existing_model_raises_error: sirius_dbsession_sirius_db (read-only check, no copy needed).

Slow test markers

  • All simulation runner tests marked @pytest.mark.slow.
  • Added -m not slow to default pytest addopts in pyproject.toml, so slow tests are skipped by default when running locally. CI workflows are unaffected since they pass explicit -m flags on the command line, which override addopts.
Original prompt

The test suite is very slow. Explore opportunities to speed it up.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

- test_database.py: Make TestDatabase.db fixture class-scoped (saves ~10 DB
  opens per run), move test_close to separate TestDatabaseClose class,
  switch test_create_existing_model_raises_error to session_sirius_db
- test_scenarios.py: Group read-only tests into TestScenarioReadOnly class
  with shared class-scoped DB connection (saves 2 DB copies + 2 DB opens)
- test_queries.py: Add class-scoped _db fixture to TestBaseQuery (saves ~11
  DB opens including 8 parametrized test_by_muid cases)
- test_simulation_runner.py: Mark all simulation tests with @pytest.mark.slow

Co-authored-by: ryan-kipawa <74630349+ryan-kipawa@users.noreply.github.com>
Copilot AI changed the title [WIP] Investigation to optimize test suite performance Speed up test suite via fixture scope optimization and slow test markers Feb 24, 2026

@ryan-kipawa ryan-kipawa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot

You marked some tests as slow. Adjust pyproject.toml pytest configuration such that it does not run slow tests by default locally. Do not update the github action - there they should run.

After running the tests locally, these were the 50 slowest tests.

===================================================== slowest 50 durations ====================================================== 
51.25s call     tests/test_simulation_runner.py::test_mike_engine_run_flood
42.30s call     tests/test_simulation_runner.py::test_mike_engine_run_swmm
20.63s call     tests/test_simulation_runner.py::test_mike_engine_run_epanet
18.75s call     tests/test_simulation_runner.py::test_mike_engine_run_1d
14.36s call     tests/database/test_table_generation.py::TestTableGeneration::test_generated_code_can_be_imported_and_used        
12.80s call     tests/database/test_database.py::TestDatabaseOpen::test_open_existing_model
9.72s call     tests/test_tools.py::test_import_tool
7.17s call     tests/test_tools.py::test_interpolate_tool
6.87s call     tests/test_tools.py::test_catch_slope_len_tool
6.10s call     tests/test_tools.py::test_connect_repair_tool
6.01s setup    tests/database/test_queries.py::TestInsertQuery::test_insert
5.80s call     tests/test_tools.py::test_topology_repair_tool
5.71s setup    tests/database/test_database.py::TestDatabase::test_is_open
5.41s call     tests/test_scenarios.py::test_api_create_and_activate_scenario
5.11s setup    tests/database/test_queries.py::TestDeleteQuery::test_safety_check
5.07s setup    tests/database/test_queries.py::TestUpdateQuery::test_safety_check
4.95s setup    tests/database/test_tables.py::TestBaseTable::test_name
3.17s call     tests/database/test_database.py::TestDatabase::test_top_level_imports
2.79s call     tests/database/test_database.py::TestDatabaseCreate::test_create_new_model
2.66s call     tests/database/test_database.py::TestDatabaseCreate::test_import_epanet
2.41s call     tests/database/test_database.py::TestDatabaseCreate::test_import_swmm
2.10s setup    tests/database/test_table_generation.py::TestTableGeneration::test_generate_single_table_class
2.03s setup    tests/database/test_table_generation.py::TestTableGeneration::test_generated_code_can_be_imported_and_used
1.95s setup    tests/database/test_table_generation.py::TestTableGeneration::test_generate_all_tables
0.68s call     tests/database/test_database.py::TestDatabaseClose::test_close
0.67s call     tests/database/test_table_generation.py::TestTableGeneration::test_generate_all_tables
0.63s call     tests/database/test_queries.py::TestBaseQuery::test_base_query_error_if_not_open
0.63s setup    tests/database/test_queries.py::TestBaseQuery::test_where_clause
0.62s setup    tests/test_scenarios.py::TestScenarioReadOnly::test_api_access_scenarios_and_alternatives
0.61s setup    tests/database/test_queries.py::TestSelectQuery::test_select_all_columns
0.56s setup    tests/database/test_table_collection.py::TestBaseTableCollection::test_initialize_tables
0.54s setup    tests/database/test_queries.py::TestUpdateQuery::test_update_computation_begin_with_string

Add '-m not slow' to pyproject.toml addopts so slow tests are excluded
by default when running pytest locally. CI workflows already pass explicit
-m flags which override this default.

Co-authored-by: ryan-kipawa <74630349+ryan-kipawa@users.noreply.github.com>

Copilot AI commented Feb 25, 2026

Copy link
Copy Markdown
Contributor Author

@copilot

You marked some tests as slow. Adjust pyproject.toml pytest configuration such that it does not run slow tests by default locally. Do not update the github action - there they should run....

Added -m not slow to pyproject.toml addopts in 0160235. Running pytest locally now skips slow tests by default. CI workflows are unchanged — they all pass explicit -m flags which override the addopts default, so slow tests still run there.

@ryan-kipawa ryan-kipawa marked this pull request as ready for review February 26, 2026 15:05
@ryan-kipawa ryan-kipawa merged commit 3ec75d4 into main Feb 26, 2026
3 of 4 checks passed
@ryan-kipawa ryan-kipawa deleted the copilot/improve-test-suite-speed branch February 26, 2026 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants