MAINT: Frontend core refactor by rlundeen2 · Pull Request #1753 · microsoft/PyRIT

rlundeen2 · 2026-05-18T22:49:25Z

This PR removes the custom logic for the front-end, using the REST APIs instead of custom CLI logic.

Verified works as intended, output is the same

In future PRs:

Progress bar is somewhat broken for a couple reasons. 1) server output if you start the server 2) inaccurate attack counts. Both will be tackled in followup PRs
API dictionary parsing is fragile. Working toward a more robust solution

…5_11_scenario_gaps

…ndeen/2026_05_13_add_initializer

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…5_13_add_initializer # Conflicts: # pyrit/backend/models/initializers.py # pyrit/backend/routes/initializers.py # pyrit/backend/services/initializer_service.py # tests/unit/backend/test_initializer_service.py

Create new pyrit/printer/ module with abstract base classes that contain all formatting logic. Data-fetching operations (CentralMemory calls) are abstract methods implemented by framework subclasses. This enables thin clients to reuse all pretty-printing by subclassing the base printers and implementing data-fetching via REST endpoints. The thin client only needs pyrit.models + pyrit.identifiers + colorama. Changes: - New pyrit/printer/ module with attack_result, scenario_result, scorer subpackages - ConsoleAttackPrinterBase: all attack console formatting, abstract get_conversation/get_scores - ConsoleScenarioPrinterBase: all scenario console formatting - ConsoleScorerPrinterBase: all scorer formatting, abstract get_objective/harm_metrics - Existing framework printers refactored to thin subclasses (backward compatible) - Added to_dict()/from_dict() to AttackResult, ScenarioResult, ScenarioIdentifier, ConversationReference, Score, MessagePiece, Message for serialization round-tripping - Message.to_full_dict() added for rich serialization (to_dict() unchanged for compat) All 675 existing tests pass with no modifications. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Move framework CentralMemory implementations into pyrit/printer/ alongside their base classes. CentralMemory is imported lazily inside constructors, so thin clients importing the module never pay the SQLAlchemy cost. - ConsoleAttackResultPrinter now lives in pyrit.printer.attack_result.console - ConsoleScenarioResultPrinter now lives in pyrit.printer.scenario_result.console - ConsoleScorerPrinter now lives in pyrit.printer.scorer.console - Old locations (executor/attack/printer/, scenario/printer/, score/printer/) become pure re-exports for backward compatibility - Updated test patch paths to match new module locations Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…6.0) Old locations now use PEP 562 __getattr__ lazy re-exports with DeprecationWarning. Only concrete classes are re-exported (not bases). - pyrit.executor.attack.printer → pyrit.printer.attack_result - pyrit.scenario.printer → pyrit.printer.scenario_result - pyrit.score.printer → pyrit.printer.scorer - Updated all internal callers to new canonical paths - Old ABC files (attack_result_printer.py, scenario_result_printer.py, scorer_printer.py) kept for now but deprecated via __init__.py Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…of bases Concrete classes that use CentralMemory/scorer registry are now named to clearly indicate their data source: - ConsoleAttackResultPrinter → ConsoleAttackMemoryPrinter - ConsoleScenarioResultPrinter → ConsoleScenarioMemoryPrinter - ConsoleScorerPrinter → ConsoleScorerMemoryPrinter Moved ScorerEvaluationIdentifier (pyrit internal) from base class into the concrete ConsoleScorerMemoryPrinter. Base classes now contain only formatting logic with no pyrit-internal imports beyond models/identifiers. Deprecated re-exports at old paths still work (mapping old names to new), scheduled for removal in 0.16.0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Created MarkdownAttackPrinterBase + MarkdownAttackMemoryPrinter in pyrit/printer/attack_result/markdown.py (same pattern as console) - Deleted dead old ABC files: - pyrit/executor/attack/printer/attack_result_printer.py - pyrit/scenario/printer/scenario_result_printer.py - pyrit/score/printer/scorer_printer.py - Old markdown_printer.py now a deprecation re-export shim - Updated all internal imports and test patches Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Changed dict[str, object] to dict[str, Any] in MessagePiece.from_dict() and Message.from_dict() to satisfy pyright (dict.get returns object otherwise) - Added Any import to message_piece.py and message.py - Wrapped get_prompt_scores return in list() for Sequence -> list coercion - Added isinstance check in display_image_async for type safety Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Added return type annotation (-> type) to all __getattr__ deprecation shims - Added noqa: B027 to display_image_async intentional no-op default - Added Returns/Raises sections to short docstrings (DOC201, DOC501) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

objective_target_identifier and objective_scorer_identifier may be None when deserializing from dicts. The printer bases already handle None. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…5_13_frontend_core_refactor # Conflicts: # .pyrit_conf_example # pyrit/backend/models/initializers.py # pyrit/backend/routes/initializers.py # pyrit/backend/services/initializer_service.py # pyrit/cli/frontend_core.py # pyrit/cli/pyrit_backend.py # pyrit/registry/class_registries/initializer_registry.py # pyrit/setup/configuration_loader.py # tests/unit/backend/test_initializer_service.py # tests/unit/cli/test_pyrit_backend.py # tests/unit/registry/test_initializer_registry.py

…rlundeen/2026_05_13_frontend_core_refactor # Conflicts: # pyrit/cli/frontend_core.py # pyrit/cli/pyrit_shell.py

…rlundeen/2026_05_13_frontend_core_refactor

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ctor Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

rlundeen2 · 2026-05-18T23:02:08Z

    },
 )
-async def get_scenario_run_results(scenario_result_id: str) -> ScenarioRunDetail:
+async def get_scenario_run_results(scenario_result_id: str) -> dict:


I have a plan for this I want to propose; but I think this is okay for this PR

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

rlundeen2 · 2026-05-19T16:47:54Z

+# ---------------------------------------------------------------------------
+
+
+def print_scenario_list(*, items: list[dict[str, Any]]) -> None:


Similarly, I feel like dict is not ideal; I think a good solution is to go towards a shared model

rlundeen2 · 2026-05-19T16:55:19Z

+        Returns:
+            dict[str, Any]: Serialized payload suitable for REST APIs or persistence.
+        """
+        from pyrit.models.conversation_reference import ConversationReference


there is another pr for this that is a dependency; see #1738

…5_13_frontend_core_refactor # Conflicts: # pyrit/cli/pyrit_shell.py # tests/unit/cli/test_pyrit_shell.py

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… CLI Fixes the failing `docker_build/Test GUI (local)` CI check and brings diff coverage above the 90% threshold for the frontend refactor. GUI / runtime fixes - `docker/start.sh` and `frontend/dev.py`: update stale references to the deleted `pyrit.cli.pyrit_backend` module (now `pyrit.backend.pyrit_backend`). - `doc/code/scenarios/0_scenarios.{py,ipynb}` and `2_custom_scenario_parameters.{py,ipynb}`: replace deleted `pyrit.cli.frontend_core` imports with `get_scenario_service` plus `print_scenario_list`. - `doc/myst.yml`: drop dead API doc entries for the removed modules and add entries for the new `api_client` and `cli_helpers` modules. Test coverage - New unit tests: - `tests/unit/cli/test_output.py` (printer functions) - `tests/unit/cli/test_api_client.py` (PyRITApiClient + error paths) - `tests/unit/cli/test_server_launcher.py` (probe/start/stop) - `tests/unit/cli/test_config_reader.py` (layered YAML loading) - Extended existing unit tests for `pyrit_backend`, `backend/main`, `pyrit_scan`, `pyrit_shell`, and `configuration_loader`. Diff coverage on the PR is now 96%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…hell) The frontend refactor regressed scenario-declared parameter handling on both CLI entry points - flags like `--max-turns 7` were rejected before the second pass could fetch metadata from the server. pyrit_scan - `parse_args` now uses `parse_known_args` (pass 1 is tolerant of scenario-specific flags) and stashes leftovers + the raw arg list on the Namespace for the second pass. - `_reparse_with_scenario_params` threads the original `args` list instead of reading `sys.argv[1:]`, so explicit-args callers work. - `main` does a strict re-parse when there are unknown args but no scenario is specified, preserving the original `exit 2` error for truly invalid flags. pyrit_shell - `do_run` now fetches `get_scenario_async` first, builds `Parameter` objects from the response's `supported_parameters`, and threads `declared_params` into `parse_run_arguments`. - Calls `extract_scenario_args` and propagates `scenario_params` on the REST request payload. Tests - New `TestScenarioParamFlow` (4 tests) and `TestShellScenarioParamFlow` (4 tests) regression cases covering forward, invalid-flag, no-params, and metadata-fetch failure paths. Diff coverage remains at 96%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…er config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

rlundeen2 and others added 30 commits May 12, 2026 17:01

Adding initializer service

2d8473a

pre-commit

40cfea2

Merge remote-tracking branch 'origin/main' into users/rlundeen/2026_0…

eebdb4e

…5_11_scenario_gaps

Merge branch 'main' into users/rlundeen/2026_05_11_scenario_gaps

31ecfc7

pr feedback

351ee5c

pr feedback

c0d5a08

Merge branch 'users/rlundeen/2026_05_11_scenario_gaps' into users/rlu…

bf1f00d

…ndeen/2026_05_13_add_initializer

adding custom initializers to rest

328ce79

style: Optional -> | None, import inspect to top-level

798c2e5

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into users/rlundeen/2026_0…

cbfc4df

…5_13_add_initializer # Conflicts: # pyrit/backend/models/initializers.py # pyrit/backend/routes/initializers.py # pyrit/backend/services/initializer_service.py # tests/unit/backend/test_initializer_service.py

adding content

abb3f62

self review

a75d767

self review

69d9c96

Merge branch 'main' into users/rlundeen/2026_05_13_add_initializer

92873dd

refactoring frontend

f5045a0

Add missing __all__ to scenario printer deprecation shim

91a417a

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Fix ty type check: make ScenarioResult identifier params optional

d117af2

objective_target_identifier and objective_scorer_identifier may be None when deserializing from dicts. The printer bases already handle None. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

pr feedback

65becc5

pre-commit

1eeb7a3

fixing test

4f29026

Merge branch 'users/rlundeen/2026_05_13_printer_refactor' into users/…

89e3202

…rlundeen/2026_05_13_frontend_core_refactor # Conflicts: # pyrit/cli/frontend_core.py # pyrit/cli/pyrit_shell.py

fixing test

69ccff3

rlundeen2 and others added 6 commits May 14, 2026 18:02

self-review

bf32513

Merge branch 'users/rlundeen/2026_05_13_printer_refactor' into users/…

880feef

…rlundeen/2026_05_13_frontend_core_refactor

Use printer module for scenario results in CLI

b5ab93c

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Show strategy-level progress during scenario runs

f97ad2f

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Merge branch 'main' into users/rlundeen/2026_05_13_frontend_core_refa…

1169f74

…ctor Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

pre-commit

e410c4b

rlundeen2 commented May 18, 2026

View reviewed changes

rlundeen2 and others added 2 commits May 18, 2026 22:21

Move setup_frontend into lifespan, share CLI helpers via pyrit.common

86bc912

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

main fix

c81b04e

rlundeen2 commented May 19, 2026

View reviewed changes

rlundeen2 marked this pull request as ready for review May 19, 2026 16:58

rlundeen2 and others added 5 commits May 19, 2026 10:09

Merge remote-tracking branch 'origin/main' into users/rlundeen/2026_0…

4b1e380

…5_13_frontend_core_refactor # Conflicts: # pyrit/cli/pyrit_shell.py # tests/unit/cli/test_pyrit_shell.py

lint fixes

2bdaf6b

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Address PR review: typed scenario params, persistent shell loop, dock…

6123bb0

…er config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MAINT: Frontend core refactor#1753

MAINT: Frontend core refactor#1753
rlundeen2 wants to merge 43 commits into
microsoft:mainfrom
rlundeen2:users/rlundeen/2026_05_13_frontend_core_refactor

rlundeen2 commented May 18, 2026 •

edited

Loading

Uh oh!

rlundeen2 May 18, 2026

Uh oh!

rlundeen2 May 19, 2026

Uh oh!

rlundeen2 May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		# ---------------------------------------------------------------------------


		def print_scenario_list(*, items: list[dict[str, Any]]) -> None:

Conversation

rlundeen2 commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rlundeen2 May 18, 2026

Choose a reason for hiding this comment

Uh oh!

rlundeen2 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

rlundeen2 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rlundeen2 commented May 18, 2026 •

edited

Loading