fixml.modules.workflow.runners.evaluator

Classes

PerFileTestEvaluator

Concrete test evaluator that performs per-file evaluation.

Module Contents

class fixml.modules.workflow.runners.evaluator.PerFileTestEvaluator(llm: langchain_core.language_models.LanguageModelLike, prompt_format: fixml.modules.workflow.prompt_format.PromptFormat, repository: fixml.modules.code_analyzer.repo.Repository, checklist: fixml.modules.checklist.checklist.Checklist, test_dirs: Iterable[str | pathlib.Path] | None = None, retries: int = 3)

Bases: fixml.modules.workflow.runners.base.PromptInjectionRunner

Concrete test evaluator that performs per-file evaluation.

static _load_test_file_into_splits(file_path: str) → List[langchain_core.documents.Document]

_validate_response(raw_response: dict) → None

Validation logics that are not covered by pydantic or langchain.

run(verbose: bool = False) → fixml.modules.workflow.response.EvaluationResponse