fixml.modules.workflow.response

Classes

`LLMInfo`
`RepositoryInfo`
`ChecklistInfo`
`TokenInfo`
`ErrorInfo`
`CallResult`
`EvaluationResponse`	A data class to store all information from test evaluation runs.

Module Contents

class fixml.modules.workflow.response.LLMInfo

Bases: pydantic.BaseModel

name: str

temperature: float

class fixml.modules.workflow.response.RepositoryInfo

Bases: pydantic.BaseModel

path: str | pathlib.Path

git_commit: str

object: fixml.modules.code_analyzer.repo.Repository

model_config

class fixml.modules.workflow.response.ChecklistInfo

Bases: pydantic.BaseModel

path: str | pathlib.Path

object: fixml.modules.checklist.checklist.Checklist

model_config

class fixml.modules.workflow.response.TokenInfo

Bases: pydantic.BaseModel

input_count: int

output_count: int

class fixml.modules.workflow.response.ErrorInfo

Bases: pydantic.BaseModel

name: str

description: str

class fixml.modules.workflow.response.CallResult

Bases: pydantic.BaseModel

start_time: datetime.datetime

end_time: datetime.datetime

tokens_used: TokenInfo

files_evaluated: List[str]

context: Dict[str, str]

prompt: str

success: bool

parsed_response: Dict | None

error: ErrorInfo | None

class fixml.modules.workflow.response.EvaluationResponse(**data)

Bases: pydantic.BaseModel, fixml.modules.mixins.WriteableMixin

A data class to store all information from test evaluation runs.

Here is the schema: {

model {
name temperature

} repository {

path git_commit object (excluded when exporting)

} checklist {

path object (excluded when exporting)

} call_results [{

start_time end_time tokens_used {

input_count output_count

} files_evaluated injected prompt success parsed_response errors [{

name description

}]

}]

}

model: LLMInfo

repository: RepositoryInfo

checklist: ChecklistInfo

call_results: List[CallResult]

to_json(output_path: str | pathlib.Path, exist_ok=False)

classmethod from_json(json_path: str | pathlib.Path)

Reconstruct an instance of EvaluationResponse from JSON file.

Requires checklist path and repository path to actually exist in the system, otherwise an error will be thrown.