fixml.cli.repository

Classes

RepositoryActions

A mixin for classes which will write content to filesystem.

Module Contents

class fixml.cli.repository.RepositoryActions

Bases: fixml.modules.mixins.WriteableMixin

A mixin for classes which will write content to filesystem.

static generate(test_output_path: str, checklist_path: str = None, model: str = 'gpt-3.5-turbo', verbose: bool = False, debug: bool = False)

Test spec generation.

This will generate function specifications for each item in the checklist. Currently, it will not be referencing existing codebase and thus the specifications created will not be aware of the existing codes or the requirements.

Parameters:

  • test_output_path (str) – Test file path that the system will write the test functions to.

  • checklist_path (str, optional) – Argument to use non-default checklist during the operation.

  • model (str, optional) – Argument to specify a specific model to be used. Default is gpt-3.5-turbo.

  • verbose (bool, optional) – If provided, the system will print out evaluation results to standard output. Default is False.

  • debug (bool, optional) – If provided, the system will enable langchain’s debug mode to expose all debug messages to the standard output.

evaluate(repo_path: str, save_response_to: str = None, export_report_to: str = None, checklist_path: str = None, model='gpt-3.5-turbo', verbose: bool = False, overwrite: bool = False, debug: bool = False, test_dirs: list[str] = None) → None

Evaluate a given repo based on the completeness of the test suites.

This will evaluate the completeness of the test suites given a git repository, based on a checklist which consists of items relevant to various aspects related to Data Science/Machine Learning projects. A JSON file containing the evaluation results would be saved. Optionally, you can provide the export_report_to flag to use the content of the JSON file to render a human-readable report in HTML or PDF format.

Parameters:

  • repo_path (str) – The path of the git repository to be analyzed.

  • save_response_to (str, optional) – If provided, the JSON file saved would be in the specified path instead of the default location.

  • export_report_to (str, optional) – If provided, the system will render the evaluation report to the specified path. The format of the evaluation report will be based on the extension provided in this path. The extensions must be either one of .html, .htm, .pdf, .qmd, or .md.

  • checklist_path (str, optional) – Argument to use non-default checklist during the operation.

  • test_dirs (list, optional) – A list of directories to indicate where the test files are located. If provided, only files inside these directories will be scanned. Otherwise, all files in the repository will be scanned, which is the default behaviour.

  • model (str, optional) – Argument to specify a specific model to be used. Default is gpt-3.5-turbo.

  • verbose (bool, optional) – If provided, the system will print out evaluation results to standard output. Default is False.

  • overwrite (bool, optional) – If provided, the system will not stop when attempting to overwrite an existing evaluation report.

  • debug (bool, optional) – If provided, the system will enable langchain’s debug mode to expose all debug messages to the standard output.

static list_tests(repo_path: str, test_dirs: list[str] = None)

List out all tests found in this repository.

Parameters:

  • repo_path (str) – The path of the git repository to be analyzed.

  • test_dirs (list, optional) – A list of directories to indicate where the test files are located. If provided, only files inside these directories will be scanned. Otherwise, all files in the repository will be scanned, which is the default behaviour.