ml_switcheroo.testing.batch_runner ================================== .. py:module:: ml_switcheroo.testing.batch_runner .. autoapi-nested-parse:: Batch Validation Runner. Iterates over all defined semantic operations and verifys their correctness. Prioritizes "Manual/Human" tests found on disk over "Auto/Robotic" fuzzing. This closes the loop for Workflow B, allowing manual fixes to count as success. Updated to handle Rich Parameter Constraints (dict items in std_args), including Dtype and Rank constraints, and **Input-Dependent Output Shape** verification. Classes ------- .. autoapisummary:: ml_switcheroo.testing.batch_runner.BatchValidator Module Contents --------------- .. py:class:: BatchValidator(semantics: ml_switcheroo.semantics.manager.SemanticsManager) Orchestrates the validation of the entire knowledge base. .. py:attribute:: semantics .. py:attribute:: runner .. py:method:: run_all(verbose: bool = False, manual_test_dir: Optional[pathlib.Path] = None) -> Dict[str, bool] Runs verification for all known APIs. Validation Hierarchy: 1. **Manual Test Priority**: If a file contains `def test_():`, we assume the developer has manually verified this operation. It is marked as Passing. 2. **Automated Fuzzing**: If no manual test exists, we unpack the arguments, type hints, and constraints from the spec, generate random inputs, and check equivalence using `EquivalenceRunner`. :param verbose: Show progress bar using Rich. :param manual_test_dir: Directory containing manual tests to scan (default: None). :returns: Validation status (True=Pass, False=Fail). :rtype: Dict[op_name, bool]