sift_client.util.test_results.context_manager ¶

CLASS	DESCRIPTION
`NewStep`	Context manager to create a new step in a test report. See usage example in init.py.
`ReportContext`	Context manager for a new TestReport. See usage example in init.py.

FUNCTION	DESCRIPTION
`format_assertion_message`	Format an ErrorInfo from just the exception line(s), no traceback frames.
`format_truncated_traceback`	Format an ErrorInfo from a traceback, keeping the first frame and the last 10.
`log_replay_instructions`	Surface replay instructions when an import/replay attempt fails.

ATTRIBUTE	DESCRIPTION
`logger`

logger `module-attribute` ¶

logger = getLogger(__name__)

NewStep ¶

NewStep(
    report_context: ReportContext,
    name: str,
    description: str | None = None,
    assertion_as_fail_not_error: bool = True,
    metadata: dict[str, str | float | bool] | None = None,
)

Bases: AbstractContextManager

Context manager to create a new step in a test report. See usage example in init.py.

Initialize a new step context.

PARAMETER	DESCRIPTION
`report_context`	The report context to create the step in. TYPE: `ReportContext`
`name`	The name of the step. TYPE: `str`
`description`	The description of the step. TYPE: `str \| None` DEFAULT: `None`
`assertion_as_fail_not_error`	Mark steps with assertion errors as failed instead of error+traceback (some users want assertions to work as simple failures especially when using pytest). TYPE: `bool` DEFAULT: `True`
`metadata`	[Optional] Structured key/value metadata to attach to the step. TYPE: `dict[str, str \| float \| bool] \| None` DEFAULT: `None`

METHOD	DESCRIPTION
`fail_if_measurements_failed`	Fail the pytest test if any measurement on this step was out of bounds.
`measure`	Measure a value and return the result.
`measure_all`	Ensure that all values in a list are within bounds and return the result. Records measurements for all values outside the bounds.
`measure_avg`	Calculate the average of a list of values, measure the average against given bounds, and return the result.
`report_outcome`	Report an outcome from some action or measurement. Creates a substep that is pass/fail with the optional reason as the description.
`substep`	Alias to return a new step context manager from the current step. The ReportContext will manage nesting of steps.
`update_step_from_result`	Update the step based on its substeps and if there was an exception while executing the step.

ATTRIBUTE	DESCRIPTION
`assertion_as_fail_not_error`	TYPE: `bool`
`client`	TYPE: `SiftClient`
`current_step`	TYPE: `TestStep \| None`
`measurements_passed`	True if every measurement recorded directly on this step has passed. TYPE: `bool`
`report_context`	TYPE: `ReportContext`

assertion_as_fail_not_error `class-attribute` `instance-attribute` ¶

assertion_as_fail_not_error: bool = (
    assertion_as_fail_not_error
)

client `instance-attribute` ¶

client: SiftClient = client

current_step `class-attribute` `instance-attribute` ¶

current_step: TestStep | None = create_step(
    name, description, metadata=metadata
)

measurements_passed `property` ¶

measurements_passed: bool

True if every measurement recorded directly on this step has passed.

Counts only step.measure, step.measure_avg, and step.measure_all calls on this NewStep instance. Pair it with fail_if_measurements_failed() at the end of a test to fail pytest on any out-of-bounds measurement without short-circuiting on the first failure (asserting on individual measure(...) return values skips every measurement after the failing one).

report_context `instance-attribute` ¶

report_context: ReportContext = report_context

fail_if_measurements_failed ¶

fail_if_measurements_failed(
    message: str = "measurements out of bounds",
) -> None

Fail the pytest test if any measurement on this step was out of bounds.

Use instead of assert step.measurements_passed: it fails via pytest.fail so the step resolves to FAILED without attaching an assertion message to error_info. No-op when every measurement passed. Call once at the end of the test so every measurement is still recorded before the failure fires.

The failure message names each out-of-bounds measurement with its recorded value and bounds. message is used as the header line.

measure ¶

measure(
    *,
    name: str,
    value: float | str | bool | int,
    bounds: dict[str, float]
    | NumericBounds
    | str
    | None = None,
    timestamp: datetime | None = None,
    unit: str | None = None,
    description: str | None = None,
    metadata: dict[str, str | float | bool] | None = None,
    channel_names: list[str] | list[Channel] | None = None,
) -> bool

Measure a value and return the result.

PARAMETER	DESCRIPTION
`name`	The name of the measurement. TYPE: `str`
`value`	The value of the measurement. TYPE: `float \| str \| bool \| int`
`bounds`	[Optional] The bounds to compare the value to. TYPE: `dict[str, float] \| NumericBounds \| str \| None` DEFAULT: `None`
`timestamp`	[Optional] The timestamp of the measurement. Defaults to the current time. TYPE: `datetime \| None` DEFAULT: `None`
`unit`	[Optional] The unit of the measurement. TYPE: `str \| None` DEFAULT: `None`
`description`	[Optional] Notes about the measurement. Server caps at 2000 characters; longer strings are truncated with a warning. TYPE: `str \| None` DEFAULT: `None`
`metadata`	[Optional] Structured key/value metadata to attach to the measurement. For metadata shared across measurements, prefer the `metadata` attribute of the enclosing `TestStep` or `TestReport`. TYPE: `dict[str, str \| float \| bool] \| None` DEFAULT: `None`
`channel_names`	[Optional] Sift channel names or `Channel` instances this measurement is associated with. Enables cross-plotting in Explore using the report's associated Run. TYPE: `list[str] \| list[Channel] \| None` DEFAULT: `None`

returns: The result of the measurement.

measure_all ¶

measure_all(
    *,
    name: str,
    values: list[float | int] | NDArray[float64] | Series,
    bounds: dict[str, float] | NumericBounds,
    timestamp: datetime | None = None,
    unit: str | None = None,
    description: str | None = None,
    metadata: dict[str, str | float | bool] | None = None,
    channel_names: list[str] | list[Channel] | None = None,
) -> bool

Ensure that all values in a list are within bounds and return the result. Records measurements for all values outside the bounds.

Note: Measurements will only be recorded for values outside the bounds. To record measurements for all values, just call measure for each value.

PARAMETER	DESCRIPTION
`name`	The name of the measurement. TYPE: `str`
`values`	The list of values to measure the average of. TYPE: `list[float \| int] \| NDArray[float64] \| Series`
`bounds`	The bounds to compare the value to. TYPE: `dict[str, float] \| NumericBounds`
`timestamp`	[Optional] The timestamp of the measurement. Defaults to the current time. TYPE: `datetime \| None` DEFAULT: `None`
`unit`	[Optional] The unit of the measurement. TYPE: `str \| None` DEFAULT: `None`
`description`	[Optional] Notes attached to each out-of-bounds measurement. Server caps at 2000 characters; longer strings are truncated with a warning. TYPE: `str \| None` DEFAULT: `None`
`metadata`	[Optional] Structured key/value metadata for each out-of-bounds measurement. TYPE: `dict[str, str \| float \| bool] \| None` DEFAULT: `None`
`channel_names`	[Optional] Sift channel names or `Channel` instances to associate with each out-of-bounds measurement. TYPE: `list[str] \| list[Channel] \| None` DEFAULT: `None`

returns: The true if all values are within the bounds, false otherwise.

measure_avg ¶

measure_avg(
    *,
    name: str,
    values: list[float | int] | NDArray[float64] | Series,
    bounds: dict[str, float] | NumericBounds,
    timestamp: datetime | None = None,
    unit: str | None = None,
    description: str | None = None,
    metadata: dict[str, str | float | bool] | None = None,
    channel_names: list[str] | list[Channel] | None = None,
) -> bool

Calculate the average of a list of values, measure the average against given bounds, and return the result.

PARAMETER	DESCRIPTION
`name`	The name of the measurement. TYPE: `str`
`values`	The list of values to measure the average of. TYPE: `list[float \| int] \| NDArray[float64] \| Series`
`bounds`	The bounds to compare the value to. TYPE: `dict[str, float] \| NumericBounds`
`timestamp`	[Optional] The timestamp of the measurement. Defaults to the current time. TYPE: `datetime \| None` DEFAULT: `None`
`unit`	[Optional] The unit of the measurement. TYPE: `str \| None` DEFAULT: `None`
`description`	[Optional] Notes about the measurement. Server caps at 2000 characters; longer strings are truncated with a warning. TYPE: `str \| None` DEFAULT: `None`
`metadata`	[Optional] Structured key/value metadata to attach to the measurement. TYPE: `dict[str, str \| float \| bool] \| None` DEFAULT: `None`
`channel_names`	[Optional] Sift channel names or `Channel` instances this measurement is associated with. TYPE: `list[str] \| list[Channel] \| None` DEFAULT: `None`

returns: The true if the average of the values is within the bounds, false otherwise.

report_outcome ¶

report_outcome(
    name: str, result: bool, reason: str | None = None
) -> bool

Report an outcome from some action or measurement. Creates a substep that is pass/fail with the optional reason as the description.

PARAMETER	DESCRIPTION
`name`	The name of the substep. TYPE: `str`
`result`	True if the action or measurement passed, False otherwise. TYPE: `bool`
`reason`	[Optional] The context to include in the description of the substep. TYPE: `str \| None` DEFAULT: `None`

returns: The given result so the function can be used in line.

substep ¶

substep(
    name: str,
    description: str | None = None,
    metadata: dict[str, str | float | bool] | None = None,
) -> NewStep

Alias to return a new step context manager from the current step. The ReportContext will manage nesting of steps.

update_step_from_result ¶

update_step_from_result(
    exc: type[Exception] | None,
    exc_value: Exception | None,
    tb: TracebackException | None,
) -> bool

Update the step based on its substeps and if there was an exception while executing the step.

PARAMETER	DESCRIPTION
`exc`	The class of Exception that was raised. TYPE: `type[Exception] \| None`
`exc_value`	The exception value. TYPE: `Exception \| None`
`tb`	The traceback object. TYPE: `TracebackException \| None`

returns: The false if step failed or errored, true otherwise.

ReportContext ¶

ReportContext(
    client: SiftClient,
    name: str,
    test_system_name: str | None = None,
    system_operator: str | None = None,
    test_case: str | None = None,
    serial_number: str | None = None,
    part_number: str | None = None,
    log_file: str | Path | bool | None = None,
    include_git_metadata: bool = False,
    replay_log_file: bool = True,
    metadata: dict[str, str | float | bool] | None = None,
)

Bases: AbstractContextManager

Context manager for a new TestReport. See usage example in init.py.

Initialize a new report context.

PARAMETER	DESCRIPTION
`client`	The Sift client to use to create the report. TYPE: `SiftClient`
`name`	The name of the report. TYPE: `str`
`test_system_name`	The name of the test system. Will default to the hostname if not provided. TYPE: `str \| None` DEFAULT: `None`
`system_operator`	The operator of the test system. Will default to the current user if not provided. TYPE: `str \| None` DEFAULT: `None`
`test_case`	The name of the test case. Will default to the basename of the file containing the test if not provided. TYPE: `str \| None` DEFAULT: `None`
`serial_number`	Optional serial_number stored on the report. Unset when None. TYPE: `str \| None` DEFAULT: `None`
`part_number`	Optional part_number stored on the report. Unset when None. TYPE: `str \| None` DEFAULT: `None`
`log_file`	If True, create a temp log file. If a path, use that path. If False/None, no log file is written and create/update calls the API. TYPE: `str \| Path \| bool \| None` DEFAULT: `None`
`include_git_metadata`	If True, include git metadata in the report. TYPE: `bool` DEFAULT: `False`
`metadata`	Structured key/value metadata to attach to the report. Merged on top of git metadata when `include_git_metadata` is True, so explicit keys win on collision. TYPE: `dict[str, str \| float \| bool] \| None` DEFAULT: `None`
`replay_log_file`	When True (the default) and `log_file` is set, spawn `import-test-result-log --incremental` to push log entries to Sift in the background during the session. When False, the log file is just a record and no worker is spawned. Replay happens later via `replay-test-result-log <path>`. Has no effect when `log_file` is None. TYPE: `bool` DEFAULT: `True`

METHOD	DESCRIPTION
`create_step`	Create a new step in the report context.
`exit_step`	Exit a step and update the report context.
`get_next_step_path`	Get the next step path for the current depth.
`mark_step_failed_after_close`	Mark a step's parent as failed after the step has already been popped from the stack.
`new_step`	Alias to return a new step context manager from this report context. Use create_step for actually creating a TestStep in the current context.
`propagate_step_result`	Propagate this step's final status to the parent step.
`record_measurement`	Retain a recorded measurement for end-of-run summaries.
`record_step_outcome`	Report a failure to the report context.

ATTRIBUTE	DESCRIPTION
`any_failures`	TYPE: `bool`
`client`	TYPE: `SiftClient`
`created_measurements`	TYPE: `list[TestMeasurement]`
`created_steps`	TYPE: `list[TestStep]`
`is_simulated`	True when this context's report came from the simulate path. TYPE: `bool`
`log_file`	TYPE: `Path \| None`
`measurement_counts`	Tally of recorded measurements keyed by `passed` (True/False). TYPE: `Counter[bool]`
`open_step_results`	TYPE: `dict[str, bool]`
`replay_incomplete`	TYPE: `bool`
`replay_log_file`
`report`	TYPE: `TestReport`
`step_is_open`	TYPE: `bool`
`step_number_at_depth`	TYPE: `dict[int, int]`
`step_stack`	TYPE: `list[TestStep]`
`step_status_counts`	Tally of every created step by its current status. TYPE: `Counter[TestStatus]`

any_failures `instance-attribute` ¶

any_failures: bool = False

client `instance-attribute` ¶

client: SiftClient = client

created_measurements `instance-attribute` ¶

created_measurements: list[TestMeasurement] = []

created_steps `instance-attribute` ¶

created_steps: list[TestStep] = []

is_simulated `property` ¶

is_simulated: bool

True when this context's report came from the simulate path.

Delegates to self.report.is_simulated; see TestReport.is_simulated for the full semantics.

log_file `instance-attribute` ¶

log_file: Path | None

measurement_counts `property` ¶

measurement_counts: Counter[bool]

Tally of recorded measurements keyed by passed (True/False).

Read at the end of a run for summaries.

open_step_results `instance-attribute` ¶

open_step_results: dict[str, bool] = {}

replay_incomplete `class-attribute` `instance-attribute` ¶

replay_incomplete: bool = False

replay_log_file `instance-attribute` ¶

replay_log_file = replay_log_file

report `instance-attribute` ¶

report: TestReport = create(create, log_file=log_file)

step_is_open `instance-attribute` ¶

step_is_open: bool = False

step_number_at_depth `instance-attribute` ¶

step_number_at_depth: dict[int, int] = {}

step_stack `instance-attribute` ¶

step_stack: list[TestStep] = []

step_status_counts `property` ¶

step_status_counts: Counter[TestStatus]

Tally of every created step by its current status.

Includes hierarchy/parametrize parent steps. Read at the end of a run for summaries; reflects late status changes since steps are mutated in place.

create_step ¶

create_step(
    name: str,
    description: str | None = None,
    metadata: dict[str, str | float | bool] | None = None,
) -> TestStep

Create a new step in the report context.

PARAMETER	DESCRIPTION
`name`	The name of the step. TYPE: `str`
`description`	The description of the step. TYPE: `str \| None` DEFAULT: `None`
`metadata`	[Optional] Structured key/value metadata to attach to the step. For metadata shared across every step in a report, prefer the `metadata` attribute of the enclosing `TestReport`. TYPE: `dict[str, str \| float \| bool] \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`TestStep`	The created step.

exit_step ¶

exit_step(step: TestStep)

Exit a step and update the report context.

get_next_step_path ¶

get_next_step_path() -> str

Get the next step path for the current depth.

mark_step_failed_after_close ¶

mark_step_failed_after_close(step: TestStep)

Mark a step's parent as failed after the step has already been popped from the stack.

Used by the pytest plugin when a teardown-phase report fires after the fixture's __exit__ has already resolved and exited the step.

new_step ¶

new_step(
    name: str,
    description: str | None = None,
    assertion_as_fail_not_error: bool = True,
    metadata: dict[str, str | float | bool] | None = None,
) -> NewStep

Alias to return a new step context manager from this report context. Use create_step for actually creating a TestStep in the current context.

propagate_step_result ¶

propagate_step_result(
    step: TestStep, status: TestStatus
) -> bool

Propagate this step's final status to the parent step.

Status is the governor: anything outside {PASSED, SKIPPED} counts as a failure for the parent. error_info is intentionally not consulted here; it is free-form diagnostic data that may sit on a step regardless of status.

record_measurement ¶

record_measurement(measurement: TestMeasurement) -> None

Retain a recorded measurement for end-of-run summaries.

record_step_outcome ¶

record_step_outcome(outcome: bool, step: TestStep)

Report a failure to the report context.

format_assertion_message ¶

format_assertion_message(
    exc: type[BaseException] | None,
    exc_value: BaseException | None,
) -> ErrorInfo

Format an ErrorInfo from just the exception line(s), no traceback frames.

For assertion failures the rewritten assert explanation lives on the exception itself, so stack frames add noise without information. Equivalent to pytest's excinfo.exconly().

format_truncated_traceback ¶

format_truncated_traceback(
    exc: type[BaseException] | None,
    exc_value: BaseException | None,
    tb: object | None,
) -> ErrorInfo

Format an ErrorInfo from a traceback, keeping the first frame and the last 10.

log_replay_instructions ¶

log_replay_instructions(
    log_file: str | Path | None,
) -> None

Surface replay instructions when an import/replay attempt fails.

Emitted as a SiftWarning (not a logger.error) so pytest and other runners surface it in their warning summary; logger.error is suppressed by default in most CLI tools.

sift_client.util.test_results.context_manager ¶

logger module-attribute ¶

NewStep ¶

assertion_as_fail_not_error class-attribute instance-attribute ¶

client instance-attribute ¶

current_step class-attribute instance-attribute ¶

measurements_passed property ¶

report_context instance-attribute ¶

fail_if_measurements_failed ¶

measure ¶

measure_all ¶

measure_avg ¶

report_outcome ¶

substep ¶

update_step_from_result ¶

ReportContext ¶

any_failures instance-attribute ¶

client instance-attribute ¶

created_measurements instance-attribute ¶

created_steps instance-attribute ¶

is_simulated property ¶

log_file instance-attribute ¶

measurement_counts property ¶

open_step_results instance-attribute ¶

replay_incomplete class-attribute instance-attribute ¶

replay_log_file instance-attribute ¶

report instance-attribute ¶

step_is_open instance-attribute ¶

step_number_at_depth instance-attribute ¶

step_stack instance-attribute ¶

step_status_counts property ¶

create_step ¶

exit_step ¶

get_next_step_path ¶

mark_step_failed_after_close ¶

new_step ¶

propagate_step_result ¶

record_measurement ¶

record_step_outcome ¶

format_assertion_message ¶

format_truncated_traceback ¶

log_replay_instructions ¶

logger `module-attribute` ¶

assertion_as_fail_not_error `class-attribute` `instance-attribute` ¶

client `instance-attribute` ¶

current_step `class-attribute` `instance-attribute` ¶

measurements_passed `property` ¶

report_context `instance-attribute` ¶

any_failures `instance-attribute` ¶

client `instance-attribute` ¶

created_measurements `instance-attribute` ¶

created_steps `instance-attribute` ¶

is_simulated `property` ¶

log_file `instance-attribute` ¶

measurement_counts `property` ¶

open_step_results `instance-attribute` ¶

replay_incomplete `class-attribute` `instance-attribute` ¶

replay_log_file `instance-attribute` ¶

report `instance-attribute` ¶

step_is_open `instance-attribute` ¶

step_number_at_depth `instance-attribute` ¶

step_stack `instance-attribute` ¶

step_status_counts `property` ¶