sift_client.util.test_results.context_manager ¶

CLASS	DESCRIPTION
`NewStep`	Context manager to create a new step in a test report. See usage example in init.py.
`ReportContext`	Context manager for a new TestReport. See usage example in init.py.

NewStep ¶

NewStep(
    report_context: ReportContext,
    name: str,
    description: str | None = None,
    assertion_as_fail_not_error: bool = True,
)

Bases: AbstractContextManager

Context manager to create a new step in a test report. See usage example in init.py.

Initialize a new step context.

PARAMETER	DESCRIPTION
`report_context`	The report context to create the step in. TYPE: `ReportContext`
`name`	The name of the step. TYPE: `str`
`description`	The description of the step. TYPE: `str \| None` DEFAULT: `None`
`assertion_as_fail_not_error`	Mark steps with assertion errors as failed instead of error+traceback (some users want assertions to work as simple failures especially when using pytest). TYPE: `bool` DEFAULT: `True`

METHOD	DESCRIPTION
`__enter__`	Enter the context manager to create a new step.
`__exit__`
`measure`	Measure a value and return the result.
`measure_all`	Ensure that all values in a list are within bounds and return the result. Records measurements for all values outside the bounds.
`measure_avg`	Calculate the average of a list of values, measure the average against given bounds, and return the result.
`report_outcome`	Report an outcome from some action or measurement. Creates a substep that is pass/fail with the optional reason as the description.
`substep`	Alias to return a new step context manager from the current step. The ReportContext will manage nesting of steps.
`update_step_from_result`	Update the step based on its substeps and if there was an exception while executing the step.

ATTRIBUTE	DESCRIPTION
`assertion_as_fail_not_error`	TYPE: `bool`
`client`	TYPE: `SiftClient`
`current_step`	TYPE: `TestStep \| None`
`report_context`	TYPE: `ReportContext`

assertion_as_fail_not_error `class-attribute` `instance-attribute` ¶

assertion_as_fail_not_error: bool = (
    assertion_as_fail_not_error
)

client `instance-attribute` ¶

client: SiftClient = client

current_step `class-attribute` `instance-attribute` ¶

current_step: TestStep | None = create_step(
    name, description
)

report_context `instance-attribute` ¶

report_context: ReportContext = report_context

enter ¶

__enter__()

Enter the context manager to create a new step.

returns: The current step.

exit ¶

__exit__(exc, exc_value, tb)

measure ¶

measure(
    *,
    name: str,
    value: float | str | bool | int,
    bounds: dict[str, float]
    | NumericBounds
    | str
    | None = None,
    timestamp: datetime | None = None,
    unit: str | None = None,
) -> bool

Measure a value and return the result.

PARAMETER	DESCRIPTION
`name`	The name of the measurement. TYPE: `str`
`value`	The value of the measurement. TYPE: `float \| str \| bool \| int`
`bounds`	[Optional] The bounds to compare the value to. TYPE: `dict[str, float] \| NumericBounds \| str \| None` DEFAULT: `None`
`timestamp`	[Optional] The timestamp of the measurement. Defaults to the current time. TYPE: `datetime \| None` DEFAULT: `None`
`unit`	[Optional] The unit of the measurement. TYPE: `str \| None` DEFAULT: `None`

returns: The result of the measurement.

measure_all ¶

measure_all(
    *,
    name: str,
    values: list[float | int] | NDArray[float64] | Series,
    bounds: dict[str, float] | NumericBounds,
    timestamp: datetime | None = None,
    unit: str | None = None,
) -> bool

Ensure that all values in a list are within bounds and return the result. Records measurements for all values outside the bounds.

Note: Measurements will only be recorded for values outside the bounds. To record measurements for all values, just call measure for each value.

PARAMETER	DESCRIPTION
`name`	The name of the measurement. TYPE: `str`
`values`	The list of values to measure the average of. TYPE: `list[float \| int] \| NDArray[float64] \| Series`
`bounds`	The bounds to compare the value to. TYPE: `dict[str, float] \| NumericBounds`
`timestamp`	[Optional] The timestamp of the measurement. Defaults to the current time. TYPE: `datetime \| None` DEFAULT: `None`
`unit`	[Optional] The unit of the measurement. TYPE: `str \| None` DEFAULT: `None`

returns: The true if all values are within the bounds, false otherwise.

measure_avg ¶

measure_avg(
    *,
    name: str,
    values: list[float | int] | NDArray[float64] | Series,
    bounds: dict[str, float] | NumericBounds,
    timestamp: datetime | None = None,
    unit: str | None = None,
) -> bool

Calculate the average of a list of values, measure the average against given bounds, and return the result.

PARAMETER	DESCRIPTION
`name`	The name of the measurement. TYPE: `str`
`values`	The list of values to measure the average of. TYPE: `list[float \| int] \| NDArray[float64] \| Series`
`bounds`	The bounds to compare the value to. TYPE: `dict[str, float] \| NumericBounds`
`timestamp`	[Optional] The timestamp of the measurement. Defaults to the current time. TYPE: `datetime \| None` DEFAULT: `None`
`unit`	[Optional] The unit of the measurement. TYPE: `str \| None` DEFAULT: `None`

returns: The true if the average of the values is within the bounds, false otherwise.

report_outcome ¶

report_outcome(
    name: str, result: bool, reason: str | None = None
) -> bool

Report an outcome from some action or measurement. Creates a substep that is pass/fail with the optional reason as the description.

PARAMETER	DESCRIPTION
`name`	The name of the substep. TYPE: `str`
`result`	True if the action or measurement passed, False otherwise. TYPE: `bool`
`reason`	[Optional] The context to include in the description of the substep. TYPE: `str \| None` DEFAULT: `None`

returns: The given result so the function can be used in line.

substep ¶

substep(
    name: str, description: str | None = None
) -> NewStep

Alias to return a new step context manager from the current step. The ReportContext will manage nesting of steps.

update_step_from_result ¶

update_step_from_result(
    exc: type[Exception] | None,
    exc_value: Exception | None,
    tb: TracebackException | None,
) -> bool

Update the step based on its substeps and if there was an exception while executing the step.

PARAMETER	DESCRIPTION
`exc`	The class of Exception that was raised. TYPE: `type[Exception] \| None`
`exc_value`	The exception value. TYPE: `Exception \| None`
`tb`	The traceback object. TYPE: `TracebackException \| None`

returns: The false if step failed or errored, true otherwise.

ReportContext ¶

ReportContext(
    client: SiftClient,
    name: str,
    test_system_name: str | None = None,
    system_operator: str | None = None,
    test_case: str | None = None,
)

Bases: AbstractContextManager

Context manager for a new TestReport. See usage example in init.py.

Initialize a new report context.

PARAMETER	DESCRIPTION
`client`	The Sift client to use to create the report. TYPE: `SiftClient`
`name`	The name of the report. TYPE: `str`
`test_system_name`	The name of the test system. Will default to the hostname if not provided. TYPE: `str \| None` DEFAULT: `None`
`system_operator`	The operator of the test system. Will default to the current user if not provided. TYPE: `str \| None` DEFAULT: `None`
`test_case`	The name of the test case. Will default to the basename of the file containing the test if not provided. TYPE: `str \| None` DEFAULT: `None`

METHOD	DESCRIPTION
`__enter__`
`__exit__`
`create_step`	Create a new step in the report context.
`exit_step`	Exit a step and update the report context.
`get_next_step_path`	Get the next step path for the current depth.
`new_step`	Alias to return a new step context manager from this report context. Use create_step for actually creating a TestStep in the current context.
`record_step_outcome`	Report a failure to the report context.
`resolve_and_propagate_step_result`	Resolve the result of a step and propagate the result to the parent step if it failed.

ATTRIBUTE	DESCRIPTION
`any_failures`	TYPE: `bool`
`open_step_results`	TYPE: `dict[str, bool]`
`report`	TYPE: `TestReport`
`step_is_open`	TYPE: `bool`
`step_number_at_depth`	TYPE: `dict[int, int]`
`step_stack`	TYPE: `list[TestStep]`

any_failures `instance-attribute` ¶

any_failures: bool = False

open_step_results `instance-attribute` ¶

open_step_results: dict[str, bool] = {}

report `instance-attribute` ¶

report: TestReport = create(create)

step_is_open `instance-attribute` ¶

step_is_open: bool = False

step_number_at_depth `instance-attribute` ¶

step_number_at_depth: dict[int, int] = {}

step_stack `instance-attribute` ¶

step_stack: list[TestStep] = []

enter ¶

__enter__()

exit ¶

__exit__(exc_type, exc_value, traceback)

create_step ¶

create_step(
    name: str, description: str | None = None
) -> TestStep

Create a new step in the report context.

PARAMETER	DESCRIPTION
`name`	The name of the step. TYPE: `str`
`description`	The description of the step. TYPE: `str \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`TestStep`	The created step.

exit_step ¶

exit_step(step: TestStep)

Exit a step and update the report context.

get_next_step_path ¶

get_next_step_path() -> str

Get the next step path for the current depth.

new_step ¶

new_step(
    name: str,
    description: str | None = None,
    assertion_as_fail_not_error: bool = True,
) -> NewStep

Alias to return a new step context manager from this report context. Use create_step for actually creating a TestStep in the current context.

record_step_outcome ¶

record_step_outcome(outcome: bool, step: TestStep)

Report a failure to the report context.

resolve_and_propagate_step_result ¶

resolve_and_propagate_step_result(
    step: TestStep, error_info: ErrorInfo | None = None
) -> bool

Resolve the result of a step and propagate the result to the parent step if it failed.

sift_client.util.test_results.context_manager ¶

NewStep ¶

assertion_as_fail_not_error class-attribute instance-attribute ¶

client instance-attribute ¶

current_step class-attribute instance-attribute ¶

report_context instance-attribute ¶

__enter__ ¶

__exit__ ¶

measure ¶

measure_all ¶

measure_avg ¶

report_outcome ¶

substep ¶

update_step_from_result ¶

ReportContext ¶

any_failures instance-attribute ¶

open_step_results instance-attribute ¶

report instance-attribute ¶

step_is_open instance-attribute ¶

step_number_at_depth instance-attribute ¶

step_stack instance-attribute ¶

__enter__ ¶

__exit__ ¶

create_step ¶

exit_step ¶

get_next_step_path ¶

new_step ¶

record_step_outcome ¶

resolve_and_propagate_step_result ¶

assertion_as_fail_not_error `class-attribute` `instance-attribute` ¶

client `instance-attribute` ¶

current_step `class-attribute` `instance-attribute` ¶

report_context `instance-attribute` ¶

enter ¶

exit ¶

any_failures `instance-attribute` ¶

open_step_results `instance-attribute` ¶

report `instance-attribute` ¶

step_is_open `instance-attribute` ¶

step_number_at_depth `instance-attribute` ¶

step_stack `instance-attribute` ¶

enter ¶

exit ¶