drytorch.lib.evaluations

Module containing classes for the evaluation of a model.

Classes

`Diagnostic`(model[, name])	Evaluate the model on inference mode without logging the metrics.
`EvaluationMixin`(args, *kwargs)	Mixin for running inference in eval mode without gradients.
`Test`(model[, name])	Evaluate model performance on a test dataset.
`Validation`(model[, name])	Evaluate model on inference mode.

class Diagnostic(model: ModelProtocol[Input, Output], name: str = '', *, loader: LoaderProtocol[tuple[Input, Target]], objective: _Objective_co)[source]

Bases: EvaluationMixin, ModelRunnerWithLogs[Input, Target, Output, Any]

Evaluate the model on inference mode without logging the metrics.

model

the model containing the weights to evaluate.

Type:: drytorch.core.protocols.ModelProtocol[drytorch.lib.runners.Input, drytorch.lib.runners.Output]

loader

provides inputs and targets in batches.

Type:: drytorch.core.protocols.LoaderProtocol[tuple[drytorch.lib.runners.Input, drytorch.lib.runners.Target]]

objective

processes the model outputs and targets.

Type:: drytorch.lib.runners._Objective_co

outputs_list

list of optionally stored outputs.

Type:: list[drytorch.lib.runners.Output]

Initialize.

Parameters:

model (ModelProtocol[Input, Output]) – the model containing the weights to evaluate.
name (str) – the name for the object for logging purposes. Defaults to class name plus eventual counter.
loader (LoaderProtocol[tuple[Input, Target]]) – provides inputs and targets in batches.
objective (_Objective_co) – processes the model outputs and targets.

class EvaluationMixin(*args, **kwargs)[source]

Bases: MonitorProtocol

Mixin for running inference in eval mode without gradients.

__call__(store_outputs: bool = False) → None[source]

Set the model in evaluation mode and PyTorch in inference mode.

Parameters:: store_outputs (bool)
Return type:: None

class Test(model: ModelProtocol[Input, Output], name: str = '', *, loader: LoaderProtocol[tuple[Input, Target]], metric: ObjectiveProtocol[Output, Target])[source]

Bases: Validation[Input, Target, Output]

Evaluate model performance on a test dataset.

model

the model containing the weights to evaluate.

Type:: drytorch.core.protocols.ModelProtocol[drytorch.lib.runners.Input, drytorch.lib.runners.Output]

loader

provides inputs and targets in batches.

Type:: drytorch.core.protocols.LoaderProtocol[tuple[drytorch.lib.runners.Input, drytorch.lib.runners.Target]]

objective

processes the model outputs and targets.

Type:: drytorch.lib.runners._Objective_co

outputs_list

list of optionally stored outputs.

Type:: list[drytorch.lib.runners.Output]

Initialize.

Parameters:

model (ModelProtocol[Input, Output]) – the model containing the weights to evaluate.
name (str) – the name for the object for logging purposes. Defaults to class name plus eventual counter.
loader (LoaderProtocol[tuple[Input, Target]]) – provides inputs and targets in batches.
metric (ObjectiveProtocol[Output, Target]) – metric to evaluate the model.

__call__(store_outputs: bool = False) → None[source]

Test the model on the dataset.

Parameters:: store_outputs (bool) – whether to store model outputs. Defaults to False.
Return type:: None

class Validation(model: ModelProtocol[Input, Output], name: str = '', *, loader: LoaderProtocol[tuple[Input, Target]], metric: ObjectiveProtocol[Output, Target])[source]

Bases: EvaluationMixin, ModelRunnerWithLogs[Input, Target, Output, Any]

Evaluate model on inference mode.

It could be used for testing (see subclass) or validating a model.

model

the model containing the weights to evaluate.

Type:: drytorch.core.protocols.ModelProtocol[drytorch.lib.runners.Input, drytorch.lib.runners.Output]

loader

provides inputs and targets in batches.

Type:: drytorch.core.protocols.LoaderProtocol[tuple[drytorch.lib.runners.Input, drytorch.lib.runners.Target]]

objective

processes the model outputs and targets.

Type:: drytorch.lib.runners._Objective_co

outputs_list

list of optionally stored outputs.

Type:: list[drytorch.lib.runners.Output]

Initialize.

Parameters:

model (ModelProtocol[Input, Output]) – the model containing the weights to evaluate.
name (str) – the name for the object for logging purposes. Defaults to class name plus eventual counter.
loader (LoaderProtocol[tuple[Input, Target]]) – provides inputs and targets in batches.
metric (ObjectiveProtocol[Output, Target]) – metric to evaluate the model.