autotorch

Real

class autotorch.Real(lower, upper, default=None, log=False)

linear search space.

Parameters

• lower (float) – the lower bound of the search space

• upper (float) – the upper bound of the search space

• default (float (optional)) – default value

• log ((True/False)) – search space in log scale

Examples

>>> learning_rate = at.Real(0.01, 0.1, log=True)
>>> learning_rate.rand
0.013396492756434304

Int

class autotorch.Int(lower, upper, default=None)

integer search space.

Parameters

• lower (int) – the lower bound of the search space

• upper (int) – the upper bound of the search space

• default (int (optional)) – default value

Examples

>>> range = at.Int(0, 100)
>>> range.rand
82

Choice

class autotorch.Choice(*data)

Choice Search Space (Nested Space) Add example for conditional space.

Parameters

data (Space or python built-in objects) – the choice candidates

Examples

>>> a = at.Choice('a', 'b', 'c', 'd')
>>> a.rand
'a'

List

class autotorch.List(*args)

A Searchable List (Nested Space)

Parameters

args (list) – a list of search spaces.

Examples

>>> sequence = at.List(
>>>     at.Choice('conv3x3', 'conv5x5', 'conv7x7'),
>>>     at.Choice('BatchNorm', 'InstanceNorm'),
>>>     at.Choice('relu', 'sigmoid'),
>>> )
>>> sequence.rand
['conv3x3', 'InstanceNorm', 'relu']

Dict

class autotorch.Dict(**kwargs)

A Searchable Dict (Nested Space)

Examples

>>> g = at.Dict(
>>>         key1=at.Choice('alpha', 'beta'),
>>>         key2=at.Int(0, 3),
>>>     )
>>> g.rand
{'key1': 'alpha', 'key2': 1}

Bool

class autotorch.Bool

Bool Search Space

Examples

>>> pretrained = at.Bool()
>>> pretrained.rand
False

args

autotorch.args(default={}, **kwvars)

Decorator for customized training script, registering arguments or searchable spaces to the decorated function. The arguments should be python built-in objects, autotorch objects (see autotorch.obj() .), or autotorch search spaces (autotorch.space.Int, autotorch.space.Real …).

Examples

>>> @at.args(batch_size=10, lr=at.Real(0.01, 0.1))
>>> def my_train(args):
...     print('Batch size is {}, LR is {}'.format(args.batch_size, arg.lr))

obj

autotorch.obj(**kwvars)

Register args or searchable spaces to the class.

Returns

a lazy init object, which allows distributed training.

Return type

instance of autotorch.space.AutoTorchObject

Examples

>>> import autotorch as at
>>> import torch
>>> @at.obj(
>>>     lr=at.Real(1e-4, 1e-1, log=True),
>>>     weight_decay=at.Real(1e-4, 1e-1),
>>> )
>>> class Adam(torch.optim.Adam):
>>>     pass

func

autotorch.func(**kwvars)

Register args or searchable spaces to the functions.

Returns

a lazy init object, which allows distributed training.

Return type

instance of autotorch.space.AutoTorchObject

Examples

>>> torchvision.models as models
>>>
>>> @at.func(pretrained=at.Bool())
>>> def resnet18(pretrained):
...     return models.resnet18(pretrained=pretrained)