Fork me on GitHub Top

Store and load skopt optimization results

Mikhail Pak, October 2016.

import numpy as np
np.random.seed(777)

Problem statement

We often want to store optimization results in a file. This can be useful, for example,

The process of converting an object into a byte stream that can be stored in a file is called serialization. Conversely, deserialization means loading an object from a byte stream.

Warning: Deserialization is not secure against malicious or erroneous code. Never load serialized data from untrusted or unauthenticated sources!

Simple example

We will use the same optimization problem as in the bayesian-optimization.ipynb notebook:

from skopt import gp_minimize

noise_level = 0.1

def obj_fun(x, noise_level=noise_level):
    return np.sin(5 * x[0]) * (1 - np.tanh(x[0] ** 2)) + np.random.randn() * noise_level

res = gp_minimize(obj_fun,            # the function to minimize
                  [(-2.0, 2.0)],      # the bounds on each dimension of x
                  x0=[0.],            # the starting point
                  acq_func="LCB",     # the acquisition function (optional)
                  n_calls=15,         # the number of evaluations of f including at x0
                  n_random_starts=0,  # the number of random initialization points
                  random_state=777)
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "
/home/ubuntu/scikit-optimize/skopt/optimizer/optimizer.py:367: UserWarning: The objective has been evaluated at this point before.
  warnings.warn("The objective has been evaluated "

As long as your Python session is active, you can access all the optimization results via the res object.

So how can you store this data in a file? skopt conveniently provides functions skopt.dump() and skopt.load() that handle this for you. These functions are essentially thin wrappers around the joblib module's dump() and load().

We will now show how to use skopt.dump() and skopt.load() for storing and loading results.

Using skopt.dump() and skopt.load()

For storing optimization results into a file, call the skopt.dump() function:

from skopt import dump, load

dump(res, 'result.pkl')

And load from file using skopt.load():

res_loaded = load('result.pkl')

res_loaded.fun
-0.17487957729512074

You can fine-tune the serialization and deserialization process by calling skopt.dump() and skopt.load() with additional keyword arguments. See the joblib documentation (dump and load) for the additional parameters.

For instance, you can specify the compression algorithm and compression level (highest in this case):

dump(res, 'result.gz', compress=9)

from os.path import getsize
print('Without compression: {} bytes'.format(getsize('result.pkl')))
print('Compressed with gz:  {} bytes'.format(getsize('result.gz')))
Without compression: 83178 bytes
Compressed with gz:  22643 bytes

Unserializable objective functions

Notice that if your objective function is non-trivial (e.g. it calls MATLAB engine from Python), it might be not serializable and skopt.dump() will raise an exception when you try to store the optimization results. In this case you should disable storing the objective function by calling skopt.dump() with the keyword argument store_objective=False:

dump(res, 'result_without_objective.pkl', store_objective=False)

Notice that the entry 'func' is absent in the loaded object but is still present in the local variable:

res_loaded_without_objective = load('result_without_objective.pkl')

print('Loaded object: ', res_loaded_without_objective.specs['args'].keys())
print('Local variable:', res.specs['args'].keys())
Loaded object:  dict_keys(['n_jobs', 'kappa', 'xi', 'n_restarts_optimizer', 'n_points', 'callback', 'verbose', 'random_state', 'y0', 'x0', 'acq_optimizer', 'acq_func', 'n_random_starts', 'n_calls', 'base_estimator', 'dimensions'])
Local variable: dict_keys(['n_jobs', 'kappa', 'xi', 'n_restarts_optimizer', 'n_points', 'callback', 'verbose', 'random_state', 'y0', 'x0', 'acq_optimizer', 'acq_func', 'n_random_starts', 'n_calls', 'base_estimator', 'dimensions', 'func'])

Possible problems

del res.specs['args']['func']

dump(res, 'result_without_objective_2.pkl')