.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/exploration-vs-exploitation.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code or to run this example in your browser via Binder .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_exploration-vs-exploitation.py: =========================== Exploration vs exploitation =========================== Sigurd Carlen, September 2019. Reformatted by Holger Nahrstaedt 2020 .. currentmodule:: skopt We can control how much the acqusition function favors exploration and exploitation by tweaking the two parameters kappa and xi. Higher values means more exploration and less exploitation and vice versa with low values. kappa is only used if acq_func is set to "LCB". xi is used when acq_func is "EI" or "PI". By default the acqusition function is set to "gp_hedge" which chooses the best of these three. Therefore I recommend not using gp_hedge when tweaking exploration/exploitation, but instead choosing "LCB", "EI" or "PI". The way to pass kappa and xi to the optimizer is to use the named argument "acq_func_kwargs". This is a dict of extra arguments for the aqcuisition function. If you want opt.ask() to give a new acquisition value immediately after tweaking kappa or xi call opt.update_next(). This ensures that the next value is updated with the new acquisition parameters. This example uses :class:`plots.plot_gaussian_process` which is available since version 0.8. .. GENERATED FROM PYTHON SOURCE LINES 33-42 .. code-block:: default print(__doc__) import numpy as np np.random.seed(1234) import matplotlib.pyplot as plt from skopt.learning import ExtraTreesRegressor from skopt import Optimizer from skopt.plots import plot_gaussian_process .. GENERATED FROM PYTHON SOURCE LINES 43-49 Toy example ----------- First we define our objective like in the ask-and-tell example notebook and define a plotting function. We do however only use on initial random point. All points after the first one is therefore chosen by the acquisition function. .. GENERATED FROM PYTHON SOURCE LINES 49-62 .. code-block:: default noise_level = 0.1 # Our 1D toy problem, this is the function we are trying to # minimize def objective(x, noise_level=noise_level): return np.sin(5 * x[0]) * (1 - np.tanh(x[0] ** 2)) +\ np.random.randn() * noise_level def objective_wo_noise(x): return objective(x, noise_level=0) .. GENERATED FROM PYTHON SOURCE LINES 63-67 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_optimizer="sampling") .. GENERATED FROM PYTHON SOURCE LINES 68-69 Plotting parameters .. GENERATED FROM PYTHON SOURCE LINES 69-75 .. code-block:: default plot_args = {"objective": objective_wo_noise, "noise_level": noise_level, "show_legend": True, "show_title": True, "show_next_point": False, "show_acq_func": True} .. GENERATED FROM PYTHON SOURCE LINES 76-77 We run a an optimization loop with standard settings .. GENERATED FROM PYTHON SOURCE LINES 77-85 .. code-block:: default for i in range(30): next_x = opt.ask() f_val = objective(next_x) opt.tell(next_x, f_val) # The same output could be created with opt.run(objective, n_iter=30) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_001.png :alt: x* = -0.2913, f(x*) = -1.0409 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 86-90 We see that some minima is found and "exploited" Now lets try to set kappa and xi using'to other values and pass it to the optimizer: .. GENERATED FROM PYTHON SOURCE LINES 90-91 .. code-block:: default acq_func_kwargs = {"xi": 10000, "kappa": 10000} .. GENERATED FROM PYTHON SOURCE LINES 92-96 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_optimizer="sampling", acq_func_kwargs=acq_func_kwargs) .. GENERATED FROM PYTHON SOURCE LINES 97-99 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_002.png :alt: x* = -0.3083, f(x*) = -0.7990 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 100-103 We see that the points are more random now. This works both for kappa when using acq_func="LCB": .. GENERATED FROM PYTHON SOURCE LINES 103-107 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_func="LCB", acq_optimizer="sampling", acq_func_kwargs=acq_func_kwargs) .. GENERATED FROM PYTHON SOURCE LINES 108-110 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_003.png :alt: x* = -0.1829, f(x*) = -0.8271 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 111-112 And for xi when using acq_func="EI": or acq_func="PI": .. GENERATED FROM PYTHON SOURCE LINES 112-116 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_func="PI", acq_optimizer="sampling", acq_func_kwargs=acq_func_kwargs) .. GENERATED FROM PYTHON SOURCE LINES 117-119 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_004.png :alt: x* = -0.3877, f(x*) = -0.8487 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 120-121 We can also favor exploitaton: .. GENERATED FROM PYTHON SOURCE LINES 121-122 .. code-block:: default acq_func_kwargs = {"xi": 0.000001, "kappa": 0.001} .. GENERATED FROM PYTHON SOURCE LINES 123-126 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_func="LCB", acq_optimizer="sampling", acq_func_kwargs=acq_func_kwargs) .. GENERATED FROM PYTHON SOURCE LINES 127-129 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_005.png :alt: x* = -0.4020, f(x*) = -0.9123 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 130-133 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_func="EI", acq_optimizer="sampling", acq_func_kwargs=acq_func_kwargs) .. GENERATED FROM PYTHON SOURCE LINES 134-136 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_006.png :alt: x* = 0.8760, f(x*) = -0.4154 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_006.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 137-140 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_func="PI", acq_optimizer="sampling", acq_func_kwargs=acq_func_kwargs) .. GENERATED FROM PYTHON SOURCE LINES 141-144 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_007.png :alt: x* = -0.2782, f(x*) = -1.1051 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_007.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 145-147 Note that negative values does not work with the "PI"-acquisition function but works with "EI": .. GENERATED FROM PYTHON SOURCE LINES 147-148 .. code-block:: default acq_func_kwargs = {"xi": -1000000000000} .. GENERATED FROM PYTHON SOURCE LINES 149-153 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_func="PI", acq_optimizer="sampling", acq_func_kwargs=acq_func_kwargs) .. GENERATED FROM PYTHON SOURCE LINES 154-156 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_008.png :alt: x* = 0.8093, f(x*) = -0.4621 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_008.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 157-160 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_func="EI", acq_optimizer="sampling", acq_func_kwargs=acq_func_kwargs) .. GENERATED FROM PYTHON SOURCE LINES 161-163 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_009.png :alt: x* = 0.8341, f(x*) = -0.4331 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_009.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 164-170 Changing kappa and xi on the go ------------------------------- If we want to change kappa or ki at any point during our optimization process we just replace opt.acq_func_kwargs. Remember to call `opt.update_next()` after the change, in order for next point to be recalculated. .. GENERATED FROM PYTHON SOURCE LINES 170-171 .. code-block:: default acq_func_kwargs = {"kappa": 0} .. GENERATED FROM PYTHON SOURCE LINES 172-175 .. code-block:: default opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_func="LCB", acq_optimizer="sampling", acq_func_kwargs=acq_func_kwargs) .. GENERATED FROM PYTHON SOURCE LINES 176-177 .. code-block:: default opt.acq_func_kwargs .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'kappa': 0} .. GENERATED FROM PYTHON SOURCE LINES 178-180 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_010.png :alt: x* = -0.2982, f(x*) = -1.0610 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_010.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 181-182 .. code-block:: default acq_func_kwargs = {"kappa": 100000} .. GENERATED FROM PYTHON SOURCE LINES 183-185 .. code-block:: default opt.acq_func_kwargs = acq_func_kwargs opt.update_next() .. GENERATED FROM PYTHON SOURCE LINES 186-188 .. code-block:: default opt.run(objective, n_iter=20) _ = plot_gaussian_process(opt.get_result(), **plot_args) .. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_011.png :alt: x* = -0.2982, f(x*) = -1.0610 :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_011.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 27.422 seconds) **Estimated memory usage:** 9 MB .. _sphx_glr_download_auto_examples_exploration-vs-exploitation.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: binder-badge .. image:: images/binder_badge_logo.svg :target: https://mybinder.org/v2/gh/scikit-optimize/scikit-optimize/master?urlpath=lab/tree/notebooks/auto_examples/exploration-vs-exploitation.ipynb :alt: Launch binder :width: 150 px .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: exploration-vs-exploitation.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: exploration-vs-exploitation.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_