Mixing
Which aquasition function to use for gp_minimize in Scikit-Optimize?
I am trying to perform a hyper-parameter optimization task on a LSTM (purely Tensorflow) using the Scikit-Optimize package. I am not familiar with Bayesian optimization or Bayesian functions. This package offers a Bayesian optimization using Gaussian Process (gp_minimize). In it's parameters we are asked to give an acquisition function (acq_func).
search_result = gp_minimize(func=fitness,
dimensions=dimensions,
acq_func='EI', # Expected Improvement.
n_calls=40,
x0=default_parameters)
The documentation lists several acquisition functions from which we can choose from. Listed below are the options:
"LCB" for lower confidence bound.
"EI" for negative
expectedimprovement.
"PI" for negative probability of
improvement.
"gp_hedge" Probabilistically choose one of the above
three acquisition functions at every iteration. The weightage given to
these gains can be set by eta through acq_func_kwargs.
My task is a sales forecasting task and I am trying to minimize the RMSE value of it. They have given an example code of a classification task.But I was unable to get much insight from that
Question: I would like to know if there is a way of determining which acquisition function is better for which scenario and what acquisition function should I use in my scenario.
python-3.x optimization deep-learning bayesian hyperparameters
asked Jan 2 at 6:58
Suleka_28Suleka_28
755618
add a comment |
I am trying to perform a hyper-parameter optimization task on a LSTM (purely Tensorflow) using the Scikit-Optimize package. I am not familiar with Bayesian optimization or Bayesian functions. This package offers a Bayesian optimization using Gaussian Process (gp_minimize). In it's parameters we are asked to give an acquisition function (acq_func).
search_result = gp_minimize(func=fitness,
dimensions=dimensions,
acq_func='EI', # Expected Improvement.
n_calls=40,
x0=default_parameters)
The documentation lists several acquisition functions from which we can choose from. Listed below are the options:
"LCB" for lower confidence bound.
"EI" for negative
expectedimprovement.
"PI" for negative probability of
improvement.
"gp_hedge" Probabilistically choose one of the above
three acquisition functions at every iteration. The weightage given to
these gains can be set by eta through acq_func_kwargs.
My task is a sales forecasting task and I am trying to minimize the RMSE value of it. They have given an example code of a classification task.But I was unable to get much insight from that
Question: I would like to know if there is a way of determining which acquisition function is better for which scenario and what acquisition function should I use in my scenario.
python-3.x optimization deep-learning bayesian hyperparameters
asked Jan 2 at 6:58
Suleka_28Suleka_28
755618
1
This is probably a better fit for CrossValidated. I don't know LSTM stuff, but I can comment generically on BO acquisition strategies. If I recall correctly, they go something like:(most conservative) LCB < EI < PI (most greedy). Where LCB focuses on ruling out regions of high uncertainty and PI on refining the region that currently looks the best. The default,gp_hedge, randomly tries a bit of each and could be a good default when approaching unfamiliar problems. I think I found this paper quite helpful.
– merv
Jan 2 at 23:20
add a comment |
I am trying to perform a hyper-parameter optimization task on a LSTM (purely Tensorflow) using the Scikit-Optimize package. I am not familiar with Bayesian optimization or Bayesian functions. This package offers a Bayesian optimization using Gaussian Process (gp_minimize). In it's parameters we are asked to give an acquisition function (acq_func).
search_result = gp_minimize(func=fitness,
dimensions=dimensions,
acq_func='EI', # Expected Improvement.
n_calls=40,
x0=default_parameters)
The documentation lists several acquisition functions from which we can choose from. Listed below are the options:
"LCB" for lower confidence bound.
"EI" for negative
expectedimprovement.
"PI" for negative probability of
improvement.
"gp_hedge" Probabilistically choose one of the above
three acquisition functions at every iteration. The weightage given to
these gains can be set by eta through acq_func_kwargs.
My task is a sales forecasting task and I am trying to minimize the RMSE value of it. They have given an example code of a classification task.But I was unable to get much insight from that
Question: I would like to know if there is a way of determining which acquisition function is better for which scenario and what acquisition function should I use in my scenario.
python-3.x optimization deep-learning bayesian hyperparameters
asked Jan 2 at 6:58
Suleka_28Suleka_28
755618
I am trying to perform a hyper-parameter optimization task on a LSTM (purely Tensorflow) using the Scikit-Optimize package. I am not familiar with Bayesian optimization or Bayesian functions. This package offers a Bayesian optimization using Gaussian Process (gp_minimize). In it's parameters we are asked to give an acquisition function (acq_func).
search_result = gp_minimize(func=fitness,
dimensions=dimensions,
acq_func='EI', # Expected Improvement.
n_calls=40,
x0=default_parameters)
The documentation lists several acquisition functions from which we can choose from. Listed below are the options:
"LCB" for lower confidence bound.
"EI" for negative
expectedimprovement.
"PI" for negative probability of
improvement.
"gp_hedge" Probabilistically choose one of the above
three acquisition functions at every iteration. The weightage given to
these gains can be set by eta through acq_func_kwargs.
My task is a sales forecasting task and I am trying to minimize the RMSE value of it. They have given an example code of a classification task.But I was unable to get much insight from that
Question: I would like to know if there is a way of determining which acquisition function is better for which scenario and what acquisition function should I use in my scenario.
python-3.x optimization deep-learning bayesian hyperparameters
python-3.x optimization deep-learning bayesian hyperparameters
asked Jan 2 at 6:58
Suleka_28Suleka_28
755618
asked Jan 2 at 6:58
Suleka_28Suleka_28
755618
asked Jan 2 at 6:58
Suleka_28Suleka_28
755618
asked Jan 2 at 6:58
Suleka_28Suleka_28
755618
asked Jan 2 at 6:58
Suleka_28Suleka_28
755618
755618
1
This is probably a better fit for CrossValidated. I don't know LSTM stuff, but I can comment generically on BO acquisition strategies. If I recall correctly, they go something like:(most conservative) LCB < EI < PI (most greedy). Where LCB focuses on ruling out regions of high uncertainty and PI on refining the region that currently looks the best. The default,gp_hedge, randomly tries a bit of each and could be a good default when approaching unfamiliar problems. I think I found this paper quite helpful.
– merv
Jan 2 at 23:20
add a comment |
1
This is probably a better fit for CrossValidated. I don't know LSTM stuff, but I can comment generically on BO acquisition strategies. If I recall correctly, they go something like:(most conservative) LCB < EI < PI (most greedy). Where LCB focuses on ruling out regions of high uncertainty and PI on refining the region that currently looks the best. The default,gp_hedge, randomly tries a bit of each and could be a good default when approaching unfamiliar problems. I think I found this paper quite helpful.
– merv
Jan 2 at 23:20
1
1
This is probably a better fit for CrossValidated. I don't know LSTM stuff, but I can comment generically on BO acquisition strategies. If I recall correctly, they go something like:
(most conservative) LCB < EI < PI (most greedy). Where LCB focuses on ruling out regions of high uncertainty and PI on refining the region that currently looks the best. The default, gp_hedge, randomly tries a bit of each and could be a good default when approaching unfamiliar problems. I think I found this paper quite helpful.– merv
Jan 2 at 23:20
This is probably a better fit for CrossValidated. I don't know LSTM stuff, but I can comment generically on BO acquisition strategies. If I recall correctly, they go something like:
(most conservative) LCB < EI < PI (most greedy). Where LCB focuses on ruling out regions of high uncertainty and PI on refining the region that currently looks the best. The default, gp_hedge, randomly tries a bit of each and could be a good default when approaching unfamiliar problems. I think I found this paper quite helpful.– merv
Jan 2 at 23:20
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54002426%2fwhich-aquasition-function-to-use-for-gp-minimize-in-scikit-optimize%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54002426%2fwhich-aquasition-function-to-use-for-gp-minimize-in-scikit-optimize%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
This is probably a better fit for CrossValidated. I don't know LSTM stuff, but I can comment generically on BO acquisition strategies. If I recall correctly, they go something like:
(most conservative) LCB < EI < PI (most greedy). Where LCB focuses on ruling out regions of high uncertainty and PI on refining the region that currently looks the best. The default,gp_hedge, randomly tries a bit of each and could be a good default when approaching unfamiliar problems. I think I found this paper quite helpful.– merv
Jan 2 at 23:20