Mixing
Accessing model in gensim wrapper
I use following gensim wrapper to train a word-vector model:
import numpy as np
import pandas as pd
from gensim.sklearn_api import W2VTransformer
from gensim.utils import simple_preprocess
# Load synthetic data
data = pd.read_csv('https://pastebin.com/raw/EPCmabvN')
data = data.head(10)
# Set random seed
np.random.seed(0)
X_train = data.apply(lambda r: simple_preprocess(r['text'], min_len=2), axis=1)
y_train = data.label
model = W2VTransformer(size=10, min_count=1)
model.fit(X_train)
model.wv.vocab
However, once I try to access the trained model, i.e. model.wv.vocab, it outputs the error:
AttributeError: 'W2VTransformer' object has no attribute 'wv'
Can I somehow access the vocabulary and other model parameters, or is this not possible with the wrapper?
Current workaround:
from gensim.models.doc2vec import TaggedDocument
from gensim.models.doc2vec import Doc2Vec
#Defining model without wrapper
documents = data.apply(lambda r: TaggedDocument(words=simple_preprocess(r['text'], min_len=2), tags=[r.label]), axis=1)
d2v = Doc2Vec(documents, window=2, vector_size=10, min_count=1, seed=0)
d2v.wv.vocab
model wrapper gensim
asked Jan 2 at 9:03
ChristopherChristopher
4071923
add a comment |
I use following gensim wrapper to train a word-vector model:
import numpy as np
import pandas as pd
from gensim.sklearn_api import W2VTransformer
from gensim.utils import simple_preprocess
# Load synthetic data
data = pd.read_csv('https://pastebin.com/raw/EPCmabvN')
data = data.head(10)
# Set random seed
np.random.seed(0)
X_train = data.apply(lambda r: simple_preprocess(r['text'], min_len=2), axis=1)
y_train = data.label
model = W2VTransformer(size=10, min_count=1)
model.fit(X_train)
model.wv.vocab
However, once I try to access the trained model, i.e. model.wv.vocab, it outputs the error:
AttributeError: 'W2VTransformer' object has no attribute 'wv'
Can I somehow access the vocabulary and other model parameters, or is this not possible with the wrapper?
Current workaround:
from gensim.models.doc2vec import TaggedDocument
from gensim.models.doc2vec import Doc2Vec
#Defining model without wrapper
documents = data.apply(lambda r: TaggedDocument(words=simple_preprocess(r['text'], min_len=2), tags=[r.label]), axis=1)
d2v = Doc2Vec(documents, window=2, vector_size=10, min_count=1, seed=0)
d2v.wv.vocab
model wrapper gensim
asked Jan 2 at 9:03
ChristopherChristopher
4071923
add a comment |
I use following gensim wrapper to train a word-vector model:
import numpy as np
import pandas as pd
from gensim.sklearn_api import W2VTransformer
from gensim.utils import simple_preprocess
# Load synthetic data
data = pd.read_csv('https://pastebin.com/raw/EPCmabvN')
data = data.head(10)
# Set random seed
np.random.seed(0)
X_train = data.apply(lambda r: simple_preprocess(r['text'], min_len=2), axis=1)
y_train = data.label
model = W2VTransformer(size=10, min_count=1)
model.fit(X_train)
model.wv.vocab
However, once I try to access the trained model, i.e. model.wv.vocab, it outputs the error:
AttributeError: 'W2VTransformer' object has no attribute 'wv'
Can I somehow access the vocabulary and other model parameters, or is this not possible with the wrapper?
Current workaround:
from gensim.models.doc2vec import TaggedDocument
from gensim.models.doc2vec import Doc2Vec
#Defining model without wrapper
documents = data.apply(lambda r: TaggedDocument(words=simple_preprocess(r['text'], min_len=2), tags=[r.label]), axis=1)
d2v = Doc2Vec(documents, window=2, vector_size=10, min_count=1, seed=0)
d2v.wv.vocab
model wrapper gensim
asked Jan 2 at 9:03
ChristopherChristopher
4071923
I use following gensim wrapper to train a word-vector model:
import numpy as np
import pandas as pd
from gensim.sklearn_api import W2VTransformer
from gensim.utils import simple_preprocess
# Load synthetic data
data = pd.read_csv('https://pastebin.com/raw/EPCmabvN')
data = data.head(10)
# Set random seed
np.random.seed(0)
X_train = data.apply(lambda r: simple_preprocess(r['text'], min_len=2), axis=1)
y_train = data.label
model = W2VTransformer(size=10, min_count=1)
model.fit(X_train)
model.wv.vocab
However, once I try to access the trained model, i.e. model.wv.vocab, it outputs the error:
AttributeError: 'W2VTransformer' object has no attribute 'wv'
Can I somehow access the vocabulary and other model parameters, or is this not possible with the wrapper?
Current workaround:
from gensim.models.doc2vec import TaggedDocument
from gensim.models.doc2vec import Doc2Vec
#Defining model without wrapper
documents = data.apply(lambda r: TaggedDocument(words=simple_preprocess(r['text'], min_len=2), tags=[r.label]), axis=1)
d2v = Doc2Vec(documents, window=2, vector_size=10, min_count=1, seed=0)
d2v.wv.vocab
model wrapper gensim
model wrapper gensim
asked Jan 2 at 9:03
ChristopherChristopher
4071923
asked Jan 2 at 9:03
ChristopherChristopher
4071923
edited Jan 2 at 9:52
Christopher
asked Jan 2 at 9:03
ChristopherChristopher
4071923
asked Jan 2 at 9:03
ChristopherChristopher
4071923
asked Jan 2 at 9:03
ChristopherChristopher
4071923
4071923
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
What makes you think W2VTransformer has a wv property? It's not listed in the class docs:
https://radimrehurek.com/gensim/sklearn_api/w2vmodel.html
And, it's not quite idiomatic (within scikit-learn) to access a Transformer's internal state like that. Instead, you would ask a model that's already been fit() to then transform() a list-of-words, to get back a list-of-word-vectors.
Indeed that's shown in the example at the top of those gensim docs, in a line which does both the fit() and `transform() in one line (even if you wouldn't want to do that):
wordvecs = model.fit(common_texts).transform(['graph', 'system'])
If you do want to access the native gensim Word2Vec model directly – a model which does have a wv property – you'd have to use a different approach. For example, you could review the W2VTransformer source code to see where that internal model is kept:
https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/sklearn_api/w2vmodel.py
There you would see that the fit() method stores the current Word2Vec instance in a property called gensim_model.
So, your line that is erroring, where model is an instance of W2VTransformer, could instead be:
model.gensim_model.wv.vocab
answered Jan 2 at 18:20
gojomogojomo
20.5k64467
1
Thank you gojomo, I found it and it works. Excuse my gensim related questions, I am a starter and all the debugging and source code inspection is still a but new to me.
– Christopher
Jan 3 at 10:54
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54003616%2faccessing-model-in-gensim-wrapper%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
What makes you think W2VTransformer has a wv property? It's not listed in the class docs:
https://radimrehurek.com/gensim/sklearn_api/w2vmodel.html
And, it's not quite idiomatic (within scikit-learn) to access a Transformer's internal state like that. Instead, you would ask a model that's already been fit() to then transform() a list-of-words, to get back a list-of-word-vectors.
Indeed that's shown in the example at the top of those gensim docs, in a line which does both the fit() and `transform() in one line (even if you wouldn't want to do that):
wordvecs = model.fit(common_texts).transform(['graph', 'system'])
If you do want to access the native gensim Word2Vec model directly – a model which does have a wv property – you'd have to use a different approach. For example, you could review the W2VTransformer source code to see where that internal model is kept:
https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/sklearn_api/w2vmodel.py
There you would see that the fit() method stores the current Word2Vec instance in a property called gensim_model.
So, your line that is erroring, where model is an instance of W2VTransformer, could instead be:
model.gensim_model.wv.vocab
answered Jan 2 at 18:20
gojomogojomo
20.5k64467
1
Thank you gojomo, I found it and it works. Excuse my gensim related questions, I am a starter and all the debugging and source code inspection is still a but new to me.
– Christopher
Jan 3 at 10:54
add a comment |
What makes you think W2VTransformer has a wv property? It's not listed in the class docs:
https://radimrehurek.com/gensim/sklearn_api/w2vmodel.html
And, it's not quite idiomatic (within scikit-learn) to access a Transformer's internal state like that. Instead, you would ask a model that's already been fit() to then transform() a list-of-words, to get back a list-of-word-vectors.
Indeed that's shown in the example at the top of those gensim docs, in a line which does both the fit() and `transform() in one line (even if you wouldn't want to do that):
wordvecs = model.fit(common_texts).transform(['graph', 'system'])
If you do want to access the native gensim Word2Vec model directly – a model which does have a wv property – you'd have to use a different approach. For example, you could review the W2VTransformer source code to see where that internal model is kept:
https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/sklearn_api/w2vmodel.py
There you would see that the fit() method stores the current Word2Vec instance in a property called gensim_model.
So, your line that is erroring, where model is an instance of W2VTransformer, could instead be:
model.gensim_model.wv.vocab
answered Jan 2 at 18:20
gojomogojomo
20.5k64467
1
Thank you gojomo, I found it and it works. Excuse my gensim related questions, I am a starter and all the debugging and source code inspection is still a but new to me.
– Christopher
Jan 3 at 10:54
add a comment |
What makes you think W2VTransformer has a wv property? It's not listed in the class docs:
https://radimrehurek.com/gensim/sklearn_api/w2vmodel.html
And, it's not quite idiomatic (within scikit-learn) to access a Transformer's internal state like that. Instead, you would ask a model that's already been fit() to then transform() a list-of-words, to get back a list-of-word-vectors.
Indeed that's shown in the example at the top of those gensim docs, in a line which does both the fit() and `transform() in one line (even if you wouldn't want to do that):
wordvecs = model.fit(common_texts).transform(['graph', 'system'])
If you do want to access the native gensim Word2Vec model directly – a model which does have a wv property – you'd have to use a different approach. For example, you could review the W2VTransformer source code to see where that internal model is kept:
https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/sklearn_api/w2vmodel.py
There you would see that the fit() method stores the current Word2Vec instance in a property called gensim_model.
So, your line that is erroring, where model is an instance of W2VTransformer, could instead be:
model.gensim_model.wv.vocab
answered Jan 2 at 18:20
gojomogojomo
20.5k64467
What makes you think W2VTransformer has a wv property? It's not listed in the class docs:
https://radimrehurek.com/gensim/sklearn_api/w2vmodel.html
And, it's not quite idiomatic (within scikit-learn) to access a Transformer's internal state like that. Instead, you would ask a model that's already been fit() to then transform() a list-of-words, to get back a list-of-word-vectors.
Indeed that's shown in the example at the top of those gensim docs, in a line which does both the fit() and `transform() in one line (even if you wouldn't want to do that):
wordvecs = model.fit(common_texts).transform(['graph', 'system'])
If you do want to access the native gensim Word2Vec model directly – a model which does have a wv property – you'd have to use a different approach. For example, you could review the W2VTransformer source code to see where that internal model is kept:
https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/sklearn_api/w2vmodel.py
There you would see that the fit() method stores the current Word2Vec instance in a property called gensim_model.
So, your line that is erroring, where model is an instance of W2VTransformer, could instead be:
model.gensim_model.wv.vocab
answered Jan 2 at 18:20
gojomogojomo
20.5k64467
edited Jan 2 at 22:37
answered Jan 2 at 18:20
gojomogojomo
20.5k64467
answered Jan 2 at 18:20
gojomogojomo
20.5k64467
answered Jan 2 at 18:20
gojomogojomo
20.5k64467
20.5k64467
1
Thank you gojomo, I found it and it works. Excuse my gensim related questions, I am a starter and all the debugging and source code inspection is still a but new to me.
– Christopher
Jan 3 at 10:54
add a comment |
1
Thank you gojomo, I found it and it works. Excuse my gensim related questions, I am a starter and all the debugging and source code inspection is still a but new to me.
– Christopher
Jan 3 at 10:54
1
1
Thank you gojomo, I found it and it works. Excuse my gensim related questions, I am a starter and all the debugging and source code inspection is still a but new to me.
– Christopher
Jan 3 at 10:54
Thank you gojomo, I found it and it works. Excuse my gensim related questions, I am a starter and all the debugging and source code inspection is still a but new to me.
– Christopher
Jan 3 at 10:54
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54003616%2faccessing-model-in-gensim-wrapper%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
