Mixing
Give scikit-learn classifier custom training data?
I have been working on this all day (struggled rather). Having read through the documentation, many other tutorials and due to my inexperience, I can't figure out how to use my own data with a MultinomialNB classifier?
Here is the code from the main tutorial:
from sklearn.datasets import fetch_20newsgroups
from sklearn.naive_bayes import MultinomialNB
categories = ['alt.atheism', 'soc.religion.christian',
'comp.graphics', 'sci.med']
text_clf = Pipeline([('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', MultinomialNB()),
])
twenty_train = fetch_20newsgroups(subset='train',
categories=categories, shuffle=True, random_state=42)
text_clf.fit(twenty_train.data, twenty_train.target)
docs_test = ['Graphics is love', 'the brain is part of the body']
predicted = text_clf.predict(docs_test)
for doc, category in zip(docs_test, predicted):
print('%r => %s' % (doc, twenty_train.target_names[category]))
Obviously, it works. But how can I replace fetch_20newsgroups with my own data (Stored in a python dictionary or the like)? And each item in the training data below is classified as one of the categories, how is this achieved?
I appreciate this is not a great question, but in this time of need, I just want to gain an understanding of how this works. Thanks
python scikit-learn training-data
edited Nov 21 '18 at 20:26
D Manokhin
599219
asked Nov 21 '18 at 20:16
ak1652ak1652
315112
add a comment |
I have been working on this all day (struggled rather). Having read through the documentation, many other tutorials and due to my inexperience, I can't figure out how to use my own data with a MultinomialNB classifier?
Here is the code from the main tutorial:
from sklearn.datasets import fetch_20newsgroups
from sklearn.naive_bayes import MultinomialNB
categories = ['alt.atheism', 'soc.religion.christian',
'comp.graphics', 'sci.med']
text_clf = Pipeline([('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', MultinomialNB()),
])
twenty_train = fetch_20newsgroups(subset='train',
categories=categories, shuffle=True, random_state=42)
text_clf.fit(twenty_train.data, twenty_train.target)
docs_test = ['Graphics is love', 'the brain is part of the body']
predicted = text_clf.predict(docs_test)
for doc, category in zip(docs_test, predicted):
print('%r => %s' % (doc, twenty_train.target_names[category]))
Obviously, it works. But how can I replace fetch_20newsgroups with my own data (Stored in a python dictionary or the like)? And each item in the training data below is classified as one of the categories, how is this achieved?
I appreciate this is not a great question, but in this time of need, I just want to gain an understanding of how this works. Thanks
python scikit-learn training-data
edited Nov 21 '18 at 20:26
D Manokhin
599219
asked Nov 21 '18 at 20:16
ak1652ak1652
315112
add a comment |
I have been working on this all day (struggled rather). Having read through the documentation, many other tutorials and due to my inexperience, I can't figure out how to use my own data with a MultinomialNB classifier?
Here is the code from the main tutorial:
from sklearn.datasets import fetch_20newsgroups
from sklearn.naive_bayes import MultinomialNB
categories = ['alt.atheism', 'soc.religion.christian',
'comp.graphics', 'sci.med']
text_clf = Pipeline([('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', MultinomialNB()),
])
twenty_train = fetch_20newsgroups(subset='train',
categories=categories, shuffle=True, random_state=42)
text_clf.fit(twenty_train.data, twenty_train.target)
docs_test = ['Graphics is love', 'the brain is part of the body']
predicted = text_clf.predict(docs_test)
for doc, category in zip(docs_test, predicted):
print('%r => %s' % (doc, twenty_train.target_names[category]))
Obviously, it works. But how can I replace fetch_20newsgroups with my own data (Stored in a python dictionary or the like)? And each item in the training data below is classified as one of the categories, how is this achieved?
I appreciate this is not a great question, but in this time of need, I just want to gain an understanding of how this works. Thanks
python scikit-learn training-data
edited Nov 21 '18 at 20:26
D Manokhin
599219
asked Nov 21 '18 at 20:16
ak1652ak1652
315112
I have been working on this all day (struggled rather). Having read through the documentation, many other tutorials and due to my inexperience, I can't figure out how to use my own data with a MultinomialNB classifier?
Here is the code from the main tutorial:
from sklearn.datasets import fetch_20newsgroups
from sklearn.naive_bayes import MultinomialNB
categories = ['alt.atheism', 'soc.religion.christian',
'comp.graphics', 'sci.med']
text_clf = Pipeline([('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', MultinomialNB()),
])
twenty_train = fetch_20newsgroups(subset='train',
categories=categories, shuffle=True, random_state=42)
text_clf.fit(twenty_train.data, twenty_train.target)
docs_test = ['Graphics is love', 'the brain is part of the body']
predicted = text_clf.predict(docs_test)
for doc, category in zip(docs_test, predicted):
print('%r => %s' % (doc, twenty_train.target_names[category]))
Obviously, it works. But how can I replace fetch_20newsgroups with my own data (Stored in a python dictionary or the like)? And each item in the training data below is classified as one of the categories, how is this achieved?
I appreciate this is not a great question, but in this time of need, I just want to gain an understanding of how this works. Thanks
python scikit-learn training-data
python scikit-learn training-data
edited Nov 21 '18 at 20:26
D Manokhin
599219
asked Nov 21 '18 at 20:16
ak1652ak1652
315112
edited Nov 21 '18 at 20:26
D Manokhin
599219
asked Nov 21 '18 at 20:16
ak1652ak1652
315112
edited Nov 21 '18 at 20:26
D Manokhin
599219
edited Nov 21 '18 at 20:26
D Manokhin
599219
edited Nov 21 '18 at 20:26
D Manokhin
599219
599219
asked Nov 21 '18 at 20:16
ak1652ak1652
315112
asked Nov 21 '18 at 20:16
ak1652ak1652
315112
asked Nov 21 '18 at 20:16
ak1652ak1652
315112
315112
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Almost all sklearn fit methods take a list of training data, and list of labels as input. In your case, the list of training data would be a list of strings (texts on which you have to train your model). Like ['this is my first training sample', 'this is second string', 'and this is third', ...], and another list of labels like ['label1', 'label2', 'label1', ...].
And you'll pass these lists to the fit method:
text_clf.fit(list_of_training_datas, list_of_labels)
predict method would remain the same, as it would also take a list of samples you want to test, and will return a list containing the predicted label for each of the test samples.
answered Nov 21 '18 at 20:51
Muhammad AhmadMuhammad Ahmad
2,1321422
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53419868%2fgive-scikit-learn-classifier-custom-training-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Almost all sklearn fit methods take a list of training data, and list of labels as input. In your case, the list of training data would be a list of strings (texts on which you have to train your model). Like ['this is my first training sample', 'this is second string', 'and this is third', ...], and another list of labels like ['label1', 'label2', 'label1', ...].
And you'll pass these lists to the fit method:
text_clf.fit(list_of_training_datas, list_of_labels)
predict method would remain the same, as it would also take a list of samples you want to test, and will return a list containing the predicted label for each of the test samples.
answered Nov 21 '18 at 20:51
Muhammad AhmadMuhammad Ahmad
2,1321422
add a comment |
Almost all sklearn fit methods take a list of training data, and list of labels as input. In your case, the list of training data would be a list of strings (texts on which you have to train your model). Like ['this is my first training sample', 'this is second string', 'and this is third', ...], and another list of labels like ['label1', 'label2', 'label1', ...].
And you'll pass these lists to the fit method:
text_clf.fit(list_of_training_datas, list_of_labels)
predict method would remain the same, as it would also take a list of samples you want to test, and will return a list containing the predicted label for each of the test samples.
answered Nov 21 '18 at 20:51
Muhammad AhmadMuhammad Ahmad
2,1321422
add a comment |
Almost all sklearn fit methods take a list of training data, and list of labels as input. In your case, the list of training data would be a list of strings (texts on which you have to train your model). Like ['this is my first training sample', 'this is second string', 'and this is third', ...], and another list of labels like ['label1', 'label2', 'label1', ...].
And you'll pass these lists to the fit method:
text_clf.fit(list_of_training_datas, list_of_labels)
predict method would remain the same, as it would also take a list of samples you want to test, and will return a list containing the predicted label for each of the test samples.
answered Nov 21 '18 at 20:51
Muhammad AhmadMuhammad Ahmad
2,1321422
Almost all sklearn fit methods take a list of training data, and list of labels as input. In your case, the list of training data would be a list of strings (texts on which you have to train your model). Like ['this is my first training sample', 'this is second string', 'and this is third', ...], and another list of labels like ['label1', 'label2', 'label1', ...].
And you'll pass these lists to the fit method:
text_clf.fit(list_of_training_datas, list_of_labels)
predict method would remain the same, as it would also take a list of samples you want to test, and will return a list containing the predicted label for each of the test samples.
answered Nov 21 '18 at 20:51
Muhammad AhmadMuhammad Ahmad
2,1321422
edited Nov 22 '18 at 14:35
answered Nov 21 '18 at 20:51
Muhammad AhmadMuhammad Ahmad
2,1321422
answered Nov 21 '18 at 20:51
Muhammad AhmadMuhammad Ahmad
2,1321422
answered Nov 21 '18 at 20:51
Muhammad AhmadMuhammad Ahmad
2,1321422
2,1321422
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53419868%2fgive-scikit-learn-classifier-custom-training-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
