Mixing
How to use spacy lemmatiser with a different pos taging
I am working on POS tagging tasks through different libraries (including pattern) as well as lemmatization tasks.
Everytime I use the spacy lemmatisation, it automaticaly generates a spacy pos tag for every word in the sentence.
However, I would like to use the pos tag generated by pattern (not from spacy) to improve the lemmatisation of the sentence.
Is that something possible ?
spacy
asked Jan 2 at 16:50
RaphaëlRaphaël
53
add a comment |
I am working on POS tagging tasks through different libraries (including pattern) as well as lemmatization tasks.
Everytime I use the spacy lemmatisation, it automaticaly generates a spacy pos tag for every word in the sentence.
However, I would like to use the pos tag generated by pattern (not from spacy) to improve the lemmatisation of the sentence.
Is that something possible ?
spacy
asked Jan 2 at 16:50
RaphaëlRaphaël
53
add a comment |
I am working on POS tagging tasks through different libraries (including pattern) as well as lemmatization tasks.
Everytime I use the spacy lemmatisation, it automaticaly generates a spacy pos tag for every word in the sentence.
However, I would like to use the pos tag generated by pattern (not from spacy) to improve the lemmatisation of the sentence.
Is that something possible ?
spacy
asked Jan 2 at 16:50
RaphaëlRaphaël
53
I am working on POS tagging tasks through different libraries (including pattern) as well as lemmatization tasks.
Everytime I use the spacy lemmatisation, it automaticaly generates a spacy pos tag for every word in the sentence.
However, I would like to use the pos tag generated by pattern (not from spacy) to improve the lemmatisation of the sentence.
Is that something possible ?
spacy
spacy
asked Jan 2 at 16:50
RaphaëlRaphaël
53
asked Jan 2 at 16:50
RaphaëlRaphaël
53
asked Jan 2 at 16:50
RaphaëlRaphaël
53
asked Jan 2 at 16:50
RaphaëlRaphaël
53
asked Jan 2 at 16:50
RaphaëlRaphaël
53
53
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
I am currently looking into this problem as well. Here are some things I found out, hope it will point you in the right direction.
- lemmatizer is created by BaseDefaults.create_lemmatizer(see https://github.com/explosion/spacy/blob/master/spacy/language.py). You can access it by calling nlp.Defaults.create_lemmatizer
- lemmatizer lives at nlp.vocab.morphology.lemmatizer (see https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
- lemmatizer is called when tokenizer's exceptions are added during tokenizer's instantiation (if lemma is not supplied as a part of the exception definition).
- lemmatizer is called from Tagger.set_annotations => vocab.morphology.assign_tag_id (see https://github.com/explosion/spacy/blob/master/spacy/pipeline.pyx for Tagger class, and https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
Tagger is a part of the spaCy pipeline.
Looks like what you need to do is:
- disable spacy POS tagger, and create and plug in your own (there's info here: https://spacy.io/usage/processing-pipelines)
- create your own lemmatizer pipe element, which will call nlp.vocab.morphology.lemmatizer with the tags your tagger assigned. Or maybe a better solution would be to create your own instance of the lemmatizer by calling nlp.Defaults.create_lemmatizer and then use that one.
Hope this helps.
answered Jan 3 at 16:56
NataliaNatalia
161
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54010166%2fhow-to-use-spacy-lemmatiser-with-a-different-pos-taging%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I am currently looking into this problem as well. Here are some things I found out, hope it will point you in the right direction.
- lemmatizer is created by BaseDefaults.create_lemmatizer(see https://github.com/explosion/spacy/blob/master/spacy/language.py). You can access it by calling nlp.Defaults.create_lemmatizer
- lemmatizer lives at nlp.vocab.morphology.lemmatizer (see https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
- lemmatizer is called when tokenizer's exceptions are added during tokenizer's instantiation (if lemma is not supplied as a part of the exception definition).
- lemmatizer is called from Tagger.set_annotations => vocab.morphology.assign_tag_id (see https://github.com/explosion/spacy/blob/master/spacy/pipeline.pyx for Tagger class, and https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
Tagger is a part of the spaCy pipeline.
Looks like what you need to do is:
- disable spacy POS tagger, and create and plug in your own (there's info here: https://spacy.io/usage/processing-pipelines)
- create your own lemmatizer pipe element, which will call nlp.vocab.morphology.lemmatizer with the tags your tagger assigned. Or maybe a better solution would be to create your own instance of the lemmatizer by calling nlp.Defaults.create_lemmatizer and then use that one.
Hope this helps.
answered Jan 3 at 16:56
NataliaNatalia
161
add a comment |
I am currently looking into this problem as well. Here are some things I found out, hope it will point you in the right direction.
- lemmatizer is created by BaseDefaults.create_lemmatizer(see https://github.com/explosion/spacy/blob/master/spacy/language.py). You can access it by calling nlp.Defaults.create_lemmatizer
- lemmatizer lives at nlp.vocab.morphology.lemmatizer (see https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
- lemmatizer is called when tokenizer's exceptions are added during tokenizer's instantiation (if lemma is not supplied as a part of the exception definition).
- lemmatizer is called from Tagger.set_annotations => vocab.morphology.assign_tag_id (see https://github.com/explosion/spacy/blob/master/spacy/pipeline.pyx for Tagger class, and https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
Tagger is a part of the spaCy pipeline.
Looks like what you need to do is:
- disable spacy POS tagger, and create and plug in your own (there's info here: https://spacy.io/usage/processing-pipelines)
- create your own lemmatizer pipe element, which will call nlp.vocab.morphology.lemmatizer with the tags your tagger assigned. Or maybe a better solution would be to create your own instance of the lemmatizer by calling nlp.Defaults.create_lemmatizer and then use that one.
Hope this helps.
answered Jan 3 at 16:56
NataliaNatalia
161
add a comment |
I am currently looking into this problem as well. Here are some things I found out, hope it will point you in the right direction.
- lemmatizer is created by BaseDefaults.create_lemmatizer(see https://github.com/explosion/spacy/blob/master/spacy/language.py). You can access it by calling nlp.Defaults.create_lemmatizer
- lemmatizer lives at nlp.vocab.morphology.lemmatizer (see https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
- lemmatizer is called when tokenizer's exceptions are added during tokenizer's instantiation (if lemma is not supplied as a part of the exception definition).
- lemmatizer is called from Tagger.set_annotations => vocab.morphology.assign_tag_id (see https://github.com/explosion/spacy/blob/master/spacy/pipeline.pyx for Tagger class, and https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
Tagger is a part of the spaCy pipeline.
Looks like what you need to do is:
- disable spacy POS tagger, and create and plug in your own (there's info here: https://spacy.io/usage/processing-pipelines)
- create your own lemmatizer pipe element, which will call nlp.vocab.morphology.lemmatizer with the tags your tagger assigned. Or maybe a better solution would be to create your own instance of the lemmatizer by calling nlp.Defaults.create_lemmatizer and then use that one.
Hope this helps.
answered Jan 3 at 16:56
NataliaNatalia
161
I am currently looking into this problem as well. Here are some things I found out, hope it will point you in the right direction.
- lemmatizer is created by BaseDefaults.create_lemmatizer(see https://github.com/explosion/spacy/blob/master/spacy/language.py). You can access it by calling nlp.Defaults.create_lemmatizer
- lemmatizer lives at nlp.vocab.morphology.lemmatizer (see https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
- lemmatizer is called when tokenizer's exceptions are added during tokenizer's instantiation (if lemma is not supplied as a part of the exception definition).
- lemmatizer is called from Tagger.set_annotations => vocab.morphology.assign_tag_id (see https://github.com/explosion/spacy/blob/master/spacy/pipeline.pyx for Tagger class, and https://github.com/explosion/spaCy/blob/master/spacy/morphology.pyx)
Tagger is a part of the spaCy pipeline.
Looks like what you need to do is:
- disable spacy POS tagger, and create and plug in your own (there's info here: https://spacy.io/usage/processing-pipelines)
- create your own lemmatizer pipe element, which will call nlp.vocab.morphology.lemmatizer with the tags your tagger assigned. Or maybe a better solution would be to create your own instance of the lemmatizer by calling nlp.Defaults.create_lemmatizer and then use that one.
Hope this helps.
answered Jan 3 at 16:56
NataliaNatalia
161
answered Jan 3 at 16:56
NataliaNatalia
161
answered Jan 3 at 16:56
NataliaNatalia
161
answered Jan 3 at 16:56
NataliaNatalia
161
161
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54010166%2fhow-to-use-spacy-lemmatiser-with-a-different-pos-taging%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
