Mixing
Java Out of Memory Error while using Stanford POS Tagger
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
1
I am getting the following error in my Jupyter notebook:
Loading default properties from tagger C:Usersbdiplus_adminDesktopABSAstanford-postagger-full-2018-10-16/models/english-bidirectional-distsim.tagger
Loading POS tagger from C:Usersbdiplus_adminDesktopABSAstanford-postagger-full-2018-10-16/models/english-bidirectional-distsim.tagger ... done [1.5 sec].
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at edu.stanford.nlp.sequences.ExactBestSequenceFinder.bestSequence(ExactBestSequenceFinder.java:129)
at edu.stanford.nlp.sequences.ExactBestSequenceFinder.bestSequence(ExactBestSequenceFinder.java:37)
at edu.stanford.nlp.tagger.maxent.TestSentence.runTagInference(TestSentence.java:341)
at edu.stanford.nlp.tagger.maxent.TestSentence.testTagInference(TestSentence.java:328)
at edu.stanford.nlp.tagger.maxent.TestSentence.tagSentence(TestSentence.java:151)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagSentence(MaxentTagger.java:1052)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagCoreLabelsOrHasWords(MaxentTagger.java:1843)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagAndOutputSentence(MaxentTagger.java:1853)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1764)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1825)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1598)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1554)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.main(MaxentTagger.java:1897)
It basically says that I am out of memory.
I've looked up the error and the most common solution I have got is to try to specify the maximum memory with -Xmx4096M. However, my issue is that I don't know where to put this argument?
I using the following lines of code in a Jupyter notebook that's running on Python 3:
from nltk.tag.stanford import StanfordPOSTagger as POS_Tag
from nltk import word_tokenize
_path_to_model = home + '/models/english-bidirectional-distsim.tagger'
_path_to_jar = home + '/stanford-postagger.jar'
stanford_tag = POS_Tag(model_filename=_path_to_model, path_to_jar=_path_to_jar)
def posTag(review):
tagged_text_list=
for text in review:
tagged_text_list.append(stanford_tag.tag(word_tokenize(text)))
return tagged_text_list
java python nlp stanford-nlp
asked Jan 3 at 6:59
VibhuVibhu
1128
add a comment |
1
I am getting the following error in my Jupyter notebook:
Loading default properties from tagger C:Usersbdiplus_adminDesktopABSAstanford-postagger-full-2018-10-16/models/english-bidirectional-distsim.tagger
Loading POS tagger from C:Usersbdiplus_adminDesktopABSAstanford-postagger-full-2018-10-16/models/english-bidirectional-distsim.tagger ... done [1.5 sec].
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at edu.stanford.nlp.sequences.ExactBestSequenceFinder.bestSequence(ExactBestSequenceFinder.java:129)
at edu.stanford.nlp.sequences.ExactBestSequenceFinder.bestSequence(ExactBestSequenceFinder.java:37)
at edu.stanford.nlp.tagger.maxent.TestSentence.runTagInference(TestSentence.java:341)
at edu.stanford.nlp.tagger.maxent.TestSentence.testTagInference(TestSentence.java:328)
at edu.stanford.nlp.tagger.maxent.TestSentence.tagSentence(TestSentence.java:151)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagSentence(MaxentTagger.java:1052)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagCoreLabelsOrHasWords(MaxentTagger.java:1843)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagAndOutputSentence(MaxentTagger.java:1853)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1764)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1825)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1598)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1554)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.main(MaxentTagger.java:1897)
It basically says that I am out of memory.
I've looked up the error and the most common solution I have got is to try to specify the maximum memory with -Xmx4096M. However, my issue is that I don't know where to put this argument?
I using the following lines of code in a Jupyter notebook that's running on Python 3:
from nltk.tag.stanford import StanfordPOSTagger as POS_Tag
from nltk import word_tokenize
_path_to_model = home + '/models/english-bidirectional-distsim.tagger'
_path_to_jar = home + '/stanford-postagger.jar'
stanford_tag = POS_Tag(model_filename=_path_to_model, path_to_jar=_path_to_jar)
def posTag(review):
tagged_text_list=
for text in review:
tagged_text_list.append(stanford_tag.tag(word_tokenize(text)))
return tagged_text_list
java python nlp stanford-nlp
asked Jan 3 at 6:59
VibhuVibhu
1128
It looks like it's a command line argument... Maybe try running it with subprocess from python?
– Novak
Jan 3 at 9:01
@Novak Hey! Thanks, but I don't know anything about subprocesses. Can you help me out with that?
– Vibhu
Jan 3 at 12:38
1
Check it out here. It's basically an interface through which you can execute your command in the system terminal. So it enables you, for example, to run the POS tagger through Command Line Interface (CLI) as you would in the terminal and capture the results in python. It's actually a fairly common way to run other programs and scripts that are not written in python.
– Novak
Jan 3 at 12:48
add a comment |
1
1
1
I am getting the following error in my Jupyter notebook:
Loading default properties from tagger C:Usersbdiplus_adminDesktopABSAstanford-postagger-full-2018-10-16/models/english-bidirectional-distsim.tagger
Loading POS tagger from C:Usersbdiplus_adminDesktopABSAstanford-postagger-full-2018-10-16/models/english-bidirectional-distsim.tagger ... done [1.5 sec].
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at edu.stanford.nlp.sequences.ExactBestSequenceFinder.bestSequence(ExactBestSequenceFinder.java:129)
at edu.stanford.nlp.sequences.ExactBestSequenceFinder.bestSequence(ExactBestSequenceFinder.java:37)
at edu.stanford.nlp.tagger.maxent.TestSentence.runTagInference(TestSentence.java:341)
at edu.stanford.nlp.tagger.maxent.TestSentence.testTagInference(TestSentence.java:328)
at edu.stanford.nlp.tagger.maxent.TestSentence.tagSentence(TestSentence.java:151)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagSentence(MaxentTagger.java:1052)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagCoreLabelsOrHasWords(MaxentTagger.java:1843)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagAndOutputSentence(MaxentTagger.java:1853)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1764)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1825)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1598)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1554)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.main(MaxentTagger.java:1897)
It basically says that I am out of memory.
I've looked up the error and the most common solution I have got is to try to specify the maximum memory with -Xmx4096M. However, my issue is that I don't know where to put this argument?
I using the following lines of code in a Jupyter notebook that's running on Python 3:
from nltk.tag.stanford import StanfordPOSTagger as POS_Tag
from nltk import word_tokenize
_path_to_model = home + '/models/english-bidirectional-distsim.tagger'
_path_to_jar = home + '/stanford-postagger.jar'
stanford_tag = POS_Tag(model_filename=_path_to_model, path_to_jar=_path_to_jar)
def posTag(review):
tagged_text_list=
for text in review:
tagged_text_list.append(stanford_tag.tag(word_tokenize(text)))
return tagged_text_list
java python nlp stanford-nlp
asked Jan 3 at 6:59
VibhuVibhu
1128
I am getting the following error in my Jupyter notebook:
Loading default properties from tagger C:Usersbdiplus_adminDesktopABSAstanford-postagger-full-2018-10-16/models/english-bidirectional-distsim.tagger
Loading POS tagger from C:Usersbdiplus_adminDesktopABSAstanford-postagger-full-2018-10-16/models/english-bidirectional-distsim.tagger ... done [1.5 sec].
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at edu.stanford.nlp.sequences.ExactBestSequenceFinder.bestSequence(ExactBestSequenceFinder.java:129)
at edu.stanford.nlp.sequences.ExactBestSequenceFinder.bestSequence(ExactBestSequenceFinder.java:37)
at edu.stanford.nlp.tagger.maxent.TestSentence.runTagInference(TestSentence.java:341)
at edu.stanford.nlp.tagger.maxent.TestSentence.testTagInference(TestSentence.java:328)
at edu.stanford.nlp.tagger.maxent.TestSentence.tagSentence(TestSentence.java:151)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagSentence(MaxentTagger.java:1052)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagCoreLabelsOrHasWords(MaxentTagger.java:1843)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.tagAndOutputSentence(MaxentTagger.java:1853)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1764)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1825)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1598)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.runTagger(MaxentTagger.java:1554)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.main(MaxentTagger.java:1897)
It basically says that I am out of memory.
I've looked up the error and the most common solution I have got is to try to specify the maximum memory with -Xmx4096M. However, my issue is that I don't know where to put this argument?
I using the following lines of code in a Jupyter notebook that's running on Python 3:
from nltk.tag.stanford import StanfordPOSTagger as POS_Tag
from nltk import word_tokenize
_path_to_model = home + '/models/english-bidirectional-distsim.tagger'
_path_to_jar = home + '/stanford-postagger.jar'
stanford_tag = POS_Tag(model_filename=_path_to_model, path_to_jar=_path_to_jar)
def posTag(review):
tagged_text_list=
for text in review:
tagged_text_list.append(stanford_tag.tag(word_tokenize(text)))
return tagged_text_list
java python nlp stanford-nlp
java python nlp stanford-nlp
asked Jan 3 at 6:59
VibhuVibhu
1128
asked Jan 3 at 6:59
VibhuVibhu
1128
asked Jan 3 at 6:59
VibhuVibhu
1128
asked Jan 3 at 6:59
VibhuVibhu
1128
asked Jan 3 at 6:59
VibhuVibhu
1128
1128
It looks like it's a command line argument... Maybe try running it with subprocess from python?
– Novak
Jan 3 at 9:01
@Novak Hey! Thanks, but I don't know anything about subprocesses. Can you help me out with that?
– Vibhu
Jan 3 at 12:38
1
Check it out here. It's basically an interface through which you can execute your command in the system terminal. So it enables you, for example, to run the POS tagger through Command Line Interface (CLI) as you would in the terminal and capture the results in python. It's actually a fairly common way to run other programs and scripts that are not written in python.
– Novak
Jan 3 at 12:48
add a comment |
It looks like it's a command line argument... Maybe try running it with subprocess from python?
– Novak
Jan 3 at 9:01
@Novak Hey! Thanks, but I don't know anything about subprocesses. Can you help me out with that?
– Vibhu
Jan 3 at 12:38
1
Check it out here. It's basically an interface through which you can execute your command in the system terminal. So it enables you, for example, to run the POS tagger through Command Line Interface (CLI) as you would in the terminal and capture the results in python. It's actually a fairly common way to run other programs and scripts that are not written in python.
– Novak
Jan 3 at 12:48
It looks like it's a command line argument... Maybe try running it with subprocess from python?
– Novak
Jan 3 at 9:01
It looks like it's a command line argument... Maybe try running it with subprocess from python?
– Novak
Jan 3 at 9:01
@Novak Hey! Thanks, but I don't know anything about subprocesses. Can you help me out with that?
– Vibhu
Jan 3 at 12:38
@Novak Hey! Thanks, but I don't know anything about subprocesses. Can you help me out with that?
– Vibhu
Jan 3 at 12:38
1
1
Check it out here. It's basically an interface through which you can execute your command in the system terminal. So it enables you, for example, to run the POS tagger through Command Line Interface (CLI) as you would in the terminal and capture the results in python. It's actually a fairly common way to run other programs and scripts that are not written in python.
– Novak
Jan 3 at 12:48
Check it out here. It's basically an interface through which you can execute your command in the system terminal. So it enables you, for example, to run the POS tagger through Command Line Interface (CLI) as you would in the terminal and capture the results in python. It's actually a fairly common way to run other programs and scripts that are not written in python.
– Novak
Jan 3 at 12:48
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54017669%2fjava-out-of-memory-error-while-using-stanford-pos-tagger%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54017669%2fjava-out-of-memory-error-while-using-stanford-pos-tagger%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It looks like it's a command line argument... Maybe try running it with subprocess from python?
– Novak
Jan 3 at 9:01
@Novak Hey! Thanks, but I don't know anything about subprocesses. Can you help me out with that?
– Vibhu
Jan 3 at 12:38
1
Check it out here. It's basically an interface through which you can execute your command in the system terminal. So it enables you, for example, to run the POS tagger through Command Line Interface (CLI) as you would in the terminal and capture the results in python. It's actually a fairly common way to run other programs and scripts that are not written in python.
– Novak
Jan 3 at 12:48