Mixing
Apache Tika not properly extracting text from PDF for Indian languages
I tried to extract text from PDF, it is working fine for English but not for Indian languages like Hindi, Tamil, Malayalam etc... See the sample below, all the bold text in the extracted content is not correct. Does Tika work with non English languages?
Original Text :
மக்களுக்குப் புத்தாண்டு பரிசாக, எல்இடி டிவி, சினிமா டிக்கெட், கம்ப்யூட்டர் மானிட்டர், மாற்றுத்திறனாளிகளுக்கான உதிரிப்பாகங்கள் உள்ளிட்ட 23 வகை பொருட்கள், சேவைகளின் குறைக்கப்பட்ட ஜிஎஸ்டி வரி இன்று முதல் அமலுக்கு வந்தது.
Extracted Text :
மக்களக்கப் பத்தாண்ட பரிசாக, எல்இடடவ, சனிமா டக்ககட,் கம்ப்யட்டர் மானிட்டர,்
மாற்றத்தறனாளிகளக்கானஉதரிப்பாகங்கள் உள்ளிட்ட 23
வகககபாரடக்ள், சசகவகளின்
ககறக்கப்படட் ஜஎஸ்ட வரி இன்றமதல் அமலக்கவந்தத.
UPDATE:
If I convert the PDF to JPG and use TesseractOCRParser and mention the language as Tamil then it works lot better than the PDFParser
apache-tika
asked Jan 1 at 5:06
wolverinewolverine
329319
add a comment |
I tried to extract text from PDF, it is working fine for English but not for Indian languages like Hindi, Tamil, Malayalam etc... See the sample below, all the bold text in the extracted content is not correct. Does Tika work with non English languages?
Original Text :
மக்களுக்குப் புத்தாண்டு பரிசாக, எல்இடி டிவி, சினிமா டிக்கெட், கம்ப்யூட்டர் மானிட்டர், மாற்றுத்திறனாளிகளுக்கான உதிரிப்பாகங்கள் உள்ளிட்ட 23 வகை பொருட்கள், சேவைகளின் குறைக்கப்பட்ட ஜிஎஸ்டி வரி இன்று முதல் அமலுக்கு வந்தது.
Extracted Text :
மக்களக்கப் பத்தாண்ட பரிசாக, எல்இடடவ, சனிமா டக்ககட,் கம்ப்யட்டர் மானிட்டர,்
மாற்றத்தறனாளிகளக்கானஉதரிப்பாகங்கள் உள்ளிட்ட 23
வகககபாரடக்ள், சசகவகளின்
ககறக்கப்படட் ஜஎஸ்ட வரி இன்றமதல் அமலக்கவந்தத.
UPDATE:
If I convert the PDF to JPG and use TesseractOCRParser and mention the language as Tamil then it works lot better than the PDFParser
apache-tika
asked Jan 1 at 5:06
wolverinewolverine
329319
2
If you select the text in Acrobat and cut-n-paste it out to something like notepad, do you get the right text, or basically the same that Tika gets?
– Gagravarr
Jan 2 at 11:44
@Gagravarr Just checked, copy paste also not working properly, same as Tika
– wolverine
Jan 2 at 13:01
1
Right, your problem is with whatever produced the PDF then. It has almost certainly done some bad OCR, and put that text behind the image of the scan. Your only option is to tell Apache Tika to re-OCR it, and ignore the text in the PDF
– Gagravarr
Jan 2 at 23:44
@Gagravarr I am now doing ocr_only strategy and it works lot better, thanks
– wolverine
Jan 3 at 5:22
add a comment |
I tried to extract text from PDF, it is working fine for English but not for Indian languages like Hindi, Tamil, Malayalam etc... See the sample below, all the bold text in the extracted content is not correct. Does Tika work with non English languages?
Original Text :
மக்களுக்குப் புத்தாண்டு பரிசாக, எல்இடி டிவி, சினிமா டிக்கெட், கம்ப்யூட்டர் மானிட்டர், மாற்றுத்திறனாளிகளுக்கான உதிரிப்பாகங்கள் உள்ளிட்ட 23 வகை பொருட்கள், சேவைகளின் குறைக்கப்பட்ட ஜிஎஸ்டி வரி இன்று முதல் அமலுக்கு வந்தது.
Extracted Text :
மக்களக்கப் பத்தாண்ட பரிசாக, எல்இடடவ, சனிமா டக்ககட,் கம்ப்யட்டர் மானிட்டர,்
மாற்றத்தறனாளிகளக்கானஉதரிப்பாகங்கள் உள்ளிட்ட 23
வகககபாரடக்ள், சசகவகளின்
ககறக்கப்படட் ஜஎஸ்ட வரி இன்றமதல் அமலக்கவந்தத.
UPDATE:
If I convert the PDF to JPG and use TesseractOCRParser and mention the language as Tamil then it works lot better than the PDFParser
apache-tika
asked Jan 1 at 5:06
wolverinewolverine
329319
I tried to extract text from PDF, it is working fine for English but not for Indian languages like Hindi, Tamil, Malayalam etc... See the sample below, all the bold text in the extracted content is not correct. Does Tika work with non English languages?
Original Text :
மக்களுக்குப் புத்தாண்டு பரிசாக, எல்இடி டிவி, சினிமா டிக்கெட், கம்ப்யூட்டர் மானிட்டர், மாற்றுத்திறனாளிகளுக்கான உதிரிப்பாகங்கள் உள்ளிட்ட 23 வகை பொருட்கள், சேவைகளின் குறைக்கப்பட்ட ஜிஎஸ்டி வரி இன்று முதல் அமலுக்கு வந்தது.
Extracted Text :
மக்களக்கப் பத்தாண்ட பரிசாக, எல்இடடவ, சனிமா டக்ககட,் கம்ப்யட்டர் மானிட்டர,்
மாற்றத்தறனாளிகளக்கானஉதரிப்பாகங்கள் உள்ளிட்ட 23
வகககபாரடக்ள், சசகவகளின்
ககறக்கப்படட் ஜஎஸ்ட வரி இன்றமதல் அமலக்கவந்தத.
UPDATE:
If I convert the PDF to JPG and use TesseractOCRParser and mention the language as Tamil then it works lot better than the PDFParser
apache-tika
apache-tika
asked Jan 1 at 5:06
wolverinewolverine
329319
asked Jan 1 at 5:06
wolverinewolverine
329319
edited Jan 2 at 10:26
wolverine
asked Jan 1 at 5:06
wolverinewolverine
329319
asked Jan 1 at 5:06
wolverinewolverine
329319
asked Jan 1 at 5:06
wolverinewolverine
329319
329319
2
If you select the text in Acrobat and cut-n-paste it out to something like notepad, do you get the right text, or basically the same that Tika gets?
– Gagravarr
Jan 2 at 11:44
@Gagravarr Just checked, copy paste also not working properly, same as Tika
– wolverine
Jan 2 at 13:01
1
Right, your problem is with whatever produced the PDF then. It has almost certainly done some bad OCR, and put that text behind the image of the scan. Your only option is to tell Apache Tika to re-OCR it, and ignore the text in the PDF
– Gagravarr
Jan 2 at 23:44
@Gagravarr I am now doing ocr_only strategy and it works lot better, thanks
– wolverine
Jan 3 at 5:22
add a comment |
2
If you select the text in Acrobat and cut-n-paste it out to something like notepad, do you get the right text, or basically the same that Tika gets?
– Gagravarr
Jan 2 at 11:44
@Gagravarr Just checked, copy paste also not working properly, same as Tika
– wolverine
Jan 2 at 13:01
1
Right, your problem is with whatever produced the PDF then. It has almost certainly done some bad OCR, and put that text behind the image of the scan. Your only option is to tell Apache Tika to re-OCR it, and ignore the text in the PDF
– Gagravarr
Jan 2 at 23:44
@Gagravarr I am now doing ocr_only strategy and it works lot better, thanks
– wolverine
Jan 3 at 5:22
2
2
If you select the text in Acrobat and cut-n-paste it out to something like notepad, do you get the right text, or basically the same that Tika gets?
– Gagravarr
Jan 2 at 11:44
If you select the text in Acrobat and cut-n-paste it out to something like notepad, do you get the right text, or basically the same that Tika gets?
– Gagravarr
Jan 2 at 11:44
@Gagravarr Just checked, copy paste also not working properly, same as Tika
– wolverine
Jan 2 at 13:01
@Gagravarr Just checked, copy paste also not working properly, same as Tika
– wolverine
Jan 2 at 13:01
1
1
Right, your problem is with whatever produced the PDF then. It has almost certainly done some bad OCR, and put that text behind the image of the scan. Your only option is to tell Apache Tika to re-OCR it, and ignore the text in the PDF
– Gagravarr
Jan 2 at 23:44
Right, your problem is with whatever produced the PDF then. It has almost certainly done some bad OCR, and put that text behind the image of the scan. Your only option is to tell Apache Tika to re-OCR it, and ignore the text in the PDF
– Gagravarr
Jan 2 at 23:44
@Gagravarr I am now doing ocr_only strategy and it works lot better, thanks
– wolverine
Jan 3 at 5:22
@Gagravarr I am now doing ocr_only strategy and it works lot better, thanks
– wolverine
Jan 3 at 5:22
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53993124%2fapache-tika-not-properly-extracting-text-from-pdf-for-indian-languages%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53993124%2fapache-tika-not-properly-extracting-text-from-pdf-for-indian-languages%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
If you select the text in Acrobat and cut-n-paste it out to something like notepad, do you get the right text, or basically the same that Tika gets?
– Gagravarr
Jan 2 at 11:44
@Gagravarr Just checked, copy paste also not working properly, same as Tika
– wolverine
Jan 2 at 13:01
1
Right, your problem is with whatever produced the PDF then. It has almost certainly done some bad OCR, and put that text behind the image of the scan. Your only option is to tell Apache Tika to re-OCR it, and ignore the text in the PDF
– Gagravarr
Jan 2 at 23:44
@Gagravarr I am now doing ocr_only strategy and it works lot better, thanks
– wolverine
Jan 3 at 5:22