Mixing
Elasticsearch find a word from local json file [PYTHON]
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
0
I try to use Elasticsearch due to get some values and frequencies from local json file with python.
Json file has multiple terms shown as below;
[{"id": "251088", "tweet": "lorem ipsum", "username": "Ahmet"},
{"id": "251059", "tweet": "bla bla bla","username": "Ali", },
...
]
The json file consists approximately 500K tweets and info.
I aim to get term frequency with elasticsearch faster.
import requests, json, os
from elasticsearch import Elasticsearch
es = Elasticsearch([{'host': 'localhost', 'port': 9200}])
i = 1
f = open("tweets_test.json")
docket_content = f.read()
#print(docket_content)
# only wait for 1 second, regardless of the client's default
es.cluster.health(wait_for_status='yellow', request_timeout=1)
es.index(index='tweets', ignore=[400, 404], doc_type='docket', id=i, body=json.loads(docket_content))
res = es.search(index="tweets", doc_type="docket", body={"query": {"match": {"tweet": "any-word"}}})
print("%d documents found" % res['hits']['total'])
for doc in res['hits']['hits']:
print("%s) %s" % (doc['_id'], doc['_source']['content']))
the output is;
0 documents found
Process finished with exit code 0
- Why didn't work this code?
- Am I wrong platform to getting term frequency?
python
elasticsearch asked Jan 3 at 13:15
babeyhbabeyh
1431110
add a comment |
0
I try to use Elasticsearch due to get some values and frequencies from local json file with python.
Json file has multiple terms shown as below;
[{"id": "251088", "tweet": "lorem ipsum", "username": "Ahmet"},
{"id": "251059", "tweet": "bla bla bla","username": "Ali", },
...
]
The json file consists approximately 500K tweets and info.
I aim to get term frequency with elasticsearch faster.
import requests, json, os
from elasticsearch import Elasticsearch
es = Elasticsearch([{'host': 'localhost', 'port': 9200}])
i = 1
f = open("tweets_test.json")
docket_content = f.read()
#print(docket_content)
# only wait for 1 second, regardless of the client's default
es.cluster.health(wait_for_status='yellow', request_timeout=1)
es.index(index='tweets', ignore=[400, 404], doc_type='docket', id=i, body=json.loads(docket_content))
res = es.search(index="tweets", doc_type="docket", body={"query": {"match": {"tweet": "any-word"}}})
print("%d documents found" % res['hits']['total'])
for doc in res['hits']['hits']:
print("%s) %s" % (doc['_id'], doc['_source']['content']))
the output is;
0 documents found
Process finished with exit code 0
- Why didn't work this code?
- Am I wrong platform to getting term frequency?
python
elasticsearch asked Jan 3 at 13:15
babeyhbabeyh
1431110
According you, what you expect searching for "ve"? If you have a default mapping, the field tweet assumes standard analyzer. If you search for "ve" in this field, es should retrieve all the "ve" term or "ve" as stem . Why you expect "ve" terms in your text?
– Lupanoide
Jan 3 at 13:27
"ve" is not important term. I has just try to find anyword.
– babeyh
Jan 4 at 11:14
Although "ve" words is absolutely exist in most tweets. But ES returns any docs. I didnt understand this output. Actually, the main problem is to find word frequencies in all tweets using ES. Due to learn ES, I just try to some exercises (find a word)
– babeyh
Jan 4 at 11:22
to find word frequencies in all tweets, take a look here: stackoverflow.com/questions/50085839/…
– Lupanoide
Jan 4 at 13:11
add a comment |
0
0
0
I try to use Elasticsearch due to get some values and frequencies from local json file with python.
Json file has multiple terms shown as below;
[{"id": "251088", "tweet": "lorem ipsum", "username": "Ahmet"},
{"id": "251059", "tweet": "bla bla bla","username": "Ali", },
...
]
The json file consists approximately 500K tweets and info.
I aim to get term frequency with elasticsearch faster.
import requests, json, os
from elasticsearch import Elasticsearch
es = Elasticsearch([{'host': 'localhost', 'port': 9200}])
i = 1
f = open("tweets_test.json")
docket_content = f.read()
#print(docket_content)
# only wait for 1 second, regardless of the client's default
es.cluster.health(wait_for_status='yellow', request_timeout=1)
es.index(index='tweets', ignore=[400, 404], doc_type='docket', id=i, body=json.loads(docket_content))
res = es.search(index="tweets", doc_type="docket", body={"query": {"match": {"tweet": "any-word"}}})
print("%d documents found" % res['hits']['total'])
for doc in res['hits']['hits']:
print("%s) %s" % (doc['_id'], doc['_source']['content']))
the output is;
0 documents found
Process finished with exit code 0
- Why didn't work this code?
- Am I wrong platform to getting term frequency?
python
elasticsearch asked Jan 3 at 13:15
babeyhbabeyh
1431110
I try to use Elasticsearch due to get some values and frequencies from local json file with python.
Json file has multiple terms shown as below;
[{"id": "251088", "tweet": "lorem ipsum", "username": "Ahmet"},
{"id": "251059", "tweet": "bla bla bla","username": "Ali", },
...
]
The json file consists approximately 500K tweets and info.
I aim to get term frequency with elasticsearch faster.
import requests, json, os
from elasticsearch import Elasticsearch
es = Elasticsearch([{'host': 'localhost', 'port': 9200}])
i = 1
f = open("tweets_test.json")
docket_content = f.read()
#print(docket_content)
# only wait for 1 second, regardless of the client's default
es.cluster.health(wait_for_status='yellow', request_timeout=1)
es.index(index='tweets', ignore=[400, 404], doc_type='docket', id=i, body=json.loads(docket_content))
res = es.search(index="tweets", doc_type="docket", body={"query": {"match": {"tweet": "any-word"}}})
print("%d documents found" % res['hits']['total'])
for doc in res['hits']['hits']:
print("%s) %s" % (doc['_id'], doc['_source']['content']))
the output is;
0 documents found
Process finished with exit code 0
- Why didn't work this code?
- Am I wrong platform to getting term frequency?
python
elasticsearch python
elasticsearch asked Jan 3 at 13:15
babeyhbabeyh
1431110
asked Jan 3 at 13:15
babeyhbabeyh
1431110
edited Jan 4 at 10:26
babeyh
asked Jan 3 at 13:15
babeyhbabeyh
1431110
asked Jan 3 at 13:15
babeyhbabeyh
1431110
asked Jan 3 at 13:15
babeyhbabeyh
1431110
1431110
According you, what you expect searching for "ve"? If you have a default mapping, the field tweet assumes standard analyzer. If you search for "ve" in this field, es should retrieve all the "ve" term or "ve" as stem . Why you expect "ve" terms in your text?
– Lupanoide
Jan 3 at 13:27
"ve" is not important term. I has just try to find anyword.
– babeyh
Jan 4 at 11:14
Although "ve" words is absolutely exist in most tweets. But ES returns any docs. I didnt understand this output. Actually, the main problem is to find word frequencies in all tweets using ES. Due to learn ES, I just try to some exercises (find a word)
– babeyh
Jan 4 at 11:22
to find word frequencies in all tweets, take a look here: stackoverflow.com/questions/50085839/…
– Lupanoide
Jan 4 at 13:11
add a comment |
According you, what you expect searching for "ve"? If you have a default mapping, the field tweet assumes standard analyzer. If you search for "ve" in this field, es should retrieve all the "ve" term or "ve" as stem . Why you expect "ve" terms in your text?
– Lupanoide
Jan 3 at 13:27
"ve" is not important term. I has just try to find anyword.
– babeyh
Jan 4 at 11:14
Although "ve" words is absolutely exist in most tweets. But ES returns any docs. I didnt understand this output. Actually, the main problem is to find word frequencies in all tweets using ES. Due to learn ES, I just try to some exercises (find a word)
– babeyh
Jan 4 at 11:22
to find word frequencies in all tweets, take a look here: stackoverflow.com/questions/50085839/…
– Lupanoide
Jan 4 at 13:11
According you, what you expect searching for "ve"? If you have a default mapping, the field tweet assumes standard analyzer. If you search for "ve" in this field, es should retrieve all the "ve" term or "ve" as stem . Why you expect "ve" terms in your text?
– Lupanoide
Jan 3 at 13:27
According you, what you expect searching for "ve"? If you have a default mapping, the field tweet assumes standard analyzer. If you search for "ve" in this field, es should retrieve all the "ve" term or "ve" as stem . Why you expect "ve" terms in your text?
– Lupanoide
Jan 3 at 13:27
"ve" is not important term. I has just try to find anyword.
– babeyh
Jan 4 at 11:14
"ve" is not important term. I has just try to find anyword.
– babeyh
Jan 4 at 11:14
Although "ve" words is absolutely exist in most tweets. But ES returns any docs. I didnt understand this output. Actually, the main problem is to find word frequencies in all tweets using ES. Due to learn ES, I just try to some exercises (find a word)
– babeyh
Jan 4 at 11:22
Although "ve" words is absolutely exist in most tweets. But ES returns any docs. I didnt understand this output. Actually, the main problem is to find word frequencies in all tweets using ES. Due to learn ES, I just try to some exercises (find a word)
– babeyh
Jan 4 at 11:22
to find word frequencies in all tweets, take a look here: stackoverflow.com/questions/50085839/…
– Lupanoide
Jan 4 at 13:11
to find word frequencies in all tweets, take a look here: stackoverflow.com/questions/50085839/…
– Lupanoide
Jan 4 at 13:11
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54023056%2felasticsearch-find-a-word-from-local-json-file-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54023056%2felasticsearch-find-a-word-from-local-json-file-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
According you, what you expect searching for "ve"? If you have a default mapping, the field tweet assumes standard analyzer. If you search for "ve" in this field, es should retrieve all the "ve" term or "ve" as stem . Why you expect "ve" terms in your text?
– Lupanoide
Jan 3 at 13:27
"ve" is not important term. I has just try to find anyword.
– babeyh
Jan 4 at 11:14
Although "ve" words is absolutely exist in most tweets. But ES returns any docs. I didnt understand this output. Actually, the main problem is to find word frequencies in all tweets using ES. Due to learn ES, I just try to some exercises (find a word)
– babeyh
Jan 4 at 11:22
to find word frequencies in all tweets, take a look here: stackoverflow.com/questions/50085839/…
– Lupanoide
Jan 4 at 13:11