Mixing
How to get a sub-field of a struct type map, in the search response of YQL query in Vespa?
Sample Data:
"fields": {
"key1":0,
"key2":"no",
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
},
"ch": {
"firstName": "维克兰特",
"lastName":"塔库尔"
}
}
}
Expected Response:
"fields": {
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
}
I have added the following in my search-definition demo.sd:
struct lang {
field firstName type string {}
field lastName type string {}
}
field Lang type map <string, lang> {
indexing: summary
struct-field key {
indexing: summary | index | attribute
}
}
I want to write a yql query something like this (This doesn't work):
http://localhost:8080/search/?yql=select Lang.en from sources demo where key2 contains 'no';
My temporary workaround approach
I have implemented a custom searcher in MySearcher.java, through which I am able to extract the required sub-field and set a new field 'defaultLang', and remove the 'Lang' field. The response generated by the searcher:
"fields": {
"defaultLang": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
I have written the following in MySearcher.java:
for (Hit hit: result.hits()) {
String language = "en"; //temporarily hard-coded
StructuredData Lang = (StructuredData) hit.getField("Lang");
Inspector o = Lang.inspect();
for (int j=0;j<o.entryCount();j++){
if (o.entry(j).field("key").asString("").equals(language)){
SlimeAdapter value = (SlimeAdapter) o.entry(j).field("value");
hit.setField("defaultLang",value);
break;
}
}
hit.removeField("Lang");
}
Edit-1: A more efficient way instead is to make use of the Inspectable interface and Inspector, like above (Thanks to @Jo Kristian Bergum)
But, in the above code, I am having to loop through all the languages to filter out the required one. I want to avoid this O(n) time-complexity and make use of the map structure to access it in O(1). (Because the languages may increase to 1000, and this would be done for each hit.)
All this is due to the StructuredData data type I am getting in the results. StructureData doesn't keep the Map Structure and rather gives an array of JSON like:
[{
"key": "en",
"value": {
"firstName": "Vikrant",
"lastName": "Thakur"
}
}, {
"key": "ch",
"value": {
"firstName": "维克兰特",
"lastName": "塔库尔"
}
}]
Please, suggest a better approach altogether, or any help with my current one. Both are appreciated.
yql vespa
asked Jan 2 at 8:54
Vikrant ThakurVikrant Thakur
334
add a comment |
Sample Data:
"fields": {
"key1":0,
"key2":"no",
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
},
"ch": {
"firstName": "维克兰特",
"lastName":"塔库尔"
}
}
}
Expected Response:
"fields": {
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
}
I have added the following in my search-definition demo.sd:
struct lang {
field firstName type string {}
field lastName type string {}
}
field Lang type map <string, lang> {
indexing: summary
struct-field key {
indexing: summary | index | attribute
}
}
I want to write a yql query something like this (This doesn't work):
http://localhost:8080/search/?yql=select Lang.en from sources demo where key2 contains 'no';
My temporary workaround approach
I have implemented a custom searcher in MySearcher.java, through which I am able to extract the required sub-field and set a new field 'defaultLang', and remove the 'Lang' field. The response generated by the searcher:
"fields": {
"defaultLang": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
I have written the following in MySearcher.java:
for (Hit hit: result.hits()) {
String language = "en"; //temporarily hard-coded
StructuredData Lang = (StructuredData) hit.getField("Lang");
Inspector o = Lang.inspect();
for (int j=0;j<o.entryCount();j++){
if (o.entry(j).field("key").asString("").equals(language)){
SlimeAdapter value = (SlimeAdapter) o.entry(j).field("value");
hit.setField("defaultLang",value);
break;
}
}
hit.removeField("Lang");
}
Edit-1: A more efficient way instead is to make use of the Inspectable interface and Inspector, like above (Thanks to @Jo Kristian Bergum)
But, in the above code, I am having to loop through all the languages to filter out the required one. I want to avoid this O(n) time-complexity and make use of the map structure to access it in O(1). (Because the languages may increase to 1000, and this would be done for each hit.)
All this is due to the StructuredData data type I am getting in the results. StructureData doesn't keep the Map Structure and rather gives an array of JSON like:
[{
"key": "en",
"value": {
"firstName": "Vikrant",
"lastName": "Thakur"
}
}, {
"key": "ch",
"value": {
"firstName": "维克兰特",
"lastName": "塔库尔"
}
}]
Please, suggest a better approach altogether, or any help with my current one. Both are appreciated.
yql vespa
asked Jan 2 at 8:54
Vikrant ThakurVikrant Thakur
334
add a comment |
Sample Data:
"fields": {
"key1":0,
"key2":"no",
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
},
"ch": {
"firstName": "维克兰特",
"lastName":"塔库尔"
}
}
}
Expected Response:
"fields": {
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
}
I have added the following in my search-definition demo.sd:
struct lang {
field firstName type string {}
field lastName type string {}
}
field Lang type map <string, lang> {
indexing: summary
struct-field key {
indexing: summary | index | attribute
}
}
I want to write a yql query something like this (This doesn't work):
http://localhost:8080/search/?yql=select Lang.en from sources demo where key2 contains 'no';
My temporary workaround approach
I have implemented a custom searcher in MySearcher.java, through which I am able to extract the required sub-field and set a new field 'defaultLang', and remove the 'Lang' field. The response generated by the searcher:
"fields": {
"defaultLang": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
I have written the following in MySearcher.java:
for (Hit hit: result.hits()) {
String language = "en"; //temporarily hard-coded
StructuredData Lang = (StructuredData) hit.getField("Lang");
Inspector o = Lang.inspect();
for (int j=0;j<o.entryCount();j++){
if (o.entry(j).field("key").asString("").equals(language)){
SlimeAdapter value = (SlimeAdapter) o.entry(j).field("value");
hit.setField("defaultLang",value);
break;
}
}
hit.removeField("Lang");
}
Edit-1: A more efficient way instead is to make use of the Inspectable interface and Inspector, like above (Thanks to @Jo Kristian Bergum)
But, in the above code, I am having to loop through all the languages to filter out the required one. I want to avoid this O(n) time-complexity and make use of the map structure to access it in O(1). (Because the languages may increase to 1000, and this would be done for each hit.)
All this is due to the StructuredData data type I am getting in the results. StructureData doesn't keep the Map Structure and rather gives an array of JSON like:
[{
"key": "en",
"value": {
"firstName": "Vikrant",
"lastName": "Thakur"
}
}, {
"key": "ch",
"value": {
"firstName": "维克兰特",
"lastName": "塔库尔"
}
}]
Please, suggest a better approach altogether, or any help with my current one. Both are appreciated.
yql vespa
asked Jan 2 at 8:54
Vikrant ThakurVikrant Thakur
334
Sample Data:
"fields": {
"key1":0,
"key2":"no",
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
},
"ch": {
"firstName": "维克兰特",
"lastName":"塔库尔"
}
}
}
Expected Response:
"fields": {
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
}
I have added the following in my search-definition demo.sd:
struct lang {
field firstName type string {}
field lastName type string {}
}
field Lang type map <string, lang> {
indexing: summary
struct-field key {
indexing: summary | index | attribute
}
}
I want to write a yql query something like this (This doesn't work):
http://localhost:8080/search/?yql=select Lang.en from sources demo where key2 contains 'no';
My temporary workaround approach
I have implemented a custom searcher in MySearcher.java, through which I am able to extract the required sub-field and set a new field 'defaultLang', and remove the 'Lang' field. The response generated by the searcher:
"fields": {
"defaultLang": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
I have written the following in MySearcher.java:
for (Hit hit: result.hits()) {
String language = "en"; //temporarily hard-coded
StructuredData Lang = (StructuredData) hit.getField("Lang");
Inspector o = Lang.inspect();
for (int j=0;j<o.entryCount();j++){
if (o.entry(j).field("key").asString("").equals(language)){
SlimeAdapter value = (SlimeAdapter) o.entry(j).field("value");
hit.setField("defaultLang",value);
break;
}
}
hit.removeField("Lang");
}
Edit-1: A more efficient way instead is to make use of the Inspectable interface and Inspector, like above (Thanks to @Jo Kristian Bergum)
But, in the above code, I am having to loop through all the languages to filter out the required one. I want to avoid this O(n) time-complexity and make use of the map structure to access it in O(1). (Because the languages may increase to 1000, and this would be done for each hit.)
All this is due to the StructuredData data type I am getting in the results. StructureData doesn't keep the Map Structure and rather gives an array of JSON like:
[{
"key": "en",
"value": {
"firstName": "Vikrant",
"lastName": "Thakur"
}
}, {
"key": "ch",
"value": {
"firstName": "维克兰特",
"lastName": "塔库尔"
}
}]
Please, suggest a better approach altogether, or any help with my current one. Both are appreciated.
yql vespa
yql vespa
asked Jan 2 at 8:54
Vikrant ThakurVikrant Thakur
334
asked Jan 2 at 8:54
Vikrant ThakurVikrant Thakur
334
edited Jan 2 at 13:39
Vikrant Thakur
asked Jan 2 at 8:54
Vikrant ThakurVikrant Thakur
334
asked Jan 2 at 8:54
Vikrant ThakurVikrant Thakur
334
asked Jan 2 at 8:54
Vikrant ThakurVikrant Thakur
334
334
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
The YQL sample query I guess is to illustrate what you want as that syntax is not valid. Picking a given key from the field Lang of type map can be done as you do in your searcher but deserializing into JSON and parsing the JSON is probably inefficient as StructuredData implements the Inspectable interface and you can inspect it directly without the need to go through JSON format. See https://docs.vespa.ai/documentation/reference/inspecting-structured-data.html
answered Jan 2 at 10:39
Jo Kristian BergumJo Kristian Bergum
1,22616
Thank you, @Jo Kristian Bergum for your answer. I created the Inspector object using the Inspectable interface. But, that too creates an array with 2 entries (for the two languages with keys "key" and "value") like the one I mentioned above, and I have to loop through the entries to pick a specific key. I don't understand why Vespa is deserializing the map data type on its own. Please, help.
– Vikrant Thakur
Jan 2 at 12:21
Yeah, I agree that handling of map or weighted sets is not optimal when working on Hit field data. AFAIK you are not able to do a direct lookup in the map but with my suggestion you avoid producing JSON, then parsing the JSON when the data could be accessed directly (Iterating though):
– Jo Kristian Bergum
Jan 2 at 12:34
Ok. I agree this would be more efficient. Can you tell me if it's somehow possible at the YQL level itself? Maybe by changing my search-definition data types, indexing, or a completely different Data Architecture. Thanks again for the help.
– Vikrant Thakur
Jan 2 at 13:07
No, there is no support for this via YQL to only chose a given key in a map for display. As far as I can tell you have an optimal solution already (given the limitations of the field api).
– Jo Kristian Bergum
Jan 2 at 13:23
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54003509%2fhow-to-get-a-sub-field-of-a-struct-type-map-in-the-search-response-of-yql-query%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The YQL sample query I guess is to illustrate what you want as that syntax is not valid. Picking a given key from the field Lang of type map can be done as you do in your searcher but deserializing into JSON and parsing the JSON is probably inefficient as StructuredData implements the Inspectable interface and you can inspect it directly without the need to go through JSON format. See https://docs.vespa.ai/documentation/reference/inspecting-structured-data.html
answered Jan 2 at 10:39
Jo Kristian BergumJo Kristian Bergum
1,22616
Thank you, @Jo Kristian Bergum for your answer. I created the Inspector object using the Inspectable interface. But, that too creates an array with 2 entries (for the two languages with keys "key" and "value") like the one I mentioned above, and I have to loop through the entries to pick a specific key. I don't understand why Vespa is deserializing the map data type on its own. Please, help.
– Vikrant Thakur
Jan 2 at 12:21
Yeah, I agree that handling of map or weighted sets is not optimal when working on Hit field data. AFAIK you are not able to do a direct lookup in the map but with my suggestion you avoid producing JSON, then parsing the JSON when the data could be accessed directly (Iterating though):
– Jo Kristian Bergum
Jan 2 at 12:34
Ok. I agree this would be more efficient. Can you tell me if it's somehow possible at the YQL level itself? Maybe by changing my search-definition data types, indexing, or a completely different Data Architecture. Thanks again for the help.
– Vikrant Thakur
Jan 2 at 13:07
No, there is no support for this via YQL to only chose a given key in a map for display. As far as I can tell you have an optimal solution already (given the limitations of the field api).
– Jo Kristian Bergum
Jan 2 at 13:23
add a comment |
The YQL sample query I guess is to illustrate what you want as that syntax is not valid. Picking a given key from the field Lang of type map can be done as you do in your searcher but deserializing into JSON and parsing the JSON is probably inefficient as StructuredData implements the Inspectable interface and you can inspect it directly without the need to go through JSON format. See https://docs.vespa.ai/documentation/reference/inspecting-structured-data.html
answered Jan 2 at 10:39
Jo Kristian BergumJo Kristian Bergum
1,22616
Thank you, @Jo Kristian Bergum for your answer. I created the Inspector object using the Inspectable interface. But, that too creates an array with 2 entries (for the two languages with keys "key" and "value") like the one I mentioned above, and I have to loop through the entries to pick a specific key. I don't understand why Vespa is deserializing the map data type on its own. Please, help.
– Vikrant Thakur
Jan 2 at 12:21
Yeah, I agree that handling of map or weighted sets is not optimal when working on Hit field data. AFAIK you are not able to do a direct lookup in the map but with my suggestion you avoid producing JSON, then parsing the JSON when the data could be accessed directly (Iterating though):
– Jo Kristian Bergum
Jan 2 at 12:34
Ok. I agree this would be more efficient. Can you tell me if it's somehow possible at the YQL level itself? Maybe by changing my search-definition data types, indexing, or a completely different Data Architecture. Thanks again for the help.
– Vikrant Thakur
Jan 2 at 13:07
No, there is no support for this via YQL to only chose a given key in a map for display. As far as I can tell you have an optimal solution already (given the limitations of the field api).
– Jo Kristian Bergum
Jan 2 at 13:23
add a comment |
The YQL sample query I guess is to illustrate what you want as that syntax is not valid. Picking a given key from the field Lang of type map can be done as you do in your searcher but deserializing into JSON and parsing the JSON is probably inefficient as StructuredData implements the Inspectable interface and you can inspect it directly without the need to go through JSON format. See https://docs.vespa.ai/documentation/reference/inspecting-structured-data.html
answered Jan 2 at 10:39
Jo Kristian BergumJo Kristian Bergum
1,22616
The YQL sample query I guess is to illustrate what you want as that syntax is not valid. Picking a given key from the field Lang of type map can be done as you do in your searcher but deserializing into JSON and parsing the JSON is probably inefficient as StructuredData implements the Inspectable interface and you can inspect it directly without the need to go through JSON format. See https://docs.vespa.ai/documentation/reference/inspecting-structured-data.html
answered Jan 2 at 10:39
Jo Kristian BergumJo Kristian Bergum
1,22616
answered Jan 2 at 10:39
Jo Kristian BergumJo Kristian Bergum
1,22616
answered Jan 2 at 10:39
Jo Kristian BergumJo Kristian Bergum
1,22616
answered Jan 2 at 10:39
Jo Kristian BergumJo Kristian Bergum
1,22616
1,22616
Thank you, @Jo Kristian Bergum for your answer. I created the Inspector object using the Inspectable interface. But, that too creates an array with 2 entries (for the two languages with keys "key" and "value") like the one I mentioned above, and I have to loop through the entries to pick a specific key. I don't understand why Vespa is deserializing the map data type on its own. Please, help.
– Vikrant Thakur
Jan 2 at 12:21
Yeah, I agree that handling of map or weighted sets is not optimal when working on Hit field data. AFAIK you are not able to do a direct lookup in the map but with my suggestion you avoid producing JSON, then parsing the JSON when the data could be accessed directly (Iterating though):
– Jo Kristian Bergum
Jan 2 at 12:34
Ok. I agree this would be more efficient. Can you tell me if it's somehow possible at the YQL level itself? Maybe by changing my search-definition data types, indexing, or a completely different Data Architecture. Thanks again for the help.
– Vikrant Thakur
Jan 2 at 13:07
No, there is no support for this via YQL to only chose a given key in a map for display. As far as I can tell you have an optimal solution already (given the limitations of the field api).
– Jo Kristian Bergum
Jan 2 at 13:23
add a comment |
Thank you, @Jo Kristian Bergum for your answer. I created the Inspector object using the Inspectable interface. But, that too creates an array with 2 entries (for the two languages with keys "key" and "value") like the one I mentioned above, and I have to loop through the entries to pick a specific key. I don't understand why Vespa is deserializing the map data type on its own. Please, help.
– Vikrant Thakur
Jan 2 at 12:21
Yeah, I agree that handling of map or weighted sets is not optimal when working on Hit field data. AFAIK you are not able to do a direct lookup in the map but with my suggestion you avoid producing JSON, then parsing the JSON when the data could be accessed directly (Iterating though):
– Jo Kristian Bergum
Jan 2 at 12:34
Ok. I agree this would be more efficient. Can you tell me if it's somehow possible at the YQL level itself? Maybe by changing my search-definition data types, indexing, or a completely different Data Architecture. Thanks again for the help.
– Vikrant Thakur
Jan 2 at 13:07
No, there is no support for this via YQL to only chose a given key in a map for display. As far as I can tell you have an optimal solution already (given the limitations of the field api).
– Jo Kristian Bergum
Jan 2 at 13:23
Thank you, @Jo Kristian Bergum for your answer. I created the Inspector object using the Inspectable interface. But, that too creates an array with 2 entries (for the two languages with keys "key" and "value") like the one I mentioned above, and I have to loop through the entries to pick a specific key. I don't understand why Vespa is deserializing the map data type on its own. Please, help.
– Vikrant Thakur
Jan 2 at 12:21
Thank you, @Jo Kristian Bergum for your answer. I created the Inspector object using the Inspectable interface. But, that too creates an array with 2 entries (for the two languages with keys "key" and "value") like the one I mentioned above, and I have to loop through the entries to pick a specific key. I don't understand why Vespa is deserializing the map data type on its own. Please, help.
– Vikrant Thakur
Jan 2 at 12:21
Yeah, I agree that handling of map or weighted sets is not optimal when working on Hit field data. AFAIK you are not able to do a direct lookup in the map but with my suggestion you avoid producing JSON, then parsing the JSON when the data could be accessed directly (Iterating though):
– Jo Kristian Bergum
Jan 2 at 12:34
Yeah, I agree that handling of map or weighted sets is not optimal when working on Hit field data. AFAIK you are not able to do a direct lookup in the map but with my suggestion you avoid producing JSON, then parsing the JSON when the data could be accessed directly (Iterating though):
– Jo Kristian Bergum
Jan 2 at 12:34
Ok. I agree this would be more efficient. Can you tell me if it's somehow possible at the YQL level itself? Maybe by changing my search-definition data types, indexing, or a completely different Data Architecture. Thanks again for the help.
– Vikrant Thakur
Jan 2 at 13:07
Ok. I agree this would be more efficient. Can you tell me if it's somehow possible at the YQL level itself? Maybe by changing my search-definition data types, indexing, or a completely different Data Architecture. Thanks again for the help.
– Vikrant Thakur
Jan 2 at 13:07
No, there is no support for this via YQL to only chose a given key in a map for display. As far as I can tell you have an optimal solution already (given the limitations of the field api).
– Jo Kristian Bergum
Jan 2 at 13:23
No, there is no support for this via YQL to only chose a given key in a map for display. As far as I can tell you have an optimal solution already (given the limitations of the field api).
– Jo Kristian Bergum
Jan 2 at 13:23
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54003509%2fhow-to-get-a-sub-field-of-a-struct-type-map-in-the-search-response-of-yql-query%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
