Mixing
How to do product catalog building with MySQL and tensorflow?
I am trying to build a product catalog with data from several e-commerce websites. The goal is to build a product catalog where every product is specified as good as possible using leveraged data across multiple sources.
This seems to be a highly complex task as there is sometimes misinformation and in some cases the unique identifier is misspelled or not even present.
The current approach is to transform the extracted data into our format and then load it into a mysql database. In this process obvious duplicates get removed and I end up with about 250.000 datasets.
Now I am facing the problem on how to brake this down even further as there are thousands of duplicates but I can not say which as some info might not be accurate.
e.g.
ref_id | title | img | color_id | size | length | diameter | dial_id
this one dataset might be incomplete or might even contain wrong values.
Looking more into the topic, this seems to be a common use case for deep learning with e.g. tensorflow.
I am looking for an answer that will help me in order to create the process on how to do that. Is tensorflow the right tool? Should I write all datasets to the db and keep the records? How could a process look like etc.
tensorflow
edited Jan 13 at 19:28
marc_s
582k13011231269
asked Jan 2 at 12:49
merlinmerlin
7761922
add a comment |
I am trying to build a product catalog with data from several e-commerce websites. The goal is to build a product catalog where every product is specified as good as possible using leveraged data across multiple sources.
This seems to be a highly complex task as there is sometimes misinformation and in some cases the unique identifier is misspelled or not even present.
The current approach is to transform the extracted data into our format and then load it into a mysql database. In this process obvious duplicates get removed and I end up with about 250.000 datasets.
Now I am facing the problem on how to brake this down even further as there are thousands of duplicates but I can not say which as some info might not be accurate.
e.g.
ref_id | title | img | color_id | size | length | diameter | dial_id
this one dataset might be incomplete or might even contain wrong values.
Looking more into the topic, this seems to be a common use case for deep learning with e.g. tensorflow.
I am looking for an answer that will help me in order to create the process on how to do that. Is tensorflow the right tool? Should I write all datasets to the db and keep the records? How could a process look like etc.
tensorflow
edited Jan 13 at 19:28
marc_s
582k13011231269
asked Jan 2 at 12:49
merlinmerlin
7761922
add a comment |
I am trying to build a product catalog with data from several e-commerce websites. The goal is to build a product catalog where every product is specified as good as possible using leveraged data across multiple sources.
This seems to be a highly complex task as there is sometimes misinformation and in some cases the unique identifier is misspelled or not even present.
The current approach is to transform the extracted data into our format and then load it into a mysql database. In this process obvious duplicates get removed and I end up with about 250.000 datasets.
Now I am facing the problem on how to brake this down even further as there are thousands of duplicates but I can not say which as some info might not be accurate.
e.g.
ref_id | title | img | color_id | size | length | diameter | dial_id
this one dataset might be incomplete or might even contain wrong values.
Looking more into the topic, this seems to be a common use case for deep learning with e.g. tensorflow.
I am looking for an answer that will help me in order to create the process on how to do that. Is tensorflow the right tool? Should I write all datasets to the db and keep the records? How could a process look like etc.
tensorflow
edited Jan 13 at 19:28
marc_s
582k13011231269
asked Jan 2 at 12:49
merlinmerlin
7761922
I am trying to build a product catalog with data from several e-commerce websites. The goal is to build a product catalog where every product is specified as good as possible using leveraged data across multiple sources.
This seems to be a highly complex task as there is sometimes misinformation and in some cases the unique identifier is misspelled or not even present.
The current approach is to transform the extracted data into our format and then load it into a mysql database. In this process obvious duplicates get removed and I end up with about 250.000 datasets.
Now I am facing the problem on how to brake this down even further as there are thousands of duplicates but I can not say which as some info might not be accurate.
e.g.
ref_id | title | img | color_id | size | length | diameter | dial_id
this one dataset might be incomplete or might even contain wrong values.
Looking more into the topic, this seems to be a common use case for deep learning with e.g. tensorflow.
I am looking for an answer that will help me in order to create the process on how to do that. Is tensorflow the right tool? Should I write all datasets to the db and keep the records? How could a process look like etc.
tensorflow
tensorflow
edited Jan 13 at 19:28
marc_s
582k13011231269
asked Jan 2 at 12:49
merlinmerlin
7761922
edited Jan 13 at 19:28
marc_s
582k13011231269
asked Jan 2 at 12:49
merlinmerlin
7761922
edited Jan 13 at 19:28
marc_s
582k13011231269
edited Jan 13 at 19:28
marc_s
582k13011231269
edited Jan 13 at 19:28
marc_s
582k13011231269
582k13011231269
asked Jan 2 at 12:49
merlinmerlin
7761922
asked Jan 2 at 12:49
merlinmerlin
7761922
asked Jan 2 at 12:49
merlinmerlin
7761922
7761922
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54006694%2fhow-to-do-product-catalog-building-with-mysql-and-tensorflow%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54006694%2fhow-to-do-product-catalog-building-with-mysql-and-tensorflow%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
