Mixing
How will I get the column data type of dataset in spark
-1
Want to compare the data types of two dataset's in spark using java.
apache-spark apache-spark-sql
asked Nov 22 '18 at 12:00
NareshNaresh
456
add a comment |
-1
Want to compare the data types of two dataset's in spark using java.
apache-spark apache-spark-sql
asked Nov 22 '18 at 12:00
NareshNaresh
456
1
Elaborate in more details. In which API? Java? The best way is to share your code because the code is our common language.
– Soheil Pourbafrani
Nov 22 '18 at 12:34
Hi Soheil , Here we want to check the schema of the existing table with the new table. if there is any change in the schema we need to drop of the existing table and create a new table and this comparsion based on the column name and column type
– Naresh
Nov 24 '18 at 15:18
add a comment |
-1
-1
-1
Want to compare the data types of two dataset's in spark using java.
apache-spark apache-spark-sql
asked Nov 22 '18 at 12:00
NareshNaresh
456
Want to compare the data types of two dataset's in spark using java.
apache-spark apache-spark-sql
apache-spark apache-spark-sql
asked Nov 22 '18 at 12:00
NareshNaresh
456
asked Nov 22 '18 at 12:00
NareshNaresh
456
asked Nov 22 '18 at 12:00
NareshNaresh
456
asked Nov 22 '18 at 12:00
NareshNaresh
456
asked Nov 22 '18 at 12:00
NareshNaresh
456
456
1
Elaborate in more details. In which API? Java? The best way is to share your code because the code is our common language.
– Soheil Pourbafrani
Nov 22 '18 at 12:34
Hi Soheil , Here we want to check the schema of the existing table with the new table. if there is any change in the schema we need to drop of the existing table and create a new table and this comparsion based on the column name and column type
– Naresh
Nov 24 '18 at 15:18
add a comment |
1
Elaborate in more details. In which API? Java? The best way is to share your code because the code is our common language.
– Soheil Pourbafrani
Nov 22 '18 at 12:34
Hi Soheil , Here we want to check the schema of the existing table with the new table. if there is any change in the schema we need to drop of the existing table and create a new table and this comparsion based on the column name and column type
– Naresh
Nov 24 '18 at 15:18
1
1
Elaborate in more details. In which API? Java? The best way is to share your code because the code is our common language.
– Soheil Pourbafrani
Nov 22 '18 at 12:34
Elaborate in more details. In which API? Java? The best way is to share your code because the code is our common language.
– Soheil Pourbafrani
Nov 22 '18 at 12:34
Hi Soheil , Here we want to check the schema of the existing table with the new table. if there is any change in the schema we need to drop of the existing table and create a new table and this comparsion based on the column name and column type
– Naresh
Nov 24 '18 at 15:18
Hi Soheil , Here we want to check the schema of the existing table with the new table. if there is any change in the schema we need to drop of the existing table and create a new table and this comparsion based on the column name and column type
– Naresh
Nov 24 '18 at 15:18
add a comment |
1 Answer
1
active
oldest
votes
0
There is a method called dtypes(). See the docs here: Dataset JavaDoc
I think this method can do your job.
answered Nov 22 '18 at 13:12
abhishek chaurasiyaabhishek chaurasiya
1225
It Works for me thanks!!
– Naresh
Nov 24 '18 at 15:12
You can accept the answer then
– abhishek chaurasiya
Nov 25 '18 at 17:03
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53430564%2fhow-will-i-get-the-column-data-type-of-dataset-in-spark%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
There is a method called dtypes(). See the docs here: Dataset JavaDoc
I think this method can do your job.
answered Nov 22 '18 at 13:12
abhishek chaurasiyaabhishek chaurasiya
1225
It Works for me thanks!!
– Naresh
Nov 24 '18 at 15:12
You can accept the answer then
– abhishek chaurasiya
Nov 25 '18 at 17:03
add a comment |
0
There is a method called dtypes(). See the docs here: Dataset JavaDoc
I think this method can do your job.
answered Nov 22 '18 at 13:12
abhishek chaurasiyaabhishek chaurasiya
1225
It Works for me thanks!!
– Naresh
Nov 24 '18 at 15:12
You can accept the answer then
– abhishek chaurasiya
Nov 25 '18 at 17:03
add a comment |
0
0
0
There is a method called dtypes(). See the docs here: Dataset JavaDoc
I think this method can do your job.
answered Nov 22 '18 at 13:12
abhishek chaurasiyaabhishek chaurasiya
1225
There is a method called dtypes(). See the docs here: Dataset JavaDoc
I think this method can do your job.
answered Nov 22 '18 at 13:12
abhishek chaurasiyaabhishek chaurasiya
1225
answered Nov 22 '18 at 13:12
abhishek chaurasiyaabhishek chaurasiya
1225
answered Nov 22 '18 at 13:12
abhishek chaurasiyaabhishek chaurasiya
1225
answered Nov 22 '18 at 13:12
abhishek chaurasiyaabhishek chaurasiya
1225
1225
It Works for me thanks!!
– Naresh
Nov 24 '18 at 15:12
You can accept the answer then
– abhishek chaurasiya
Nov 25 '18 at 17:03
add a comment |
It Works for me thanks!!
– Naresh
Nov 24 '18 at 15:12
You can accept the answer then
– abhishek chaurasiya
Nov 25 '18 at 17:03
It Works for me thanks!!
– Naresh
Nov 24 '18 at 15:12
It Works for me thanks!!
– Naresh
Nov 24 '18 at 15:12
You can accept the answer then
– abhishek chaurasiya
Nov 25 '18 at 17:03
You can accept the answer then
– abhishek chaurasiya
Nov 25 '18 at 17:03
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53430564%2fhow-will-i-get-the-column-data-type-of-dataset-in-spark%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Elaborate in more details. In which API? Java? The best way is to share your code because the code is our common language.
– Soheil Pourbafrani
Nov 22 '18 at 12:34
Hi Soheil , Here we want to check the schema of the existing table with the new table. if there is any change in the schema we need to drop of the existing table and create a new table and this comparsion based on the column name and column type
– Naresh
Nov 24 '18 at 15:18