Hive inserting parquet files with different schema











up vote
0
down vote

favorite












I am trying to insert 2 parquet files to HDFS via hive.
the first one looks like



----------------

|col1|col2|col3|

----------------

 |1   |2   |3   |

----------------


when I select in HUE, I see this row correctly.
Then I insert the second file to the same table that looks like this:



---------------

 col3|col4|col5|

---------------

 31   |4   |5   |

---------------


The insert query works here but when I do a select query from all the rows, I get this output:



-------------------------

 col1|col2|col3|col4|col5|

-------------------------

 1   |2   |3   |null|null|

------------------------

NULL|NULL|31  |NULL|NULL|

------------------------


The second row should be null, null, 31,4,5
what am I doing wrong here?






hadoop hive







share|improve this question
























  • Could you please post your code so that people here can help you out?
    – VIN
    yesterday















up vote
0
down vote

favorite












I am trying to insert 2 parquet files to HDFS via hive.
the first one looks like



----------------

|col1|col2|col3|

----------------

 |1   |2   |3   |

----------------


when I select in HUE, I see this row correctly.
Then I insert the second file to the same table that looks like this:



---------------

 col3|col4|col5|

---------------

 31   |4   |5   |

---------------


The insert query works here but when I do a select query from all the rows, I get this output:



-------------------------

 col1|col2|col3|col4|col5|

-------------------------

 1   |2   |3   |null|null|

------------------------

NULL|NULL|31  |NULL|NULL|

------------------------


The second row should be null, null, 31,4,5
what am I doing wrong here?






hadoop hive







share|improve this question
























  • Could you please post your code so that people here can help you out?
    – VIN
    yesterday













up vote
0
down vote

favorite









up vote
0
down vote

favorite











I am trying to insert 2 parquet files to HDFS via hive.
the first one looks like



----------------

|col1|col2|col3|

----------------

 |1   |2   |3   |

----------------


when I select in HUE, I see this row correctly.
Then I insert the second file to the same table that looks like this:



---------------

 col3|col4|col5|

---------------

 31   |4   |5   |

---------------


The insert query works here but when I do a select query from all the rows, I get this output:



-------------------------

 col1|col2|col3|col4|col5|

-------------------------

 1   |2   |3   |null|null|

------------------------

NULL|NULL|31  |NULL|NULL|

------------------------


The second row should be null, null, 31,4,5
what am I doing wrong here?






hadoop hive







share|improve this question















I am trying to insert 2 parquet files to HDFS via hive.
the first one looks like



----------------

|col1|col2|col3|

----------------

 |1   |2   |3   |

----------------


when I select in HUE, I see this row correctly.
Then I insert the second file to the same table that looks like this:



---------------

 col3|col4|col5|

---------------

 31   |4   |5   |

---------------


The insert query works here but when I do a select query from all the rows, I get this output:



-------------------------

 col1|col2|col3|col4|col5|

-------------------------

 1   |2   |3   |null|null|

------------------------

NULL|NULL|31  |NULL|NULL|

------------------------


The second row should be null, null, 31,4,5
what am I doing wrong here?






hadoop hive




hadoop hive






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited yesterday









mustaccio

14k83637




14k83637










asked 2 days ago









user1997656

158313




158313












  • Could you please post your code so that people here can help you out?
    – VIN
    yesterday


















  • Could you please post your code so that people here can help you out?
    – VIN
    yesterday
















Could you please post your code so that people here can help you out?
– VIN
yesterday




Could you please post your code so that people here can help you out?
– VIN
yesterday

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53372507%2fhive-inserting-parquet-files-with-different-schema%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53372507%2fhive-inserting-parquet-files-with-different-schema%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

android studio warns about leanback feature tag usage required on manifest while using Unity exported app?

Image
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 0 This is a simple project showing cubes on Unity, exported to android project and imported on Android studio. On maifest file it showed following warning: Expecting <uses-feature android:name="android.software.leanback" android:required="false" /> tag I have no information about it. And also on default emptyview android project does not use/mention this feature. I know it is required by TV's and such but why its not added by default? Why is this? And Should I concern about it? android unity3d
Read more

SQL update select statement

Image
0 I'm trying to update an entry in my database (postgresql). I'm having a problem with updating the query: INSERT INTO prices (price, product_id) SELECT product_price, commn_id FROM products_temp as prod WHERE NOT EXISTS (SELECT * FROM prices od WHERE prod.commn_id = od.product_id) OR prod.product_price != ( SELECT price FROM prices as p WHERE p.product_id = prod.commn_id order by p.end desc LIMIT 1 ) Query works fine when i delete: OR prod.product_price != ( SELECT price FROM prices as p WHERE p.product_id = prod.commn_id order by p.end desc LIMIT 1 ) So it seems to me that it's looping through this operation. My question is, how can I fix it? sql postgresql
Read more

WPF add header to Image with URL pettitions [duplicate]

Image
up vote 0 down vote favorite This question already has an answer here: Image source URI requiring authorization 3 answers Add header to all* HTTP requests on a machine 1 answer first of all sorry for my english. Recently I added some security in the backend of my project, and now I need to send a validation code(apiKey) inside the headers to retrieve the images. I have a problem with ALL images in my project that I had attached in my UI. For example, before I had: <Image Height="90" Source="{Binding Video.Thumbnail}"/> Now I need to change my code to create a CUSTOM petition add the apiKey in the header:
Read more