Parquet String to timestamp conversion in hiveHive UDTF returning ArrayList ColumnSchema Evolution in Parquet Hive tableUsing hive table over parquet in PigGetting ClassCastException in Hive ORC tableLoading parquet files in hive table returns all NULLTimestamp datatype not supporting in hive, when reading a parquet fileParquet Files Generation with hiveInsert timestamp type field in parquet file using AvroParquetReader to get it in hive external tableAthena/Hive timestamp in parquet files written by sparkHive: Error while fetching data
Randomness of Python's random
How did Kirk identify Gorgan in "And the Children Shall Lead"?
Why do only some White Walkers shatter into ice chips?
Position of past participle and extent of the Verbklammer
What is the difference between 'unconcealed' and 'revealed'?
How do LIGO and VIRGO know that a gravitational wave has its origin in a neutron star or a black hole?
How do I tell my manager that his code review comment is wrong?
Missing Piece of Pie - Can you find it?
Why is this entry signal showing green?
How can I get a job without pushing my family's income into a higher tax bracket?
Can you complete the sequence?
Point of the the Dothraki's attack in GoT S8E3?
Have I damaged my car by attempting to reverse with hand/park brake up?
Is it possible to know which is the correct temperature range and speed for any model?
Out of scope work duties and resignation
Is this homebrew life-stealing melee cantrip unbalanced?
Does a card have a keyword if it has the same effect as said keyword?
I'm in your subnets, golfing your code
How to convert a MULTIPOLYGON field stored as text to geometry data type in postGIS so that it can be plotted in QGIS
Are there any Final Fantasy Spirits in Super Smash Bros Ultimate?
Hyperlink on red background
On which topic did Indiana Jones write his doctoral thesis?
What is Edge to Edge Routing?
How to model the curly cable part of the phone
Parquet String to timestamp conversion in hive
Hive UDTF returning ArrayList ColumnSchema Evolution in Parquet Hive tableUsing hive table over parquet in PigGetting ClassCastException in Hive ORC tableLoading parquet files in hive table returns all NULLTimestamp datatype not supporting in hive, when reading a parquet fileParquet Files Generation with hiveInsert timestamp type field in parquet file using AvroParquetReader to get it in hive external tableAthena/Hive timestamp in parquet files written by sparkHive: Error while fetching data
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I have parquet files generated by some code. I created a DDL for that data, added table in hive and pointed to those parquet files in hdfs. When I try to query the table, all fields look perfect. But, for timestamp fields, hive is giving an exception.
HiveException: java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to org.apache.hadoop.hive.serde2.io.TimestampWritable
Actually, timestamp fields are stored as strings in parquet, but my hive table fields are of type timestamp. I think this causes the issue, but what is right way to do this?
hadoop hive parquet
asked Mar 22 at 22:10
Dinesh NamburiDinesh Namburi
9611
add a comment |
I have parquet files generated by some code. I created a DDL for that data, added table in hive and pointed to those parquet files in hdfs. When I try to query the table, all fields look perfect. But, for timestamp fields, hive is giving an exception.
HiveException: java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to org.apache.hadoop.hive.serde2.io.TimestampWritable
Actually, timestamp fields are stored as strings in parquet, but my hive table fields are of type timestamp. I think this causes the issue, but what is right way to do this?
hadoop hive parquet
asked Mar 22 at 22:10
Dinesh NamburiDinesh Namburi
9611
add a comment |
I have parquet files generated by some code. I created a DDL for that data, added table in hive and pointed to those parquet files in hdfs. When I try to query the table, all fields look perfect. But, for timestamp fields, hive is giving an exception.
HiveException: java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to org.apache.hadoop.hive.serde2.io.TimestampWritable
Actually, timestamp fields are stored as strings in parquet, but my hive table fields are of type timestamp. I think this causes the issue, but what is right way to do this?
hadoop hive parquet
asked Mar 22 at 22:10
Dinesh NamburiDinesh Namburi
9611
I have parquet files generated by some code. I created a DDL for that data, added table in hive and pointed to those parquet files in hdfs. When I try to query the table, all fields look perfect. But, for timestamp fields, hive is giving an exception.
HiveException: java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to org.apache.hadoop.hive.serde2.io.TimestampWritable
Actually, timestamp fields are stored as strings in parquet, but my hive table fields are of type timestamp. I think this causes the issue, but what is right way to do this?
hadoop hive parquet
hadoop hive parquet
asked Mar 22 at 22:10
Dinesh NamburiDinesh Namburi
9611
asked Mar 22 at 22:10
Dinesh NamburiDinesh Namburi
9611
edited Mar 22 at 22:17
Dinesh Namburi
asked Mar 22 at 22:10
Dinesh NamburiDinesh Namburi
9611
asked Mar 22 at 22:10
Dinesh NamburiDinesh Namburi
9611
asked Mar 22 at 22:10
Dinesh NamburiDinesh Namburi
9611
9611
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
This is an open bug in Hive.
And documented at https://issues.apache.org/jira/browse/HIVE-15079
answered 4 hours ago
Dinesh NamburiDinesh Namburi
9611
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55308423%2fparquet-string-to-timestamp-conversion-in-hive%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
This is an open bug in Hive.
And documented at https://issues.apache.org/jira/browse/HIVE-15079
answered 4 hours ago
Dinesh NamburiDinesh Namburi
9611
add a comment |
This is an open bug in Hive.
And documented at https://issues.apache.org/jira/browse/HIVE-15079
answered 4 hours ago
Dinesh NamburiDinesh Namburi
9611
add a comment |
This is an open bug in Hive.
And documented at https://issues.apache.org/jira/browse/HIVE-15079
answered 4 hours ago
Dinesh NamburiDinesh Namburi
9611
This is an open bug in Hive.
And documented at https://issues.apache.org/jira/browse/HIVE-15079
answered 4 hours ago
Dinesh NamburiDinesh Namburi
9611
answered 4 hours ago
Dinesh NamburiDinesh Namburi
9611
answered 4 hours ago
Dinesh NamburiDinesh Namburi
9611
answered 4 hours ago
Dinesh NamburiDinesh Namburi
9611
9611
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55308423%2fparquet-string-to-timestamp-conversion-in-hive%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
