2015년 1월 9일 금요일

[cdh-user] How to use Avro timestamp in CDH 5.3?

CDH 5.3 release note says, Apache Avro has new date, time, timestamp, and duration binary types.
What I need is a timestamp in the impala table created from a parquet file, how to achieve this with the new Avro feature introduced by AVRO-739
I write a schema with
{"name": "ServerTime", "type": "long", "logicalType": "timestamp-millis", "default": 0}
but when I use mvn generate-sourcesServerTime is still a java long, and if I write the record to parquet using AvroParquetWriter,
it’s int64, and then I load the file to impala, ServerTime is now bigint, not timestamp .
Note that I've updated avro, parquet-avro, avro-maven-plugin to the latest cdh5.3.0 version


댓글 없음:

댓글 쓰기