Datax MySQL2Hive抽数ClassCastException: java.lang.String cannot be cast to java.lang.Integer问题解决

1、现象

com.alibaba.datax.common.exception.DataXException: Code:[HdfsWriter-04], Description:[您配置的文件在写入时出现IO异常.]. - java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer

at org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaIntObjectInspector.get(JavaIntObjectInspector.java:40)

at .orc.WriterImpl$IntegerTreeWriter.write(WriterImpl.java:922)

at .orc.WriterImpl$StructTreeWriter.write(WriterImpl.java:1599)

at .orc.WriterImpl.addRow(WriterImpl.java:2259)

at .orc.OrcOutputFormat$OrcRecordWriter.write(OrcOutputFormat.java:76)

at .orc.OrcOutputFormat$OrcRecordWriter.write(OrcOutputFormat.java:55)

at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.orcFileStartWrite(HdfsHelper.java:392)

at com.alibaba.datax.plugin.writer.hdfswriter.HdfsWriter$Task.startWrite(HdfsWriter.java:364)

at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)

at java.lang.Thread.run(Thread.java:748)

• java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer

2、原因

mysql int类型存储的NULL值抽取到hive的orc表对应的int类型时会报类型转换的错误。

原因：该建MySQL表的默认值修改过，之前是NULL，后来改为默认值为0,在对应hive的int类型时，NULL被识别为string无法被转为integer类型.

3、解决

1、改为textfile格式的hive目标表表，此问题可以解决。（在配合gzip压缩，也是可以的）

2、清洗源头库将格式统一（一般比较困难动不了）

如果你由更好的方案，欢迎留言！