Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Iceberg Connector ORC writer writes incorrect file size #9810

Closed
Tracked by #1324
kekwan opened this issue Oct 28, 2021 · 1 comment · Fixed by #10042
Closed
Tracked by #1324

Iceberg Connector ORC writer writes incorrect file size #9810

kekwan opened this issue Oct 28, 2021 · 1 comment · Fixed by #10042
Labels
bug Something isn't working

Comments

@kekwan
Copy link
Contributor

kekwan commented Oct 28, 2021 •
edited
Loading

ORC files in Iceberg connector are being written with incorrect file_size_in_bytes in the metadata.

I am noticing this behaviour when the table has varchar data types. It seems to add 5 extra bytes per varchar column.

Using io.trino.plugin.iceberg.IcebergQueryRunner#main:

1 varchar column adds 5 extra bytes.

trino:tpch> create table test_size_varchar (c1 varchar);
CREATE TABLE

trino:tpch> insert into test_size_varchar values ('abc');
INSERT: 1 row

trino:tpch> select file_size_in_bytes, file_path, file_format, record_count from "test_size_varchar$files";
 file_size_in_bytes |                                                                            file_path                                                                             | file_format | record_count
--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+--------------
                326 | /var/folders/tt/j8ffvrld6335mlspxh020p700000gp/T/TrinoTest17702232825155028672/iceberg_data/tpch/test_size_varchar/data/f4940823-f793-4f4b-8832-801978eb7a4c.orc | ORC         |            1

➜  ~ stat -f%z /var/folders/tt/j8ffvrld6335mlspxh020p700000gp/T/TrinoTest17702232825155028672/iceberg_data/tpch/test_size_varchar/data/f4940823-f793-4f4b-8832-801978eb7a4c.orc
321

2 varchar columns adds 10 extra bytes.

trino:tpch> create table test_size_2_varchar (c1 varchar, c2 varchar);
CREATE TABLE

trino:tpch> insert into test_size_2_varchar values ('123', 'abc');
INSERT: 1 row

trino:tpch> select file_size_in_bytes, file_path, file_format, record_count from "test_size_2_varchar$files";
 file_size_in_bytes |                                                                             file_path                                                                              | file_format | record_count
--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+--------------
                406 | /var/folders/tt/j8ffvrld6335mlspxh020p700000gp/T/TrinoTest17702232825155028672/iceberg_data/tpch/test_size_2_varchar/data/776d2c0a-eff8-4a71-af43-e91b83429335.orc | ORC         |            1

➜  ~ stat -f%z /var/folders/tt/j8ffvrld6335mlspxh020p700000gp/T/TrinoTest17702232825155028672/iceberg_data/tpch/test_size_2_varchar/data/776d2c0a-eff8-4a71-af43-e91b83429335.orc
396

Other data types seem fine. Haven't tested them all though.

trino:tpch> create table test_size_other_types (c1 int, c2 timestamp(6), c3 boolean, c4 decimal);
CREATE TABLE

trino:tpch> insert into test_size_other_types values (1, current_timestamp(6), false, 1.1);
INSERT: 1 row

trino:tpch> select file_size_in_bytes, file_path, file_format, record_count from "test_size_other_types$files";
 file_size_in_bytes |                                                                              file_path                                                                               | file_format | record_count
--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+--------------
                544 | /var/folders/tt/j8ffvrld6335mlspxh020p700000gp/T/TrinoTest17702232825155028672/iceberg_data/tpch/test_size_other_types/data/f34878db-00e2-4d81-af6f-0ec467ad8377.orc | ORC         |            1

➜  ~ stat -f%z /var/folders/tt/j8ffvrld6335mlspxh020p700000gp/T/TrinoTest17702232825155028672/iceberg_data/tpch/test_size_other_types/data/f34878db-00e2-4d81-af6f-0ec467ad8377.orc
544

Slack thread with more discussion: https://trinodb.slack.com/archives/CGB0QHWSW/p1631656773196400
@kekwan
Copy link
Contributor Author

kekwan commented Oct 28, 2021 •
edited
Loading

This may be leading to the following OrcCorruptionException when I try to read from 1 of my particular larger Iceberg tables. However, this behaviour is not deterministic within this same table.

trino:huron_iceberg_daily_data> select count(*) from huron_daily_ailtn;

Query 20211028_190759_37378_gea99 failed: Error opening Iceberg split s3://presto-322055953712-us-west-2-bdmpresto-dev-001/huron_iceberg_daily_data/huron_daily_ailtn/data/_event_timestamp_day=2021-10-09/compact_ts_=2021-10-12T04%3A30/compact_try_number_=1/c32532c2-091b-446c-8a8f-dad58fa90b7f.orc (offset=939524096, length=134217728): Malformed ORC file. Invalid file metadata [s3://presto-322055953712-us-west-2-bdmpresto-dev-001/huron_iceberg_daily_data/huron_daily_ailtn/data/_event_timestamp_day=2021-10-09/compact_ts_=2021-10-12T04%3A30/compact_try_number_=1/c32532c2-091b-446c-8a8f-dad58fa90b7f.orc]
io.trino.spi.TrinoException: Error opening Iceberg split s3://presto-322055953712-us-west-2-bdmpresto-dev-001/huron_iceberg_daily_data/huron_daily_ailtn/data/_event_timestamp_day=2021-10-09/compact_ts_=2021-10-12T04%3A30/compact_try_number_=1/c32532c2-091b-446c-8a8f-dad58fa90b7f.orc (offset=939524096, length=134217728): Malformed ORC file. Invalid file metadata [s3://presto-322055953712-us-west-2-bdmpresto-dev-001/huron_iceberg_daily_data/huron_daily_ailtn/data/_event_timestamp_day=2021-10-09/compact_ts_=2021-10-12T04%3A30/compact_try_number_=1/c32532c2-091b-446c-8a8f-dad58fa90b7f.orc]
	at io.trino.plugin.iceberg.IcebergPageSourceProvider.createOrcPageSource(IcebergPageSourceProvider.java:372)
	at io.trino.plugin.iceberg.IcebergPageSourceProvider.createDataPageSource(IcebergPageSourceProvider.java:213)
	at io.trino.plugin.iceberg.IcebergPageSourceProvider.createPageSource(IcebergPageSourceProvider.java:171)
	at io.trino.spi.connector.ConnectorPageSourceProvider.createPageSource(ConnectorPageSourceProvider.java:68)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorPageSourceProvider.createPageSource(ClassLoaderSafeConnectorPageSourceProvider.java:66)
	at io.trino.split.PageSourceManager.createPageSource(PageSourceManager.java:64)
	at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:298)
	at io.trino.operator.Driver.processInternal(Driver.java:387)
	at io.trino.operator.Driver.lambda$processFor$9(Driver.java:291)
	at io.trino.operator.Driver.tryWithLock(Driver.java:683)
	at io.trino.operator.Driver.processFor(Driver.java:284)
	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1075)
	at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
	at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
	at io.trino.$gen.Trino_325_5727_g8f55fe1____20211028_091527_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.trino.orc.OrcCorruptionException: Malformed ORC file. Invalid file metadata [s3://presto-322055953712-us-west-2-bdmpresto-dev-001/huron_iceberg_daily_data/huron_daily_ailtn/data/_event_timestamp_day=2021-10-09/compact_ts_=2021-10-12T04%3A30/compact_try_number_=1/c32532c2-091b-446c-8a8f-dad58fa90b7f.orc]
	at io.trino.orc.metadata.ExceptionWrappingMetadataReader.propagate(ExceptionWrappingMetadataReader.java:119)
	at io.trino.orc.metadata.ExceptionWrappingMetadataReader.readMetadata(ExceptionWrappingMetadataReader.java:64)
	at io.trino.orc.OrcReader.<init>(OrcReader.java:198)
	at io.trino.orc.OrcReader.createOrcReader(OrcReader.java:117)
	at io.trino.orc.OrcReader.createOrcReader(OrcReader.java:94)
	at io.trino.plugin.iceberg.IcebergPageSourceProvider.createOrcPageSource(IcebergPageSourceProvider.java:286)
	... 17 more
Caused by: io.trino.orc.OrcCorruptionException: Malformed ORC file. Requested read size (1346595 bytes) is greater than max buffer size (16292 bytes [s3://presto-322055953712-us-west-2-bdmpresto-dev-001/huron_iceberg_daily_data/huron_daily_ailtn/data/_event_timestamp_day=2021-10-09/compact_ts_=2021-10-12T04%3A30/compact_try_number_=1/c32532c2-091b-446c-8a8f-dad58fa90b7f.orc]
	at io.trino.orc.stream.CompressedOrcChunkLoader.ensureCompressedBytesAvailable(CompressedOrcChunkLoader.java:156)
	at io.trino.orc.stream.CompressedOrcChunkLoader.nextChunk(CompressedOrcChunkLoader.java:124)
	at io.trino.orc.stream.OrcInputStream.advance(OrcInputStream.java:204)
	at io.trino.orc.stream.OrcInputStream.read(OrcInputStream.java:96)
	at io.trino.orc.protobuf.CodedInputStream$StreamDecoder.tryRefillBuffer(CodedInputStream.java:2715)
	at io.trino.orc.protobuf.CodedInputStream$StreamDecoder.isAtEnd(CodedInputStream.java:2654)
	at io.trino.orc.protobuf.CodedInputStream$StreamDecoder.readTag(CodedInputStream.java:2004)
	at io.trino.orc.proto.OrcProto$Metadata.<init>(OrcProto.java:21978)
	at io.trino.orc.proto.OrcProto$Metadata.<init>(OrcProto.java:21950)
	at io.trino.orc.proto.OrcProto$Metadata$1.parsePartialFrom(OrcProto.java:22647)
	at io.trino.orc.proto.OrcProto$Metadata$1.parsePartialFrom(OrcProto.java:22642)
	at io.trino.orc.protobuf.AbstractParser.parseFrom(AbstractParser.java:91)
	at io.trino.orc.protobuf.AbstractParser.parseFrom(AbstractParser.java:96)
	at io.trino.orc.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
	at io.trino.orc.protobuf.GeneratedMessageV3.parseWithIOException(GeneratedMessageV3.java:311)
	at io.trino.orc.proto.OrcProto$Metadata.parseFrom(OrcProto.java:22176)
	at io.trino.orc.metadata.OrcMetadataReader.readMetadata(OrcMetadataReader.java:114)
	at io.trino.orc.metadata.ExceptionWrappingMetadataReader.readMetadata(ExceptionWrappingMetadataReader.java:61)

Setting set session iceberg.use_file_size_from_metadata = false; makes the SELECT queries work.

trino:huron_iceberg_daily_data> set session iceberg.use_file_size_from_metadata = false;
SET SESSION

trino:huron_iceberg_daily_data> select count(*) from huron_daily_ailtn;
   _col0
------------
 1988039657

@findepi findepi added the bug Something isn't working label Oct 29, 2021
@findepi findepi mentioned this issue Oct 29, 2021
Closed
93 tasks
@findepi findepi mentioned this issue Nov 23, 2021
Merged
@findepi findepi mentioned this issue Nov 24, 2021
Closed
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix file_size_in_bytes in Iceberg manifest for ORC findepi/trino
2 participants
@findepi @kekwan