Trino
Trino is an open-source distributed SQL query engine used for interactive analytical queries against big data sources. It supports federated queries and connectors for a variety of data sources.
Timeline (last 90 days)
In engineering practice, Logic42 deployed an asynchronous compaction pipeline with the goal of keeping Trino's file planning time within 50 ms.
When a table contains 4 million 120 KB small files, the manifest files themselves can number in the thousands and total several GB, and merely reading this metadata tree can exhaust the Trino coordinator's heap space.
The performance data provided by Logic42 shows that when the number of files per partition increases from 4 (512 MB/file) to 18,500 (120 KB/file), Trino planning latency rises from 45 ms to 38,000 ms, and end-to-end query time goes from 480 ms to timeout (OOM).
In a severely fragmented scenario (18,500 120 KB files per partition), a single dashboard refresh triggers about 140,000 S3 GET calls, AWS throttles with HTTP 503 SlowDown, and the redundant API cost is about $4,200 per month.
To balance sub-second query performance with real-time visibility, streaming write buffering should be decoupled from the queryable analytical partitions, and Logic42 uses independent Spark or PyIceberg workers to asynchronously merge files smaller than 64 MB into 512 MB columnar blocks.
Trino injects policies through its built-in SystemAccessControl SPI.
The lakehouse architecture combines a data lake in open formats (such as Parquet and Iceberg) with the transactional and query capabilities of a database, allowing the same data to be accessed by both the Postgres engine and analytical engines such as Spark and Trino.
Oracle, Snowflake, Trino/Presto, and Apache Flink have provided MATCH_RECOGNIZE support, while PostgreSQL, MySQL, and others have not yet natively supported the syntax
When enterprises simultaneously use multiple compute engines such as Spark, Trino, and Flink and access data across different metadata catalogs, governance consistency becomes a core pain point
All Facts (9)
In engineering practice, Logic42 deployed an asynchronous compaction pipeline with the goal of keeping Trino's file planning time within 50 ms.
50%UnverifiedWhen a table contains 4 million 120 KB small files, the manifest files themselves can number in the thousands and total several GB, and merely reading this metadata tree can exhaust the Trino coordinator's heap space.
50%UnverifiedThe performance data provided by Logic42 shows that when the number of files per partition increases from 4 (512 MB/file) to 18,500 (120 KB/file), Trino planning latency rises from 45 ms to 38,000 ms, and end-to-end query time goes from 480 ms to timeout (OOM).
50%UnverifiedIn a severely fragmented scenario (18,500 120 KB files per partition), a single dashboard refresh triggers about 140,000 S3 GET calls, AWS throttles with HTTP 503 SlowDown, and the redundant API cost is about $4,200 per month.
50%UnverifiedTo balance sub-second query performance with real-time visibility, streaming write buffering should be decoupled from the queryable analytical partitions, and Logic42 uses independent Spark or PyIceberg workers to asynchronously merge files smaller than 64 MB into 512 MB columnar blocks.
50%UnverifiedTrino injects policies through its built-in SystemAccessControl SPI.
50%UnverifiedThe lakehouse architecture combines a data lake in open formats (such as Parquet and Iceberg) with the transactional and query capabilities of a database, allowing the same data to be accessed by both the Postgres engine and analytical engines such as Spark and Trino.
50%UnverifiedOracle, Snowflake, Trino/Presto, and Apache Flink have provided MATCH_RECOGNIZE support, while PostgreSQL, MySQL, and others have not yet natively supported the syntax
50%UnverifiedWhen enterprises simultaneously use multiple compute engines such as Spark, Trino, and Flink and access data across different metadata catalogs, governance consistency becomes a core pain point
50%