Using Presto For Long Running Batch Queries At Meta Tim Meehan Arjun Gupta Meta

27 Long Running Queries 169166 Pdf In this paper, we discuss several successful evolutions in recent years that have improved presto latency as well as scalability by several orders of magnitude in production at meta. Presto is used for low latency interactive use cases as well as long running etl jobs at meta. it was originally launched at meta in 2013 and donated to the linux foundation in 2019.

Meta Arajun Arjun Mehta Janudi Threads Say More Presto is an open source distributed sql query engine that supports analytics workloads involving multiple exabyte scale data sources. presto is used for low latency interactive use cases as well as long running etl jobs at meta. Before i start talking about presto, let me tell you that it has been renamed trino. presto is an open source distributed sql query engine. presto supports analytics workloads for multiple exabyte scale of data. it can be used for low latency interactive use cases and long running etl jobs. Prior to ibm, tim was a software engineer at meta where he also worked on presto, focused on resource management and reliability. he’s spent a lot of his career wrangling data, and chooses to work on presto because of its versatility, extensibility and performance. One of the earliest challenges we faced as presto’s footprint at meta grew rapidly was deploying the query engine to a high volume of clusters while ensuring consistent availability and reliability.

Comparison Between Using Meta Batch And Not Using Meta Batch Download Prior to ibm, tim was a software engineer at meta where he also worked on presto, focused on resource management and reliability. he’s spent a lot of his career wrangling data, and chooses to work on presto because of its versatility, extensibility and performance. One of the earliest challenges we faced as presto’s footprint at meta grew rapidly was deploying the query engine to a high volume of clusters while ensuring consistent availability and reliability. In this paper, we discuss several successful evolutions in recent years that have improved presto latency as well as scalability by several orders of magnitude in production at meta. Presto is central to meta’s data infrastructure, particularly for interactive use cases, where a human often waits for query results. this makes presto vital for employee productivity and everyday workflows. In this paper, we'd like to introduce some of the most important features and performance improvements the open source presto community made in recent years, which enables companies running presto at scale, supporting millions of queries per day, with hundreds of thousands of machines. In this paper, we’d like to introduce some of the most important features and performance improvements the open source presto community made in recent years, which enables companies running presto at scale, supporting millions of queries per day, with hundreds of thousands of machines.