Nov 5, 2024

A Chat With Tomer Shiran, Founder at Dremio Dana LeighNovember 5, 2024 Dremio was founded with a vision to enable companies to harness their data. Today, it is the open and easy data lakehouse, providing self-service SQL analytics, data warehouse performance and functionality, and data lake flexibility across all of a company’s data. Dremio features an open architecture that separates data and compute. Data is stored on object storage in the Apache Iceberg format, eliminating vendor lock-in concerns. This allows companies to run any data warehousing workload directly on the data lake, without the skyrocketing costs of cloud data warehouses. Users can also query across multiple data sources and clouds, thereby enabling analytics on any data in the organisation without complex and expensive data transfers (ETL). Dremio provides a variety of unique capabilities that make data engineers more productive and analysts more self-sufficient. First, Dremio’s query acceleration ensures response times in less than a second, eliminating the need to manage thousands of business intelligence (BI) extracts/imports. Second, Dremio allows querying across dozens of data sources, including data lakes, data warehouses, relational databases and NoSQL databases. Third, Dremio provides a revolutionary approach to data management we call git-for-data. Inspired by GitHub, it enables users to utilise zero-copy branches to load, transform and test data more efficiently than ever. Dremio achieves all this through a fully open architecture. Data is stored and transferred in open-source formats, thereby guaranteeing interoperability across the vast ecosystem of applications, tools, and engines. Notably, Dremio is the creator of Apache Arrow, a widely adopted open source in-memory representation that is downloaded over 70 million times per month. Additionally, Dremio is one of the main contributors to the Apache Iceberg project, which has become the standard table format for data lakes. Dremio has been innovating rapidly over the last couple years. For example, Dremio was the first vendor to back Apache Iceberg, thereby significantly expanding the use of data lakes and enabling companies to transition from closed data warehouse architectures to open data lake/lakehouse architectures. In addition, Dremio was the first data lakehouse platform to introduce GenAI capabilities, such as querying data using natural language and automatically documenting and labeling data. As a company, Dremio has continued to grow over the last couple years. Hundreds of the world’s leading companies now use Dremio to power their data analytics in the cloud. With companies seeking to capitalise on AI opportunities and lower their costs, we are seeing many migrations from closed, all-in-one products to open and flexible architectures with Dremio as a foundational component in the new stack. Dremio will continue to innovate and release game-changing capabilities in the market, with the objective of making data engineering effortless, and data analytics as easy as using ChatGPT. Related Articles