Lead developer of Spark SQL @databricks, formerly @ucberkeley. Distributed databases, query languages, scala, other nerdy stuff...
Joined January 2012
- Tweets272
- Following0
- Followers6.1K
- Likes353
Michael Armbrust retweeted
Insightful benchmark of Linux Foundation Delta Lake and Apache Iceberg by @BrooklynData that shows Delta is up to 8x faster in workloads with data updates. Most storage benchmarks only test reads, but with updates, care is needed to maintain performance. brooklyndata.co/blog/benchma…
Michael Armbrust retweeted
.@DeltaLakeOSS just got even better. Meet #DeltaLake 2.0, now *entirely* open source. @michaelarmbrust shares how the latest features improve performance & manageability. #DataAISummit
Michael Armbrust retweeted
Databricks just set a new record on the official TPC-DS data warehousing benchmark, showing that a lakehouse system based on open data formats can outperform previous DW systems. Don't listen to folks who say open means bad performance! databricks.com/blog/2021/11/…
As the #lakehouse gains momentum, we've been getting a lot of questions about what it means and how it compares to other architectures. Here are some answers to these and other common questions! databricks.com/blog/2021/08/…
Michael Armbrust retweeted
Why do you have to pick between diversity and merit? Make your work env. inclusive, decrease bias in hiring, as a leader don't make statements that alienate large groups. That'll give you a competitive advantage to a talent pool that won't join unwelcoming companies.
Unlock your data and securely query it anywhere with #deltasharing on @DeltaLakeOSS!
Happy to announce #deltasharing - an open standard for secure data sharing delta.io/sharing/
#DeltaLake cc @databricks
Really excited to announce Delta Live Tables, a system that turns your SQL queries and pyspark DataFrame code into production-ready ETL pipelines! We've been working on this for almost 3 years, and it powers some of our largest pipelines internally. youtube.com/watch?v=fJhlTsh3…
If you want to jump into the details check out our docs: docs.databricks.com/data-eng…
And if you want to try it out, sign up for the preview! databricks.com/product/delta…
Michael Armbrust retweeted
Did you know you can now use Delta Lake straight from #Python?
Just a quick: pip install deltalake pandas
Michael Armbrust retweeted
Excited that our #Lakehouse paper got published at #CIDR21: it shares our vision of the Lakehouse: a new type of data platforms that are completely open, have full support for #machinelearning, while supporting all traditional #datawarehouse workloads.
cidrdb.org/cidr2021/papers/c…
Michael Armbrust retweeted
What Is a Data Lakehouse? 🆕 post on a data management paradigm for the age of #MachineLearning and #AI (written with some of the founders @databricks ↦ @rxin @matei_zaharia @michaelarmbrust and @alighodsi) databricks.com/blog/2020/01/…
Michael Armbrust retweeted
I gave a keynote at @ACMSoCC about lessons from building a large-scale cloud service at @Databricks. Did you know that Databricks runs millions of VMs/day to process exabytes of data with <200 engineers? Slides here: slideshare.net/matei/lessons…
Michael Armbrust retweeted
Delta Lake is now part of the Linux Foundation! EBs of data/month, in production 1000s of organizations. Can't wait to see how the community will shape its future and establish it as a standard for data lakes.
databricks.com/blog/2019/10/…
Had a great time chatting with @TobiasMacey on the @DataEngPodcast about @DeltaLakeOSS - dataengineeringpodcast.com/d…
Exciting news from the #Presto community!
[News] @starburstdata brings #DeltaLakeOSS compatibility to #Presto. Read more here: dbricks.co/2Kh1K7N
cc: @databricks
Excited to get into more details of how to use delta and also how delta works.
Need data lake reliability? Learn from principal engineer @MichaelArmbrust how Delta Lake makes your data reliable. Webinar on May 23. dbricks.co/30h0JRX #UnifiedAnalytics #Analytics
H/T to @AbeGong for inspiring the expectations feature of #DeltaLake with his project github.com/great-expectation… #SparkAISummit
Michael Armbrust retweeted
Welcome to Delta Lake, the open-source storage layer that brings ACID transactions to Apache Spark™ and big data workloads. Join us in our pursuit to address data reliability with data lakes: Delta.io
