@apachehudi

Official twitter handle of Apache Hudi, an open data lakehouse platform. https://xitter.cf/t.co/SXay7oHNah

hudi.apache.org/
Joined January 2019
  • Tweets1.1K
  • Following124
  • Followers4K
304Media
Hudi 1.0 is the most powerful release to date for data lakehouses. Read the blog for details: Secondary Indexing, Expression Indexes, Partial Updates, Non-blocking Concurrency Control, New LSM timeline, +more: hudi.apache.org/blog/2024/12… #datalakehouse #opentableformat

Announcing Apache Hudi 1.0 and the Next Generation of Data Lakehouses | Apache Hudi

game-changing major release, that reimagines Hudi and Data Lakehouses.

hudi.apache.org
10
36
2,966
A Hudi upsert is just a Spark DAG. Tune it like one. Two phases ↓
1
6
277
more replies
• hoodie.memory.merge.fraction — bump if upserts spill to temp folder • Timeseries / log data — tune bloom accuracy OR use bucket index; time-prefix keys for range pruning Links ↓ #ApacheHudi #SparkTuning
1
41
Tuning guide docs: hudi.apache.org/docs/tuning-… Indexes overview: hudi.apache.org/docs/indexes

Spark Tuning Guide | Apache Hudi

To get a better understanding of where your Hudi jobs is spending its time, use a tool like YourKit Java Profiler, to obtain heap dumps/flame graphs.

hudi.apache.org
20
Zendesk ran CDC from 1,800+ Aurora MySQL databases across 8 AWS regions into one Apache Hudi data lake. 10+ billion ticket events. Petabytes of data ↓
1
1
220
more replies
Cross-region, multi-database CDC at meaningful scale. Links ↓ #ApacheHudi #CDC
1
1
34
AWS Architecture Blog (co-authored with Johnathan Hwang, Zendesk): aws.amazon.com/blogs/archite…

Insights for CTOs: Part 3 – Growing your business with modern data capabilities | Amazon Web...

This post was co-wrtiten with Jonathan Hwang, head of Foundation Data Analytics at Zendesk. In my role as a Senior Solutions Architect, I have spoken to chief technology officers (CTOs) and executive...

aws.amazon.com
40
Unsafe (workarounds required): non-nullable column (make it nullable), datatype demotion (rewrite), field reorder (breaks partial-coverage upsert reads).
1
26
more replies
For column renames + deletes + complex evolution: opt into schema-on-read with hoodie.schema.on.read.enable=true. Spark 3.2.1+, can't be disabled once on. Proven at scale at Uber, Walmart, LinkedIn. Links ↓ #ApacheHudi #SchemaEvolution
1
36
Schema evolution docs: hudi.apache.org/docs/schema_… Mastering Schema Evolution blog: hudi.apache.org/blog/2025/12…

Schema Evolution | Apache Hudi

Schema evolution is an essential aspect of data management, and Hudi supports schema evolution on write out-of-the-box,

hudi.apache.org
21
Upstream added a column. Did your pipeline survive? Hudi supports schema evolution on write out of the box — the changes that happen in production all the time.
1
1
168
Safe (writes succeed, reads return the full dataset): add nullable column at root or nested level, add complex type with default, promote int→long / float→double / string↔bytes, add Hudi meta columns.
1
46
Upstox — one of India's largest online trading platforms — ingested 300 GB of new data per day for 12 million customers on Apache Hudi. Capital markets workloads stress every part of a lakehouse ↓
1
2
140
more replies
Streaming-first lakehouse, built for upserts on S3 — what trading-day workloads require. 300k data requests served daily. Links ↓ #ApacheHudi #FinTech
1
19
Upstox engineering blog hudi.apache.org/blog/2025/10… Hudi concurrency control: hudi.apache.org/docs/next/co…

Modernizing Upstox's Data Platform with Apache Hudi, dbt, and EMR Serverless | Apache Hudi

How Upstox modernized its data platform with Apache Hudi, dbt and EMR Serverless.

hudi.apache.org
23
Load more