Real Time For Your Analytics Events
Joined August 2023
- Tweets73
- Following22
- Followers41
- Likes698
Aggregations.io retweeted
Quadrillion scale is fun.
Disconnecting entire regions for science is fun.
Combining the two is a certified party.
@ClickHouseDB + @Cloudflare is amazing.
clickhouse.com/blog/cloudfla…
Aggregations.io retweeted
I ❤️ ClickHouse.
There are 2 columns:
id
updated_at
I need to do a cursor/pagination esque thing, was just doing updated_at before but there can be giant spikes of same updated_at. So it was infinite looping when trying to offset.
First thought:
select max(updated_at), argMax(id, updated_at)
Saw inconsistent behavior.
argMax isn't deterministic if there are updated_at ties.
BUT
ArgMax is a valid combinator
select max(updated_at), maxArgMax(id, updated_at)
Boom. It works.
Aggregations.io retweeted
ClickHouse is pretty amazing tech
Alexey said it best:
“We optimize ClickHouse… and then optimize it again… and again.”
clickhou.se/4pmS0IZ
25.11 keeps that tradition alive
Parallel GROUP BY merges. Projections as true secondary indexes. Faster DISTINCT.
More speed everywhere.
Aggregations.io retweeted
Why you should join @aggregationsio
We have 1 engineer per 86400 daily seconds.
Aggregations.io retweeted
I love these scaled stories
Netflix - 5PB per day, 10m events per second.
ClickHouse 🐐
clickhouse.com/blog/netflix-…
Aggregations.io retweeted
It's amazing what 2 little words can do to your query performance.
In this case, it was mySQL's STRAIGHT_JOIN
Silly analyzer, you think you know better than ME?
Aggregations.io retweeted
Shipping a big new feature for Aggregations.
I built it in <1 week.
Relied on:
- foundations I’ve built by not taking shortcuts.
- experience building similar thing at prev job.
- not caring about the UI (too much).
It’s all coming together.
Thank you past Jeff.
Aggregations.io retweeted
Ever try to use a complex payload to extract real time metrics?
It’s usually difficult, if your pipeline supports it at all.
Well we support it now, and it’s very powerful.
Fantastic resource for folks interested in Data Engineering & Analytics 👇
I created a Github repo with over 11,000 stars with all the resources, books, companies, and social media accounts you should be following to stay current on data engineering topics.
I'm accepting PRs so we can crowdsource this effort and have a comprehensive list of resources!
github.com/DataExpert-io/dat…
RT @jsneedles: What if I told you you could:
- Get rid of JOINs
- Increase query speed
- Kill annoying ETLs
- Not sacrifice the final que…
Aggregations.io retweeted
Improving the efficiency of your #data org is hard.
You've got constant asks for new stuff and you still need to keep everything working.
Who has time for niceties like documentation and schema management?
Well - that's where @aggregationsio comes in.
Aggregations.io retweeted
You know what's awesome?
Crafting the perfect complex event schema for your #Analytics.
Modern data tooling handles these structures without issue.
But observability? NOPE.
Well, @aggregationsio does.
aggregations.io/blog/aggrega…