ClickHouse

I've been taking @ClickHouseDB managed Postgres and the new WAL Shadow integration for a test drive. This stack gives us the ability to create fully managed databases and then replicate data from Postgres to ClickHouse with sub second performance (~200 ms). This is more than just another boring ETL mechanism. With this level of latency, we can build applications where half of a screen or half of a user journey is served from a transactional database, and half is served by an analytical database. One blended application or workflow pulling consistent data from the most appropriate data store. It also opens up new use cases where we can look at low level event data, and combine it with real time analytics across a whole population. For instance, we can take every trade, and compare it against the 100k previous trades in real time. This could be really interesting for fraud, personalisation or the capital markets. In the example below, we have an trading scenario. Orders are placed, updated and cancelled for a book of exchange traded derivatives. The current state order book is shown and served out of Postgres. The data is immediately replicated into ClickHouse where analytics are then surfaced on the screen. We are pushing around 2.5k orders per second into a pair of small instances which would cost less than $500 per month. Great work @saisrirampur and team!