Shrikanth Shankar retweeted
Love this integration between @huggingface and @databricks. Concretely, you will be able to train your own LLM from using Spark and Transformer/Dataset with this tight integration:
databricks.com/blog/contribu…
Shrikanth Shankar retweeted
Very excited to return to UC Berkeley as a professor starting this week. I’ll be collaborating with the Sky Lab, @UCBEPIC, @berkeley_ai and others!
@Berkeley_EECS welcomes @matei_zaharia, who returns to Berkeley EECS as an Associate Professor. Matei’s research interests include computer systems and machine learning. He is also the co-founder and Chief Technologist of Databricks. Welcome back, Matei!
Shrikanth Shankar retweeted
The upcoming Data and AI Summit keynote lineup looks insane: Eric Schmidt, Satya Nadella, Marc Andreessen, Nat Friedman (Github CEO introduced copilot), LangChain creator, PyTorch co-creator, DuckDB creator, and a few top researchers in AI and robotics databricks.com/dataaisummit/
Shrikanth Shankar retweeted
Awesome to see Oracle adopt #DeltaSharing (from @DeltaLakeOSS)! This means you can share data between Oracle, Databricks, Apache Spark, pandas, PowerBI, Excel and any system that supports the open protocol. Open makes sense for a sharing solution.
infoworld.com/article/369521…
Shrikanth Shankar retweeted
Dolly LLM now on @huggingface, check it out! Cost under $100 to produce on a few machines for a few hours!
Shrikanth Shankar retweeted
Databricks Model Serving is now GA! Key benefits:
- Simple: pick a model and get HA serving and fast autoscaling to minimize cost.
- Integrated with your data platform: build features, serve them, monitor and debug quality all on the same platform.
databricks.com/blog/2023/03/…
Shrikanth Shankar retweeted
#DataAISummit Call for Presentations ends Jan 13! Submit your case studies, research or best practices around #Lakehouse, MLOps, streaming, and data. Last year's event included 60,000 data experts. dbricks.co/3Wh1Mx6
Shrikanth Shankar retweeted
Lots of folks are under the misguided impression that switching to Delta Lake is a big change. For many organizations, it's trivially easy. You can oftentimes just change a few lines of code to switch from #parquet to #deltalake. 🦀
Credit: Matthew Powers
#opensource #oss
Shrikanth Shankar retweeted
Insightful benchmark of Linux Foundation Delta Lake and Apache Iceberg by @BrooklynData that shows Delta is up to 8x faster in workloads with data updates. Most storage benchmarks only test reads, but with updates, care is needed to maintain performance. brooklyndata.co/blog/benchma…
Shrikanth Shankar retweeted
This one is for people interested in @getdbt and @databricks. We just added support for dbt-databricks in dbt Cloud. Which means #UnityCatalog support!! More details : getdbt.com/blog/dbt-cloud-da…
Shrikanth Shankar retweeted
New Python streaming operators for #ApacheSpark are coming as part of Project Lightspeed. Check them out in Spark 3.4 or Databricks Runtime 11.3. databricks.com/blog/2022/10/…
Shrikanth Shankar retweeted
1/ This week on #TheDataExchangePod: @karthikz, Head of Streaming @databricks. We discussed the next generation Spark Streaming engine & its key goals:
- Improve latency and make it predictable
- Enhance the ecosystem support for connectors
thedataexchange.media/projec…
Shrikanth Shankar retweeted
It was great talking with @TiernanRayTech about the technology we're developing at Databricks: thetechnologyletter.com/the-…
Shrikanth Shankar retweeted
In-person meetups tomorrow at Hacker Dojo, Mountain View. Project #Lightspeed is a new initiative that will take Spark Structured Streaming to the next generation.
@karthikz @databricks @DataRiders #lightspeed
#stream_processing #apache_spark xitter.cf/DataRiders/status/1571…
Shrikanth Shankar retweeted
Join me on Sept 20th for a webinar on how #lakehouse systems are becoming the best data warehouse, through improved performance, scalability, governance features and real-time support. We'll include a tour of the latest Databricks SQL features launched. sprou.tt/1pRJ6cag6i0
Shrikanth Shankar retweeted
Check out Karthik's talk on the new things we're doing in @ApacheSpark Structured Streaming. It's one of the fastest growing use cases we see on Databricks and we have a lot coming to improve performance and usability.
Excited to talk about Project #Lightspeed - Next Generation Spark Structured Streaming at @DataRiders meetup on Sept 21st - meetup.com/datariders/events…
Shrikanth Shankar retweeted
We are excited to announce the release of Delta Sharing 0.5.0, which introduces support for Change Data Feed and more! 🦀
View the release notes: github.com/delta-io/delta-sh…
#linuxfoundation #opensource #deltalake
Shrikanth Shankar retweeted
How can #DuckDB return this Query in less than 100 ms ? it is the magic of @DeltaLakeOSS file pruning based on columns statistics, thanks @databricks for making it Open Source
Shrikanth Shankar retweeted
Excited that Ryan Harris will be joining me for our first @DeltaLakeOSS Discussions (i.e., #D3L2) on #cybersecurity, #datamesh, and #deltalake at @HSBC on Tuesday, August 30th at 09:00 PT. Come with your questions - it will be a fun discussion! community.linuxfoundation.or…
Shrikanth Shankar retweeted
There are a lot of great features in the Delta Sharing GA release on Databricks, including sharing Change Data Feeds and easy connectivity to other Databricks accounts. Check out what's new and a demo: databricks.com/blog/2022/08/…