TensorChord retweeted
With the 0.4.11 release, Hindsight now supports VectorChord (vchord) from @TensorChord, a high-performance, open-source PostgreSQL extension for similarity search
TensorChord retweeted
II-Commons
Infrastructure for shared knowledge.
Transparent. Distributed. Open source.
The foundation for trustworthy AI.
TensorChord retweeted
Building Earth Index as a searchable planet has always been ambitious & we've made sure to use the right tools to make this a reality. We're happy to use @TensorChord's PostgreSQL extension to enable environmental action at this scale.
Read more here➡️blog.vectorchord.ai/3-billio…
TensorChord retweeted
Explore the counterintuitive insights behind the performant RaBitQ algorithm for binary and scalar quantization in vector databases!
dev.to/gaoj0017/quantization…
#RAG #vectorsearch #vectordatabase #rabitq
TensorChord retweeted
> First, we’re introducing explicit functions that treat external object storage locations as first-class data citizens. Second, we’re integrating the ability to run LLMs directly within the Postgres platform.
Interview with my senior coworker, Torsten.
odbms.org/2024/10/on-edb-pos…
TensorChord retweeted
We're excited to announce the release of pg_bestmatch.rs, a PostgreSQL extension that brings the power of Best Matching 25 Score (BM25) text queries to your database, enhancing your ability to perform efficient and accurate text retrieval. blog.pgvecto.rs/pgbestmatchr…
TensorChord retweeted
Replying to @mattshumer_
I ended up writing it out myself using @modal_labs for getting embeddings, and @qdrant_engine for vectors, then realised I need lots of filtering and switched to pgvecto.rs
TensorChord retweeted
There’s thousands of RAG techniques and tutorials, but which ones perform the best?
ARAGOG by Matous Eibich is one of the most comprehensive evaluation surveys on advanced RAG techniques, testing everything from “classic vector database” to reranking (@cohere, LLM) to MMR to @llama_index native advanced techniques (sentence window retrieval, document summary index).
The findings 💡:
✅ HyDE and LLM reranking enhance retrieval precision
⚠️ MMR and multi-query techniques didn’t seem to be as effective
✅ Sentence window retrieval, Auto-merging retrieval, and the document summary index (all native @llama_index techniques) offer promising benefits in either retrieval precision and answer similarity! (And also interesting tradeoffs).
It’s definitely worth giving the full paper a skim. Check it out: arxiv.org/pdf/2404.01037.pdf
TensorChord retweeted
tensorchord / pgvecto.rs: Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database. ★1271 github.com/tensorchord/pgvec…
TensorChord retweeted
TIL about binary vector search... apparently there's a trick where you can take an embedding vector like [0.0051, 0.017, -0.0186, -0.0185...] and turn that into a binary vector just reflecting if each value is > 0 - so [1, 1, -1, -1, ...] and still get useful cosine similarities!
TensorChord retweeted
Conducted benchmarks on the @TensorChord pgvecto.rs new binary vector recently and found that @canva is an excellent platform for creating charts.
The data and style components are separate, allowing for easy editing and customization in the UI
TensorChord retweeted
Here's my take on the Sora technical report, with a good dose of speculation that could be totally off. First of all, really appreciate the team for sharing helpful insights and design decisions – Sora is incredible and is set to transform the video generation community.
What we have learned so far:
- Architecture: Sora is built on our diffusion transformer (DiT) model (published in ICCV 2023) — it's a diffusion model with a transformer backbone, in short:
DiT = [VAE encoder + ViT + DDPM + VAE decoder].
According to the report, it seems there are not much additional bells and whistles.
- "Video compressor network": Looks like it's just a VAE but trained on raw video data. Tokenization probably plays a significant role in getting good temporal consistency. By the way, VAE is a ConvNet, so DiT technically is a hybrid model ;) (1/n)
TensorChord retweeted
Why is Postgres well-suited (or even the best choice) for apps using vector similarity search?
Similarity search alone is insufficient for general-purpose apps that typically filter data using several columns/fields. Pure vector databases address this requirement by allowing you to add text/numeric metadata to vectors and pre-filter by the metadata before initiating the computationally intensive vector similarity search.
This pre-filtering capability has existed in Postgres since its inception. You pre-filter by specifying a condition on non-vector columns in the WHERE clause of a query statement. :)
For instance, imagine a user selecting the `Science Fiction` category and asking to suggest movies with a rating of 7 or higher. Then, the user prompts for `A movie about a space adventure`. The final SQL query might look as shown in the picture.
The data is initially filtered by the vote_average (rank from 1 through 10) and genres (an array of JSON objects) columns, followed by the similarity search on the overview_vector column.
TensorChord retweeted
My friend @gaocegege is looking for an internship at @TensorChord to contribute to pgvecto.rs: a scalable vector database plugin for Postgres, written in Rust.
- 300 RMB/day
- Remote
- #Rust + #Database + #AI
Welcome to take a look!
github.com/tensorchord/pgvec…
TensorChord retweeted
pgvector is the de facto standard for vectorized processing in Postgres.
But its dominance can be challenged soon by pgvecto.rs - a Postgres extension for vector similarity search written in Rust. The extension is still in beta, but its capabilities already look compelling.
github.com/tensorchord/pgvec… – at Florida, USA
TensorChord retweeted
Turn PostgreSQL Into A Vector Store-"pg_vector + pg_vector.rs are extensions for #PostgreSQL that renders it a viable alternative to specialized vector stores used in LLMs"
#llm #langchain #postgres #vector #embedings
@TensorChord
TensorChord retweeted
We are thrilled to announce the release of pgvecto.rs, a powerful Postgres extension for vector similarity search written in Rust. The HNSW algorithm is 20x faster than pgvector at 90% recall.
modelz.ai/blog/pgvecto-rs
#pgvector #vector #LLM #ArtificialInteligence
TensorChord retweeted
LLMOps has its own awesome repo👇⚡️
@TensorChord
github.com/tensorchord/Aweso…
TensorChord retweeted
envd: 🏕️ Reproducible development environment for AI/ML
Lang: Go
⭐️ 1653
Author: @TensorChord
#MachineLearning
github.com/tensorchord/envd
TensorChord retweeted
Github見に行ったら、早速LLMOpsに有益なツール群をまとめてくれている人がいた。みんなトレンドに乗るのが早いなー
github.com/tensorchord/Aweso…



























