I like to make computers learn things
Joined December 2020
- Tweets108
- Following138
- Followers38
- Likes1.7K
Zach Robertson retweeted
the techno-optimist crowd often assumes Musk and Thiel share the same political alignment because of their backgrounds. in reality, nothing could be further from the truth.
Zach Robertson retweeted
New open weights LLM from @MistralAI
params.json:
- hidden_dim / dim = 14336/4096 => 3.5X MLP expand
- n_heads / n_kv_heads = 32/8 => 4X multiquery
- "moe" => mixture of experts 8X top 2 👀
Likely related code:
github.com/mistralai/megablo…
Oddly absent: an over-rehearsed professional release video talking about a revolution in AI.
If people are wondering why there is so much AI activity right around now, it's because the biggest deep learning conference (NeurIPS) is next week.
magnet:?xt=urn:btih:5546272da9065eddeb6fcd7ffddeef5b75be79a7&dn=mixtral-8x7b-32kseqlen&tr=udp%3A%2F%2Fopentracker.i2p.rocks%3A6969%2Fannounce&tr=http%3A%2F%https://nitter.gq/t.co/g0m9cEUz0T%3A80%2Fannounce
RELEASE a6bbd9affe0c2725c1b7410d66833e24
Zach Robertson retweeted
magnet:?xt=urn:btih:5546272da9065eddeb6fcd7ffddeef5b75be79a7&dn=mixtral-8x7b-32kseqlen&tr=udp%3A%2F%2Fopentracker.i2p.rocks%3A6969%2Fannounce&tr=http%3A%2F%https://nitter.gq/t.co/g0m9cEUz0T%3A80%2Fannounce
RELEASE a6bbd9affe0c2725c1b7410d66833e24
Zach Robertson retweeted
I spent some time building this obsidian-meets-chatgpt-meets-tab style application. It can scrape the web when I add a link in my notes, and it will both vectorize and create graph-based relationships for context later on.
I ended up dropping the project for more pressing work, but want to revisit it if nothing else like it exists. Anyone know of something useful that basically does this? Would anyone use this?
(in the example, I'm using my notes and code to query the app about itself. the graph above represents the related notes and tags)
Zach Robertson retweeted
At the core of BitSurf’s discovery engine is a content-first recommendation algorithm. We forego a virality and engagement driven algorithm because doing so enables deeper and more relevant recommendations than traditional social media websites.
Here’s a simple demonstration.
A blog post shared by Andrej Karpathy details the surprising power of a MacBook’s ability to run Large Language Models.
On the X platform, the discover more feature does not really demonstrate clear information relationship between the original content and the recommended posts.
The same content, indexed on BitSurf, produces a recommendation of a post giving instructions on installing and running an LLM on an M1 MacBook Pro.
The quality of recommendations speaks for itself.
What will you discover?
Zach Robertson retweeted
We call this product market fit.
Follow for updates over at @BitSurfApp
This Post is from an account that no longer exists. Learn more
Zach Robertson retweeted
The search experience is fundamentally broken. This is because Google was originally designed for a dense flat internet of honest actors.
Now, the internet is siloed within a feudalistic landscape of large internet conglomerates. Bots run amok, and synthetic content is adding more and more redundant information to the landscape. The web is now far more deep than wide, and search indexes will miss all of this content.
LLMs aren’t going to fix this. The tools to scan and scrape the internet are now a part of a perpetual arms race against those who want to protect their data silos. Twitter vs OpenAI comes to mind.
The way we discover information across the web needs to fundamentally change.
I’ve been building a tool called BitSurf (@BitSurfApp ) that aims to solve this problem. You can make an account for free and try out our early proof of concept for a platform based solution. There’s a lot more to come.
Here’s a great video that further elaborates on the deterioration of search.
youtu.be/48AOOynnmqU
Zach Robertson retweeted
This is a good point.
My company is working on building the guide and the remote control.
Zach Robertson retweeted
"This submarine uses the same technology SpaceX uses to build rockets!"
"How many atmospheres of pressure can that withstand?"
"Well, it's for a spaceship, so somewhere between zero and one."
Zach Robertson retweeted
BitSurf Hackathon Retrospective:
Team gathered in Austin, TX this weekend to set the stage for the next era of BitSurf. In short:
🤖🪄 More AI
💭📢 More social features
🚀🌴 More pro on-ramps
Are you ready?
photo by @ZachRTech on BitSurf
Zach Robertson retweeted
Introducing MeZO - a memory-efficient zeroth-order optimizer that can fine-tune large language models with forward passes while remaining performant. MeZO can train a 30B model on 1x 80GB A100 GPU.
Paper: arxiv.org/abs/2305.17333
Code: github.com/princeton-nlp/MeZ…
Zach Robertson retweeted
Nvidia just released ATT3D: Amortized Text-To-3D Object Synthesis
project page: research.nvidia.com/labs/tor…
Text-to-3D modeling has seen exciting progress by combining generative text-to-image models with image-to-3D methods like Neural Radiance Fields. DreamFusion recently achieved high-quality results but requires a lengthy, per-prompt optimization to create 3D objects. To address this, we amortize optimization over text prompts by training on many prompts simultaneously with a unified model, instead of separately. With this, we share computation across a prompt set, training in less time than per-prompt optimization. Our framework -- Amortized text-to-3D (ATT3D) -- enables sharing of knowledge between prompts to generalize to unseen setups and smooth interpolations between text for novel assets and simple animations.