Dan Busbridge retweeted
Stop by poster #596 at 10A-1230P tomorrow (Fri 25 April) at #ICLR2025 to hear more about Sigmoid Attention!
We just pushed 8 trajectory checkpoints each for two 7B LLMs for Sigmoid Attention and a 1:1 Softmax Attention (trained with a deterministic dataloader for 1T tokens):
- Sigmoid: gs://axlearn-public/experiments/gala-7B-sigmoid-hybridnorm-alibi-sprp-2024-12-03-1002/checkpoints/
- Softmax: gs://axlearn-public/experiments/gala-7B-hybridnorm-alibi-sprp-2024-12-02-1445/checkpoints/
Inference code at github.com/apple/ml-sigmoid-…
Dan Busbridge retweeted
Parameterized Transforms 🚀
Here is a new tool that provides a modular and extendable implementation of torchvision-based image augmentations that provides access to their parameterization. [1/5]
Dan Busbridge retweeted
🚨 One question that has always intrigued me is the role of different ways to increase a model's capacity: parameters, parallelizable compute, or sequential compute?
We explored this through the lens of MoEs:
Dan Busbridge retweeted
We attempted to make Normalizing Flows work really well, and we are happy to report our findings in paper arxiv.org/pdf/2412.06329, and code github.com/apple/ml-tarflow. [1/n]
Dan Busbridge retweeted
Thrilled to share the latest work from our team at @Apple where we achieve interpretable and fine-grained control of LLMs and Diffusion models via Activation Transport 🔥
📄 arxiv.org/abs/2410.23054
🛠️ github.com/apple/ml-act
1/9 🧵
Dan Busbridge retweeted
📢Internships at Apple ML Research🍏
We’re looking for a PhD research intern with interests in uncertainty quantification, LLMs, probabilistic ML and/or decision making under uncertainty!
See thread for more details 👇
[1/3]
Dan Busbridge retweeted
Check out ‘Dataset Decomposition’, a new paper from the Machine Learning Research team @Apple that improves the training speed of LLMs by up to 3x:
arxiv.org/abs/2405.13226
Hadi Pouransari et al.
#EfficientML #AI #ML #Apple
Dan Busbridge retweeted
Have a research idea at the intersection of generative models and RL, IL or planning? Consider submitting a short (4 pages) or long (9 pages) paper to our GenAI+Decision Making workshop at ICLR 2024: sites.google.com/view/GenAI4…
More information coming early January!
Dan Busbridge retweeted
Interested in Bayes + Self-Supervised Learning? Come to the SSL workshop at #NeurIPS2023 Room 217
We show that coupling Bootstrap Your Own Latent (BYOL) with Bayes-by-Backprop (BBB) learns a distrbutionally aligned predictive variance with supervised learning without any labels.
Dan Busbridge retweeted
I'll present Manifold Diffusion Fields as an oral at the Diffusion Models workshop @NeurIPSConf, today 2:30 pm New Orleans time!!
work with @YuyangW95 @jsusskin and @itsbautistam
Don't forget to join and let's know your questions and feedback!
openreview.net/pdf?id=0AMMdK…
Dan Busbridge retweeted
Stocked to present « Bilevel Optimization to Learn Training Distributions under Domain Shift » at the Neurips Distribution shift workshop!
Joint work w. my amazing colleagues @GrangierDavid and @awnihannun !
Paper : arxiv.org/abs/2311.11973
Dan Busbridge retweeted
MDF is an oral at the Diffusion Models workshop tomorrow at @NeurIPSConf! Catch @Ahmed_AI035’s talk (via zoom because well…visas) and also @YuyangW95 and I will be around in the poster session! Come say hi and lets chat about practical diffusion models on manifolds!
Introducing Manifold Diffusion Fields (MDF), our new work on learning generative models over fields defined on curved geometries. This is joint work with our intern @Ahmed_AI035 (who hasn’t even started his PhD yet!) and @jsusskin at @Apple MLR arxiv.org/abs/2305.15586 🧵
Dan Busbridge retweeted
MLR at Apple just opened an office in Copenhagen, Denmark 🎉🎉💃🇩🇰 And we are hiring!
jobs.apple.com/en-us/details…
Please reach out if you're interested! This office opening is pretty neat just on its own, if I do say so myself. 😊 1/
Dan Busbridge retweeted
I wanted to advertise that we have opportunities in my team at Apple MLR in scalable/distributed ML for multimodal foundational models, embodied AI, planning and reasoning. More info below but feel free to send me a note. tinyurl.com/ms77zkam
Dan Busbridge retweeted
Just in time for the holidays, we are releasing some new software today from Apple machine learning research.
MLX is an efficient machine learning framework specifically designed for Apple silicon (i.e. your laptop!)
Code: github.com/ml-explore/mlx
Docs: ml-explore.github.io/mlx/bui…
Dan Busbridge retweeted
Apple ML Research in Cambridge, UK is looking for a PhD intern 🎓
Topics: Self supervised representation learning, generative modeling, diffusion models.
Job Req: jobs.apple.com/en-gb/details…
Dan Busbridge retweeted
Our Apple ML Research team in Barcelona is looking for a PhD intern! 🎓
Curiosity-driven research 🧠 with the goal to publish 📝
Topics: Confidence/uncertainty quantification and reliability of LLMs 🤖
Apple here: jobs.apple.com/en-gb/details…
Dan Busbridge retweeted
Introducing Manifold Diffusion Fields (MDF), our new work on learning generative models over fields defined on curved geometries. This is joint work with our intern @Ahmed_AI035 (who hasn’t even started his PhD yet!) and @jsusskin at @Apple MLR arxiv.org/abs/2305.15586 🧵
Manifold Diffusion Fields
present Manifold Diffusion Fields (MDF), an approach to learn generative models of continuous functions defined over Riemannian manifolds. Leveraging insights from spectral geometry analysis, we define an intrinsic coordinate system on the manifold via the eigen-functions of the Laplace-Beltrami Operator. MDF represents functions using an explicit parametrization formed by a set of multiple input-output pairs. Our approach allows to sample continuous functions on manifolds and is invariant with respect to rigid and isometric transformations of the manifold. Empirical results on several datasets and manifolds show that MDF can capture distributions of such functions with better diversity and fidelity than previous approaches
paper page: huggingface.co/papers/2305.1…
Dan Busbridge retweeted
Planning to attend @CVPR? Check out the workshop sessions on Sunday, I will be talking about generative modeling for fields and manifolds at the @_LXAI workshop!
Dan Busbridge retweeted
We are hiring an ML Research Engineer to work on the forefront of AI in the health space. If you are passionate about building robust DL models, want to work with outstanding ML researchers + engineers, want to have real impact with your work: apply! jobs.apple.com/en-us/details…































