@PeterNickl_

PhD @ Pouget Lab @NeuroGeneva @pouget_alex | previously RA @ Adaptive Bayesian Intelligence Team @RIKEN_AIP_EN @RIKEN_EN | MSc @TUDarmstadt @ias_tudarmstadt

 Geneva, Switzerland
pnickl.github.io
Joined July 2019
  • Tweets89
  • Following1.9K
  • Followers337
If you are NeurIPS, feel free to drop by our poster on improving LoRA for LLMs using variational learning. Saturday East Exhibition Hall, 4:40 p.m.–5:30 p.m. (Poster session II). I'm not at NeurIPS this year but please say Hi to Cong Bai who is presenting arxiv.org/abs/2411.04421

Variational Low-Rank Adaptation Using IVON

We show that variational learning can significantly improve the accuracy and calibration of Low-Rank Adaptation (LoRA) without a substantial increase in the cost. We replace AdamW by the Improved...

arxiv.org
NeurIPS24 Workshop: Fine-tuning in ML@neurips24fitml10 Dec 2024
Welcome to our fine-tuning workshop at Saturday East Exhibition Hall A!
4
21
1,740
Next Thursday, Sept. 12: Seminar on Advances in Probabilistic Machine Learning returns, with @tmoellenhoff: Variational Learning is Effective for Large Deep Networks. Register here for the Zoom link: aaltoml.github.io/apml/ cc @arnosolin @CSAalto @UnivHelsinkiCS @RIKEN_AIP_EN

Seminar on Advances in Probabilistic Machine Learning

This seminar series aims to provide a platform for young researchers (PhD student or post-doc level) to give invited talks about their research, intending to have a diverse set of talks & speakers on...

aaltoml.github.io
7
8
1,575
Peter Nickl retweeted
Yuesong Shen@ysngshn30 Jul 2024
The JAX implementation of IVON is now available at github.com/ysngshn/ivon-opta… Let us know if you have any issues or success stories to share! #JAX #IVON #deeplearning

GitHub - ysngshn/ivon-optax: An Optax-based JAX implementation of the IVON optimizer for large-sc...

An Optax-based JAX implementation of the IVON optimizer for large-scale VI training of NNs (ICML'24 spotlight) - ysngshn/ivon-optax

github.com
Yuesong Shen@ysngshn21 Jul 2024
Good news for JAX users! We are preparing a JAX version of our IVON optimizer (github.com/team-approx-bayes…). I have implemented it and we are double-checking now. Feel free to ping us if you are impatient to try it out 😁 #ICML2024 #JAX #deeplearning #IVON

GitHub - team-approx-bayes/ivon: IVON optimizer for neural networks based on variational learning.

IVON optimizer for neural networks based on variational learning. - team-approx-bayes/ivon

github.com
3
1
13
1,150
Peter Nickl retweeted
UKP Lab@UKPLab9 Jul 2024
✨#ICML2024 Spotlight✨ 🤔 Variational learning is often thought to be impractical. 🔥 Plot twist: it actually works, with improvements over Adam! Meet IVON, a new LLM optimizer that brings the best out of variational learning – 🧵 (1/4) 📰 arxiv.org/abs/2402.17641 #NLProc
1
8
27
1,961
We don't expect Bayesian methods to do so well at large scale, but we can now get decent improvements with variational learning to GPT-2. I wrote a blog about this (first one in a long time). Check it out! team-approx-bayes.github.io/… Paper: arxiv.org/abs/2402.17641 A thread below.
12
55
4
268
33,923
Peter Nickl retweeted
myamada0
@myamada01 Mar 2024
MLSS 2024 is ready!!! See you next Monday!
5
32
3,732
Peter Nickl retweeted
Etash Guha
@etash_guha17 Jan 2024
Excited and humbled that my paper “Conformal Prediction via Regression-as-Classification” was accepted at #ICLR2024! Huge shoutout to my collaborators @EmtiyazKhan, @tmoellenhoff, @eugene_ndiaye, @Shlok_Natarajan! More details coming soon! Paper: etash.me/papers/Bayesian_Con…
3
7
32
3,333
Really excited that our work on model merging has been accepted as a poster to ICLR 2024! There have been many discussions recently on model merging on here and many people are questioning why it should work. We try to answer exactly this, as briefly summarized in this thread.
Nico Carlsen Daheim@ndaheim_17 Jan 2024
Replying to @natolambert @__dipam__
We tried to answer this in our recent work: arxiv.org/abs/2310.12808 TL;DR (Sec. 3): merging models trained on individual data approximates a target model trained on all data and is expected to work if the individual models are close to this model in terms of their gradients.

Model Merging by Uncertainty-Based Gradient Matching

Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail? Here, we connect the inaccuracy of weighted-averaging to...

arxiv.org
1
6
48
3,336
Duality dinner at Fogo de Chao, with @ZeldaMariet @tmoellenhoff and so many other folks. We will be doing more such meetings in the future (it started at ICML 2023 dp4ml.github.io/) Hope more will join in our efforts in the future. Also check out bayesduality.github.io/
1
3
33
3,299
I'm at NeurIPS for the first time, looking forward to talking about our framework to estimate sensitivity during training at our poster on Wed! Come by if you are interested: Wed 13 Dec 5 p.m. CST — 7 p.m. CST Great Hall & Hall B1+B2 (level 1) #1310 neurips.cc/virtual/2023/post…
Thomas Möllenhoff@tmoellenhoff10 Dec 2023
First time at NeurIPS since 2019. I'm presenting our work on deriving influence measures from a Bayesian viewpoint (with @EmtiyazKhan @PeterNickl_ @dtailor17 @L_u_X_u). Suggests cheap ways of computing influence also during training. Please say hi:) arxiv.org/abs/2310.19273
10
551
@tmoellenhoff and @PeterNickl_ from our group will also be there presenting their paper on memory perturbation, which generalizes influence function to all sorts of algorithms (arxiv.org/abs/2310.19273), and remove the need to invert matrices. Hope to see you all soon! /end

The Memory Perturbation Equation: Understanding Model's...

Understanding model's sensitivity to its training data is crucial but can also be challenging and costly, especially during training. To simplify such issues, we present the Memory-Perturbation...

arxiv.org
6
17
1,745
The tech-staff position is a really great pre-doc opportunity to jump start your research career in ML. Not only do you get to work with brilliant people at the cutting edge of the field but you get to live/work in the centre of Tokyo. (1/2)
Emtiyaz Khan@EmtiyazKhan27 Sep 2023
We have two open positions in our group in Tokyo (start date is April 2024). Postdoc/research-scientist: riken.jp/en/careers/research… Postdoc or tech-staff (pre-PhD): riken.jp/en/careers/research… Help me spread the word!
1
2
10
3,907
We have two open positions in our group in Tokyo (start date is April 2024). Postdoc/research-scientist: riken.jp/en/careers/research… Postdoc or tech-staff (pre-PhD): riken.jp/en/careers/research… Help me spread the word!
48
7
103
39,881
We spent yesterday with Yoshida-team who have another @JST_Kisokenkyu CREST grant (just like our @BayesDuality project). Wonderful to visit the Komaba campus of UTokyo, hear about wonderful work on stochastic processes and spatiotemporal models (and have some French food).
2
24
3,712
Inspiring talks by @krikamol and @mundt_martin on „(Im)possibility of Collective Intelligence“ and „Self-Expanding Neural Networks“! Thank you for visiting us at RIKEN AIP. 😊
4
2
25
15,848
The third day of MLRS 2023. @EmtiyazKhan and his team are convincing our participants of the importance of uncertainty in lifelong learning. #MLRS2023 #Bayes #Uncertainty
6
1
48
9,153
At #ICML2023 we will have a workshop on Duality Principles. "Duality is a principle, it gives two different views of the same object" Duality is not a niche topic, it's for everybody. Hope to see you there! dp4ml.github.io/schedule/ @tmoellenhoff @ZeldaMariet @mblondel_ml
1
27
2
146
28,570
Replying to @miniapeur
@geowolfer nominated for top Israeli math prize
1
3
7
3,659
Peter Nickl retweeted
hessian.AI@Hessian_AI27 Jun 2023
👋 Meet @EmtiyazKhan at The Hessian #AICon on July 5! The Team Leader @RIKEN_AIP_EN gives a talk on the topic "How to make machines that adapt quickly" around 1:45 p.m. Check out the agenda & speakers 👉 hessian.ai/aicon-2023/day-2/. Get your free ticket 👉 hessian.ai/aicon-2023/.
2
1
7
5,110
Peter Nickl retweeted
Joe Watson@JoeMWatson9 Jun 2023
Excited to share my internship project from my time at @DeepMind, looking at sample-efficient imitation learning using entropy-regularized reinforcement learning TL;DR: do behavioral cloning (BC), get inverse reinforcement learning (IRL) for free! [1/6]
2
20
3
246
38,525
Load more