@keyutian

first author of NeurIPS 2024 best paper been in【lab mode】 more soon

 world
scholar.google.com/citations?user=6FdkbygAAAAJ&hl=en-US
Joined October 2021
  • Tweets30
  • Following43
  • Followers3.1K
Bloomberg has the deets on what we’ve been up to👇🏻As for the "world model" label, no rush to name it or rename it. We’re keeping it simple: build the foundation model, scale it, and let the results speak. More soon.
Bloomberg
@businessOct 8
A former ByteDance intern has secured funding from venture heavyweights including 5Y for a stealth startup to take on AI pioneers such as Fei-Fei Li in the nascent realm of world models bloomberg.com/news/articles/…

Ex-ByteDance Intern’s $200 Million AI Lab Takes On Fei-Fei Li

A former ByteDance Ltd. intern has secured funding from venture heavyweights including 5Y Capital for a stealth startup to take on AI pioneers such as Fei-Fei Li in the nascent realm of world models.

bloomberg.com
38
6
4
193
54,421
hello world. it’s been a while. this is Sha Sha 🐰 my sidekick for the past year. he’s the best. the thing about rabbits: they can be super random and surprisingly hard to predict. working on that. 😉
5
2
19
8,332
SparK - [ICLR'23 Spotlight] The first successful BERT-style pretraining on any *convolutional network*; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling" github.com/keyu-tian/SparK

GitHub - keyu-tian/SparK: [ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on...

[ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchi...

github.com
2
6
9,885
Keyu Tian 田柯宇 retweeted
/MachineLearning@slashML23 Jan 2023
: The first BERT-style pretraining on CNNs! #nlp #cnn reddit.com/r/MachineLearning…
1
7
18
11,479
Now this paper has been accepted to #ICLR2023 as a spotlight🌟! I truly appreciate everyone's efforts. Check out our repo github.com/keyu-tian/SparK for the latest demo video and updates! Title: "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"
Sebastian Raschka
@rasbt17 Jan 2023
How can we leverage successful pretraining techniques from transformers to improve purely convolutional networks? The answer is *Sparse Convolutions*! Let's see what happens when purely convolutional networks are pretrained with 1.28 million unlabeled images ... 1/7
4
12
82
42,660
PS: our original submission in Oct. 2022 was titled "Sparse and Hierarchical Masked Modeling for Convolutional Representation Learning"; we've changed it to the new one to more clearly express our vision for bringing the powerful BERT-style self-supervised learning to CNNs.
1
6
10,038
画像認識の事前学習で、ViTは一部の入力をMaskし、それを予測するタスクが成功していたがConvは隣り合うパッチ間で重複があり情報が漏れ成功してなかった。SparKは出力も入力と同じマスクを維持するSparseConvを使うことで問題を解決、また復号器側も階層を持ちさらに改善 openreview.net/forum?id=NRxy…
9
60
14,578
Keyu Tian 田柯宇 retweeted
Sebastian Raschka
@rasbt17 Jan 2023
How can we leverage successful pretraining techniques from transformers to improve purely convolutional networks? The answer is *Sparse Convolutions*! Let's see what happens when purely convolutional networks are pretrained with 1.28 million unlabeled images ... 1/7
9
101
5
482
146,957
Keyu Tian 田柯宇 retweeted
fly51fly@fly51fly14 Jan 2023
[CV] Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling K Tian, Y Jiang, Q Diao, C Lin, L Wang, Z Yuan [Peking University & Bytedance Inc & University of Oxford] (2023) arxiv.org/abs/2301.03580 #MachineLearning #ML #AI #CV [1/2]
1
10
1
30
9,500
Keyu Tian 田柯宇 retweeted
DeepAI
@DeepAI12 Jan 2023
🤩The first successful BERT-style #SelfSupervisedLearning on any convolutional network! #ResNet now enjoys masked autoencoding! 🚀A breakthrough paper "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling" by @keyutian et al. deepai.org/publication/desig…

DeepAI

Artificially intelligent tools for naturally creative humans.

deepai.org
1
3
8
3,389
Keyu Tian 田柯宇 retweeted
Dmytro Mishkin
@ducha_aiki11 Jan 2023
Designing BERT for convolutional networks: sparse and hierarchical masked modeling Keyu Tian, Yi Jiang, Qishuai Diao, Chen Lin, Liwei Wang, Zehuan Yuan tl;dr: create image, which looks to CNN same, as transformers -> MIM starts working arxiv.org/abs/2301.03580.pdf
4
25
1
116
25,355
Load more