Diffusion Transformers

Diffusion transformers, or DiT, are discussed in 133. 对谢赛宁的7小时马拉松访谈：世界模型、逃出硅谷、AMI Labs、两次拒绝Ilya、杨立昆、李飞飞和42 as a FAIR-period architecture result connected to Xie Saining’s broader Representation Learning line. Xie says DiT began from thinking about diffusion-model representations rather than simply deciding to make a diffusion model.

Source View

The source treats DiT as meaningful but not as a field-defining revolution on the scale of LeNet, AlexNet, ImageNet, ResNet, Transformer, GPT-3, BERT, CLIP, ViT, or GAN. It then connects later REPA and RAE work to the question of whether strong representations can guide or become the encoder foundation for generative models.

Connections

• Xie Saining and FAIR — researcher and lab context.
• Representation Learning and Self-Supervised Learning — source of the representation question.
• Video Models, World Models, and Multimodal Intelligence — adjacent generative and predictive domains.
• Research Taste — DiT is used as an example of research emerging through exploration rather than a linear plan.