About this channel
Yannic Kilcher's channel centres on examining AI research, with paper analyses and reviews forming a substantial part of the selection. Language model architectures, memory, retrieval and the effects…
Yannic Kilcher's channel centres on examining AI research, with paper analyses and reviews forming a substantial part of the selection. Language model architectures, memory, retrieval and the effects of context length all appear as subjects. The channel also includes responses to technical arguments, broader claims about AI's direction and more informal projects and live streams.
The paper discussions name particular approaches and questions. TiDAR provides an entry point into diffusion and autoregressive generation, while Titans is discussed through the idea of learning to memorise at test time. Other material looks at energy-based transformers and the theoretical limitations of embedding-based retrieval. These topics invite viewers to consider the mechanisms behind a proposed approach and the boundaries of what it can do.
Practical consequences run alongside architecture. A discussion of context rot examines the relationship between increasing input length and model performance, and a response about code mode addresses MCP and tool calling. The tone also has room for provocation and humour, visible in a claim about AGI's prospects and a project described as an automatic mansplainer. Holiday streams provide an additional informal format.
The strongest fit is for viewers who already encounter machine-learning terminology and want to engage with research discussion. Engineers, students and technically curious viewers can choose a paper analysis or a response to a specific debate, then compare the arguments involved. The combination of close technical subjects and opinionated framing makes the channel useful for following questions within AI research, especially when the interest is in why a method might work and where its limitations may lie.
Latest episode
I BUILT A FULLY AUTOMATIC MANSPLAINER
All information about GTC and the DGX Spark Raffle is here: https://www.ykilcher.com/gtc
Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube…
All information about GTC and the DGX Spark Raffle is here: https://www.ykilcher.com/gtc
Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
LinkedIn: https://www.linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content 🙂
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
YouTube Shorts
Adversarial Examples
Adversarial Machine Learning
AGI
Applications of ML
Best Of
Channel Updates
Classic Papers
Computer Vision
Deep Learning Architectures
Drama
General Machine Learning
Generative Models
More episodes from this channel
TiDAR: Think in Diffusion, Talk in Autoregression (Paper Analysis)
27.12.2025
Paper: https://arxiv.org/abs/2511.08923 Abstract: Diffusion language models hold the promise of fast parallel generation, while autoregressive (AR) models typically excel in quality due to their causal structure aligning naturally with language modeling. This raises a...
Titans: Learning to Memorize at Test Time (Paper Analysis)
14.12.2025
Paper: https://arxiv.org/abs/2501.00663 Abstract: Over more than a decade there has been an extensive research effort on how to effectively utilize recurrent models and attention. While recurrent models aim to compress the data into a...
[Paper Analysis] The Free Transformer (and some Variational Autoencoder stuff)
1.11.2025
https://arxiv.org/abs/2510.17558 Abstract: We propose an extension of the decoder Transformer that conditions its generative process on random latent variables which are learned without supervision thanks to a variational procedure. Experimental evaluations show that allowing...
[Video Response] What Cloudflare's code mode misses about MCP and tool calling
19.10.2025
Theo's Video: https://www.youtube.com/watch?v=bAYZjVAodoo Cloudflare article: https://blog.cloudflare.com/code-mode/ Links: Homepage: https://ykilcher.com Merch: https://ykilcher.com/merch YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ykilcher.com/discord LinkedIn: https://www.linkedin.com/in/ykilcher If you want to support me, the best thing to do is to share out...
[Paper Analysis] On the Theoretical Limitations of Embedding-Based Retrieval (Warning: Rant)
11.10.2025
Paper: https://arxiv.org/abs/2508.21038 Abstract: Vector embeddings have been tasked with an ever-increasing set of retrieval tasks over the years, with a nascent rise in using them for reasoning, instruction-following, coding, and more. These new benchmarks...
Context Rot: How Increasing Input Tokens Impacts LLM Performance (Paper Analysis)
23.7.2025
Paper: https://research.trychroma.com/context-rot Abstract: Large Language Models (LLMs) are typically presumed to process context uniformly—that is, the model should handle the 10,000th token just as reliably as the 100th. However, in practice, this assumption does...
Energy-Based Transformers are Scalable Learners and Thinkers (Paper Review)
19.7.2025
Paper: https://arxiv.org/abs/2507.02092 Code: https://github.com/alexiglad/EBT Website: https://energy-based-transformers.github.io/ Abstract: Inference-time computation techniques, analogous to human System 2 Thinking, have recently become popular for improving model performances. However, most existing approaches suffer from several limitations: they are...
On the Biology of a Large Language Model (Part 2)
3.5.2025
An in-depth look at Anthropic's Transformer Circuit Blog Post Part 1 here: https://youtu.be/mU3g2YPKlsA Discord here: https;//ykilcher.com/discord https://transformer-circuits.pub/2025/attribution-graphs/biology.html Abstract: We investigate the internal mechanisms used by Claude 3.5 Haiku — Anthropic's lightweight production model —...
On the Biology of a Large Language Model (Part 1)
5.4.2025
An in-depth look at Anthropic's Transformer Circuit Blog Post https://transformer-circuits.pub/2025/attribution-graphs/biology.html Abstract: We investigate the internal mechanisms used by Claude 3.5 Haiku — Anthropic's lightweight production model — in a variety of contexts, using our...
[GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
26.1.2025
#deepseek #llm #grpo GRPO is one of the core advancements used in Deepseek-R1, but was introduced already last year in this paper that uses a combination of new RL techniques and iterative data collection...
Traditional Holiday Live Stream
27.12.2024
https://ykilcher.com/discord Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yannic-kilcher Minds: https://www.minds.com/ykilcher Parler: https://parler.com/profile/YannicKilcher LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/ BiliBili: https://space.bilibili.com/1824646584 If you want to support me, the best thing to do...
Byte Latent Transformer: Patches Scale Better Than Tokens (Paper Explained)
24.12.2024
#tokenization #llm #meta This paper does away with tokenization and creates an LLM architecture that operates on dynamically sized "patches" instead of tokens. By controlling the patch size, they gain a level of control...
Safety Alignment Should be Made More Than Just a Few Tokens Deep (Paper Explained)
10.12.2024
This paper demonstrates in a series of experiments that current safety alignment techniques of LLMs, as well as corresponding jailbreaking attacks, are in large part focusing on modulating the distribution of the first few...
TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters (Paper Explained)
23.11.2024
A deep dive into the TokenFormer and an opinion about its impact, novelty, and relation to prior work. Paper: https://arxiv.org/abs/2410.23168 Abstract: Transformers have become the predominant architecture in foundation models due to their excellent...
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
19.10.2024
This paper (by Apple) questions the mathematical reasoning abilities of current LLMs and designs a synthetic template-based dataset distribution to investigate various aspects around LLM performance of high-school level math questions. Paper: https://arxiv.org/abs/2410.05229 Abstract:...
Were RNNs All We Needed? (Paper Explained)
12.10.2024
This paper posits the interesting question: How much of the performance of Mamba, S4, and other state-space-like models is actually just attributable to some very core concepts - rather than their elaborate architectures. The...
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (Paper)
5.10.2024
How can one best use extra FLOPS at test time? Paper: https://arxiv.org/abs/2408.03314 Abstract: Enabling LLMs to improve their outputs by using more test-time computation is a critical step towards building generally self-improving agents that...
Privacy Backdoors: Stealing Data with Corrupted Pretrained Models (Paper Explained)
4.8.2024
#llm #privacy #finetuning Can you tamper with a base model in such a way that it will exactly remember its fine-tuning data? This paper presents a method of doing exactly that, and implements it...
Scalable MatMul-free Language Modeling (Paper Explained)
8.7.2024
Matrix multiplications (MatMuls) are pervasive throughout modern machine learning architectures. However, they are also very resource intensive and require special accelerators (GPUs). This paper explores architectures that do away with MatMuls and use quantization...
Continue reading
You may also like
Innohub TV
Watch this content also on Innohub TV
We picked clips from Innohub TV that continue the same topic.
