SpeakCoveTen landmark deep learning papers to read first are AlexNet (2012), Word2Vec and DQN (2013), GANs and Dropout (2014), Adam and Batch Normalization (2015), ResNet (2016), the Transformer paper "Attention Is All You Need" (2017) and BERT (2019). All ten have free PDFs: eight on arXiv, AlexNet in the NeurIPS proceedings and Dropout in JMLR. Years are the conference or journal publication year.
These are the papers later work keeps citing, so understanding them gives you a framework for evaluating everything that comes after. Reading dense technical papers on a screen is slow and tiring. Listening to them lets you take in the key ideas during commutes, workouts or walks, then go back and study the equations on screen later. On a computer, open any of these PDFs in SpeakCove's free web reader at speakcove.com/pdf-to-speech; on iPhone, the app's paper search finds the arXiv ones by title.
The books
1“Attention Is All You Need” by Vaswani et al. (2017), NeurIPS
The paper that introduced the Transformer, the architecture behind today's large language models. The prose sections explaining self-attention, the encoder-decoder structure, and positional encoding are surprisingly clear when listened to. Understanding this paper puts much of modern ML in context.
2“ImageNet Classification with Deep Convolutional Neural Networks (AlexNet)” by Krizhevsky, Sutskever, and Hinton (2012), NeurIPS
The paper that showed deep convolutional networks trained on GPUs could win ImageNet by a wide margin. The architectural decisions and training tricks described here became the template for years of computer vision research. The methodology section packs many practical insights into a short paper.
3“Generative Adversarial Networks” by Goodfellow et al. (2014), NeurIPS
The GAN paper is well written for a technical paper. The core idea, two networks competing against each other, is intuitive enough that the prose sections are engaging as audio. Skip the proofs on first listen and focus on the framework.
4“BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” by Devlin et al. (2019), NAACL; arXiv preprint 2018
BERT showed that pre-training on unlabeled text could produce representations useful for a wide range of NLP tasks. The paper is clearly structured and the masked language modeling idea is easy to follow by ear. Essential context for why pre-training and fine-tuning became the dominant approach.
5“Deep Residual Learning for Image Recognition (ResNet)” by He et al. (2016), CVPR; arXiv preprint 2015
Residual (skip) connections, one of the simplest and most important ideas in deep learning, are introduced here. The paper explains why deeper networks were getting worse and how residual connections fix it. A model of concise technical writing that works well as audio.
6“Efficient Estimation of Word Representations in Vector Space (Word2Vec)” by Mikolov et al. (2013), arXiv / ICLR workshop
The paper that introduced the CBOW and Skip-gram models and helped make word embeddings mainstream. It builds on the well-known 'king minus man plus woman is close to queen' result from a companion paper. The intuitions are explained clearly in prose, so it works as audio even if you skip the math.
7“Playing Atari with Deep Reinforcement Learning (DQN)” by Mnih et al. (2013), NeurIPS Deep Learning Workshop
DeepMind's paper showed that one architecture and learning algorithm, with no game-specific changes, could learn seven Atari 2600 games from raw pixels, beating previous approaches on six and a human expert on three. It opened deep reinforcement learning as a research direction.
8“Dropout: A Simple Way to Prevent Neural Networks from Overfitting” by Srivastava, Hinton, Krizhevsky, Sutskever, and Salakhutdinov (2014), JMLR
One of the most practically useful techniques in deep learning, explained with unusual clarity. The biological motivation and the ensemble interpretation make the prose sections engaging and easy to follow as audio.
9“Adam: A Method for Stochastic Optimization” by Kingma and Ba (2015), ICLR
The optimizer many practitioners use by default. The paper explains the motivation behind adaptive learning rates with enough intuition that you can follow the key ideas by ear, even though the algorithm details require reading the equations on screen.
10“Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift” by Ioffe and Szegedy (2015), ICML
Batch norm changed how many neural networks are trained. The paper's argument about internal covariate shift is clear to follow as audio, but later work (Santurkar et al., 2018) suggests the technique helps mainly by smoothing the optimization landscape, so listen to both.
Listening Tips
- Listen to the abstract, introduction, and conclusion first to get the big picture. Then go back for the methodology section on a second listen.
- Do not worry about equations on your first audio pass. Focus on the intuitions and motivations: the prose around the math often contains the real insights.
- Use SpeakCove's sentence highlighting to follow along with technical terminology. Seeing unfamiliar terms while hearing them makes them easier to place.
- Listen at 1x for technical papers, and drop to 0.75x for dense sections. The information density is much higher than in a typical book.
- Pair audio listening with a later screen review. Listen during your commute to get the concepts, then skim the paper on screen to study figures and equations.
Why SpeakCove
On a computer, open a paper's PDF at speakcove.com/pdf-to-speech and press Play: it's free, with 10 voices, no sign-up and no page limit, and the file never leaves your browser. On iPhone and iPad, SpeakCove's paper search covers arXiv, Semantic Scholar and OpenAlex and lists open-access papers with a PDF, so most papers here can be found and imported by title without leaving the app. For the AlexNet and Dropout papers, download the free PDF and import it from the Files app. Text extraction and speech run on your device; only searches and downloads go online. Equations and tables may not read cleanly, so check the math on screen.
Free Text to Speech, Right in Your Browser
Paste text or open any file (PDF, EPUB, Word, scans and more) and press Play. SpeakCove reads it aloud with natural voices in 31 languages. It's free, with no sign-up and no length limit, and your text never leaves your browser.
Open the reader →Try SpeakCove Free. No sign-up required. Start listening in seconds.
One-time purchase, no subscription
Free to start with 2 voices and every format. $14.99 once unlocks all 10 voices, background playback and CarPlay. No subscription, ever.
Listen in the car with CarPlay
Your documents on your car's screen and speakers, with CarPlay in Premium.
Works offline, fully on-device
Your documents never leave your device. The app has no analytics and no ads, and reading and voices work in airplane mode.
Frequently Asked Questions
- How do I import ML papers into SpeakCove?
- Use SpeakCove's paper search, which covers arXiv, Semantic Scholar, and OpenAlex, and search by title or keywords. Tap a result to import it; the app extracts the text from the PDF and makes it ready to play. You can also import PDFs directly from the Files app.
- Can TTS handle technical ML papers?
- TTS handles the prose sections of papers well: introductions, related work, method explanations, and conclusions. Mathematical notation and code blocks are less suited to audio, which is why we recommend listening for concepts first and reviewing equations on screen later.
- How long does it take to listen to a typical ML paper?
- Most of these papers are 8 to 15 pages, roughly 30 to 60 minutes of audio at 1x, depending on how many equations and tables you skip. Long survey papers can run 2 hours or more.
- Is SpeakCove free for listening to papers?
- Yes. Paper search, PDF text extraction, playback, speed control, and sentence highlighting are all free. The optional $14.99 one-time Premium purchase adds 8 more voices, background and lock-screen playback, and CarPlay.
- Can I listen to papers offline?
- Yes. Once you import a paper, all text-to-speech processing happens on your device, so you can listen without an internet connection. Searching for new papers needs a connection.
- What order should I read these papers in?
- Chronologically works well: AlexNet (2012), then Word2Vec and DQN (2013), GANs and Dropout (2014), Adam and Batch Normalization (2015), ResNet (2016), Attention Is All You Need (2017), and finally BERT (2019). Each builds on ideas from the earlier ones.
Sources
- Vaswani et al. (2017). Attention Is All You Need. arXiv:1706.03762
- Krizhevsky, Sutskever & Hinton (2012). ImageNet Classification with Deep Convolutional Neural Networks. NeurIPS
- Goodfellow et al. (2014). Generative Adversarial Networks. arXiv:1406.2661
- Devlin et al. (2019). BERT. arXiv:1810.04805 (2018 preprint)
- He et al. (2016). Deep Residual Learning for Image Recognition. arXiv:1512.03385 (2015 preprint)
- Mikolov et al. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv:1301.3781
- Mnih et al. (2013). Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602
- Srivastava et al. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting. JMLR 15
- Kingma & Ba (2015). Adam: A Method for Stochastic Optimization. arXiv:1412.6980 (2014 preprint)
- Ioffe & Szegedy (2015). Batch Normalization. arXiv:1502.03167
- Santurkar et al. (2018). How Does Batch Normalization Help Optimization? arXiv:1805.11604