Read any paper
like it’s from your field.

The hard terms explained before you reach them. Any passage, in plain words. Every citation a click away. All on the original paper, never a summary.

orupload a PDF(up to 50 MB and 100 pages)

3.2.1 Scaled Dot-Product Attentionfrom Attention Is All You Need, 2017

We compute the of the with all keys, divide each by √d_k, and apply a function to obtain the weights on the values.

Click a highlighted term. Your paper gets the same treatment.

Twenty tabs, or one desk.

A tab for every term, then a tab for every term in that tab. By page three, the paper is lost. PaperParker brings the answers to the page.

Attention Is All You Need.pdfsoftmax functionwhat is a query vector?layer normalization, explainedBahdanau 2014.pdfpositional encoding (video)residual connectionswhy divide by sqrt(d_k)?an AI chat about the paperbeam searchlabel smoothingBLEU scoreencoder-decoderbyte-pair encodingAdam optimizerdropoutSutskever 2014.pdfmulti-head attentionwarmup stepswhat is a logit?

arxiv.org/pdf/1706.03762

Attention Is All You Need

How it works

[1]
Bring a paper
An arXiv link, a DOI or your own PDF.
[2]
We read it first
Hard terms marked, cards ready, references linked.
[3]
You read on your desk
Everything opens beside the line it belongs to.

Terms explained before you reach them.

Every highlight opens a card: plain or technical, what this paper does with it, what to know first. Stuck on a whole sentence? Select it and hit Explain this.

3.2.1 Scaled Dot-Product Attention

While for small values of dₖ the two mechanisms perform similarly, additive attention outperforms dot product attention without scaling for larger values of dₖ. We suspect that for large values of dₖ, the dot products grow large in magnitude, pushing the softmax function into regions where it has extremely small gradients. To counteract this effect, we scale the dot products by 1/⁠√dₖ.

Explanations you can check.

Every card points to the page and the exact sentence behind it. The paper always has the last word.

Term in this paper

scaled dot-product attention

In this paper

The attention used in every layer of the Transformer. p. 3 The scores are divided by √d_k to keep the softmax from saturating. p. 4

Read more on Wikipedia: Attention (machine learning)

Report a problem with this card

Hover p. 4 and the sentence lights up on the page.

While for small values of d_k the two mechanisms perform similarly, additive attention outperforms dot product attention without scaling for larger values of d_k. We suspect that for large values of d_k, the dot products grow large in magnitude, pushing the softmax function into regions where it has extremely small gradients. To counteract this effect, we scale the dot products by 1/⁠√d_k.

Follow the literature without losing your place.

See what each citation names and why it’s there. Open any reference on its own desk, then find your way back.

Attention Is All You Need
Vaswani and others, 2017
Why it’s cited: Additive attention, which this paper weighs its own attention against.
Neural machine translation by jointly learning to align and translate
Bahdanau, Cho and Bengio, 2014
Why it’s cited: The encoder–decoder model this paper sets out to improve.
Sequence to sequence learning with neural networks
Sutskever, Vinyals and Le, 2014

Your notes, on the line they’re about.

Highlight, comment, draw, clip a figure, mark what you already know. Your thinking stays where it happened.

Attention Is All You Need

Vaswani, Shazeer, Parmar and others, 2017

3.2 Attention

An attention function can be described as mapping a

query and a set of key-value pairs to an output, where

the query, keys, values, and output are all vectors.

3.2.1 Scaled Dot-Product Attention

The two most commonly used attention functions are

dot-product (multiplicative) and additive attention [2].

Dot-product attention is identical to our algorithm,

except for the scaling factor of 1/√d_k.

3.2.2 Multi-Head Attention

Instead of performing a single attention function, we

found it beneficial to linearly project the queries,

keys and values h times. Multi-head attention

allows the model to jointly attend to information

from different representation subspaces at different

positions.

So it is a soft dictionary lookup: the query decides which values count, and by how much.

MatMulSoftMaxMask (opt.)ScaleMatMulQKV

Figure 2, clipped from page 4

Term in this paper

multi-head attention

Several attention functions run side by side, each on its own projection of the input, and their outputs are joined.

In this paper

Eight heads, each working in 64 dimensions.

Reference [2]

Neural machine translation by jointly learning to align and translate

Bahdanau, Cho and Bengio, 2014

PreviewOpen as new canvas

Who it’s for

Papers are written for people already in the field. PaperParker is for everyone else who reads them.

Students
Forty papers on the reading list, in a field you met last month.
Start Simple, go Technical as you learn, and keep every note where you wrote it.
Researchers
An expert in your field. A newcomer next door.
Technical cards, Explain this for the derivations, and the references a click away.
Anyone curious
You read the headline. Now read the paper.
Simple cards and Explain this take you as far as your curiosity goes.

Appendix A

Questions you might have

Is this a chatbot?: No prompts to write. The explanations are ready before you start, and Explain this handles anything you select.
Who writes the explanations?: A language model, held to the paper: every card shows the page and sentence it rests on, and anything off can be reported in a click.
Which papers can I open?: Anything on arXiv, open-access DOIs, or any PDF up to 50 MB and 100 pages. English, for now.
What about paywalled papers?: Upload the PDF you have. It works the same.
Will my notes be there tomorrow?: Yes. Your desk saves as you go, in this browser.
Does it work on a phone?: It shines on a laptop or desktop, where the paper and its explanations sit side by side.

References

[1]
Your next paper (forthcoming).
arXiv link or DOI
orupload a PDF(up to 50 MB and 100 pages)
[2]
Parker, P. (2026). PaperParker: read any paper like it’s from your field. paperparker.com

Read any paperlike it’s from your field.

Twenty tabs, or one desk.

How it works

Bring a paper

We read it first

You read on your desk