RoPE: How to find the pair index that rotates N times
Finding the pair index whose RoPE frequency makes N rotations over the sequence length.
Finding the pair index whose RoPE frequency makes N rotations over the sequence length.
The 6NP rule is a shortcut for estimating how many floating-point operations are needed to train a dense neural network.
A blog post where I try to give a rough estimate of my codex usage impact.
Walking our way through masked image modeling
A blog post to breakdown the maths behind attention pooling.
A new video from my YouTube channel.
A blog post where I am introducing a simple first approach to describe a given place on Earth using wikidata
A few words on why I am starting a youtube channel
A new video from my YouTube channel.
A new video from my YouTube channel.