> For the complete documentation index, see [llms.txt](https://ykkim.gitbook.io/dlip/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ykkim.gitbook.io/dlip/programming/pytorch/autograd-torch.md).

# Autograd in PyTorch

## AUTOGRAD: AUTOMATIC DIFFERENTIATION

Central to all neural networks in PyTorch is the `autograd` package.

* provides automatic differentiation for all operations on Tensors.
* Generally speaking, `torch.autograd` is an engine for computing vector-Jacobian product.

  `torch.Tensor` is the central class of the package.
* `.requires_grad` as `True`, starts to track all operations on it
  * you can call `.backward()` and have all the gradients computed automatically.
  * accumulated into `.grad` attribute.
* `with torch.no_grad():` to prevent tracking history (and using memory)
* `.backward()` compute the derivatives,
* `.grad_fn` attribute references a `Function`

### Autograd Tutorial

[Pytorch Autograd\_tutorial](https://github.com/ykkimhgu/gitbook_docs/blob/master/deep-learning-framework/pytorch/autograd_tutorial_ykk.ipynb)