Pytorch reshape tensor dimension

Question

For example  I have 1D vector with dimension  5   I would like to reshape it into 2D matrix  1 5    Here is how I do it with numpy   gt  gt  gt  import numpy as np  gt  gt  gt  a   np array  1 2 3 4 5    gt  gt  gt  a shape  5    gt  gt  gt  a   np reshape a   1 5    gt  gt  gt  a shape  1  5   gt  gt  gt  a array   1  2  3  4  5     gt  gt  gt     But how can I do that with Pytorch Tensor  and Variable   I don t want to switch back to numpy and switch to Torch variable again  because it will loss backpropagation information   Here is what I have in Pytorch   gt  gt  gt  import torch  gt  gt  gt  from torch autograd import Variable  gt  gt  gt  a   torch Tensor  1 2 3 4 5    gt  gt  gt  a   1  2  3  4  5  torch FloatTensor of size 5    gt  gt  gt  a size    5L    gt  gt  gt  a var   variable a   gt  gt  gt  a var   Variable a   gt  gt  gt  a var size    5L        do some calculation in forward function  gt  gt  gt  a var size    5L     Now I want it size to be  1  5   How can I resize or reshape the dimension of pytorch tensor in Variable without loss grad information   because I will feed into another model before backward

User · Answer

For in-place modification of the shape of the tensor  you should use   tensor resize      In  23   a   torch Tensor  1  2  3  4  5    In  24   a shape Out 24   torch Size  5       tensor resize    new shape        In  25   a resize   1 5   Out 25      1  2  3  4  5  torch FloatTensor of size 1x5   In  26   a shape Out 26   torch Size  1  5     In PyTorch  if there s an underscore at the end of an operation  like tensor resize     then that operation does in-place modification to the original tensor     Also  you can simply use np newaxis in a torch Tensor to increase the dimension  Here is an example   In  34   list    range 5  In  35   a   torch Tensor list   In  36   a shape Out 36   torch Size  5    In  37   new a   a np newaxis     In  38   new a shape Out 38   torch Size  1  5

User · Answer

or you can use this  the  -1  means you don t have to specify the number of the elements   In  3   a view 1 -1  Out 3     1  2  3  4  5  torch FloatTensor of size 1x5

User · Answer

torch reshape   is made to dupe the numpy reshape method  It came after the view   and torch resize    and it is inside the dir torch  package  import torch x torch arange 24  print x  x shape  x view   x view 1 2 3 4    works on is contiguous   tensor print x view shape  x reshaped   x reshape 1 2 3 4    works on any tensor print x reshaped shape  x reshaped2   torch reshape x reshaped   -1      part of torch package  while view   and resize    are not print x reshaped2 shape   Out  tensor   0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17          18  19  20  21  22  23   torch Size  24   torch Size  1  2  3  4   torch Size  1  2  3  4   torch Size  24    But did you know it can also work as a replacement for squeeze   and unsqueeze   x   torch tensor  1  2  3  4   print x shape  x1   torch unsqueeze x  0  print x1 shape  x2   torch unsqueeze x1  1  print x2 shape  x3 x reshape 1 1 4  print x3 shape  x4 x reshape 4  print x4 shape  x5 x3 squeeze   print x5 shape    Out  torch Size  4   torch Size  1  4   torch Size  1  1  4   torch Size  1  1  4   torch Size  4   torch Size  4

User · Answer

you might use  a view 1 5  Out     1  2  3  4  5  torch FloatTensor of size 1x5

User · Answer

There are multiple ways of reshaping a PyTorch tensor  You can apply these methods on a tensor of any dimensionality   Let s start with a 2-dimensional 2 x 3 tensor   x   torch Tensor 2  3  print x shape    torch Size  2  3     To add some robustness to this problem  let s reshape the 2 x 3 tensor by adding a new dimension at the front and another dimension in the middle  producing a 1 x 2 x 1 x 3 tensor   Approach 1  add dimension with None  Use NumPy-style insertion of None  aka np newaxis  to add dimensions anywhere you want  See here   print x shape    torch Size  2  3    y   x None     None       Add new dimensions at positions 0 and 2  print y shape    torch Size  1  2  1  3     Approach 2  unsqueeze  Use torch Tensor unsqueeze i   a k a  torch unsqueeze tensor  i  or the in-place version unsqueeze     to add a new dimension at the i th dimension  The returned tensor shares the same data as the original tensor  In this example  we can use unqueeze   twice to add the two new dimensions    print x shape    torch Size  2  3      Use unsqueeze twice  y   x unsqueeze 0    Add new dimension at position 0 print y shape    torch Size  1  2  3    y   y unsqueeze 2    Add new dimension at position 2 print y shape    torch Size  1  2  1  3     In practice with PyTorch  adding an extra dimension for the batch may be important  so you may often see unsqueeze 0    Approach 3  view  Use torch Tensor view  shape  to specify all the dimensions  The returned tensor shares the same data as the original tensor   print x shape    torch Size  2  3    y   x view 1  2  1  3  print y shape    torch Size  1  2  1  3     Approach 4  reshape  Use torch Tensor reshape  shape   aka torch reshape tensor  shapetuple   to specify all the dimensions  If the original data is contiguous and has the same stride  the returned tensor will be a view of input  sharing the same data   otherwise it will be a copy  This function is similar to the NumPy reshape   function in that it lets you define all the dimensions and can return either a view or a copy   print x shape    torch Size  2  3    y   x reshape 1  2  1  3  print y shape    torch Size  1  2  1  3     Furthermore  from the O Reilly 2019 book Programming PyTorch for Deep Learning  the author writes   Now you might wonder what the difference is between view   and reshape    The answer is that view   operates as a view on the original tensor  so if the underlying data is changed  the view will change too  and vice versa   However  view   can throw errors if the required view is not contiguous  that is  it doesn   t share the same block of memory it would occupy if a new tensor of the required shape was created from scratch  If this happens  you have to call tensor contiguous   before you can use view    However  reshape   does all that behind the scenes  so in general  I recommend using reshape   rather than view     Approach 5  resize   Use the in-place function torch Tensor resize   sizes  to modify the original tensor  The documentation states   WARNING  This is a low-level method  The storage is reinterpreted as C-contiguous  ignoring the current strides  unless the target size equals the current size  in which case the tensor is left unchanged   For most purposes  you will instead want to use view    which checks for contiguity  or reshape    which copies data if needed  To change the size in-place with custom strides  see set      print x shape    torch Size  2  3    x resize  1  2  1  3  print x shape    torch Size  1  2  1  3     My observations  If you want to add just one dimension  e g  to add a 0th dimension for the batch   then use unsqueeze 0   If you want to totally change the dimensionality  use reshape     See also   What s the difference between reshape and view in pytorch   What is the difference between view   and unsqueeze     In PyTorch 0 4  is it recommended to use reshape than view when it is possible

User · Answer

Use  torch unsqueeze input  dim  out None    gt  gt  gt  import torch  gt  gt  gt  a   torch Tensor  1 2 3 4 5    gt  gt  gt  a   1  2  3  4  5  torch FloatTensor of size 5    gt  gt  gt  a   a unsqueeze 0   gt  gt  gt  a   1  2  3  4  5  torch FloatTensor of size 1x5

User · Answer

This question has been thoroughly answered already  but I want to add for the less experienced python developers that you might find the   operator helpful in conjunction with view     For example if you have a particular tensor size that you want a different tensor of data to conform to  you might try   img   Variable tensor randn 20 30 3     tensor with goal shape flat size   20 30 3 X   Variable tensor randn 50  flat size     data tensor  X   X view -1   img size      sweet maneuver print X size      size is  50  20  30  3    This works with numpy shape too   img   np random randn 20 30 3  flat size   20 30 3 X   Variable tensor randn 50  flat size   X   X view -1   img shape  print X size      size is  50  20  30  3

User · Answer

import torch  gt  gt  gt a   torch Tensor  1 2 3 4 5    gt  gt  gt a size   torch Size  5    use view to reshape   gt  gt  gt b   a view 1 a shape 0    gt  gt  gt b tensor   1   2   3   4   5      gt  gt  gt b size   torch Size  1  5    gt  gt  gt b type    torch FloatTensor

User · Answer

import torch t   torch ones  2  3  4   t size      gt  gt torch Size  2  3  4     a   t view -1 t size   1  t size   2   a size      gt  gt torch Size  2  12

User · Answer

Assume the following code   import torch import numpy as np a   torch tensor  1  2  3  4  5     The following three calls have the exact same effect   res 1   a unsqueeze 0  res 2   a view 1  5  res 3   a np newaxis    res 1 shape    res 2 shape    res 3 shape     1 5     Returns true   Notice that for any of the resulting tensors  if you modify the data in them  you are also modifying the data in a  because they don t have a copy of the data  but reference the original data in a   res 1 0 0    2 a 0     res 1 0 0     2    Returns true   The other way of doing it would be using the resize  in place operation   a shape    res 1 shape    Returns false a reshape   1  5   a shape    res 1 shape   Returns true   Be careful of using resize  or other in-place operation with autograd   See the following discussion  https   pytorch org docs stable notes autograd html in-place-operations-with-autograd

[python] Pytorch reshape tensor dimension

Examples related to python

Examples related to numpy

Examples related to deep-learning

Examples related to pytorch

Examples related to tensor