What is the difference between SAME and VALID padding in tf nn max pool of tensorflow

Question

What is the difference between  SAME  and  VALID  padding in tf nn max pool of tensorflow   In my opinion   VALID  means there will be no zero padding outside the edges when we do max pool    According to A guide to convolution arithmetic for deep learning  it says that there will be no padding in pool operator  i e  just use  VALID  of tensorflow  But what is  SAME  padding of max pool in tensorflow

User · Answer

Tensorflow 2 0 Compatible Answer  Detailed Explanations have been provided above  about  Valid  and  Same  Padding    However  I will specify different Pooling Functions and their respective Commands in Tensorflow 2 x   gt   2 0   for the benefit of the community   Functions in 1 x   tf nn max pool  tf keras layers MaxPool2D  Average Pooling   gt  None in tf nn  tf keras layers AveragePooling2D  Functions in 2 x   tf nn max pool if used in 2 x and tf compat v1 nn max pool v2 or tf compat v2 nn max pool  if migrated from 1 x to 2 x   tf keras layers MaxPool2D if used in 2 x and   tf compat v1 keras layers MaxPool2D or tf compat v1 keras layers MaxPooling2D or  tf compat v2 keras layers MaxPool2D or tf compat v2 keras layers MaxPooling2D  if migrated from 1 x to 2 x   Average Pooling   gt  tf nn avg pool2d or tf keras layers AveragePooling2D if used in TF 2 x and   tf compat v1 nn avg pool v2 or tf compat v2 nn avg pool or tf compat v1 keras layers AveragePooling2D or tf compat v1 keras layers AvgPool2D or tf compat v2 keras layers AveragePooling2D or tf compat v2 keras layers AvgPool2D   if migrated from 1 x to 2 x   For more information about Migration from Tensorflow 1 x to 2 x  please refer to this Migration Guide

User · Answer

I am quoting this answer from official tensorflow docs https   www tensorflow org api guides python nn Convolution For the  SAME  padding  the output height and width are computed as   out height   ceil float in height    float strides 1    out width    ceil float in width    float strides 2      and the padding on the top and left are computed as   pad along height   max  out height - 1    strides 1                        filter height - in height  0  pad along width   max  out width - 1    strides 2                       filter width - in width  0  pad top   pad along height    2 pad bottom   pad along height - pad top pad left   pad along width    2 pad right   pad along width - pad left   For the  VALID  padding  the output height and width are computed as   out height   ceil float in height - filter height   1    float strides 1    out width    ceil float in width - filter width   1    float strides 2      and the padding values are always zero

User · Answer

Based on the explanation here and following up on Tristan s answer  I usually use these quick functions for sanity checks     a function to help us stay clean def getPaddings pad along height pad along width         if even   easy       if pad along height 2    0          pad top   pad along height   2         pad bottom   pad top       if odd     else          pad top   np floor  pad along height   2           pad bottom   np floor  pad along height   2    1       check if width padding is odd or even       if even   easy       if pad along width 2    0          pad left   pad along width   2         pad right  pad left       if odd     else          pad left   np floor  pad along width   2           pad right   np floor  pad along width   2    1               return pad top pad bottom pad left pad right    strides  image index  y  x  depth    padding  SAME  or  VALID    bottom and right sides always get the one additional padded pixel  if padding is odd  def getOutputDim  inputWidth inputHeight filterWidth filterHeight strides padding       if padding     SAME           out height   np ceil float inputHeight    float strides 1            out width    np ceil float inputWidth    float strides 2                      pad along height     out height - 1    strides 1    filterHeight - inputHeight          pad along width     out width - 1    strides 2    filterWidth - inputWidth                      now get padding         pad top pad bottom pad left pad right   getPaddings pad along height pad along width                    print  output height   out height         print  output width    out width         print  total pad along height    pad along height         print  total pad along width    pad along width         print  pad at top    pad top         print  pad at bottom   pad bottom         print  pad at left    pad left         print  pad at right   pad right      elif padding     VALID           out height   np ceil float inputHeight - filterHeight   1    float strides 1            out width    np ceil float inputWidth - filterWidth   1    float strides 2                      print  output height   out height         print  output width    out width         print  no padding      use like so getOutputDim  80 80 4 4  1 1 1 1   SAME

User · Answer

Quick Explanation  VALID  Don t apply any padding  i e   assume that all dimensions are valid so that input image fully gets covered by filter and stride you specified   SAME  Apply padding to input  if needed  so that input image gets fully covered by filter and stride you specified  For stride 1  this will ensure that output image size is same as input   Notes   This applies to conv layers as well as max pool layers in same way The term  valid  is bit of a misnomer because things don t become  invalid  if you drop part of the image  Sometime you might even want that  This should have probably be called NO PADDING instead  The term  same  is a misnomer too because it only makes sense for stride of 1 when output dimension is same as input dimension  For stride of 2  output dimensions will be half  for example  This should have probably be called AUTO PADDING instead  In SAME  i e  auto-pad mode   Tensorflow will try to spread padding evenly on both left and right  In VALID  i e  no padding mode   Tensorflow will drop right and or bottom cells if your filter and stride doesn t full cover input image

User · Answer

There are three choices of padding  valid  no padding   same  or half   full  You can find explanations  in Theano  here  http   deeplearning net software theano tutorial conv arithmetic html   Valid or no padding    The valid padding involves no zero padding  so it covers only the valid input  not including artificially generated zeros  The length of output is   the length of input  -  k-1   for the kernel size k if the stride s 1    Same or half padding    The same padding makes the size of outputs be the same with that of inputs when s 1  If s 1  the number of zeros padded is  k-1     Full padding    The full padding means that the kernel runs over the whole inputs  so at the ends  the kernel may meet the only one input and zeros else  The number of zeros padded is 2 k-1  if s 1  The length of output is   the length of input     k-1   if s 1   Therefore  the number of paddings   valid   lt    same   lt    full

User · Answer

Padding is an operation to increase the size of the input data  In case of 1-dimensional data you just append prepend the array with a constant  in 2-dim you surround matrix with these constants  In n-dim you surround your n-dim hypercube with the constant  In most of the cases this constant is zero and it is called zero-padding   Here is an example of zero-padding with p 1 applied to 2-d tensor      You can use arbitrary padding for your kernel but some of the padding values are used more frequently than others they are    VALID padding  The easiest case  means no padding at all  Just leave your data the same it was  SAME padding sometimes called HALF padding  It is called SAME because for a convolution with a stride 1   or for pooling  it should produce output of the same size as the input  It is called HALF because for a kernel of size k  FULL padding is the maximum padding which does not result in a convolution over just padded elements  For a kernel of size k  this padding is equal to k - 1      To use arbitrary padding in TF  you can use tf pad

User · Answer

Here  W and H are width and height of input        F are filter dimensions         P is padding size  i e   number of rows or columns to be padded   For SAME padding      For VALID padding

User · Answer

Complementing YvesgereY s great answer  I found this visualization extremely helpful   Padding  valid  is the first figure  The filter window stays inside the image  Padding  same  is the third figure  The output is the same size   Found it on this article Visualization credits  vdumoulin GitHub

User · Answer

Padding on off  Determines the effective size of your input  VALID  No padding  Convolution etc  ops are only performed at locations that are  valid   i e  not too close to the borders of your tensor  With a kernel of 3x3 and image of 10x10  you would be performing convolution on the 8x8 area inside the borders  SAME  Padding is provided  Whenever your operation references a neighborhood  no matter how big   zero values are provided when that neighborhood extends outside the original tensor to allow that operation to work also on border values  With a kernel of 3x3 and image of 10x10  you would be performing convolution on the full 10x10 area

User · Answer

When stride is 1  more typical with convolution than pooling   we can think of the following distinction     SAME   output size is the same as input size  This requires the filter window to slip outside input map  hence the need to pad    VALID   Filter window stays at valid position inside input map  so output size shrinks by filter size - 1  No padding occurs

User · Answer

If you like ascii art     VALID    without padding      inputs          1  2  3  4  5  6  7  8  9  10 11  12 13                                                      dropped                                                        SAME    with zero padding                  pad                                        pad    inputs       0  1  2  3  4  5  6  7  8  9  10 11 12 13 0  0                                                                                                                                                        In this example    Input width   13 Filter width   6 Stride   5   Notes     VALID  only ever drops the right-most columns  or bottom-most rows    SAME  tries to pad evenly left and right  but if the amount of columns to be added is odd  it will add the extra column to the right  as is the case in this example  the same logic applies vertically  there may be an extra row of zeros at the bottom     Edit   About the name    With  SAME  padding  if you use a stride of 1  the layer s outputs will have the same spatial dimensions as its inputs  With  VALID  padding  there s no  made-up  padding inputs  The layer only uses valid input data

User · Answer

VALID padding  this is with zero padding  Hope there is no confusion    x   tf constant   1   2   3     4   5   6     7   8   9      7   8   9     x   tf reshape x   1  4  3  1   valid pad   tf nn max pool x   1  2  2  1    1  2  2  1   padding  VALID   print  valid pad get shape      output-- gt  1  2  1  1    SAME  padding  This is kind of tricky to understand in the first place because we have to consider two conditions separately as mentioned in the official docs    Let s take input as    output as   padding as    stride as  and kernel size as   only a single dimension is considered   Case 01      Case 02         is calculated such that the minimum value which can be taken for padding  Since value of  is known  value of  can be found using this formula     Let s work out this example   x   tf constant   1   2   3     4   5   6     7   8   9      7   8   9     x   tf reshape x   1  4  3  1   same pad   tf nn max pool x   1  2  2  1    1  2  2  1   padding  SAME   print  same pad get shape      -- gt  output  1  2  2  1    Here the dimension of x is  3 4   Then if the horizontal direction is taken  3      If the vertial direction is taken  4      Hope this will help to understand how actually SAME padding works in TF

User · Answer

I ll give an example to make it clearer    x  input image of shape  2  3   1 channel valid pad  max pool with 2x2 kernel  stride 2 and VALID padding  same pad  max pool with 2x2 kernel  stride 2 and SAME padding  this is the classic way to go    The output shapes are    valid pad  here  no padding so the output shape is  1  1  same pad  here  we pad the image to the shape  2  4   with -inf and then apply max pool   so the output shape is  1  2      x   tf constant   1   2   3                      4   5   6      x   tf reshape x   1  2  3  1      give a shape accepted by tf nn max pool  valid pad   tf nn max pool x   1  2  2  1    1  2  2  1   padding  VALID   same pad   tf nn max pool x   1  2  2  1    1  2  2  1   padding  SAME    valid pad get shape       1  1  1  1     valid pad is  5   same pad get shape       1  1  2  1      same pad is   5   6

User · Answer

The TensorFlow Convolution example gives an overview about the difference between SAME and VALID     For the SAME padding  the output height and width are computed as   out height   ceil float in height    float strides 1    out width    ceil float in width    float strides 2       And   For the VALID padding  the output height and width are computed as   out height   ceil float in height - filter height   1    float strides 1    out width    ceil float in width - filter width   1    float strides 2

User · Answer

To sum up   valid  padding means no padding  The output size of the convolutional layer shrinks depending on the input size  amp  kernel size    On the contrary   same  padding means using padding  When the stride is set as 1  the output size of the convolutional layer maintains as the input size by appending a certain number of  0-border  around the input data when calculating convolution   Hope this intuitive description helps

[python] What is the difference between 'SAME' and 'VALID' padding in tf.nn.max_pool of tensorflow?

Examples related to python

Examples related to tensorflow

Examples related to deep-learning