[assembly] Assembly - JG/JNLE/JL/JNGE after CMP

Addition and subtraction in two's complement is the same for signed and unsigned numbers

The key observation is that CMP is basically subtraction, and:

In two's complement (integer representation used by x86), signed and unsigned addition are exactly the same operation

This allows for example hardware developers to implement it more efficiently with just one circuit.

So when you give input bytes to the x86 ADD instruction for example, it does not care if they are signed or not.

However, ADD does set a few flags depending on what happened during the operation:

  • carry: unsigned addition or subtraction result does not fit in bit size, e.g.: 0xFF + 0x01 or 0x00 - 0x01

    For addition, we would need to carry 1 to the next level.

  • sign: result has top bit set. I.e.: is negative if interpreted as signed.

  • overflow: input top bits are both 0 and 0 or 1 and 1 and output inverted is the opposite.

    I.e. signed operation changed sigedness in an impossible way (e.g. positive + positive or negative

We can then interpret those flags in a way that makes comparison match our expectations for signed or unsigned numbers.

This interpretation is exactly what JA vs JG and JB vs JL do for us!

Code example

Here is GNU GAS a code snippet to make this more concrete:

/* 0x0 ==
 *
 * * 0 in 2's complement signed
 * * 0 in 2's complement unsigned
 */
mov $0, %al

/* 0xFF ==
 *
 * *  -1 in 2's complement signed
 * * 255 in 2's complement unsigned
 */
mov $0xFF, %bl

/* Do the operation "Is al < bl?" */
cmp %bl, %al

Note that AT&T syntax is "backwards": mov src, dst. So you have to mentally reverse the operands for the condition codes to make sense with cmp. In Intel syntax, this would be cmp al, bl

After this point, the following jumps would be taken:

  • JB, because 0 < 255
  • JNA, because !(0 > 255)
  • JNL, because !(0 < -1)
  • JG, because 0 > -1

Note how in this particular example the signedness mattered, e.g. JB is taken but not JL.

Runnable example with assertions.

Equals / Negated versions like JLE / JNG are just aliases

By looking at the Intel 64 and IA-32 Architectures Software Developer's Manuals Volume 2 section "Jcc - Jump if Condition Is Met" we see that the encodings are identical, for example:

Opcode  Instruction  Description
7E cb   JLE rel8     Jump short if less or equal (ZF=1 or SF ? OF).
7E cb   JNG rel8     Jump short if not greater (ZF=1 or SF ? OF).