[c] How to do encryption using AES in Openssl

I am trying to write a sample program to do AES encryption using Openssl. I tried going through Openssl documentation( it's a pain), could not figure out much. I went through the code and found the API's using which i wrote a small program as below (please omit the line numbers). I don't see any encryption happening... am i missing something?

PS: I don't get any errors upon compilation.

  1 #include <stdio.h> 
  2 #include <openssl/aes.h>   
  3 
  4 static const unsigned char key[] = {
  5   0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77,
  6     0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff,
  7       0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
  8         0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f
  9         };
 10 
 11 void main()
 12 {
 13     unsigned char text[]="virident";
 14     unsigned char out[10]; 
 15     unsigned char decout[10];
 16 
 17     AES_KEY wctx;
 18 
 19     AES_set_encrypt_key(key, 128, &wctx);
 20     AES_encrypt(text, out, &wctx);  
 21 
 22     printf("encryp data = %s\n", out);
 23     
 24     AES_decrypt(out, decout, &wctx);
 25     printf(" Decrypted o/p: %s \n", decout);
 26 
 27 
 28 }

Please help me to figure this out...

Check out this link it has a example code to encrypt/decrypt data using AES256CBC using EVP API.

https://github.com/saju/misc/blob/master/misc/openssl_aes.c

Also you can check the use of AES256 CBC in a detailed open source project developed by me at https://github.com/llubu/mpro

The code is detailed enough with comments and if you still need much explanation about the API itself i suggest check out this book Network Security with OpenSSL by Viega/Messier/Chandra (google it you will easily find a pdf of this..) read chapter 6 which is specific to symmetric ciphers using EVP API.. This helped me a lot actually understanding the reasons behind using various functions and structures of EVP.

and if you want to dive deep into the Openssl crypto library, i suggest download the code from the openssl website (the version installed on your machine) and then look in the implementation of EVP and aeh api implementation.

One more suggestion from the code you posted above i see you are using the api from aes.h instead use EVP. Check out the reason for doing this here OpenSSL using EVP vs. algorithm API for symmetric crypto nicely explained by Daniel in one of the question asked by me..

I don't know what's wrong with yours but one thing for sure is you need to call AES_set_decrypt_key() before decrypting the message. Also don't try to print out as %s because the encrypted message isn't composed by ascii characters anymore.. For example:

static const unsigned char key[] = {
    0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77,
    0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff,
    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
    0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f
};

int main()
{
    unsigned char text[]="hello world!";
    unsigned char enc_out[80];
    unsigned char dec_out[80];

    AES_KEY enc_key, dec_key;

    AES_set_encrypt_key(key, 128, &enc_key);
    AES_encrypt(text, enc_out, &enc_key);      

    AES_set_decrypt_key(key,128,&dec_key);
    AES_decrypt(enc_out, dec_out, &dec_key);

    int i;

    printf("original:\t");
    for(i=0;*(text+i)!=0x00;i++)
        printf("%X ",*(text+i));
    printf("\nencrypted:\t");
    for(i=0;*(enc_out+i)!=0x00;i++)
        printf("%X ",*(enc_out+i));
    printf("\ndecrypted:\t");
    for(i=0;*(dec_out+i)!=0x00;i++)
        printf("%X ",*(dec_out+i));
    printf("\n");

    return 0;
} 

U1: your key is 192 bit isn't it...

I am trying to write a sample program to do AES encryption using Openssl.

This answer is kind of popular, so I'm going to offer something more up-to-date since OpenSSL added some modes of operation that will probably help you.

First, don't use AES_encrypt and AES_decrypt. They are low level and harder to use. Additionally, it's a software-only routine, and it will never use hardware acceleration, like AES-NI. Finally, its subject to endianess issues on some obscure platforms.

Instead, use the EVP_* interfaces. The EVP_* functions use hardware acceleration, like AES-NI, if available. And it does not suffer endianess issues on obscure platforms.

Second, you can use a mode like CBC, but the ciphertext will lack integrity and authenticity assurances. So you usually want a mode like EAX, CCM, or GCM. (Or you manually have to apply a HMAC after the encryption under a separate key.)

Third, OpenSSL has a wiki page that will probably interest you: EVP Authenticated Encryption and Decryption. It uses GCM mode.

Finally, here's the program to encrypt using AES/GCM. The OpenSSL wiki example is based on it.

#include <openssl/evp.h>
#include <openssl/aes.h>
#include <openssl/err.h>
#include <string.h>   

int main(int arc, char *argv[])
{
    OpenSSL_add_all_algorithms();
    ERR_load_crypto_strings();     

    /* Set up the key and iv. Do I need to say to not hard code these in a real application? :-) */

    /* A 256 bit key */
    static const unsigned char key[] = "01234567890123456789012345678901";

    /* A 128 bit IV */
    static const unsigned char iv[] = "0123456789012345";

    /* Message to be encrypted */
    unsigned char plaintext[] = "The quick brown fox jumps over the lazy dog";

    /* Some additional data to be authenticated */
    static const unsigned char aad[] = "Some AAD data";

    /* Buffer for ciphertext. Ensure the buffer is long enough for the
     * ciphertext which may be longer than the plaintext, dependant on the
     * algorithm and mode
     */
    unsigned char ciphertext[128];

    /* Buffer for the decrypted text */
    unsigned char decryptedtext[128];

    /* Buffer for the tag */
    unsigned char tag[16];

    int decryptedtext_len = 0, ciphertext_len = 0;

    /* Encrypt the plaintext */
    ciphertext_len = encrypt(plaintext, strlen(plaintext), aad, strlen(aad), key, iv, ciphertext, tag);

    /* Do something useful with the ciphertext here */
    printf("Ciphertext is:\n");
    BIO_dump_fp(stdout, ciphertext, ciphertext_len);
    printf("Tag is:\n");
    BIO_dump_fp(stdout, tag, 14);

    /* Mess with stuff */
    /* ciphertext[0] ^= 1; */
    /* tag[0] ^= 1; */

    /* Decrypt the ciphertext */
    decryptedtext_len = decrypt(ciphertext, ciphertext_len, aad, strlen(aad), tag, key, iv, decryptedtext);

    if(decryptedtext_len < 0)
    {
        /* Verify error */
        printf("Decrypted text failed to verify\n");
    }
    else
    {
        /* Add a NULL terminator. We are expecting printable text */
        decryptedtext[decryptedtext_len] = '\0';

        /* Show the decrypted text */
        printf("Decrypted text is:\n");
        printf("%s\n", decryptedtext);
    }

    /* Remove error strings */
    ERR_free_strings();

    return 0;
}

void handleErrors(void)
{
    unsigned long errCode;

    printf("An error occurred\n");
    while(errCode = ERR_get_error())
    {
        char *err = ERR_error_string(errCode, NULL);
        printf("%s\n", err);
    }
    abort();
}

int encrypt(unsigned char *plaintext, int plaintext_len, unsigned char *aad,
            int aad_len, unsigned char *key, unsigned char *iv,
            unsigned char *ciphertext, unsigned char *tag)
{
    EVP_CIPHER_CTX *ctx = NULL;
    int len = 0, ciphertext_len = 0;

    /* Create and initialise the context */
    if(!(ctx = EVP_CIPHER_CTX_new())) handleErrors();

    /* Initialise the encryption operation. */
    if(1 != EVP_EncryptInit_ex(ctx, EVP_aes_256_gcm(), NULL, NULL, NULL))
        handleErrors();

    /* Set IV length if default 12 bytes (96 bits) is not appropriate */
    if(1 != EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, 16, NULL))
        handleErrors();

    /* Initialise key and IV */
    if(1 != EVP_EncryptInit_ex(ctx, NULL, NULL, key, iv)) handleErrors();

    /* Provide any AAD data. This can be called zero or more times as
     * required
     */
    if(aad && aad_len > 0)
    {
        if(1 != EVP_EncryptUpdate(ctx, NULL, &len, aad, aad_len))
            handleErrors();
    }

    /* Provide the message to be encrypted, and obtain the encrypted output.
     * EVP_EncryptUpdate can be called multiple times if necessary
     */
    if(plaintext)
    {
        if(1 != EVP_EncryptUpdate(ctx, ciphertext, &len, plaintext, plaintext_len))
            handleErrors();

        ciphertext_len = len;
    }

    /* Finalise the encryption. Normally ciphertext bytes may be written at
     * this stage, but this does not occur in GCM mode
     */
    if(1 != EVP_EncryptFinal_ex(ctx, ciphertext + len, &len)) handleErrors();
    ciphertext_len += len;

    /* Get the tag */
    if(1 != EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_GET_TAG, 16, tag))
        handleErrors();

    /* Clean up */
    EVP_CIPHER_CTX_free(ctx);

    return ciphertext_len;
}

int decrypt(unsigned char *ciphertext, int ciphertext_len, unsigned char *aad,
            int aad_len, unsigned char *tag, unsigned char *key, unsigned char *iv,
            unsigned char *plaintext)
{
    EVP_CIPHER_CTX *ctx = NULL;
    int len = 0, plaintext_len = 0, ret;

    /* Create and initialise the context */
    if(!(ctx = EVP_CIPHER_CTX_new())) handleErrors();

    /* Initialise the decryption operation. */
    if(!EVP_DecryptInit_ex(ctx, EVP_aes_256_gcm(), NULL, NULL, NULL))
        handleErrors();

    /* Set IV length. Not necessary if this is 12 bytes (96 bits) */
    if(!EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, 16, NULL))
        handleErrors();

    /* Initialise key and IV */
    if(!EVP_DecryptInit_ex(ctx, NULL, NULL, key, iv)) handleErrors();

    /* Provide any AAD data. This can be called zero or more times as
     * required
     */
    if(aad && aad_len > 0)
    {
        if(!EVP_DecryptUpdate(ctx, NULL, &len, aad, aad_len))
            handleErrors();
    }

    /* Provide the message to be decrypted, and obtain the plaintext output.
     * EVP_DecryptUpdate can be called multiple times if necessary
     */
    if(ciphertext)
    {
        if(!EVP_DecryptUpdate(ctx, plaintext, &len, ciphertext, ciphertext_len))
            handleErrors();

        plaintext_len = len;
    }

    /* Set expected tag value. Works in OpenSSL 1.0.1d and later */
    if(!EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_TAG, 16, tag))
        handleErrors();

    /* Finalise the decryption. A positive return value indicates success,
     * anything else is a failure - the plaintext is not trustworthy.
     */
    ret = EVP_DecryptFinal_ex(ctx, plaintext + len, &len);

    /* Clean up */
    EVP_CIPHER_CTX_free(ctx);

    if(ret > 0)
    {
        /* Success */
        plaintext_len += len;
        return plaintext_len;
    }
    else
    {
        /* Verify failed */
        return -1;
    }
}

My suggestion is to run

openssl enc -aes-256-cbc -in plain.txt -out encrypted.bin

under debugger and see what exactly what it is doing. openssl.c is the only real tutorial/getting started/reference guide OpenSSL has. All other documentation is just an API reference.

U1: My guess is that you are not setting some other required options, like mode of operation (padding).

U2: this is probably a duplicate of this question: AES CTR 256 Encryption Mode of operation on OpenSSL and answers there will likely help.