[linux] How to encrypt a large file in openssl using public key

How can I encrypt a large file with a public key so that no one other than who has the private key be able to decrypt it?

I can make RSA public and private keys but when it comes to encrypting a large file using this command:

openssl rsautl -encrypt -pubin -inkey public.pem -in myLargeFile.xml -out myLargeFile_encrypted.xml

and how can i perform the decryption also....

i create my private and public key by the following commands

openssl genrsa -out private.pem 1024
openssl rsa -in private.pem -out public.pem -outform PEM -pubout

I get this error:

RSA operation error
3020:error:0406D06E:rsa routines:RSA_padding_add_PKCS1_type_2:data too large for key size:.\crypto\rsa\rsa_pk1.c:151:

I tried to make keys with sizes from 1024 to 1200 bits, no luck, same error

Solution for safe and high secured encode anyone file in OpenSSL and command-line:

You should have ready some X.509 certificate for encrypt files in PEM format.

Encrypt file:

openssl smime -encrypt -binary -aes-256-cbc -in plainfile.zip -out encrypted.zip.enc -outform DER yourSslCertificate.pem

What is what:

• smime - ssl command for S/MIME utility (smime(1))
• -encrypt - chosen method for file process
• -binary - use safe file process. Normally the input message is converted to "canonical" format as required by the S/MIME specification, this switch disable it. It is necessary for all binary files (like a images, sounds, ZIP archives).
• -aes-256-cbc - chosen cipher AES in 256 bit for encryption (strong). If not specified 40 bit RC2 is used (very weak). (Supported ciphers)
• -in plainfile.zip - input file name
• -out encrypted.zip.enc - output file name
• -outform DER - encode output file as binary. If is not specified, file is encoded by base64 and file size will be increased by 30%.
• yourSslCertificate.pem - file name of your certificate's. That should be in PEM format.

That command can very effectively a strongly encrypt big files regardless of its format.
Known issue: Something wrong happens when you try encrypt huge file (>600MB). No error thrown, but encrypted file will be corrupted. Always verify each file! (or use PGP - that has bigger support for files encryption with public key)

Decrypt file:

openssl smime -decrypt -binary -in encrypted.zip.enc -inform DER -out decrypted.zip -inkey private.key -passin pass:your_password

What is what:

• -inform DER - same as -outform above
• -inkey private.key - file name of your private key. That should be in PEM format and can be encrypted by password.
• -passin pass:your_password - your password for private key encrypt. (passphrase arguments)

Maybe you should check out the accepted answer to this (How to encrypt data in php using Public/Private keys?) question.

Instead of manually working around the message size limitation (or perhaps a trait) of RSA, it shows how to use the S/mime feature of OpenSSL to do the same thing and not needing to juggle with the symmetric key manually.

I found the instructions at http://www.czeskis.com/random/openssl-encrypt-file.html useful.

To paraphrase the linked site with filenames from your example:

Generate a symmetric key because you can encrypt large files with it

openssl rand -base64 32 > key.bin

Encrypt the large file using the symmetric key

openssl enc -aes-256-cbc -salt -in myLargeFile.xml \
  -out myLargeFile.xml.enc -pass file:./key.bin

Encrypt the symmetric key so you can safely send it to the other person

openssl rsautl -encrypt -inkey public.pem -pubin -in key.bin -out key.bin.enc

Destroy the un-encrypted symmetric key so nobody finds it

shred -u key.bin

At this point, you send the encrypted symmetric key (key.bin.enc) and the encrypted large file (myLargeFile.xml.enc) to the other person

The other person can then decrypt the symmetric key with their private key using

openssl rsautl -decrypt -inkey private.pem -in key.bin.enc -out key.bin

Now they can use the symmetric key to decrypt the file

openssl enc -d -aes-256-cbc -in myLargeFile.xml.enc \
  -out myLargeFile.xml -pass file:./key.bin

And you're done. The other person has the decrypted file and it was safely sent.

Encrypting a very large file using smime is not advised since you might be able to encrypt large files using the -stream option, but not decrypt the resulting file due to hardware limitations see: problem decrypting big files

As mentioned above Public-key crypto is not for encrypting arbitrarily long files. Therefore the following commands will generate a pass phrase, encrypt the file using symmetric encryption and then encrypt the pass phrase using the asymmetric (public key). Note: the smime includes the use of a primary public key and a backup key to encrypt the pass phrase. A backup public/private key pair would be prudent.

Random Password Generation

Set up the RANDFILE value to a file accessible by the current user, generate the passwd.txt file and clean up the settings

export OLD_RANDFILE=$RANDFILE
RANDFILE=~/rand1
openssl rand -base64 2048 > passwd.txt
rm ~/rand1
export RANDFILE=$OLD_RANDFILE

Encryption

Use the commands below to encrypt the file using the passwd.txt contents as the password and AES256 to a base64 (-a option) file. Encrypt the passwd.txt using asymetric encryption into the file XXLarge.crypt.pass using a primary public key and a backup key.

openssl enc -aes-256-cbc -a -salt -in XXLarge.data -out XXLarge.crypt -pass file:passwd.txt
openssl smime -encrypt -binary -in passwd.txt -out XXLarge.crypt.pass -aes256 PublicKey1.pem PublicBackupKey.pem
rm passwd.txt

Decryption

Decryption simply decrypts the XXLarge.crypt.pass to passwd.tmp, decrypts the XXLarge.crypt to XXLarge2.data, and deletes the passwd.tmp file.

openssl smime -decrypt -binary -in XXLarge.crypt.pass -out passwd.tmp -aes256 -recip PublicKey1.pem -inkey PublicKey1.key
openssl enc -d -aes-256-cbc -a -in XXLarge.crypt -out XXLarge2.data -pass file:passwd.tmp
rm passwd.tmp

This has been tested against >5GB files..

5365295400 Nov 17 10:07 XXLarge.data
7265504220 Nov 17 10:03 XXLarge.crypt
      5673 Nov 17 10:03 XXLarge.crypt.pass
5365295400 Nov 17 10:07 XXLarge2.data

In more explanation for n. 'pronouns' m.'s answer,

Public-key crypto is not for encrypting arbitrarily long files. One uses a symmetric cipher (say AES) to do the normal encryption. Each time a new random symmetric key is generated, used, and then encrypted with the RSA cipher (public key). The ciphertext together with the encrypted symmetric key is transferred to the recipient. The recipient decrypts the symmetric key using his private key, and then uses the symmetric key to decrypt the message.

There is the flow of Encryption:

+---------------------+      +--------------------+
|                     |      |                    |
| generate random key |      |   the large file   |
|        (R)          |      |        (F)         |
|                     |      |                    |
+--------+--------+---+      +----------+---------+
         |        |                     |
         |        +------------------+  |
         |                           |  |
         v                           v  v
+--------+------------+     +--------+--+------------+
|                     |     |                        |
| encrypt (R) with    |     | encrypt (F)            |
| your RSA public key |     | with symmetric key (R) |
|                     |     |                        |
|  ASym(PublicKey, R) |     |     EF = Sym(F, R)     |
|                     |     |                        |
+----------+----------+     +------------+-----------+
           |                             |
           +------------+ +--------------+
                        | |
                        v v
         +--------------+-+---------------+
         |                                |
         |   send this files to the peer  |
         |                                |
         |     ASym(PublicKey, R) + EF    |
         |                                |
         +--------------------------------+

And the flow of Decryption:

   +----------------+        +--------------------+
   |                |        |                    |
   | EF = Sym(F, R) |        | ASym(PublicKey, R) |
   |                |        |                    |
   +-----+----------+        +---------+----------+
         |                             |
         |                             |
         |                             v
         |   +-------------------------+-----------------+
         |   |                                           |
         |   |             restore key (R)               |
         |   |                                           |
         |   | R <= ASym(PrivateKey, ASym(PublicKey, R)) |
         |   |                                           |
         |   +---------------------+---------------------+
         |                         |
         v                         v
     +---+-------------------------+---+
     |                                 |
     |       restore the file (F)      |
     |                                 |
     |      F <= Sym(Sym(F, R), R)     |
     |                                 |
     +---------------------------------+

Besides, you can use this commands:

# generate random symmetric key
openssl rand -base64 32 > /config/key.bin

# encryption
openssl rsautl -encrypt -pubin -inkey /config/public_key.pem -in /config/key.bin -out /config/key.bin.enc
openssl aes-256-cbc -a -pbkdf2 -salt -in  $file_name -out $file_name.enc -k $(cat /config/key.bin)

# now you can send this files: $file_name.enc + /config/key.bin.enc

# decryption
openssl rsautl -decrypt -inkey /config/private_key.pem -in /config/key.bin.enc -out /config/key.bin
openssl aes-256-cbc -d -a -in $file_name.enc -out $file_name -k $(cat /config/key.bin)

To safely encrypt large files (>600MB) with openssl smime you'll have to split each file into small chunks:

# Splits large file into 500MB pieces
split -b 500M -d -a 4 INPUT_FILE_NAME input.part.

# Encrypts each piece
find -maxdepth 1 -type f -name 'input.part.*' | sort | xargs -I % openssl smime -encrypt -binary -aes-256-cbc -in % -out %.enc -outform DER PUBLIC_PEM_FILE

For the sake of information, here is how to decrypt and put all pieces together:

# Decrypts each piece
find -maxdepth 1 -type f -name 'input.part.*.enc' | sort | xargs -I % openssl smime -decrypt -in % -binary -inform DEM -inkey PRIVATE_PEM_FILE -out %.dec

# Puts all together again
find -maxdepth 1 -type f -name 'input.part.*.dec' | sort | xargs cat > RESTORED_FILE_NAME

You can't directly encrypt a large file using rsautl. instead, do something like the following:

1. Generate a key using openssl rand, eg. openssl rand 32 -out keyfile
2. Encrypt the key file using openssl rsautl
3. Encrypt the data using openssl enc, using the generated key from step 1.
4. Package the encrypted key file with the encrypted data. the recipient will need to decrypt the key with their private key, then decrypt the data with the resulting key.