How to correct TypeError Unicode-objects must be encoded before hashing

Question

I have this error   Traceback  most recent call last     File  python md5 cracker py   line 27  in  lt module gt    m update line  TypeError  Unicode-objects must be encoded before hashing   when I try to execute this code in Python 3 2 2   import hashlib  sys m   hashlib md5   hash      hash file   input  What is the file name in which the hash resides      wordlist   input  What is your wordlist    Enter the file name      try    hashdocument   open hash file   r   except IOError    print  Invalid file      raw input     sys exit   else    hash   hashdocument readline     hash   hash replace   n        try    wordlistfile   open wordlist   r   except IOError    print  Invalid file      raw input     sys exit   else    pass for line in wordlistfile      Flush the buffer  this caused a massive problem when placed      at the beginning of the script  because the buffer kept getting     overwritten  thus comparing incorrect hashes    m   hashlib md5     line   line replace   n         m update line    word hash   m hexdigest     if word hash    hash      print  Collision  The word corresponding to the given hash is   line      input       sys exit    print  The hash given does not correspond to any supplied word in the wordlist    input   sys exit

User · Answer

This program is the bug free and enhanced version of the above MD5 cracker that reads the file containing list of hashed passwords and checks it against hashed word from the English dictionary word list. Hope it is helpful.

I downloaded the English dictionary from the following link https://github.com/dwyl/english-words

# md5cracker.py
# English Dictionary https://github.com/dwyl/english-words 

import hashlib, sys

hash_file = 'exercise\hashed.txt'
wordlist = 'data_sets\english_dictionary\words.txt'

try:
    hashdocument = open(hash_file,'r')
except IOError:
    print('Invalid file.')
    sys.exit()
else:
    count = 0
    for hash in hashdocument:
        hash = hash.rstrip('\n')
        print(hash)
        i = 0
        with open(wordlist,'r') as wordlistfile:
            for word in wordlistfile:
                m = hashlib.md5()
                word = word.rstrip('\n')            
                m.update(word.encode('utf-8'))
                word_hash = m.hexdigest()
                if word_hash==hash:
                    print('The word, hash combination is ' + word + ',' + hash)
                    count += 1
                    break
                i += 1
        print('Itiration is ' + str(i))
    if count == 0:
        print('The hash given does not correspond to any supplied word in the wordlist.')
    else:
        print('Total passwords identified is: ' + str(count))
sys.exit()

User · Answer

Please take a look first at that answer   Now  the error message is clear  you can only use bytes  not Python strings  what used to be unicode in Python  lt  3   so you have to encode the strings with your preferred encoding  utf-32  utf-16  utf-8 or even one of the restricted 8-bit encodings  what some might call codepages    The bytes in your wordlist file are being automatically decoded to Unicode by Python 3 as you read from the file  I suggest you do   m update line encode wordlistfile encoding     so that the encoded data pushed to the md5 algorithm are encoded exactly like the underlying file

User · Answer

You could open the file in binary mode   import hashlib  with open hash file  as file      control hash   file readline   rstrip   n    wordlistfile   open wordlist   rb         for line in wordlistfile      if hashlib md5 line rstrip b  n r    hexdigest      control hash           collision

User · Answer

You must have to define encoding format like utf-8   Try this easy way   This example generates a random number using the SHA256 algorithm    gt  gt  gt  import hashlib  gt  gt  gt  hashlib sha256 str random getrandbits 256   encode  utf-8    hexdigest    cd183a211ed2434eac4f31b317c573c50e6c24e3a28b82ddcb0bf8bedf387a9f

User · Answer

encoding this line fixed it for me   m update line encode  utf-8

User · Answer

If it s a single line string  wrapt it with b or B  e g   variable   b This is a variable    or   variable2   B This is also a variable

User · Answer

import hashlib string to hash    123  hash object   hashlib sha256 str string to hash  encode  utf-8    print  Hash   hash object hexdigest

User · Answer

The error already says what you have to do  MD5 operates on bytes  so you have to encode Unicode string into bytes  e g  with line encode  utf-8

User · Answer

To store the password  PY3    import hashlib  os password salt   os urandom 32  hex   password    12345   hash   hashlib sha512   hash update    s s     password salt  password   encode  utf-8    password hash   hash hexdigest

User · Answer

It is probably looking for a character encoding from wordlistfile  wordlistfile   open wordlist  quot r quot  encoding  utf-8    Or  if you re working on a line-by-line basis  line encode  utf-8     EDIT Per the comment below and this answer  My answer above assumes that the desired output is a str from the wordlist file  If you are comfortable in working in bytes  then you re better off using open wordlist   quot rb quot    But it is important to remember that your hashfile should NOT use rb if you are comparing it to the output of hexdigest  hashlib md5 value  hashdigest   outputs a str and that cannot be directly compared with a bytes object   abc     b abc    There s a lot more to this topic  but I don t have the time ATM   It should also be noted that this line  line replace  quot  n quot    quot  quot    Should probably be line strip    That will work for both bytes and str s  But if you decide to simply convert to bytes  then you can change the line to  line replace b quot  n quot   b quot  quot

[python] How to correct TypeError: Unicode-objects must be encoded before hashing?

Examples related to python

Examples related to python-3.x

Examples related to unicode

Examples related to syntax-error

Examples related to hashlib