When I need to do something like this I prebake an array with the values I want.
const static int lookup[256] = { -1, ..., 0,1,2,3,4,5,6,7,8,9, .... };
Then the conversion is easy
int digit_to_int( unsigned char c ) { return lookup[ static_cast<int>(c) ]; }
This is basically the approach taken by many implementations of the ctype library. You can trivially adapt this to work with hex digits too.