With current versions of Docker, you can use the --device
flag to achieve what you want, without needing to give access to all USB devices.
For example, if you wanted to make only /dev/ttyUSB0
accessible within your Docker container, you could do something like:
docker run -t -i --device=/dev/ttyUSB0 ubuntu bash