Hope this will help someone.
-> open to keyboard shortcut
-> search for "workbench.action.togglePanelPosition"
-> assign your desired shortcut
I've assigned keybinding "cmd+`"
{
"key": "cmd+`",
"command": "workbench.action.togglePanelPosition"
}
now I can toggle the terminal by pressing "cmd + `"