As with all msys installs whenever you press backspace and there is no more text to delete or you press TAB for auto-complete when there are multiple options, your speaker utters an horrendous, ugly beep.
To disable this, load msys and find out where your home directory is:
Go to that directory (either in Windows or through msys) and create a new file called .inputrc
Put one line in that file:
set bell-style none
Save it and restart msys. You should now have no system bell! I’ve also seen it advised that you can disable your system bell by disabling the hardware emulation for this device in Device Manager, but this didn’t seem to work for me in Vista.