For surfers, the special draw would be the waves that break in tubing right-handers; busy days see the waters full of longboarders. Just be aware that locals take their surf seriously, so take turns and don't steal anyone's wave.
Dive boats take off from The Pass; surfers and swimmers are told to keep an eye out to avoid getting in their way – and possibly getting injured.
Swimming appears to be the less common activity at The Pass given the longshore currents that make the task more difficult. Regardless, the beaches are beautiful along the cape and deserve to be enjoyed even if that doesn't involve getting wet.