Emergency services were called to Kurnell National Park just before 12.15pm, after reports that a man was unresponsive after he was pulled from the water off the coast of Inscription Point.
Divers conducted CPR on the man until paramedics arrived however he died at the scene.
The man is yet to be formally identified.
NSW Ambulance Inspector David Matthews said it was “a very sad scene for everyone involved”.
“Our thoughts are with the patient’s friends and family,” Mr Matthews said.
“The bystanders did an incredible job to get the patient to shore, and immediately start CPR. They were remarkable in devastating circumstances.
“As we head into the summer months, it’s important to understand the dangers of the ocean and waterways. It’s imperative to have a safety plan in case something goes wrong.”
A crime scene has been established and a report will be prepared for the information of the coroner.