OK - I see the problem then.
Some not logged in users should be able to see the calendar and the content and some not logged in users should not be able to see the calendar.
And that is why you can't use any of the settings that could restrict access.
Then we are back to some change of code or a setting for the mail. I think only Roel can give the answer to that.
Even if the logical solution would be to restrict public access when public access to the calendar should not be possible, I see the problem with all the persons that would then need to make logins. And I see the problem for the people that have none or little knowledge to IT.