"Use SMTP to send system mails" should work with default smtp settings
If you check the "Use SMTP to send system mails" checkbox and leave the other smtp settings the same, password reset emails stop working and there is no error/indication when this happens. Using 'localhost' as a server name in particular should be allowed, as that is the default.
https://www.howtoforge.com/community/threads/password-reset-email-not-sent.84349/