When the Reject unknown IP address option is enabled, InterScan Messaging Security Virtual Appliance (IMSVA) rejects the email from the domain which does not match the reverse DNS.
Know the message that the sender might receive when an email is declined.
The sender is supposed to receive a non-delivery report (NDR) mail from the upstream MTA.
Take the following mail flow as an example:
Sender MUA (e.g. Outlook) > Sender company's mail server (e.g. Microsoft Exchange) > IMSVA > Recipient's mail server
When IMSVA rejects the SMTP connection because of the Reject unknown IP address option, it will return a "450 error code" to the sender company's mail server. Afterwards, the sender company's mail server is supposed to keep retrying for some time and finally it will generate a non-delivery report mail to the sender.
Please be reminded that when IMSVA rejected a connection, it is NOT IMSVA but the upstream MTA responsible for sending a NDR back to the sender. Depending on the actual mail flow, the upstream might not be the sender company's mail server, but some other third-party MTA in the middle.
For more information about the rule of non-delivery reports, refer to Section 3.6.3 of RFC5321.
3.6.3. Message Submission Servers as Relays
If an SMTP server has accepted the task of relaying the mail and later finds that the destination is incorrect or that the mail cannot be delivered for some other reason, then it MUST construct an "undeliverable mail" notification message and send it to the originator of the undeliverable mail (as indicated by the reverse- path). Formats specified for non-delivery reports by other standards (see, for example, RFC 3461  and RFC 3464 ) SHOULD be used if possible.