You get to this window by going to the Services -> SMTP page in the VPOP3 settings.

The Require SMTP Authentication option tells VPOP3 that the email clients must use SMTP authentication before they can send email messages through VPOP3. If this option is enabled and you receive your incoming messages using SMTP, you must have the Do not require SMTP Authentication for internal/incoming mail option enabled as well, otherwise it will prevent you receiving messages! Note that some email clients do not yet support SMTP authentication, so check that the ones you use do support this facility, before you enable this option in VPOP3.
The Require "POP3 then SMTP" authentication option tells VPOP3 only to allow people to send mail if they have checked their VPOP3 POP3 mailbox just before trying to send mail. The Timeout setting related to this setting tells VPOP3 for how long after someone has checked their mail it will allow them to start sending mail. For instance, if it is set to '5 minutes', VPOP3 will allow them to start sending mail any time from them successfully logging onto the VPOP3 POP3 server to 5 minutes after they have logged off. Once they have started sending mail, VPOP3 will allow them to continue sending mail until their email client disconnects from the VPOP3 SMTP server.
If you have both Require SMTP Authentication and Require POP3 then SMTP authentication enabled, then VPOP3 will accept either authentication method.
If you turn on Do not require SMTP Authentication for internal mail, then VPOP3 will only require the above two authentication methods to send mail to external users. Messages to internal users will be allowed even if the sender did not authenticate first.
The SMTP Anti-Relay Protection options allow you to restrict who can send outgoing messages through VPOP3. This is most useful if VPOP3 has a permanent connection to the Internet as it can be used to stop email SPAMMERS sending their junk email through VPOP3. See here for more information.
The Refuse SMTP Connections From ... option tells VPOP3 to refuse incoming SMTP connections from certain mail clients. By default VPOP3 refuses SMTP connections from '<anything>.mail.demon.net'. This is because we recommend using the POP3 service offered by Demon Internet instead of their SMTP service, because there is much more functionality that way. If you want to use Demon Internet's SMTP service, then clear this box - and make sure that you do not connect to Demon's POP3 server as well! (You can only specify a single name here. VPOP3 checks the HELO SMTP command sent by the SMTP client and checks the this string inside the HELO data string - if it matches, then VPOP3 disconnects the SMTP client). Note that VPOP3 will try to set this field automatically.
If you set a Maximum Message Size: this makes VPOP3 reject any messages over this size. If individual users have a maximum outgoing message size, the size limit is the most restrictive of that setting and this one. [Tech Info]
The Reject unrecognised local recipients tells VPOP3 to refuse to accept messages for addresses in the Local Domains which are not recognised. [Tech Info]
If you enable Don't allow addresses with % in, VPOP3 will not allow messages to be sent to the VPOP3 SMTP server which contain a '%' character. The '%' character is used for SMTP forwarding, and it is sometimes used by junk emailers to bypass security and is rarely, if ever, used for legitimate purposes, so usually this option should be enabled.
The Edit Realtime Blacklist Rules button lets you define the use of anti-spam database to filter incoming SMTP mail. This is only useful if your incoming SMTP mail comes directly from the sender, rather than going through your ISP. See here for more information. This facility is only available with VPOP3 Enterprise.
Access Restrictions are used to limit who can use the VPOP3 SMTP server. You can restrict access to a certain range of IP addresses. The format of the restriction definitions is described here.
SMTP Rules let you define rules for messages sent through VPOP3 or received using the SMTP protocol. See here for a description of SMTP Rules.
If you enable the Load SMTP Rules on each connection option, VPOP3 will load the SMTP Rules file each time the SMTP Service accepts a new incoming connection. This will slow down performance, but makes it easier to test SMTP rule definitions.
The Edit SMTP Rules button takes you to an editor where you can create/modify SMTP Rules. The rules will automatically be reloaded after this.
The Reload SMTP Rules button tells VPOP3 to reload the SMTP Rules file after it has been modified from outside VPOP3.