Connection failure between InterScan Web Security Virtual Appliance (IWSVA) 6.5 and its upstream HTTP proxy

Scenario 1

The upstream HTTP proxy often reuses the same TCP client port for a new connection despite the relevant TCP session being still left in IWSVA. This leads to the upstream HTTP proxy failing to connect to IWSVA.

When the upstream HTTP proxy uses the same TCP client port as the TIME_WAIT state session left in IWSVA and sends TCP SYN to IWSVA, IWSVA regards it as the response for the TIME_WAIT state session and replies TCP ACK (not TCP SYN/ACK). The upstream HTTP proxy gets the ACK reply and sends TCP RCT to IWSVA because the reply is invalid.

However, IWSVA disregards the TCP RCT response according to RFC 1337 by default without closing the TIME_WAIT state session. Therefore, the upstream HTTP proxy continues to reuse the same TCP client port for a new connection until it gives up as TCP 3-way handshake failure.

Workaround

The following makes IWSVA accept TCP RST for the TIME_WAIT session and immediately close the session. As a reference, to increase "IP addresses/TCP ports for TCP client" in the upstream HTTP proxy also resolves the issue.

To accept TCP RST for the TIME_WAIT session:

1. Log on to IWSVA as root via SSH (for example with a SSH shell such as PuTTy).

2. Get a backup copy of /etc/sysctl.conf:

   # cp /etc/sysctl.conf /etc/sysctl.conf.bak

3. Use vi to edit the file /etc/sysctl.conf:

   # vi /etc/sysctl.conf

4. Change the following value:

   net.ipv4.tcp_rfc1337=1 (default)
   TO
   net.ipv4.tcp_rfc1337=0

5. Save the file and quit.

6. Run the following command:

   # sysctl -p

Scenario 2

The load balancer located between the upstream HTTP proxy and IWSVA might change the information of the TCP segments. When this happens, IWSVA's firewall might discard the TCP segments by default when it classifies them as invalid. This leads to the upstream HTTP proxy failing to connect to IWSVA.

Workaround

Add an IWSVA firewall rule so that any TCP segments from the upstream HTTP proxy are accepted.

To add the rule:

1. Log on to IWSVA as root via SSH (for example with a SSH shell such as PuTTy).

2. Get a backup copy of /usr/iwss/iwsvafw.sh

   # cp /usr/iwss/iwsvafw.sh /usr/iwss/iwsvafw.sh.bak

3. Use vi to edit the file /usr/iwss/iwsvafw.sh:

   # vi /usr/iwss/iwsvafw.sh

4. Find the following part:

   ----------------------------------  IPTABLE -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT # stateful  # Create the custom chains
    IPTABLE -N SCAN_SERV_IN  IPTABLE -N ACL_IN   IPTABLE -N LOCAL_SERV_IN  ----------------------------------  

5. Add the following new line under the "# stateful" line:

   ----------------------------------  IPTABLE ipv4 -A INPUT -p tcp -s {IP address of the upstream HTTP proxy} -j ACCEPT  ----------------------------------

   Example: For the upstream HTTP proxy "192.0.2.1"

   ----------------------------------   IPTABLE -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT # stateful  IPTABLE ipv4 -A INPUT -p tcp -s 192.0.2.1 -j ACCEPT  # Create the custom chains  IPTABLE -N SCAN_SERV_IN  IPTABLE -N ACL_IN  IPTABLE -N LOCAL_SERV_IN  ----------------------------------

6. Save the file and quit.

7. Run the following command:

   # /etc/init.d/iptables restart

   The following should be the output if the settings are correct:

   # iptables -nvL INPUT  Chain INPUT (policy DROP 0 packets, 0 bytes)  pkts bytes target     prot opt in     out     source               destination  ...     0     0 ACCEPT     tcp  --  *      *       192.0.2.1            0.0.0.0/0  ...