Sign In with your
Trend Micro Account
Need Help?
Need More Help?

Create a technical support case if you need further support.

"Out of Connection" and "packet denied" log entries appear in Deep Security

    • Updated:
    • 27 Sep 2016
    • Product/Version:
    • Deep Security 8.0
    • Deep Security 9.0
    • Platform:
    • Windows 2008 Standard 64-bit
Summary

When a firewall rule is modified and the policy is applied to a group of virtual machines (VMs), you see "packet denied" log entries with the reason "Out of Connection". The error comes from a VM that had a policy successfully deployed to it earlier.

Details
Public

Below are the possible reasons why the issue occurs and their corresponding solutions:

When Stateful Inspection is enabled, an "Out of Connection" error occurs if a non-SYN packet is dropped and the existing connection cannot be found in the state table.

Stateful Configuration

When Stateful Inspection is enabled and a change in Deep Security occurs, the Deep Security Agent (DSA) or Deep Security Virtual Appliance (DSVA) attempts to learn about existing connections within the Coldstart Timeout Period. It then tries to set up a state table for established connections according to packets detected.

The coldstart period applies to service restarts and profile changes. It starts when the DSA or DSVA receives the new profile. In the case of multiple VMs, the DSVA has different profiles for each virtual agent and records a coldstart time stamp for each one. By default, this timeout period is 60 seconds in Deep Security 8.0 SP1, and five minutes in SP2 and other Deep Security versions.

If an established connection sits idle for more than the default period after a change, the DSA or DSVA does not get the chance to learn about the connection and its record is not established in the state table. On the other hand, any connection detected within the default period for both TCP/UDP will be added to the state tables and allowed to pass through.

Below are sample scenarios:

In the following scenario, the session is not learned and is not added to the state table. An "Out of Connection" error is raised.

10:00:00 AM - SSH Connection is ESTABLISHED with the server.
10:00:30 AM - SSH Connection is left open but idle
10:01:00 AM - Deep Security Agent is started
10:01:30 AM - Coldstart Timeout Period is reached
10:05:00 AM - SSH Connection is used. "Out of Connection" error in logs.

In the following scenario, the session is learned and no error is raised:

10:00:00 AM - SSH Connection is ESTABLISHED with the server.
10:00:30 AM - SSH Connection is left open but idle
10:01:00 AM - Deep Security Agent is started
10:01:15 AM - SSH Connection is used. Coldstart Timeout has not been reached. Connection added to  state table.
10:01:30 AM - Coldstart Timeout Period is reached
10:05:00 AM - SSH Connection is used. Entry is in state table. Everything is fine.

Recommendation

 
This issue has been fixed in Deep Security 9.5 Service Pack (SP) 1 Patch 3.

Because the environment is receiving "Out of Connection" within the default timeout setting after restarts or policy changes, we recommend increasing the Coldstart timeout period. To change this setting in the Deep Security Manager (DSM) console, go to System Settings > Custom Driver Settings > Cold Start Timeout and adjust the setting. There is no service impact when you change this setting.

The other option is to disable the stateful inspection if it is not a business requirement.

When Deep Security detects abnormal TCP flag and TCP flag synchronizes, these two (2) alerts may appear:

  • Maximum connection reached
  • TCP connection out of memory

If neither of the two conditions above is met, it will show "Out of Connection".

To fix the alert:

  1. On the DSM console, go to Settings > Network Engine.
  2. Increase the value of Established Timeout. By default, its value is three (3) hours.

    Setting the Established Timeout

?
Premium
Internal
Rating:
Category:
Troubleshoot
Solution Id:
1096766
Feedback
Did this article help you?

Thank you for your feedback!

To help us improve the quality of this article, please leave your email here so we can clarify further your feedback, if neccessary:
We will not send you spam or share your email address.

*This form is automated system. General questions, technical, sales, and product-related issues submitted through this form will not be answered.

If you need additional help, you may try to contact the support team. Contact Support


To help us improve the quality of this article, please leave your email here so we can clarify further your feedback, if neccessary:
We will not send you spam or share your email address.

*This form is automated system. General questions, technical, sales, and product-related issues submitted through this form will not be answered.


Need More Help?

Create a technical support case if you need further support.