VMware has confirmed that this issue could happen when downloading and installing the Deep Security Filter Driver, or other vendors' VIB files, on ESXi 5.5.
This issue does not happen on ESXi 5.1 and older versions, because it is caused by a newly-added logic on ESXi 5.5. This new logic processes multiple partner CRLs in /usr/share/certs/vmparter.crl, but it does not clear the “PEM_R_NO_START_LINE” error in the openSSL error queue. This causes inaccurate communication during VIB payload downloading.
VMware will include the fix in ESXi 5.5 Update 2, which will be released in Q3 2014.
While waiting for the fix from VMware, do the following workaround:
- On the DSM, do the following:
- For Windows: Open the Windows command line and change the directory path to C:\Program Files\Trend Micro\Deep Security Manager\. Execute the following command:
dsm_c -action changesetting -name "settings.configuration.filterDriverNoSigCheck" -value true
- For Linux: Log in via SSH and run the following command:
/opt/dsm/dsm_c -action changesetting -name "settings.configuration.filterDriverNoSigCheck" -value true
The DSM service will stop and start again during the process.
- For Windows: Open the Windows command line and change the directory path to C:\Program Files\Trend Micro\Deep Security Manager\. Execute the following command:
- Go to the DSM console and prepare the ESX again.
The Filter Driver installation should now be successful.
- Download FilterDriver-ESX_5.0-9.0.0-2636.x86_64.zip.
- Use the WinSCP tool to upload the file to the /tmp directory of the ESXi host.
- Use Putty to log in to the ESXi host via SSH.
- Go to the /tmp directory and run the following command:
md5sum FilterDriver-ESX_5.0-9.0.0-2636.x86_64.zip
The following MD5 value will appear:
11e199e14e852e3a5da7028176d6a062 FilterDriver-ESX_5.0-9.0.0-2636.x86_64.zip
- Run the command “unzip FilterDriver-ESX_5.0-9.0.0-2636.x86_64.zip”.
- Run the following command:
esxcli software vib install --maintenance-mode -v /tmp/vib20/dvfilter-dsa/Trend_bootbank_dvfilter- dsa_9.0.0-2636.vib
The following message will appear after 20-30 seconds:
Installation Result
Message: The update completed successfully, but the system needs to be rebooted for the changes
to be effective.
Reboot Required: true
VIBs Installed: Trend_bootbank_dvfilter-dsa_9.0.0-2636Do not reboot the ESXi host yet.
- Go to the vCenter and click the host machine.
- Click Configuration > Networking.
- Under Standard Switch (vmservice-vswitch), click Properties > Add.
- Select Virtual Machine then click Next.
- Type "vmservice-trend-pg" as the Network Label.
- Click Next > Finish.
The vmservice-trend-pg port group will be created.
- Reboot the ESXi host.
- Use Putty to log in to the ESXi host again and run the following commands to verify the installation:
~ # esxcli software vib list | grep Trend
dvfilter-dsa 9.0.0-995 Trend VMwareAccepted 2014-03-24
~ # vmkload_mod -l | grep dvfilter
dvfilter 12 144
vmkapi_v2_0_0_0_dvfilter_shim1 8
vmkapi_v2_1_0_0_dvfilter_shim0 8
dvfilter-switch-security 1 192
dvfilter-generic-fastpath0 180
dvfilter-dsa 0 448 - Go to the DSM console and click Synchronize Now in the vCenter.
ESXi will appear "Prepared" and vShield Endpoint will appear "Installed".
- Proceed with the usual DSVA deployment steps.