From the logs above, the following applications using most of the available memory are as follows:

• TrustedInstaller.exe (59793940484 bytes or 5.9GB RAM)
• svchost.exe (1608499200 bytes or 1.6GB RAM)

The DBServer.exe only needs 551MB (551055360 bytes) of memory to load, which the server failed to allocate for the said service.

To solve this issue, you may apply either or both of the following options to resolve this issue:

• For the TrustedInstaller.exe that constantly uses 80% of RAM, help from the MS Support Team is needed to identify which parent application is causing this process to load. There is a similar issue in the web as shown in the link below stating that it may be related to a virtual desktop application such as Citrix:

  Service TrustedInstaller.EXE permanently active

• Increase the VM's virtual memory. This approach is easier to do. Perhaps you can do this as an interim solution to be able to bring the OfficeScan server online while waiting for the MS Support Team to help us resolve the issue with the TrustedInstaller.exe high memory consumption.
• You may also attempt to reboot the VM Image and observe if you can reclaim available memory to allocate for the DBServer.exe.