To resolve this issue, the following steps may be done:

1. Check the SG-SPS version under Trend Vision One console:

   

   ^{Click the image to enlarge.}

    

   • If your SPS version is already 2.0.1, then it should work well.
   • If your SPS version is 2.0.0 then please try to upgrade it on the Cross Detection and Response (XDR) console.

    
2. Usually this upgrade will finish within 30 minutes. If the upgrade does not finish, please login through the SG backend by following this KB article: How to log in to Service Gateway with root permission in Trend Vision One
3. Check the SG running status with command “kubectl get pods -A“;, if the command returns the pods, then wait for the upgrade to finish. This is OK. A manual fix of the issue is needed, if the output displays similar to the following:

   "internal error, please report: running "microk8s.kubectl" failed: cannot create transient scope: DBus error "System.Error.ENOSPC": [No space left on device]"
   	

4. If a manual fix is needed, execute the following command to delete the files:

   cd /run/systemd/propagatem;
   rm -rf systemd-coredump@10* &&rm -rf systemd-coredump@11* && rm -rf systemd-coredump@12* && rm -rf systemd-coredump@13* && rm -rf systemd-coredump@14* && rm -rf systemd-coredump@15* && rm -rf systemd-coredump@16* && rm -rf systemd-coredump@1*;
   rm -rf systemd-coredump@20* &&rm -rf systemd-coredump@21* && rm -rf systemd-coredump@22* && rm -rf systemd-coredump@23* && rm -rf systemd-coredump@24* && rm -rf systemd-coredump@25* && rm -rf systemd-coredump@26* && rm -rf systemd-coredump@2*;
   rm -rf systemd-coredump@30* &&rm -rf systemd-coredump@31* && rm -rf systemd-coredump@32* && rm -rf systemd-coredump@33* && rm -rf systemd-coredump@34* && rm -rf systemd-coredump@35* && rm -rf systemd-coredump@36* && rm -rf systemd-coredump@3*;
   rm -rf systemd-coredump@40* &&rm -rf systemd-coredump@41* && rm -rf systemd-coredump@42* && rm -rf systemd-coredump@43* && rm -rf systemd-coredump@44* && rm -rf systemd-coredump@45* && rm -rf systemd-coredump@46* && rm -rf systemd-coredump@4*;
   rm -rf systemd-coredump@50* &&rm -rf systemd-coredump@51* && rm -rf systemd-coredump@52* && rm -rf systemd-coredump@53* && rm -rf systemd-coredump@54* && rm -rf systemd-coredump@55* && rm -rf systemd-coredump@56* && rm -rf systemd-coredump@5*;
   rm -rf systemd-coredump@60* &&rm -rf systemd-coredump@61* && rm -rf systemd-coredump@62* && rm -rf systemd-coredump@63* && rm -rf systemd-coredump@64* && rm -rf systemd-coredump@65* && rm -rf systemd-coredump@66* && rm -rf systemd-coredump@6*;
   rm -rf systemd-coredump@70* &&rm -rf systemd-coredump@71* && rm -rf systemd-coredump@72* && rm -rf systemd-coredump@73* && rm -rf systemd-coredump@74* && rm -rf systemd-coredump@75* && rm -rf systemd-coredump@76* && rm -rf systemd-coredump@7*;
   rm -rf systemd-coredump@80* &&rm -rf systemd-coredump@81* && rm -rf systemd-coredump@82* && rm -rf systemd-coredump@83* && rm -rf systemd-coredump@84* && rm -rf systemd-coredump@85* && rm -rf systemd-coredump@86* && rm -rf systemd-coredump@8*;
   rm -rf systemd-coredump@90* &&rm -rf systemd-coredump@91* && rm -rf systemd-coredump@92* && rm -rf systemd-coredump@93* && rm -rf systemd-coredump@94* && rm -rf systemd-coredump@95* && rm -rf systemd-coredump@96* && rm -rf systemd-coredump@9*;

5. After executing the command above, check the SG status again with the command “kubectl get pods -A“;. This should now be working. You can now return to the XDR console and upgrade SG-SPS.
6. If after executing the commands in Step 4 and Step 5 the issue still persists, then execute the command “microk8s inspect“ and ““journactl -u snap.microk8s.daemon-k8s-dqlite.service” to check the service status. You may get the following output:

   fatal msg="Failed to start server: start node: raft_start(): io: load closed segment 0000000014232389-0000000014232597: entries batch 394 starting at byte 8384000: entries count in preamble is zero\n"
   		

7. According to the output message above, you have to recover the microk8s service. To do this, use the following command:

   microk8s stop;
   rm /var/snap/microk8s/current/var/kubernetes/backend/0000000014232389-0000000014232597;
   microk8s start;
   		

    

   These files were taken from Step 6.

    
8. Once this is done, restart microK8s and then check the status using the command “kubectl get pods -A“;. You should be able to upgrade SG-SPS from Trend Vision One console again.