Skip to main content

Troubleshoot Endpoint Detection and Response backups and Ransomware Rollback in Nebula

The Ransomware Rollback feature in Endpoint Detection and Response (EDR) allows you to revert file changes made by malware or ransomware on Windows endpoints. This article provides a deeper understanding of the EDR backup and remediation solution and how to troubleshoot it.

Select one of the following topics:

Checking for the EDR Service

To ensure EDR backups are being created, check that the EDR service is running on an endpoint:

  • From Nebula, verify the Endpoint Detection and Response component is listed under Agent Information in the Endpoint Overview.
  • From the endpoint, perform one of the following:
    • Hold control and right-click the system tray icon on the endpoint and click About. The Endpoint Detection and Response plugin is displayed if it is installed.
    • In cmd prompt, run the following command to check if the service is running:
      • SC flightrecorder
    • In PowerShell, run the following command to check if the service is running:
      • Get-Service -Name flightrecorder

Backups

Before a file is modified or a registry entry is changed, a backup is made in the local folder C:\ProgramData\Malwarebytes Endpoint Agent\Plugins\EDRPlugin\Backups\. For servers, this path can be modified to a separate drive. For more information, see Configure Endpoint Detection and Response options in Nebula.

Notes

  • The folder and contents are self-protected by Malwarebytes drivers against attack, preventing even local administrators from deleting the folder.
  • Each backup is encrypted to avoid interference and scanning by other security products. Exclude this folder from other security products to avoid unwanted detections and false positives. For more information, see Network access requirements and firewall settings for Nebula.
  • All file types can be backed up (docs, xls, json, xml, exe, dll, etc.)
    • There is a 14-day self-learning process. After that period, for space and performance optimization, backups are ignored by trusted processes. For example:
      • A document edited by Word.exe would be ignored
      • Backups would occur for a document edited by an untrusted or malicious process
  • Files are named like 0000001670324876267_2D7E74B2.frb.
    • The first part of the name contains the backup time with a Unix timestamp. (0000001670324876267 = Tue Dec 06 2022 22:07:56). For information on converting the timestamp, see https://www.unixtimestamp.com/.
    • The second part of the name (2D7E74B2) is randomly generated.
    • The extensions are either Flight Recorder Backup (FRB) and Flight Recorder Backup Registry (FRBR).
    • The file's datetime viewed by Windows, is the creation date of the original or source file, not the backup time.
  • Backups are to a local drive for very fast recovery.
    Note: Offline backups are still required to cover cases of hard drive or SSD crash, theft of device, and for files ignored by Ransomware Rollback.

Remediation and Ransomware Rollback

A remediation can be triggered for any suspicious activity alert. When a remediation is triggered, a scan is run to clean the identified processes.

Additionally, remediating a [Ransomware] suspicious activity alert automatically begins the ransomware rollback process.

The rollback uses the processes identified in the alert, then restores files modified by that process by overwriting files changed with the prior copies. This design takes away the need to discover the exact date and time of the start of the attack.

Self cleaning

The backups perform self cleaning to meet the thresholds set in the Endpoint Detection and Response policy configuration.

A task runs every 10 minutes to check the disk quota and delete old backups or unindexed files if the quota is exceeded. This activity is logged in C:\ProgramData\Malwarebytes Endpoint Agent\Logs\EndpointAgent.txt.

The following is an example of successful cleanups.

INFO FRCoreManager [FRSDK] Running cleanup ALL. RollbackTTL: 72 LearningMode: 2
INFO FRCoreManager [FRSDK] FR cleanup ALL started. Cleaning up events older than 2022-11-28 17:22:34+1100. Current backup files total number: 1579 and disk stats size/free/usage/quota/quota%: 84880125952/1781522432/537416136/695681570/30%
INFO FRCoreManager [FRSDK] FR cleanup ALL finished. Deleted 11574 backup events and 298 backup files. New backup files total number: 1281 and disk size/free/usage/quota/quota%: 84880125952/1902772224/390548912/687996340/30%
INFO FRCoreManager [FRSDK] Next backup cleanup scheduled for 2022-12-02 17:22:33+1100
INFO FRCoreManager Checking for orphaned backup files under "C:\ProgramData\Malwarebytes Endpoint Agent\Plugins\EDRPlugin\Backup"
INFO FRCoreManager Finished checking for orphaned backup files under "C:\ProgramData\Malwarebytes Endpoint Agent\Plugins\EDRPlugin\Backup"

Backup folder exceeds configured quota

In scenarios where the backup folder is larger than expected, complete the following:

  1. Check the Windows Services and verify the MBEndpointAgent service is running, as this service controls cleaning.
  2. Enable debug logging. For more information, see Enable debug logging on the Malwarebytes Endpoint Agent.
  3. Check the C:\ProgramData\Malwarebytes Endpoint Agent\Logs\EndpointAgent.txt for the following:
    • The self cleaning process is running every 10 minutes. 
      DEBUG FRCoreManager FRCallbackWrapper -- Received Message: Disk size/free/usage/quota/quota%: 510823661568/404060053504/476145280/121360859635/30%
      DEBUG FRCoreManager FRCallbackWrapper -- Received Message: Disk size/free/usage/quota/quota%: 510823661568/404060053504/476145280/121360859635/30%
    • Any log messages containing ERROR FRCoreManager.
  4. Contact Support

Manual cleanup of backup folder

In case it is urgent to free up disk space, complete the following:

  1. Collect diagnostic logs. For more information, see Collect Malwarebytes Endpoint Agent diagnostic logs.
  2. Report the issue to Support.
  3. Clean up the backup folder with one of the following methods:
    • Disable and re-enable EDR in the policy settings.
      1. Create a Policy with all EDR policy settings disabled.
        For more information, see Configure Endpoint Detection and Response options in Nebula.
      2. Create a Group and assign with the new policy assigned.
      3. Move the affected endpoints into that group. This will force the EDR plugin to unload and clean up. A reboot may be required.
      4. Move the affected endpoints back to previous group and EDR will reinstall.
    • Uninstall and reinstall Malwarebytes.

Contacting Support

When submitting a support case, the following information is required:

  • Endpoint names.
  • Malwarebytes Diagnostic logs. For more information, see Collect Malwarebytes Endpoint Agent diagnostic logs.
    • If you are unable to collect the Malwarebytes Diagnostic logs, manually obtain the following files:
      • C:\ProgramData\Malwarebytes Endpoint Agent\Logs\EndpointAgent.txt
      • C:\ProgramData\Malwarebytes Endpoint Agent\Plugins\EDRPlugin\Database\2B455663142B495843A6F3DCB6B55CCE
    •  

Related articles