Skip to main content

Troubleshoot Endpoint Detection and Response backups and Ransomware Rollback in Nebula

The Ransomware Rollback feature in Endpoint Detection and Response (EDR) allows you to revert file changes made by malware or ransomware on Windows endpoints. This article provides a deeper understanding of the EDR backup and remediation solution and how to troubleshoot it.

Select one of the following topics:

Checking for the EDR Service

To ensure EDR backups are being created, check that the EDR plugin service is running:

  • In the console.
    1. Go to Manage > Endpoints.
    2. Click on an affected endpoint.
    3. Click See more details.
    4. Check the Agent and plugins section for the following:
      1. Presence of Endpoint Detection and Response plugin.
      2. Date listed for Agent info last refreshed is the current day.
    5. Example: 
      Agent and plugins
      Agent info last refreshed: 06/23/2023 10:50:09 AM*
      Agent version: 1.2.0.1048
      Endpoint Detection and Response: 1.2.0.387
  • On the endpoint.
    • Check the About screen.
      1. Hold control and right-click the system tray icon on the endpoint and click About.
      2. Verify the Endpoint Detection and Response version in the list.
    • Check with command prompt.
      1. Open command prompt.
      2. Run the following commands.
        1. SC QUERY MBEndpointAgent
        2. SC QUERY flightrecorder
    • Check with Powershell
      1. Open PowerShell.
      2. Run the following command.
        • Get-Service -Name flightrecorder,MBEndpointAgent.

Backups

Before a file is modified or a registry entry is changed, a backup is made in the following folder:

  • C:\ProgramData\Malwarebytes Endpoint Agent\Plugins\EDRPlugin\Backups\

For servers, this path can be modified to a separate drive. For more information, see Configure Endpoint Detection and Response options in Nebula.

Notes

  • The folder and contents are self-protected by Malwarebytes drivers against attack, preventing even local administrators from deleting the folder.
  • Each backup is encrypted to avoid interference and scanning by other security products.
  • Exclude this folder from other security products to avoid unwanted detections and false positives. For more information, see Network access requirements and firewall settings for Nebula.
  • All file types can be backed up (docs, xls, json, xml, exe, dll, etc.)
    • There is a 14-day self-learning process. After that period, for space and performance optimization, backups are ignored for trusted processes. For example:
      • A document edited by Word.exe would be ignored
      • Backups would occur for a document edited by an untrusted or malicious process
  • Files are named like 0000001670324876267_2D7E74B2.frb.
    • The first part of the name contains the backup time with a Unix timestamp. (0000001670324876267 = Tue Dec 06 2022 22:07:56). For information on converting the timestamp, see https://www.unixtimestamp.com/.
    • The second part of the name (2D7E74B2) is randomly generated.
    • The extensions are either Flight Recorder Backup (FRB) and Flight Recorder Backup Registry (FRBR).
    • The file's datetime viewed by Windows, is the creation date of the original or source file, not the backup time.
  • Backups are to a local drive for very fast recovery.

Self cleaning

The Endpoint Agent triggers self-cleaning to meet the space and hours duration thresholds set in the Endpoint Detection and Response policy configuration.

A task runs every 10 minutes to check the disk quota and delete old backups or unindexed files if the quota is exceeded. This activity is logged in the following location:

  • C:\ProgramData\Malwarebytes Endpoint Agent\Logs\EndpointAgent.txt.

The following is an example of successful cleanups.

INFO FRCoreManager [FRSDK] Running cleanup ALL. RollbackTTL: 72 LearningMode: 2
INFO FRCoreManager [FRSDK] FR cleanup ALL started. Cleaning up events older than 2022-11-28 17:22:34+1100. Current backup files total number: 1579 and disk stats size/free/usage/quota/quota%: 84880125952/1781522432/537416136/695681570/30%
INFO FRCoreManager [FRSDK] FR cleanup ALL finished. Deleted 11574 backup events and 298 backup files. New backup files total number: 1281 and disk size/free/usage/quota/quota%: 84880125952/1902772224/390548912/687996340/30%
INFO FRCoreManager [FRSDK] Next backup cleanup scheduled for 2022-12-02 17:22:33+1100
INFO FRCoreManager Checking for orphaned backup files under "C:\ProgramData\Malwarebytes Endpoint Agent\Plugins\EDRPlugin\Backup"
INFO FRCoreManager Finished checking for orphaned backup files under "C:\ProgramData\Malwarebytes Endpoint Agent\Plugins\EDRPlugin\Backup"

Backup folder very large or exceeds configured quota

In scenarios where the backup folder is larger than expected, complete the following:

  1. Check the Windows Services and verify the Malwarebytes Endpoint Agent service is Running, as this service controls cleaning.
  2. Enable debug logging. For more information, see Enable debug logging on the Malwarebytes Endpoint Agent.
  3. Check the C:\ProgramData\Malwarebytes Endpoint Agent\Logs\EndpointAgent.txt for the following:
    • The self cleaning process running every 10 minutes. 
      DEBUG FRCoreManager FRCallbackWrapper -- Received Message: Disk size/free/usage/quota/quota%: 510823661568/404060053504/476145280/121360859635/30%
      DEBUG FRCoreManager FRCallbackWrapper -- Received Message: Disk size/free/usage/quota/quota%: 510823661568/404060053504/476145280/121360859635/30%
    • Any log messages containing ERROR FRCoreManager.
  4. Contact Support. For more information, see Contacting Support.

Manual or emergency cleanup of backup folder

In case it is urgent to free up disk space, complete the following:

  1. Collect diagnostic logs. For more information, see Collect Malwarebytes Endpoint Agent diagnostic logs.
  2. Report the issue to Support.
  3. Clean up the backup folder with one of the following methods:
    • Disable and re-enable EDR in the policy settings.
      • Create a Policy with all EDR policy settings disabled. For more information, see Configure Endpoint Detection and Response options in Nebula.
      • Create a Group with the new policy assigned.
      • Move the affected endpoints into that group. This will force the EDR plugin to unload and clean up. A reboot may be required.
      • Move the affected endpoints back to the previous group, and EDR will reinstall.
    • Uninstall and reinstall Malwarebytes.
      • Use Add or Remove Programs.
      • Use the Discovery and Deployment Tool.

Contacting Support

When submitting a support case, the following information is required:

  • Endpoint names.
  • Malwarebytes Diagnostic logs. For more information, see Collect Malwarebytes Endpoint Agent diagnostic logs.
    • If you are unable to collect the Malwarebytes Diagnostic logs, manually obtain the following files:
      • C:\ProgramData\Malwarebytes Endpoint Agent\Logs\EndpointAgent.txt
      • C:\ProgramData\Malwarebytes Endpoint Agent\Plugins\EDRPlugin\Database\2B455663142B495843A6F3DCB6B55CCE
    •  

Related articles