Windows Logs Automation

Offline Artifacts Collection

Unix Collector

Getting Started - UAC Documentationtclahr.github.io

Velociraptor Offline Agent

In certain scenarios, it may be necessary to collect artifacts from target machines in an offline mode, ensuring minimal alteration or contamination of the digital environment. Various tools are available for this purpose, designed to extract information discreetly and effectively. These tools enable forensic investigators to gather crucial data without compromising the integrity of the evidence.

Triage and acquisition :: Velociraptor - Digging deeper!docs.velociraptor.app

Importing an offline collection can be done via the Server.Utils.ImportCollection artifact. This artifact will inspect the zip file from a path specified on the server and import it as a new collection (with a new collection ID) into either a specified client or a new randomly generated client.

Using the KAPE GUI to analyze the artifacts

Instead of the velociraptor GUI, you can use the KAPE GUI to analyze and process all the artifacts.

If you are running it locally, the "Module Source" should be the folder where the artifacts obtained are. 😎

Next, you can use "TimeLine Explorer " to analyze the result.

KAPE Agent Collector

KAPE agent can be used to collect also the data.

Kroll Artifact Parser and Extractor (KAPE) is an efficient and highly configurable triage program that will target essentially any device or storage location, find forensically useful artifacts, and parse them within a few minutes.Kroll

The GUI can be used to select what kind of collection we want to do:

For collecting the artifacts on the remote machine, you just need to use the kape.exe collector with the Modules and Targets folder. See below:

To execute it, open a new command line with administrator rights, and paste the command obtained from step 4 on the GUI.

.\kape.exe --tsource C: --tdest C:\temp\kape --tflush --target !SANS_Triage --zip SERVER-DC01 --debug --gui

Next, the data will be available on: C:\temp\kap inside a zip file.

Run modules on the target machine

In addition, you can also run the modules on the target machine or locally.

If you are running it locally, the "Module Source" should be the folder where the artifacts obtained are. 😎

Next, you can use "TimeLine Explorer " to analyze the result.

Remote Collections with KAPE

kape.exe --tsource \\target-1\c --target LnkFilesAndJumpLists --tdest c:\kape_out\test

or accessing the target machine, mount the kape server folder, and run it remotely without touch with the binary on disk.

net use k: \\kape-server-vm\triage /user:kape-server-vm\analyst

k:\kape\kape.exe --tsource C --target RegistryHives --tdest k:\kape_out\tdest --vss

MDwikiericzimmerman.github.io

Building your agent collector

By developing an agent, you can collect also the raw files from the target Windows machine. This can be very useful for post-analysis in a depth way.

Next, see some of the raw artifacts collected.

Example of the content of the "amcache directory":

Full disk image

In order to obtain a complete snapshot of the target machines the following tools can be used:

FTK IMAGER

DFIR FTK Imager | Red Teaming and Malware Analysisgitbook.seguranca-informatica.pt

Disk2vhd

Disk2vhd - SysinternalsMicrosoftLearn

More details on how to use it can be found here:

File Analysis | Red Teaming and Malware Analysisgitbook.seguranca-informatica.pt

chntpwn

Also the usage of chntpwn to change Windows local administration passwords before executing an analysis.

Autopsy

Autopsy | Digital ForensicsAutopsy

TestDisk & PhotoRec

This tool can be used to recover files from damaged devices.

Velociraptor Analysis

After getting the ZIP files with all the artifacts, the file must be imported into the GUI.

Importing an offline collection can be done via the Server.Utils.ImportCollection artifact. This artifact will inspect the zip file from a path specified on the server and import it as a new collection (with new collection id) into either a specified client or a new randomly generated client.

After that, click on "Search" and select the target machine ID you want to analyze.

After that, select the artifacts FLOWID, click on Notebook, and all the data is presented! 👍

In addition, you can also create a new hunting and add the notebook logs into the hunting process. This is just a way how to split the results to perform a better analysis.

Eventx Analysis

chainsaw

GitHub - WithSecureLabs/chainsaw: Rapidly Search and Hunt through Windows Forensic ArtefactsGitHub

git clone https://github.com/WithSecureLabs/chainsaw.git
./chainsaw hunt EVTX-ATTACK-SAMPLES/ -s sigma/ --mapping mappings/sigma-event-logs-all.yml

Search all .evtx files for the case-insensitive string "mimikatz"

./chainsaw search mimikatz -i evtx_attack_samples/

*Search all .evtx files for powershell script block events (Event ID 4014

./chainsaw search -t 'Event.System.EventID: =4104' evtx_attack_samples/

Search a specific evtx log for logon events, with a matching regex pattern, output in JSON format

./chainsaw search -e "DC[0-9].insecurebank.local" evtx_attack_samples --json

Hunting

Hunt through all evtx files using Sigma rules for detection logic

./chainsaw hunt evtx_attack_samples/ -s sigma/ --mapping mappings/sigma-event-logs-all.yml

Hunt through all evtx files using Sigma rules and Chainsaw rules for detection logic and output in CSV format to the results folder

./chainsaw hunt evtx_attack_samples/ -s sigma/ --mapping mappings/sigma-event-logs-all.yml -r rules/ --csv --output results

Hunt through all evtx files using Sigma rules for detection logic, only search between specific timestamps, and output the results in JSON format

 ./chainsaw hunt evtx_attack_samples/ -s sigma/ --mapping mappings/sigma-event-logs-all.yml --from "2019-03-17T19:09:39" --to "2019-03-17T19:09:50" --json

$ ./chainsaw hunt -r rules/ evtx_attack_samples -s sigma/rules --mapping mappings/sigma-event-logs-all.yml --level critical

     ██████╗██╗  ██╗ █████╗ ██╗███╗   ██╗███████╗ █████╗ ██╗    ██╗
    ██╔════╝██║  ██║██╔══██╗██║████╗  ██║██╔════╝██╔══██╗██║    ██║
    ██║     ███████║███████║██║██╔██╗ ██║███████╗███████║██║ █╗ ██║
    ██║     ██╔══██║██╔══██║██║██║╚██╗██║╚════██║██╔══██║██║███╗██║
    ╚██████╗██║  ██║██║  ██║██║██║ ╚████║███████║██║  ██║╚███╔███╔╝
     ╚═════╝╚═╝  ╚═╝╚═╝  ╚═╝╚═╝╚═╝  ╚═══╝╚══════╝╚═╝  ╚═╝ ╚══╝╚══╝
        By WithSecure Countercept (@FranticTyping, @AlexKornitzer)

    [+] Loading detection rules from: ../../rules/, /tmp/sigma/rules
    [+] Loaded 129 detection rules (198 not loaded)
    [+] Loading event logs from: ../../evtx_attack_samples (extensions: .evtx)
    [+] Loaded 268 EVTX files (37.5 MB)
    [+] Hunting: [========================================] 268/268

    [+] Group: Antivirus
    ┌─────────────────────┬────────────────────┬──────────┬───────────┬─────────────┬────────────────────────────────┬──────────────────────────────────┬────────────────────┐
    │      timestamp      │     detections     │ Event ID │ Record ID │  Computer   │          Threat Name           │           Threat Path            │        User        │
    ├─────────────────────┼────────────────────┼──────────┼───────────┼─────────────┼────────────────────────────────┼──────────────────────────────────┼────────────────────┤
    │ 2019-07-18 20:40:00 │ ‣ Windows Defender │ 1116     │ 37        │ MSEDGEWIN10 │ Trojan:PowerShell/Powersploit. │ file:_C:\AtomicRedTeam\atomic-   │ MSEDGEWIN10\IEUser │
    │                     │                    │          │           │             │ M                              │ red-team-master\atomics\T1056\   │                    │
    │                     │                    │          │           │             │                                │ Get-Keystrokes.ps1               │                    │
    ├─────────────────────┼────────────────────┼──────────┼───────────┼─────────────┼────────────────────────────────┼──────────────────────────────────┼────────────────────┤
    │ 2019-07-18 20:53:31 │ ‣ Windows Defender │ 1117     │ 106       │ MSEDGEWIN10 │ Trojan:XML/Exeselrun.gen!A     │ file:_C:\AtomicRedTeam\atomic-   │ MSEDGEWIN10\IEUser │
    │                     │                    │          │           │             │                                │ red-team-master\atomics\T1086\   │                    │
    │                     │                    │          │           │             │                                │ payloads\test.xsl                │                    │
    └─────────────────────┴────────────────────┴──────────┴───────────┴─────────────┴────────────────────────────────┴──────────────────────────────────┴────────────────────┘

    [+] Group: Log Tampering
    ┌─────────────────────┬───────────────────────────────┬──────────┬───────────┬────────────────────────────────┬───────────────┐
    │      timestamp      │          detections           │ Event ID │ Record ID │            Computer            │     User      │
    ├─────────────────────┼───────────────────────────────┼──────────┼───────────┼────────────────────────────────┼───────────────┤
    │ 2019-01-20 07:00:50 │ ‣ Security Audit Logs Cleared │ 1102     │ 32853     │ WIN-77LTAPHIQ1R.example.corp   │ Administrator │
    └─────────────────────┴───────────────────────────────┴──────────┴───────────┴────────────────────────────────┴───────────────┘

    [+] Group: Sigma
    ┌─────────────────────┬────────────────────────────────┬───────┬────────────────────────────────┬──────────┬───────────┬──────────────────────────┬──────────────────────────────────┐
    │      timestamp      │           detections           │ count │     Event.System.Provider      │ Event ID │ Record ID │         Computer         │            Event Data            │
    ├─────────────────────┼────────────────────────────────┼───────┼────────────────────────────────┼──────────┼───────────┼──────────────────────────┼──────────────────────────────────┤
    │ 2019-04-29 20:59:14 │ ‣ Malicious Named Pipe         │ 1     │ Microsoft-Windows-Sysmon       │ 18       │ 8046      │ IEWIN7                   │ ---                              │
    │                     │                                │       │                                │          │           │                          │ Image: System                    │
    │                     │                                │       │                                │          │           │                          │ PipeName: "\\46a676ab7f179e511   │
    │                     │                                │       │                                │          │           │                          │ e30dd2dc41bd388"                 │
    │                     │                                │       │                                │          │           │                          │ ProcessGuid: 365ABB72-D9C4-5CC   │
    │                     │                                │       │                                │          │           │                          │ 7-0000-0010EA030000              │
    │                     │                                │       │                                │          │           │                          │ ProcessId: 4                     │
    │                     │                                │       │                                │          │           │                          │ RuleName: ""                     │
    │                     │                                │       │                                │          │           │                          │ UtcTime: "2019-04-29 20:59:14.   │
    │                     │                                │       │                                │          │           │                          │ 430"                             │
    ├─────────────────────┼────────────────────────────────┼───────┼────────────────────────────────┼──────────┼───────────┼──────────────────────────┼──────────────────────────────────┤
    │ 2019-04-30 20:26:51 │ ‣ CobaltStrike Service         │ 1     │ Microsoft-Windows-Sysmon       │ 13       │ 9806      │ IEWIN7                   │ ---                              │
    │                     │ Installations in Registry      │       │                                │          │           │                          │ Details: "%%COMSPEC%% /b /c st   │
    │                     │                                │       │                                │          │           │                          │ art /b /min powershell.exe -no   │
    │                     │                                │       │                                │          │           │                          │ p -w hidden -noni -c \"if([Int   │
    │                     │                                │       │                                │          │           │                          │ Ptr]::Size -eq 4){$b='powershe   │
    │                     │                                │       │                                │          │           │                          │ ll.exe'}else{$b=$env:windir+'\   │
    │                     │                                │       │                                │          │           │                          │ \syswow64\\WindowsPowerShell\\   │
    │                     │                                │       │                                │          │           │                          │ v1.0\\powershell.exe'};$s=New-   │
    │                     │                                │       │                                │          │           │                          │ Object System.Diagnostics.Proc   │
    │                     │                                │       │                                │          │           │                          │ essStartInfo;$s.FileName=$b;$s   │
    │                     │                                │       │                                │          │           │                          │ .Arguments='-noni -nop -w hidd   │
    │                     │                                │       │                                │          │           │                          │ en -c &([scriptblock]::create(   │
    │                     │                                │       │                                │          │           │                          │ (New-Object IO.StreamReader(Ne   │
    │                     │                                │       │                                │          │           │                          │ w-Object IO.Compression.GzipSt   │
    │                     │                                │       │                                │          │           │                          │ ream((New-Object IO.MemoryStre   │
    │                     │                                │       │                                │          │           │                          │ am(,[Convert]::FromBase64Strin   │
    │                     │                                │       │                                │          │           │                          │ g(''H4sIAIuvyFwCA7VW+2/aSBD+OZ   │
    │                     │                                │       │                                │          │           │                          │ H6P1...                          │
    │                     │                                │       │                                │          │           │                          │ (use --full to show all content) │
    │                     │                                │       │                                │          │           │                          │ EventType: SetValue              │
    │                     │                                │       │                                │          │           │                          │ Image: "C:\\Windows\\system32\   │
    │                     │                                │       │                                │          │           │                          │ \services.exe"                   │
    │                     │                                │       │                                │          │           │                          │ ProcessGuid: 365ABB72-2586-5CC   │
    │                     │                                │       │                                │          │           │                          │ 9-0000-0010DC530000              │
    │                     │                                │       │                                │          │           │                          │ ProcessId: 460                   │
    │                     │                                │       │                                │          │           │                          │ RuleName: ""                     │
    │                     │                                │       │                                │          │           │                          │ TargetObject: "HKLM\\System\\C   │
    │                     │                                │       │                                │          │           │                          │ urrentControlSet\\services\\he   │
    │                     │                                │       │                                │          │           │                          │ llo\\ImagePath"                  │
    │                     │                                │       │                                │          │           │                          │ UtcTime: "2019-04-30 20:26:51.   │
    │                     │                                │       │                                │          │           │                          │ 934"                             │
    ├─────────────────────┼────────────────────────────────┼───────┼────────────────────────────────┼──────────┼───────────┼──────────────────────────┼──────────────────────────────────┤
    │ 2019-05-12 12:52:43 │ ‣ Meterpreter or Cobalt        │ 1     │ Service Control Manager        │ 7045     │ 10446     │ IEWIN7                   │ ---                              │
    │                     │ Strike Getsystem Service       │       │                                │          │           │                          │ AccountName: LocalSystem         │
    │                     │ Installation                   │       │                                │          │           │                          │ ImagePath: "%COMSPEC% /c ping    │
    │                     │                                │       │                                │          │           │                          │ -n 1 127.0.0.1 >nul && echo 'W   │
    │                     │                                │       │                                │          │           │                          │ inPwnage' > \\\\.\\pipe\\WinPw   │
    │                     │                                │       │                                │          │           │                          │ nagePipe"                        │
    │                     │                                │       │                                │          │           │                          │ ServiceName: WinPwnage           │
    │                     │                                │       │                                │          │           │                          │ ServiceType: user mode service   │
    │                     │                                │       │                                │          │           │                          │ StartType: demand start          │
    ├─────────────────────┼────────────────────────────────┼───────┼────────────────────────────────┼──────────┼───────────┼──────────────────────────┼──────────────────────────────────┤
    │ 2019-06-21 07:35:37 │ ‣ Dumpert Process Dumper       │ 1     │ Microsoft-Windows-Sysmon       │ 11       │ 238375    │ alice.insecurebank.local │ ---                              │
    │                     │                                │       │                                │          │           │                          │ CreationUtcTime: "2019-06-21 0   │
    │                     │                                │       │                                │          │           │                          │ 6:53:03.227"                     │
    │                     │                                │       │                                │          │           │                          │ Image: "C:\\Users\\administrat   │
    │                     │                                │       │                                │          │           │                          │ or\\Desktop\\x64\\Outflank-Dum   │
    │                     │                                │       │                                │          │           │                          │ pert.exe"                        │
    │                     │                                │       │                                │          │           │                          │ ProcessGuid: ECAD0485-88C9-5D0   │
    │                     │                                │       │                                │          │           │                          │ C-0000-0010348C1D00              │
    │                     │                                │       │                                │          │           │                          │ ProcessId: 3572                  │
    │                     │                                │       │                                │          │           │                          │ RuleName: ""                     │
    │                     │                                │       │                                │          │           │                          │ TargetFilename: "C:\\Windows\\   │
    │                     │                                │       │                                │          │           │                          │ Temp\\dumpert.dmp"               │
    │                     │                                │       │                                │          │           │                          │ UtcTime: "2019-06-21 07:35:37.   │
    │                     │                                │       │                                │          │           │                          │ 324"                             │
    └─────────────────────┴────────────────────────────────┴───────┴────────────────────────────────┴──────────┴───────────┴──────────────────────────┴──────────────────────────────────┘

Shimcache

From the collected raw files. there is the SYSTEM hive.

./chainsaw analyse shimcache ./SYSTEM --regexfile ./analysis/shimcache_patterns.txt --amcache ./Amcache.hve --tspair --output ./output.csv

SRUM

Analyse the SRUM database (the SOFTWARE hive is mandatory)

./chainsaw analyse srum --software ./SOFTWARE ./SRUDB.dat --output ./output.json

$ ./chainsaw analyse srum --software ./SOFTWARE ./SRUDB.dat -o ./output.json

     ██████╗██╗  ██╗ █████╗ ██╗███╗   ██╗███████╗ █████╗ ██╗    ██╗
    ██╔════╝██║  ██║██╔══██╗██║████╗  ██║██╔════╝██╔══██╗██║    ██║
    ██║     ███████║███████║██║██╔██╗ ██║███████╗███████║██║ █╗ ██║
    ██║     ██╔══██║██╔══██║██║██║╚██╗██║╚════██║██╔══██║██║███╗██║
    ╚██████╗██║  ██║██║  ██║██║██║ ╚████║███████║██║  ██║╚███╔███╔╝
     ╚═════╝╚═╝  ╚═╝╚═╝  ╚═╝╚═╝╚═╝  ╚═══╝╚══════╝╚═╝  ╚═╝ ╚══╝╚══╝
        By WithSecure Countercept (@FranticTyping, @AlexKornitzer)

    [+] ESE database file loaded from "/home/user/Documents/SRUDB.dat"
    [+] Parsing the ESE database...
    [+] SOFTWARE hive loaded from "/home/user/Documents/SOFTWARE"
    [+] Parsing the SOFTWARE registry hive...
    [+] Analysing the SRUM database...
    [+] Details about the tables related to the SRUM extensions:
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | Table GUID                               | Table Name                                 | DLL Path                             | Timeframe of the data   | Expected Retention Time |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {5C8CF1C7-7257-4F13-B223-970EF5939312}   | App Timeline Provider                      | %SystemRoot%\System32\eeprov.dll     | 2022-03-10 16:34:59 UTC | 7 days                  |
    |                                          |                                            |                                      | 2022-03-10 21:10:00 UTC |                         |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {B6D82AF1-F780-4E17-8077-6CB9AD8A6FC4}   | Tagged Energy Provider                     | %SystemRoot%\System32\eeprov.dll     | No records              | 3 days                  |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {D10CA2FE-6FCF-4F6D-848E-B2E99266FA86}   | WPN SRUM Provider                          | %SystemRoot%\System32\wpnsruprov.dll | 2022-03-10 20:09:00 UTC | 60 days                 |
    |                                          |                                            |                                      | 2022-03-10 21:09:00 UTC |                         |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {D10CA2FE-6FCF-4F6D-848E-B2E99266FA89}   | Application Resource Usage Provider        | %SystemRoot%\System32\appsruprov.dll | 2022-03-10 16:34:59 UTC | 60 days                 |
    |                                          |                                            |                                      | 2022-03-10 21:10:00 UTC |                         |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {FEE4E14F-02A9-4550-B5CE-5FA2DA202E37}   | Energy Usage Provider                      | %SystemRoot%\System32\energyprov.dll | No records              | 60 days                 |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {FEE4E14F-02A9-4550-B5CE-5FA2DA202E37}LT | Energy Usage Provider (Long Term)          | %SystemRoot%\System32\energyprov.dll | No records              | 1820 days               |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {973F5D5C-1D90-4944-BE8E-24B94231A174}   | Windows Network Data Usage Monitor         | %SystemRoot%\System32\nduprov.dll    | 2022-03-10 16:34:59 UTC | 60 days                 |
    |                                          |                                            |                                      | 2022-03-10 21:10:00 UTC |                         |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {7ACBBAA3-D029-4BE4-9A7A-0885927F1D8F}   | vfuprov                                    | %SystemRoot%\System32\vfuprov.dll    | 2022-03-10 20:09:00 UTC | 60 days                 |
    |                                          |                                            |                                      | 2022-03-10 21:10:00 UTC |                         |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {DA73FB89-2BEA-4DDC-86B8-6E048C6DA477}   | Energy Estimation Provider                 | %SystemRoot%\System32\eeprov.dll     | No records              | 7 days                  |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    | {DD6636C4-8929-4683-974E-22C046A43763}   | Windows Network Connectivity Usage Monitor | %SystemRoot%\System32\ncuprov.dll    | 2022-03-10 16:34:59 UTC | 60 days                 |
    |                                          |                                            |                                      | 2022-03-10 21:10:00 UTC |                         |
    +------------------------------------------+--------------------------------------------+--------------------------------------+-------------------------+-------------------------+
    [+] SRUM database parsed successfully
    [+] Saving output to "/home/user/Documents/output.json"
    [+] Saved output to "/home/user/Documents/output.json"

zircolite

python3 zircolite.py --evtx Logs/ --package

GitHub - wagga40/Zircolite: A standalone SIGMA-based detection tool for EVTX, Auditd and Sysmon for Linux logsGitHub

Hayabusa

Import the logs into Timeline Explorer and add visualization filter by: LEVEL > COMPUTER > RULE

https://mahim-firoj.medium.com/incident-response-and-threat-hunting-using-hayabusa-tool-383da273183amahim-firoj.medium.com

GitHub - Yamato-Security/hayabusa: Hayabusa (隼) is a sigma-based threat hunting and fast forensics timeline generator for Windows event logs.GitHub

DeepBlueCLI & WELA & APT-Hunter

These tools are also useful to collect pieces of evidence from eventx. For example, WELA can detail authentication on the machine by user and their types.

Windows EventViewer Analysis | DFIR | Red Teaming and Malware Analysisgitbook.seguranca-informatica.pt

Manual Analysis with ericzimmerman Tools

MDwikiericzimmerman.github.io

Each facet of analysis is delineated into subsections accessible through the navigation menu on this page.

The files under analysis are:

Timeline Explorer, EZViewer, and Hasher are proficient tools for concurrently examining all artifacts.

Timeline Explorer

Opening all the CSV files post-normalization tailored by the specific tools.

EZViewer

Opening single files (docx, csv, pdf, etc).

Hasher

Hash everything.

Dissect

Dissect is an incident response framework build from various parsers and implementations of file formats. Tying this all together, Dissect allows you to work with tools named target-query and target-shell to quickly gain access to forensic artefacts, such as Runkeys, Prefetch files, and Windows Event Logs, just to name a few!

And the best thing: all in a singular way, regardless of underlying container (E01, VMDK, QCoW), filesystem (NTFS, ExtFS, FFS), or Operating System (Windows, Linux, ESXi) structure / combination. You no longer have to bother extracting files from your forensic container, mount them (in case of VMDKs and such), retrieve the MFT, and parse it using a separate tool, to finally create a timeline to analyse. This is all handled under the hood by Dissect in a user-friendly manner.

If we take the example above, you can start analysing parsed MFT entries by just using a command like target-query -f mft <PATH_TO_YOUR_IMAGE>!

target-shell `<PATH_TO_YOUR_IMAGE>`

Download artifacts from image raw (VMDK, E01, RAW, etc)

target-dump -o . --restart -f mft,usnjrnl,amcache.applaunches,amcache.application_files,amcache.applications,amcache.device_containers,amcache.drivers,amcache.files,amcache.programs,amcache.shortcuts,defender.evtx,defender.exclusions,defender.quarantine,shimcache,lnk,services,runkeys,shellbags,browser.history,browser.downloads,tasks,remoteaccess,shell,ssh,windows.users,jumplist,prefetch,recyclebin,recentfilecache,sru,startupinfo,syscache,thumbcache,wer,wua_history <TARGET_PATH.RAW>

After that, convert all the jsonl files from the dissect output into CSV files to import them in the Timeline Explorer!

import os
import csv
import json
import sys

# Function to convert JSONL to CSV, skipping the first line
def convert_jsonl_to_csv(jsonl_file, csv_file):
    with open(jsonl_file, 'r') as json_file:
        with open(csv_file, 'w', newline='') as csv_out:
            csv_writer = csv.writer(csv_out)
            # Skip the first line
            next(json_file)
            for line in json_file:
                data = json.loads(line)
                if csv_out.tell() == 0:  # Write the header in the first line
                    csv_writer.writerow(data.keys())
                csv_writer.writerow(data.values())

# Ask the user for the base directory
base_dir = input('Please enter the base directory path: ')

# Check if the directory exists
if not os.path.isdir(base_dir):
    print(f"The path {base_dir} does not exist. Please try again.")
else:
    # Count total number of files to process
    total_files = sum(len(files) for _, _, files in os.walk(base_dir) if any(file.endswith('.jsonl') for file in files))
    progress = 0
    finished_files = []  # List to keep track of finished files
    
    # Walk through the directories and subdirectories recursively
    for root, dirs, files in os.walk(base_dir):
        for file in files:
            if file.endswith('.jsonl'):
                jsonl_path = os.path.join(root, file)
                csv_path = os.path.join(root, file.replace('.jsonl', '.csv'))
                convert_jsonl_to_csv(jsonl_path, csv_path)
                progress += 1
                finished_files.append(file)

                # Show files converted so far
                print("\nFiles converted so far:")
                for finished in finished_files:
                    print(finished)

                # Print progress bar with file name
                bar_length = 40  # Length of the progress bar
                progress_bar = '#' * int(bar_length * progress / total_files)
                sys.stdout.write(f'\rConverting {file}: [{progress_bar:<{bar_length}}] {progress}/{total_files} files converted')
                sys.stdout.flush()

    print("\nAll files converted!")

Drop all the security logs from Collectors zip files

import zipfile
import os
import shutil

# Lista das pastas
folders = [
    "cxxxx000031/",
    "Cxxxx00010CV/"
]

# Diretório de saída
output_dir = "output_evtx"
os.makedirs(output_dir, exist_ok=True)

# Loop por cada pasta para processar o ficheiro ZIP
for folder in folders:
    # Localizar qualquer arquivo ZIP que comece com "Collection-"
    for filename in os.listdir(folder):
        if filename.startswith("Collection-") and filename.endswith(".zip"):
            zip_path = os.path.join(folder, filename)
            prefix = os.path.basename(os.path.normpath(folder))  # Prefixo com o nome da pasta

            with zipfile.ZipFile(zip_path, 'r') as zip_ref:
                # Procurar o ficheiro Security.evtx dentro do ZIP
                for file in zip_ref.namelist():
                    if "Security.evtx" in file:
                        # Extrair e renomear o ficheiro com o prefixo da pasta
                        output_file_path = os.path.join(output_dir, f"{prefix}_Security.evtx")
                        with zip_ref.open(file) as source, open(output_file_path, "wb") as target:
                            shutil.copyfileobj(source, target)
                        print(f"Extraído: {output_file_path}")
                        break  # Parar após encontrar o Security.evtx
            break  # Prosseguir para a próxima pasta após encontrar um ZIP que comece com "Collection-"

Yara scan in raw formats

target-query -f yara xxxx-flat.vmdk -p '/c:' --check -r yara/

target-yara xxx-flat.vmdk -p 'c:\Users' --check -r yara/ | tee -a output.log
target-yara xxx-flat.vmdk -p 'c:\Windows\Temp' --check -r yara/ | tee -a out
target-yara xxx-flat.vmdk -p 'c:\Users\ProgramData' --check -r yara/ | tee -a out

My bundle:

7MB

yara_.zip

Convert from yara output into CSV


import csv
import re

def parse_yara_output(line):
    pattern = r"hostname='(?P<hostname>.*?)' domain='(?P<domain>.*?)' path='(?P<path>.*?)' digest=\(md5=(?P<md5>.*?), sha1=(?P<sha1>.*?), sha256=(?P<sha256>.*?)\) rule='(?P<rule>.*?)' tags=\[(?P<tags>.*?)\] namespace='(?P<namespace>.*?)'"
    match = re.search(pattern, line)
    if match:
        return match.groupdict()
    return None

def main():
    input_filename = input("Digite o nome do ficheiro de output YARA: ")

    output_filename = input_filename.split('.')[0] + ".csv"

    with open(input_filename, 'r') as infile:
        yara_data = infile.readlines()

    parsed_data = [parse_yara_output(line) for line in yara_data if parse_yara_output(line)]

    fields = ["hostname", "domain", "path", "md5", "sha1", "sha256", "rule", "tags", "namespace"]

    with open(output_filename, 'w', newline='') as csvfile:
        csvwriter = csv.DictWriter(csvfile, fieldnames=fields)
        csvwriter.writeheader()
        for data in parsed_data:
            # Converte a lista de tags para string, se existir
            if data['tags']:
                data['tags'] = ", ".join(data['tags'].split(", "))
            csvwriter.writerow(data)

    print(f"Ficheiro CSV gerado: {output_filename}")

if __name__ == "__main__":
    main()

After that you can import the CSV in Timeline Explorer and group the output by matched rules ;)

Yara Repositories:

Loki Yara

GitHub - Neo23x0/signature-base: YARA signature and IOC database for my scanners and toolsGitHub

Awesome Yara

GitHub - InQuest/awesome-yara: A curated list of awesome YARA rules, tools, and people.GitHub

Dissect Documentation:

GitHub - fox-it/dissect: Dissect is a digital forensics & incident response framework and toolset that allows you to quickly access and analyse forensic artefacts from various disk and file formats, developed by Fox-IT (part of NCC Group).GitHub

Dissect Tutorials

Dissect vs SysInternals Case Part 1: Planning and TestingPending Investigations

Dissect vs SysInternals Case, Part 2Pending Investigations

gKAPE (offline parser)

Using the gkape tool to parse the telemetry obtained from the collector (raw log files from Windows).

After getting all the zip outputs from the target machines, the following procedure should be executed:

The files should be prepared to analyze

The following python script executed on the root folder:

import os
import shutil

def list_folders(directory):
    try:
        # Construct the full path
        full_path = os.path.abspath(directory)
        
        # Check if the directory exists
        if not os.path.exists(full_path):
            raise FileNotFoundError(f"Directory '{directory}' does not exist.")
        
        # List all entries in the directory
        entries = os.listdir(full_path)
        
        # Filter only directories
        folders = [entry for entry in entries if os.path.isdir(os.path.join(full_path, entry))]
        
        return folders
    
    except Exception as e:
        print(f"Error: {e}")
        return []

def copy_thumbcache_files(root_folder, output_folder):
    try:
        # Create output folder if it doesn't exist
        if not os.path.exists(output_folder):
            os.makedirs(output_folder)
        
        # List all folders in the root folder
        folders = list_folders(root_folder)
        
        # Iterate through each folder
        for folder in folders:
            folder_path = os.path.join(root_folder, folder)
            explorer_path = os.path.join(folder_path, 'AppData', 'Local', 'Microsoft', 'Windows', 'Explorer')
            
            # Check if the Explorer folder exists
            if os.path.exists(explorer_path):
                # Iterate through files in Explorer folder
                files = os.listdir(explorer_path)
                
                # Copy thumbcache files to output folder
                for file in files:
                    if file.startswith("thumbcache"):
                        file_path = os.path.join(explorer_path, file)
                        shutil.copy(file_path, os.path.join(output_folder, file))
                        print(f"Copied '{file}' to '{output_folder}'")
    
    except Exception as e:
        print(f"Error: {e}")

# Root folder to search for user folders
root_folder = 'uploads\\auto\\C%3A\\Users'

# Output folder for copied thumbcache files
output_folder = 'output_thumbcache'

# Copy thumbcache files to output folder
copy_thumbcache_files(root_folder, output_folder)

Changing the following mkape files:

Description: 'thumbcache_viewer_cmd.exe: process Windows Thumbcache files'
Category: FileKnowledge
Author: Dennis Reneau, Kevin Pagano
Version: 2.0
Id: 8896483c-563a-4a28-ad8a-07ba74a54a63
BinaryUrl: https://github.com/thumbcacheviewer/thumbcacheviewer/releases/download/v1.0.1.8/thumbcache_viewer_cmd.zip
ExportFormat: html
Processors:
    -
        Executable: thumbcache_viewer_cmd.exe
        CommandLine: -o %destinationDirectory%\ThumbCache_Results -w -c -z -d %sourceDirectory%\output_thumbcache
        ExportFormat: html
        ExportFile: thumbcache_results.csv

# Documentation
# Uses Thumbcache Viewer (https://github.com/thumbcacheviewer)
# Designed to work with the Thumbcache DB Target collection created by Eric Zimmerman.
# Executable author Eric Kutcher.
# Point msource (Module Source) to the Thumbcache folder or use the Target/Module option of KAPE.
# Options  -w HTML Report | -c CSV Report | -z Exclude 0 byte files | -n Prevent Thumbnail extraction | -o Output
# 2023-06-27 Updated by Kevin Pagano: Updated binary URL, changed source to directory for parsing to HMTL properly if more than DB one file

Description: Tool to parse Windows Background Intelligent Transfer Service database files
Category: GitHub
Author: Pedro Sanchez Cordero (conexioninversa)
Version: 1.0
Id: acdc62ed-b1a1-426f-8d5e-e53687284410
BinaryUrl: https://github.com/conexioninversa/BitsParser/blob/master/BitsParser.exe
ExportFormat: json
Processors:
    -
        Executable: BitsParser.exe
        CommandLine: -i %sourceDirectory%\uploads\auto\C%3A\ProgramData\Microsoft\Network\Downloader\ -o %destinationDirectory%\BitsParser_Results.json
        ExportFormat: json

# Documentation
# https://github.com/fireeye/BitsParser
# By default BitsParser will process files in the %ALLUSERSPROFILE%\Microsoft\Network\Downloader. The script can be used with offline files from alternate operating systems.
# By default BitsParser will only parse and output active jobs and files. To carve deleted entries from the database use --carvedb. To carve entries from all file types, including transaction logs, use --carveall
# https://www.sans.org/reading-room/whitepapers/forensics/bits-forensics-39195
# https://cyberforensicator.com/2019/05/12/using-mitre-attck-for-forensics-bits-jobs-t1197/

Description: 'Ese2csv: Parsing SRUM Database'
Category: SRUMDatabase
Author: Max Ye
Version: 1.0
Id: 852b64c1-fd0e-47ec-8aa4-0994dbf5d8d1
BinaryUrl: https://github.com/MarkBaggett/ese-analyst/archive/master.zip
ExportFormat: csv
Processors:
    -
        Executable: ese-analyst\ese2csv.exe
        CommandLine: -o %destinationDirectory% -p srudb_plugin --plugin-args "%sourceDirectory%\uploads\auto\C%3A\Windows\System32\config\SOFTWARE" -- "%sourceDirectory%\uploads\auto\C%3A\Windows\System32\sru\SRUDB.dat"
        ExportFormat: csv

# Documentation
# https://github.com/MarkBaggett/ese-analyst
# Create a folder "ese-analyst" within the ".\KAPE\Modules\bin" folder
# Place both files "ese2csv.exe" and "srudb_plugin.py" into ".\KAPE\Modules\bin\ese-analyst"
# When using this Module, the Module source should be set to OS drive root directory (e.g. C:\), because parameters use absolute paths

Use the following modules:

!!Tools-Sync
!EZParser
Bits-Parser
BMC-Tools
ThumbCacheViewer
DHParser
LogParser (Run all)
--or--
LogParser_RDPUsageEvents
LogParser_xxx

Also, run the following manually:

-----CHAINSAW----
Chainsaw.exe search -i mimikatz z:\Collection-contabilxxxx\uploads\auto\C%3A\Windows\System32\winevt\Logs\

C:\Users\malwr\Desktop\kape-gui\KAPE\Modules\Chainsaw>Chainsaw.exe hunt z:\Collection-contabilixxxxxx\uploads\auto\C%3A\Windows\System32\winevt\Logs\ -s sigma/ --mapping mappings/sigma-event-logs-all.yml -r rules/ --level critical

C:\Users\malwr\Desktop\kape-gui\KAPE\Modules\Chainsaw>Chainsaw.exe hunt z:\Collection-contabilxxxxx\uploads\auto\C%3A\Windows\System32\winevt\Logs\ -s sigma/ --mapping mappings/sigma-event-logs-all.yml -r rules/ -o teste

---DEEP BLUE CLI-------
DeepblueCLI

---- LOG PARSER----
 LogParser.exe: -stats:OFF -i:EVT "SELECT TO_UTCTIME(TimeGenerated) AS Date, EventID, 'Client attempted to access SMB via an anonymous logon.' AS Description, EXTRACT_TOKEN(Strings,8,'|') AS UserName,  EXTRACT_TOKEN(Strings,10,'|') AS ClientName INTO 'Z:\Contabilidade\Scan2\RemoteAccess\logparser-SMBServer-Anonymous-Logon.csv' FROM '"Z:\Collection-xxxxx\uploads\auto\C%3A\Windows\System32\winevt\Logs\Microsoft-Windows-SMBServer%254Security.evtx"' WHERE EventID=551" -filemode:0

Have a good hunting! 😎

GRR

GRR is a python client (agent) that is installed on target systems, and python server infrastructure that can manage and talk to clients.

GitHub - google/grr: GRR Rapid Response: remote live forensics for incident responseGitHub

References

PreviousParsing Large Raw Files and Excluding Country IP Address Ranges Nextamcache.hve

Last updated 1 year ago

Was this helpful?