Configure Custom Detections - Advanced with ClamAV SIGTOOL.EXE on Windows

Introduction

This document describes how to create Custom Detections - Advanced using ClamAV sigtool.exe on Windows.

About Advanced Custom Detections

Advanced Custom Detections are like traditional antivirus signatures, but they are written by the user. In order to detect malware and other file-based threats, ClamAV relies on signatures to differentiate clean and malicious/unwanted files. ClamAV signatures are primarily text-based and conform to one of the ClamAV-specific signature formats associated with a given method of detection These signatures can inspect various aspects of a file and have different signature formats. Some of the available signature formats are:

• MD5 signatures
• MD5, PE section-based signatures
• File body-based signatures
• Extended signature format (offsets, wildcards, regular expressions)
• Logical signatures
• Icon signatures

The ClamAV project distributes a collection of signatures in the form of CVD (ClamAV Virus Database) files. The CVD file format provides a digitally-signed container that encapsulates the signatures and ensures that they cannot be modified by a malicious third-party. This signature set is actively maintained by Cisco Talos and can be downloaded using the freshclam application that ships with ClamAV

Why ClamAV

We use ACD to match complex detections/file attributes which is cannot be detected using SHA256 Hashes like those in the examples bellow:

Body-based Signature Content Format

ClamAV stores all body-based (content-based) signatures in a hexadecimal format, with exception to ClamAV YARA rule support.  By a hex-signature we mean a fragment of malware’s body converted into a hexadecimal string which can be additionally extended using various wildcards.

• Logical signatures

Logical signatures allow combining of multiple signatures in extended format using logical operators. They can provide both more detailed and flexible pattern matching.

• Extended signature format

The extended signature format is ClamAV most basic type of body-based signature since the deprecation of the original .db database format.

Extended sigantures allow for specification of additional information beyond just hexidecimal content such as a file "target type", virus offset, or engine functionality level (FLEVEL), making the detection more reliable

Phishing Signatures

ClamAV can detect HTML links that look suspicious when the display text is a URL that is a different domain than than in the actual URL. Unfortunately, it is pretty common for a company to contract out web services and to use HTML link display text to make it look like it is a link to the company website. Because this practice is commonplace, ClamAV only does phishing checks for specific websites that are popularly targeted by phishing campaigns

Bytecode Signatures

Bytecode Signatures are the means by which more complex matching can be performed by writing C code to parse sample content at various stages in file extraction.

Signatures based on container metadata

ClamAV 0.96 allows creating generic signatures matching files stored inside different container types which meet specific conditions. The signature format is:

VirusName:ContainerType:ContainerSize:FileNameREGEX:
FileSizeInContainer:FileSizeReal:IsEncrypted:FilePos:
Res1:Res2[:MinFL[:MaxFL]]

where the corresponding fields are:

VirusName: Virus name to be displayed when signature matches.

ContainerType: The file type containing the target file. For example:

CL_TYPE_ZIP,
CL_TYPE_RAR,
CL_TYPE_ARJ,
CL_TYPE_MSCAB,
CL_TYPE_7Z,
CL_TYPE_MAIL,
CL_TYPE_POSIX_TAR,
CL_TYPE_OLD_TAR,
CL_TYPE_CPIO_OLD,
CL_TYPE_CPIO_ODC,
CL_TYPE_CPIO_NEWC,
CL_TYPE_CPIO_CRC
and so on.

This document is focusing on hash signatures.

Note: The easiest way to create signatures for ClamAV is to use filehash checksums, however this method can be only used against static malware.

Caution: Please note that information in this document are subject to change with newer releases of ClamAV. Always corelate and verify with official guide for ClamAV.

More information on signature formats can be found at: ClamAV Website

These signatures are compiled into a file that is downloaded to the endpoint. In order to create advanced custom detections, go to Outbreak Control > Advanced. Click Create Signature Set to create a new Advanced Custom Detection set, give it a name, and click Create.

After you create the Advanced Custom Detection set, click Edit and you can see the Add Signature link. Enter the name of your signature and click Create.

After all of your signatures are listed, select Build a Database from Signature Set. If you accidentally add a signature you did not want, you can delete it by clicking Remove.

Warning:  Any time you add or remove a signature you MUST click on Build a Database from Signature Set

Note: When you create an advanced custom detection for a file, it is subject to caching for an hour. If a file is added to an advanced custom detection set, the cache time must expire before the detection takes effect. For example, if you add an advanced custom detection for an unknown file 5 minutes after it was cached, the detection is not going to take effect for another 55 minutes.

Warning: Advanced Custom Detections only work on files of unknown disposition.

How to Create Custom Detections - Advanced with sigtool.exe

Step 1: First we  need to obtain the sigtool by navigation to the ClamAV website downloads

In my case I download the ZIP package clamav-1.1.0.win.x64.zip

Step 2: Unzip the file in to preferred location. In my case I used C:/ClamAV

Please note highlighted folders and files.

clamscan.exe – is a command line tool which is used to scan files and/or directories for viruses. Unlike clamdscan, clamscan does not require a running clamd instance to function. Instead, clamscan create a new engine and load in the virus database each time it is run. It going to scan the files and/or directories specified at the command line, create a scan report, and exit.

sigtool.exe – Tool that creates and format signature and write in to *.hdb file. The hdb extension refers to Hash-based Signatures. sigtool pulls in libclamav and provides shortcuts to doing tasks that clamscan does behind the scenes. These can be really useful when writing a signature or trying to get information about a signature that can be causing FPs or performance problems.

VirusVault – This is my own folder that was created to drop files for which I wanted to create custom signature.

VirusRV.hdb – This file contains formatted signatures. I named this file VirusRV but you can use any meaningful name.

Step 3a: Launch CMD line and navigate to the location where you previously unzip your ClamAV.

Step 3b Execute this line.

C:\ClamAV>sigtool --md5 location-of-the-custom-file > name-of-the-output-file.hdb

You can change the name (by default sigtool uses the name of the file) and place it inside a *.hdb file. A single database file can include any number of signatures. To get them automatically loaded each time clamscan/clamd starts just copy the database file(s) into the local virus database directory (eg. /usr/local/share/clamav).

C:\ClamAV>sigtool --md5 C:\ClamAV\VirusVault\TestVirusRV.exe > VirusRV.hdb

This  creates the signature in this case for the file named TestVirusRV.exe and then write the signature to file named VirusRV.hdb

Note: If you like to create signature based on SHA value the corresponding file is then saved as *.hsb

C:\ClamAV>sigtool --md5 C:\ClamAV\VirusVault\TestVirusRV.exe > VirusRVsha256.hsb

Caution: The hash-based signatures shall not be used for text files, HTML and any other data that gets internally preprocessed before pattern matching. If you really want to use a hash signature in such a case, run clamscan with --debug and --leave-temps and create a signature for a preprocessed file left in /tmp. Please keep in mind that a hash signature going to stop matching as soon as a single byte changes in the target file. More can be found here

These sigtool flags can be especially useful for signature writing:

--md5 / --sha1 / --sha256: Generate the MD5/SHA1/SHA256 hash and calculate the file size, outputting both as a properly-formatted .hdb/.hsb signature

Signature names

ClamAV signatures must only use alphanumeric characters, dash (-), dot (.), underscores (_) to delimit words. Never use a space, apostrophe, colon, semi-colon, or quote mark.

ClamAV signature names found in the official signature databases generally use this format:

{platform}.{category}.{name}-{signature id}-{revision}

Naming conventions in 3rd party databases vary. You can find Cisco-Talos guidelines for naming signatures for the official database here.

Tip:  If you want to create multiple signatures on files located in the directory that you created earlier in my case I named mine VirusVault you can run the command using wild card. In this case I ran mine as:

C:\ClamAV>sigtool --md5 C:\ClamAV\VirusVault\wildcard\ > VirusRV.hdb

C:\ClamAV>sigtool --md5 C:\ClamAV\VirusVault\*\ > VirusRV.hdb

Which created signature against each file located in the directory VirusVault

Step 4: Verify the signature by running this command.

C:\ClamAV>clamscan -d VirusRV.hdb C:\ClamAV\VirusVault\TestVirusRV.exe

To verify all signatures you can again use wild card mask.

C:\ClamAV>clamscan -d VirusRV.hdb C:\ClamAV\VirusVault\*\

Step 5: Open newly created *.hdb file and copy created signature and then navigate to your secure endpoint console under Outbreak Control > Advanced click Edit and you can see the Add Signature link.

Step 6: Make sure you click on Build a Database from Signature Set then apply the new Custom Detections to your policy.

Step 7: Sync up your policy on your endpoint and test your new signature with manual scan. You can see results like those listed below.

Manual Scan

Detections in Secure Endpoint Console

Event Expanded

Requirements to Save Signature in Secure Endpoint Console

If you only have MD5 hash and you don’t know the size of the file you can still create signature set but you must use these rules:

A: MD5 cannot contain all uppercase letters

Correct MD5 Hash:
5b852928a129d63dc5c895bd62cf2ab7

Incorrect MD5 Hash:
5B852928A129D63DC5C895BD62CF2AB7

Error in Secure Endpoint Console with ALL upper case letters

B: You must use upper case letter for naming and it could be your own name.

Correct Name:
Dm-launcher.msi

Incorrect Name:
dm-launcher.msi

C: You can use wild card for the size but you must use :73 for minimum FLEVEL

Correct Wild Card:

5b852928a129d63dc5c895bd62cf2ab7:*:dm-launcher.msi:73

Incorrect Wild Card:

5b852928a129d63dc5c895bd62cf2ab7:*:dm-launcher.msi

Error in Secure Endpoint Console for all lower case name and no FLEVEL specified

Note: Hash signatures with unknown size - ClamAV 0.98 has also added support for hash signatures where the size is not known but the hash is. It is much more performance-efficient to use signatures with specific sizes, so be cautious when using this feature. For these cases, the ’*’ character can be used in the size field. To ensure proper backwards compatibility with older versions of ClamAV, these signatures must have a minimum functional level of 73 or higher. Signatures that use the wildcard size without this level set is rejected as malformed.