Web Exploit Detector: Node.js security scanner

Introduction

After detecting some new exploits on one of my web servers, I decided to start work on a new application called the “Web Exploit Detector”. This project is open-source and hosted on GitHub, meaning that it’s free for anyone to use and, more importantly, anyone can contribute their own rules to make the tool better.

I now use this tool on my own servers to check for infections on a daily basis and it works well so far. It is my hope that, given time and the help of others, it can become even more useful and capable of detecting even the most recent of exploits.

The two major reasons that I have made this project available as open source are: -

1. So that you and others can benefit from it freely.
2. So that others, who will probably encounter different exploits from those that I do, can extend the tool so that it can detect as many exploits as possible.

The tool aims to detect exploits and suspicious files only, not to remove them. However as you might know already, I am the author of the Cybersecurity Blog here on this site. Each detection in the application will, where possible, include a link to the exact blog post that relates to that type of infection. In these blog posts you can find a technical analysis of the exploit as well as instructions on how to remove it and prevent it in the future.

This application is a work in progress. There are a number of features that I want to add and improve upon and I will do so as soon as possible. Even now, the application works and is useful for detecting a limited number of exploits and suspicious files, but soon it will be able to detect more and will also be easier to use. I will follow up on this post when these improvements have been made and also with another post about how to use the tool effectively.

Update 2017-04-22

This project is now available as an NPM module: see this blog post for further details.

Application introduction

The Web Exploit Detector is a Node.js application (and NPM module) used to detect possible infections, malicious code and suspicious files in web hosting environments. This application is intended to be run on web servers hosting one or more websites. Running the application will generate a list of files that are potentially infected together with a description of the infection and references to online resources relating to it.

The application is hosted here on GitHub so that others can benefit from it, as well as allowing others to contribute their own detection rules.

Links

• My website: https://www.polaris64.net/
• My cybersecurity blog, which contains articles describing some of these exploits and how to remove them
• Contact me
• NPM module

Installation

Regular users

The simplest way to install Web Exploit Detector is as a global NPM module: -

npm install -g web_exploit_detector

If you are running Linux or another Unix-based OS you might need to run this command as root (e.g. sudo npm install -g web_exploit_detector).

Updating

The module should be updated regularly to make sure that all of the latest detection rules are present. Running the above command will always download the latest stable (tested) version. To update a version that has already been installed, simply run the following: -

npm update -g web_exploit_detector

Again, you may have to use the sudo command as above.

Technical users

You can also clone the Git repository and run the script directly like so: -

git clone https://github.com/polaris64/web_exploit_detector
cd web_exploit_detector
npm install

Running

From NPM module

If you have installed Web Exploit Detector as an NPM module (see above) then running the scanner is as simple as running the following command, passing in the path to your webroot (location of your website files): -

wed-scanner --webroot=/var/www/html

Other command-line options are available, simply run wed-scanner --help to see a help message describing them.

Running the script in this way will produce human-readable output to the console. This is very useful when running the script with cron for example as the output can be sent as an e-mail whenever the script runs.

The script also supports the writing of results to a more computer-friendly JSON format for later processing. To enable this output, see the --output command line argument.

From cloned Git repository

Simply call the script via node and pass the path to your webroot as follows: -

node index.js --webroot=/var/www/html

Usage as a module

The src/web-exploit-detector.js script is an ES6 module that exports the set of rules as rules as well as a number of functions: -

• executeTests(settings): runs the exploit checker based on the passed settings object. For usage, please consult the index.js script.
• formatResult(result): takes a single test result from the array returned from executeTests() and generates a string of results ready for output for that test.
• getFileList(path): returns an array of files from the base path using readDirRecursive().
• processRulesOnFile(file, rules): processes all rules from the array rules on a single file (string path).
• readDirRecursive(path): recursive function which returns a Promise which will be resolved with an array of all files in path and sub-directories.

The src/cli.js script is a simple command-line interface (CLI) to this module as used by the wed-scanner script, so reading this script shows one way in which this module can be used.

The project uses Babel to compile the ES6 modules in “src” to plain JavaScript modules in “lib”. If you are running an older version of Node.js then modules can be require()’d from the “lib” directory instead.

Building

The package contains Babel as a dev-dependency and the "build" and "watch:build" scripts. When running the "build" script (npm run build), the ES6 modules in "./src" will be compiled and saved to "./lib", where they are included by the CLI scripts.

The "./lib" directory is included in the repository so that any user can clone the repository and run the application directly without having to install dev-dependencies and build the application.

Excluding results per rule

Sometimes rules, especially those tagged with suspicion, will identify a clean file as a potential exploit. Because of this, a system to allow files to be excluded from being checked for a rule is also included.

The wed-results-to-exceptions script takes an output file from the main detector script (see the --output option) and gives you the choice to exclude each file in turn for each specific rule. All excluded files are stored in a file called wed-exceptions.json (in the user’s home directory) which is read by the main script before running the scan. If a file is listed in this file then all attached rules (by ID) will be skipped when checking this file.

For usage instructions, simply run wed-results-to-exceptions. You will need to have a valid output JSON from a previous run of the main detector first using the --output option.

For users working directly with the Git repository, run node results_to_exceptions.js in the project root directory.

Rule engine

The application operates using a collection of “rules” which are loaded when the application is run. Each rule consists of an ID, name, description, list of URLs, tags, deprecation flag and most importantly a set of tests.

Each individual test must be one of the following: -

• A regular expression: the simplest type of test, any value matching the regex will pass the test.
• A Boolean callback: the callback function must return a Boolean value indicating if the value passes the test. The callback is free to perform any synchronous operations.
• A Promise callback: the callback function must return a Promise which is resolved with a Boolean value indicating if the value passes the test. This type of callback is free to perform any asynchronous operations.

The following test types are supported: -

• “path”: used to check the file path. This test must exist and should evaluate to true if the file path is considered to match the rule.
• “content”: used to check the contents of a file. This test is optional and file contents will only be read and sent to rules that implement this test type. When this test is a function, the content (string) will be passed as the first argument and the file path will be passed as the second argument, allowing the test to perform additional file operations.

Expanding on the rules

As web-based exploits are constantly evolving and new exploits are being created, the set of rules need to be updated too. As I host a number of websites I am constantly observing new kinds of exploits, so I will be adding to the set of rules whenever I can. I run this tool on my own servers, so of course I want it to be as functional as possible!

This brings me onto the reasons why I have made this application available as an open-source project: firstly so that you and others can benefit from it and secondly so that we can all collaborate to contribute detection rules so that the application is always up to date.

Contributing rules

If you have discovered an exploit that is not detected by this tool then please either contact me to let me know or even better, write your own rule and add it to the third-party rule-set (rules/third-party/index.js), then send me a pull request.

Don’t worry if you don’t know how to write your own rules; the most important thing is that the rule gets added, so feel free to send me as much information as you can about the exploit and I will try to create my own rule for it.

Rules are categorised, but the simplest way to add your own rule is to add it to the third-party rule-set mentioned above. Rule IDs are written in the following format: "author:type:sub-type(s):rule-id". For example, one of my own rules is "P64:php:cms:wordpress:wso_webshell". "P64" is me (the author), "php:cms:wordpress" is the grouping (a PHP-specific rule, for the Content Management System (CMS) called WordPress) and "wso_webshell" is the specific rule ID. When writing your own rules, try to follow this format, and replace "P64" with your own GitHub username or other unique ID.

Unit tests and linting

The project contains a set of Jasmine tests which can be run using npm test. It also contains an ESLint configuration, and ESLint can be run using npm run lint.

When developing, tests can also be run whenever a source file changes by running npm run watch:test. To run tests and ESLint, the npm run watch:all script can be used.

Please note that unless you already have Jasmine and/or nodemon installed, you should run npm install in non-production mode to ensure that the dev-dependencies are installed.