Hex dump of malware content.

System Security Auditing/Monitoring Tools

Common Criteria

Also see:
OS-specific security
Also see:
Cloud security standards

The Common Criteria evolved out of ITSEC, CTCPEC, and TCSEC, from Europe, Canada, and the U.S.

ITSEC was developed in the early 1990s by France, Germany, the Netherlands, and the UK.

TCSEC, or the Trusted Computer System Evaluation Criteria, is U.S. Department of Defense standard DoD 5200.28, famously called the Orange Book. CTCPEC was a Canadian standard that started with TCSEC and fixed some problems.

The Orange Book largely defined systems that are too locked down to be useful. CTCPEC and ITSEC defined far more practical systems. Common Criteria used the three as a starting point.

Common Criteria or CC is formalized as an international standard, ISO/IEC 15408.

The Common Criteria defines EALs or Evaluation Assurance Levels. These specify how thoroughly the product has been analyzed and tested. There are EAL1 through EAL7, higher numbers are better.

The simple wording here makes it look like a semantic game. The Wikipedia description provides useful further details. As it says, commercial operating systems are typically evaluated as EAL4. See the formal definitions for meaning, if you can afford the rather expensive ISO/IEC documents.

Dark Reading on Integrity OS EAL6+

Speaking of expensive, prices for anything above EAL4 rise quickly. And, choice drops. The INTEGRITY-178B real-time operating system is beyond closed-source, with classified source code. It's used in U.S. military aircraft, and an unclassified version is used in the Airbus A380. It's rated as EAL 6+, which the manufacturer says is the highest-ever EAL rating for an operating system. That's right: EAL 7 is defined, but nothing has been certified as meeting it.

That level of stability means giving up features like dynamic memory allocation.

EAL1 Functionally tested
EAL2 Structurally tested
EAL3 Methodically tested and checked
EAL4 Methodically designed, tested, and reviewed
EAL5 Semi-formally designed and tested
EAL6 Semi-formally verified design and tested
EAL7 Formally verified design and tested

FIPS 140-2

FIPS 140-2

FIPS 140-2 or the Federal Information Process Standard Publication 140-2, "Security Requirements for Cryptographic Modules", specifies four levels of security. Level 1 means that it has been analyzed and tested for correctness — it will produce the correct output when a given input.

Moving beyond Level 1 gets into tamper-evident systems, then tamper-proof, automated detection and response to access attempts, and finally automated immediate zeroization of plaintext data. The increasing levels come with increasingly rigorous testing for adequacy, effectiveness, and completeness.

FIPS 140-2 Level 1 can be done in software. For example, the Linux kernel contains cryptographic modules that can be certified, but only in an environment where the operating system is supported by a trusted provider. CentOS Linux is exactly the same software as Red Hat Enterprise Linux, minus the Red-Hat-specific boot splash screen and graphic desktop backgrounds. The U.S. Department of Defense trusts the Red Hat company to provide appropriate patches and other support, it does not trust the organization that is the CentOS project, even though CentOS joined with Red Hat in January 2014.

US DOD STIGs and Evaluation and Mitigation Tools

DISA or Defense Information Systems Agency, part of the U.S. Department of Defense, maintains a wide range of documents known as the STIGs — Security Technical Implementation Guide. These describe DISA's view of best practices in the form of checklists. There are STIGs for operating systems, for network services, for use of biometrics, for instant messaging, for configuring browsers, and more. You can download the STIG documents from DISA's site.


If you're in US DOD or a contractor to it, you will be regularly audited. If you convince the auditor that you're following the official checklist items, the audit will succeed quickly. If you don't follow a checklist item, that may be acceptable as long as you can provide an explanation of why you are doing that and what you are doing to provide the needed security in that area.

If you are not connected to DOD, then the STIGs may seem somewhat paranoid and of little interest. But they do provide a good starting point for your policy. At the very least, they provide an organized set of concerns to be addressed in your policy.

Linux STIG

If you were in US DOD or are a contractor, it used to be that DISA would give you their "Gold Disk" and its SRR or the System Requirements Review checklist, and the associated scripts that you could run to automate the process. However, unless you were part of DOD or one of their contractors, you could not download the SRR scripts. You could certainly use Google to locate what claim to be the SRR scripts for your operating system, but you can't tell if they really are the real ones or not.

The plan was to replace the Gold Disk scripts with new SRR software that tests a system for compliance to the STIG. And that leads us to...

SCAP Products

SCAP is the Security Content Automation Protocol. SCAP is much more than just a communication protocol. It is a method for automating vulnerability detection, patch and configuration management, policy compliance, and the communication between the needed components and for the generation of reports.

The U.S. Government pays contractors to test products and validate those capable of managing the Federal Desktop Core Configuration or FDCC control of system patching and configuration.

Vendor Product SCAP Validations
FDCC Scanner Authenticated Configuration Scanner Authenticated Vulnerability and Patch Scanner Unauthenticated Configuration Scanner
Arellia Security Analysis Solution
BigFix Security Configuration and Vulnerability Management Pack
bmc software BMC Automation Server
BMC BladeLogic Client Automation
CA Technologies CA IT Client Manager
The Center for Internet Security CIS Configuration Audit Tool
Core Security Technologies Core IMPACT Professional
Digital Defense Incorporated Frontline Vulnerability Manager
Dell KACE Dell KACE K1000 System Management Appliance
eEye Digital Security Retina
eIQ Networks SecureVue
Greenbone Greenbone Security Manager
HP SCAP Scanner
IBM Tivoli Endpoint Manager for Security and Compliance
LANDesk LANDesk Patch Manager 9.0 Extensions for Federal Desktops
LANDesk Security Suite 9.0 Extensions for Federal Desktops
Lumesion Security PatchLink Security Configuration Management for PatchLink Update
PatchLink Security Configuration Management for PatchLink Scan
McAfee Policy Auditor
Vulnerability Manager
Microsoft System Center Configuration Manager Extensions for SCAP
nCircle Configuration Compliance Manager
NetIQ NetIQ Secure Configuration Manager
Prism Microsystems EventTracker Enterprise
Qualys QualysGuard FDCC Scanner
Rapid 7 Nexpose
SAINT Vulnerability Scanner
Shavlik Shavlik Security Suite: Netchk Configure
Shavlik Security Suite: Netchk Protect
SignaCert Enterprise Trust Server
SPAWAR Systems Center Atlantic SCAP Compliance Checker
Symantec Control Compliance Suite
Symantec Risk Automation Suite
Telos Xacta IA Manager (Xacta Hostinfo)
Vendor Product FDCC Scanner Authenticated Configuration Scanner Authenticated Vulnerability and Patch Scanner Unauthenticated Configuration Scanner
SCAP Validations

Printer Security

Many printers run complex operating systems that have many capabilities, some of them unneeded or even unwanted. There is a very limited interface, and it is often impractical to patch them. Most of them run unattended and ignored, beyond feeding them paper and toner as needed.

Hacking Printers Printer Security Testing Cheat Sheet

Apple OS X Security Auditing Tools

osxlockdown was built to audit and remediate security configuration settings on OS X 10.11 (El Capitan).

Windows System Security Auditing Tools

There used to be a number of these types of tools, but they have disappeared. The Windows Security Configuration and Analysis Snap-In is one of the few tools available.
Introduction to the Security Configuration and Analysis Snap-In Applying security policies with the Security Configuration and Analysis Snap-In.

Windows Server 2003 has a Security Configuration Wizard. Get to know Windows Server 2003's Security Configuration Wizard Using the Secedit tool to work with security templates

Registry Live Watch can monitor the Windows registry and alert you to any changes. Someone told me that they had installed the PACE Anti-Piracy software, which then stealthily modifies the registry on some hidden schedule. If you know when the registry is being changed, then you have a chance of figuring out what is making the change.

PACE Anti-Piracy is described as software which "installs a kernel-level driver onto your Windows machine, does its best to blend into the woodwork (the device driver is named "TPKD", the support files get buried in a common-apps directory, and at no point in the its process of validating a trial period does it display its name or other information to the user), and uses some method of compiling unique information about your computer in order to do its anti-piracy thing. And there's no obvious way to get the PACE Anti-Piracy software off of your computer once it's there."

Network Vulnerability Scanners

Network Vulnerability Scanners

Lynis — Open-source and enterprise security auditing toolkits for Linux and other Unix-family operating systems

Lynis runs on Linux; the BSD family of OpenBSD, FreeBSD, NetBSD and others; macOS; and commercial Unixes including HP-UX and Solaris.

There is an open-source version, a collection of scripts.

Lynis Enterprise adds centralized management and reporting, further plugins, integration with other projects, and support.

Older Open-Source Tools for Linux/UNIX Family Security Auditing

Be aware that most of these have not been updated since the mid to late 2000s! For example, in mid-2014 I noticed that Bastille had not been updated since 2008.

TARA, the Tiger Analytical Research Assistant, is an automated system administrator's assistant.

COPS has been a standard auditing tool, although it's getting awfully old.

Titan automatically changes your system configuration to increase security, possibly breaking some functionality.

Bastille takes you through a series of questions, educating you along the way, possibly making configuration changes to increase security.

More tool FTP sites: Purdue CERIAS U.S. Department of Energy funet.fi