System Security Auditing/Monitoring Tools

Common Criteria

The Common Criteria evolved out of ITSEC, CTCPEC, and TCSEC, from Europe, Canada, and the U.S.

ITSEC was developed in the early 1990s by France, Germany, the Netherlands, and the UK.

TCSEC, or the Trusted Computer System Evaluation Criteria, is U.S. Department of Defense standard DoD 5200.28, famously called the Orange Book. CTCPEC was a Canadian standard that started with TCSEC and fixed some problems.

The Orange Book largely defined systems that are too locked down to be useful. CTCPEC and ITSEC defined far more practical systems. Common Criteria used the three as a starting point.

Common Criteria or CC is formalized as an international standard, ISO/IEC 15408.

The Common Criteria defines EALs or Evaluation Assurance Levels. These specify how thoroughly the product has been analyzed and tested. There are EAL1 through EAL7, higher numbers are better.

The simple wording here makes it look like a semantic game. The Wikipedia description provides useful further details. As it says, commercial operating systems are typically evaluated as EAL4. See the formal definitions for meaning, if you can afford the rather expensive ISO/IEC documents.

Speaking of expensive, prices for anything above EAL4 rise quickly. And, choice drops. The INTEGRITY-178B real-time operating system is beyond closed-source, with classified source code. It's used in U.S. military aircraft, and an unclassified version is used in the Airbus A380. It's rated as EAL 6+, which the manufacturer says is the highest-ever EAL rating for an operating system. That's right: EAL 7 is defined, but nothing has been certified as meeting it.

That level of stability means giving up features like dynamic memory allocation.

FIPS 140-2

FIPS 140-2 or the Federal Information Process Standard Publication 140-2, "Security Requirements for Cryptographic Modules", specifies four levels of security. Level 1 means that it has been analyzed and tested for correctness — it will produce the correct output when a given input.

Moving beyond Level 1 gets into tamper-evident systems, then tamper-proof, automated detection and response to access attempts, and finally automated immediate zeroization of plaintext data. The increasing levels come with increasingly rigorous testing for adequacy, effectiveness, and completeness.

FIPS 140-2 Level 1 can be done in software. For example, the Linux kernel contains cryptographic modules that can be certified, but only in an environment where the operating system is supported by a trusted provider. CentOS Linux is exactly the same software as Red Hat Enterprise Linux, minus the Red-Hat-specific boot splash screen and graphic desktop backgrounds. The U.S. Department of Defense trusts the Red Hat company to provide appropriate patches and other support, it does not trust the organization that is the CentOS project, even though CentOS joined with Red Hat in January 2014.

US DOD STIGs and Evaluation and Mitigation Tools

DISA or Defense Information Systems Agency, part of the U.S. Department of Defense, maintains a wide range of documents known as the STIGs — Security Technical Implementation Guide. These describe DISA's view of best practices in the form of checklists. There are STIGs for operating systems, for network services, for use of biometrics, for instant messaging, for configuring browsers, and more. You can download the STIG documents from DISA's site.

If you're in US DOD or a contractor to it, you will be regularly audited. If you convince the auditor that you're following the official checklist items, the audit will succeed quickly. If you don't follow a checklist item, that may be acceptable as long as you can provide an explanation of why you are doing that and what you are doing to provide the needed security in that area.

If you are not connected to DOD, then the STIGs may seem somewhat paranoid and of little interest. But they do provide a good starting point for your policy. At the very least, they provide an organized set of concerns to be addressed in your policy.

If you were in US DOD or are a contractor, it used to be that DISA would give you their "Gold Disk" and its SRR or the System Requirements Review checklist, and the associated scripts that you could run to automate the process. However, unless you were part of DOD or one of their contractors, you could not download the SRR scripts. You could certainly use Google to locate what claim to be the SRR scripts for your operating system, but you can't tell if they really are the real ones or not.

The plan was to replace the Gold Disk scripts with new SRR software that tests a system for compliance to the STIG. And that leads us to...

SCAP Products

SCAP is the Security Content Automation Protocol. SCAP is much more than just a communication protocol. It is a method for automating vulnerability detection, patch and configuration management, policy compliance, and the communication between the needed components and for the generation of reports.

The U.S. Government pays contractors to test products and validate those capable of managing the Federal Desktop Core Configuration or FDCC control of system patching and configuration.

Printer Security

Many printers run complex operating systems that have many capabilities, some of them unneeded or even unwanted. There is a very limited interface, and it is often impractical to patch them. Most of them run unattended and ignored, beyond feeding them paper and toner as needed.

Hacking Printers Printer Security Testing Cheat Sheet

Apple OS X Security Auditing Tools

osxlockdown was built to audit and remediate security configuration settings on OS X 10.11 (El Capitan).

Windows System Security Auditing Tools

There used to be a number of these types of tools, but they have disappeared. The Windows Security Configuration and Analysis Snap-In is one of the few tools available.
Introduction to the Security Configuration and Analysis Snap-In Applying security policies with the Security Configuration and Analysis Snap-In.

Windows Server 2003 has a Security Configuration Wizard. Get to know Windows Server 2003's Security Configuration Wizard Using the Secedit tool to work with security templates

Registry Live Watch can monitor the Windows registry and alert you to any changes. Someone told me that they had installed the PACE Anti-Piracy software, which then stealthily modifies the registry on some hidden schedule. If you know when the registry is being changed, then you have a chance of figuring out what is making the change.

PACE Anti-Piracy is described as software which "installs a kernel-level driver onto your Windows machine, does its best to blend into the woodwork (the device driver is named "TPKD", the support files get buried in a common-apps directory, and at no point in the its process of validating a trial period does it display its name or other information to the user), and uses some method of compiling unique information about your computer in order to do its anti-piracy thing. And there's no obvious way to get the PACE Anti-Piracy software off of your computer once it's there."

Windows running inside the QEMU emulator on an OpenBSD desktop.

Network Vulnerability Scanners

Network Vulnerability Scanners

Lynis — Open-source and enterprise security auditing toolkits for Linux and other Unix-family operating systems

Lynis runs on Linux; the BSD family of OpenBSD, FreeBSD, NetBSD and others; macOS; and commercial Unixes including HP-UX and Solaris.

There is an open-source version, a collection of scripts.

Lynis Enterprise adds centralized management and reporting, further plugins, integration with other projects, and support.

Older Open-Source Tools for Linux/UNIX Family Security Auditing

Be aware that most of these have not been updated since the mid to late 2000s! For example, in mid-2014 I noticed that Bastille had not been updated since 2008.

TARA, the Tiger Analytical Research Assistant, is an automated system administrator's assistant.

COPS has been a standard auditing tool, although it's getting awfully old.

Titan automatically changes your system configuration to increase security, possibly breaking some functionality.

Bastille takes you through a series of questions, educating you along the way, possibly making configuration changes to increase security.

More tool FTP sites: Purdue CERIAS U.S. Department of Energy funet.fi