EAL4 Without Source Code?

TL;DR

Achieving EAL4 (or any Common Criteria certification) without source code access is extremely difficult, but not impossible. It relies heavily on black-box testing, reverse engineering, and strong assurance in the build process. Expect significantly higher costs and a longer evaluation timeline compared to evaluations with full source code.

Understanding EAL4

EAL4 (Evaluation Assurance Level 4) requires an independent assessment of a product’s security against identified threats. It focuses on design, implementation, and testing. Key areas include:

• Design Analysis: Reviewing the architecture to identify potential vulnerabilities.
• Code Review (usually): Examining source code for flaws. This is problematic without access.
• Vulnerability Testing: Attempting to exploit weaknesses.
• Penetration Testing: Simulating real-world attacks.
• Build Process Assurance: Verifying the integrity of the compilation and distribution chain.

Steps to EAL4 Without Source Code

1. Define a Clear Scope: Precisely define what parts of the application/OS are in scope for evaluation. The narrower the scope, the more manageable the process.
2. Reverse Engineering (Significant Effort): Disassemble and decompile the binary code to understand its functionality. Tools like IDA Pro or Ghidra can help.

   # Example using objdump (Linux) - very basic disassembly

   objdump -d application_name | less

   This is time-consuming and may not reveal the complete picture, especially with obfuscation techniques.

3. Black-Box Testing: Treat the software as a ‘black box’ – you know what it should do, but not how. Focus on:
4. Fuzzing: Providing invalid or unexpected inputs to identify crashes and errors. Tools like AFL (American Fuzzy Lop) are useful.

   # Example using AFL

   afl-fuzz -i input_directory -o output_directory //path/to/application

5. Penetration Testing: Employ ethical hackers to simulate real-world attacks.
6. Boundary Value Analysis: Test inputs at the edges of acceptable ranges.
7. Equivalence Partitioning: Divide input data into classes and test representative values from each class.
8. Build Process Verification (Critical): This becomes much more important without source code access.
   • Secure Development Environment: Audit the environment where the software is built.
   • Tamper-Proof Build System: Ensure the build process cannot be compromised. This might involve hardware security modules (HSMs) to protect signing keys.
   • Code Signing: Verify that all binaries are digitally signed by a trusted authority.
   • Dependency Analysis: Identify and assess the security of all third-party libraries used.

     # Example using DependencyCheck (Java)

     dependency-check scan --scan /path/to/application.jar --out output_directory

9. Static Binary Analysis: Use tools to analyze the binary code for potential vulnerabilities without executing it.
   • Binary Code Scanning Tools: Identify common security flaws (e.g., buffer overflows, format string bugs).
   • Control Flow Graph Analysis: Understand the program’s execution paths.
10. Documentation Review: Thoroughly review any available documentation (architecture diagrams, user manuals, API specifications) for inconsistencies or potential weaknesses.
11. Independent Assessment: Engage a Common Criteria accredited testing laboratory (CATL). They will perform the evaluation based on the defined scope and EAL4 requirements. Be prepared for extensive questioning and requests for evidence.

Challenges

• Completeness of Testing: Without source code, it’s impossible to guarantee complete test coverage.
• Obfuscation: If the software uses obfuscation techniques, reverse engineering becomes significantly harder.
• Cost and Time: Evaluations without source code are considerably more expensive and time-consuming.
• Trust Assumptions: You’ll need to make strong assumptions about the trustworthiness of the vendor and their build process.