This schedule is an estimate. It may be updated based on the actual pacing of the course material.

DateTopicAssignment
Week 1
Mon 13 Jan

Overview / Malware Terminology

 [ 
slides: intrologistics
| screencapture (browser  or download mp4 webm audio ) ]
  • malware, generally
  • course overview, logistics
  • (if time) x86-64 assembly review
References:
Wed 15 Jan

x86 asm

 [ 
slides: asm
| screencapture (browser  or download mp4 webm audio ) ]
  • AT&T, Intel syntax
  • official sources for calling convention info
  • some x86-64 nits
    • floating point/SSE2
References:
Fri 17 Jan

x86 asm 2 / executable formats

 [ 
slides: asmexec-encoding
| screencapture (browser  or download mp4 webm audio ) ]
  • misc. ASM things (label(%rip); segmentation)
  • ELF executable format (start)
References:
RE1 released
Week 2
Mon 20 Jan
(no class)
Wed 22 Jan

executable formats / x86-64 encoding

 [ | screencapture (browser  or download mp4 webm audio ) ]
  • ELF executable format (finish)
  • dynamic loading realities
    • procedure linkage table
    • global offset table
  • x86-64 encoding (start)
Quiz week02 (quiz for week 2) released, due 2025-01-27 12:45
Fri 24 Jan

x86-64 encoding / Reverse engineering tools

 [ 
slides: re-tools
| screencapture (browser  or download mp4 webm audio ) ]
  • x86-64 encoding (finish)

  • Ghidra as example
  • annotation, tricky cases for disassembly
  • control flow graphs
  • intermediate representation
  • decompiling
RE1 due by 11:59pm
RE2 released
Week 3
Mon 27 Jan

Virus

 [ 
slides: virus
| screencapture (browser  or download mp4 webm audio ) ]
  • virus example: Vienna
Quiz week02 (quiz for week 2) due 12:45 (released 2025-01-22)
Wed 29 Jan

Virus 2

 [ 
slides: virus
| screencapture (browser  or download mp4 webm audio ) ]
  • options for viruses/worms:
  • where to put code
    • appending, replacing
    • “cavities” in executables
    • system files, bootloaders
  • how to get to code to run
    • bootloaders
    • replacing jumps, returns
    • dynamic linking information
    • default start-up program
Quiz week03 (quiz for week 3) released, due 2025-02-03 12:45
Fri 31 Jan

Heuristic detection

 [ 
slides: heur-detect
| screencapture (browser  or download mp4 webm audio ) ]
  • whitelisting
  • signatures as regexes
RE2 due by 11:59pm
TRICKY released
Week 4
Mon 03 Feb

Signatures

 [ 
slides: heur-detect
| screencapture (browser  or download mp4 webm audio ) ]
  • state machines for pattern matching
  • combining patterns
  • heuristic matching on “weird” executables
  • behavior monitoring
Quiz week03 (quiz for week 3) due 12:45 (released 2025-01-29)
Wed 05 Feb

Anti-signature/analysis techniques

 [ 
slides: antianti
| screencapture (browser  or download mp4 webm audio ) ]
  • RE2, part B
  • obfuscation techniques to hide flow-control
    • merging/splitting functions
    • Tigress’s flatten pattern
  • “encrypted” code
Quiz week04 (quiz for week 4) released, due 2025-02-10 12:45
Fri 07 Feb

Anti-signature/analysis techniques

 [ 
slides: antiantire-tools
| screencapture (browser  or download mp4 webm audio ) ]
  • “encrypted” code and decrypter generators
  • emulation as analysis tool
  • anti-emulation/virtualization techniques
  • (start) “mutation engines”
TRICKY due by 11:59pm
LEX released
Week 5
Mon 10 Feb

Anti-signature/analysis techniques

 [ 
slides: antianti
| screencapture (browser  or download mp4 webm audio ) ]
  • (finish) “mutation engines”
  • anti-debugging techniques
  • rootkit-style malware
Quiz week04 (quiz for week 4) due 12:45 (released 2025-02-05)
Wed 12 Feb

Command injection / Basic overflows

 [ | screencapture (browser  or download mp4 webm audio ) ]
  • command injection vulnerabilities
  • dynamic taint tracking
  • buffer overflows
Quiz week05 (quiz for week 5) released, due 2025-02-17 12:45
Fri 14 Feb

Stack smashing 1

 [ | screencapture (browser  or download mp4 webm audio ) ]
  • stack smashing pattern
OBFUSCATE released
Week 6
Mon 17 Feb

Stack smashing 2 / Shellcode / Stack Canaries

 [ | screencapture (browser  or download mp4 webm audio ) ]
  • stack smashing pattern (finish)
  • dealing with restrictions on shellcode
  • stack canaries
Quiz week05 (quiz for week 5) due 12:45 (released 2025-02-12)
LEX due by 11:59pm
Wed 19 Feb

Skipping Stack Canaries / Information Disclosure / Shadow Stacks

 [ | screencapture (browser  or download mp4 webm audio ) ]
  • notes on quiz; reviewing stack canary overwrite
  • information disclosure
  • shadow stacks
Quiz week06 (quiz for week 6) released, due 2025-02-24 12:45
Fri 21 Feb

Pointer subterfuge 1

 [ | screencapture (browser  or download mp4 webm audio ) ]
  • pointers on the stack
  • “write gadgets”
  • targets for write gadgets
    • global offset table
    • VTables (start)
OBFUSCATE due by 11:59pm
OVER released
Week 7
Mon 24 Feb

Pointer subterfuge 2

 [ | screencapture (browser  or download mp4 webm audio ) ]
  • targets for write gadgets
    • VTables (finish)
      • exercise
  • arc injection
  • ntpd exploit example
  • exercise
Quiz week06 (quiz for week 6) due 12:45 (released 2025-02-19)
Wed 26 Feb

More buffer overflow mitigations (guard pages / write XOR execute)

 [ | screencapture (browser  or download mp4 webm audio ) ]
  • memory protection, guard pages
  • write XOR execute
Quiz week07 (quiz for week 7) released, due 2025-03-03 12:45
Fri 28 Feb

More buffer overflow mitigations (ASLR)

 [ 
slides: mitigate-aslr
| screencapture (browser  or download mp4 webm audio ) ]

(Reiss out of town)

  • ASLR: making addresses hard to predict
    • limits on entropy
    • things that must be kept together
OVER due by 11:59pm
SUBTERFUGE released
Week 8
Mon 03 Mar

Return oritented programming 1

 [ 
slides: rop
| screencapture (browser  or download mp4 webm audio ) ]
  • idea of using gadgets
  • chaining gadgets together
Quiz week07 (quiz for week 7) due 12:45 (released 2025-02-26)
Wed 05 Mar

return-oriented programming 2

 [ 
slides: rop
| screencapture (browser  or download mp4 webm audio ) ]
  • finding gadgets automatically
  • automatic chain generation
  • gadgets without RET
Fri 07 Mar

return-oriented programming 3

 [ 
slides: rop
| screencapture (browser  or download mp4 webm audio ) ]
SUBTERFUGE due by 11:59pm
ROP released
Week 9
Mon 10 Mar
(no class)
Wed 12 Mar
(no class)
Fri 14 Mar
(no class)
Week 10
Mon 17 Mar

heap structure exploits

 [ 
slides: overflow-heap
| screencapture (browser  or download mp4 webm audio ) ]
  • overflows on the heap
    • into other objects (sudo example)
    • into heap metadata
  • consistency and the heap
  • pointer subterfuge via heap metadata
Wed 19 Mar

heap structure exploits / use-after-free

 [ 
slides: overflow-heapuaf
| screencapture (browser  or download mp4 webm audio ) ]
  • pointer subterfuge via heap metadata (con’t)
  • double-free vulnerabilities
  • use-after-free
    • prevalence
    • using type confusion
Quiz week10 (quiz for week 10) released, due 2025-03-24 12:45
Fri 21 Mar

use-after-free / integer overflow

 [ 
slides: uafoverflow-int
| screencapture (browser  or download mp4 webm audio ) ]
  • use-after free exercises / examples

  • integer overflow

ROP due by 11:59pm
UAF released
Week 11
Mon 24 Mar

memory-safe languages 1

 [ 
slides: betterpl
| screencapture (browser  or download mp4 webm audio ) ]
  • integer overflow (con’t)

  • why people like C/C++

  • Rust intro

  • ownership rule

Quiz week10 (quiz for week 10) due 12:45 (released 2025-03-19)
Wed 26 Mar

memory-safe languages 2

 [ 
slides: betterpl
| screencapture (browser  or download mp4 webm audio ) ]
  • Rust borrowing
Quiz week11 (quiz for week 11) released, due 2025-03-31 12:45
Fri 28 Mar

memory-safe languages 3

 [ 
slides: betterpl
| screencapture (browser  or download mp4 webm audio ) ]
  • escape hatch in Rust; smart pointer
  • Rc (reference counting)
  • RefCell (dynamic borrow tracking)
UAF due by 11:59pm
RUST released
Week 12
Mon 31 Mar

coverage-guided (fuzz-)testing

 [ 
slides: testing
| screencapture (browser  or download mp4 webm audio ) ]
  • greybox fuzz testing
Quiz week11 (quiz for week 11) due 12:45 (released 2025-03-26)
Wed 02 Apr

symbolic/concolic execution 1

 [ 
slides: symbolic
| screencapture (browser  or download mp4 webm audio ) ]
  • symbolic execution idea
    • solving equations
  • splitting execution on if statements
  • collecting/solving for constraints
  • automatic overflows
  • optimizing symbolic execution in practice
Quiz week12 (quiz for week 12) released, due 2025-04-07 12:45
Fri 04 Apr

static analysis / sanboxing 0

 [ | screencapture (browser  or download mp4 webm audio ) ]
  • briefly static analysis — tracking approximations
    • example for use-after-free
    • points-to analysis difficulty
  • sandboxing idea
  • problem of applications doing too much
  • privilege seperation intro
RUST due by 11:59pm
FUZZ released
Week 13
Mon 07 Apr

sandboxing 1

 [ 
slides: sandbox
| screencapture (browser  or download mp4 webm audio ) ]
  • challenges with selecting system call filters
  • privilege seperation con’t
    • example interface
    • limits on what it does/does not mitigate
Quiz week12 (quiz for week 12) due 12:45 (released 2025-04-02)
Wed 09 Apr

sandboxing 2

 [ 
slides: sandbox
| screencapture (browser  or download mp4 webm audio ) ]
  • limiting naming — chroot, Linux namespaces

  • sandbox escapes (start)

Quiz week13 (quiz for week 13) released, due 2025-04-14 12:45
Fri 11 Apr

sandboxing 3

 [ 
slides: sandbox
| screencapture (browser  or download mp4 webm audio ) ]
  • sandbox escapes (finish)
  • whole-application sandboxing
  • usability issues with mobile permissions
  • sandboxing without OS help
FUZZ due by 11:59pm
SANDBOX released
Week 14
Mon 14 Apr

fast? bounds checking 1

 [ 
slides: bounds
| screencapture (browser  or download mp4 webm audio ) ]
  • “fat” pointers (pointer has objcet bounds)
  • baggy bounds checking (lookup object bounds)
Quiz week13 (quiz for week 13) due 12:45 (released 2025-04-09)
Wed 16 Apr

fast? bounds checking 2

 [ 
slides: bounds
| screencapture (browser  or download mp4 webm audio ) ]

Note: there are more detailed answer slides for the last exericse in lecture in the slide PDF (added after lecture).

  • baggy bounds checking (con’t)
  • AddressSanitizer
  • comparing bounds checking
Quiz week14 (quiz for week 14) released, due 2025-04-21 12:45
Fri 18 Apr

control-flow integrity 1

 [ 
slides: cfi
 ]
SANDBOX due by 11:59pm
Week 15
Mon 21 Apr

control flow integrity 2

 [ 
slides: cfi
 ]
Quiz week14 (quiz for week 14) due 12:45 (released 2025-04-16)
Wed 23 Apr

[tentative] same-origin policy and XSS mitigation

Quiz week15 (quiz for week 15) released, due 2025-04-28 12:45
Fri 25 Apr

[tentative] same-origin policy and XSS mitigation

Week 16
Mon 28 Apr

Exam review

Quiz week15 (quiz for week 15) due 12:45 (released 2025-04-23)
Tue 29 Apr
(no class)
CHALLENGE due by 11:59pm
Week 17
Wed 05 May

Final exam

9AM-12PM.