Software Testing

Spring 2024 — Schedule

Tentative class schedule (may be updated without prior notice)

Note: AO-Ch x means chapter x in Ammann and Offutt text book
K-Ch x means chapter x in Koskela's Test Driven (This is recommended reading. You don't have to buy the book. Free trial available.)

Meet	Date	Topics and Handouts	Readings
Course overview and intro to software testing
1	Wed 01/17	Get Familiar with Our Course, Let's Set Expectation A bit about this course and how we shall work together
2	Fri 01/19	Why Test? Who cares? So what?	AO-Ch1 Some examples of why CWE Top 25 Most Dangerous Software Weaknesses
3	Mon 01/22	Intro to Software Testing What is Software Testing? What are differences between testing and debugging? What are some testing categories? How mature is your testing?	AO-Ch1, AO-Ch2.1
4	Wed 01/24	Faults−Errors−Failures, and RIPR model What aspects must be analyzed when designing tests and how do we analyze them?	AO-Ch1, AO-Ch2.1
5	Fri 01/26	Model Driven Test Design (MDTD) How does testing fit in the software development process? What are testing activities? How do we work at a higher level of abstraction, using mathematical structures to design tests?	AO-Ch2
Test automation and test framework
6	Mon 01/29	Test Automation Designing for testability Test code quality How do we test software automatically? What makes test automation hard? What are characteristics of good tests?	AO-Ch3 Flaky tests
7	Wed 01/31 Add deadline 01/31	JUnit One of the most commonly used test automation frameworks for Java	AO-Ch3, JUnit 5 user guide
8	Fri 02/02	JUnit Let's explore more JUnit features	AO-Ch3
9	Mon 02/05	Selenium One of the most commonly used test automation frameworks for web apps	Selenium documentation
10	Wed 02/07	Selenium Let's explore more Selenium features
Test design, generation, execution, evaluation and analysis, and improvement
11	Fri 02/09	Coverage-based Test Design How do we know if we have tested enough? Why coverage criteria? How should we design tests? Why don't we need all possible tests? How to design tests using coverage criteria?	AO-Ch.5
12	Mon 02/12	Coverage-based Test Design in Action Let's put the concept into action — invent our own coverage criterion, derive tests to satisfy the criterion, execute and evaluate our tests, and analyze the coverage level.	AO-Ch.5
Input Space Partitioning (ISP) Testing
13	Wed 02/14	Input Space Partition (ISP) Testing How do we design tests without having source code (of a program under test)? Is ISP black-box testing? An Industrial Study of Applying Input Space Partitioning to Test Financial Calculation Engines (This is an example of how the concepts have been applied to industry; recommended reading, it will not be tested)	AO-Ch6.1
14	Fri 02/16	ISP: Coverage Criteria All Combination Each Choice Pair-Wise Base Choice Multiple Base Choice Which ISP coverage criterion should we use? Why? How do we apply the criterion to derive tests? How many tests do we need?	AO-Ch6.2, 6.3
15	Mon 02/19	General Ideas: Web App Testing ISP: Apply to Web Apps Can we test web apps the same way we test traditional software? How should we test What are web-specific features that must be taken into account when designing tests? Here is one way to test web apps systematically.
Graph-based Testing
16	Wed 02/21	Graph-based Testing Graph: Structural Coverage Criteria Node Edge One of the most commonly used coverage criteria	AO-Ch7.1 AO-Ch7.2
17	Fri 02/23	Graph: Structural Coverage Criteria Per our class conversation on 21-Feb, we decided to move today's topics to Monday. No class meeting on 23-Feb. Students will work on assignment 3.
18	Mon 02/26	Graph: Structural Coverage Criteria Edge-Pair Complete Path Prime Path How should we handle loops in graphs? Execute loop once, 0 time, many times?	AO-Ch7.2
19	Wed 02/28 Drop deadline 02/28	Structural Graph Coverage: Apply to Source Code How do we create graph representation of the software under test? Let's put graph coverage criteria into action.	AO-Ch7.3.1
20	Fri 03/01	Graph: Data Flow Coverage Criteria All-Defs All-Uses All-DU-Paths How do we analyze data flow of a software artifact and integrate them into a graph model? How do we design tests to ensure that the program states / values are created and used correctly?	AO-Ch7.2
	03/02−03/10	Spring recess, no class (refer to UVA Academic Calendar)
21	Mon 03/11	Data flow Coverage: Apply to Source Code Putting it all together and designing unit-level tests to satisfy data flow coverage criteria.	AO-Ch7.3.2
22	Wed 03/13 Withdraw deadline 03/13	Graph Coverage for Design Elements Software usually consists of multiple components (or modules or subsystems) and there are data flow couplings between these components. How do we design tests beyond unit-level testing? Graph coverage for Specification How do we apply graph coverage criteria to other kinds of software artifacts / non-source code? — (Recommended reading. Read on your own. It will not be tested)	AO-Ch7.4 AO-Ch7.5
Logic-based Testing
23	Fri 03/15	Logic-based Testing Logic: Coverage Criteria Predicate Clause Combinatorial How should we design tests based on semantic of the logic expressions? Will tests designed for equivalent predicates always be the same?	AO-Ch8.1
24	Mon 03/18	Logic: Determination A clause determines the value of its predicate when the other clauses have certain values. How do we know what values the other clauses should have? Under which condition(s) will the clause determine the predicate?	AO-Ch8.1
25	Wed 03/20	Logic: Active Clause Coverage Criteria (ACC) GACC RACC CACC When some aspects are supposed to impact the behavior of a software under test, how should we design tests to check if they actually do impact? ACC – A form of "Modified Condition Decision Coverage" (MCDC), required by the US Federal Aviation Administration (FAA) for safety critical avionics software	AO-Ch8.1
26	Fri 03/22	Logic: Apply to Source Code How do we identify test requirements according to logical expressions found in program source code? How should we derive tests to ensure proper evaluation of the expressions? How are reachability and controllability issues handled when applying logic coverage criteria?	AO-Ch8.3
27	Mon 03/25	Logic: Apply to Source Code Can we reuse tests when a software is refactored (or gets evolved)? How do we analyze? How much can we reuse? Some ideas toward regression testing.	AO-Ch8.3
28	Wed 03/27	Logic: Apply to Source Code More practice with logic-based testing for source code	AO-Ch8.3
Syntax-based Testing
29	Fri 03/29	Syntax-based testing: Grammar-based mutation How to design tests based on the syntax of the software artifacts. How do we design valid and invalid test inputs using regular expressions and context free grammars?	AO-Ch9.1
30	Mon 04/01	Syntax: Apply to input space Let's mutate the software artifact, apply grammar-based mutation testing to some simple real-world scenarios	AO-Ch9.1
31	Wed 04/03	Syntax-based testing: Program-based mutation Why do we need variations of the software under tests? Why mimic developers' common mistakes or difficult to detect faults. How to mimic the mistakes and encourage common test heuristics.	AO-Ch9.2, 9.3
32	Fri 04/05	Syntax: Apply to Source Code Though mutation testing is the strongest coverage criterion, it is very expensive. What can we do to reduce the testing cost? How is it used in practice? Let's mutate the software artifact to create effective test requirements	AO-Ch9.2, 9.3
33	Mon 04/08	Syntax: Apply to Source Code More practice on mutation testing.	AO-Ch9.2, 9.3
34	Wed 04/10	Putting it all together RIPR → mutation testing → test automation	AO-Ch9.2, 9.3
34	Wed 04/10	Winners: Phillip Bongiorno, Ningxi Huang, Camryn Kim, Mia McCarrick, Maria Salamone, Tsvetelin Tsvetkov Congratulations!! Please remember to claim your prize. (winners ordered by last name)
Testing as central activity, TDD
35	Fri 04/12	Shift Testing Left, Test-Driven Development (TDD) Users always change their mind and software continues to evolve. How do we use testing as central activity to embrace evolutionary design? How do we use test harnesses as guideline in software development?	AO-Ch4, K-Ch1 to Ch4, TDD (wikipedia)
36	Mon 04/15	TDD More TDD practice. How do we use tests to guide the refactoring? We try to be productive with TDD, but when not to use TDD.	K-Ch1 to Ch4
37	Web 04/17	TDD web app We have used TDD to build a standalone Java program. How do we use TDD to build a web app?	K-Ch1 to Ch4
38	Fri 04/19	Test Doubles What do we do if some parts of the software are unavailable but we need them to test another part? Using stubs, fakes, and mocks to simplify testing. What to mock, when to mock, and when not to mock.
Course wrap-up
39	Mon 04/22	Bypass testing Although client-side and server-side input validations can be used as double safeguard to prevent unacceptable users' entries, users can still bypass the validations. How should we design tests targeting the validations and web-specific constraints?
39	Wed 04/24	Course wrap-up and final exam guide Q&A. No additional topic/discussion.
41	Fri 04/26	Q&A, Office hours on demand No class meeting; feel free to drop by to chat or discuss about anything.
42	Mon 04/29	Last minutes Q&A No class meeting. Please use Piazza for Q&A.
FE	Tue 05/07	Final Exam: Tuesday, May 7, 2024, 9:00am - 12:00pm EST, Room: Rice 130 Please refer to Course wrap-up and final exam guide for time and detail (please refer to the university final exam schedule)

CC-BY-NC-SA 4.0 license.
Last updated 2024-04-29 10:42

Top