Software Analysis and Testing

Course Overview

Aspects of software development besides programming, such as diagnosing bugs, testing, and debugging, comprise over 50% of the cost of software development. Modern technology can help automate these aspects and thereby improve programmer productivity and software reliability. At the heart of this technology lies software analysis: a body of work that encompasses techniques for automatically discovering facts about a given program. Many diverse such techniques exist with their own strengths and limitations. This course teaches the principles underlying these techniques as well as imparts hands-on experience with applying them to automate testing, debugging, and finding bugs in complex real-world programs.

Intended Audience

This is a graduate-level course intended for students who wish to:
  • prepare for research in the areas of programming languages, compilers, or software engineering;
  • conduct research in other areas of computer science (e.g., security or systems) that involve applying software analysis; and
  • become more productive software engineers in industry.

Learning Objectives

Upon successfully completing the course, you will be able to:
  • Describe the abstract properties of different techniques for analyzing and testing software.
  • Compute the outcome of the techniques on concrete examples.
  • Evaluate the suitability of different techniques for a given software and/or set of constraints.
  • Run analysis and testing tools on actual software and interpret their results.

Pre-Requisites

The lectures require basic knowledge of discrete mathematics (e.g., set theory, logic, and probability).
The assignments presume experience with the following:
  • Programming in Java, C/C++, or equivalent (most assignments will use Java).
  • Using virtual machines (e.g., VirtualBox or VMWare).
  • Writing and running shell scripts.

Lessons

The lectures consist of 11 lessons delivered in two forms: videos (streaming from YouTube) and slides (in PDF and PPTX formats).

 #  Topic Videos Slides
URL Size (mm:ss) PDF PPTX
L1 Introduction to Software Analysis [video] 19:36 [PDF] [PPTX]
L2 Introduction to Software Testing [video] 33:17 [PDF] [PPTX]
L3 Random Testing [video] 35:28 [PDF] [PPTX]
L4 Automated Test Generation [video] 45:52 [PDF] [PPTX]
L5 Dataflow Analysis [video] 40:00 [PDF] [PPTX]
L6 Pointer Analysis [video] 50:09 [PDF] [PPTX]
L7 Constraint-Based Analysis [video] 40:53 [PDF] [PPTX]
L8 Type Systems [video] 59:37 [PDF] [PPTX]
L9 Statistical Debugging [video] 51:45 [PDF] [PPTX]
L10 Delta Debugging [video] 35:51 [PDF] [PPTX]
L11 Dynamic Symbolic Execution [video] 46:47 [PDF] [PPTX]

Assignments

There are 8 assignments to complement the lectures and impart hands-on experience. Each assignment is of one of the following two kinds:
  • Using a software analysis tool based on a technique covered in lectures, reporting your findings, and answering conceptual questions.
  • Implementing a software analysis algorithm in a high-level programming language (usually Java).
A VM image compatible with both Virtualbox and VMWare is provided for relevant assignments. (It should also work with other virtualization software but only these two are officially supported.)

 #  Title Associated Lesson
A1DafnyL1 (Intro to Software Analysis)
A2MonkeyL3 (Random Testing)
A3KoratL4 (Automated Test Generation)
A4RandoopL4 (Automated Test Generation)
A5DataflowL5 (Dataflow Analysis)
A6ConstraintsL7 (Constraint-Based Analysis)
A7CBIL9 (Statistical Debugging)
A8DeltaL10 (Delta Debugging)

Questions or Feedback?

We are happy to answer questions and receive feedback from anyone who uses material from this course. Please email your comments to mhnaik@cis.upenn.edu.

Acknowledgments

This course was created by Mayur Naik together with assistance from Udacity course developer Chris Pryby as part of Georgia Tech's Online Masters Program in Computer Science (OMSCS).