Exam Test Case Variables

Overview

When you run a test cell during an exam, the autograder creates variables that you can use to debug your solution. These variables let you inspect the actual inputs and expected outputs for test cases.

This page is a quick reference

For detailed videos and additional grading information, see the supplemental course content:

• GT: Canvas > Modules > Supplemental Topic: How Student Code is Tested on Exams and Homework Notebooks
• EdX: Module 0: Fundamentals > Supplemental Topic: How Student Code is Tested on Exams and Homework Notebooks

How to Access Test Case Variables

Variable	Purpose
input_vars	The inputs passed to your function
original_input_vars	A copy of inputs before your function ran
returned_output_vars	What your function returned
true_output_vars	The expected (correct) output

Typical Workflow

1. Run solution/demo cell -> Confirm your output and the printed demo look the same
2. Run test cell
3. If it doesn't pass, access test case variables and identify differences, then use them to debug

Variables persist until they're overwritten

Since the test case variables have the same name for each exercise (but may have different keys), the variables will hold values until you run the relevant test cell. That means if you are skipping around and forget to run the test cell that generates the test case variables you're looking at, you may be looking at ones previously generated. This has happened to people before.

Accessing Variables

Each variable is a dict containing the inputs or outputs. To access, look at the variable's keys, then grab the value.

print(returned_output_vars.keys())

See what the key is, then you can access it via:

print(returned_output_vars[{key_name}])

For quick debugging, assign values to shorter variable names:

my_out = returned_output_vars['result']
true_out = true_output_vars['result']

Comparing Variables

Comparison approach depends on data type. Keep code short since copy/paste is disabled.

Lists/Sets

len(my_out), len(true_out)

my_out == true_out

set(my_out) - set(true_out)  # items in yours but not expected

Dictionaries

my_out.keys() == true_out.keys()

[k for k in my_out if my_out[k] != true_out[k]]  # differing keys

DataFrames

my_out.shape, true_out.shape

my_out.equals(true_out)

my_out.compare(true_out)  # shows differences

Numbers/Floats

abs(my_out - true_out) < 1e-6

Tips

• Test case variables let you programmatically compare data. While a visual inspection may work if the error is obvious, you should plan on using code to debug your solution. Things like data types, for example, may not be obvious.
• Since copy/paste is disabled, you will want to be efficient with your test code, using shortened, quick versions of things. You can also create functions to help you compare.

For example, a quick compare function may save you a few seconds here and there, at the cost of a few seconds at the start. You can save this code and type it from your notes at the start of the exam if you wish.

def qc():
    """Quick compare after running test cell."""
    m = returned_output_vars['output']
    t = true_output_vars['output']
    print("Match:", m == t)
    print("Mine:", m)
    print("True:", t)
    return m, t