CSE 6040 - Exam Guide

For information on how to best prep for the exam, please refer to the Exam Prep page. If you have questions that weren't answered here, please check out the FAQ.

Overview

Exams are open-note, open-internet, timed assignments, similar to notebooks with an unlimited number of submissions, but within a limited time in which you can access and complete them. You are proctored and able to use most resources that do not involve collaborating with other people. You are not allowed to ask for help or clarification from TAs.

Please read this entire guide. You are responsible for understanding the test rules and procedures. We are unable to grant exemptions or make accommodations due to a misunderstanding of the rules and/or failure to follow the provided issue resolution steps.

This may seem repetitive and overly detailed This guide was developed over the years based on experience administering these tests. Everything below corresponds to a question asked in the past in some form.

You are expected to understand all of the rules and exam policies listed on this page and in the Rules. The Exam Summary page has highlights of the most important things to help guide you. Hopefully it all makes sense as you read through it, but if there is something that surprises you then it's very important that you take note of that. We don't want you to lose out on exam time, get deducted points, or otherwise have your experience hampered because of a misunderstanding.

Exam Guide

Best practice: Submit after every exercise

Grading

Scoring Cap

Demo Cells

Testing Cells

Autograder

Autograder - Timeout

Requests for Manual Grading

Partial Credit

Missed Exams

GT Academic Policies

Proctoring

Debugging

Getting support from TA's

Reporting Exam Bugs

Best practice: Submit after every Exercise

Every time you submit your solution for grading, the Autograder runs and adds points to your total. To ensure you lock in your points and do not run out of time at the end, submit after every exercise. You can move on to the next exercise while it runs and then check to make sure your points are there. At the same time, if you don’t get the points, you can return to the exercise while it’s still fresh in your mind. This also allows you to see your points add up, which can be a boost in the middle of the exam. There is no reason not to submit after each exercise. This might be the most important tip in this entire document. Comment out print statements before submitting to allow your code to run more quickly.

Grading

Summary: you should submit after every exercise and confirm that the autograder awarded you points.

The exam grades are handled just like the notebooks, except with a much shorter time period during which you can submit. The below policies are the same as notebooks, but in this case you won't have multiple days + an extended deadline to resolve the problem and get points. Once your test ends it ends, no exceptions.

Scoring Cap

The scoring cap means that your score will be [# pts]/[scoring cap]. You may not have to complete all exercises on the exam in order to score 100%. We will post a “cutoff” for achieving 100%. That number may be adjusted based on the overall class performance, but adjustments will only be in your favor. Ex: there may be 18 possible points but the scoring cap is 15 points, meaning that if you earned 12 points, you would have 80% (12 out of 15). Your score will be X/15 with a cap of 100% The cap on the grade is 100%, so there will be no extra credit unfortunately (therefore you may keep working after you reach the scoring cap, but will not receive any extra credit).

Demo Cells

Each exercise has a demo cell to help illustrate what the problem is asking for. The demo cells are not a complete test of your solution. It is certainly possible to get the correct response for a simplified demo problem and miss corner cases in the real test. We will not give any credit for solutions that do not pass the test cells as run by the grading script, even if they seem to get the correct result for the demos.

Testing Cells

The code in the testing cells executes as many as 20 randomized test cases written to more thoroughly test your code and generate the output cleanly. While you probably won't be able to deconstruct the test code during the exam window, you will see an error traceback as well as the generated variables to show your generated solution vs. the real solution (see practice midterm 1 problem #26). This is more useful than printing this out for you, because you can use the variables to view the data however you want and more dynamically compare if you choose. It is recommended to print these variables if you want to compare visually, just be careful that it's 1) not too large, and 2) that you don't go down a rabbit hole trying to visually compare 100+ characters.

Autograder

The Autograder is the final arbiter of grades. If your code doesn't pass the Autograder, you will not receive credit. Note: 'passing' is indicated by the grade card on the right, seeing a 'passed' below a cell doesn't guarantee points. For an explanation of how you can get “Passed!” and not receive points, see “Demo Cells” above. It is important that you understand this distinction.

The Autograder Explained

The Autograder is not complicated. It runs your code in a clean environment, in order from top to bottom. It is similar to restarting the kernel and running from top to bottom. There are only a few differences:

  • It runs in a clean environment, meaning any packages you install or update will not be carried over. Importing is fine, it's installing that is the issue. See the Exam Rules for more details.

  • If code for a specific exercise generates an error it skips it and keeps running your code. This gives you more points so you can potentially fail ex2, but still get credit for ex0, ex1, and ex3.

  • There is a time limit (see Autograder Timeout below), meaning that it stops running if your code takes too long. Jupyter interactive mode (when you're sitting there and running cells directly) has more processing power and a higher time limit, so being able to run a cell is not a guarantee that the Autograder will run it to completion.

Autograder Timeout

"The process runtime may have reached the allocated threshold of x second(s)"

If the Autograder times out you receive no credit for that submission. This will be visible to you when you look at the grade report after submission--it will clearly state the above error message and the grade card on the right won't be updated. This is a common issue and something you should be aware of. If you fail to submit after every exercise and only submit after the exam ends, you may end up with 0 points.

Requests for Manual Grading

We do not do manual grading under any circumstances. If you believe your code should have passed but failed due to a bug, see “Support for Potential Bugs” below.

Partial Credit

We do not grant partial credit for exercises on the exam. Please do not ask.

Missed Exams

If you don’t take the exam because you misunderstood the exam window or the exam length, you will receive a 0 for the exam. We cannot extend the window for you or consider alternative grading structures or assignments.

Missed exams (MSA/OMSA only): If you cannot take the exam during the designated period, but you do have compelling reason with documented justification (e.g., medical condition, family emergency), you may be eligible to receive an Incomplete in the course. Note that this is granted at the discretion of 6040 staff so you must make a private piazza post to discuss it with us. If the staff agrees, you will then need to reach out to OMSA Advising to document your case. For your own privacy you only need to share the specifics with OMSA Advising. You do not need to share that private info with us in your Piazza post.

Assuming OMSA accepts your documentation and forwards us confirmation, you will then be granted an Incomplete at the end of the semester. This means you will continue working this semester but will receive an "I" at the end, which you can resolve by coming back in a future semester and completing the missed exam. In your piazza post with instructors we will give you the full details and rules regarding resolving your Incomplete.

Per Georgia Tech’s policies, incomplete grades are reserved for making up a “small” amount of work only. If your circumstances are more severe, you may need to withdraw and retake the course. Please refer to GT’s information for incompletes for more detail.

Missed exams (edX VMM only): If you cannot take the exam during the designated period, but you do have compelling and documented justification (e.g., medical condition, family emergency), you can reach out to edX support and explain your situation to them. They will then reach out to the instructors and help come up with options. edX will probably offer you the option to re-enroll in the class in the future for free. Your progress can be transferred at the sole discretion of 6040 staff.

Georgia Tech Academic Policies

Enrolling in graduate courses is a serious commitment. Once you are enrolled in a class you are subject to official Georgia Tech academic policies, meaning that you should assume no changes will be made to the grading structure, withdrawal date, or any rule mentioned in this class or official Georgia Tech documentation. There may be cases where you can make an appeal for special consideration, but you need to speak to CSE 6040 staff (via private piazza post) as well as with OMSA Advising to go through the formal process. This is by no means meant to stress you out further during serious situations. This is meant to remind you that 1) CSE 6040 staff and OMSA Advising are here to help you, and 2) there are serious policies that we have to follow, so please take your commitment seriously and communicate with the appropriate party as soon as possible.

Proctoring

Your exam is proctored using Honorlock (MSA and online MSA students) or Proctortrack (edX VMM students). Both tools can be installed right before you start your exam, but you should budget time to install and test it if there are issues.

What Proctoring does:

  • Quick scan of your room (not always)
  • Records your webcam
  • Records your screen
  • Confirms you're using only one monitor
  • It may do other simple checks on your setup, but the above items are the ones we rely on

Testing your setup:

Go to the practice proctored exam and start it up. If proctoring starts and you are able to get to the exam and see it, your setup is good. There is no need to go further.

For edX VMM students: The systems only lets you start the exam once. If you already tested and want to test again, please make a private piazza post with this request and include your edX username.

If you experience issues with the Proctoring tool, see “Proctoring Troubleshooting”

Debugging

  • Test cells on exams run your solution on many randomly generated test cases
  • The test case inputs, expected outputs, and your solution's returned outputs are available as variables in the notebook after running a test cell. You can use these variables for your debugging. This can be especially helpful when your solution runs but doesn't produce the correct output (the AssertionError only tells you that your output is incorrect)
  • Visually comparing your output to the test output does not mean they are the equivalent. If the code says they're different, they probably are, so plan on comparing the test variables to identify data type differences and other common issues
  • If you cannot find the difference, try using a site like www.diffchecker.com
  • If you cannot find the difference between two numbers, try subtracting one from the other. If the result is not zero, they are not equal
  • Read the entire traceback given when an error is raised
  • Assume the test is bug-free and continue working. You are encouraged to submit a bug report (template link) to us if you have good reason for thinking there's a bug, but you should not spend more than a few minutes doing this. The later in the exam period that you are taking the exam, after hundreds of students have taken the exam, the more unlikely it is that you have found a bug. The vast majority of bugs (if there are any) are identified within the first 24 hours of the exam window. If you think you have found a bug, see “Support for Potential Bugs” below.

Getting Support from TAs

  • Reach out to the TAs for support on Piazza by making a private post to the Instructors. Include your email and the relevant information in your post. As per class policy, piazza is the only valid method of communication to reach out to the teaching staff.
  • TAs provide support to help with platform issues and bug inquiries only. In the interest of consistency and fairness, TAs cannot discuss the exam material at all, including clarifying the exercise instructions.
  • If you think you are encountering a platform issue (Proctoring or Vocareum), then: Attempt the steps listed in the Vocareum Troubleshooting Guide or Platform and Proctoring Troubleshooting guide section before posting! If there is a platform issue requiring further troubleshooting such as an outage, TAs will adjust your exam timer to make up for the lost time. If the issue was caused by your code or is resolved by carrying out the posted troubleshooting steps, your time will not be adjusted. See reporting template on this page.

If you encounter issues, your workflow should be: Troubleshoot, Document, Continue working

  • Troubleshoot: follow the troubleshooting steps listed on the above Vocareum and Platform/Proctoring troubleshooting pages, as well as any extra notes in the exam release notes
  • Document: follow the steps above to document your issue with us
  • Do not spend a lot of time on this, but you must provide all of the required info in order for it to count as fully documented. You do not want to spend >5 minutes doing this in case the error is related to your code
  • Continue working: Don't wait for a response, keep working as best you can

Reporting Possible Exam Bugs

We thoroughly test the exams prior to release. The overwhelming majority of reported "bugs" are actually issues with student solutions. However, we investigate all reports. If a bug is found, we will make appropriate adjustments. If the issue is your code, we will let you know the reason after the exam window closes.

Bug inquiry: If you think you are encountering a bug where your solution is incorrectly failing a test cell, do the following:

  1. Check your output against the expected output thoroughly using the test case variables described in the debugging section.
  2. Submit your work - this confirms a clean test of your work is still resulting in an error. This also gives us a snapshot that we can use to reproduce the bug, which is critical
  3. Make a private Piazza post visible to the instructors with a short description of the problem. You can find the template for this on the Templates page.
  4. Continue working to debug your solution or move on to another exercise. It takes some time for us to find the issue. If the problem is found to be in your solution, we do not give any concessions for lost time spent debugging.
  5. The later in the exam period that you are taking the exam, after hundreds of students have taken the exam, the more unlikely it is that you have found a bug. The vast majority of bugs (if any) are identified within the first 24 hours of the exam window.

tl;dr - the Exam in Seven Points

  • The exam is open-notes, open-book, open-internet
  • Collaboration is strictly prohibited (a few examples: posting on Stackoverflow, posting in Slack, asking a bot like ChatGPT)
  • The exam time limit will be announced right before the exam window opens, in the exam 'Release Notes' piazza post. The time limit will be in the 3-4.5 hour range, it varies based on our internal testing.
  • The exam is structured similarly to the notebooks. The main difference will be the testing cells have improved code that makes it easier for you to debug.
  • There is no TA assistance except for platform issues. This means you are responsible for completing the test on your own with help from the approved resources.
  • You are responsible for reading and understanding all of the information regarding test rules and troubleshooting procedures.
  • Troubleshoot, document, continue working
Updated: 2024-01-08