CSE 6040 Exam Rules - FA22

For information on how to best prep for the exam, please refer to the Exam Prep page. If you have questions that weren't answered here, please check out the FAQ page.

Overview:

Exams are open-note, open-internet, timed assignments, similar to notebooks with unlimited submissions, but with a limited time in which you can access and complete them. You will be proctored, but will be able to use most resources that don't involve collaborating with other people. You will also not be allowed to ask for help or clarification from TA's.

Please spend time reading this entire page. You are responsible for understanding the test rules and procedures for resolving issues. We are unable to grant exemptions or make accommodations due to a misunderstanding of the rules and/or failure to follow the provided issue resolution steps.

A lot of this will be repetitive and may sound overly verbose, but this is written based on experience administering these tests for the past few semesters. Everything below corresponds to a question asked in the past in some form, so please trust us when we say this is all important.

Student responsibilities before beginning the exam

You are expected to understand all of the rules and exam policies listed in this document. The exam guide page has a list of the most important things to help guide you. Hopefully it all makes sense as you read through it, but if there is something that surprises you then it's very important that you take note of that. We don't want you to lose out on exam time, get deducted points, or otherwise have your experience hampered because of a misunderstanding.


Grading

Summary: you should submit after every exercise and confirm that the autograder awarded you points.

The exam grades are handled just like the notebooks, except with a much shorter time period during which you can submit. The below policies are the same as notebooks, but in this case you won't have multiple days + an extended deadline to resolve the problem and get points. Once your test ends it ends, no exceptions.

The autograder

The autograder is the final arbiter of grades. This means that if your code doesn't pass the autograder, you will not receive credit. Note: 'passing' is indicated by the grade card on the right, seeing a 'passed' below a cell doesn't guarantee points. It is important that you understand this distinction.

Autograder timeout

Error text: "the process runtime may have reached the allocated threshold of x second(s)"

If the autograder times out you will receive no credit for that submission. This will be visible to you when you look at the grade report after submission--it will clearly state the above error message and the grade card on the right won't be updated. This is a common issues and something you should aware of. If you fail to submit after every exercise and only submit after the exam ends, you may end up with 0 points.

Best practice: submit after every exercise you complete

There are multiple reasons why this is good, but the takeaway is that you need to submit after every exercise. There is no reason not to, it should take a few seconds to hit submit. You don't need to wait for it to complete, but you should go back and check on it 30-60 seconds later to confirm you were awarded the points. This might be the most important tip in this entire document.

Background: how the autograder works

The autograder is not complicated, it simply runs your code in a clean environment, in order from top to bottom. It should be nearly identical to you restarting the kernel and running from top to bottom. There are only a few differences:

  • It runs in a clean environment, meaning any packages you install will not be carried over (importing is fine, it's installing that is the issue)
  • If code for a specific exercise generates an error it will skip it and keep running your code (this will give you more points so you can potentially fail ex2, but still get credit for ex0, ex1, and ex3)
  • There is a time limit (see Autograder timeout above), meaning that it will stop running your code if it takes too long. Jupyter interactive mode (when you're sitting there and running cells directly) has a much higher limit, so being able to run a cell is not a guarantee that the autograder will run it to completion.

Testing cells

The code in the testing cells is sophisticated because it imports other code we wrote that more thoroughly tests your code and generates the output cleanly. Therefore, while you probably won't be able to deconstruct the test code during your time window, you will still see an error traceback as well as the generated variables to show your generated solution vs. the real solution (see practice midterm 1 problem #26). This is more useful than us simply printing this out for you, because you can use the variables to view the data however you want and more dynamically compare if you choose. You can even print these out if you want to compare visually, just be careful that it's 1) not too large, and 2) that you don't go down a rabbit hole trying to visually compare 100+ characters.

Scoring cap

The exams will usually have a scoring cap that is announced as part of the exam release notes. This scoring cap means that your score will be [# pts]/[scoring cap]. For example, if there are 15 possible points and the scoring cap is 12, your score of 9 points would get you the grade of 9/12 = 75%. The cap on the grade is 100%, so there will be no extra credit unfortunately (therefore you may keep working after you reach the scoring cap, but will not receive any extra credit).

Requests for manual grading

We do not do manual grading under any circumstances. If you believe your code should've passed but failed due to a bug, see the section below about bug reporting and follow that process.

Partial credit

There is no 'partial credit' beyond the points awarded at the exercise level.


Rules

The rules below are by no means exhaustive, but should cover everything you need to know. If something is unclear, then you need to ask the staff via Piazza before the exam (after carefully reading the below 'Rules' and Exam Rules - FAQ page). To be direct: You must complete this exam within the time limit without collaborating with other people. Any attempts to circumvent this policy will be considered an Honor Code Violation.

Honor Code Violations (aka 'No Collaboration Policy')

These rules pertain to Honor Code violations, which will result in you being reported to the Georgia Tech Office of Student Integrity and likely receiving a 0 on the exam at a minimum.

  • No collaboration of any kind. This includes:

    • Working with any person via any means of communication
    • Attempting to use any homework help service (ex: Chegg)
    • Receiving or sending exam solutions (including code fragments specifically related to the exam)
    • Searching for exam solutions (accidentally stumbling upon them is ok, but you must report it to us and close the page)
  • No circumventing exam rules, including misleading TA's about timing or platform issues:

    • Vocareum extensively logs everything, so we will be validating all reports including issues with the timer, technical issues, etc.
    • This rule is specifically mentioned because this has unfortunately happened before. It's unfair to other students and is an attempt to take advantage of the teaching staff who dedicate many nights and weekends to teaching this class. We will generally give you the benefit of the doubt, but will be very thorough to ensure that no student gets an unfair advantage due to any of these strategies.

List of rules

  1. No collaboration (see above "Honor Code Violations" section)
  2. No discussion of exams until after the solutions are released. This means:
    • No discussing exam content, structure, question order, difficulty, or anything about the exam
    • No posting of this information on any platform
  3. TA's will not be able to assist. During the exam the TA's will be on 'radio silence' mode, meaning we will monitor piazza, but will not be replying to posts unless they involve platform/technical issues, or other special issues. We will not help you debug, nor will we be able to 'clarify' any questions for you.
  4. TA's will be able to assist if you have technical issues. See the 'Support' section below.
    • It is important that you distinguish between technical issues and issues caused by your code. You should be conservative during you assessment. That means that if the test code says your output is wrong, it's probably wrong, absolutely do not assume it's a bug. If the issue is that Vocareum is generating an error or the page isn't loading, then that's obviously a technical issue.
  5. Test/debugging steps: You are responsible for following the steps listed in the test "Release Notes" piazza post as well as the Vocareum Troubleshooting Guide. There are some technical annoyances that happen (rarely), but are 100% fixable with the steps listed in those two resources. If you make a Piazza post about a technical issue and you didn't follow the troubleshooting steps, our response will be to remind you of those steps.
  6. Time limit: You will have the number of hours specified (usually 3-4.5), or until the exam window closes to complete your test, whichever comes first. If you start the exam at 1059 UTC Wednesday, you will only have one hour to complete the exam, so make sure you don't wait until the last minute.
  7. Test window: see schedule.
  8. Technology restrictions:
    • Only one computer/device allowed (no secondary computer, no tablet, etc.)
    • Only one monitor allowed
      • You may have multiple plugged in, but the ones you aren't using will need to be deactivated. Honorlock/Proctortrack will let you know if there's an issue. This may take a few minutes to figure out, so please test your setup beforehand and confirm there are no issue.
    • For MSA/OMSA: Chrome is required for Honorlock, but you may have other browsers open.
    • For edX VMM: Any Proctortrack-compatible browser is required, then we recommend having one more ready to go per the troubleshooting steps.
    • Webcam required. A built-in one will work, but it needs to be positioned so you're looking towards it during the exam (ex: if you're using a laptop with an external monitor, you will need to position the laptop so it's very close to the other screen)
  9. Room/workspace restrictions: The below rules are less strict than the rest, but please try to follow them. All of these will generate flags in the proctoring tool that we will have to review and approve. If what happened won't be obvious to us during review, please make a private piazza post after the exam with a quick note explaining the circumstances (please be brief!).

    • No other people around if possible
      • It's ok if you're in the living room and family walks by, or if you cat needs to investigate what's going on
    • No talking (to people in the room, to people on your phone)
      • It's ok if a family member has a quick question, or you get an urgent call and have to tell them you're taking a test
    • No other technology around you (no phones, tablets, other computers, etc.)
      • Please make sure everything is visibly 'off' and not being used
  10. Environment Package Restrictions

    • Installing packages: install packageX is not allowed. Vocareum will allow you to install them, but they will not work in the autograder, therefore there is no reason to do this.
    • Importing packages: import packageX is allowed on Vocareum. You can find a full list of installed packages here. Note: if you are working locally, your computer environment will be different. That means if you use import you may be importing a package not in Vocareum, or a different version of the Vocareum package. If you do this locally be prepared to debug issues related to that (or resolve these package issues ahead of time).

What's allowed during the test:

Now that we got all of the restrictions out of the way, these are resources that you are specifically allowed to have and things you are allowed to do during your exam. This list is not exhaustive--if you are not breaking the above rules and the spirit of the test, it is allowed. No collaboration is still the key.

Resources allowed: The test is open-note, open-book, and open-internet. This means you are allowed to use any resources that don't involve collaborating with another person:

  • Notebooks and solutions (your copy + instructor solutions)
  • Practice exams and solutions
  • Notes (hand-written, digital, etc.)
  • Internet-based resources subject to the no collaboration policy. This means you can Google search and view general coding forums, but you may not post on there. You can read and search piazza posts, but you may not post to other students.
    • For example, you can search Stackoverflow posts and read everything there, but you can not make your own post
  • Use computer software that's not collaboration-related (any note app is fine, any music app, screenshot tool, etc.)
  • Use local IDE's (see FAQ question about local IDE's)
  • Use sites like regex101, pythontutor, etc. (interactive sites where you aren't collaborating with another person!)
  • Take screenshots of the exam to report issues to us, or for your own notes
  • Calculators (physical, on-screen, web-based)

Other things allowed:

  • Take short breaks (~5 minutes; if longer, that's ok but please make a private piazza post so we can resolve the proctoring flag)
  • Eat/drink
  • Listen to music with headphones using any music app (no speakers please!)
  • Scratch paper, writing utensils, etc.
  • Plenty of other things, as long as they don't violate rules 1-10 and the spirit of the test

For anything not mentioned here or in the above rules area, please make a private piazza post if you want us to confirm yes/no. Generally as long as it doesn't conflict with the "no collaboration" and proctoring/tech requirement rules, it's fine.


Debugging

  • Test cells on exams will run your solution on many randomly generated test cases
  • The test case inputs, expected outputs, and your solution's returned outputs will be available as variables in the notebook after running a test cell. You can use these variables for your debugging. This can be especially helpful when your solution runs but doesn't produce the correct output (the AssertionError will only tell you that your output is incorrect)
  • Visually comparing your output to the test output doesn't mean they're the same. If the code says they're different then they probably are, whether it's data types, or even something that's visible but you missed
  • Read the entire traceback given when an error is raised
  • Assume the test is bug-free and continue working. You are encouraged to submit a bug report (template link) to us if you have a good reason for thinking there's a bug, but you should not spend more than a few minutes doing this. While it's technically possible that there are bugs, it's highly unlikely and you should not waste any of your limited time thinking that the test is bugged.

Proctoring

Your exam will be proctored using Honorlock (MSA and online MSA students) or Proctortrack (edX VMM students). Both tools can be installed right before you start your exam, but you should budget time to install and test it if there are issues.

Things that Proctoring does:

  • Does a quick scan of your room (not always)
  • Records your webcam
  • Records your screen
  • Confirms you're using only one monitor
  • It may do other simple checks on your setup, but the above items are the ones we rely on

Testing your setup

Go to the practice proctored exam for [current exam] and start it up. If proctoring starts and you are able to get to the exam and see it, then your setup is good. There is no need to go further.

For edX VMM students: The systems will only let you start the exam once, so if you already tested and want to test again, please make a private piazza post with this request and include your edX username.


Getting Support (Troubleshooting)

  • You can reach out to the TAs for support on Piazza by making a private post to the Instructors. Include your email and the relavent information in your post. As per class policy, piazza is the only valid method of communication to reach out to the teaching staff. That is also the fastest method too.
  • TAs will provide support to help with platform issues and bug inquiries only. In the interest of consistency and fairness, TAs are not able to discuss the exam material at all (including clarifying the exercise instructions).
  • If think you are encountering a platform issue: attempt the steps listed in the Vocareum Guide's Platform Troubleshooting section before posting! If it is discovered that there is a platform issue requiring further troubleshooting or there is an outage, TAs will adjust your exam timer to make up for the lost time. If the issue was caused by your code or is resolved by carrying out the posted troubleshooting steps, your time will not be adjusted. See reporting template on this page.

If you encounter issues your workflow should be: Troubleshoot, Document, Continue working

  • Troubleshoot: follow the troubleshooting steps listed on the Vocareum Troubleshooting page, as well as any extra notes in the exam release notes
  • Document: follow the steps above to document your issue with us
  • Do not spend a lot of time on this, but you must provide all of the required info in order for it to count as fully documented. You do not want to spend >5 minutes doing this in case the error is related to your code
  • Continue working: Don't wait for a response, keep working as best you can

Support for potential bugs

Bug inquiry: If you think you are encountering a bug where your solution is incorrectly failing a test cell do the following:

  1. Check your output against the expected output thoroughly using the test case variables described in the debugging seciton.
  2. Submit your work - this gives us a snapshot that we can use to reproduce the bug.
  3. Make a private Piazza post visible to the instructors with a short description of the problem. You can find the template on this page.
  4. Continue working to debug your solution or move on to another problem. It will take some time for us to find the issue. If the problem is found to be in your solution, we will not give any concessions for lost time spent debugging.
    • Note - We thorouglhly test the exams prior to release. The overwhelming majority of reported "bugs" are actually issues with student solutions. We will investigate all of them and give an answer as to what the problem was after the exam window has closed. If there is a bug found, we will make the appropriate adjustments.

Note on academic policies (incl. missed exams)

CSE 6040 Grading Policy: If you don't take the exam, or misunderstand the exam window (ex: UTC time confustion) and miss it, you will receive a 0 on the exam. There is no flexibility here unfortunately. We won't be able to extend the window for you, nor will we consider alternative grading structures. See exception below

Missed exams (MSA/OMSA only): If you cannot take the exam during the designated period, but you do have compelling and documented justification (e.g., medical condition, family emergency), we can give you an incomplete grade for the course. To document your situation, reach out to OMSA Advising as soon as you can and explain your situation to them. Then, make a private piazza post to all instructors explaining the situation and confirm you're working with OMSA Advising. For your own privacy you will only need to share the specifics with OMSA Advising. You do not need to share that private info with us in your Piazza post.

Assuming OMSA approves and confirms it with 6040 staff, you will then be granted an Incomplete at the end of the semester. This means you will continue working this semester but will receive an "I" at the end, which you can resolve by coming back in a future semester and completing the missing work.

Per Georgia Tech’s policies, incomplete grades are reserved for making up a “small” amount of work only. If your circumstances are more severe, you may need to drop and retake the course. Please refer to GT’s information incompletes for more detail.

Missed exams (edX VMM only): If you cannot take the exam during the designated period, but you do have compelling and documented justification (e.g., medical condition, family emergency), you can reach out to edX support and explain your situation to them. They will then reach out to the instructors and help come up with options. Unfortunately we can't shift the exam window or offer alternative grading structures, so edX will probably offer you the option to re-enroll in the class in the future for free.

Enrolling in graduate courses is a serious commitment. Once you are enrolled in a class you are subject to official Georgia Tech academic policies, meaning that you should assume no changes will be made to the grading structure, withdrawal date, or any rule mentioned in this class or official Georgia Tech documentation. There may be cases where you can make an appeal for special consideration, but you will need to speak to CSE 6040 staff (via private piazza post) as well as with OMSA Advising to go through the formal process. This is by no means meant to stress you out further during serious situations. This is meant to remind you that 1) CSE 6040 staff and OMSA Advising are here to help you, and 2) there are serious policies that we have to follow, so please take your commitment seriously and communicate with the appropriate party as soon as possible.


Rules -

See Exam Rules - FAQ page


tl;dr - the exam in seven bullet points

  • The exam is open-notes, open-internet
  • The exam time limit will be announced right before the exam window opens, in the exam 'Release Notes' piazza post. The time limit will be in the 3-4.5 hour range, it varies based on our internal testing.
  • The exam is structured similarly to the notebooks. The content will change, but the style will be similar. The testing cells will have improved code that makes it easier for you to debug.
  • Collaboration is strictly prohibited
  • There will be no TA assistance other than for platform issues. This means you are responsible for completing on the test on your own with help from the approved resources.
  • You are responsible for reading and understanding all of the information regarding test rules and troubleshooting procedures
  • Troubleshoot, document, continue working
Updated: 2022-12-16