├── README.md ├── announcements.md ├── course-info.md ├── deliverables ├── 1 │ ├── coffeemaker.jar │ ├── deliverable1-tips.md │ ├── deliverable1.md │ ├── grading_rubric.txt │ └── requirements.txt ├── 2 │ ├── deliverable2.md │ ├── grading_rubric.txt │ └── requirements.txt ├── 3 │ ├── deliverable3.md │ └── grading_rubric.md ├── 4 │ ├── deliverable4.md │ ├── grading_rubric_combinatorial.md │ └── grading_rubric_property_based.md ├── 5 │ └── deliverable5.md └── 6 │ └── deliverable6.md ├── grading.md ├── lectures ├── lecture1-intro.pdf ├── lecture11-intro-to-tdd.pdf ├── lecture12-writing-testable-code.pdf ├── lecture13-intro-to-bdd.pdf ├── lecture14-selenium.pdf ├── lecture15-web-testing-with-junit.pdf ├── lecture16-combinatorial-testing.pdf ├── lecture17-property-based-formal-verification.pdf ├── lecture18-perf-testing-1.pdf ├── lecture19-perf-testing-2.pdf ├── lecture2-testing-basics.pdf ├── lecture20-security-testing.pdf ├── lecture21-stakeholder-interaction.pdf ├── lecture22-stakeholder-interaction-2.pdf ├── lecture23-penetration-testing.pdf ├── lecture3-requirements.pdf ├── lecture4-SUPPLEMENTAL-exploratory.pdf ├── lecture4-test-plans.pdf ├── lecture5-defects.pdf ├── lecture6-breaking-software.pdf ├── lecture7-automated-tests.pdf ├── lecture8-intro-unit-tests.pdf └── lecture9-advanced-unit-testing.pdf ├── old_deliverables ├── deliverable1 │ ├── coffeemaker.jar │ ├── deliverable1-tips.md │ ├── deliverable1.md │ └── requirements.txt ├── deliverable2 │ ├── deliverable2.md │ ├── requirements.md │ └── sample_output.txt ├── deliverable3 │ └── deliverable3.md ├── deliverable4 │ └── deliverable4.md ├── deliverable5 │ └── deliverable5.md └── deliverable6 │ └── deliverable6.md ├── old_study_guides ├── exam-2-study-guide.md └── midterm-study-guide.md ├── sample_code ├── FizzBuzz.rb ├── LinkedList.java ├── LinkedListTest.java └── selenium │ └── RedditTest.java ├── sample_tests └── midterm.txt ├── study-guides ├── midterm-1-study-guide.md └── midterm-2-study-guide.md ├── syllabus.md └── test └── Foo.java /README.md: -------------------------------------------------------------------------------- 1 | CS 1632: Software Quality Assurance 2 | 3 | ==== 4 | PULL REQUESTS ACCEPTED! 5 | ==== 6 | 7 | Pitt CS 1632 - Lectures and Notes 8 | 9 | This contains all of the lectures, course information, and sample code for 1632: Software Quality Assurance, taught by Bill Laboon. 10 | 11 | Note that you may need to clone the repo to read some of the PDF lecture slides, since GitHub has a limit on how large files can be before they're not displayed. 12 | -------------------------------------------------------------------------------- /announcements.md: -------------------------------------------------------------------------------- 1 | ## Announcements (newest at top) 2 | 3 | * Link to Alex Shenoy's Git Slides: https://speakerdeck.com/alexshenoy/git-basics 4 | -------------------------------------------------------------------------------- /course-info.md: -------------------------------------------------------------------------------- 1 | # CS 1632 2 | Software Quality Assurance 3 | 4 | ## Course Information 5 | 6 | **Taught by:** Bill Laboon (bill.laboon@pitt.edu) 7 | **Professor's Office Hours:** SENSQ 6305 8 | * Thursday 2:30-4:00 PM 9 | * Wednesday 1:00-2:30 PM 10 | * or by appointment. 11 | 12 | **Class Time:** T/H 4:00-5:15 PM. 13 | **Room:** SENSQ 5129 14 | 15 | **TA**: 16 | **Office Hours**: 17 | 18 | **Class GitHub repo:** https://www.github.com/laboon/cs1632 19 | **Required Text:** _A Friendly Introduction to Software Testing_ by Bill Laboon. 20 | * This is a PDF and is available on Github 21 | * https://github.com/laboon/software-testing/blob/master/software-testing-laboon-ebook.pdf 22 | * If you spot a typo or mistake, or suggest an improvement that I use, and file it as a pull request or issue on the source repository (ebook), you can get 0.5 extra percentage point (out of 100), up to 2 percentage points. 23 | * In other words, if you finish the semester with a 90, but submitted 4 valid issues / typo fixes / whatever, your final grade is 90 + 2, 92. 24 | 25 | **Recommended Texts:** 26 | 27 | Test-Driven Development: By Example, Kent Beck ISBN-13: 978-0321146533 28 | 29 | Software Testing: A Craftsman's Approach, Paul Jorgensen, ISBN-13: 978-1466560680 30 | 31 | I have both of these texts available for you to borrow. 32 | 33 | This course provides students with a broad understanding of modern 34 | software testing and quality assurance. Although it will cover testing 35 | theory, the emphasis is on providing practical skills in software 36 | testing currently used in industry. To that end, it will cover: manual 37 | and automated tests, test-driven and behavior-driven development, 38 | performance testing, and understanding and developing a testing 39 | process. 40 | 41 | ## Grading 42 | 43 | * Mid-term Exam - 15% 44 | * Final Exam - 15% 45 | * Projects: 46 | * Deliverable 1 - 10% 47 | * Deliverable 2 - 10% 48 | * Deliverable 3 - 10% 49 | * Deliverable 4 - 10% 50 | * Deliverable 5 - 10% 51 | * Final Deliverable - 15% 52 | * Class Participation - 5% 53 | 54 | Although you are not required to come to every class (as you can see, it's technically possible to get an A without ever showing up except for turning in papers and tests), it is strongly recommended that you do. I also expect students to take an active part in discussions. Attendance _will_ be taken for project days, and this will count towards class participation. 55 | 56 | The following grading scale will be used. 57 | 58 | Score | Grade 59 | -----: | ------------------------------ 60 | 100.00-94.00 | A (A+ for extraordinary work) 61 | 93.99-91.00 | A- 62 | 90.99-88.00 | B+ 63 | 87.99-84.00 | B 64 | 83.99-81.00 | B- 65 | 80.99-78.00 | C+ 66 | 77.99-74.00 | C 67 | 73.99-71.00 | C- 68 | 70.99-68.00 | D+ 69 | 67.99-64.00 | D 70 | 63.99-61.00 | D- 71 | 60.99-0.00 | F 72 | 73 | All groups are expected to do their own work on the group project, but 74 | are more than welcome to collaborate and ask questions with other 75 | groups, the Internet, or other colleagues. 76 | 77 | However, any student caught collaborating or cheating on an exam will 78 | automatically receive a 0 (zero) for that exam, and may be penalized 79 | more harshly based on University of Pittsburgh academic policy. 80 | 81 | Assignments should be committed and pushed to GitHub by the beginning 82 | of class on the due date. Write-ups are due at the beginning of class. 83 | Late assignments will not be accepted. 84 | 85 | If any disputes from groups arise, I will assume that GitHub is the "ground 86 | truth". 87 | 88 | The final exam will be cumulative. 89 | 90 | It is recommended you keep all of your graded assignments until final 91 | grades are posted and accepted, in order to resolve any discrepancies 92 | in grading. 93 | 94 | ## Attendance 95 | 96 | Lecture attendance is not required, but is STRONGLY recommended. 97 | The instructor will try to ensure that all information on the exams 98 | will be available via slides, but simply reading them may be 99 | insufficient to understand the concepts thoroughly. 100 | 101 | Presence for group exercise days is required in order to get the full 102 | class participation score. These will not be rescheduled individually. 103 | 104 | Presence for the mid-term and final exam are REQUIRED. They will be 105 | individually re-scheduled only in the event of an emergency. If you 106 | are facing an emergency, please contact the instructor IMMEDIATELY (if 107 | it is safe to do so, of course). Failure to show up for an exam 108 | without clearing it first with the instructor will result in a 0 109 | (zero) for that exam. 110 | 111 | ## Project Details 112 | 113 | For deliverables 1, 3 and 5, groups of two will be assigned and will be 114 | different each time. For the final deliverable, you can choose to work 115 | alone, or choose your own partner (who must, of course, also agree to work with you). This partner can be someone with whom you have worked previously. 116 | 117 | Students will perform the role of QA team on a project which can be selected either by themselves, or assigned by the instructor. Although students can select to work on a different project, it must first be cleared with the instructor. Additionally, it should be noted that the instructor and/or TA may not be able to help as much if you use a different project or language! The group should check with the instructor as to the feasibility of 118 | the particular project they are working on. 119 | 120 | * **Deliverable 1:** A test plan and traceability matrix 121 | * **Deliverable 2:** Unit tests for a console-based application. 122 | * **Deliverable 3:** Automated acceptance tests for a web application. 123 | * **Deliverable 4:** Performance testing of an application. 124 | * **Deliverable 5:** Security analysis of an application. 125 | * **Final Deliverable:** Will vary based on the group. See the Final Deliverable description for details of what is expected. 126 | 127 | Deliverables must be handed in by close of class on the day that it is 128 | due. Late deliverables will NOT be accepted. 129 | 130 | ## Programming Language Selection 131 | 132 | For deliverables 1 through 5, the class will use Java with the appropriate 133 | frameworks (JUnit, Mockito, Selenium). For the final project, student 134 | groups are free to choose their own language/framework if they wish. 135 | However, the professor may not be able to help them if he is not familiar 136 | with the language/framework of their choice! 137 | 138 | If you want to use Java but aren't a fan of Eclipse, information on setting up the [Gradle build tool](https://gradle.org/) with the required packages for this course can be found [here](https://gist.github.com/alexlafroscia/c6757de349b27e34eff6). 139 | 140 | ## Disability Services Statement 141 | 142 | "The Office of Disability Resources and 143 | Services (DRS) provides a broad range of support services to assist 144 | students with disabilities. Services include, but are not limited to, 145 | tape-recorded textbooks, sign language interpreters, adaptive and 146 | transportation. Contact DRS at 412-648-7890 or 412-383-1355 (TTY) in 147 | 216 William Pitt Union or see www.drs.pitt.edu for more computer 148 | technology, Braille translation, and nonstandard exam arrangements, 149 | DRS can also assist students with accessibility to campus housing 150 | information." 151 | 152 | ## Academic Integrity Statement 153 | 154 | "As members of the University of 155 | Pittsburgh community, A&S students are expected to meet the obligation 156 | to exhibit honesty and to respect the ethical standards of the 157 | University community and of their chosen field of study in carrying 158 | out academic assignments. A&S students are therefore expected to 159 | familiarize themselves with the published rules and regulations go to 160 | http://www.fcas.pitt.edu/academicintegrity.html 161 | 162 | -------------------------------------------------------------------------------- /deliverables/1/coffeemaker.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/deliverables/1/coffeemaker.jar -------------------------------------------------------------------------------- /deliverables/1/deliverable1-tips.md: -------------------------------------------------------------------------------- 1 | 1. Be as precise as possible. Don't say "Ensure that you see the correct value"; say that "You should see the text 'A Smart door leads South'". Don't say that you should "go north"; say "Type n ". 2 | 3 | 2. Remember to put down all the necessary steps and preconditions! 4 | 5 | 3. The "Test Case" label is for a description of the test case in plain English. For example, "Ensure that user can go North from the starting room." 6 | 7 | 4. You do not need to exhaustively test the application! If you find yourself with more than three test cases per requirement, you may be over-testing. This is admirable in software where human life is at stake, but this is only your grade. 8 | 9 | 5. This is a test plan, not a test run - remember the difference. 10 | 11 | 6. Note that if you find yourself in a magical land - even if you get transported back to where you came from - you may have gone through a door which did not exist. Think about if that meets the requirements or not. 12 | 13 | 7. Remember that tests should be reproducible - starting with all preconditions met, anybody should be able to reproduce the same steps and get the same results. If they can't, rethink your preconditions! -------------------------------------------------------------------------------- /deliverables/1/deliverable1.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Testing 2 | Spring Semester 2016 3 | 4 | * DUE: 28 JAN 2016 5 | 6 | ## Deliverable 1 7 | 8 | For this assignment, your group will determine a test plan for the simple game Coffee Maker Quest, based on the requirements listed. There are several known defects in the software; you will need to find and report on at least three. Additionally, a traceability matrix showing the mapping of test cases to requirements is required. 9 | 10 | Time will be given in class to group yourself into groups. There should be two and only two members of a group. 11 | 12 | There should be at least as many test cases as requirements, although I would normally expect several test cases per requirement. As a general rule, I'd estimate that the number of test cases is at least 2x the number of requirements. If the number of test cases is more than 3x the number of requirements, you are probably overtesting. 13 | 14 | Each requirement should have AT LEAST one test case associated with it, and each test should have EXACTLY ONE requirement associated with it. This can easily be checked via a traceability matrix (which you should also deliver). 15 | 16 | Test cases should mention all necessary preconditions, input values, execution steps, output values, and postconditions. Remember when they are necessary and when they are not... 17 | 18 | It is NOT necessary to make multiple test plans inside a test suite; it is enough for there to be one test plan. 19 | 20 | I expect you to test several edge cases as part of the test plan. 21 | 22 | It is expected that you actually execute the test plan in order to find the defects. There are AT LEAST three. Full credit will be given only to those who properly find and describe at least three. While you are not expected to find all of the defects, a reasonable test plan should definitely find at least three. This is an intentionally target-rich environment. 23 | 24 | ## Format 25 | Please hand in the paper to me with a cover page with: 26 | * The name of the project under test 27 | * The names of the people in the group 28 | * The title "CS 1632 - DELIVERABLE 1: Test Plan and Traceability Matrix" 29 | 30 | There should be a short introduction (a few paragraphs) in which you may note any concerns or difficulties you may have had or anticipate with the testing process. You should also note why you considered certain test cases, how you thought of edge cases, etc. 31 | 32 | This should be followed ON A NEW PAGE by the list of test cases. You may name or number them any way you wish, but be consistent. You may wish to write them out in this format - 33 | 34 | IDENTIFIER: 35 | TEST CASE: 36 | PRECONDITIONS: 37 | INPUT VALUES: 38 | EXECUTION STEPS: 39 | OUTPUT VALUES: 40 | POSTCONDITIONS: 41 | 42 | ON A SEPARATE PAGE, a traceability matrix should be provided mapping the test cases with their associated requirements. Remember that all requirements should map to AT LEAST ONE test case, and all test cases should map to EXACTLY ONE requirement. 43 | 44 | Finally, ON A SEPARATE PAGE, list at least three defects found. The defects should follow the defect reporting template: 45 | 46 | DESCRIPTION: 47 | SUMMARY: 48 | REPRODUCTION STEPS: 49 | EXPECTED BEHAVIOR: 50 | OBSERVED BEHAVIOR: 51 | 52 | Other attributes of a defect (e.g., SEVERITY or IMPACT) are not necessary. The test case which found the defect should be listed as part of the SUMMARY. 53 | 54 | ## Grading 55 | * Introduction: 10% of grade 56 | * Test Plan: 40% of grade 57 | * Traceability Matrix: 20% of grade 58 | * Defects Found and Described: 30% of grade 59 | 60 | ## Coffee Maker Quest 61 | Coffee Maker Quest is a simple game. The goal is get coffee, sugar, and cream, and then drink it so that you can stay up and study. In order to do so, you need to visit several rooms in a house and look around. Once you have obtained all the necessary elements, you win. If you decide to drink before getting all of the necessary elements, you lose. 62 | 63 | You can run it downloading the coffeemaker.jar file and running: 64 | ```bash 65 | java -jar coffeemaker.jar 66 | ``` 67 | 68 | The requirements are listed in the file requirements.txt. 69 | 70 | Please feel free to email me at laboon@cs.pitt.edu or come to office hours to discuss any problems you have. 71 | 72 | -------------------------------------------------------------------------------- /deliverables/1/grading_rubric.txt: -------------------------------------------------------------------------------- 1 | Grading Rubric for D1 2 | 3 | * Introduction: 10% of grade 4 | 5 | Concerns and/or difficulties listed: ____ / 5 6 | 7 | Clearly written: ____ / 5 8 | 9 | * Test Plan: 40% of grade 10 | 11 | Test cases are valid (actual 12 | expected behavior according to reqs) ____ / 15 13 | 14 | Test cases include only EXPECTED 15 | behavior, not OBSERVED behavior: ____ / 5 16 | 17 | Expected behavior and execution 18 | steps are SPECIFIC: ____ / 10 19 | 20 | Clearly written: ____ / 10 21 | 22 | * Traceability Matrix: 20% of grade 23 | 24 | Each requirement & TC listed: ____ / 5 25 | 26 | >= 1 Test case per requirement, and 27 | exactly 1 requirement per TC: ____ / 5 28 | 29 | Test case covers listed reqs: ____ / 10 30 | 31 | * Defects Found and Described: 30% of grade 32 | 33 | Must be >= 3 defects! 10% off for each missing one. 34 | 35 | Defects are Reproducible: ____ / 10 36 | 37 | Include expected/observed behavior: ____ / 10 38 | 39 | Clearly written: ____ / 10 40 | 41 | *** 42 | 43 | Total: ____ / 100 44 | -------------------------------------------------------------------------------- /deliverables/1/requirements.txt: -------------------------------------------------------------------------------- 1 | Requirements for Coffee Maker Quest 2 | 3 | FUN-ITERATION - At each iteration of the game, the user shall be able enter one of six commands - "N" to go North, "S" to go South, "L" to Look for items, "I" for Inventory, "H" for Help, or "D" to Drink. 4 | 5 | FUN-UNKNOWN-COMMAND - If a player enters a command not specified by FUN-ITERATION, the system shall respond with the phrase "What?". 6 | 7 | FUN-INPUT-CAPS - The system shall be case-insensitive in regards to input values; that is, it shall accept capital and lower-case letters and treat them as equivalent. 8 | 9 | FUN-MOVE - The system shall allow a player to move North only if a door exists going North, and South only if a door exists going South. 10 | 11 | FUN-WIN - The player shall win the game if and only if Coffee, Sugar, and Cream have been collected by the player and then drunk. 12 | 13 | FUN-LOSE - The player shall lose the game if and only if the player Drinks but has not collected all of the items (Coffee, Sugar, and Cream). 14 | 15 | FUN-INVENTORY - Upon entering "I" for inventory, the player shall be informed of the items that he/she has collected (consisting of Coffee, Sugar, and Cream). 16 | 17 | FUN-LOOK - Upon entering "L" for Look, the player shall collect any items in the room and those items will be added to the player's inventory. 18 | 19 | FUN-HELP - Upon entering "H" for Help, the player shall be shown a listing of possible commands and what their effects are. 20 | 21 | FUN-UNIQ-ROOM - Each room in the house shall have a unique adjective describing it. 22 | 23 | FUN-UNIQ-ROOM-FURNISHING - Each room in the house shall have one and only one unique furnishing visible to the user upon entering the room. 24 | -------------------------------------------------------------------------------- /deliverables/2/deliverable2.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Quality Assurance 2 | 3 | Spring Semester 2016 4 | DUE 16 FEB 2016 5 | 6 | ## Deliverable 2 7 | 8 | For this assignment, your group will write code and unit tests for an authorized reproduction of Coffee Maker Quest. Groups can be made up of two people, selected by yourselves. If you prefer to work alone, you can also do that. 9 | 10 | Requirements for this program are in the requirements.txt file in this directory. In case of ambiguity, please see the original program as an example of what to display and how the system should work. However, you should not code in defects! 11 | 12 | All code and tests should be on GitHub by the beginning of class on the due date. 13 | 14 | Code coverage should be at an absolute minimum of 80%. 15 | 16 | Tests should be written in JUnit. You should test every public method _except_ main. For each method that you test, ensure that you get a variety of different equivalence classes! Think about different edge and base cases, what the happy path is, etc. 17 | 18 | ## Format 19 | You should turn in a title page with: 20 | * Your name 21 | * The URL of your code and tests on GitHub or a similar repository 22 | * The title "CS 1632 - DELIVERABLE 2: Unit Testing and Code Coverage" 23 | 24 | 25 | Add a short ( < 1 page ) description of issues you faced when writing this code and tests. If any tests you wrote fail, they should be included here, along with why you think that they are failing. 26 | 27 | After this, ON A SEPARATE PAGE, include a screen shot of the executed unit tests. Note that not all tests have to pass! However, if a test doesn't pass, it should be included in the concerns section above. 28 | 29 | ON A SEPARATE PAGE, include either a screenshot or output from a code coverage tool of your tests and the object they cover. 30 | 31 | There is no need to print out the code. It should be on GitHub (or a similar publicly-accessible version control system such as BitKeeper) BY THE BEGINNING OF CLASS. 32 | 33 | At least three (3) unit tests should use test doubles. 34 | 35 | At least three (3) unit tests should use stubbing of methods. 36 | 37 | I expect unit tests for AT LEAST each public method that returns a value (excluding the main method), using a variety of assertions and looking at different failure modes and edge cases. Keep in mind some of the things we learned when doing manual testing; you should be cognizant of equivalence classes, boundary values, etc. and focus on them. 38 | 39 | As a general guideline, code coverage should exceed 80%. 40 | 41 | The program should use appropriate object-oriented design. 42 | 43 | Before each test, add some comments (two or three sentences, on average) explaining what the test is checking. For example... 44 | 45 | // Two LLs with different sizes should never be equal. 46 | // Create two linked lists, both of which have the same value in the initial node, 47 | // but one has several more nodes. 48 | // They should not be equal to each other. 49 | @Test 50 | public void testNotEqualsDiffSizes() { 51 | LinkedList ll11 = new LinkedList(); 52 | LinkedList ll_3elems = new LinkedList(); 53 | 54 | ll11.addToFront(new Node(new Integer(1))); 55 | ll_3elems.addToFront(new Node(new Integer(3))); 56 | ll_3elems.addToFront(new Node(new Integer(2))); 57 | ll_3elems.addToFront(new Node(new Integer(1))); 58 | 59 | assertFalse(ll_3elems.equals(ll11)); 60 | } 61 | 62 | ## Grading 63 | I remind you that grammar and good code count as well as functionality. By good code, I mean - 64 | 65 | No commented-out code (unless there's an EXPLICIT reason, e.g. 66 | ```java 67 | // I couldn't get this assertion to work, but instead used a different assertion, below 68 | // assertEquals(foo, 3); 69 | assertTrue(foo == 3); 70 | ``` 71 | 72 | Properly indented code, e.g. 73 | ```java 74 | public void doSomething() { 75 | doStuff(); 76 | } 77 | ``` 78 | NOT 79 | ```java 80 | public 81 | void 82 | doSomething() 83 | { doStuff(); } 84 | ``` 85 | 86 | Don't misspell words or use bad grammar, in comments or summaries. 87 | 88 | For this project, you should endeavour to get a good sampling of different equivalence classes and success/failure cases. 89 | 90 | ## Grading Breakdown 91 | * Summary and Testing Concerns- 5% 92 | * Screenshot of executed unit tests - 5% 93 | * Unit test coverage report - 10% 94 | * Program Code - 40% 95 | * Includes functionality *and* code quality! 96 | * Test Code - 40% 97 | 98 | Please feel free to email me at bill@billlaboon.com or come to office hours to discuss any problems you have. 99 | -------------------------------------------------------------------------------- /deliverables/2/grading_rubric.txt: -------------------------------------------------------------------------------- 1 | Grading Rubric for D2 2 | 3 | * Summary and Testing Concerns - 5% of grade 4 | 5 | Valid testing concerns and ideas: ____ / 3 6 | 7 | Clearly written: ____ / 2 8 | 9 | * Screenshot of executed unit tests - 5% of grade 10 | 11 | Screenshot included of unit tests: ____ / 5 12 | 13 | * Test coverage report - 10% of grade 14 | 15 | Screenshot included: ____ / 2 16 | 17 | Code coverage >= 80% ____ / 8 18 | 19 | * Program Functionality & Quality - 40% of grade 20 | 21 | Relevant classes: ____ / 5 22 | 23 | Testable OOP design: ____ / 10 24 | 25 | Player can move N/S properly: ____ / 5 26 | 27 | Player can look & check inventory: ____ / 5 28 | 29 | Player can ask for help: ____ / 5 30 | 31 | Player can drink and win/lose: ____ / 5 32 | 33 | Rooms display correctly: ____ / 5 34 | 35 | * Test Code - 40% of grade 36 | 37 | Test happy path for methods: ____ / 8 38 | 39 | Edge cases tested: ____ / 8 40 | 41 | At least 3 test doubles: ____ / 8 42 | 43 | At least 3 stubs: ____ / 8 44 | 45 | Comments included & descriptive: ____ / 8 46 | 47 | *** 48 | 49 | Total: ____ / 100 50 | -------------------------------------------------------------------------------- /deliverables/2/requirements.txt: -------------------------------------------------------------------------------- 1 | Requirements for Coffee Maker Quest 2 | 3 | FUN-ITERATION - At each iteration of the game, the user shall be able enter one of six commands - "N" to go North, "S" to go South, "L" to Look for items, "I" for Inventory, "H" for Help, or "D" to Drink. 4 | 5 | FUN-UNKNOWN-COMMAND - If a player enters a command not specified by FUN-ITERATION, the system shall respond with the phrase "What?". 6 | 7 | FUN-INPUT-CAPS - The system shall be case-insensitive in regards to input values; that is, it shall accept capital and lower-case letters and treat them as equivalent. 8 | 9 | FUN-MOVE - The system shall allow a player to move North only if a door exists going North, and South only if a door exists going South. 10 | 11 | FUN-WIN - The player shall win the game if and only if Coffee, Sugar, and Cream have been collected by the player and then drunk. 12 | 13 | FUN-LOSE - The player shall lose the game if and only if the player Drinks but has not collected all of the items (Coffee, Sugar, and Cream). 14 | 15 | FUN-INVENTORY - Upon entering "I" for inventory, the player shall be informed of the items that he/she has collected (consisting of Coffee, Sugar, and Cream). 16 | 17 | FUN-LOOK - Upon entering "L" for Look, the player shall collect any items in the room and those items will be added to the player's inventory. 18 | 19 | FUN-HELP - Upon entering "H" for Help, the player shall be shown a listing of possible commands and what their effects are. 20 | 21 | FUN-UNIQ-ROOM - Each room in the house shall have a unique adjective describing it. 22 | 23 | FUN-UNIQ-ROOM-FURNISHING - Each room in the house shall have one and only one unique furnishing visible to the user upon entering the room. 24 | -------------------------------------------------------------------------------- /deliverables/3/deliverable3.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Quality Assurance 2 | Spring Semester 2015 3 | 4 | DUE 3 Mar 2032 5 | 6 | ## Deliverable 3 7 | 8 | For this assignment, your group will write systems-level, automated black-box tests for a web site of your choice using the BDD model discussed in class. That is, you will write user stories (features) and scenarios, and then use JUnit and Selenium tests. You should substantially cover a subset of functionality for the project, and note in the "Testing Concerns" section what other aspects you would additionally add for full testing if this were a professional product. 9 | 10 | Tests and code should be on GitHub or another publicly accessible repository by the beginning of class on the due date. 11 | 12 | You are allowed to choose your partner for this deliverable - the only stipulation is that it CANNOT be the same person you partnered with for the first deliverable. You may work on your own if you like, but the requirements for the program are the same as for those people working with a partner. 13 | 14 | ## Format 15 | Every group should have a title page with: 16 | * The name of the project under test 17 | * The names of the people in the group 18 | * The title "CS 1632 - DELIVERABLE 3: Web Testing with BDD" 19 | 20 | First, include a short summary (a few paragraphs) of what you decided to test, and why. I would recommend trying a few test runs with the Selenium IDE on a web page before deciding to test it "for real". Sites with lots of JavaScript or very dynamic page layouts are often poor choices for this deliverable. In particular, Amazon and Facebook have been very difficult for previous projects. You will probably also want to choose a website which you 1) understand well and 2) does not cost money to use. 21 | 22 | Secondly, add a description of issues you faced when writing these tests and problems you would expect going forward based on your experiences. If any tests you wrote fail, they should be included here. Also note if there are any special steps to get the tests running. 23 | 24 | At the end of this section, note where your code (test code and code of project under test, if available) is located. 25 | 26 | After this, there should be a printout or screen shot of the test execution results. 27 | 28 | There is no need to print out the code. It should be put on a publicly-available site such as GitHub BY THE BEGINNING OF CLASS. 29 | 30 | There shall be AT LEAST 4 user stories per group, but there can be more. Each user story should have multiple (at least two) scenarios. There shall be at least an average ("arithmetic mean", if you would like to be specific) of 5 scenarios per user story. 31 | 32 | User stories should all follow the Connextra template. 33 | 34 | Scenarios should all follow the Given/When/Then template (but note that some scenarios may not require all three elements). 35 | 36 | The feature files shall contain a feature and all scenarios for that feature. The JUnit tests shall have the scenario that they are testing written in the comments above the particular test. All tests shall correspond to a scenario and vice-versa. 37 | 38 | Remember that scenarios are FEATURE-level tests; they should discuss things in a way that an intelligent but non-technical user of the software would understand. Remember the differences between scenario tests and specification/unit tests! 39 | 40 | Also remember to focus on a particular subset of functionality in order to get good testing coverage. 41 | 42 | ## Grading 43 | * Summary - 5% 44 | * Testing concerns - 10% 45 | * Screen shot or printout of test results - 5% 46 | * User Stories and Scenarios, WRITTEN IN THE PROPER FORMAT - 30% 47 | * JUnit Tests - 50% 48 | 49 | Reminder that all code (project under test AND test code) should be publicly available, or at least available to me. 50 | 51 | Please feel free to email me or come to office hours to discuss any problems you have. 52 | 53 | -------------------------------------------------------------------------------- /deliverables/3/grading_rubric.md: -------------------------------------------------------------------------------- 1 | ## CS 1632 - Deliverable 3 Grading Rubric 2 | 3 | 4 | ``` 5 | WRITEUP: 6 | 7 | Summary: ____ / 5 8 | 9 | Testing Concerns: ____ / 10 10 | 11 | Screenshot / Printout of Results: ____ / 5 12 | 13 | USER STORIES - 14 | 15 | Written using proper templates: _____ / 5 16 | 17 | User Story quality: _____ / 10 18 | (User-facing, valid) 19 | 20 | Scenario quality: _____ / 15 21 | (Associated with US, well-written, 22 | valid, cover happy path & edge cases) 23 | 24 | JUNIT TESTS - 25 | 26 | Required number of tests: _____ / 10 27 | 28 | Assertions used properly: _____ / 10 29 | 30 | Test the scenarios properly: _____ / 20 31 | 32 | Coded well (incl. comments): _____ / 10 33 | 34 | Total: 35 | 36 | _________________ / 100 37 | 38 | ``` -------------------------------------------------------------------------------- /deliverables/4/deliverable4.md: -------------------------------------------------------------------------------- 1 | # CS1632 Deliverable 4 2 | Spring Semester 2015 3 | 4 | DUE 24 MAR 2016 5 | 6 | ## Deliverable 4 7 | 8 | For this assignment, you will either: 9 | 1. Develop a combinatorial test and explain it 10 | 2. Write your own JUnit-based property-based test to check Arrays.sort(int[] arr) method in Java 11 | 12 | There are no partners for this deliverable. 13 | 14 | For the combinatorial testing, you will need to develop a combinatorial test with at least five parameters (variables). You may generate the combinatorial table using the NIST ACTS tool or another combinatorial generator. To download the NIST ACTS tool, you may follow the instructions here: http://csrc.nist.gov/groups/SNS/acts/documents/comparison-report.html. I recommend you do this as soon as possible to ensure everything works on your system and it can be downloaded. Since this requires at least one manual step on the part of NIST researchers, do it as soon as possible! 15 | 16 | You will come up with your own system (software) to test. This can be already existing software, or software that you make up. Build the system using NIST ACTS (or a similar tool), which will generate a covering array. You should make tests which covers all pairs (2-way interactions). Based on this covering array, generate a test plan (in the same format as Deliverable 1) which indicates how to test all of pairs (as discussed in class). 17 | 18 | You may not use "fonts and font effects" as your software. You should come up with your own idea. 19 | 20 | For the JUnit-based property-based tests, generate a minimum of 100 different random arrays of different sizes, and test different properties (many examples were discussed in the lecture on property-based testing) of sorting them. You may use Java's built-in Arrays.sort() method. You should test at least three different properties of each sorted array. You should use traditional JUnit test procedures (e.g., use assertions, don't use System.out.println during normal execution, etc.) Since we are testing a built-in Java method, I don't expect any failures, but who knows, you may be the one to find a bug in Java's own libraries! 21 | 22 | You may do this either all in one JUnit test method, or with one method per property. 23 | 24 | For either of these projects, I expect an approximately one-page (3 or 4 paragraphs) description of why you chose this project, how you went about doing it, any issues you faced, and what you learned. If you are doing the combinatorial testing option, use this space to explain what you are testing. If it is existing software, please provide a link; if it is software that you have thought of yourself, please explain it in enough detail that I can understand what you are testing. 25 | 26 | Both of these options also will require a screenshot. Please put this on its own page. For the combinatorial testing project, I expect a screenshot of NIST ACTS or whatever other tool you are using, specifically showing the all-pairs covering array. For the property-based testing project, I expect a screenshot of the executed JUnit test(s). 27 | 28 | ## Format 29 | Every assignment should have a title page with: 30 | * The name of the student 31 | * The title "CS 1632 - DELIVERABLE 4" 32 | * The name of the project you worked on - "COMBINATORIAL TESTING" or "PROPERTY-BASED TESTING" 33 | 34 | ## Grading (Combinatorial Testing) 35 | * Summary - 10% 36 | * Screenshot of NIST ACTS (or similar) tool with your system - 20% 37 | * Test Plan - 70% 38 | 39 | ## Grading (Property-Based Testing) 40 | * Summary - 10% 41 | * Screenshot of completed test - 20% 42 | * Test Code - 70% 43 | 44 | Please feel free to email me at bill@billlaboon.com or come to office hours to discuss any problems you have. 45 | 46 | -------------------------------------------------------------------------------- /deliverables/4/grading_rubric_combinatorial.md: -------------------------------------------------------------------------------- 1 | ## Grading (Combinatorial Testing) 2 | 3 | Summary - 10% 4 | 5 | Why student chose project: ______ / 2 6 | 7 | How it was done: ______ / 3 8 | 9 | Issues faced: ______ / 5 10 | 11 | Screenshot of completed test - 20% 12 | 13 | Screenshot included: ______ / 5 14 | 15 | Shows all-pairs covering array: ______ / 15 16 | 17 | Test Plan - 70% 18 | 19 | If _observed behavior_ is shown instead of _expected behavior_, 20 | then -15 points. 21 | 22 | Test cases written properly (incl. 23 | description, expected behavior, etc) ______ / 25 24 | 25 | Test cases cover all pairs in 26 | covering array: ______ / 25 27 | 28 | 29 | Test case steps are specific: ______ / 10 30 | 31 | Test case are clearly written: ______ / 10 32 | 33 | -------------------------------------------------------------------------------- /deliverables/4/grading_rubric_property_based.md: -------------------------------------------------------------------------------- 1 | ## Property-Based Testing Grading Rubric 2 | 3 | Summary - 10% 4 | 5 | Why student chose project: ______ / 2 6 | 7 | How it was done: ______ / 3 8 | 9 | Issues faced: ______ / 5 10 | 11 | Screenshot of completed test - 20% 12 | 13 | Screenshot included: _____ / 5 14 | 15 | Shows successful completion: _____ / 15 16 | (or if failed, explained why in 17 | summary) 18 | 19 | Test Code - 70% 20 | 21 | Note: if < 3 properties tested for, then maximum score for this section is 35. 22 | If not in JUnit format, as specified, then maximum score for this section is 23 | 35. If neither, maximum score is 20. If written as standard unit tests instead 24 | of testing properties, maximum score is 20. 25 | 26 | >= 100 arrays passed in: _____ / 10 27 | 28 | Valid and different properties 29 | selected: _____ / 25 30 | (idempotent, item in input is also 31 | in output, item not in input is 32 | not in output, etc.) 33 | 34 | Unit tests testing what they say _____ / 25 35 | they are testing: 36 | 37 | Code aesthetics (proper indentation, _____ / 10 38 | tabs, good comments, etc.) 39 | -------------------------------------------------------------------------------- /deliverables/5/deliverable5.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Quality Assurance 2 | Spring Semester 2016 3 | 4 | DUE 5 APR 2016 5 | 6 | ## Deliverable 5 7 | 8 | For this assignment, you (NOT a group) will profile a Conway's Game of Life simulation, and improve its performance by refactoring several methods (to be determined by the results of the profiling). This will consist of several parts: 9 | 10 | 1. Profiling (before) to determine which method is the most CPU-intensive 11 | 2. Adding pinning tests (in the form of manual and/or unit tests) to show that the functionality is unchanged by your modifications 12 | 2. Refactoring the method to be more performant 13 | 3. Profiling (after) showing that your rewrite helped make your method more performant 14 | 15 | Code will be on Github (https://github.com/laboon/SlowLifeGUI). Please _fork_ it and create your own repository. 16 | 17 | There are no partners for this deliverable. 18 | 19 | ## Format 20 | Every assignment should have a title page with: 21 | * The name of the project under test (JavaLife) 22 | * The name of the student 23 | * The title "CS 1632 - DELIVERABLE 5: Performance Testing Conway's Game of Life" 24 | 25 | There is no need to print out the code. It should be available on GitHub or a similar code-sharing site BY THE BEGINNING OF CLASS. 26 | 27 | In order to determine the "hot spots" of the application, you will need to run a profiler such as VisualVM (download at https://visualvm.java.net/). Using a profiler, determine a method you can use to measurably increase the speed of the application without modifying behavior. 28 | 29 | The program is an implementation of Conway's Game of Life (https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life). You can change the state of a cell (from living to dead) by clicking on one of the buttons. Cells which are currently alive have an X and a red background; cells that are dead now, but were at any point alive during the current run, will have a green background. 30 | 31 | There are several other buttons which perform different functions: 32 | 33 | 1. Run - this will run one iteration of the Game of Life 34 | 2. Run Continuous - This will run iterations until you press the Stop button. 35 | 3. Stop - This will stop the current "Run Continuous" run. It will have no effect if the program is not running continuously already. 36 | 4. Write - This will write the state of the system to a backup file, to be loaded later. 37 | 5. Undo - This will undo the previous iteration. It will only work for one iteration (that is, you cannot do multiple undos to get back multiple iterations). 38 | 6. Load - This will load a previously-saved backup file (created using the Write button) to the current world. 39 | 7. Clear - This will clear the current world. 40 | 41 | The application accepts one command line argument, specifying the size of the world (e.g., if you enter 10, then you will create a 10 x 10 world). I recommend you have a size of 15 or thereabouts, depending on the size of the screen. 42 | 43 | There are at least THREE major performance issues with this code. They could be in any of the pieces of functionality of the program! I recommend you use exploratory testing to determine where they are before profiling the system. 44 | 45 | For the summary, describe how you profiled the application and determined the methods to refactor, and why you changed what you did. The summary should not be more than a few paragraphs - definitely less than a page. 46 | 47 | After this, include screenshots of VisualVM (or another profiler, if you use that) both before and after the refactor. These screenshots should include the relevant sections that let you see what method to refactor. 48 | 49 | As part of this assignment, you should create "pinning tests" (as described in the section on legacy code earlier - please see the textbook if you need a refresher). These pinning tests should check that the behavior of a modified method was not changed by your refactor. This program should work EXACTLY the same as before, except it should be faster and take up less CPU time. The only exception is if you come across an error and fix it - no points will be taken off as long as you note it in your summary. 50 | 51 | There should be a *bare minimum* of least three pinning tests per method modified (check different edge cases). Some method modifications may be difficult to unit test; you may write several manual test cases instead of unit tests for these. If you do so, however, please note in the Summary why you did it. 52 | 53 | ## Grading 54 | * Summary - 10% 55 | * VisualVM (or other profiler) screenshots (before and after) - 10% 56 | * Method choice and refactoring - 40% 57 | * Pinning Tests - 40% 58 | 59 | Please feel free to email me at laboon@cs.pitt.edu or come to office hours to discuss any problems you have. 60 | 61 | -------------------------------------------------------------------------------- /deliverables/6/deliverable6.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Quality Assurance 2 | Spring Semester 2016 3 | ASSIGNED 5 APR 2016 4 | DUE 19 APR 2016 5 | 6 | ## Deliverable 6 - THE FINAL DELIVERABLE 7 | 8 | For this assignment, you will perform some software testing of whatever kind you find interesting. It can be from the following list of topics gone over in class, or you can research and use your own. If you are using your own, please check with me before working on it. You may work alone or in a group of two (with a partner of your choice, even if it is somebody with whom you already worked). 9 | 10 | This software testing project should *not* be a simple re-do of something you have already done. It should extend the work that you have already done, or work on an entirely different field of testing (such as penetration testing). 11 | 12 | ### Possible Ideas (not an exhaustive list!) 13 | 14 | 1. Deeper testing of a web application (perhaps including JavaScript) 15 | 2. Detailed analysis and performance testing of an application 16 | 3. Security Testing 17 | 3. Penetration Testing (be sure it's on something you HAVE PERMISSION TO DO!) 18 | 6. More In-Depth Property-Based Testing 19 | 7. Formal Analysis / Verification of a program 20 | 8. Algorithmic analysis of a combinatorial algorithm 21 | 9. Writing a new app from scratch using TDD or BDD 22 | 10. Adding tests to an existing program 23 | 11. In-depth metrics analysis (code coverage, etc.) of a project 24 | 25 | The final project should be approximately as much work as one of the 'larger' assignments for this class (e.g. Deliverable 1 or 2). Please see the grading rubric below for how the final deliverable will be graded. 26 | 27 | You may use a project you have already worked on, or a totally new one if you prefer. Any tests and/or code should be on GitHub or another publicly accessible repository. 28 | 29 | The quality of your write-up can and will impact your grade. For a perfect grade, I expect professional quality on ALL aspects of your testing, including: 30 | 31 | 1. Proper grammar, punctuation, etc. 32 | 2. No commented-out code, UNLESS there is a specified reason for it, e.g.: 33 | * `// This doesn't work, but this is what I would do if I could...` 34 | * `// Uncomment the following line to run additional long-running tests)` 35 | 3. No inconsistent spacing/tabs/brace settings. I don't care which you use, but make 36 | it consistent. 37 | 4. Good variable names, good coding style in general. 38 | 39 | Since there are so many possible projects, this deliverable has a much more subjective grading scale. 40 | 41 | * **Perfect Score** - Absolutely no problems, working on a challenging system with excellent 42 | test coverage. Tests and kinds of tests make perfect sense and are described well. 43 | Summary, testing concerns, and quality assessment are easy to read with no grammatical 44 | errors. 45 | * **A** - Very minor problems, working on a reasonable system with good test coverage. Tests and 46 | kinds of tests are appropriate and described well. Summary, testing concerns, and 47 | quality assessment have only minor issues. 48 | * **B** - A few issues, but overall decent test coverage. Working on a simpler system. 49 | Tests and kinds of tests could have been better. Summary, testing concerns, and 50 | quality assessment have a few issues. 51 | * **C** - Test coverage is not good; many tests are repetitive or unnecessary. Working on a 52 | trivial system (e.g. a Linked List). Tests and kinds of tests are only vaguely appropriate. 53 | Summary, testing concerns, and quality assessment have major issues. 54 | * **D** - Test coverage is horrible; tests do not do what they say, there aren't enough of them, 55 | and are not described at all. Summary, testing concerns, and quality assessment are 56 | filled with errors, grammatical and factual. 57 | * **F** - Test coverage is basically nonexistent; tests are testing the wrong thing entirely 58 | (e.g. a "usability test" which only checks a math function repeatedly) or missing. 59 | Summary, testing concerns, and quality assessment are unreadable or missing 60 | entirely. 61 | 62 | ## Format 63 | Every group should have a title page with: 64 | * The name of the project under test 65 | * The names of the people in the group 66 | * The title "CS 1632 - FINAL DELIVERABLE" 67 | * A title for the project 68 | 69 | First, include a short summary (a few paragraphs) of what you decided to test, what kinds of 70 | testing you are doing, and why. 71 | 72 | Then, add a description of issues you faced when writing these tests and problems you would 73 | expect going forward based on your experiences. Include here any additional tests or kinds of 74 | tests you would like to run to get a better assessment of the quality of the system. This should 75 | also be a few paragraphs. 76 | 77 | I also expect an assessment of quality of the project under test. This will include: 78 | 79 | 1. A list of any failed tests or problem areas 80 | 2. An overall assessment of quality 81 | 3. Your recommendation on whether or not the product is ready to be released 82 | 83 | You should write this in the style of convincing a manager whether or not a system is ready to be released. If what you are doing does not lend itself to this (e.g., algorithmic analysis), then write how your analysis would be useful to a commercial system, as if you are writing to a manager. 84 | 85 | At the end of this section, note where your code (test code and code of project under test) is located, if applicable. 86 | 87 | After this, there should be printouts or screen shots of the test execution results, if applicable. There is no need to print out the code, if any. It should be put on a publicly-available site such as GitHub BY THE BEGINNING OF CLASS. 88 | 89 | ## Grading 90 | * Summary - 10% 91 | * Testing concerns / Concerns going forward - 10% 92 | * Assessment of Quality - 10% 93 | * Project - 70% 94 | 95 | If an assessment of quality or testing concerns would not fit into your project (for example, you are doing algorithmic analysis), please inform me ahead of time. 96 | 97 | Reminder that all code (project under test AND test code) should be publicly available, or at least available to me. 98 | 99 | Please feel free to email me at laboon@cs.pitt.edu or come to office hours to discuss any problems you have. 100 | 101 | I urge you to talk to me as soon as possible if you experience any problems. 102 | -------------------------------------------------------------------------------- /grading.md: -------------------------------------------------------------------------------- 1 | # Grading Guidelines 2 | 3 | The quality of your write-up can and will impact your grade. For a perfect grade, I expect 4 | professional quality on ALL aspects of your testing, including: 5 | 6 | 1. Proper grammar, punctuation, etc. 7 | 2. No commented-out code, UNLESS there is a specified reason for it, e.g.: 8 | * `// This doesn't work, but this is what I would do if I could...` 9 | * `// Uncomment the following line to run additional long-running tests)` 10 | 11 | 3. No inconsistent spacing/tabs/brace settings. I don't care which you use, but make 12 | it consistent. 13 | 4. Good variable names, good coding style in general. 14 | 15 | * **Perfect Score** - Absolutely no problems, working on a challenging system with excellent 16 | test coverage. Tests and kinds of tests make perfect sense and are described well. 17 | Summary, testing concerns, and quality assessment are easy to read with no grammatical 18 | errors. 19 | * **A** - Minor problems, working on a reasonable system with good test coverage. Tests and 20 | kinds of tests are appropriate and described well. Summary, testing concerns, and 21 | quality assessment have only minor issues. 22 | * **B** - A few issues, but overall decent test coverage. Tests and kinds of tests could have 23 | been better. Summary, testing concerns, and quality assessment have a few issues. 24 | * **C** - Test coverage is not good; many tests are repetitive or unnecessary. Tests and kinds 25 | of tests are only vaguely appropriate. Summary, testing concerns, and quality 26 | assessment have major issues. 27 | * **D** - Test coverage is horrible; tests do not do what they say, there aren't enough of them, 28 | and are not described at all. Summary, testing concerns, and quality assessment are 29 | filled with errors, grammatical and factual. 30 | * **F** - Test coverage is basically nonexistent; tests are testing the wrong thing entirely 31 | (e.g. a "usability test" which only checks a math function repeatedly) or missing. 32 | Summary, testing concerns, and quality assessment are unreadable or missing 33 | entirely. 34 | -------------------------------------------------------------------------------- /lectures/lecture1-intro.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture1-intro.pdf -------------------------------------------------------------------------------- /lectures/lecture11-intro-to-tdd.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture11-intro-to-tdd.pdf -------------------------------------------------------------------------------- /lectures/lecture12-writing-testable-code.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture12-writing-testable-code.pdf -------------------------------------------------------------------------------- /lectures/lecture13-intro-to-bdd.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture13-intro-to-bdd.pdf -------------------------------------------------------------------------------- /lectures/lecture14-selenium.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture14-selenium.pdf -------------------------------------------------------------------------------- /lectures/lecture15-web-testing-with-junit.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture15-web-testing-with-junit.pdf -------------------------------------------------------------------------------- /lectures/lecture16-combinatorial-testing.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture16-combinatorial-testing.pdf -------------------------------------------------------------------------------- /lectures/lecture17-property-based-formal-verification.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture17-property-based-formal-verification.pdf -------------------------------------------------------------------------------- /lectures/lecture18-perf-testing-1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture18-perf-testing-1.pdf -------------------------------------------------------------------------------- /lectures/lecture19-perf-testing-2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture19-perf-testing-2.pdf -------------------------------------------------------------------------------- /lectures/lecture2-testing-basics.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture2-testing-basics.pdf -------------------------------------------------------------------------------- /lectures/lecture20-security-testing.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture20-security-testing.pdf -------------------------------------------------------------------------------- /lectures/lecture21-stakeholder-interaction.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture21-stakeholder-interaction.pdf -------------------------------------------------------------------------------- /lectures/lecture22-stakeholder-interaction-2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture22-stakeholder-interaction-2.pdf -------------------------------------------------------------------------------- /lectures/lecture23-penetration-testing.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture23-penetration-testing.pdf -------------------------------------------------------------------------------- /lectures/lecture3-requirements.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture3-requirements.pdf -------------------------------------------------------------------------------- /lectures/lecture4-SUPPLEMENTAL-exploratory.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture4-SUPPLEMENTAL-exploratory.pdf -------------------------------------------------------------------------------- /lectures/lecture4-test-plans.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture4-test-plans.pdf -------------------------------------------------------------------------------- /lectures/lecture5-defects.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture5-defects.pdf -------------------------------------------------------------------------------- /lectures/lecture6-breaking-software.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture6-breaking-software.pdf -------------------------------------------------------------------------------- /lectures/lecture7-automated-tests.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture7-automated-tests.pdf -------------------------------------------------------------------------------- /lectures/lecture8-intro-unit-tests.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture8-intro-unit-tests.pdf -------------------------------------------------------------------------------- /lectures/lecture9-advanced-unit-testing.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/lectures/lecture9-advanced-unit-testing.pdf -------------------------------------------------------------------------------- /old_deliverables/deliverable1/coffeemaker.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laboon/cs1632/df2dd2c6ddc0839e504580ba88615d6917cd0396/old_deliverables/deliverable1/coffeemaker.jar -------------------------------------------------------------------------------- /old_deliverables/deliverable1/deliverable1-tips.md: -------------------------------------------------------------------------------- 1 | 1. Be as precise as possible. Don't say "Ensure that you see the correct value"; say that "You should see the text 'A Smart door leads South'". Don't say that you should "go north"; say "Type n ". 2 | 3 | 2. Remember to put down all the necessary steps and preconditions! 4 | 5 | 3. The "Test Case" label is for a description of the test case in plain English. For example, "Ensure that user can go North from the starting room." 6 | 7 | 4. You do not need to exhaustively test the application! If you find yourself with more than three test cases per requirement, you may be over-testing. This is admirable in software where human life is at stake, but this is only your grade. 8 | 9 | 5. This is a test plan, not a test run - remember the difference. 10 | 11 | 6. Note that if you find yourself in a magical land - even if you get transported back to where you came from - you may have gone through a door which did not exist. Think about if that meets the requirements or not. 12 | 13 | 7. Remember that tests should be reproducible - starting with all preconditions met, anybody should be able to reproduce the same steps and get the same results. If they can't, rethink your preconditions! -------------------------------------------------------------------------------- /old_deliverables/deliverable1/deliverable1.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Testing 2 | Fall Semester 2015 3 | 4 | * DUE: 24 SEP 2015 5 | 6 | ## Deliverable 1 7 | 8 | For this assignment, your group will determine a test plan for the simple game Coffee Maker Quest, based on the requirements listed. There are several known defects in the software; you will need to find and report on at least three. Additionally, a traceability matrix showing the mapping of test cases to requirements is required. 9 | 10 | There should be at least as many test cases as requirements, although I would normally expect several test cases per requirement. As a general rule, I'd estimate that the number of test cases is at least 2x the number of requirements. If the number of test cases is more than 3x the number of requirements, you are probably overtesting. 11 | 12 | Each requirement should have AT LEAST one test case associated with it, and each test should have EXACTLY ONE requirement associated with it. This can easily be checked via a traceability matrix (which you should also deliver). 13 | 14 | Test cases should mention all necessary preconditions, input values, execution steps, output values, and postconditions. Remember when they are necessary and when they are not... 15 | 16 | It is NOT necessary to make multiple test plans inside a test suite; it is enough for there to be one test plan. 17 | 18 | I expect you to test several edge cases as part of the test plan. 19 | 20 | It is expected that you actually execute the test plan in order to find the defects. There are AT LEAST three. Full credit will be given only to those who properly find and describe at least three. 21 | 22 | ## Format 23 | Please hand in the paper to me with a cover page with: 24 | * The name of the project under test 25 | * The names of the people in the group 26 | * The title "CS 1632 - DELIVERABLE 1: Test Plan and Traceability Matrix" 27 | 28 | There should be a short introduction (a few paragraphs) in which you may note any concerns or difficulties you may have had or anticipate with the testing process. You should also note why you considered certain test cases, how you thought of edge cases, etc. 29 | 30 | This should be followed ON A NEW PAGE by the list of test cases. You may name or number them any way you wish, but be consistent. You may wish to write them out in this format - 31 | 32 | IDENTIFIER: 33 | TEST CASE: 34 | PRECONDITIONS: 35 | INPUT VALUES: 36 | EXECUTION STEPS: 37 | OUTPUT VALUES: 38 | POSTCONDITIONS: 39 | 40 | ON A SEPARATE PAGE, a traceability matrix should be provided mapping the test cases with their associated requirements. Remember that all requirements should map to AT LEAST ONE test case, and all test cases should map to EXACTLY ONE requirement. 41 | 42 | Finally, ON A SEPARATE PAGE, list at least three defects found. The defects should follow the defect reporting template: 43 | 44 | DESCRIPTION: 45 | SUMMARY: 46 | REPRODUCTION STEPS: 47 | EXPECTED BEHAVIOR: 48 | OBSERVED BEHAVIOR: 49 | 50 | Other attributes of a defect (e.g., SEVERITY or IMPACT) are not necessary. The test case which found the defect should be listed as part of the SUMMARY. 51 | 52 | ## Grading 53 | * Introduction: 10% of grade 54 | * Test Plan: 40% of grade 55 | * Traceability Matrix: 20% of grade 56 | * Defects Found and Described: 30% of grade 57 | 58 | ## Coffee Maker Quest 59 | Coffee Maker Quest is a simple game. The goal is get coffee, sugar, and cream, and then drink it so that you can stay up and study. In order to do so, you need to visit several rooms in a house and look around. Once you have obtained all the necessary elements, you win. If you decide to drink before getting all of the necessary elements, you lose. 60 | 61 | You can run it downloading the coffeemaker.jar file and running: 62 | ```bash 63 | java -jar coffeemaker.jar 64 | ``` 65 | 66 | The requirements are listed in the file requirements.txt. 67 | 68 | Please feel free to email me at bill@billlaboon.com or come to office hours to discuss any problems you have. 69 | 70 | -------------------------------------------------------------------------------- /old_deliverables/deliverable1/requirements.txt: -------------------------------------------------------------------------------- 1 | Requirements for Coffee Maker Quest 2 | 3 | FUN-ITERATION - At each iteration of the game, the user shall be able enter one of six commands - "N" to go North, "S" to go South, "L" to Look for items, "I" for Inventory, "H" for Help, or "D" to Drink. 4 | 5 | FUN-UNKNOWN-COMMAND - If a player enters a command not specified by FUN-ITERATION, the system shall respond with the phrase "What?". 6 | 7 | FUN-INPUT-CAPS - The system shall be case-insensitive in regards to input values; that is, it shall accept capital and lower-case letters and treat them as equivalent. 8 | 9 | FUN-MOVE - The system shall allow a player to move North only if a door exists going North, and South only if a door exists going South. 10 | 11 | FUN-WIN - The player shall win the game if and only if Coffee, Sugar, and Cream have been collected by the player and then drunk. 12 | 13 | FUN-LOSE - The player shall lose the game if and only if the player Drinks but has not collected all of the items (Coffee, Sugar, and Cream). 14 | 15 | FUN-INVENTORY - Upon entering "I" for inventory, the player shall be informed of the items that he/she has collected (consisting of Coffee, Sugar, and Cream). 16 | 17 | FUN-LOOK - Upon entering "L" for Look, the player shall collect any items in the room and those items will be added to the player's inventory. 18 | 19 | FUN-HELP - Upon entering "H" for Help, the player shall be shown a listing of possible commands and what their effects are. 20 | 21 | FUN-UNIQ-ROOM - Each room in the house shall have a unique adjective describing it. 22 | 23 | FUN-UNIQ-ROOM-FURNISHING - Each room in the house shall have one and only one unique furnishing visible to the user upon entering the room. 24 | -------------------------------------------------------------------------------- /old_deliverables/deliverable2/deliverable2.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Quality Assurance 2 | Fall Semester 2015 3 | DUE 13 OCT 2015 4 | 5 | ## Deliverable 2 6 | 7 | For this assignment, you (not a group) will write code and unit tests for CitySim9000. 8 | 9 | Requirements for this program are in the requirements.txt file in this directory. Sample output is also provided for several runs of the program. In case of ambiguity, please see the sample output as an example of what to display and how the system should work. 10 | 11 | All code and tests should be on GitHub or a similar repository accessible to me. 12 | 13 | ## Format 14 | You should turn in a title page with: 15 | * Your name 16 | * The URL of your code and tests on GitHub or a similar repository 17 | * The title "CS 16932 - DELIVERABLE 2: Unit Testing and Code Coverage" 18 | 19 | 20 | Add a short ( < 1 page ) description of issues you faced when writing this code and tests. If any tests you wrote fail, they should be included here, along with why you think that they are failing. 21 | 22 | After this, ON A SEPARATE PAGE, include a screen shot of the executed unit tests. Note that not all tests have to pass! However, if a test doesn't pass, it should be included in the concerns section above. 23 | 24 | ON A SEPARATE PAGE, include either a screenshot or output from a code coverage tool of your tests and the object they cover. 25 | 26 | There is no need to print out the code. It should be on GitHub (or a similar publicly-accessible version control system such as BitKeeper) BY THE BEGINNING OF CLASS. 27 | 28 | At least three (3) unit tests should use test doubles. 29 | 30 | At least three (3) unit tests should use stubbing of methods. 31 | 32 | I expect unit tests for AT LEAST each public method that returns a value (excluding the main method), using a variety of assertions and looking at different failure modes and edge cases. Keep in mind some of the things we learned when doing manual testing; you should be cognizant of equivalence classes, boundary values, etc. and focus on them. 33 | 34 | As a general guideline, code coverage should exceed 80%. 35 | 36 | The program should use appropriate object-oriented design. 37 | 38 | Before each test, add some comments (two or three sentences, on average) explaining what the test is checking. For example... 39 | 40 | // Two LLs with different sizes should never be equal. 41 | // Create two linked lists, both of which have the same value in the initial node, 42 | // but one has several more nodes. 43 | // They should not be equal to each other. 44 | @Test 45 | public void testNotEqualsDiffSizes() { 46 | LinkedList ll11 = new LinkedList(); 47 | LinkedList ll_3elems = new LinkedList(); 48 | 49 | ll11.addToFront(new Node(new Integer(1))); 50 | ll_3elems.addToFront(new Node(new Integer(3))); 51 | ll_3elems.addToFront(new Node(new Integer(2))); 52 | ll_3elems.addToFront(new Node(new Integer(1))); 53 | 54 | assertFalse(ll_3elems.equals(ll11)); 55 | } 56 | 57 | ## Grading 58 | I remind you that grammar and good code count as well as functionality. By good code, I mean - 59 | 60 | No commented-out code (unless there's an EXPLICIT reason, e.g. 61 | ```java 62 | // I couldn't get this assertion to work, but instead used a different assertion, below 63 | // assertEquals(foo, 3); 64 | assertTrue(foo == 3); 65 | ``` 66 | 67 | Properly indented code, e.g. 68 | ```java 69 | public void doSomething() { 70 | doStuff(); 71 | } 72 | ``` 73 | NOT 74 | ```java 75 | public 76 | void 77 | doSomething() 78 | { doStuff(); } 79 | ``` 80 | 81 | Don't misspell words or use bad grammar, in comments or summaries. 82 | 83 | For this project, you should endeavour to get a good sampling of different equivalence classes and success/failure cases. 84 | 85 | ## Grading Breakdown 86 | * Summary and Testing Concerns- 10% 87 | * Screenshot of executed unit tests - 10% 88 | * Unit test coverage report - 10% 89 | * Program Code - 30% 90 | * Test Code - 40% 91 | 92 | Please feel free to email me at bill@billlaboon.com or come to office hours to discuss any problems you have. 93 | -------------------------------------------------------------------------------- /old_deliverables/deliverable2/requirements.md: -------------------------------------------------------------------------------- 1 | ## CitySim9000 Requirements 2 | 3 | FUN-CITY-LOCS. The program shall simulate a city with four locations: Mall, Bookstore, Coffee Shop, and University. 4 | 5 | FUN-OUTSIDE-CITY. A fifth location, Outside City, shall stand for a driver outside the city limits. 6 | 7 | FUN-AVENUES. The city shall have two one-way avenues, Fourth Ave and Fifth Ave. Fourth Ave shall connect, in order, Outside City -> Mall -> Bookstore -> Outside City. Fifth Ave shall connect, in order, Outside City -> University -> Coffee Shop -> Outside City. 8 | 9 | FUN-STREETS. The city shall have two two-way streets, Meow St. and Chirp St. Meow St. shall connect Coffee Shop and Mall. Chirp St. shall connect Bookstore and University. 10 | 11 | FUN-FIVE-DRIVERS. Five drivers, numbered 0 through 4, shall traverse the city, one after the other. 12 | 13 | FUN-START-LOC. A driver may start in any of the five locations listed in FUN-CITY-LOCS or FUN-OUTSIDE-CITY. 14 | 15 | FUN-ITERATION. At each iteration, a Driver will drive from the current Location to one of the possible Locations that can be reached from the original Location. The decision shall be made pseudorandomly based on a seed passed in from the command line, as covered by FUN-ARGS. 16 | 17 | FUN-END. The simulation shall end if a Driver encounters the Outside City Location and it is not the first iteration of the simulation. If it is the first iteration of the simulation, behavior is covered by FUN-FIRST-LOC. 18 | 19 | FUN-FIRST-LOC. If the Location the Driver starts is Outside City, the Driver will pseudorandomly choose to go to University (via Fifth Ave) or Mall (via Fourth Ave). 20 | 21 | FUN-ARGS. The system shall accept an integer seed from the command line for the pseudorandom number generator. No other arguments shall be accepted. If no arguments are provided, more than one argument is provided, or the single argument is not an integer, the system shall inform the user and exit. 22 | 23 | It may be easier to understand the map of the city visually. 24 | 25 | ``` 26 | City Map 27 | 28 | ---> [Mall] ----> [Bookstore] ----> Fourth Ave. 29 | A | A | 30 | Meow St.| | | | Chirp St. 31 | | V | V 32 | <--- [Coffee] <-- [University] <--- Fifth Ave. 33 | ``` 34 | -------------------------------------------------------------------------------- /old_deliverables/deliverable2/sample_output.txt: -------------------------------------------------------------------------------- 1 | (712) $ java CitySim9000 9 2 | Driver 0 heading from Outside City to University via Fifth Ave. 3 | Driver 0 heading from University to Coffee Shop via Fifth Ave. 4 | Driver 0 heading from Coffee Shop to Mall via Meow St. 5 | Driver 0 heading from Mall to Bookstore via Fourth Ave. 6 | Driver 0 heading from Bookstore to University via Chirp St. 7 | Driver 0 heading from University to Bookstore via Chirp St. 8 | Driver 0 heading from Bookstore to Outside City via Fourth Ave. 9 | Driver 0 has left the city! 10 | ----- 11 | Driver 1 heading from Coffee Shop to Mall via Meow St. 12 | Driver 1 heading from Mall to Bookstore via Fourth Ave. 13 | Driver 1 heading from Bookstore to University via Chirp St. 14 | Driver 1 heading from University to Bookstore via Chirp St. 15 | Driver 1 heading from Bookstore to Outside City via Fourth Ave. 16 | Driver 1 has left the city! 17 | ----- 18 | Driver 2 heading from Bookstore to Outside City via Fourth Ave. 19 | Driver 2 has left the city! 20 | ----- 21 | Driver 3 heading from Coffee Shop to Mall via Meow St. 22 | Driver 3 heading from Mall to Bookstore via Fourth Ave. 23 | Driver 3 heading from Bookstore to University via Chirp St. 24 | Driver 3 heading from University to Bookstore via Chirp St. 25 | Driver 3 heading from Bookstore to University via Chirp St. 26 | Driver 3 heading from University to Bookstore via Chirp St. 27 | Driver 3 heading from Bookstore to Outside City via Fourth Ave. 28 | Driver 3 has left the city! 29 | ----- 30 | Driver 4 heading from University to Coffee Shop via Fifth Ave. 31 | Driver 4 heading from Coffee Shop to Mall via Meow St. 32 | Driver 4 heading from Mall to Bookstore via Fourth Ave. 33 | Driver 4 heading from Bookstore to Outside City via Fourth Ave. 34 | Driver 4 has left the city! 35 | ----- 36 | 37 | (713) $ java CitySim9000 10 38 | Driver 0 heading from University to Bookstore via Chirp St. 39 | Driver 0 heading from Bookstore to University via Chirp St. 40 | Driver 0 heading from University to Bookstore via Chirp St. 41 | Driver 0 heading from Bookstore to Outside City via Fourth Ave. 42 | Driver 0 has left the city! 43 | ----- 44 | Driver 1 heading from University to Bookstore via Chirp St. 45 | Driver 1 heading from Bookstore to University via Chirp St. 46 | Driver 1 heading from University to Bookstore via Chirp St. 47 | Driver 1 heading from Bookstore to University via Chirp St. 48 | Driver 1 heading from University to Coffee Shop via Fifth Ave. 49 | Driver 1 heading from Coffee Shop to Mall via Meow St. 50 | Driver 1 heading from Mall to Coffee Shop via Meow St. 51 | Driver 1 heading from Coffee Shop to Outside City via Fifth Ave. 52 | Driver 1 has left the city! 53 | ----- 54 | Driver 2 heading from University to Bookstore via Chirp St. 55 | Driver 2 heading from Bookstore to University via Chirp St. 56 | Driver 2 heading from University to Bookstore via Chirp St. 57 | Driver 2 heading from Bookstore to Outside City via Fourth Ave. 58 | Driver 2 has left the city! 59 | ----- 60 | Driver 3 heading from University to Coffee Shop via Fifth Ave. 61 | Driver 3 heading from Coffee Shop to Mall via Meow St. 62 | Driver 3 heading from Mall to Bookstore via Fourth Ave. 63 | Driver 3 heading from Bookstore to Outside City via Fourth Ave. 64 | Driver 3 has left the city! 65 | ----- 66 | Driver 4 heading from Bookstore to Outside City via Fourth Ave. 67 | Driver 4 has left the city! 68 | ----- 69 | 70 | 71 | (725) $ java CitySim9000 111 72 | Driver 0 heading from University to Coffee Shop via Fifth Ave. 73 | Driver 0 heading from Coffee Shop to Outside City via Fifth Ave. 74 | Driver 0 has left the city! 75 | ----- 76 | Driver 1 heading from Mall to Coffee Shop via Meow St. 77 | Driver 1 heading from Coffee Shop to Mall via Meow St. 78 | Driver 1 heading from Mall to Bookstore via Fourth Ave. 79 | Driver 1 heading from Bookstore to Outside City via Fourth Ave. 80 | Driver 1 has left the city! 81 | ----- 82 | Driver 2 heading from Mall to Coffee Shop via Meow St. 83 | Driver 2 heading from Coffee Shop to Mall via Meow St. 84 | Driver 2 heading from Mall to Bookstore via Fourth Ave. 85 | Driver 2 heading from Bookstore to University via Chirp St. 86 | Driver 2 heading from University to Bookstore via Chirp St. 87 | Driver 2 heading from Bookstore to University via Chirp St. 88 | Driver 2 heading from University to Bookstore via Chirp St. 89 | Driver 2 heading from Bookstore to University via Chirp St. 90 | Driver 2 heading from University to Bookstore via Chirp St. 91 | Driver 2 heading from Bookstore to University via Chirp St. 92 | Driver 2 heading from University to Coffee Shop via Fifth Ave. 93 | Driver 2 heading from Coffee Shop to Mall via Meow St. 94 | Driver 2 heading from Mall to Coffee Shop via Meow St. 95 | Driver 2 heading from Coffee Shop to Mall via Meow St. 96 | Driver 2 heading from Mall to Coffee Shop via Meow St. 97 | Driver 2 heading from Coffee Shop to Mall via Meow St. 98 | Driver 2 heading from Mall to Coffee Shop via Meow St. 99 | Driver 2 heading from Coffee Shop to Mall via Meow St. 100 | Driver 2 heading from Mall to Coffee Shop via Meow St. 101 | Driver 2 heading from Coffee Shop to Outside City via Fifth Ave. 102 | Driver 2 has left the city! 103 | ----- 104 | Driver 3 heading from Bookstore to University via Chirp St. 105 | Driver 3 heading from University to Coffee Shop via Fifth Ave. 106 | Driver 3 heading from Coffee Shop to Outside City via Fifth Ave. 107 | Driver 3 has left the city! 108 | ----- 109 | Driver 4 heading from University to Coffee Shop via Fifth Ave. 110 | Driver 4 heading from Coffee Shop to Outside City via Fifth Ave. 111 | Driver 4 has left the city! 112 | ----- 113 | -------------------------------------------------------------------------------- /old_deliverables/deliverable3/deliverable3.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Quality Assurance 2 | Fall Semester 2015 3 | 4 | DUE 27 OCT 2015 5 | 6 | ## Deliverable 3 7 | 8 | For this assignment, your group will write systems tests (aka acceptance or integration tests) for a web site of your choice using the BDD model discussed in class. That is, you will write user stories (features) and scenarios, and then use JUnit and Selenium tests. You should substantially cover a subset of functionality for the project, and note in the "Testing Concerns" section what other aspects you would additionally add for full testing if this were a professional product. 9 | 10 | Tests and code should be on GitHub or another publicly accessible repository by the beginning of class on the due date. 11 | 12 | You are allowed to choose your partner for this deliverable - the only stipulation is that it CANNOT be the same person you partnered with for the first deliverable. There should be TWO and ONLY TWO students per group. If you cannot find a partner, please let me know and I will help you find one. 13 | 14 | ## Format 15 | Every group should have a title page with: 16 | * The name of the project under test 17 | * The names of the people in the group 18 | * The title "CS 1632 - DELIVERABLE 3: Web Testing with BDD" 19 | 20 | First, include a short summary (a few paragraphs) of what you decided to test, and why. I would recommend trying a few test runs with the Selenium IDE on a web page before deciding to test it "for real". Sites with lots of JavaScript or very dynamic page layouts are often poor choices for this deliverable. In particular, Amazon and Facebook have been very difficult for previous projects. You will probably also want to choose a website which you 1) understand well and 2) does not cost money to use. 21 | 22 | Secondly, add a description of issues you faced when writing these tests and problems you would expect going forward based on your experiences. If any tests you wrote fail, they should be included here. Also note if there are any special steps to get the tests running. 23 | 24 | At the end of this section, note where your code (test code and code of project under test, if available) is located. 25 | 26 | After this, there should be a printout or screen shot of the test execution results. 27 | 28 | There is no need to print out the code. It should be put on a publicly-available site such as GitHub BY THE BEGINNING OF CLASS. 29 | 30 | There shall be AT LEAST 4 user stories per group, but there can be more. Each user story should have multiple (at least two) scenarios. There shall be at least an average ("arithmetic mean", if you would like to be specific) of 5 scenarios per user story. 31 | 32 | User stories should all follow the Connextra template. 33 | 34 | Scenarios should all follow the Given/When/Then template (but note that some scenarios may not require all three elements). 35 | 36 | The feature files shall contain a feature and all scenarios for that feature. The JUnit tests shall have the scenario that they are testing written in the comments above the particular test. All tests shall correspond to a scenario and vice-versa. 37 | 38 | Remember that scenarios are FEATURE-level tests; they should discuss things in a way that an intelligent but non-technical user of the software would understand. Remember the differences between scenario tests and specification/unit tests! 39 | 40 | Also remember to focus on a particular subset of functionality in order to get good testing coverage. 41 | 42 | ## Grading 43 | * Summary - 5% 44 | * Testing concerns - 10% 45 | * Screen shot or printout of test results - 5% 46 | * User Stories and Scenarios, written in the proper format - 30% 47 | * JUnit Tests - 50% 48 | 49 | Reminder that all code (project under test AND test code) should be publicly available, or at least available to me. 50 | 51 | Please feel free to email me at bill@billlaboon.com or come to office hours to discuss any problems you have. 52 | 53 | -------------------------------------------------------------------------------- /old_deliverables/deliverable4/deliverable4.md: -------------------------------------------------------------------------------- 1 | # CS1632 Deliverable 4 2 | Fall Semester 2015 3 | 4 | DUE 10 NOV 2015 5 | 6 | ## Deliverable 4 7 | 8 | For this assignment, you will either: 9 | 1. Develop a combinatorial test and explain it 10 | 2. Write your own JUnit-based property-based test to check Arrays.sort(int[] arr) method in Java 11 | 12 | There are no partners for this deliverable. 13 | 14 | For the combinatorial testing, you will need to develop a combinatorial test with at least five parameters (variables). You may generate the combinatorial table using the NIST ACTS tool or another combinatorial generator. To download the NIST ACTS tool, you may follow the instructions here: http://csrc.nist.gov/groups/SNS/acts/documents/comparison-report.html. I recommend you do this as soon as possible to ensure everything works on your system and it can be downloaded. Since this requires at least one manual step on the part of NIST researchers, do it as soon as possible! 15 | 16 | You will come up with your own system (software) to test. This can be already existing software, or software that you make up. Build the system using NIST ACTS (or a similar tool), which will generate a covering array. You should make tests which covers all pairs (2-way interactions). Based on this covering array, generate a test plan (in the same format as Deliverable 1) which indicates how to test all of pairs (as discussed in class). 17 | 18 | You may not use "fonts and font effects" as your software. You should come up with your own idea. 19 | 20 | For the JUnit-based property-based tests, generate a minimum of 100 different random arrays of different sizes, and test different properties (many examples were discussed in the lecture on property-based testing) of sorting them. You may use Java's built-in Arrays.sort() method. You should test at least three different properties of each sorted array. You should use traditional JUnit test procedures (e.g., use assertions, don't use System.out.println during normal execution, etc.) Since we are testing a built-in Java method, I don't expect any failures, but who knows, you may be the one to find a bug in Java's own libraries! 21 | 22 | You may do this either all in one JUnit test method, or with one method per property. 23 | 24 | For either of these projects, I expect an approximately one-page (3 or 4 paragraphs) description of why you chose this project, how you went about doing it, any issues you faced, and what you learned. If you are doing the combinatorial testing option, use this space to explain what you are testing. If it is existing software, please provide a link; if it is software that you have thought of yourself, please explain it in enough detail that I can understand what you are testing. 25 | 26 | Both of these options also will require a screenshot. Please put this on its own page. For the combinatorial testing project, I expect a screenshot of NIST ACTS or whatever other tool you are using, specifically showing the all-pairs covering array. For the property-based testing project, I expect a screenshot of the executed JUnit test(s). 27 | 28 | ## Format 29 | Every assignment should have a title page with: 30 | * The name of the student 31 | * The title "CS 1632 - DELIVERABLE 4" 32 | * The name of the project you worked on - "COMBINATORIAL TESTING" or "PROPERTY-BASED TESTING" 33 | 34 | ## Grading (Combinatorial Testing) 35 | * Summary - 10% 36 | * Screenshot of NIST ACTS (or similar) tool with your system - 20% 37 | * Test Plan - 70% 38 | 39 | ## Grading (Property-Based Testing) 40 | * Summary - 10% 41 | * Screenshot of completed test - 20% 42 | * Test Code - 70% 43 | 44 | Please feel free to email me at bill@billlaboon.com or come to office hours to discuss any problems you have. 45 | 46 | -------------------------------------------------------------------------------- /old_deliverables/deliverable5/deliverable5.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Quality Assurance 2 | Fall Semester 2015 3 | 4 | DUE 19 NOV 2015 5 | 6 | ## Deliverable 5 7 | 8 | For this assignment, you (NOT a group) will profile a Conway's Game of Life simulation, and improve its performance by refactoring a single method (to be determined by the results of the profiling). This will consist of three parts: 9 | 10 | 1. Profiling (before) to determine which method is the most CPU-intensive 11 | 2. Refactoring the method to be more performant 12 | 3. Profiling (after) showing that your rewrite helped make your method more performant 13 | 14 | Code will be on Github (https://github.com/laboon/JavaLife). Please _fork_ it and create your own repository (do not issue a pull request against the original). 15 | 16 | There are no partners for this deliverable. 17 | 18 | ## Format 19 | Every assignment should have a title page with: 20 | * The name of the project under test (JavaLife) 21 | * The name of the student 22 | * The title "CS 1632 - DELIVERABLE 5: Performance Testing Conway's Game of Life" 23 | 24 | There is no need to print out the code. It should be available on GitHub or a similar code-sharing site BY THE BEGINNING OF CLASS. 25 | 26 | In order to determine the "hot spots" of the application, you will need to run a profiler such as VisualVM (download at https://visualvm.java.net/). Using a profiler, determine a method you can use to measurably increase the speed of the application without modifying behavior. You may wish to use "time" or a similar command to ensure that you have in fact reduced the amount of time necessary to execute n iterations of the World. 27 | 28 | The application accepts four command line arguments. All of these should be positive integers. They are: 29 | 30 | 1. The size of the world (n x n) 31 | 2. The seed for the random number generator 32 | 3. The percentage of cells alive initially 33 | 4. The number of iterations to run the simulation 34 | 35 | Note that iterations can go by pretty quickly, so for your debugging, you may want to set the number of iterations very high (e.g. 30000) and just kill the application when you are done profiling. You also may want to try running relatively large arrays (15x15 or greater). Remember that performance testing often means running the same test multiple times, or under different conditions, to determine actual performance under real-life load. Try different values if you are not seeing a relatively obvious outlier. 36 | 37 | For the summary, describe how you profiled the application and determined the method to refactor, and why you changed what you did. The summary should not be more than a few paragraphs - definitely less than a page. 38 | 39 | After this, include screenshots of VisualVM (or another profiler, if you use that) both before and after the refactor. These screenshots should include the relevant sections that let you see what method to refactor. 40 | 41 | ## Grading 42 | * Summary - 25% 43 | * VisualVM (or other profile) screenshots (before and after) - 50% 44 | * Method refactoring - 25% 45 | 46 | Please feel free to email me at bill@billlaboon.com or come to office hours to discuss any problems you have. 47 | 48 | -------------------------------------------------------------------------------- /old_deliverables/deliverable6/deliverable6.md: -------------------------------------------------------------------------------- 1 | # CS 1632 - Software Quality Assurance 2 | Fall Semester 2015 3 | ASSIGNED 19 NOV 2015 4 | DUE 8 DEC 2015 5 | 6 | ## Deliverable 6 - THE FINAL DELIVERABLE 7 | 8 | For this assignment, you will perform some software testing of whatever kind you find interesting. It can be from the following list of topics gone over in class, or you can research and use your own. If you are using your own, please check with me before working on it. You may work alone or in a group of two (with a partner of your choice). 9 | 10 | ### Possible Ideas (not an exhaustive list!) 11 | 12 | 1. Deeper testing of a web application (perhaps including JavaScript) 13 | 2. Detailed analysis and performance testing of an application 14 | 3. Rewriting an application to be more test-friendly (e.g. Coffee Maker Quest) 15 | 3. Security Testing 16 | 3. Penetration Testing (be sure it's on something you HAVE PERMISSION TO DO!) 17 | 6. More In-Depth Property-Based Testing 18 | 7. Formal Analysis / Verification of a program 19 | 8. Algorithmic analysis of a combinatorial algorithm 20 | 9. Writing a new app from scratch using TDD or BDD 21 | 10. Adding tests to an existing program 22 | 11. In-depth metrics analysis (code coverage, etc.) of a project 23 | 24 | The final project should be approximately as much work as one of the 'larger' assignments for this class (e.g. Deliverable 1 or 2). Please see the grading rubric below for how the final deliverable will be graded. 25 | 26 | You may use a project you have already worked on, or a totally new one if you prefer. Any tests and/or code should be on GitHub or another publicly accessible repository. 27 | 28 | The quality of your write-up can and will impact your grade. For a perfect grade, I expect 29 | professional quality on ALL aspects of your testing, including: 30 | 31 | 1. Proper grammar, punctuation, etc. 32 | 2. No commented-out code, UNLESS there is a specified reason for it, e.g.: 33 | * `// This doesn't work, but this is what I would do if I could...` 34 | * `// Uncomment the following line to run additional long-running tests)` 35 | 3. No inconsistent spacing/tabs/brace settings. I don't care which you use, but make 36 | it consistent. 37 | 4. Good variable names, good coding style in general. 38 | 39 | * **Perfect Score** - Absolutely no problems, working on a challenging system with excellent 40 | test coverage. Tests and kinds of tests make perfect sense and are described well. 41 | Summary, testing concerns, and quality assessment are easy to read with no grammatical 42 | errors. 43 | * **A** - Minor problems, working on a reasonable system with good test coverage. Tests and 44 | kinds of tests are appropriate and described well. Summary, testing concerns, and 45 | quality assessment have only minor issues. 46 | * **B** - A few issues, but overall decent test coverage. Working on a simpler system. 47 | Tests and kinds of tests could have been better. Summary, testing concerns, and 48 | quality assessment have a few issues. 49 | * **C** - Test coverage is not good; many tests are repetitive or unnecessary. Working on a 50 | trivial system (e.g. a Linked List). Tests and kinds of tests are only vaguely appropriate. 51 | Summary, testing concerns, and quality assessment have major issues. 52 | * **D** - Test coverage is horrible; tests do not do what they say, there aren't enough of them, 53 | and are not described at all. Summary, testing concerns, and quality assessment are 54 | filled with errors, grammatical and factual. 55 | * **F** - Test coverage is basically nonexistent; tests are testing the wrong thing entirely 56 | (e.g. a "usability test" which only checks a math function repeatedly) or missing. 57 | Summary, testing concerns, and quality assessment are unreadable or missing 58 | entirely. 59 | 60 | ## Format 61 | Every group should have a title page with: 62 | * The name of the project under test 63 | * The names of the people in the group 64 | * The title "CS 1699 - FINAL DELIVERABLE" 65 | 66 | First, include a short summary (a few paragraphs) of what you decided to test, what kinds of 67 | testing you are doing, and why. 68 | 69 | Then, add a description of issues you faced when writing these tests and problems you would 70 | expect going forward based on your experiences. Include here any additional tests or kinds of 71 | tests you would like to run to get a better assessment of the quality of the system. This should 72 | also be a few paragraphs. 73 | 74 | I also expect an assessment of quality of the project under test. This will include: 75 | 76 | 1. A list of any failed tests or problem areas 77 | 2. An overall assessment of quality 78 | 3. Your recommendation on whether or not the product is ready to be released 79 | 80 | You should write this in the style of convincing a manager whether or not a system is ready to be released. If what you are doing does not lend itself to this (e.g., algorithmic analysis), then write how your analysis would be useful to a commercial system, as if you are writing to a manager. 81 | 82 | At the end of this section, note where your code (test code and code of project under test) is located, if applicable. 83 | 84 | After this, there should be printouts or screen shots of the test execution results, if applicable. There is no need to print out the code, if any. It should be put on a publicly-available site such as GitHub BY THE BEGINNING OF CLASS. 85 | 86 | ## Grading 87 | * Summary - 10% 88 | * Testing concerns / Concerns going forward - 10% 89 | * Assessment of Quality - 10% 90 | * Project - 70% 91 | 92 | Reminder that all code (project under test AND test code) should be publicly available, or at least available to me. 93 | 94 | Please feel free to email me at laboon@cs.pitt.edu or come to office hours to discuss any problems you have. 95 | 96 | I urge you to talk to me as soon as possible if you experience any problems. 97 | -------------------------------------------------------------------------------- /old_study_guides/exam-2-study-guide.md: -------------------------------------------------------------------------------- 1 | # CS 1632 Exam 2 Study Guide 2 | Spring 2015 3 | 4 | Note that the second exam is _not_ cumulative, except in the sense that there 5 | 6 | ## EXPLORATORY TESTING 7 | * Be able to define 8 | * When is it useful? When is it not? 9 | * Benefits/drawbacks 10 | 11 | ## SMOKE TESTS 12 | * Be able to define 13 | * Given a system, be able to determine a smoke test for it 14 | * Scripted vs unscripted 15 | * Sanity testing 16 | 17 | ## COMBINATORIAL TESTING 18 | * Understand what a covering array is 19 | * Be able to make a simple covering array 20 | 21 | ## PROPERTY-BASED TESTING 22 | * Be able to write simple property-based tests 23 | * What is property-based testing good for? What is it bad for? 24 | 25 | ## FORMAL VERIFICATION 26 | * Understand what it is 27 | * Drawbacks: why is not used often? 28 | * How can we formally verify that something will halt, if our languages are Turing-complete? 29 | 30 | ## STOCHASTIC ("FUZZ") TESTING 31 | * Be able to define, list benefits/drawbacks 32 | * Think of good inputs to push (e.g., JSON, executable code, JavaScript, SQL, etc.) 33 | * Understand dumb, smart, evil, chaos monkey testing 34 | * Other ways to test similar to chaos monkey: reduce bandwidth, modified permissions, etc. 35 | 36 | ## PERFORMANCE TESTING 37 | * Understand concepts on how to test performance 38 | * Be able to write test plans for different performance indicators and systems 39 | * Terminology: Service-Oriented vs Efficiency-Oriented Indicators 40 | * Availability, Response Time, Throughput, Utilization 41 | * Performance targets, performance thresholds, KPIs - understand and be able to generate! 42 | * Measuring response time - methodologies 43 | * Understand different concepts of time: user, system, total, real 44 | * Measuring availability, concurrency, scalability, throughput 45 | * Understand n 9's (e.g., 5 9s vs 6 9s) 46 | * Load testing - baseline, soak/stability, stress tests 47 | * Know different performance monitoring tools - perfmon, Activity Monitor/Instruments, top/iostat 48 | * Key resources to watch, and why 49 | * When to use a packet analyzer or profiler 50 | * Be able to answer questions about profiling 51 | 52 | ## SECURITY TESTING 53 | * The CIA/InfoSec Triad 54 | * Terminology: passive vs active, public-key cryptography, vulnerability, exploit 55 | * Terminology: interruption, interception, modification, fabrication 56 | * Be able to define different kinds of malware (virus, worm, etc.) 57 | * Be able to describe/test for common attacks: injection, broken authentication, XSS, insecure storage, buffer overruns, smashing the stack, security misconfiguration, insecure object references, social engineering 58 | * Understand the steps of penetration testing, and why it is useful 59 | * How does security testing differ from other kinds of testing? 60 | 61 | ## INTERACTING WITH STAKEHOLDERS 62 | * Be able to name some stakeholders and what is important to them (upper management, project management, testers, other developers) 63 | * Be prepared for some "fake" interaction with various stakeholders 64 | * Be able to put together a red-yellow-green template report 65 | * Know the one most important thing when dealing with stakeholders or other co-workers -------------------------------------------------------------------------------- /old_study_guides/midterm-study-guide.md: -------------------------------------------------------------------------------- 1 | # CS 1632 Midterm Exam Study Guide 2 | Fall 2015 3 | 4 | The midterm will cover everything we have covered up to the lecture of 15 OCT 2015. 5 | 6 | However, here are the key topics to study in preparation for the test. 7 | 8 | ## TESTING THEORY AND TERMINOLOGY 9 | * Equivalence class partitioning 10 | * Boundary and "interior" values 11 | * Base, Edge, and Corner cases 12 | * Static vs Dynamic testing 13 | * Know the differences and examples of each 14 | * Black/White/Grey box testing 15 | * Know the differences and examples of each 16 | 17 | ## REQUIREMENTS ANALYSIS 18 | * What makes a good or bad requirement? 19 | * Testability - requirements must be: 20 | * Complete, consistent, unambiguous, quantitative, feasible 21 | * Functional Requirements vs Non-Functional (Quality Attributes) 22 | * Be able to define and write your own 23 | * Traceability Matrices 24 | * Be able to define and write your own 25 | 26 | ## TEST PLANS 27 | * Terminology: test cases, test plans, test suites, test runs 28 | * The Seven Testing Principles 29 | * You don't need to remember their names, but use them in developing test plans 30 | * Verification vs validation 31 | 32 | ## DEFECT REPORTING 33 | * Be prepared to report a defect based on a test case 34 | * Remember the defect template: 35 | * SUMMARY, DESCRIPTION, REPRODUCTION STEPS, EXPECTED BEHAVIOR, OBSERVED BEHAVIOR 36 | * Optionally: SEVERITY/IMPACT, NOTES 37 | * Levels of severity: BLOCKER, CRITICAL, MAJOR, NORMAL, MINOR, TRIVIAL 38 | * Enhancements vs defects 39 | * Be prepared to argue if something is a defect or enhancement 40 | * Coding mistakes vs defects 41 | 42 | ## AUTOMATED TESTING 43 | * Pros and cons of automated testing 44 | * Unit tests vs system/acceptance/integration tests 45 | * Writing automated tests: 46 | * PRECONDITIONS, POSTCONDITIONS, EXECUTION STEPS, INPUT/OUTPUT VALUES 47 | 48 | ## UNIT TESTING 49 | * Be prepared to write some unit tests in JUnit 50 | * Pay special attention to assertions 51 | * Stubs, test doubles, mocks, and verification 52 | * Be able to explain code coverage and what it's good for/not good for 53 | 54 | ## WRITING TESTABLE CODE 55 | * Basic strategies for testable code 56 | * The DRY Principle 57 | * The SOLID Principles 58 | * The Law of Demeter 59 | 60 | ## TDD 61 | * The red-green-refactor loop 62 | * Principles of TDD: 63 | * YAGNI, KISS, "Fake it 'til you make it", Avoid interdependency, avoid slow tests 64 | * Benefits and drawbacks of TDD 65 | * Testing private methods (you won't need to use reflection, but remember why/why not one might not test them) 66 | 67 | ## BDD 68 | * Be able to define it 69 | * Understand and be able to use the Connextra template and Gherkin syntax 70 | * Be prepared to write out Gherkin feature files in the appropriate format 71 | * Be able to create user stories/scenarios 72 | * How to use automated tests 73 | 74 | ## WEB TESTING 75 | * Be able to explain how you would test a web page with Selenium 76 | * You do NOT need to know Selenese (Selenium scripting language) 77 | * Be able to discuss issues with testing web apps (page loading timing issues, javascript issues, etc) 78 | -------------------------------------------------------------------------------- /sample_code/FizzBuzz.rb: -------------------------------------------------------------------------------- 1 | class String 2 | def is_i? 3 | !!(self =~ /^[-+]?[0-9]+$/) 4 | end 5 | end 6 | 7 | def fizzbuzz(max_val, fail = false) 8 | 9 | (1..max_val).each do |n| 10 | if ((n % 3) == 0) 11 | puts "fizz" 12 | elsif ((n % 5) == 0) 13 | puts "buzz" 14 | elsif ((n % 5 == 0) && (n % 3 == 0)) 15 | puts "fizzbuzz" 16 | else 17 | puts n.to_s 18 | end 19 | end 20 | 21 | puts "FAIL FAIL FAIL!" if fail 22 | 23 | end 24 | 25 | # EXECUTION STARTS HERE 26 | 27 | print "Enter n: " 28 | n = gets.chomp 29 | if n.is_i? && n.to_i > 100 && n.to_i < 500 then 30 | fizzbuzz(n.to_i, true) 31 | elsif n.is_i? && n.to_i < 1001 then 32 | fizzbuzz(n.to_i, false) 33 | elsif n.is_i? then 34 | raise "ERROR: NUMBER TOO LARGE!!!!" 35 | else 36 | raise "ERROR: NOT A POSITIVE INTEGER!!!!" 37 | end 38 | -------------------------------------------------------------------------------- /sample_code/LinkedList.java: -------------------------------------------------------------------------------- 1 | package com.laboon; 2 | 3 | import java.util.ArrayList; 4 | 5 | public class LinkedList { 6 | 7 | Node _head = null; 8 | 9 | public LinkedList() { 10 | 11 | } 12 | 13 | public void traverse() { 14 | Node ptr = _head; 15 | while (ptr != null) { 16 | ptr = ptr.getNext(); 17 | } 18 | } 19 | 20 | public void prettyPrint() { 21 | Node ptr = _head; 22 | if (_head == null) { 23 | System.out.println(""); 24 | } 25 | while (ptr != null) { 26 | if (ptr.getNext() != null) { 27 | System.out.print(ptr.getData() + " -> "); 28 | } else { 29 | System.out.println(ptr.getData()); 30 | } 31 | ptr = ptr.getNext(); 32 | } 33 | } 34 | 35 | public void addToFront(Node toAdd) { 36 | if (_head == null) { 37 | _head = toAdd; 38 | } else { 39 | Node oldHead = _head; 40 | _head = toAdd; 41 | _head.setNext(oldHead); 42 | } 43 | } 44 | 45 | public void deleteLast() { 46 | if (_head == null) { 47 | // Nothing to do! 48 | return; 49 | } 50 | 51 | if (_head.getNext() == null) { 52 | _head = null; 53 | return; 54 | } 55 | 56 | Node ptr = _head; 57 | Node last = ptr; 58 | 59 | while (ptr != null) { 60 | 61 | ptr = ptr.getNext(); 62 | if (ptr != null && ptr.getNext() != null) { 63 | 64 | last = ptr; 65 | } 66 | } 67 | 68 | last.setNext(null); 69 | } 70 | 71 | public void deleteFront() { 72 | if (_head == null) { 73 | // Nothing to do! 74 | return; 75 | } 76 | 77 | if (_head.getNext() == null) { 78 | _head = null; 79 | return; 80 | } 81 | 82 | _head = _head.getNext(); 83 | 84 | } 85 | 86 | public T getFrontData() { 87 | if (_head == null) { 88 | return null; 89 | } else { 90 | return _head.getData(); 91 | } 92 | } 93 | 94 | public Node getFront() { 95 | return _head; 96 | } 97 | 98 | public void addToEnd(Node toAdd) { 99 | 100 | toAdd.setNext(null); 101 | 102 | if (_head == null) { 103 | _head = toAdd; 104 | } else { 105 | 106 | Node ptr = _head; 107 | Node last = ptr; 108 | while (ptr != null) { 109 | 110 | last = ptr; 111 | ptr = ptr.getNext(); 112 | } 113 | 114 | last.setNext(toAdd); 115 | 116 | } 117 | 118 | } 119 | 120 | /** 121 | * Delete all duplicate data 122 | */ 123 | 124 | public void deleteDupes() { 125 | 126 | // If there are zero or one elements, there can't be dupes 127 | 128 | if (_head == null || _head.getNext() == null) { 129 | return; 130 | } 131 | 132 | ArrayList prevObjs = new ArrayList(); 133 | Node ptr = _head; 134 | Node last = ptr; 135 | 136 | // Traverse list 137 | 138 | while (ptr != null) { 139 | if (prevObjs.contains(ptr.getData())) { 140 | // delete node - note this will never occur on first pass-through 141 | last.setNext(ptr.getNext()); 142 | } else { 143 | // add it to the list of previous data elements 144 | last = ptr; 145 | prevObjs.add(ptr.getData()); 146 | } 147 | 148 | ptr = ptr.getNext(); 149 | } 150 | 151 | } 152 | 153 | /** 154 | * Will delete first element with that value 155 | * @param valToCheck 156 | */ 157 | 158 | public void deleteByVal(T valToCheck) { 159 | 160 | if (_head == null) { 161 | // nothing here, return 162 | return; 163 | } 164 | 165 | if (_head.getNext() == null) { 166 | // Only one item 167 | if (_head.getData() == valToCheck) { 168 | _head = null; 169 | } 170 | return; 171 | } 172 | 173 | if (_head.getData() == valToCheck) { 174 | _head = _head.getNext(); 175 | } 176 | 177 | Node ptr = _head; 178 | Node last = ptr; 179 | boolean foundIt = false; 180 | while (ptr != null && !(foundIt)) { 181 | if (ptr.getData() == valToCheck) { 182 | // Found it! 183 | foundIt = true; 184 | last.setNext(ptr.getNext()); 185 | } else { 186 | last = ptr; 187 | ptr = ptr.getNext(); 188 | } 189 | } 190 | } 191 | 192 | public void clear() { 193 | _head = null; 194 | } 195 | 196 | public T kthToLast(int k) { 197 | 198 | if (_head == null) { 199 | return null; 200 | } 201 | Node ptr = _head; 202 | Node candidate = null; 203 | int ctr = 0; 204 | while (ptr != null) { 205 | 206 | if (ctr == k) { 207 | candidate = _head; 208 | } else if (ctr > k) { 209 | candidate = candidate.getNext(); 210 | } 211 | ctr++; 212 | ptr = ptr.getNext(); 213 | } 214 | if (candidate == null) { 215 | return null; 216 | } else { 217 | return candidate.getData(); 218 | } 219 | } 220 | 221 | @Override 222 | public boolean equals(Object rhs) { 223 | 224 | // Trivial cases - if exact same object, return true, 225 | // if not of correct class or if null, return false 226 | 227 | if (this == rhs) return true; 228 | else if (!(rhs instanceof LinkedList) || rhs == null) return false; 229 | LinkedList other = (LinkedList) rhs; 230 | boolean toReturn = true; 231 | Node ptr1 = this.getFront(); 232 | Node ptr2 = other.getFront(); 233 | boolean cont = true; 234 | while (ptr1 != null && ptr2 != null && cont == true) { 235 | // System.out.println("Ptr1 = " + ptr1.getData() + " / ptr2 = " + ptr2.getData()); 236 | if (ptr1.getData().equals(ptr2.getData())) { 237 | ptr1 = ptr1.getNext(); 238 | ptr2 = ptr2.getNext(); 239 | } else { 240 | // System.out.println("Data not equal, setting to false"); 241 | toReturn = false; 242 | cont = false; 243 | } 244 | } 245 | if (ptr1 == null ^ ptr2 == null) { 246 | // System.out.println("XOR failure - lists are diff sizes"); 247 | toReturn = false; 248 | } 249 | 250 | return toReturn; 251 | } 252 | 253 | @Override 254 | public int hashCode() { 255 | return 1; 256 | } 257 | 258 | /** 259 | * @param args 260 | */ 261 | public static void main(String[] args) { 262 | /* 263 | LinkedList ll = new LinkedList(); 264 | Node node0 = new Node(0); 265 | Node node1 = new Node(1); 266 | Node node2 = new Node(2); 267 | Node node3 = new Node(3); 268 | Node node4 = new Node(4); 269 | Node node5 = new Node(5); 270 | ll.addToEnd(node1); 271 | ll.addToEnd(node2); 272 | ll.addToEnd(node3); 273 | ll.addToEnd(node4); 274 | ll.addToEnd(node5); 275 | ll.addToFront(node0); 276 | 277 | System.out.println("Original.."); 278 | ll.traverse(); 279 | 280 | System.out.println("Pretty..."); 281 | ll.prettyPrint(); 282 | 283 | System.out.println("Delete front.."); 284 | ll.deleteFront(); 285 | ll.traverse(); 286 | 287 | System.out.println("Delete end..."); 288 | ll.deleteLast(); 289 | ll.traverse(); 290 | 291 | System.out.println("Delete non-existent 1000.."); 292 | ll.deleteByVal(1000); 293 | ll.traverse(); 294 | 295 | System.out.println("Delete 2.."); 296 | ll.deleteByVal(2); 297 | ll.traverse(); 298 | 299 | System.out.println("Delete 1.."); 300 | ll.deleteByVal(1); 301 | ll.traverse(); 302 | 303 | System.out.println("Delete 4.."); 304 | ll.deleteByVal(4); 305 | ll.traverse(); 306 | 307 | System.out.println("Delete 3.."); 308 | ll.deleteByVal(3); 309 | ll.traverse(); 310 | 311 | System.out.println("Delete 1 on null.."); 312 | ll.deleteByVal(1); 313 | ll.traverse(); 314 | 315 | 316 | ll.clear(); 317 | System.out.println("Delete dupes NULL: orig"); 318 | ll.prettyPrint(); 319 | ll.deleteDupes(); 320 | System.out.println("Delete dupes NULL: after"); 321 | ll.prettyPrint(); 322 | ll.addToEnd(new Node(new Integer(1))); 323 | ll.addToEnd(new Node(new Integer(1))); 324 | ll.addToEnd(new Node(new Integer(2))); 325 | ll.addToEnd(new Node(new Integer(3))); 326 | ll.addToEnd(new Node(new Integer(3))); 327 | ll.addToEnd(new Node(new Integer(2))); 328 | ll.addToEnd(new Node(new Integer(1))); 329 | ll.addToEnd(new Node(new Integer(3))); 330 | ll.addToEnd(new Node(new Integer(2))); 331 | ll.addToEnd(new Node(new Integer(1))); 332 | System.out.println("Delete dupes: orig"); 333 | ll.prettyPrint(); 334 | System.out.println("Delete dupes: after"); 335 | ll.deleteDupes(); 336 | ll.prettyPrint(); 337 | 338 | System.out.println("Long ordered list"); 339 | ll.clear(); 340 | ll.addToEnd(new Node(new Integer(1))); 341 | ll.addToEnd(new Node(new Integer(2))); 342 | ll.addToEnd(new Node(new Integer(3))); 343 | ll.addToEnd(new Node(new Integer(4))); 344 | ll.addToEnd(new Node(new Integer(5))); 345 | ll.addToEnd(new Node(new Integer(6))); 346 | ll.addToEnd(new Node(new Integer(7))); 347 | ll.addToEnd(new Node(new Integer(8))); 348 | ll.addToEnd(new Node(new Integer(9))); 349 | ll.prettyPrint(); 350 | for (int j=0; j<10; j++) { 351 | Integer n = ll.kthToLast(j); 352 | if (n == null) { 353 | System.out.println("Returned null"); 354 | } else { 355 | System.out.println(j + "[st/nd/th] to last = " + n.intValue()); 356 | } 357 | } 358 | */ 359 | } 360 | 361 | } 362 | -------------------------------------------------------------------------------- /sample_code/LinkedListTest.java: -------------------------------------------------------------------------------- 1 | package com.laboon; 2 | 3 | import static org.junit.Assert.*; 4 | 5 | import org.junit.After; 6 | import org.junit.Before; 7 | import org.junit.Test; 8 | 9 | // You could also do this to make this a little cleaner. 10 | // import static org.mockito.Mockito.*; 11 | 12 | import org.mockito.*; 13 | 14 | public class LinkedListTest { 15 | 16 | @SuppressWarnings("unchecked") 17 | 18 | @Mock 19 | LinkedList mockedLinkedList = Mockito.mock(LinkedList.class); 20 | 21 | @Before 22 | public void setUp() throws Exception { 23 | // If you use @Mock, you need to do this 24 | MockitoAnnotations.initMocks(mockedLinkedList); 25 | 26 | } 27 | 28 | @After 29 | public void tearDown() throws Exception { 30 | // any necessary teardown - none needed here 31 | } 32 | 33 | 34 | // -------------------------------------------------------------- 35 | // ZERO-LENGTH TESTS 36 | // -------------------------------------------------------------- 37 | 38 | // Test that a zero-length list will return null if you try to get 39 | // the first element 40 | @Test 41 | public void testZeroList() { 42 | LinkedList ll = new LinkedList(); 43 | ll.clear(); 44 | assertNull(ll.getFront()); 45 | } 46 | 47 | // Test that the .clear() methods works, by first adding an item, and then 48 | // clearing the list. It should be empty (tested by ensuring that 49 | // the first element is null). 50 | @Test 51 | public void testClearedList() { 52 | LinkedList ll = new LinkedList(); 53 | ll.addToFront(new Node(new Integer(7))); 54 | ll.clear(); 55 | assertNull(ll.getFront()); 56 | } 57 | 58 | // This tests whether a multiple item linked lit will 59 | // clear down to zero items when clear method is called. 60 | 61 | @Test 62 | public void testMultiList() { 63 | LinkedList ll = new LinkedList(); 64 | for (int j=0; j < 10; j++) { 65 | ll.addToFront(new Node(new Integer(j))); 66 | } 67 | ll.clear(); 68 | assertNull(ll.getFront()); 69 | } 70 | 71 | // -------------------------------------------------------------- 72 | // ADD TO FRONT TESTS 73 | // -------------------------------------------------------------- 74 | 75 | // Add ten nodes, then add one more (testNode). Check that setNext() has been 76 | // called to the last of the first ten nodes, and that the added 77 | // node testNode is the same as the one that is at the front of the list. 78 | 79 | @SuppressWarnings("unchecked") 80 | @Test 81 | public void testAddToNoItemLL() { 82 | LinkedList ll = new LinkedList(); 83 | Node[] nodes = new Node[10]; 84 | 85 | for (int j = 0; j < 10; j++) { 86 | nodes[j] = Mockito.mock(Node.class); 87 | ll.addToFront(nodes[j]); 88 | } 89 | 90 | Node testNode = Mockito.mock(Node.class); 91 | ll.addToFront(testNode); 92 | Mockito.verify(testNode).setNext(Matchers.eq(nodes[9])); 93 | assertSame(ll.getFront(), testNode); 94 | 95 | } 96 | 97 | // Add only one node, then add one more (testNode). Check that setNext() has been 98 | // called to the original node and that the added 99 | // node testNode is the same as the one that is at the front of the list. 100 | 101 | @SuppressWarnings("unchecked") 102 | @Test 103 | public void testAddToOneItemLL() { 104 | LinkedList ll = new LinkedList(); 105 | Node existingNode = Mockito.mock(Node.class); 106 | Node testNode = Mockito.mock(Node.class); 107 | ll.addToFront(existingNode); 108 | ll.addToFront(testNode); 109 | Mockito.verify(testNode).setNext(Matchers.eq(existingNode)); 110 | assertSame(ll.getFront(), testNode); 111 | } 112 | 113 | // -------------------------------------------------------------- 114 | // DELETE FROM FRONT TESTS 115 | // -------------------------------------------------------------- 116 | 117 | // Check that attempting to delete a node from a Linked List with 118 | // no elements will not throw an error. 119 | 120 | @Test 121 | public void testDeleteFrontNoItem() { 122 | LinkedList ll = new LinkedList(); 123 | ll.deleteFront(); 124 | assertEquals(ll.getFront(), null); 125 | 126 | } 127 | 128 | // Check that deleting a node from a Linked List with 129 | // one elements will not throw an error, and will result in an 130 | // empty LinkedList (front is null). 131 | 132 | @SuppressWarnings("unchecked") 133 | @Test 134 | public void testDeleteFrontOneItem() { 135 | LinkedList ll = new LinkedList(); 136 | ll.addToFront(Mockito.mock(Node.class)); 137 | ll.deleteFront(); 138 | assertEquals(ll.getFront(), null); 139 | } 140 | 141 | // Check that deleting a node from a Linked List with 142 | // multiple elements will properly delete the first node 143 | // and leave the old second node as the new first node.. 144 | 145 | @SuppressWarnings("unchecked") 146 | @Test 147 | public void testDeleteFrontMultipleItems() { 148 | LinkedList ll = new LinkedList(); 149 | Node[] nodes = new Node[10]; 150 | 151 | for (int j = 0; j < 10; j++) { 152 | nodes[j] = new Node(new Integer(1)); 153 | ll.addToFront(nodes[j]); 154 | } 155 | 156 | ll.deleteFront(); 157 | 158 | assertSame(ll.getFront(), nodes[8]); 159 | } 160 | 161 | // -------------------------------------------------------------- 162 | // EQUALITY TESTS 163 | // -------------------------------------------------------------- 164 | 165 | // Check that a new linked list equals itself. 166 | @Test 167 | public void testEqualsSelf() { 168 | LinkedList ll = new LinkedList(); 169 | assertEquals(ll, ll); 170 | } 171 | 172 | // Check that two new linked lists with no elements equal each other. 173 | @Test 174 | public void testEquals0Elems() { 175 | LinkedList ll01 = new LinkedList(); 176 | LinkedList ll02 = new LinkedList(); 177 | assertEquals(ll01, ll02); 178 | } 179 | 180 | // An instantiated linked list should not equal null. 181 | @Test 182 | public void testNotEqualsNull() { 183 | LinkedList ll01 = new LinkedList(); 184 | assertFalse(ll01.equals(null)); 185 | } 186 | 187 | // Check that a LL object does equal a non-LinkedList, e.g. Object 188 | @Test 189 | public void testNotEqualsRegularObject() { 190 | LinkedList ll01 = new LinkedList(); 191 | Object obj = new Object(); 192 | assertFalse(ll01.equals(obj)); 193 | } 194 | 195 | @Test 196 | public void throwException() { 197 | try { 198 | doSomethingThatThrowsException(); 199 | fail(); 200 | } catch (Exceptio e) { 201 | } 202 | 203 | } 204 | 205 | 206 | // Check that two LLs with the same Node value with a single node are equal 207 | @Test 208 | public void testEqualsOneNodeSameVals() { 209 | LinkedList ll11 = new LinkedList(); 210 | LinkedList ll12 = new LinkedList(); 211 | ll11.addToFront(new Node(new Integer(1))); 212 | ll12.addToFront(new Node(new Integer(1))); 213 | assertEquals(ll11, ll12); 214 | } 215 | 216 | // Check that two LL with different Node values with a single node are NOT equal 217 | @Test 218 | public void testEqualsOneNodeDiffVals() { 219 | LinkedList ll11 = new LinkedList(); 220 | LinkedList ll2 = new LinkedList(); 221 | ll11.addToFront(new Node(new Integer(1))); 222 | ll2.addToFront(new Node(new Integer(2))); 223 | assertFalse(ll11.equals(ll2)); 224 | } 225 | 226 | // Check that two LLs with different sizes, but the same front node value, 227 | // are not considered equal. 228 | @Test 229 | public void testNotEqualsDiffSizes() { 230 | LinkedList ll11 = new LinkedList(); 231 | LinkedList ll_3elems = new LinkedList(); 232 | 233 | ll11.addToFront(new Node(new Integer(1))); 234 | ll_3elems.addToFront(new Node(new Integer(3))); 235 | ll_3elems.addToFront(new Node(new Integer(2))); 236 | ll_3elems.addToFront(new Node(new Integer(1))); 237 | 238 | assertFalse(ll_3elems.equals(ll11)); 239 | } 240 | 241 | // Check that a LL which is just a reference to another instance of itself 242 | // equals itself 243 | @Test 244 | public void testEqualsRef() { 245 | LinkedList ll11 = new LinkedList(); 246 | ll11.addToFront(new Node(new Integer(1))); 247 | LinkedList ll11_new = ll11; 248 | assertSame(ll11, ll11_new); 249 | } 250 | 251 | // Check that LLs with the same size, but different data in the nodes, 252 | // do not equal each other. 253 | @Test 254 | public void testNotEqualsDiffData() { 255 | LinkedList ll_3elems = new LinkedList(); 256 | LinkedList ll_321 = new LinkedList(); 257 | ll_3elems.addToFront(new Node(new Integer(3))); 258 | ll_3elems.addToFront(new Node(new Integer(2))); 259 | ll_3elems.addToFront(new Node(new Integer(1))); 260 | 261 | ll_321.addToFront(new Node(new Integer(1))); 262 | ll_321.addToFront(new Node(new Integer(2))); 263 | ll_321.addToFront(new Node(new Integer(3))); 264 | assertFalse(ll_321.equals(ll_3elems)); 265 | } 266 | 267 | // Check that two multiple-node LLs with the same data equal each other 268 | @Test 269 | public void testEqualsSameData() { 270 | LinkedList ll_321 = new LinkedList(); 271 | LinkedList ll_321_2 = new LinkedList(); 272 | 273 | ll_321.addToFront(new Node(new Integer(1))); 274 | ll_321.addToFront(new Node(new Integer(2))); 275 | ll_321.addToFront(new Node(new Integer(3))); 276 | 277 | ll_321_2.addToFront(new Node(new Integer(1))); 278 | ll_321_2.addToFront(new Node(new Integer(2))); 279 | ll_321_2.addToFront(new Node(new Integer(3))); 280 | 281 | assertTrue(ll_321.equals(ll_321_2)); 282 | 283 | } 284 | 285 | } 286 | -------------------------------------------------------------------------------- /sample_code/selenium/RedditTest.java: -------------------------------------------------------------------------------- 1 | import static org.junit.Assert.*; 2 | 3 | import org.junit.Before; 4 | import org.junit.Test; 5 | import org.openqa.selenium.*; 6 | import org.openqa.selenium.firefox.FirefoxDriver; 7 | import org.openqa.selenium.htmlunit.HtmlUnitDriver; 8 | 9 | /** 10 | * As a user, 11 | * I would like to see reddit links in all sorts of ways, 12 | * So that I can know what is happening in the world 13 | * @author wlaboon 14 | * 15 | */ 16 | 17 | public class RedditTest { 18 | 19 | static WebDriver driver = new HtmlUnitDriver(); 20 | 21 | // Start at the home page for reddit for each test 22 | @Before 23 | public void setUp() throws Exception { 24 | driver.get("https://www.reddit.com"); 25 | } 26 | 27 | // Given that I am on the main page 28 | // When I view the title 29 | // Then I see that it contains the word "reddit" 30 | @Test 31 | public void testShowsCorrectTitle() { 32 | 33 | // Simply check that the title contains the word "reddit" 34 | 35 | String title = driver.getTitle(); 36 | assertTrue(title.contains("reddit")); 37 | } 38 | 39 | // Given that I am on the main page 40 | // When I view the header 41 | // Then I see that it contains "new", "rising", and "top" links 42 | @Test 43 | public void testHasCorrectHeaderLinks() { 44 | 45 | // Check for new, rising, and top links - if any of 46 | // these is not found, fail the test 47 | 48 | try { 49 | driver.findElement(By.linkText("new")); 50 | driver.findElement(By.linkText("rising")); 51 | driver.findElement(By.linkText("top")); 52 | } catch (NoSuchElementException nseex) { 53 | fail(); 54 | } 55 | } 56 | 57 | // Given that I am on the main page 58 | // When I view the Remember Me section 59 | // Then I should see that it contains the phrase "remember me" 60 | @Test 61 | public void testHasRememberMe() { 62 | 63 | // Check that there is a remember-me element 64 | // that contains the text "remember me" 65 | // If it does not exist, or text is incorrect, fail test 66 | 67 | try { 68 | WebElement e = driver.findElement(By.id("remember-me")); 69 | String elementText = e.getText(); 70 | assertTrue(elementText.contains("remember me")); 71 | } catch (NoSuchElementException nseex) { 72 | fail(); 73 | } 74 | } 75 | 76 | // Given that I am on the main page 77 | // When I click on the "new" link 78 | // Then I should be redirected to the "new" page 79 | @Test 80 | public void testSeeNewLinks() { 81 | 82 | // find the "new" link and click on it 83 | // The page you go to should include "newest submissions" 84 | // in the title 85 | 86 | driver.findElement(By.linkText("new")).click(); 87 | String newPageTitle = driver.getTitle(); 88 | assertTrue(newPageTitle.contains("newest submissions")); 89 | } 90 | 91 | // Given that I am on the main page 92 | // And I am not logged in 93 | // When I try to login with an valid username and invalid password 94 | // Then I am given the opportunity to reset the password 95 | @Test 96 | public void testBadPasswordResetLink() { 97 | 98 | // Enter username "meow", password "meow" 99 | 100 | driver.findElement(By.name("user")).sendKeys("meow"); 101 | driver.findElement(By.name("passwd")).sendKeys("meow"); 102 | 103 | // Look for the submit button (in the login div) and click 104 | // to attempt to login 105 | 106 | WebElement loginDiv = driver.findElement(By.id("login_login-main")); 107 | 108 | WebElement submitButton = loginDiv.findElement(By.className("btn")); 109 | submitButton.click(); 110 | 111 | // Check that there is a link to reset password and it is visible 112 | 113 | try { 114 | WebElement resetPw = driver.findElement(By.linkText("reset password")); 115 | assertTrue(resetPw.isDisplayed()); 116 | } catch (NoSuchElementException nseex) { 117 | fail(); 118 | } 119 | } 120 | 121 | 122 | } 123 | -------------------------------------------------------------------------------- /sample_tests/midterm.txt: -------------------------------------------------------------------------------- 1 | 2 | 1. (5 pts) 3 | A coffee shop offers some special deals for their coffee. A regular coffee costs $2.00. However, 4 | military veterans can get coffee for $1.00. Seniors (over age 65) can get coffee for $1.00. All 5 | students can get 20% off of the price of coffee, regardless of their military status or age. 6 | 7 | Partition this into equivalence classes and write them down below. 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 2. (3 pts) 17 | An online toy store is offering a deal for the holidays. Any order of $100.00 or more will get a 18 | free teddy bear. Any order of $1,000.00 or more will get a free tricycle, but no teddy bear. Any order 19 | of $10,000.00 or more will get a free tricycle and a teddy bear. Any order not meeting these requirements 20 | will get a free lollipop. 21 | 22 | What are the boundary values? 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 3. (3 pts) 34 | Your company offers an exciting new product, "Fibonacci-as-a-service". The customer 35 | emails your company a number n, and the company will email back with the nth Fibonacci 36 | number (remember the Fibonacci sequence is [0, 1] and each additional number is the sum 37 | of the previous two, e.g. [0, 1, 1, 2, 3, 5, 8...]. You are in charge of testing this and thinking 38 | of test cases. 39 | 40 | Include a description of the case as well as a (reasonable) expected value. 41 | 42 | Write a base case: 43 | 44 | 45 | 46 | 47 | 48 | Write an edge case: 49 | 50 | 51 | 52 | 53 | 54 | Write a corner case: 55 | 56 | 57 | 58 | 59 | 60 | 61 | 4. (2 pts) 62 | Explain the difference between black-box and white-box testing. 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 5. (3 pts) 73 | Explain the difference between static and dynamic testing. Give one example of each. 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 6. (4 pts) 83 | For each of the following requirements, write down one reason why it is not a good requirement. 84 | 85 | a. The system shall use a Hashmap to store all login attempts. 86 | 87 | 88 | 89 | 90 | 91 | b. The system shall be extremely responsive. 92 | 93 | 94 | 95 | 96 | 97 | c. Whenever the system retrieves an invalid name and birthdate for a user, it will be set to the default. 98 | 99 | 100 | 101 | 102 | 103 | d. The system will run continuously and never have any downtime. 104 | 105 | 106 | 107 | 108 | 109 | 110 | 7. (4 pts) 111 | a. Give an example of a functional requirement for a database system. 112 | 113 | 114 | 115 | 116 | 117 | 118 | b. Give an example of a non-functional requirement (quality attribute) for a database system. 119 | 120 | 121 | 122 | 123 | 124 | 125 | 8. (15 pts) 126 | Develop a test plan (with three test cases) for a Collatz Conjecture Tester. 127 | 128 | The Collatz Conjecture is that for any natural number n, if n is even, divide it by 2 129 | (n = n * 2); if odd, multiply by 3 and add 1 (n = 3n + 1). Repeat the process with the 130 | new number until 1 is reached (e.g., 5 -> 16 -> 8 -> 4 -> 2 -> 1). The conjecture 131 | is that every natural number subjected to this sequence will eventually reach 1; 132 | this has been tested up to 5.764 x 10^18 but never proven. 133 | 134 | Upon initial execution, the system shall prompt the user with the prompt: 135 | "Enter number >" without the quotes. 136 | 137 | After accepting input from the user, if the input is not a natural number (i.e., an 138 | integer of value 1 or greater), then the system shall display "NOT A NATURAL NUMBER" 139 | and exit. 140 | 141 | If the input is a natural number, the system shall calculate the sequence according to the 142 | rules of the Collatz conjecture, and display each iteration of the sequence on a separate line, 143 | including the initial value input by the user. 144 | 145 | Upon reaching the value 1, the system shall display the number of iterations required 146 | to reach the value of 1 in parentheses ( "(" and ")" ) and exit immediately afterwards. 147 | 148 | Sample Output: 149 | Enter number > 5 150 | 5 151 | 16 152 | 8 153 | 4 154 | 2 155 | 1 156 | (5) 157 | 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | 174 | 175 | 176 | 177 | 178 | 179 | 180 | 181 | 182 | 183 | 184 | 185 | 186 | 9. (4 pts) 187 | For the following runs of these test cases, write what their status would be listed as. 188 | 189 | a. The test case had an expected behavior of displaying 7. Instead, the system displayed 4. 190 | 191 | 192 | 193 | 194 | b. The case had an expected behavior of display "FOO". The system displayed "FOO". 195 | 196 | 197 | 198 | 199 | c. The test case could not be executed because the functionality has not yet been developed. 200 | 201 | 202 | 203 | 204 | d. The test case has the tester to do something impossible, such as press a button which does 205 | not exist and is not planned to be added to the system (because it is command line only). 206 | 207 | 208 | 209 | 210 | 10. (10 pts) 211 | After testing the Collatz Conjecture tester, several defects were found. Write a defect 212 | report using the template described in class to write a defect report for each. 213 | 214 | a. The user entered 7. The following output appeared: "NOT A NATURAL NUMBER". 215 | 216 | 217 | 218 | 219 | 220 | 221 | 222 | 223 | 224 | 225 | b. The user entered -4. The following output appeared: 226 | -4 227 | -2 228 | -1 229 | (-2) 230 | 231 | 232 | 233 | 234 | 235 | 236 | 237 | 238 | 239 | 240 | 11. (4 pts) 241 | Suppose we have a function, int billify( int n), which has the following requirements: 242 | If the value of n is less than 5, return 1. 243 | If the value of n greater than or equal to 5, but less than 20, return 2. 244 | If the value is 20 or greater, return 3. 245 | 246 | Assume no integer overflows (MAXINT and MININT are infinity and -infinity). 247 | 248 | Does the following function definition contain a defect? Why or why not? 249 | 250 | public class Bill { 251 | public int billify(int n) { 252 | int toReturn = 6; 253 | if (toReturn == 6) { 254 | toReturn++; 255 | } else { 256 | toReturn = 7; 257 | } 258 | if (toReturn > 0) { 259 | if (n - 5 < 0) { 260 | return 1; 261 | } else { 262 | if (n - 19 <= 0) { 263 | return 2; 264 | } else { 265 | return 3; 266 | } 267 | } 268 | return toReturn; 269 | } 270 | } 271 | 272 | 273 | 274 | 275 | 276 | 277 | 278 | 12. (15 pts) 279 | Write three unit tests in JUnit for the billify function in the previous question, 280 | each one testing a different equivalence class. 281 | There is no need for writing out the import statements, but remember to include the 282 | appropriate annotations. 283 | 284 | 285 | 286 | 287 | 288 | 289 | 290 | 291 | 292 | 293 | 294 | 295 | 296 | 297 | 298 | 299 | 300 | 301 | 302 | 303 | 13. (3 pts) 304 | A user complains that he wants to test the Collatz Conjecture on negative numbers, 305 | but the user receives an error when he/she enters a negative number into the program. 306 | Is this a defect? Why or why not? 307 | 308 | 309 | 310 | 311 | 312 | 313 | 314 | 14. (3 pts) 315 | In unit testing... 316 | a. Define the following: "stub". 317 | 318 | 319 | 320 | b. Define the following: "mock". 321 | 322 | 323 | 324 | 325 | 15. (3 pts) 326 | Describe the red-green-refactor loop in TDD. 327 | 328 | 329 | 330 | 331 | 332 | 333 | 334 | 335 | 16. (2 pts) 336 | In TDD, what does YAGNI stand for? 337 | 338 | 339 | 340 | 341 | 342 | 343 | 344 | 17. (5 pts) 345 | You are a user of the Rent-A-Cat platform. However, you have noticed that there is some 346 | functionality missing which you would really like. Using the Connextra template discussed in class, 347 | write out a user story for this missing functionality. 348 | 349 | 350 | 351 | 352 | 353 | 354 | 355 | 356 | 357 | 358 | 18. (4 pts) 359 | Write out one scenario for that user story, using the given/when/then template discussed in class. 360 | 361 | 362 | 363 | 364 | 365 | 366 | 367 | 368 | 369 | 370 | 19. (6 pts) 371 | You are testing software for a data repository for sale to a large hospital in the Pittsburgh region. This 372 | data repository must meet all sorts of legal and regulatory hurdles due to it storing confidential 373 | patient data. Although the hospital will buy it, it will be used primarily by doctors and nurses in the 374 | system. They will look up data for individual patients, and it will let them know what courses of action 375 | have been taken in the past. The development team is very large and has a project manager who 376 | likes to keep a close eye on the project, which has been running a little late. Due to this, developers are 377 | staying late at work and eating lots of pizza, which is causing the janitor much grief since the garbage 378 | cans are overflowing with pizza boxes. They are also drinking much more coffee than normal, much to the 379 | delight of the barista in the coffee shop next to the office (as the developers are generous tippers). 380 | 381 | If the sale goes through, the financials of this company will improve markedly. The CEO has remarked 382 | several times that the success of the project is essential to the success of the company. The CFO has 383 | mentioned numerous times that without the revenue generated by the sale of this project, earnings 384 | will dip below Wall Street expectations. 385 | 386 | List five classes of stakeholder for this project, and what they are likely care about. 387 | 388 | 389 | 390 | 391 | 392 | 393 | 394 | 395 | 396 | 397 | 398 | 399 | 400 | 20. (2 pts) 401 | Fill in the blank with the correct term. 402 | 403 | a. _______________________ is building the RIGHT software. That is, it is ensuring that the software 404 | as developed and specified meets the needs of the users and customers. 405 | 406 | b. _______________________ is building the software RIGHT. That is, it is ensuring that the software 407 | is stable and meets all of the specified requirements. 408 | -------------------------------------------------------------------------------- /study-guides/midterm-1-study-guide.md: -------------------------------------------------------------------------------- 1 | # CS 1632 Midterm Exam Study Guide 2 | Spring 2016 3 | 4 | The midterm will cover everything we have covered up to the lecture of 18 FEB 2016. 5 | 6 | However, here are the key topics to study in preparation for the test. 7 | 8 | ## TESTING THEORY AND TERMINOLOGY 9 | * Equivalence class partitioning 10 | * Boundary and "interior" values 11 | * Base, Edge, and Corner cases 12 | * Static vs Dynamic testing 13 | * Know the differences and examples of each 14 | * Black/White/Grey box testing 15 | * Know the differences and examples of each 16 | 17 | ## REQUIREMENTS ANALYSIS 18 | * What makes a good or bad requirement? 19 | * Testability - requirements must be: 20 | * Complete, consistent, unambiguous, quantitative, feasible 21 | * Functional Requirements vs Non-Functional (Quality Attributes) 22 | * Be able to define and write your own 23 | * Traceability Matrices 24 | * Be able to define and write your own 25 | 26 | ## TEST PLANS 27 | * Terminology: test cases, test plans, test suites, test runs 28 | * The Seven Testing Principles 29 | * You don't need to remember their names, but use them in developing test plans 30 | * Verification vs validation 31 | 32 | ## DEFECT REPORTING 33 | * Be prepared to report a defect based on a test case 34 | * Remember the defect template: 35 | * SUMMARY, DESCRIPTION, REPRODUCTION STEPS, EXPECTED BEHAVIOR, OBSERVED BEHAVIOR 36 | * Optionally: SEVERITY/IMPACT, NOTES 37 | * Levels of severity: BLOCKER, CRITICAL, MAJOR, NORMAL, MINOR, TRIVIAL 38 | * Enhancements vs defects 39 | * Be prepared to argue if something is a defect or enhancement 40 | * Coding mistakes vs defects 41 | 42 | ## AUTOMATED TESTING 43 | * Pros and cons of automated testing 44 | * Unit tests vs system/acceptance/integration tests 45 | * Writing automated tests: 46 | * PRECONDITIONS, POSTCONDITIONS, EXECUTION STEPS, INPUT/OUTPUT VALUES 47 | 48 | ## UNIT TESTING 49 | * Be prepared to write some unit tests in JUnit 50 | * Pay special attention to assertions 51 | * Stubs, test doubles, mocks, and verification 52 | * Be able to explain code coverage and what it's good for/not good for 53 | 54 | ## WRITING TESTABLE CODE 55 | * Basic strategies for testable code 56 | * The DRY Principle 57 | * The SOLID Principles 58 | * The Law of Demeter 59 | 60 | ## TDD 61 | * The red-green-refactor loop 62 | * Principles of TDD: 63 | * YAGNI, KISS, "Fake it 'til you make it", Avoid interdependency, avoid slow tests 64 | * Benefits and drawbacks of TDD 65 | * Testing private methods (you won't need to use reflection, but remember why/why not one might not test them) 66 | 67 | ## BDD 68 | * Be able to define it and differentiate from TDD 69 | * Understand and be able to use the Connextra template 70 | * Be able to create user stories/scenarios in the proper format 71 | * How to use automated tests 72 | 73 | ## WEB TESTING 74 | * Be able to explain how you would test a web page with Selenium 75 | * You do NOT need to know Selenese (Selenium scripting language) 76 | 77 | -------------------------------------------------------------------------------- /study-guides/midterm-2-study-guide.md: -------------------------------------------------------------------------------- 1 | # CS 1632 Exam 2 Study Guide 2 | Spring 2016 3 | 4 | Note that the second exam is _not_ cumulative, except in the sense that there 5 | 6 | ## COMBINATORIAL TESTING 7 | * Understand what a covering array is 8 | * Be able to make a simple covering array 9 | 10 | ## PROPERTY-BASED TESTING 11 | * Be able to write simple property-based tests 12 | * What is property-based testing good for? What is it bad for? 13 | 14 | ## FORMAL VERIFICATION 15 | * Understand what it is 16 | * Drawbacks: why is not used often? 17 | * How can we formally verify that something will halt, if our languages are Turing-complete? 18 | 19 | ## STOCHASTIC ("FUZZ") TESTING 20 | * Be able to define, list benefits/drawbacks 21 | * Think of good inputs to push (e.g., JSON, executable code, JavaScript, SQL, etc.) 22 | * Understand dumb, smart, evil, chaos monkey testing 23 | * Other ways to test similar to chaos monkey: reduce bandwidth, modified permissions, etc. 24 | 25 | ## PERFORMANCE TESTING 26 | * Understand concepts on how to test performance 27 | * Be able to write test plans for different performance indicators and systems 28 | * Terminology: Service-Oriented vs Efficiency-Oriented Indicators 29 | * Availability, Response Time, Throughput, Utilization 30 | * Performance targets, performance thresholds, KPIs - understand and be able to generate! 31 | * Measuring response time - methodologies 32 | * Understand different concepts of time: user, system, total, real 33 | * Measuring availability, concurrency, scalability, throughput 34 | * Understand n 9's (e.g., 5 9s vs 6 9s) 35 | * Load testing - baseline, soak/stability, stress tests 36 | * Know different performance monitoring tools - perfmon, Activity Monitor/Instruments, top/iostat 37 | * Key resources to watch, and why 38 | * When to use a packet analyzer or profiler 39 | * Be able to answer questions about profiling 40 | 41 | ## SECURITY TESTING 42 | * The CIA/InfoSec Triad 43 | * Terminology: passive vs active, public-key cryptography, vulnerability, exploit 44 | * Terminology: interruption, interception, modification, fabrication 45 | * Be able to define different kinds of malware (virus, worm, etc.) 46 | * Be able to describe/test for common attacks: injection, broken authentication, XSS, insecure storage, buffer overruns, smashing the stack, security misconfiguration, insecure object references, social engineering 47 | * Understand the steps of penetration testing, and why it is useful 48 | * How does security testing differ from other kinds of testing? 49 | 50 | ## LANGUAGE AND QUALITY 51 | * How does Rust protect from common program failures? 52 | 53 | 54 | ## INTERACTING WITH STAKEHOLDERS 55 | * Be able to name some stakeholders and what is important to them (upper management, project management, testers, other developers) 56 | * Be prepared for some "fake" interaction with various stakeholders 57 | * Be able to put together a red-yellow-green template report 58 | * Know the one most important thing when dealing with stakeholders or other co-workers -------------------------------------------------------------------------------- /syllabus.md: -------------------------------------------------------------------------------- 1 | # Syllabus - Spring 2016 2 | CS1632: Software Quality Assurance 3 | 4 | _Although the professor will make a best effort to have the class topic on the day listed, occasionally a change must be made (e.g., a lecture going long, or a guest lecturer unable to make it to class that day). However, these are the topics that will be covered and the expected date that they will be taught._ 5 | 6 | AFIST = _A Friendly Introduction to Software Testing_ by Bill Laboon 7 | 8 | ## WEEK 1 (Week of 4 Jan) 9 | * Introduction - What is Software Testing? 10 | 11 | ## WEEK 2 (Week of 11 Jan) 12 | 13 | * Testing Basics 14 | * READING: AFIST, Chapters 2 - 4 15 | 16 | * Requirements Development and Testing 17 | * READING: AFIST, Chapter 5 18 | 19 | ## WEEK 3 (Week of 18 Jan) 20 | 21 | * Test Plans, Exploratory and Smoke Testing 22 | * READING: AFIST, Chapter 6-8 23 | 24 | * Defects 25 | * READING: AFIST, Chapter 9 26 | 27 | ## WEEK 4 (Week of 25 Jan) 28 | 29 | * Breaking Software 30 | * READING: AFIST, Chapter 7 31 | 32 | * Automated and Manual Testing, Writing Unit Tests (The Basics) 33 | * READING: AFIST, Chapter 12-13 34 | 35 | ## WEEK 5 (Week of 1 Feb) 36 | 37 | * Advanced Unit Testing 38 | * READING: AFIST, Chapter 14 39 | 40 | * Test-driven Development 41 | * READING: AFIST, Chapter 15 42 | 43 | ## WEEK 6 (Week of 8 Feb) 44 | 45 | * Writing Testable Code 46 | * READING: AFIST, Chapter 16 47 | 48 | * Behavior-Driven Development 49 | * Writing Acceptance Tests for Behavior-Driven Development 50 | 51 | ## WEEK 7 (Week of 15 Feb) 52 | 53 | * Web Testing with Selenium 54 | 55 | * Automated Web Testing with Selenium / JUnit 56 | 57 | ## WEEK 8 (Week of 22 Feb) 58 | 59 | * MIDTERM 60 | 61 | * Pairwise and Combinatorial Testing 62 | * READING: AFIST, Chapter 17 63 | 64 | ## WEEK 9 (Week of 29 Feb) 65 | 66 | * Stochastic and Property-Based Testing 67 | * READING: AFIST, Chapter 18 68 | 69 | * Performance Testing 70 | * READING: AFIST, Chapter 19 71 | 72 | 73 | ## WEEK 10 (Week of 7 Mar) 74 | 75 | * NO CLASS THIS WEEK - SPRING BREAK 76 | 77 | ## WEEK 11 (Week of 14 Mar) 78 | 79 | * Performance Testing, continued 80 | 81 | * Performance Exercise 82 | 83 | ## WEEK 12 (Week of 21 Mar) 84 | 85 | * Security Testing 86 | * READING: AFIST, Chapter 20 87 | 88 | * Security Testing (Penetration Testing) 89 | 90 | ## WEEK 13 (Week of 4 Apr) 91 | 92 | * Security Exercise 93 | 94 | * Interacting With Stakeholders 95 | * READING: AFIST, Chapter 21 96 | * Stakeholder Interaction Exercise 97 | 98 | ## WEEK 14 (Week of 11 Apr) 99 | 100 | * (GUEST LECTURE) Carol Nichols, Quality with Rust 101 | 102 | * Stakeholder Interaction Exercise 103 | 104 | ## WEEK 15 (Week of 18 Apr) 105 | 106 | * (GUEST LECTURE) Usability Testing 107 | 108 | * MIDTERM 2 109 | 110 | -------------------------------------------------------------------------------- /test/Foo.java: -------------------------------------------------------------------------------- 1 | public class Foo { 2 | 3 | public static void main(String[] args) { 4 | System.out.println("Monkeys!"); 5 | System.out.println("Birds!"); 6 | System.out.println("Owls!"); 7 | } 8 | } 9 | --------------------------------------------------------------------------------